How to see why an SSAS cube fails to refresh a table - ssas

We have a tabular model cube that is refreshed each day via an SQL Agent job calling an SSAS Command. The command is simply:
{
"refresh": {
"type": "full",
"objects": [
{
"database": "KPIDashboardv1"
}
]
}
}
The job runs and succeeds each day. But I have noticed that some of the tables in the cube have not refreshed. select * from $system.TMSCHEMA_partitions shows these tables with a RefreshedTime of weeks ago.
How can I find what error or problem SSAS is having when attempting to process these tables?
I have tried:
Taking the Table Properties query from $system.TMSCHEMA_partitions
and executing that query against the relevant Data Source. It
succeeds promptly and contains up-to-date data.
Looked at the Output file configured for the SQL Agent job step. It
is empty.

Have you tried to check the data that XE collects?
You needed "ProgressReportEnd" event
<Create xmlns="http://schemas.microsoft.com/analysisservices/2003/engine">
<ObjectDefinition>
<Trace>
<ID>ETLExecutionLogs</ID>
<Name>ETLExecutionLogs</Name>
<AutoRestart>true</AutoRestart>
<XEvent xmlns="http://schemas.microsoft.com/analysisservices/2011/engine/300/300">
<event_session name="ETLExecutionLogs" dispatchLatency="0" maxEventSize="0" maxMemory="4" memoryPartition="none" eventRetentionMode="AllowSingleEventLoss" trackCausality="true" xmlns="http://schemas.microsoft.com/analysisservices/2003/engine">
<event package="AS" name="ProgressReportEnd" />
<target package="package0" name="event_file">
<parameter name="filename" value="d:\YourServerLogPath\OLAP\Log\ETLExecutionLogs.xel" />
<parameter name="max_file_size" value="250" />
<parameter name="max_rollover_files" value="4" />
<parameter name="increment" value="50" />
</target>
</event_session>
</XEvent>
</Trace>
</ObjectDefinition>
</Create>
After that you can use powershell to extract info:
$SharedPath = "C:\Program Files\Microsoft SQL Server\150\Shared";
$xeCore = [System.IO.Path]::Combine($SharedPath, "Microsoft.SqlServer.XE.Core.dll");
$xeLinq = [System.IO.Path]::Combine($SharedPath, "Microsoft.SqlServer.XEvent.Linq.dll");
Add-Type -Path $xeLinq;
if( [System.IO.File]::Exists($xeCore) )
{
Add-Type -Path $xeCore;
}
[Microsoft.SqlServer.XEvent.Linq.QueryableXEventData] $xEvents = $null;
$files = [System.IO.Directory]::EnumerateFiles("d:\YourServerLogPath\OLAP\Log","ETLExecutionLogs*.xel")
$xEvents =
New-Object -TypeName Microsoft.SqlServer.XEvent.Linq.QueryableXEventData(
$files
);
$dt = New-Object System.Data.Datatable;
$dt.Columns.Add("EventSubclass",[int]);
$dt.Columns.Add("ObjectType",[int]);
$dt.Columns.Add("NTCanonicalUserName",[String]);
$dt.Columns.Add("DatabaseName",[String]);
$dt.Columns.Add("ObjectPath",[String]);
$dt.Columns.Add("ObjectID",[String]);
$dt.Columns.Add("EndTime",[DateTimeOffset]);
$dt.Columns.Add("StartTime",[DateTimeOffset]);
$dt.Columns.Add("RequestID",[Guid]);
$dt.Columns.Add("ReEncoding",[String]);
foreach($publishedEvent in $xEvents)
{
if( ($publishedEvent.Fields["EventSubclass"].value -In 55) -or ($publishedEvent.Fields["EventSubclass"].Value -In 59 -And $publishedEvent.Fields["ObjectType"].Value -In 802016,802015,802014, 802013, 802018 -And
$publishedEvent.Name -eq "ProgressReportEnd" -And $publishedEvent.Fields["Success"].Value -eq "1" ))
{
$row = $dt.NewRow();
$row.EventSubclass = $publishedEvent.Fields["EventSubclass"].Value;
$row.ObjectType = $publishedEvent.Fields["ObjectType"].Value;
$row.NTCanonicalUserName = $publishedEvent.Fields["NTCanonicalUserName"].Value;
$row.DatabaseName = $publishedEvent.Fields["DatabaseName"].Value;
$row.ObjectPath = $publishedEvent.Fields["ObjectPath"].Value;
$row.ObjectID = $publishedEvent.Fields["ObjectID"].Value;
$row.EndTime = $publishedEvent.Fields["EndTime"].Value;
$row.StartTime = $publishedEvent.Fields["StartTime"].Value;
$row.RequestID = $publishedEvent.Fields["RequestID"].Value;
$row.ReEncoding = $publishedEvent.Fields["TextData"].Value;
$dt.Rows.Add($row)
}
}
$dt
}

Related

Get count of rows in a partition of an azure table using Azure PowerShell

I would like to get count of rows in a partition. I have the code for getting the total count of rows. How can I alter it to get count for a particular partition. Also I am getting warning for fetching count of all rows and not getting the count on powershell window. Is there any documentation on this?
function GetTable($connectionString, $tableName)
{
$context = New-AzureStorageContext -ConnectionString $connectionString
$azureStorageTable = Get-AzureStorageTable $tableName -Context $context
$azureStorageTable
}
function GetTableCount($table)
{
#Create a table query.
$query = New-Object Microsoft.WindowsAzure.Storage.Table.TableQuery
#Define columns to select.
$list = New-Object System.Collections.Generic.List[string]
$list.Add("PartitionKey")
#Set query details.
$query.SelectColumns = $list
#Execute the query.
$entities = $table.CloudTable.ExecuteQuery($query)
($entities | measure).Count
}
$connectionString = "xyz"
$table = GetTable $connectionString SystemAudit
GetTableCount $table
How can I alter it to get count for a particular partition
There is a function Get-AzureStorageTableRowByPartitionKey you could use, and the following is the sample code
function GetTable($connectionString, $tableName)
{
$context = New-AzureStorageContext -ConnectionString $connectionString
$azureStorageTable = Get-AzureStorageTable $tableName -Context $context
$azureStorageTable
}
function GetTableCount($table)
{
$list = Get-AzureStorageTableRowByPartitionKey -table $table –partitionKey “storage” | measure
$list.Count
}
Import-Module AzureRmStorageTable
$connectionString = xyz"
$table = GetTable $connectionString <yourTableName>
GetTableCount $table
You can know more information on this blog

SQL Server backup status Report with PowerShell

I got a PowerShell script for reporting SQL backup status on multiple servers, it works fine but it had no function to send mail. I added that part and now I am able to get the mail with the attachment.
The only concern is, I want the report to show "NA" and not a default date where the satabase is in Simple Recovery Model or if backup has not happened. Can someone please advise?
Here is the code, just in case someone needs it unlike my requirement:
$ServerList = Get-Content "Serverlist location"
$OutputFile = "to save the report location"
$titleDate = Get-Date -UFormat "%m-%d-%Y - %A"
$HTML = '<style type="text/css">
#Header{font-family:"Trebuchet MS", Arial, Helvetica, sans-serif;width:100%;border-collapse:collapse;}
#Header td, #Header th {font-size:14px;border:1px solid #98bf21;padding:3px 7px 2px 7px;}
#Header th {font-size:14px;text-align:left;padding-top:5px;padding-bottom:4px;background-color:#A7C942;color:#fff;}
#Header tr.alt td {color:#000;background-color:#EAF2D3;}
</Style>'
$HTML += "<HTML><BODY><Table border=1 cellpadding=0 cellspacing=0 width=100% id=Header>
<TR>
<TH><B>Database Name</B></TH>
<TH><B>RecoveryModel</B></TD>
<TH><B>Last Full Backup Date</B></TH>
<TH><B>Last Differential Backup Date</B></TH>
<TH><B>Last Log Backup Date</B></TH>
</TR>"
[System.Reflection.Assembly]::LoadWithPartialName('Microsoft.SqlServer.SMO') | Out-Null
foreach ($ServerName in $ServerList)
{
$HTML += "<TR bgColor='#ccff66'><TD colspan=5 align=center><B>$ServerName</B></TD></TR>"
$SQLServer = New-Object ('Microsoft.SqlServer.Management.Smo.Server') $ServerName
foreach ($Database in $SQLServer.Databases)
{
$HTML += "<TR>
<TD>$($Database.Name)</TD>
<TD>$($Database.RecoveryModel)</TD>
<TD>$($Database.LastBackupDate)</TD>
<TD>$($Database.LastDifferentialBackupDate)</TD>
<TD>$($Database.LastLogBackupDate)</TD>
</TR>"
}
}
$HTML += "</Table></BODY></HTML>"
$HTML | Out-File $OutputFile
$emailFrom = "send email address"
$emailTo = "recipient email address"
$subject = "Xyz Report"
$body = "your words "
$smtpServer = "Smptp server"
$filePath = "location of the file you want to attach"
function sendEmail([string]$emailFrom, [string]$emailTo, [string]$subject,[string]$body,[string]$smtpServer,[string]$filePath)
{
$email = New-Object System.Net.Mail.MailMessage
$email.From = $emailFrom
$email.To.Add($emailTo)
$email.Subject = $subject
$email.Body = $body
$emailAttach = New-Object System.Net.Mail.Attachment $filePath
$email.Attachments.Add($emailAttach)
$smtp = New-Object Net.Mail.SmtpClient($smtpServer)
$smtp.Send($email)
}
sendEmail $emailFrom $emailTo $subject $body $smtpServer $filePath
Replace
<TD>$($Database.LastBackupDate)</TD>
with something like
<TD>$(if ($Database.RecoveryModel -eq 'Simple' -or $Database.LastBackupDate -eq '01/01/0001 00:00:00') {'NA'} else {$Database.LastBackupDate})</TD>
Do the same for LastDifferentialBackupDate and LastLogBackupDate.
With that said, I strongly recommend looking into calculated properties, ConvertTo-Html, and Send-MailMessage, which would allow you to greatly simplify your code:
[Reflection.Assembly]::LoadWithPartialName('Microsoft.SqlServer.SMO') | Out-Null
$emailFrom = 'sender#example.com'
$emailTo = 'recipient#example.com'
$subject = 'Xyz Report'
$smtpServer = 'mail.example.com'
$style = #'
<style type="text/css">
...
</style>
'#
$msg = Get-Content 'C:\path\to\serverlist.txt' |
ForEach-Object {New-Object 'Microsoft.SqlServer.Management.Smo.Server' $_} |
Select-Object -Expand Databases |
Select-Object Name, RecoveryModel,
#{n='LastBackupDate';e={if ($_.RecoveryModel -eq 'Simple' -or $_.LastBackupDate -eq '01/01/0001 00:00:00') {'NA'} else {$_.LastBackupDate}}},
#{n='LastDifferentialBackupDate';e={if ($_.RecoveryModel -eq 'Simple' -or $_.LastDifferentialBackupDate -eq '01/01/0001 00:00:00') {'NA'} else {$_.LastDifferentialBackupDate}}},
#{n='LastLogBackupDate';e={if ($_.RecoveryModel -eq 'Simple' -or $_.LastLogBackupDate -eq '01/01/0001 00:00:00') {'NA'} else {$_.LastLogBackupDate}}} |
ConvertTo-Html -Head $style | Out-String
Send-MailMessage -From $emailFrom -To $emailTo -Subject $subject -Body $msg -BodyAsHtml -SmtpServer $smtpServer
See also.

update sql table for Active Directory createdon and disabled on information

I have a user table in the database that i am trying to update with Createdon date and disabled on date with the data from Active Directory. So far this is what I have:
$SearchRoot = "OU=NonAIQ,OU=FrontOffice,DC=dev,DC=local"
$serverName = "localhost"
#$SearchRoot = "OU=NonAIQ,OU=FrontOffice,DC=dmz,DC=local"
#$serverName = "spoproddb3.dmz.local"
try {
Import-Module "sqlps" -DisableNameChecking
if ((Get-PSSnapin -Name "Quest.ActiveRoles.ADManagement" -ErrorAction SilentlyContinue) -eq $null ) {
Add-PsSnapin "Quest.ActiveRoles.ADManagement"
}
$externalUsers = Get-QADUser -SizeLimit 0 -SearchRoot $SearchRoot | Select-Object whencreated, whenchanged
$externalUsers | % {
$query = #"
Update tbl_EdgeUsers Set CompanyName = '$_.CreationDate'
Where UserUPN = $_.UserPrincipalName;
"#
Write-Host "The query is $query"
Invoke-SqlCmd -ServerInstance $serverName -Query $query -Database "EdgeDW"
}
} finally {
Remove-Module "sqlps" -ErrorAction SilentlyContinue
Remove-PsSnapin "Quest.ActiveRoles.ADManagement"
}
Now for when created, we just grab all the values.
But since AD does not track the Disabled in date, I am using the when changed date since we dont make changes to an account once it is changed.
The part that I am stuck on is about the logic for when changed date. For this I have to check if an account is disabled. If it is the update the table with that date. If an account is not disabled, then ignore that value and set the value in the sql table as '1/1/9999'.
can you guys please help with this logic?
Thank you in advance for any help.
of top of my head maybe something such as this, although thinking about it now, its a nasty way having the invoke-sql inside the foreach loop if the dataset is large, probably better to output the results of the if statement to csv or somewhere then run the invoke-sql against that.
$users = Get-ADUser -Filter * -Properties whenchanged | Select-Object -Property UserPrincipalName, whenchanged, enabled
$disabledsql = #"
update tbl_EdgeUsers Set date = '$user.whenchanged'
Where UserUPN = '$user.UserPrincipalName';
"#
$activesql = #"
update tbl_EdgeUsers Set date = '1/1/9999
Where UserUPN = '$user.UserPrincipalName';
"#
foreach ($user in $users)
{
if ($user.enabled -eq 'False')
{
Invoke-Sqlcmd -ServerInstance $serverName -Query $disabledsql -Database 'EdgeDW'
}
else
{
Invoke-Sqlcmd -ServerInstance $serverName -Query $activesql -Database 'EdgeDW'
}
}

Dynamics CRM- update record of contact entity

I have created contact record using ASP.NET. Now I need to Check if the contact record exists. If exists, update the same record. Through advance find have downloaded FetchXML and added to my FetchXML variable. Please suggest the logic. Below is my code.
// Establish a connection to crm and get the connection proxy
string connectionString = "xyz; Username= xyz ;Password=xyz";
CrmConnection connect = CrmConnection.Parse(connectionString);
OrganizationService service;
using (service = new OrganizationService(connect))
{
WhoAmIRequest request = new WhoAmIRequest();
Guid userId = ((WhoAmIResponse)service.Execute(request)).UserId;
ContactDetails contact = new ContactDetails();
//Check if the contact record exists . If exists , update the same record.
//Fecthxml query
string fetchXml = #" <fetch version='1.0' output-format='xml-platform' mapping='logical' distinct='false'>
<entity name='contact'>
<attribute name='fullname' />
<attribute name='parentcustomerid' />
<attribute name='telephone1' />
<attribute name='emailaddress1' />
<attribute name='contactid' />
<order attribute='fullname' descending='false' />
<filter type='and'>
<condition attribute= 'mobilephone' operator='not-null' />
</filter>
</entity>
</fetch>" ;
FetchExpression query = new FetchExpression(fetchXml);
EntityCollection results = service.RetrieveMultiple(query);
if (results.Entities.Count > 0)
{
Entity contactRecord = results[0];
contactRecord["firstname"] = contactInfo.FirstName;
contactRecord["lastname"] = contactInfo.LastName;
contactRecord["emailaddress1"] = contactInfo.EmailId;
contactRecord["mobilephone"] = contactInfo.MobilePhone;
contactRecord["address1_line1"] = contactInfo.Street1;
contactRecord["address1_line2"] = contactInfo.Street2;
contactRecord["address1_line3"] = contactInfo.Street3;
contactRecord["address1_city"] = contactInfo.City;
service.Update(contactRecord);
}
//Else, Create the contact record
else
{
Entity entity = new Entity();
entity.LogicalName = "contact";
entity["firstname"] = contactInfo.FirstName;
entity["lastname"] = contactInfo.LastName;
entity["emailaddress1"] = contactInfo.EmailId;
entity["mobilephone"] = contactInfo.MobilePhone;
entity["address1_line1"] = contactInfo.Street1;
entity["address1_line2"] = contactInfo.Street2;
entity["address1_line3"] = contactInfo.Street3;
entity["address1_city"] = contactInfo.City;
entity["address1_stateorprovince"] = contactInfo.State;
entity["address1_country"] = contactInfo.Country;
entity["spousesname"] = contactInfo.SpouseName;
entity["birthdate"] = contactInfo.Birthday;
entity["anniversary"] = contactInfo.Anniversary;
//Create entity gender with option value
if (contactInfo.Gender == "Male")
{
entity["gendercode"] = new OptionSetValue(1);
}
else
{
entity["gendercode"] = new OptionSetValue(2);
}
//entity["familystatuscode"] = contactInfo.MaritalStatus;
if (contactInfo.MaritalStatus == "Single")
{
entity["familystatuscode"] = new OptionSetValue(1);
}
else
{
entity["familystatuscode"] = new OptionSetValue(2);
}
service.Create(entity);
}
}
// Create the entity
your logic seems ok, with the exception of the FectchXML query. The way you have your code, you will always end up updating the first record retrieved that has its mobilephone field filled. That does not seem a good way to check if a contact already exists.
I suggest you to change the filter of your fetch. In your filter condition you have to use an attribute that represents uniqueness for all contacts.
Apart of this your code looks ok.
Like nunoalmeieda says, you need to have a better way to ascertain if the Contact already exists. A common way of identifying if the Contact already exists would be to check if the email address already exists as it is very unlikely that two people will have the same email address.
I have updated your basic code to show how it is done with FetchXML.
<fetch version="1.0" output-format="xml-platform" mapping="logical" distinct="false">
<entity name="contact">
<attribute name='fullname' />
<attribute name='parentcustomerid' />
<attribute name='telephone1' />
<attribute name='emailaddress1' />
<attribute name='contactid' />
<order attribute="fullname" descending="false" />
<filter type="and">
<condition attribute="emailaddress1" operator="eq" value=contactInfo.EmailId />
</filter>
</entity>
</fetch>
The logic here is that I am checking if the value of emailaddress1 (field in the contact entity of CRM) is equal to the value of your contactInfo.EmailId. I am assuming that contactInfo is the record that you get from ASP.NET.
The rest of your code is fine (I have formatted it a bit to make the question more readable).

Powershell SQL query results convertto-XML

I have been having quite the time trying to figure this out. Let me try to explain what I am trying to accomplish, I hope i can be clear enough.
I am sending two queries to an MSSQL database and receiving them back. The below code works perfect, however I would like to manipulate the format of the XML a bit before it writes to the XML file. I currently get 3 columns (serviceGroupName, numAccounts, numDevices) I would like to accomplish 1 of 2 things:
1) Add a new column named "ReportType" and have it fill in "Monthly" Or "Total" depending on if it is pass 1 or 2 of the foreach loop (SQLQuery1 is Monthly report, and SQLQuery2 is Total number since inception)
2) Create a new PSObject and have it fill in the appropriate information such as the data it receives back (serviceGroupName, numAccounts, numDevices)
Below is my current code. As i mentioned it does work and it generated an XML but i would like to add some more information before the pipe to ConvertTo-XML if possible.
### Dates to use
$Date = (Get-Date -f MM-dd-yyyy)
$FDoTM = ((Get-Date -Day 01).AddMonths(0)).AddDays(0)
$LDo2PM = ((Get-Date -Day 01).AddMonths(-1)).AddDays(-1)
$TempDir = "C:\Temp"
$WebDir = #("\\x.x.x.x\c$\inetpub\wwwroot\Reports\Accounts","\\x.x.x.x\c$\inetpub\wwwroot\Reports\Accounts")
### Something
$OutputXML = "$Date-Monthly-AccountReport.xml"
### Connection settings, uses windows authentication
$DBServer = "OMMITED"
$databasename = "OMMITED"
$Connection = new-object system.data.sqlclient.sqlconnection #Set new object to connect to sql database
$Connection.ConnectionString ="server=$DBServer;database=$databasename;trusted_connection=True" # Connectiongstring setting for local machine database with window authentication
Write-host "Connection Information:" -foregroundcolor yellow -backgroundcolor black
$Connection #List connection information
### Connect to Database and Run Query
$SqlCmd = New-Object System.Data.SqlClient.SqlCommand #setting object to use sql commands
$OutputHeader1 = "This Month's counts"
$SqlQuery1 = #"
SET NOCOUNT ON;
WITH AccountDeviceStats(serviceGroupName,numAccounts,numDevices)
AS
(
SELECT svg.name,COUNT(acct.serviceGroupId) as Accounts, NULL FROM bm_account acct WITH (NOLOCK)
INNER JOIN bm_servicegroup svg WITH (NOLOCK) ON svg.servicegroupId = acct.serviceGroupId
where acct.CreateStamp between '$($LDo2PM)' and '$($FDoTM)'
GROUP BY acct.serviceGroupId,svg.name
UNION ALL
SELECT svg.name, NULL, COUNT(device.serviceGroupId) as Devices FROM bm_device device WITH (NOLOCK)
INNER JOIN bm_servicegroup svg WITH (NOLOCK) ON svg.servicegroupId = device.serviceGroupId, bm_account acct
where device.accountID=acct.accountId and acct.CreateStamp between '$($LDo2PM)' and '$($FDoTM)'
GROUP BY device.serviceGroupId,svg.name
)
SELECT ad1.serviceGroupName,ad1.numAccounts,ad2.numDevices FROM AccountDeviceStats ad1
INNER JOIN AccountDeviceStats ad2 ON ad1.serviceGroupName = ad2.serviceGroupName
WHERE ad1.numAccounts IS NOT NULL AND ad2.numDevices IS NOT NULL
ORDER BY numAccounts DESC,numDevices DESC
"#
$OutputHeader2 = "Total Counts"
$SqlQuery2 = #"
SET NOCOUNT ON;
WITH AccountDeviceStats(serviceGroupName,numAccounts,numDevices)
AS
(
SELECT svg.name,COUNT(acct.serviceGroupId) as Accounts, NULL FROM bm_account acct WITH (NOLOCK)
INNER JOIN bm_servicegroup svg WITH (NOLOCK) ON svg.servicegroupId = acct.serviceGroupId
where acct.CreateStamp < '12-31-2099'
GROUP BY acct.serviceGroupId,svg.name
UNION ALL
SELECT svg.name, NULL, COUNT(device.serviceGroupId) as Devices FROM bm_device device WITH (NOLOCK)
INNER JOIN bm_servicegroup svg WITH (NOLOCK) ON svg.servicegroupId = device.serviceGroupId, bm_account acct
where device.accountID=acct.accountId and acct.CreateStamp < '12-31-2099'
GROUP BY device.serviceGroupId,svg.name
)
SELECT ad1.serviceGroupName,ad1.numAccounts,ad2.numDevices FROM AccountDeviceStats ad1
INNER JOIN AccountDeviceStats ad2 ON ad1.serviceGroupName = ad2.serviceGroupName
WHERE ad1.numAccounts IS NOT NULL AND ad2.numDevices IS NOT NULL
ORDER BY numAccounts DESC,numDevices DESC
"#
$sqlQueries = #($SqlQuery1, $SqlQuery2)
$Results = #()
Foreach ($Query in $sqlQueries){
$Connection.open()
Write-host "Connection to database successful." -foregroundcolor green -backgroundcolor black
$SqlCmd.CommandText = $Query
$SqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter
$SqlAdapter.SelectCommand = $SqlCmd
$SqlCmd.Connection = $Connection
$DataSet = New-Object System.Data.DataSet
$SqlAdapter.Fill($DataSet)
$Connection.Close()
$Results += $DataSet.Tables[0]
($Results | ConvertTo-XML -NoTypeInformation).Save("$TempDir\$OutputXML")
}
if ((Get-ChildItem $TempDir -filter "$Date-*.xml").count -gt 0){
Foreach ($file in (Get-ChildItem $TempDir -filter "$Date-*.xml" -recurse)){
Foreach ($webserver in $WebDir){
Copy-Item $file.fullname "$webserver\$file" -force
}
Remove-Item $file.fullname -force
}
}
Here is the output formatting of the XML
<?xml version="1.0"?>
<Objects>
<Object>
<Property Name="serviceGroupName">ServiceGroup1</Property>
<Property Name="numAccounts">15</Property>
<Property Name="numDevices">28</Property>
<Property Name="RowError" />
<Property Name="RowState">Unchanged</Property>
<Property Name="Table">
<Property>System.Data.DataRow</Property>
</Property>
<Property Name="ItemArray">
<Property>ServiceGroup1</Property>
<Property>15</Property>
<Property>28</Property>
</Property>
<Property Name="HasErrors">False</Property>
</Object>
<Object>
<Property Name="serviceGroupName">ServiceGroup1</Property>
<Property Name="numAccounts">45</Property>
<Property Name="numDevices">69</Property>
<Property Name="RowError" />
<Property Name="RowState">Unchanged</Property>
<Property Name="Table">
<Property>System.Data.DataRow</Property>
</Property>
<Property Name="ItemArray">
<Property>ServiceGroup1</Property>
<Property>45</Property>
<Property>69</Property>
</Property>
<Property Name="HasErrors">False</Property>
And one last thing. If it's possible to remove the excess bloat from the XML, as you can see it doubles the data output because it creates a node named ItemArray with all of the same information.
I hope this is easy enough to understand. If you need any more information, please let me know. And thank you in advance for any and all help.
I think all you need to do is to update your two T-sql queries within the powershell script. First one, add code like following:
...., "Monthly" as ReportType FROM AccountDeviceStats ad1...
Second one, add code like following:
...., "Total" as ReportType FROM AccountDeviceStats ad1...
### Dates to use
$Date = (Get-Date -f MM-dd-yyyy)
$FDoTM = ((Get-Date -Day 01).AddMonths(0)).AddDays(0)
$LDo2PM = ((Get-Date -Day 01).AddMonths(-1)).AddDays(-1)
$TempDir = "C:\Temp"
$WebDir = #("\\x.x.x.x\c$\inetpub\wwwroot\Reports\Accounts","\\x.x.x.x\c$\inetpub\wwwroot\Reports\Accounts")
### Something
$OutputXML = "$Date-Monthly-AccountReport.xml"
### Connection settings, uses windows authentication
$DBServer = "OMMITED"
$databasename = "OMMITED"
$Connection = new-object system.data.sqlclient.sqlconnection #Set new object to connect to sql database
$Connection.ConnectionString ="server=$DBServer;database=$databasename;trusted_connection=True" # Connectiongstring setting for local machine database with window authentication
Write-host "Connection Information:" -foregroundcolor yellow -backgroundcolor black
$Connection #List connection information
### Connect to Database and Run Query
$SqlCmd = New-Object System.Data.SqlClient.SqlCommand #setting object to use sql commands
$OutputHeader1 = "This Month's counts"
$SqlQuery1 = #"
SET NOCOUNT ON;
WITH AccountDeviceStats(serviceGroupName,numAccounts,numDevices)
AS
(
SELECT svg.name,COUNT(acct.serviceGroupId) as Accounts, NULL FROM bm_account acct WITH (NOLOCK)
INNER JOIN bm_servicegroup svg WITH (NOLOCK) ON svg.servicegroupId = acct.serviceGroupId
where acct.CreateStamp between '$($LDo2PM)' and '$($FDoTM)'
GROUP BY acct.serviceGroupId,svg.name
UNION ALL
SELECT svg.name, NULL, COUNT(device.serviceGroupId) as Devices FROM bm_device device WITH (NOLOCK)
INNER JOIN bm_servicegroup svg WITH (NOLOCK) ON svg.servicegroupId = device.serviceGroupId, bm_account acct
where device.accountID=acct.accountId and acct.CreateStamp between '$($LDo2PM)' and '$($FDoTM)'
GROUP BY device.serviceGroupId,svg.name
)
SELECT ad1.serviceGroupName,ad1.numAccounts,ad2.numDevices, ""Monthly"" as ReportType FROM AccountDeviceStats ad1
INNER JOIN AccountDeviceStats ad2 ON ad1.serviceGroupName = ad2.serviceGroupName
WHERE ad1.numAccounts IS NOT NULL AND ad2.numDevices IS NOT NULL
ORDER BY numAccounts DESC,numDevices DESC
"#
$OutputHeader2 = "Total Counts"
$SqlQuery2 = #"
SET NOCOUNT ON;
WITH AccountDeviceStats(serviceGroupName,numAccounts,numDevices)
AS
(
SELECT svg.name,COUNT(acct.serviceGroupId) as Accounts, NULL FROM bm_account acct WITH (NOLOCK)
INNER JOIN bm_servicegroup svg WITH (NOLOCK) ON svg.servicegroupId = acct.serviceGroupId
where acct.CreateStamp < '12-31-2099'
GROUP BY acct.serviceGroupId,svg.name
UNION ALL
SELECT svg.name, NULL, COUNT(device.serviceGroupId) as Devices FROM bm_device device WITH (NOLOCK)
INNER JOIN bm_servicegroup svg WITH (NOLOCK) ON svg.servicegroupId = device.serviceGroupId, bm_account acct
where device.accountID=acct.accountId and acct.CreateStamp < '12-31-2099'
GROUP BY device.serviceGroupId,svg.name
)
SELECT ad1.serviceGroupName,ad1.numAccounts,ad2.numDevices, ""Total"" as ReportType FROM AccountDeviceStats ad1
INNER JOIN AccountDeviceStats ad2 ON ad1.serviceGroupName = ad2.serviceGroupName
WHERE ad1.numAccounts IS NOT NULL AND ad2.numDevices IS NOT NULL
ORDER BY numAccounts DESC,numDevices DESC
"#
$sqlQueries = #($SqlQuery1, $SqlQuery2)
$Results = #()
Foreach ($Query in $sqlQueries){
$Connection.open()
Write-host "Connection to database successful." -foregroundcolor green -backgroundcolor black
$SqlCmd.CommandText = $Query
$SqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter
$SqlAdapter.SelectCommand = $SqlCmd
$SqlCmd.Connection = $Connection
$DataSet = New-Object System.Data.DataSet
$SqlAdapter.Fill($DataSet)
$Connection.Close()
$Results += $DataSet.Tables[0]
($Results | ConvertTo-XML -NoTypeInformation).Save("$TempDir\$OutputXML")
}
if ((Get-ChildItem $TempDir -filter "$Date-*.xml").count -gt 0){
Foreach ($file in (Get-ChildItem $TempDir -filter "$Date-*.xml" -recurse)){
Foreach ($webserver in $WebDir){
Copy-Item $file.fullname "$webserver\$file" -force
}
Remove-Item $file.fullname -force
}
}
The original question asked how to remove the bloat from the XML as well. I was looking for a solution where the XML that I was generating from the SQL results had to be in an absolute specific format with the correct tags and everything in place. What I discovered was that once you have your dataset object ($DataSet) then if you look to see what methods and properties are available to it, ($DataSet | gm) then one of them is GetXML().
This automatically formats your SQL output such that each returned column (or column alias) is returned as a separate tag (although note, it does not generate an empty tag for a null value) so in this instance if you use $DataSet.GetXML() I would have expected to see output something along the lines of
<NewDataSet>
<Table>
<serviceGroupName>ServiceGroup1</serviceGroupName>
<numAccounts>15</numAccounts>
<numDevices>28</numDevices>
</Table>
</NewDataSet>
so no bloat!
As this is just a series of strings, you can then do things like ($Dataset.GetXML()).Replace('NewDataSet','OuterTag').Replace('Table','InnerTag') to give better labels to the XML. Once you are happy with this you can output
SET-CONTENT -PATH $xmlfilename -VALUE '<?xml version="1.0" ?>'
or some such to a file and then append the output from your GetXML() method so you have a much neater formatted piece of XML!
($DataSet.GetXML()).Replace('NewDataSet','OuterTagName').Replace('Table','InnerTagName') | ADD-CONTENT -PATH $xmlfilename