Powershell outputs mixed between loops - SQL datasets [duplicate] - sql

This question already has an answer here:
PowerShell output is crossing between functions
(1 answer)
Closed 2 years ago.
I have a PowerShell script that loops through list of 3 servers. A SQL script is run with Invoke-Sqlcmd and the result set is stored to variable $DS. At the end of the loop I return the records with with $DS.Tables.Rows.
But the results sets are getting mixed together. I tried using a Write-Host message to breakup the results. But they are still getting mixed together.
Why are the result getting mixed together in the output?
How can I separate the outputs between each loop?
Thanks
Object type
$DS | gm ............... TypeName: System.Data.DataSet
$DS.Tables | gm ........ TypeName: System.Data.DataTable
$DS.Tables.Rows | gm ... TypeName: System.Data.DataRow
Script
#########################>
# SQL servers
$PCList= #("GCOD139","GCOD039","GCOP039")
Write-Host ($PCList -join ", ")
# Query multiple servers
foreach ($PC in $PCList) {
Write-Host ($PC + "...") -ForegroundColor Yellow
# SQL parameters
$Params = #{
'ServerInstance' = $PC;
'Database' = 'master';
# 'Username' = 'svcBIPOC';
# 'Password' = 'bipoc2020*';
# 'InputFile' = "C:\ScriptFolder\TestSqlCmd.sql"
'Query' = '
SELECT
[Server]= ##SERVERNAME
--MB to GB
, REPLACE(name, ''MB'', ''GB'')
,[value]= CAST(value as int)/1000
, [value_in_use]= CAST(value_in_use as int)/1000
--, value, value_in_use, [description]
FROM sys.configurations
WHERE name like ''%server memory%''
ORDER BY name desc
OPTION (RECOMPILE);
'
}
# Capture SQL Dataset
# (Get-Date).ToSTring('s') + " SQL query start..."
$DS = Invoke-Sqlcmd #Params -As DataSet
#(Get-Date).ToSTring('s') + " SQL query end..."
Write-host "-----"
Write-host "SQL"
sleep -Seconds 5
$DS.Tables.Rows
sleep -Seconds 5
}
#########################

Stop using Write-Host to convey progress information - use Write-Progress for that instead!
$PCList= #("GCOD139","GCOD039","GCOP039")
Write-Progress -Activity "Query servers" -Status "About to query: $($PCList -join ", ")"
# Query multiple servers
foreach ($PC in $PCList) {
Write-Progress -Activity "Query servers" -Status "Querying: $PC"
# SQL parameters
$Params = #{
'ServerInstance' = $PC;
'Database' = 'master';
# 'Username' = 'svcBIPOC';
# 'Password' = 'bipoc2020*';
# 'InputFile' = "C:\ScriptFolder\TestSqlCmd.sql"
'Query' = '
SELECT
[Server]= ##SERVERNAME
--MB to GB
, REPLACE(name, ''MB'', ''GB'')
,[value]= CAST(value as int)/1000
, [value_in_use]= CAST(value_in_use as int)/1000
--, value, value_in_use, [description]
FROM sys.configurations
WHERE name like ''%server memory%''
ORDER BY name desc
OPTION (RECOMPILE);
'
}
# Capture SQL Dataset
$DS = Invoke-Sqlcmd #Params -As DataSet
$DS.Tables.Rows
}
Write-Progress -Activity "Query servers" -Completed
Now the progress messages won't interfere with the actual output from the function

Related

Multi-level json to SQL outputs per element instead of per row

I'm using the Microsoft 365 Defender API to receive all recent events/incidents.
I get a json file as following: link to example json
And use following script to try and convert this for easy import to an SQL server:
(Echoes only as test)
# Send the request and get the results.
$response = Invoke-WebRequest -UseBasicParsing -Method Get -Uri $url -Headers $headers -ErrorAction Stop
# Extract the incidents from the results.
$alerts = ($response | ConvertFrom-Json)
$devices = ($response | ConvertFrom-Json ).value.alerts.devices
$entities = ($response | ConvertFrom-Json ).value.alerts.entities
Foreach($row in $alerts){
$IncidentID = $alerts.value.incidentID
$Createdtime = $alerts.value.creationTime
$Status = $alerts.value.status
$Severity = $alerts.value.severity
$Classification = $alerts.value.classification
$IncidentName = $alerts.value.incidentName
$URL = $alerts.incidentUri
$Klant = $afkorting
$Username = $entities.accountname
$device = $devices.deviceDnsName
echo $IncidentID
echo $Createdtime
echo $Status
echo $Severity
echo $Classification
echo $IncidentName
echo $URL
echo $Klant
echo $Username
echo $device
Invoke-Sqlcmd -ServerInstance "SQL.domain.local\MSQL2016" -Database "private" -Username private -Password 'private' -Query "INSERT Into dbo.private ( [IncidentID], [Createdtime], [Status], [Severity], [Classification], [IncidentName], [URL], [Klant], [Username], [device]) VALUES ('$IncidentID', '$Createdtime', '$Status', '$Severity', '$Classification', '$IncidentName', '$URL', '$Klant', '$Username', '$device')"
}
However, the output in case of 3 incidents looks like:
IncidentID
IncidentID
IncidentID
Createdtime
Createdtime
Createdtime
Status
Status
Status
So grouped by element instead of grouped by IncidentID.
I can't find a way to get the output like:
IncidentID
Createdtime
Status
Severity
Classification
IncidentName
URL
Klant
Username
device
I "solved" this with an intermediary step exporting to CSV's and merging them and piping those to SQL for now, but that's too inefficient.
Move resolution of $devices and $entities into the loop, then use the iterator variable $row instead of referencing all $alerts inside the loop body:
# Send the request and get the results.
$response = Invoke-WebRequest -UseBasicParsing -Method Get -Uri $url -Headers $headers -ErrorAction Stop
# Extract the incidents from the results.
$alerts = ($response | ConvertFrom-Json)
foreach($row in $alerts){
$IncidentID = $row.value.incidentID
$Createdtime = $row.value.creationTime
$Status = $row.value.status
$Severity = $row.value.severity
$Classification = $row.value.classification
$IncidentName = $row.value.incidentName
$URL = $row.incidentUri
$Klant = $afkorting # where does `$afkorting` come from?
$Username = $row.value.entities.accountname
$device = $row.value.devices.deviceDnsName
# Insert into SQL Server here
}

Powershell -join Object property values

I'm attempting to produce a variable containing a string of Object properties that are joined with "','". This is to pass into a SQL select where clause. My input looks like this:
foreach ($csv in $CSVFiles) {
$csvOutput = Import-Csv $csv.FullName
$group = $csvOutput | Group-Object order-id, amount-type | Where-Object {$_.Group.'order-id' -ne '' -and $_.Group.'amount-type' -eq 'ItemPrice'}}
Within the above loop. I'm looking to retrieve the order-id and pass it into a new variable $OrdNum. I'm doing this like so:
$OrdNum = $group | Select-Object #{Name='order-id';Expression={$_.Values[0]}}
To perform the join I have attempted:
$OrdNum = ($group | Select-Object #{Name='order-id';Expression={$_.Values[0]}}) -join "','"
This gives ','','','','','','','','','','','',' with no values.
I have also tried:
$OrdNum = ($group | Select-Object -Property 'order-id') -join "','"
Which produces the same result.
I'm expecting $OrdNum to look like 12345','43567','76334','23765 etc.
I'm working under the assumption that $OrdNum is required in that format to pass to this SQL query:
$query = “SELECT ARIBH.ORDRNBR AS [ORDER No'],AROBP.IDRMIT AS [RECPT No'], FROM [XXXX].[dbo].[AROBP] FULL JOIN [XXXX].[dbo].[ARIBH] ON [XXXX].[dbo].[AROBP].[IDMEMOXREF] = [XXXX].[dbo].[ARIBH].[IDINVC] where ARIBH.ORDRNBR IN ('$OrdNum')"
Any assistance on the -join greatly appreciated OR if there is an alternative method to pass the values into SQL avoiding the -join then I'm open to suggestions. Thanks very much.
Thanks to Theo for the updated code. This works as expected.
I have also reworked my existing example with the following. Preserving the original grouping, this also works:
foreach ($csv in $CSVFiles) {
$csvOutput = Import-Csv $csv.FullName -Delimiter "`t"
$group = $csvOutput | Group-Object order-id, amount-type | Where-Object {$_.Group.'order-id' -ne '' -and $_.Group.'amount-type' -eq 'ItemPrice'}
($OrdNum = $csvOutput | Where-Object {![string]::IsNullOrWhiteSpace($_.'order-id')}).'order-id' | Out-Null
$OrdNum = ($OrdNum.'order-id' | Select-Object -Unique) -join "','"
}
I'm not quite sure if I understand the question properly, but I don't really see the need for grouping at all, when all you seem to want is an array of 'order-id' values joined with a comma.
# assuming $CSVFiles is a collection of FileInfo objects
$OrdNum = foreach ($csv in $CSVFiles) {
# import the csv and output the order numbers that match your where condition
(Import-Csv -Path $csv.FullName | Where-Object { ![string]::IsNullOrWhiteSpace($_.'order-id') -and $_.'amount-type' -eq 'ItemPrice'}).'order-id'
}
# if needed you can de-dupe the returned arrau with either `$OrdNum | Select-Object -Unique` or `$OrdNum | Sort-Object -Unique`
# join the array elements with a comma to use in your query
$OrdNum = $OrdNum -join ','
As per your comment, you need the grouping for other purposes later on.
In that case, something like this could work for you:
# create a List object to collect the order-id values
$orders = [System.Collections.Generic.List[string]]::new()
# loop through the CSV files and collect the grouped data in variable $group
$group = foreach ($csv in $CSVFiles) {
# import the csv and output objects that match your where condition
$items = Import-Csv -Path $csv.FullName | Where-Object { ![string]::IsNullOrWhiteSpace($_.'order-id') -and $_.'amount-type' -eq 'ItemPrice'}
if ($items) {
# add the 'order-id' values to the list
$orders.AddRange([string[]]($items.'order-id'))
# output the grouped items to collect in variable $group
$items | Group-Object order-id, amount-type
}
}
# join the elements with a comma to use in your query
$OrdNum = $orders -join ','
P.S. You need the [string[]] cast for the AddRange() method to avoid exception: Cannot convert argument "collection", with value: "System.Object[]", for "AddRange" to type "System.Collections.Generic.IEnumerable``1[System.String]": "Cannot convert the "System.Object[]
" value of type "System.Object[]" to type "System.Collections.Generic.IEnumerable``1[System.String]"."

Get count of rows in a partition of an azure table using Azure PowerShell

I would like to get count of rows in a partition. I have the code for getting the total count of rows. How can I alter it to get count for a particular partition. Also I am getting warning for fetching count of all rows and not getting the count on powershell window. Is there any documentation on this?
function GetTable($connectionString, $tableName)
{
$context = New-AzureStorageContext -ConnectionString $connectionString
$azureStorageTable = Get-AzureStorageTable $tableName -Context $context
$azureStorageTable
}
function GetTableCount($table)
{
#Create a table query.
$query = New-Object Microsoft.WindowsAzure.Storage.Table.TableQuery
#Define columns to select.
$list = New-Object System.Collections.Generic.List[string]
$list.Add("PartitionKey")
#Set query details.
$query.SelectColumns = $list
#Execute the query.
$entities = $table.CloudTable.ExecuteQuery($query)
($entities | measure).Count
}
$connectionString = "xyz"
$table = GetTable $connectionString SystemAudit
GetTableCount $table
How can I alter it to get count for a particular partition
There is a function Get-AzureStorageTableRowByPartitionKey you could use, and the following is the sample code
function GetTable($connectionString, $tableName)
{
$context = New-AzureStorageContext -ConnectionString $connectionString
$azureStorageTable = Get-AzureStorageTable $tableName -Context $context
$azureStorageTable
}
function GetTableCount($table)
{
$list = Get-AzureStorageTableRowByPartitionKey -table $table –partitionKey “storage” | measure
$list.Count
}
Import-Module AzureRmStorageTable
$connectionString = xyz"
$table = GetTable $connectionString <yourTableName>
GetTableCount $table
You can know more information on this blog

update sql table for Active Directory createdon and disabled on information

I have a user table in the database that i am trying to update with Createdon date and disabled on date with the data from Active Directory. So far this is what I have:
$SearchRoot = "OU=NonAIQ,OU=FrontOffice,DC=dev,DC=local"
$serverName = "localhost"
#$SearchRoot = "OU=NonAIQ,OU=FrontOffice,DC=dmz,DC=local"
#$serverName = "spoproddb3.dmz.local"
try {
Import-Module "sqlps" -DisableNameChecking
if ((Get-PSSnapin -Name "Quest.ActiveRoles.ADManagement" -ErrorAction SilentlyContinue) -eq $null ) {
Add-PsSnapin "Quest.ActiveRoles.ADManagement"
}
$externalUsers = Get-QADUser -SizeLimit 0 -SearchRoot $SearchRoot | Select-Object whencreated, whenchanged
$externalUsers | % {
$query = #"
Update tbl_EdgeUsers Set CompanyName = '$_.CreationDate'
Where UserUPN = $_.UserPrincipalName;
"#
Write-Host "The query is $query"
Invoke-SqlCmd -ServerInstance $serverName -Query $query -Database "EdgeDW"
}
} finally {
Remove-Module "sqlps" -ErrorAction SilentlyContinue
Remove-PsSnapin "Quest.ActiveRoles.ADManagement"
}
Now for when created, we just grab all the values.
But since AD does not track the Disabled in date, I am using the when changed date since we dont make changes to an account once it is changed.
The part that I am stuck on is about the logic for when changed date. For this I have to check if an account is disabled. If it is the update the table with that date. If an account is not disabled, then ignore that value and set the value in the sql table as '1/1/9999'.
can you guys please help with this logic?
Thank you in advance for any help.
of top of my head maybe something such as this, although thinking about it now, its a nasty way having the invoke-sql inside the foreach loop if the dataset is large, probably better to output the results of the if statement to csv or somewhere then run the invoke-sql against that.
$users = Get-ADUser -Filter * -Properties whenchanged | Select-Object -Property UserPrincipalName, whenchanged, enabled
$disabledsql = #"
update tbl_EdgeUsers Set date = '$user.whenchanged'
Where UserUPN = '$user.UserPrincipalName';
"#
$activesql = #"
update tbl_EdgeUsers Set date = '1/1/9999
Where UserUPN = '$user.UserPrincipalName';
"#
foreach ($user in $users)
{
if ($user.enabled -eq 'False')
{
Invoke-Sqlcmd -ServerInstance $serverName -Query $disabledsql -Database 'EdgeDW'
}
else
{
Invoke-Sqlcmd -ServerInstance $serverName -Query $activesql -Database 'EdgeDW'
}
}

Powershell SQL query results convertto-XML

I have been having quite the time trying to figure this out. Let me try to explain what I am trying to accomplish, I hope i can be clear enough.
I am sending two queries to an MSSQL database and receiving them back. The below code works perfect, however I would like to manipulate the format of the XML a bit before it writes to the XML file. I currently get 3 columns (serviceGroupName, numAccounts, numDevices) I would like to accomplish 1 of 2 things:
1) Add a new column named "ReportType" and have it fill in "Monthly" Or "Total" depending on if it is pass 1 or 2 of the foreach loop (SQLQuery1 is Monthly report, and SQLQuery2 is Total number since inception)
2) Create a new PSObject and have it fill in the appropriate information such as the data it receives back (serviceGroupName, numAccounts, numDevices)
Below is my current code. As i mentioned it does work and it generated an XML but i would like to add some more information before the pipe to ConvertTo-XML if possible.
### Dates to use
$Date = (Get-Date -f MM-dd-yyyy)
$FDoTM = ((Get-Date -Day 01).AddMonths(0)).AddDays(0)
$LDo2PM = ((Get-Date -Day 01).AddMonths(-1)).AddDays(-1)
$TempDir = "C:\Temp"
$WebDir = #("\\x.x.x.x\c$\inetpub\wwwroot\Reports\Accounts","\\x.x.x.x\c$\inetpub\wwwroot\Reports\Accounts")
### Something
$OutputXML = "$Date-Monthly-AccountReport.xml"
### Connection settings, uses windows authentication
$DBServer = "OMMITED"
$databasename = "OMMITED"
$Connection = new-object system.data.sqlclient.sqlconnection #Set new object to connect to sql database
$Connection.ConnectionString ="server=$DBServer;database=$databasename;trusted_connection=True" # Connectiongstring setting for local machine database with window authentication
Write-host "Connection Information:" -foregroundcolor yellow -backgroundcolor black
$Connection #List connection information
### Connect to Database and Run Query
$SqlCmd = New-Object System.Data.SqlClient.SqlCommand #setting object to use sql commands
$OutputHeader1 = "This Month's counts"
$SqlQuery1 = #"
SET NOCOUNT ON;
WITH AccountDeviceStats(serviceGroupName,numAccounts,numDevices)
AS
(
SELECT svg.name,COUNT(acct.serviceGroupId) as Accounts, NULL FROM bm_account acct WITH (NOLOCK)
INNER JOIN bm_servicegroup svg WITH (NOLOCK) ON svg.servicegroupId = acct.serviceGroupId
where acct.CreateStamp between '$($LDo2PM)' and '$($FDoTM)'
GROUP BY acct.serviceGroupId,svg.name
UNION ALL
SELECT svg.name, NULL, COUNT(device.serviceGroupId) as Devices FROM bm_device device WITH (NOLOCK)
INNER JOIN bm_servicegroup svg WITH (NOLOCK) ON svg.servicegroupId = device.serviceGroupId, bm_account acct
where device.accountID=acct.accountId and acct.CreateStamp between '$($LDo2PM)' and '$($FDoTM)'
GROUP BY device.serviceGroupId,svg.name
)
SELECT ad1.serviceGroupName,ad1.numAccounts,ad2.numDevices FROM AccountDeviceStats ad1
INNER JOIN AccountDeviceStats ad2 ON ad1.serviceGroupName = ad2.serviceGroupName
WHERE ad1.numAccounts IS NOT NULL AND ad2.numDevices IS NOT NULL
ORDER BY numAccounts DESC,numDevices DESC
"#
$OutputHeader2 = "Total Counts"
$SqlQuery2 = #"
SET NOCOUNT ON;
WITH AccountDeviceStats(serviceGroupName,numAccounts,numDevices)
AS
(
SELECT svg.name,COUNT(acct.serviceGroupId) as Accounts, NULL FROM bm_account acct WITH (NOLOCK)
INNER JOIN bm_servicegroup svg WITH (NOLOCK) ON svg.servicegroupId = acct.serviceGroupId
where acct.CreateStamp < '12-31-2099'
GROUP BY acct.serviceGroupId,svg.name
UNION ALL
SELECT svg.name, NULL, COUNT(device.serviceGroupId) as Devices FROM bm_device device WITH (NOLOCK)
INNER JOIN bm_servicegroup svg WITH (NOLOCK) ON svg.servicegroupId = device.serviceGroupId, bm_account acct
where device.accountID=acct.accountId and acct.CreateStamp < '12-31-2099'
GROUP BY device.serviceGroupId,svg.name
)
SELECT ad1.serviceGroupName,ad1.numAccounts,ad2.numDevices FROM AccountDeviceStats ad1
INNER JOIN AccountDeviceStats ad2 ON ad1.serviceGroupName = ad2.serviceGroupName
WHERE ad1.numAccounts IS NOT NULL AND ad2.numDevices IS NOT NULL
ORDER BY numAccounts DESC,numDevices DESC
"#
$sqlQueries = #($SqlQuery1, $SqlQuery2)
$Results = #()
Foreach ($Query in $sqlQueries){
$Connection.open()
Write-host "Connection to database successful." -foregroundcolor green -backgroundcolor black
$SqlCmd.CommandText = $Query
$SqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter
$SqlAdapter.SelectCommand = $SqlCmd
$SqlCmd.Connection = $Connection
$DataSet = New-Object System.Data.DataSet
$SqlAdapter.Fill($DataSet)
$Connection.Close()
$Results += $DataSet.Tables[0]
($Results | ConvertTo-XML -NoTypeInformation).Save("$TempDir\$OutputXML")
}
if ((Get-ChildItem $TempDir -filter "$Date-*.xml").count -gt 0){
Foreach ($file in (Get-ChildItem $TempDir -filter "$Date-*.xml" -recurse)){
Foreach ($webserver in $WebDir){
Copy-Item $file.fullname "$webserver\$file" -force
}
Remove-Item $file.fullname -force
}
}
Here is the output formatting of the XML
<?xml version="1.0"?>
<Objects>
<Object>
<Property Name="serviceGroupName">ServiceGroup1</Property>
<Property Name="numAccounts">15</Property>
<Property Name="numDevices">28</Property>
<Property Name="RowError" />
<Property Name="RowState">Unchanged</Property>
<Property Name="Table">
<Property>System.Data.DataRow</Property>
</Property>
<Property Name="ItemArray">
<Property>ServiceGroup1</Property>
<Property>15</Property>
<Property>28</Property>
</Property>
<Property Name="HasErrors">False</Property>
</Object>
<Object>
<Property Name="serviceGroupName">ServiceGroup1</Property>
<Property Name="numAccounts">45</Property>
<Property Name="numDevices">69</Property>
<Property Name="RowError" />
<Property Name="RowState">Unchanged</Property>
<Property Name="Table">
<Property>System.Data.DataRow</Property>
</Property>
<Property Name="ItemArray">
<Property>ServiceGroup1</Property>
<Property>45</Property>
<Property>69</Property>
</Property>
<Property Name="HasErrors">False</Property>
And one last thing. If it's possible to remove the excess bloat from the XML, as you can see it doubles the data output because it creates a node named ItemArray with all of the same information.
I hope this is easy enough to understand. If you need any more information, please let me know. And thank you in advance for any and all help.
I think all you need to do is to update your two T-sql queries within the powershell script. First one, add code like following:
...., "Monthly" as ReportType FROM AccountDeviceStats ad1...
Second one, add code like following:
...., "Total" as ReportType FROM AccountDeviceStats ad1...
### Dates to use
$Date = (Get-Date -f MM-dd-yyyy)
$FDoTM = ((Get-Date -Day 01).AddMonths(0)).AddDays(0)
$LDo2PM = ((Get-Date -Day 01).AddMonths(-1)).AddDays(-1)
$TempDir = "C:\Temp"
$WebDir = #("\\x.x.x.x\c$\inetpub\wwwroot\Reports\Accounts","\\x.x.x.x\c$\inetpub\wwwroot\Reports\Accounts")
### Something
$OutputXML = "$Date-Monthly-AccountReport.xml"
### Connection settings, uses windows authentication
$DBServer = "OMMITED"
$databasename = "OMMITED"
$Connection = new-object system.data.sqlclient.sqlconnection #Set new object to connect to sql database
$Connection.ConnectionString ="server=$DBServer;database=$databasename;trusted_connection=True" # Connectiongstring setting for local machine database with window authentication
Write-host "Connection Information:" -foregroundcolor yellow -backgroundcolor black
$Connection #List connection information
### Connect to Database and Run Query
$SqlCmd = New-Object System.Data.SqlClient.SqlCommand #setting object to use sql commands
$OutputHeader1 = "This Month's counts"
$SqlQuery1 = #"
SET NOCOUNT ON;
WITH AccountDeviceStats(serviceGroupName,numAccounts,numDevices)
AS
(
SELECT svg.name,COUNT(acct.serviceGroupId) as Accounts, NULL FROM bm_account acct WITH (NOLOCK)
INNER JOIN bm_servicegroup svg WITH (NOLOCK) ON svg.servicegroupId = acct.serviceGroupId
where acct.CreateStamp between '$($LDo2PM)' and '$($FDoTM)'
GROUP BY acct.serviceGroupId,svg.name
UNION ALL
SELECT svg.name, NULL, COUNT(device.serviceGroupId) as Devices FROM bm_device device WITH (NOLOCK)
INNER JOIN bm_servicegroup svg WITH (NOLOCK) ON svg.servicegroupId = device.serviceGroupId, bm_account acct
where device.accountID=acct.accountId and acct.CreateStamp between '$($LDo2PM)' and '$($FDoTM)'
GROUP BY device.serviceGroupId,svg.name
)
SELECT ad1.serviceGroupName,ad1.numAccounts,ad2.numDevices, ""Monthly"" as ReportType FROM AccountDeviceStats ad1
INNER JOIN AccountDeviceStats ad2 ON ad1.serviceGroupName = ad2.serviceGroupName
WHERE ad1.numAccounts IS NOT NULL AND ad2.numDevices IS NOT NULL
ORDER BY numAccounts DESC,numDevices DESC
"#
$OutputHeader2 = "Total Counts"
$SqlQuery2 = #"
SET NOCOUNT ON;
WITH AccountDeviceStats(serviceGroupName,numAccounts,numDevices)
AS
(
SELECT svg.name,COUNT(acct.serviceGroupId) as Accounts, NULL FROM bm_account acct WITH (NOLOCK)
INNER JOIN bm_servicegroup svg WITH (NOLOCK) ON svg.servicegroupId = acct.serviceGroupId
where acct.CreateStamp < '12-31-2099'
GROUP BY acct.serviceGroupId,svg.name
UNION ALL
SELECT svg.name, NULL, COUNT(device.serviceGroupId) as Devices FROM bm_device device WITH (NOLOCK)
INNER JOIN bm_servicegroup svg WITH (NOLOCK) ON svg.servicegroupId = device.serviceGroupId, bm_account acct
where device.accountID=acct.accountId and acct.CreateStamp < '12-31-2099'
GROUP BY device.serviceGroupId,svg.name
)
SELECT ad1.serviceGroupName,ad1.numAccounts,ad2.numDevices, ""Total"" as ReportType FROM AccountDeviceStats ad1
INNER JOIN AccountDeviceStats ad2 ON ad1.serviceGroupName = ad2.serviceGroupName
WHERE ad1.numAccounts IS NOT NULL AND ad2.numDevices IS NOT NULL
ORDER BY numAccounts DESC,numDevices DESC
"#
$sqlQueries = #($SqlQuery1, $SqlQuery2)
$Results = #()
Foreach ($Query in $sqlQueries){
$Connection.open()
Write-host "Connection to database successful." -foregroundcolor green -backgroundcolor black
$SqlCmd.CommandText = $Query
$SqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter
$SqlAdapter.SelectCommand = $SqlCmd
$SqlCmd.Connection = $Connection
$DataSet = New-Object System.Data.DataSet
$SqlAdapter.Fill($DataSet)
$Connection.Close()
$Results += $DataSet.Tables[0]
($Results | ConvertTo-XML -NoTypeInformation).Save("$TempDir\$OutputXML")
}
if ((Get-ChildItem $TempDir -filter "$Date-*.xml").count -gt 0){
Foreach ($file in (Get-ChildItem $TempDir -filter "$Date-*.xml" -recurse)){
Foreach ($webserver in $WebDir){
Copy-Item $file.fullname "$webserver\$file" -force
}
Remove-Item $file.fullname -force
}
}
The original question asked how to remove the bloat from the XML as well. I was looking for a solution where the XML that I was generating from the SQL results had to be in an absolute specific format with the correct tags and everything in place. What I discovered was that once you have your dataset object ($DataSet) then if you look to see what methods and properties are available to it, ($DataSet | gm) then one of them is GetXML().
This automatically formats your SQL output such that each returned column (or column alias) is returned as a separate tag (although note, it does not generate an empty tag for a null value) so in this instance if you use $DataSet.GetXML() I would have expected to see output something along the lines of
<NewDataSet>
<Table>
<serviceGroupName>ServiceGroup1</serviceGroupName>
<numAccounts>15</numAccounts>
<numDevices>28</numDevices>
</Table>
</NewDataSet>
so no bloat!
As this is just a series of strings, you can then do things like ($Dataset.GetXML()).Replace('NewDataSet','OuterTag').Replace('Table','InnerTag') to give better labels to the XML. Once you are happy with this you can output
SET-CONTENT -PATH $xmlfilename -VALUE '<?xml version="1.0" ?>'
or some such to a file and then append the output from your GetXML() method so you have a much neater formatted piece of XML!
($DataSet.GetXML()).Replace('NewDataSet','OuterTagName').Replace('Table','InnerTagName') | ADD-CONTENT -PATH $xmlfilename