I'm attempting to produce a variable containing a string of Object properties that are joined with "','". This is to pass into a SQL select where clause. My input looks like this:
foreach ($csv in $CSVFiles) {
$csvOutput = Import-Csv $csv.FullName
$group = $csvOutput | Group-Object order-id, amount-type | Where-Object {$_.Group.'order-id' -ne '' -and $_.Group.'amount-type' -eq 'ItemPrice'}}
Within the above loop. I'm looking to retrieve the order-id and pass it into a new variable $OrdNum. I'm doing this like so:
$OrdNum = $group | Select-Object #{Name='order-id';Expression={$_.Values[0]}}
To perform the join I have attempted:
$OrdNum = ($group | Select-Object #{Name='order-id';Expression={$_.Values[0]}}) -join "','"
This gives ','','','','','','','','','','','',' with no values.
I have also tried:
$OrdNum = ($group | Select-Object -Property 'order-id') -join "','"
Which produces the same result.
I'm expecting $OrdNum to look like 12345','43567','76334','23765 etc.
I'm working under the assumption that $OrdNum is required in that format to pass to this SQL query:
$query = “SELECT ARIBH.ORDRNBR AS [ORDER No'],AROBP.IDRMIT AS [RECPT No'], FROM [XXXX].[dbo].[AROBP] FULL JOIN [XXXX].[dbo].[ARIBH] ON [XXXX].[dbo].[AROBP].[IDMEMOXREF] = [XXXX].[dbo].[ARIBH].[IDINVC] where ARIBH.ORDRNBR IN ('$OrdNum')"
Any assistance on the -join greatly appreciated OR if there is an alternative method to pass the values into SQL avoiding the -join then I'm open to suggestions. Thanks very much.
Thanks to Theo for the updated code. This works as expected.
I have also reworked my existing example with the following. Preserving the original grouping, this also works:
foreach ($csv in $CSVFiles) {
$csvOutput = Import-Csv $csv.FullName -Delimiter "`t"
$group = $csvOutput | Group-Object order-id, amount-type | Where-Object {$_.Group.'order-id' -ne '' -and $_.Group.'amount-type' -eq 'ItemPrice'}
($OrdNum = $csvOutput | Where-Object {![string]::IsNullOrWhiteSpace($_.'order-id')}).'order-id' | Out-Null
$OrdNum = ($OrdNum.'order-id' | Select-Object -Unique) -join "','"
}
I'm not quite sure if I understand the question properly, but I don't really see the need for grouping at all, when all you seem to want is an array of 'order-id' values joined with a comma.
# assuming $CSVFiles is a collection of FileInfo objects
$OrdNum = foreach ($csv in $CSVFiles) {
# import the csv and output the order numbers that match your where condition
(Import-Csv -Path $csv.FullName | Where-Object { ![string]::IsNullOrWhiteSpace($_.'order-id') -and $_.'amount-type' -eq 'ItemPrice'}).'order-id'
}
# if needed you can de-dupe the returned arrau with either `$OrdNum | Select-Object -Unique` or `$OrdNum | Sort-Object -Unique`
# join the array elements with a comma to use in your query
$OrdNum = $OrdNum -join ','
As per your comment, you need the grouping for other purposes later on.
In that case, something like this could work for you:
# create a List object to collect the order-id values
$orders = [System.Collections.Generic.List[string]]::new()
# loop through the CSV files and collect the grouped data in variable $group
$group = foreach ($csv in $CSVFiles) {
# import the csv and output objects that match your where condition
$items = Import-Csv -Path $csv.FullName | Where-Object { ![string]::IsNullOrWhiteSpace($_.'order-id') -and $_.'amount-type' -eq 'ItemPrice'}
if ($items) {
# add the 'order-id' values to the list
$orders.AddRange([string[]]($items.'order-id'))
# output the grouped items to collect in variable $group
$items | Group-Object order-id, amount-type
}
}
# join the elements with a comma to use in your query
$OrdNum = $orders -join ','
P.S. You need the [string[]] cast for the AddRange() method to avoid exception: Cannot convert argument "collection", with value: "System.Object[]", for "AddRange" to type "System.Collections.Generic.IEnumerable``1[System.String]": "Cannot convert the "System.Object[]
" value of type "System.Object[]" to type "System.Collections.Generic.IEnumerable``1[System.String]"."
Related
Basically, I want the data to show up in an excel file like it shows in the SQL database.
This is much more simplified version of the work that I need to do but in essence this what it is.
I retrieve the data from SQL and for each item retrieved(which is the primary key) I want the data corresponding to it to be added in the hashtable. I then export this hashtable as a CSV
The CSV file is generated but with some weird data
I am not sure what exactly is wrong because when I Write-host $hashObject I can see the data is in there.
Code
$server = "DESKTOP\SQLEXPRESS"
$database = "AdventureWorks2019"
$hashTable = #{}
$hashObject = #([PSCustomObject]$hashTable)
$query = "SELECT[DepartmentID] FROM [AdventureWorks2019].[HumanResources].[Department]"
$invokeSql = Invoke-Sqlcmd -ServerInstance $server -Database $database -Query $query
$departmentResult = $invokeSql.DepartmentID
ForEach($department in $departmentResult){
$queryAll = "SELECT [Name],[GroupName],[ModifiedDate]FROM [AdventureWorks2019].[HumanResources].[Department] Where DepartmentID=$department"
$invokeSql = Invoke-Sqlcmd -ServerInstance $server -Database $database -Query $queryAll
$name = $invokeSql.Name
$groupName = $invokeSql.GroupName
$modifiedDate = $invokeSql.ModifiedDate
$hashObject+=("Department",$department, "Name",$name,"GroupName",$groupName,"ModifiedDate",$modifiedDate)
}
ConvertTo-Csv $hashObject| Export-Csv -Path "C:\Users\Desktop\PowerShell\HashTable_OutputFiles\HashOutput.csv"
This is a simplified version of what you're attempting to do, in this case you should be able to use the SQL IN Operator in your second query instead of querying your Database on each loop iteration.
As aside, is unclear what you wanted to do when declaring a hash table to then convert it to a PSCustomObject instance and then wrap it in an array:
$hashTable = #{}
$hashObject = #([PSCustomObject] $hashTable)
It's also worth noting that ConvertTo-Csv and Import-Csv are coded in such a way that they are intended to receive objects from the pipeline. This answer might help clarifying the Why. It's also unclear why are you attempting to first convert the objects to Csv and then exporting them when Import-Csv can (and in this case, must) receive the objects, convert them to a Csv string and then export them to a file.
$server = "DESKTOP\SQLEXPRESS"
$database = "AdventureWorks2019"
$query = "SELECT [DepartmentID] FROM [AdventureWorks2019].[HumanResources].[Department]"
$invokeSql = Invoke-Sqlcmd -ServerInstance $server -Database $database -Query $query
$department = "'{0}'" -f ($invokeSql.DepartmentID -join "','")
$query = #"
SELECT [Name],
[GroupName],
[ModifiedDate]
FROM [AdventureWorks2019].[HumanResources].[Department]
WHERE DepartmentID IN ($department);
"#
Invoke-Sqlcmd -ServerInstance $server -Database $database -Query $query |
Export-Csv -Path "C:\Users\Desktop\PowerShell\HashTable_OutputFiles\HashOutput.csv"
If you want to query the database per ID from the first query, you could do it this way (note this is similar to what you where looking to accomplish, merge the ID with the second results from the second query):
$invokeSql = Invoke-Sqlcmd -ServerInstance $server -Database $database -Query $query
$query = #"
SELECT [Name],
[GroupName],
[ModifiedDate]
FROM [AdventureWorks2019].[HumanResources].[Department]
WHERE DepartmentID = '{0}';
"#
& {
foreach($id in $invokeSql.DepartmentID) {
$queryID = $query -f $id
Invoke-Sqlcmd -ServerInstance $server -Database $database -Query $queryID |
Select-Object #{ N='DepartmentID'; E={ $id }}, *
}
} | Export-Csv -Path "C:\Users\Desktop\PowerShell\HashTable_OutputFiles\HashOutput.csv"
I have a Linq query in VB.Net which works fine.
I now need to transform this to work in PowerShell by using the static Linq methods.
I figured out how to do simple select and where queries but especially the grouping and "new with" I am not sure how to translate...
The goal is to find out if the entries (dataRows) in a Data Table that do have the same AttributeKey also have the same AttributeValue AndAlso the same AttributeType or not.
Dim dt As New DataTable
dt.Columns.Add("AttributeKey")
dt.Columns.Add("AttributeValue")
dt.Columns.Add("AttributeType")
dt.Rows.Add("Action", "MakeItSo", "System.String")
dt.Rows.Add("Action", "MakeItSo", "System.String")
dt.Rows.Add("Action", "True", "System.Boolean")
dt.Rows.Add("Single", "MyVal", "System.String")
Dim srcGroupBy = dt.AsEnumerable.GroupBy(Function(Attrib) New With
{
Key .AttributeKey = Attrib.Field(Of String)("AttributeKey")
}).
Where(Function(grp) grp.Count() > 1).
Select(Function(grp) New With {
Key .AttributeKey = grp.Key.AttributeKey,
Key .CounterVal = grp.GroupBy(Function(AttributeVal) AttributeVal.Item("AttributeValue").ToString().ToUpper()).Count(),
Key .CounterType = grp.GroupBy(Function(AttributeType) AttributeType.Item("AttributeType").ToString().ToUpper()).Count(),
Key .ExistsInInconsistentStates = grp.GroupBy(Function(x) x.Item("AttributeValue").ToString().ToUpper()).Count() > 1 Or
grp.GroupBy(Function(x) x.Item("AttributeType").ToString().ToUpper()).Count() > 1
})
I could also solve this without Linq in PS like this:
[PSObject]$res=$DV |
Group-Object -Property "AttributeKey" |
Where-Object -FilterScript {$PsItem.Count -gt 1} |
Select-Object -Property #("Name","Count",#{Name='CountDistinctVals';Expression=`
{
$PsItem.Group.AttributeValue |
Select-Object -Unique |
Measure-Object |
Select-Object -ExpandProperty "Count"
}`
},#{Name='CountVals';Expression=`
{
$PsItem.Group.AttributeValue |
Select-Object |
Measure-Object |
Select-Object -ExpandProperty "Count"
}`
},#{Name='CountDistinctTypes';Expression=`
{
$PsItem.Group.AttributeType |
Select-Object -Unique |
Measure-Object |
Select-Object -ExpandProperty "Count"
}`
},#{Name='CountTypes';Expression=`
{
$PsItem.Group.AttributeType |
Select-Object |
Measure-Object |
Select-Object -ExpandProperty "Count"
}`
})
I can then evaluate the first row and check if distinct count = all count which would also meet my requirement for that particular use case.
Anyway, it would be still interesting for me to know whether it could be done also in Linq with PS. (Not saying that this would always be the preferred way)
Thanks
Chris
I'm using the Microsoft 365 Defender API to receive all recent events/incidents.
I get a json file as following: link to example json
And use following script to try and convert this for easy import to an SQL server:
(Echoes only as test)
# Send the request and get the results.
$response = Invoke-WebRequest -UseBasicParsing -Method Get -Uri $url -Headers $headers -ErrorAction Stop
# Extract the incidents from the results.
$alerts = ($response | ConvertFrom-Json)
$devices = ($response | ConvertFrom-Json ).value.alerts.devices
$entities = ($response | ConvertFrom-Json ).value.alerts.entities
Foreach($row in $alerts){
$IncidentID = $alerts.value.incidentID
$Createdtime = $alerts.value.creationTime
$Status = $alerts.value.status
$Severity = $alerts.value.severity
$Classification = $alerts.value.classification
$IncidentName = $alerts.value.incidentName
$URL = $alerts.incidentUri
$Klant = $afkorting
$Username = $entities.accountname
$device = $devices.deviceDnsName
echo $IncidentID
echo $Createdtime
echo $Status
echo $Severity
echo $Classification
echo $IncidentName
echo $URL
echo $Klant
echo $Username
echo $device
Invoke-Sqlcmd -ServerInstance "SQL.domain.local\MSQL2016" -Database "private" -Username private -Password 'private' -Query "INSERT Into dbo.private ( [IncidentID], [Createdtime], [Status], [Severity], [Classification], [IncidentName], [URL], [Klant], [Username], [device]) VALUES ('$IncidentID', '$Createdtime', '$Status', '$Severity', '$Classification', '$IncidentName', '$URL', '$Klant', '$Username', '$device')"
}
However, the output in case of 3 incidents looks like:
IncidentID
IncidentID
IncidentID
Createdtime
Createdtime
Createdtime
Status
Status
Status
So grouped by element instead of grouped by IncidentID.
I can't find a way to get the output like:
IncidentID
Createdtime
Status
Severity
Classification
IncidentName
URL
Klant
Username
device
I "solved" this with an intermediary step exporting to CSV's and merging them and piping those to SQL for now, but that's too inefficient.
Move resolution of $devices and $entities into the loop, then use the iterator variable $row instead of referencing all $alerts inside the loop body:
# Send the request and get the results.
$response = Invoke-WebRequest -UseBasicParsing -Method Get -Uri $url -Headers $headers -ErrorAction Stop
# Extract the incidents from the results.
$alerts = ($response | ConvertFrom-Json)
foreach($row in $alerts){
$IncidentID = $row.value.incidentID
$Createdtime = $row.value.creationTime
$Status = $row.value.status
$Severity = $row.value.severity
$Classification = $row.value.classification
$IncidentName = $row.value.incidentName
$URL = $row.incidentUri
$Klant = $afkorting # where does `$afkorting` come from?
$Username = $row.value.entities.accountname
$device = $row.value.devices.deviceDnsName
# Insert into SQL Server here
}
I have the below code to get data from a SQL DB and export it into a CSV file:
#Server and Database names
$SQLServer = "Servername"
$DB = "DatabaseName"
#SQL Command
$FullScriptSQL="Select * from MyTable WHERE Column = 'TestData'"
#Invoke the command, rename the column headers and export to CSV file
$FullScriptCallLinked = Invoke-Sqlcmd -ServerInstance $SQLServer -Database $DB -Query $FullScriptSQL | select-object #{ expression={$_.Column1};label='Column1 Name'},#{ expression={$_.Column2};label='Column2 Name'},#{ expression={$_.Column3}; label='Column3 Name' },#{ expression={$_.Column4} ; label='Column4 Name' }
Export-CSV -Path ".\ResultFile\FullScript.csv" -inputobject $FullScriptCallLinked -Append -NoTypeInformation
This works perfectly if there is one result. But if there is more than one result, it will show the below in the csv file
I am at my wits end as to why it is doing this. It's obviously the DB parameter data or something to that effect. Been googling for a few days with no luck. Anyone smarter than I able to assist please?
Instead of using Select-Object to rename your columns, which is quite inefficient, you could give the alias to your columns on the query itself:
$SQLServer = "Servername"
$DB = "DatabaseName"
$query = #'
SELECT Column1 AS "Column1 Name",
Column2 AS "Column2 Name",
Column3 AS "Column3 Name",
Column4 AS "Column4 Name"
FROM MyTable
WHERE ColumnX = 'TestData'
'#
Invoke-Sqlcmd -ServerInstance $SQLServer -Database $DB -Query $query |
Export-CSV -Path ".\ResultFile\FullScript.csv" -NoTypeInformation
Also, as in my comment, the code you have on your question is fine and should work, the only problem was using -InputObject instead of piping the results to Export-Csv:
$FullScriptCallLinked | Export-Csv -Path ".\ResultFile\FullScript.csv" -NoTypeInformation
Figured it out. I knew I was close!
#Server and Database names
$SQLServer = "Servername"
$DB = "DatabaseName"
#SQL Command
$FullScriptSQL="Select * from MyTable WHERE Column = 'TestData'"
#Invoke the command, rename the column headers and export to CSV file
$FullScriptCallLinked = Invoke-Sqlcmd -ServerInstance $SQLServer -Database $DB -Query $FullScriptSQL
foreach($i in $FullScriptCallLinked){
select-object #{ expression={$_.Column1};label='Column1 Name'},#{ expression={$_.Column2};label='Column2 Name'},#{ expression={$_.Column3}; label='Column3 Name' },#{ expression={$_.Column4} ; label='Column4 Name' }
Export-CSV -Path ".\ResultFile\FullScript.csv" -inputobject $i -Append -NoTypeInformation
}
I have a user table in the database that i am trying to update with Createdon date and disabled on date with the data from Active Directory. So far this is what I have:
$SearchRoot = "OU=NonAIQ,OU=FrontOffice,DC=dev,DC=local"
$serverName = "localhost"
#$SearchRoot = "OU=NonAIQ,OU=FrontOffice,DC=dmz,DC=local"
#$serverName = "spoproddb3.dmz.local"
try {
Import-Module "sqlps" -DisableNameChecking
if ((Get-PSSnapin -Name "Quest.ActiveRoles.ADManagement" -ErrorAction SilentlyContinue) -eq $null ) {
Add-PsSnapin "Quest.ActiveRoles.ADManagement"
}
$externalUsers = Get-QADUser -SizeLimit 0 -SearchRoot $SearchRoot | Select-Object whencreated, whenchanged
$externalUsers | % {
$query = #"
Update tbl_EdgeUsers Set CompanyName = '$_.CreationDate'
Where UserUPN = $_.UserPrincipalName;
"#
Write-Host "The query is $query"
Invoke-SqlCmd -ServerInstance $serverName -Query $query -Database "EdgeDW"
}
} finally {
Remove-Module "sqlps" -ErrorAction SilentlyContinue
Remove-PsSnapin "Quest.ActiveRoles.ADManagement"
}
Now for when created, we just grab all the values.
But since AD does not track the Disabled in date, I am using the when changed date since we dont make changes to an account once it is changed.
The part that I am stuck on is about the logic for when changed date. For this I have to check if an account is disabled. If it is the update the table with that date. If an account is not disabled, then ignore that value and set the value in the sql table as '1/1/9999'.
can you guys please help with this logic?
Thank you in advance for any help.
of top of my head maybe something such as this, although thinking about it now, its a nasty way having the invoke-sql inside the foreach loop if the dataset is large, probably better to output the results of the if statement to csv or somewhere then run the invoke-sql against that.
$users = Get-ADUser -Filter * -Properties whenchanged | Select-Object -Property UserPrincipalName, whenchanged, enabled
$disabledsql = #"
update tbl_EdgeUsers Set date = '$user.whenchanged'
Where UserUPN = '$user.UserPrincipalName';
"#
$activesql = #"
update tbl_EdgeUsers Set date = '1/1/9999
Where UserUPN = '$user.UserPrincipalName';
"#
foreach ($user in $users)
{
if ($user.enabled -eq 'False')
{
Invoke-Sqlcmd -ServerInstance $serverName -Query $disabledsql -Database 'EdgeDW'
}
else
{
Invoke-Sqlcmd -ServerInstance $serverName -Query $activesql -Database 'EdgeDW'
}
}