I am quite new to Powershell and I use it only for some Azure work I am recently assigned with.
I noticed a behavior that I can't understand. When I run Get-AzNetworkInterface cmdlet, the field IpConfigurations will displays all details like this
IpConfigurations : [
{
"Name": "ipconfig1",
"Etag": "blabla",
"Id": "blabla",
"PrivateIpAddress": "10.1.2.3",
....
}
]
However, if I append select-object, the result becomes just the {ipconfig}, which is just the name of the field
> Get-AzNetworkInterface | Where-Object { $_.Name -eq "some nic name" } | Select-Object "IpConfigurations"
IpConfigurations
----------------
{ipconfig1}
I have the same result if I view the output with | Out-Gridview. The IpConfigurations field would only show the name. However, it would also show an IpConfigurationsText field that contains all the details. It got truncated so I can't see the full text. By the way, is there a way for the popup GridView window to show the full content? Anyhow, I then tried to select this field in the Shell console but still got nothing.
> Get-AzNetworkInterface | Where-Object { $_.Name -eq "some nic name" } | Select-Object IpConfigurationsText
IpConfigurationsText
--------------------
[…
What is the logic here for the Azure Powershell cmdlet to expand fields here?
Thanks!
Difan
You can use the below cmdlet to expand the Ipconfigurations property
Get-AzNetworkInterface | Where-Object { $_.Name -eq "NICName" } |Select-Object -ExpandProperty IpConfigurations
Here is the sample output for reference:
If you want to expand the IpConfigurations Property & project the output in Gridview we would suggest you to use the below PowerShell cmdlet
Get-AzNetworkInterface | Where-Object { $_.Name -eq "NICName" } |Select-Object -ExpandProperty IpConfigurations|Out-GridView -PassThru
Here is the sample output for reference :
Related
I am trying to loop around all of our subscriptions and get Policy Exemptions, but only get the ones that we have created. The loop appears fine, but the Match element appears to bring back some Exemptions that don't meet the -Match criteria.
$allSubscriptions = Get-AzSubscription
$baseFolder = "C:\source\PowerShell Exemptions Dump\"
# loop subscriptions
foreach($sub in $allSubscriptions){
$subName = $sub.Name
# Get Exemptions at Sub level
Set-AzContext -Subscription $subName
# Write to File
$exemptionsIn = Get-AzPolicyExemption|ConvertTo-Json
$fileName = $baseFolder + $subName + ".json"
$exemptionsOut = ''
foreach($ex in $exemptionsIn|ConvertFrom-Json){
if($ex.Properties.PolicyAssignmentId -Match "abc") {
$exemptionsOut += $ex|ConvertTo-Json
}
}
if ($exemptionsOut -ne '') {
$exemptionsOut | Out-File -filepath $fileName
$exemptionsOut = ''
}
}
It does work to a certain extent i.e. if a Subscription has a 0% match in everything it brings back, then it doesn't create a file. but it appears if it finds one match, then it saves Exemptions to the file that don't match.
Here is some example Json that was saved to one of the files:
[
{
"Properties": {
"PolicyAssignmentId": "/providers/Microsoft.Management/managementGroups/abc-mg/providers/Microsoft.Authorization/policyAssignments/abc-mg",
"PolicyDefinitionReferenceIds": "",
"ExemptionCategory": "Waiver",
"DisplayName": "abc - abc-mg Policy Assignment",
"Description": "AIB Testing",
"ExpiresOn": "\/Date(1662134400000)\/",
"Metadata": ""
},
"SystemData": null,
"Name": "456",
"ResourceId": "/subscriptions/123/providers/Microsoft.Authorization/policyExemptions/789",
"ResourceName": "456",
"ResourceGroupName": null,
"ResourceType": "Microsoft.Authorization/policyExemptions",
"SubscriptionId": "123"
},
{
"Properties": {
"PolicyAssignmentId": "/providers/Microsoft.Management/managementGroups/root-mg/providers/Microsoft.Authorization/policyAssignments/111",
"PolicyDefinitionReferenceIds": "installEndpointProtection",
"ExemptionCategory": "Waiver",
"DisplayName": "root-mg - Azure Security Benchmark",
"Description": "currently use sophos and not defender",
"ExpiresOn": null,
"Metadata": ""
},
"SystemData": null,
"Name": "345",
"ResourceId": "/providers/Microsoft.Management/managementGroups/root-mg/providers/Microsoft.Authorization/policyExemptions/345",
"ResourceName": "345",
"ResourceGroupName": null,
"ResourceType": "Microsoft.Authorization/policyExemptions",
"SubscriptionId": null
}
]
Finally, I don't appear to get all Exemptions back in this loop i.e. some are set at Resource Group or Resource Level. Do I need to drill further beyond Set-AzContext?
After reproducing the same code from my end, I could able to see the expected results. However, make sure you are checking in the right file and the location to which you are sending your data to.
Finally, I don't appear to get all Exemptions back in this loop i.e. some are set at Resource Group or Resource Level.
This might be due to the scope that you are looking into. After setting the scope to the required level I could able to get the expected results. Below is the code that worked for me.
$Resource = Get-AzResource -ResourceGroupName <YOUR_RESOURCEGROUP_NAME>
for($I=0;$I -lt $Resource.ResourceId.Count;$I++)
{
$a=Get-AzPolicyExemption -Scope $Resource.ResourceId[$I]
for($J=0;$J -lt $a.Count;$J++)
{
If($a.ResourceId[$J] -Match $Resource.ResourceId[$I])
{
$exemptionsIn = Get-AzPolicyExemption -Scope $Resource.ResourceId[$I] | ConvertTo-Json
$fileName = "sample2" + ".json"
$exemptionsOut = ''
foreach($ex in $exemptionsIn|ConvertFrom-Json){
if($ex.Properties.PolicyAssignmentId -Match "Swetha*") {
$exemptionsOut += $ex|ConvertTo-Json
}
}
if ($exemptionsOut -ne '') {
$exemptionsOut | Out-File -filepath $fileName
$exemptionsOut = ''
}
}
}
}
I have few policy exemptions in my subscription but above script gave me the results at Resource level which -Match with Swetha.
RESULTS:
I'm working with a simplified example in which there are workers which can have multiple lifecycles in which they perform tasks. (This is similar to the example of users logging into different sessions and performing shell commands given in https://community.splunk.com/t5/Splunk-Search/Any-example-for-MAP-command/m-p/88473).
When a task is started, a taskID and lifecycleID is logged. However, I would also like to look up the corresponding workerID which would have been logged together with the lifecycleID in a previous log line when the lifecycle started.
Consider the following example data:
{
"level": "info",
"lifecycleID": "af331787-654f-441f-ac06-21b6b7e0c984",
"msg": "Started lifecycle",
"time": "2022-04-02T21:15:38.07991-07:00",
"workerID": "c51df20b-f157-4002-8292-4583ebd3ba9e"
}
{
"level": "info",
"lifecycleID": "af331787-654f-441f-ac06-21b6b7e0c984",
"msg": "Started task",
"taskID": "9de93d09-5e6e-4648-9488-dda0e3e58765",
"time": "2022-04-02T21:15:38.181107-07:00"
}
{
"level": "info",
"lifecycleID": "03d2148c-b697-4d8e-a3ca-f0fb68d2bbb9",
"msg": "Started lifecycle",
"time": "2022-04-02T21:15:38.282264-07:00",
"workerID": "c51df20b-f157-4002-8292-4583ebd3ba9e"
}
{
"level": "info",
"lifecycleID": "03d2148c-b697-4d8e-a3ca-f0fb68d2bbb9",
"msg": "Started task",
"taskID": "243bf757-85c6-4c6e-9eec-6d74886ec407",
"time": "2022-04-02T21:15:38.383176-07:00"
}
{
"level": "info",
"lifecycleID": "9cab44b4-5600-47b3-9acd-47b2641cb0d5",
"msg": "Started lifecycle",
"time": "2022-04-02T21:15:38.483304-07:00",
"workerID": "0b82966c-cc98-48f0-9a36-a699e2cee48c"
}
{
"level": "info",
"lifecycleID": "9cab44b4-5600-47b3-9acd-47b2641cb0d5",
"msg": "Started task",
"taskID": "864819ed-208d-4d3d-96b9-1af4c4c42b08",
"time": "2022-04-02T21:15:38.584478-07:00"
}
{
"level": "info",
"lifecycleID": "9cab44b4-5600-47b3-9acd-47b2641cb0d5",
"msg": "Finished task",
"taskID": "864819ed-208d-4d3d-96b9-1af4c4c42b08",
"time": "2022-04-02T21:15:38.684633-07:00"
}
I would like to generate a table which shows the workerID, lifecycleID, and taskID for each of the three tasks started. So far what I've come up with is
index="workers" msg="Started task"
| stats count by lifecycleID
| map search="search index=workers msg=\"Started lifecycle\" lifecycleID=$lifecycleID$"
| table workerID, lifecyleID, taskID
However, this doesn't appear to retain the lifecycleID and taskID (like it would if I were to omit the map and simply count by lifecycleID, taskID):
How can I make it such that I can display all three values in the table?
Update
I've attempted RichG's answer using a subsearch,
index=workers msg="Started lifecycle"
[ search index="workers" msg="Started task"
| stats count by lifecycleID
| fields lifecycleID
| format ]
| table workerID, lifecyleID, taskID
but it generates output that is identical to the one generated in my own attempt using a map, i.e. without the lifecycleID or taskID:
Try using a subsearch instead of map. In the subsearch below (the part inside square brackets), a list of unique lifecycleID values is produced and formatted into (lifecycleID="foo" OR lifecycleID="bar"). That string is substituted for the subsearch to produce a search for all "Started lifecycle" events with one of the specified lifecycleID's.
index=workers msg="Started lifecycle"
[ search index="workers" msg="Started task"
| stats count by lifecycleID
| fields lifecycleID
| format ]
| table workerID, lifecyleID, taskID
Another method for combining events is the stats command. See the run-anywhere example below.
| makeresults
| eval data="{\"level\": \"info\",\"lifecycleID\": \"af331787-654f-441f-ac06-21b6b7e0c984\",\"msg\": \"Started lifecycle\",\"time\": \"2022-04-02T21:15:38.07991-07:00\",\"workerID\": \"c51df20b-f157-4002-8292-4583ebd3ba9e\"}
{\"level\": \"info\",\"lifecycleID\": \"af331787-654f-441f-ac06-21b6b7e0c984\",\"msg\": \"Started task\",\"taskID\": \"9de93d09-5e6e-4648-9488-dda0e3e58765\",\"time\": \"2022-04-02T21:15:38.181107-07:00\"}
{\"level\": \"info\",\"lifecycleID\": \"03d2148c-b697-4d8e-a3ca-f0fb68d2bbb9\",\"msg\": \"Started lifecycle\",\"time\": \"2022-04-02T21:15:38.282264-07:00\",\"workerID\": \"c51df20b-f157-4002-8292-4583ebd3ba9e\"}
{\"level\": \"info\",\"lifecycleID\": \"03d2148c-b697-4d8e-a3ca-f0fb68d2bbb9\",\"msg\": \"Started task\",\"taskID\": \"243bf757-85c6-4c6e-9eec-6d74886ec407\",\"time\": \"2022-04-02T21:15:38.383176-07:00\"}
{\"level\": \"info\",\"lifecycleID\": \"9cab44b4-5600-47b3-9acd-47b2641cb0d5\",\"msg\": \"Started lifecycle\",\"time\": \"2022-04-02T21:15:38.483304-07:00\",\"workerID\": \"0b82966c-cc98-48f0-9a36-a699e2cee48c\"}
{\"level\": \"info\",\"lifecycleID\": \"9cab44b4-5600-47b3-9acd-47b2641cb0d5\",\"msg\": \"Started task\",\"taskID\": \"864819ed-208d-4d3d-96b9-1af4c4c42b08\",\"time\": \"2022-04-02T21:15:38.584478-07:00\"}
{\"level\": \"info\",\"lifecycleID\": \"9cab44b4-5600-47b3-9acd-47b2641cb0d5\",\"msg\": \"Finished task\",\"taskID\": \"864819ed-208d-4d3d-96b9-1af4c4c42b08\",\"time\": \"2022-04-02T21:15:38.684633-07:00\"}"
| eval data=split(data,"
")
| mvexpand data
| eval _raw=data
| extract
```Everything above is just to set up test data. Omit IRL```
```Combine events that share the same taskID```
| stats values(*) as * by lifecycleID
| table workerID, lifecycleID, taskID
I realized that this could be achieved by a join query:
index=workers msg="Started lifecycle"
| join lifecycleID
[ search index=workers msg="Started task"]
| table workerID, lifecycleID, taskID
The results are shown below.
I cannot find any doc for Fish Shell regarding using Command Substitution more than once.
I'm trying to assign the state, city from the JSON result set (jq parser) piped from a curl API query of LocationIQ. 2 Command Substitution 1:(curl) and 2:(jq). I don't need the location variable assignment if I can get the address variable assignment
Purpose of Function:
#Take 2 arguments (Latitude, Longitude) and return 2 variables $State, $City
The JSON:
{
"address": {
"city": "Aurora",
"country": "United States of America",
"country_code": "us",
"county": "Kane County",
"postcode": "60504",
"road": "Ridge Road",
"state": "Illinois"
},
"boundingbox": [
"41.729347",
"41.730247",
"-88.264466",
"-88.261979"
],
"display_name": "Ridge Road, Aurora, Kane County, Illinois, 60504, USA",
"importance": 0.2,
"lat": "41.729476",
"licence": "https://locationiq.com/attribution",
"lon": "-88.263423",
"place_id": "333878957973"
}
My Function:
function getLocation
set key 'hidden'
set exifLat $argv[1]
set exifLon $argv[2]
set location (curl -s "https://us1.locationiq.com/v1/reverse.phpkey=$key&lat=$exifLat&lon=$exifLon&format=json" | set address (jq --raw-output '.address.state,.address.city') )
echo "Location: $location
echo "state: $address[1]"
echo "city: $address[2]"
end
Error: fish Command substitution not allowed
Works fine using only the curl Command substitution ->removing the: set address & parens for jq.
set location (curl -s "https://us1.locationiq.com/v1/reverse.phpkey=$key&lat=$exifLat&lon=$exifLon&format=json" | jq --raw-output '.address.state,.address.city')
I'm still pretty novice - maybe there is a better way to achieve my desired result: Assign the JSON State to a variable and City to a variable?
I originally tried (slicing the location[17] - City, location[19] - State) and getting inconsistent results as the fields seem to be dynamic and affecting how many results which affects the ordering.
Any help appreciated!
I find the nested set confusing. Did you intend to do use $location to hold the downloaded JSON data, and $address to hold the results of jq? If yes, split them out into separate statements
set url "https://us1.locationiq.com/v1/reverse.phpkey=$key&lat=$exifLat&lon=$exifLon&format=json"
set location (curl -s $url)
set address (echo $location | jq --raw-output '.address.state,.address.city')
I have an array:
[
{
"AssetId": 14462955,
"Name": "Cultural Item"
},
{
"AssetId": 114385498,
"Name": "Redspybot"
},
{
"AssetId": 29715011,
"Name": "American Cowboy"
},
{
"AssetId": 98253651,
"Name": "Mahem"
}
]
I would like to loop through each object in this array, and pick out the value of each key called AssetId and output it.
How would I do this using jq for the command line?
The command-line tool jq writes to STDOUT and/or STDERR. If you want to write the .AssetId information to STDOUT, then one possibility would be as follows:
jq -r ".[] | .AssetId" input.json
Output:
14462955
114385498
29715011
98253651
A more robust incantation would be: .[] | .AssetId? but your choice will depend on what you want if there is no key named "AssetId".
You can also do it via this command.
jq ".[].AssetId" input.json
if array like be that which is in my case
{
"resultCode":0,
"resultMsg":"SUCCESS",
"uniqueRefNo":"111222333",
"list":[
{
"cardType":"CREDIT CARD",
"isBusinessCard":"N",
"memberName":"Bank A",
"memberNo":10,
"prefixNo":404591
},
{
"cardType":"DEBIT CARD",
"isBusinessCard":"N",
"memberName":"Bank A",
"memberNo":10,
"prefixNo":407814
},
{
"cardType":"CREDIT CARD",
"isBusinessCard":"N",
"memberName":"Bank A",
"memberNo":10,
"prefixNo":413226
}
]
}
you can get the prefixNo with below jq command.
jq ".list[].prefixNo" input.json
For more specific case on array iterating on jq you can check this blogpost
you have a couple of choices to do the loop itself. you can apply peak's awesome answer and wrap a shell loop around it. replace echo with the script you want to run.
via xargs
$ jq -r ".[] | .AssetId" input.json | xargs -n1 echo # this would print
14462955
114385498
29715011
98253651
via raw loop
$ for i in $(jq -r ".[] | .AssetId" input.json)
do
echo $i
done
14462955
114385498
29715011
98253651
An alternative using map:
jq "map ( .AssetId ) | .[]"
For your case jq -r '.[].AssetId' should work
You can also use online JQ Parser : https://jqplay.org/
If you want to loop through the each value then can use below :
for i in $(echo $api_response | jq -r ".[].AssetId")
do
echo echo $i
done
I need to interact with an API that is expecting an array of objects, among other parameters. Example:
{
"fields": {
"somefield": "somevalue",
"someobject": {
"name": "foobar"
},
"versions": [
{
"name": "1.0"
}
]
}
}
With the help of this answer, I've tried two different ways of handling this. I've combined them into a single code example:
$versionName = New-Object -TypeName PSObject
$versionName | Add-Member -Name "name" -MemberType NoteProperty -Value "1.0"
$versionName2 = #{}
$versionName2.name = "1.0"
$postIssueBody = #{}
$postIssueBody.fields = #{}
$postIssueBody.fields.somefield = "somevalue"
$postIssueBody.fields.someobject = #{}
$postIssueBody.fields.someobject.name = "foobar"
$postIssueBody.fields.version = #($versionName)
$postIssueBody.fields.version2 = #()
$postIssueBody.fields.version2 += [pscustomobject]$versionName2
$postIssueRequestJson = $postIssueBody | ConvertTo-Json
$postIssueRequestJson
This results in the following output:
{
"fields": {
"somefield": "somevalue",
"someobject": {
"name": "foobar"
},
"version": [
"#{name=1.0}"
],
"version2": [
"#{name=1.0}"
]
}
}
As you can see, that's not going to fly as valid JSON. What is the best way to handle this assignment so that the version names are properly formed after going through ConvertTo-Json?
The ConvertTo-Json function has a switch called Depth. It informs the Convert function how deep it should go when converting data to the JSON format. By default, it is set at 2. Since the data that isn't being converted properly sits at a depth of 3, we simply need to set the depth to that, like so:
$postIssueRequestJson = $postIssueBody | ConvertTo-Json -Depth 3
And now we have well-formed JSON.
{
"fields": {
"somefields": "somevalue",
"someobject": {
"name": "foobar"
},
"versions": [
{
"name": "1.0"
}
]
}
}
Ok, I think I understand. So you need a string that starts with "versions": and is followed by an array of objects, yes? So, let's start with an empty array.
$Array = #()
Then we can create objects, and add those to the array:
$Array += [PSCustomObject]#{"Name1"="1.0.0"}
$Array += [PSCustomObject]#{"Name2"="3.10.0"}
Now we have a PowerShell array with PSCustomObjects in it. We can pipe that to ConvertTo-JSON and it will output:
[
{
"Name1": "1.0.0"
},
{
"Name2": "3.10.0"
}
]
Which is the array of objects you wanted. If you want an object to have that as it's value you could simply create another object to do that with:
$Versions = [PSCustomObject]#{'versions'=$Array}
Then you can convert that to JSON if you want and get:
{
"versions": [
{
"Name1": "1.0.0"
},
{
"Name2": "3.10.0"
}
]
}
That's what you were looking for, right? Or if you really want it on one line:
PS C:\> ($Versions|convertto-json).split() -join ""
{"versions":[{"Name1":"1.0.0"},{"Name2":"3.10.0"}]}
To be formatted exactly as your first example we would have to get rid of the { } surrounding that result I suppose, you can do that with Trim() as such:
PS C:\> ($Versions|convertto-json).trim("{}").split() -join ""
"versions":[{"Name1":"1.0.0"},{"Name2":"3.10.0"}]
Edit: Ok, so you just need to add objects as property values of other objects as needed, much like I did for setting the array as a value in the object in my example.
I think the easiest way to understand what needs to be done is to take your example (minus the last comma, since that makes it throw errors), and pipe it into ConvertFrom-JSON and assign it a variable. Then you can see how that is formed in Powersehll. Once I do that (I named my variable $JSON), I can see that $JSON has 1 NoteProperty of 'fields'. That NoteProperty has 3 NoteProperties of 'somefield', 'someobject', and 'versions'. When I do a $JSON.fields|Get-Member I find out more about those.
somefield is just a String. That will be easy enough to deal with.
someobject is a PSCustomObject, basically a HashTable where name=foobar.
versions just shows that it's a System.Object, so I'll do $JSON.fields.versions.GetType() and it shows that the basetpe is System.Array. After looking at versions it looks like an array with 1 object in it, and that object has one noteproperty that is a string (like the first object we had).
So, there's two ways to go about doing this. You can either try and create your objects and array in-line, or you can make them ahead of time, starting at the deepest nested layer, and work your way up. I'll be showing you the later.
$name = [PSCustomObject]#{'name'='1.0'}
$versions=#($name)
$Someobject = [PSCustomObject]#{'name'='foobar'}
$Fields = [PSCustomObject]#{
'somefields'='somevalue'
'someobject'=$someobject
'versions'=$versions}
$NewJSON = [PSCustomObject]#{'Fields'=$fields}
$NewJSON | ConvertTo-Json