Output the results of a Log Analytics Workspace query to Event Hub - azure-powershell

I'm performing a query to output logs captured in an Azure Log Analytics Workspace, for example:
Invoke-AzOperationalInsightsQuery -WorkspaceId '' -Query "AzureDiagnostics | where Category == 'AzureFirewallApplicationRule'"
However I need to send the results of this to an Event Hub for further processing.
I'm trying to use the REST API (https://learn.microsoft.com/en-us/rest/api/eventhub/send-batch-events) but struggling to dynamically generate a request body to send to the Event Hub based on the output fields of the query. This may not be the best way to do it, any suggestions?

I suggest you can use Send event api by sending a simple json data one by one. Because if you use send batch api, you should build a more complex source data.
You can use the following powershell code to send data to event hub using send event api.
$queryResults = Invoke-AzOperationalInsightsQuery -WorkspaceId "xxx" -Query "your query"
#generate sas token
$URI_1 = "event_hub_namespace.servicebus.windows.net/eventhub_path"
$Access_Policy_Name="RootManageSharedAccessKey"
$Access_Policy_Key="the key"
#Token expires now+3000
$Expires=([DateTimeOffset]::Now.ToUnixTimeSeconds())+3000
$SignatureString=[System.Web.HttpUtility]::UrlEncode($URI_1)+ "`n" + [string]$Expires
$HMAC = New-Object System.Security.Cryptography.HMACSHA256
$HMAC.key = [Text.Encoding]::ASCII.GetBytes($Access_Policy_Key)
$Signature = $HMAC.ComputeHash([Text.Encoding]::ASCII.GetBytes($SignatureString))
$Signature = [Convert]::ToBase64String($Signature)
$SASToken = "SharedAccessSignature sr=" + [System.Web.HttpUtility]::UrlEncode($URI_1) + "&sig=" + [System.Web.HttpUtility]::UrlEncode($Signature) + "&se=" + $Expires + "&skn=" + $Access_Policy_Name
$SASToken
$method = "POST"
$url = "https://event_hub_namespace.servicebus.windows.net/eventhub_path/messages"
$signature = $SASToken
# API headers
$headers = #{
"Authorization"=$signature;
"Content-Type"="application/atom+xml;type=entry;charset=utf-8";
}
#use foreach to send data
foreach($s in $queryResults.Results){
#Write-Output "hello"
$json = $s | ConvertTo-Json
#Write-Output $json
Invoke-WebRequest -Method $method -Headers $headers -Body $json -uri $url
}
Write-Output "**completed**"
After execute the powershell, I use code to receive the data from event hub, and I can confirm that all the data are sent to event hub. The screenshot as below:

Related

How to make a rest call in Azure pipeline then use the response to make another call?

I am creating an Azure pipeline where I need to make a rest api call from the pipeline then use part of the api call response in the request of another consecutive rest api call.
How can I achieve that?
In my test, I use the script below to get the work item state value in the pipeline, and you could use this variable in your script or pipeline for other api calls.
1.I transfer the api result into json with variable $field
2.Write-host the value $field
3.Get the element of system.state and set it as variable $witstate
4.Write-host the value.
# Define organization base url, PAT, linked wit, Target WIT state and API version variables
$orgUrl = "https://dev.azure.com/{yourORG}/{yourPROJECT}"
$pat = ""
$queryString = "fields=system.state&api-version=7.0"
$witID = {YourWitID}
# Create header with PAT
$token = [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes(":$($pat)"))
$header = #{authorization = "Basic $token"}
# Get the linked wit state
$projectsUrl = "$orgUrl/_apis/wit/workitems/$witID?$queryString"
$field = Invoke-RestMethod -Uri $projectsUrl -Method Get -ContentType "application/json" -Headers $header | ConvertTo-Json | ConvertFrom-Json | Select-Object -ExpandProperty fields
write-host $field
$witstate = $field.'System.State'
Write-Host $witstate

Azure DevOps Services REST API 6.0: Deployments/Release - List not returning count more than 100

Azure DevOps Services REST API 6.0: Deployments/Release - List not returning count more than 100
I'm using doc (Build, Release & Deployment List)
https://learn.microsoft.com/en-us/rest/api/azure/devops/release/deployments/list?view=azure-devops-rest-6.0&tabs=HTTP
Below API call is giving me list of total builds (approx 930)
GET https://dev.azure.com/{organization}/{project}/_apis/build/builds?$top=999&api-version=6.0
Whereas it wont show more than 100 releases/deployments with below queries
GET https://vsrm.dev.azure.com/{organization}/{project}/_apis/release/releases?$top=999&api-version=6.0
GET https://vsrm.dev.azure.com/{organization}/{project}/_apis/release/deployments?$top=999&api-version=6.0
please suggest further if you have any solutions.
Please try this PowerShell script:
[string]$orgurl = "https://vsrm.dev.azure.com/{organization}"
[string]$project = "ProjectName"
[string]$user = ""
[string]$token = "PAT"
[System.Net.ServicePointManager]::SecurityProtocol = "tls12"
# Ignore the error
$ErrorActionPreference = 'SilentlyContinue'
# Base64-encodes the Personal Access Token (PAT) appropriately
$base64AuthInfo = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(("{0}:{1}" -f $user,$token)))
cls
$headers = #{Authorization = ("Basic {0}" -f $base64AuthInfo)}
$continuationToken = ''
$deployments = #()
Do {
$uri = "$orgurl/$project/_apis/release/deployments?api-version=6.0&continuationToken=$ContinuationToken"
$result = Invoke-WebRequest -Uri $uri -Headers $headers
$deployments += ($result.Content | ConvertFrom-Json).Value
$continuationToken = $result.Headers.'x-ms-continuationtoken'
Write-Host "continuationToken:" $continuationToken
}
While($continuationToken)
$deployments.Count
$deployments.release | Select id, name, url
I got it to work by the first time not passing the ContinuationToken parameter the first call.
$uri = "$orgurl/$project/_apis/release/deployments?api-version=6.0"
Do {
$result = Invoke-WebRequest -Uri $uri -Headers $headers
$uri = "$orgurl/$project/_apis/release/deployments?api-version=6.0&continuationToken=$ContinuationToken"
$deployments += ($result.Content | ConvertFrom-Json).Value
$continuationToken = $result.Headers.'x-ms-continuationtoken'
}

How to do a web request call to sqlpad?

Hi I have been trying to do a web request call to sqlpad.
I have got the basis of the script to make the connection
Invoke-WebRequest -uri
However when I run the command I get connection 200 showing it has made a connection but how do I use cached cookies or how do I sign into sqlpad using credentials and run query all from using web request.
Sorry I am new to powershell and webrequest so I appreciate all your advice thank you.
That should be doable. You'd need to use sqlpad api. To keep cookies you need to create a session variable when calling signin and reuse it in later calls. To extract query data you'd need to use query-result end point. You need to know connectionId (can lookup at api/connections using browser) and SQL code (query text). Turns out there is no direct way to run query by query name. So you either need to know the sql of the query or you can extract it from /api/queries for specific query
$baseUrl = "http://localhost:39325/api"
$user = "yourEmail"
$password = "yourPassword"
$signinUrl = "$baseUrl/signin?email=$user&password=$password&redirect=false"
# sign in and create session variable $ws
if(!$ws) { $r = Invoke-WebRequest -Uri $signinUrl -SessionVariable ws -Method Post } else { Write-Host "connected"}
# list of available queries and connections. May need to run this to determine connection id or existing query sql
$QueryList = Invoke-RestMethod -Uri "$baseUrl/queries" -WebSession $ws
$ConnectionList = Invoke-RestMethod -Uri "$baseUrl/connections" -WebSession $ws
Write-Host "Available queries:"
$QueryList.queries | select name, connectionId, queryText | ft -AutoSize
# Execute Query
$params = #{
connectionId = "vhsNXMXCJeI9QlUL" #use $ConnectionList var or just look up on http://localhost:39325/api/connections/
cacheKey = "null" #just some dummy value, parameter is required but it's not really affecting anything
queryName = "test2" #optional
queryText = "select top 15 * from sys.columns" # required
} | ConvertTo-Json
$head = #{'Content-Type'='application/json'}
$data = Invoke-RestMethod -Uri "$baseUrl/query-result" -Method Post -Body $params -Headers $head -WebSession $ws
$data.queryResult.rows | ft

Coinbase API invalid signature with Powershell

I would like to retrieve account balance through Coinbase API with Powershell.
I coded the following reading from coinbase api documentation but the last request throws the following error:
Invoke-RestMethod : {"errors":[{"id":"authentication_error","message":"invalid signature"}]}
Here is my code.
What's wrong? Thank you.
$accounts = 'https://api.coinbase.com/v2/accounts'
$time = 'https://api.coinbase.com/v2/time'
$epochtime = [string]((Invoke-WebRequest $time | ConvertFrom-Json).data).epoch
$method = 'GET'
$requestpath = '/v2/accounts'
$secret_key = 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
$sign = $epochtime + $method + $requestpath
$hmacsha = New-Object System.Security.Cryptography.HMACSHA256
$hmacsha.key = [Convert]::FromBase64String($secret_key)
$signature = $hmacsha.ComputeHash([Text.Encoding]::ASCII.GetBytes($sign))
$signature = [Convert]::ToBase64String($signature)
$header = #{
"CB-ACCESS-SIGN"=$signature
"CB-ACCESS-TIMESTAMP"=$epochtime
"CB-VERSION" = '2017-08-07'
"CB-ACCESS-KEY"='xxxxxxxxxxxxxx'
}
Invoke-WebRequest $accounts -Headers $header
Hopefully this will get you going. I just started working on a module today and got stuck on the same thing. I came across your question while trying to solve the problem myself. Figured I would share what I found. Good luck!
$accounts = 'https://api.coinbase.com/v2/accounts'
$time = 'https://api.coinbase.com/v2/time'
$epochtime = [string]((Invoke-WebRequest $time | ConvertFrom-Json).data).epoch
$method = 'GET'
$requestpath = '/v2/accounts'
$secret_key = (Get-CoinBaseAPIKeys).Secret
$sign = $epochtime + $method + $requestpath
$hmacsha = New-Object System.Security.Cryptography.HMACSHA256
$hmacsha.key = [Text.Encoding]::UTF8.GetBytes($secret_key)
$computeSha = $hmacsha.ComputeHash([Text.Encoding]::UTF8.GetBytes($sign))
The LONG WAY, for reference:
$signature = ""
foreach ( $c in $computeSha )
{
$signature += "{0:x2}" -f $c
}
The short way. Oddly I got stuck on this SAME issue, because the short way
produces UPPER CASE HEX and the long way ^^above^^ converts to lower case HEX.
The CoinBase API will ONLY accept the signature in HEX in lower case.
$signature = ([System.BitConverter]::ToString($computeSha) -replace "-").ToLower()
Now that we have the signature figured out, the rest should work great. I removed the CB_VERSION because it will default to your OWN API version. My default was different, so I simply removed it.
$header = #{
"CB-ACCESS-SIGN"=$signature
"CB-ACCESS-TIMESTAMP"=$epochtime
"CB-ACCESS-KEY"=(Get-CoinBaseAPIKeys).Key
}
$result = Invoke-WebRequest $accounts -Headers $header -Method Get -ContentType "application/json"
$accounts = $result.Content | ConvertFrom-Json
Write-Output $accounts.data
As an aside on storing PRIVATE KEY/SECRET you can find some ideas here:
https://github.com/cmaahs/BittrexAPI/tree/master/Encryption. Feel free to grab it and modify as you will. Better to store your KEY/SECRET encrypted in the registry rather than as plain text in your script or as environment variables.

Powershell Salesforce SOAP API SessionHeader Type Converion Issue

Hi I am having trouble setting the SessionHeaderValue. I am basing my code on c#.Net. The login works and I receive the serviceUrl and sessionId in the login result but I can't get the session Id set in the session header
Here is the code
$uri = "c:\installs\sforce.wsdl"
$username = "username"
$password = "password"
# Proxy
$service = New-WebServiceProxy -Uri $uri -Namespace sforce -UseDefaultCredential
# Login
$loginResult = $service.login($username, $password)
$service.Url = $loginResult.serverUrl
$service.SessionHeaderValue = New-Object sforce.SessionHeader
This is the error I get which is a bit odd.
Exception setting "SessionHeaderValue": "Cannot convert the "sforce.SessionHeader" value of type "sforce.SessionHeader" to type "sforce.SessionHeader"."
I have been playing with this for a few hours now and have run out of ideas.
Any help is appreciated.
Anthony
The real problem is that you can't reuse $service. Your instance of $service is only good for the login, and that's it. I think it's due to the way New-WebServiceProxy works. Check this little script out:
$uri = 'file://C:\projects\CRM\SalesForce\Integration\enterprise.xml'
$api = new-webserviceproxy -uri $uri -NameSpace SalesForce
$api.GetType().Module.Assembly.ManifestModule.ScopeName
$api = new-object SalesForce.SforceService.ScopeName
$api.GetType().Module.Assembly.ManifestModule
If you were to run this script, (of course you'd need to substitute your own WSDL), you'd see something like this (the DLLs are dynamic, so the names will be different):
mhgl0l5w.dll
vzecopaq.dll
Notice that not only are the object references for $api different, but the dynamic assembly that New-WebServiceProxy creates is different for each one, which isn't what you might expect. That's why your cast is failing; your objects with the same name are different because they come from different dynamic assemblies. I'm not sure why the behavior is this way - if it's a peculiarity of New-WebServiceProxy, or somehow in SalesForce's WSDL.
The solution is actually pretty simple. Recreate your service object off the namespace generated by New-WebServiceProxy like so:
$uri = "c:\installs\sforce.wsdl"
$username = "username"
$password = "password"
# Proxy
$service = New-WebServiceProxy -Uri $uri -Namespace sforce -UseDefaultCredential
# Login
$loginResult = $service.login($username, $password)
$service = New-Object sforce.SforceService
$service.Url = $loginResult.serverUrl
$service.SessionHeaderValue = New-Object sforce.SessionHeader
I found a webpage that implies the the $service.SessionHeaderValue instance should be created using something like:
$service.SessionHeaderValue = New-ObjectFromProxy -proxy $service -proxyAttributeName "SessionHeaderValue" -typeName "SessionHeader"
There was a definition for the New-ObjectFromProxy function in PowerShell + SOAP + AuthenticationInfoValue:
function New-ObjectFromProxy {
param($proxy, $proxyAttributeName, $typeName)
# Locate the assembly for $proxy
$attribute = $proxy | gm | where { $_.Name -eq $proxyAttributeName }
$str = "`$assembly = [" + $attribute.TypeName + "].assembly"
invoke-expression $str
# Instantiate an AuthenticationHeaderValue object.
$type = $assembly.getTypes() | where { $_.Name -eq $typeName }
return $assembly.CreateInstance($type)
}
I don't profess to be a powershell expert, but it appears the instance of the object that New-Object creates isn't really the same type that the proxy object is expecting. Confusingly, they do have the same name.
It's also worth noting that you cannot define a web service proxy twice for the same namespace. This is required for a flow where you need to go to SOAP API to login, then pass the session id to the metadata API to make your requests.
Compare:
$LoginResponse = $sf.login($username, $password)
$newSession = $LoginResponse.sessionId
$newURL = $LoginResponse.serverUrl
$service = New-Object sforce.SforceService
$service.Url = $LoginResponse.serverUrl
$service.SessionHeaderValue = New-Object sforce.SessionHeader
$service.SessionHeaderValue.sessionId = $LoginResponse.sessionId
# Set the batch size to 2000
# Though, if it contains two long text area fields, it will set it as 200
# This is to avoid long SOAP messages and is controlled by SF
$service.QueryOptionsValue = new-Object sforce.QueryOptions
$service.QueryOptionsValue.batchSize = 2000
$service.QueryOptionsValue.batchSizeSpecified = $true
$mdservice= new-WebServiceProxy -URI $mdwsdl -Namespace sforce
$mdservice = New-Object sforce.MetadataService
$mdservice.Url = $LoginResponse.metadataServerUrl
$mdservice.SessionHeaderValue = New-Object sforce.SessionHeader
$mdservice.SessionHeaderValue.sessionId = $LoginResponse.sessionId
This will generate the following error:
"sforce.SessionHeader" to type "sforce.SessionHeader"."
At line:22 char:5
+ $mdservice.SessionHeaderValue = New-Object sforce.SessionHeader
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [], SetValueInvocationException
+ FullyQualifiedErrorId : ExceptionWhenSetting
The property 'sessionId' cannot be found on this object. Verify that the property exists and can be set.
At line:23 char:5
+ $mdservice.SessionHeaderValue.sessionId = $LoginResponse.sessionI ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (:) [], RuntimeException
+ FullyQualifiedErrorId : PropertyNotFound
Whereas the below code will not error:
$sf = new-WebServiceProxy -URI $pwsdl -Namespace sforce
$LoginResponse = $sf.login($username, $password)
$newSession = $LoginResponse.sessionId
$newURL = $LoginResponse.serverUrl
$service = New-Object sforce.SforceService
$service.Url = $LoginResponse.serverUrl
$service.SessionHeaderValue = New-Object sforce.SessionHeader
$service.SessionHeaderValue.sessionId = $LoginResponse.sessionId
# Set the batch size to 2000
# Though, if it contains two long text area fields, it will set it as 200
# This is to avoid long SOAP messages and is controlled by SF
$service.QueryOptionsValue = new-Object sforce.QueryOptions
$service.QueryOptionsValue.batchSize = 2000
$service.QueryOptionsValue.batchSizeSpecified = $true
$mdservice= new-WebServiceProxy -URI $mdwsdl #Don't set namespace here
# Notice that we are getting the namespace dynamically
$type = $mdservice.GetType().NameSpace
$mdservice = New-Object ($type + '.MetadataService')
$mdservice.Url = $LoginResponse.metadataServerUrl
$mdservice.SessionHeaderValue = New-Object ($type + '.SessionHeader')
$mdservice.SessionHeaderValue.sessionId = $LoginResponse.sessionId
Exception setting "SessionHeaderValue": "Cannot convert the "sforce.SessionHeader" value of type "sforce.SessionHeader" to type "sforce.SessionHeader"."
$service.SessionHeaderValue = New-Object sforce.SessionHeader
After New-Object you have to specify the correct class name. Try this:
#Login to Salesforce
$loginResults = $service.login($username,$password)
#set the session Id in partner Object
$service.Url = $loginResults.serverUrl
$sessionHeaderObjClassName = ($service.GetType().FullName) -replace "SforceService","SessionHeader"
$service.SessionHeaderValue = New-Object $sessionHeaderObjClassName
$service.SessionHeaderValue.sessionId = $loginResults.sessionId