Can I mass-update VSTS build pipeline definitions? - migration

I'm in the process of migrating from an on-premise TFS instance to VSTS. I have a lot of build pipelines (vNext build definitions) that were migrated to VSTS, but now I have to update them all to use a specific Agent.
There is no option available in the UI nor in the commandline client.
Am I missing an option available to me so I can update them all at once?

Based on the migration work I did with Manuel (referred to the post Jesse mentions), I have made some scripts available that get the TFS queues and then use that for updating the VSTS build definitions.
Read-QueuesFromTfs.ps1
Repair-BuildDefinitions.ps1
Both scripts require a parameters PersonalAccesToken - One is a PAT for the VSTS account you are targeting and one for targeting the TFS environment.
First script helps you get a queues.json file that holds all TFS Queues. Second script iterates the VSTS projects you are targeting for updating the build definitions. Scripts should be quite self-explanatory.
# Get all queues and based on previous names get the id's
(Invoke-RestMethod `
-Uri "https://$account.visualstudio.com/$_/_apis/distributedtask/queues" `
-Headers #{Authorization = "Basic $auth"; Accept = "application/json; api-version=3.2-preview" } `
-Method Get `
-ContentType "application/json" -Verbose).value | % { $vstsqueues[$_.name] = $_.id }
# get all the builds
$builds = (Invoke-RestMethod `
-Uri "https://$account.visualstudio.com/$_/_apis/build/definitions" `
-Headers #{Authorization = "Basic $auth"; Accept = "application/json; api-version=4.1-preview.6" } `
-Method Get `
-ContentType "application/json").value
# get the full build definition
$build = Invoke-RestMethod `
-Uri $_.url `
-Headers #{Authorization = "Basic $auth"; Accept = "application/json; api-version=4.1-preview.6" } `
-Method Get `
-ContentType "application/json"
# get queue
$queuename = $tfsqueues[$_.queue.id]
Write-Output " queue name: $queuename"
# update build
$build.queue = #{ id = $vstsqueues[$queuename] }
# post changes
Invoke-RestMethod `
-Uri $_.url `
-Headers #{Authorization = "Basic $auth"; Accept = "application/json; api-version=4.1-preview.6" } `
-Method Put `
-ContentType "application/json" `
-Body ($build | ConvertTo-Json -Depth 100 -Compress) | Out-Null
}
}
Described in this file. https://github.com/JasperGilhuis/VSTS-RestAPI/blob/master/README.md#update-vsts-build-definitions-based-on-tfs-queues
Look that the Builds folder in the repository https://github.com/JasperGilhuis/VSTS-RestAPI/tree/master/Builds

Related

Azure Data Factory API Connector

I am trying to access an application via API REST but I am stuck in the process using ADF.
Basically this process consists in 2 steps.
1 - Getting a session (https://horizon.akixi.com/CCS/API/v1/session)
2 - Authenticating (https://horizon.akixi.com/CCS/API/v1/login?locale=en_GB") using Username and Password.
I have created a Linked Service (Rest) and set the Base URL as (https://horizon.akixi.com/CCS/API/v1/session)
Authentication Anonymous.
Then I create a Data Set to point this API and finally I create a "Copy Data" and tried to set on the Additional Header the authentication's details, but it is not working, I got an error 405.
Basically I can run the below script using Powershell and it is working as expected, but is required to use it on ADF instead.
The script in Powershell is this one.
$user = "xxxxxx#mail.com"
$pass = "123456789"
$secpasswd = ConvertTo-SecureString $pass -AsPlainText -Force
$base64AuthInfo = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(("{0}:{1}" -f $user,$pass)))
#Get Session
$url = "https://horizon.akixi.com/CCS/API/v1/session"
$response = Invoke-RestMethod -uri $url -Method Post -SessionVariable se
#Authentication
Write-Host "Authenticating" -ForegroundColor Green
$headers = #{
'Authorization' = "Basic $base64AuthInfo"
}
$headers
$url = "https://horizon.akixi.com/CCS/API/v1/login?locale=en_GB"
$response = Invoke-RestMethod -uri $url -Headers $headers -Method Get -WebSession $se
#List Reports
$url = "https://horizon.akixi.com/CCS/API/v1/report"
$response = Invoke-RestMethod -uri $url -Method Get -WebSession $se
$response
Thank you all.
As the 3 steps are separate, and only the last one fetches data, you should break up the process into multiple activities.
Use Web Activity for the first two steps,
https://horizon.akixi.com/CCS/API/v1/session
https://horizon.akixi.com/CCS/API/v1/login?locale=en_GB"
extract the session variable or Authentication from the web activities. Pass it to the copy activity. The copy activity should target
https://horizon.akixi.com/CCS/API/v1/report

Azure pipelines: How to add failed selenium Xunit test case attachment in VS test task

I use the Xunit2 selenium framework for automated test cases. Some case fails in pipeline. I want to see the attachment of the fail test case in the Test tab. How can I do this using VS test task?
The option publishRunAttachments: true, it will update attachments to Test run attachment tab, check the pic below
After the VS test task runs, we can get the run ID through the variable VSTEST_TESTRUNID.
Then we could call the RESST API Create Test Result Attachment to add attachments to the test result attachment tab.
Request API:
POST https://dev.azure.com/{organization}/{project}/_apis/test/Runs/{runId}/attachments?api-version=6.0-preview.1
Request Body:
{
"stream": "iVBORw0KGgoAAAANSUhEUgAAABgAAAAYCAIAAABvFaqvAAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAAJcEhZcwAADsMAAA7DAcdvqGQAAABlSURBVDhP7cxBCsAgDERR739pG/CnGJI0FopQ8O2cjNP6R85QbeNQU7wT1dkijaQ3vkZoWElaoTeJojW01cYh0jwfgiFBV/lEjOZtacijN/nLkOBHhIaVDgn+Wdycp6FXzlCl9wt0Y0cAzHo/zgAAAABJRU5ErkJggg==",
"fileName": "imageAsFileAttachment.png",
"comment": "Test attachment upload",
"attachmentType": "GeneralAttachment"
}
You could also check this thread
Update1
We could add task power shell and call the rest api via below script:
$connectionToken="{PAT}"
$base64AuthInfo= [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes(":$($connectionToken)"))
$URL = "https://dev.azure.com/{organization}/{project}/_apis/test/Runs/{runId}/attachments?api-version=6.0-preview.1"
$body =#"
{
"stream": "iVBORw0KGgoAAAANSUhEUgAAABgAAAAYCAIAAABvFaqvAAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAAJcEhZcwAADsMAAA7DAcdvqGQAAABlSURBVDhP7cxBCsAgDERR739pG/CnGJI0FopQ8O2cjNP6R85QbeNQU7wT1dkijaQ3vkZoWElaoTeJojW01cYh0jwfgiFBV/lEjOZtacijN/nLkOBHhIaVDgn+Wdycp6FXzlCl9wt0Y0cAzHo/zgAAAABJRU5ErkJggg==",
"fileName": "imageAsFileAttachment.png",
"comment": "Test attachment upload",
"attachmentType": "GeneralAttachment"
}
"#
$Result = Invoke-RestMethod -Uri $URL -ContentType "application/json" -Body $body -Headers #{Authorization=("Basic {0}" -f $base64AuthInfo)} -Method POST
Update2
Check the pic below and notice the URL
According to the screenshot you shared before, it seems that you want to add attachments to the test run instead of test result. So we just need test run ID and we could get the test run ID from variable after the task vs test ends.
If you want to add attachments to test result, we could list all test result via test run ID.
Sample URL:
GET https://dev.azure.com/{organization}/{project}/_apis/test/Runs/{RunID}/results
The add attachments to the test result via test result ID and test run ID.
Sample power shell script to list all test result ID
$connectionToken="{PAT}"
$base64AuthInfo= [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes(":$($connectionToken)"))
$URL = "https://dev.azure.com/{organization}/{project}/_apis/test/Runs/{RunID}/results"
$Result = Invoke-RestMethod -Uri $URL -Headers #{authorization = "Basic $base64AuthInfo"} -Method Get
foreach($Run in $Result.value){
Write-Host "This test run contain" $Run.id "and the test reuslt name is" $Run.testCase.name
}
Update3
$connectionToken="{PAT}"
$base64AuthInfo= [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes(":$($connectionToken)"))
$URL = "https://dev.azure.com/{organization}/{project}/_apis/test/Runs/134/results"
$Result = Invoke-RestMethod -Uri $URL -Headers #{authorization = "Basic $base64AuthInfo"} -Method Get
#List all test result get the test result ID via result
foreach($Run in $Result.value){
#Get the test result ID via result
If($Run.outcome -eq "Failed"){
$TestResultID = $Run.id
#Write-Host $TestResultID
#Add attachment via test run ID and test result ID
$TestResultAttachmentURL = "https://dev.azure.com/{organization}/{project}/_apis/test/Runs/{runId}/Results/$($TestResultID)/attachments?api-version=6.0-preview.1"
$body =#"
{
"stream": "VXNlciB0ZXh0IGNvbnRlbnQgdG8gdXBsb2FkLg==",
"fileName": "textAsFileAttachment.txt",
"comment": "Test attachment upload",
"attachmentType": "GeneralAttachment"
}
"#
$TestResultAttachmentResult = Invoke-RestMethod -Uri $TestResultAttachmentURL -ContentType "application/json" -Body $body -Headers #{Authorization=("Basic {0}" -f $base64AuthInfo)} -Method POST
}
}
Update4
$AzureDevOpsPAT = {PAT}
$AzureDevOpsAuthenicationHeader = #{Authorization = 'Basic ' + [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(":$($AzureDevOpsPAT)")) }
$UriOrga = "https://dev.azure.com/{organization}/{project}/"
$uriAccount = $UriOrga + "_apis/test/runs?api-version=6.0"
$response = Invoke-RestMethod -Uri $uriAccount -Headers $AzureDevOpsAuthenicationHeader -Method Get
$testRunsIdSorted = $response.value | sort-object id -Descending
Write-Host "##vso[task.setvariable variable=runId]$($testRunsIdSorted[0].id | ConvertTo-Json -Depth 100)"
$result = Invoke-RestMethod -Uri https://dev.azure.com/{organization}/{project}/_apis/test/runs/$($testRunsIdSorted[0].id)/results?api-version=6.0 -Headers $AzureDevOpsAuthenicationHeader -Method Get
#List all test result get the test result ID via result
foreach($Run in $result.value){
#Get the test result ID via result
If($Run.outcome -eq "Failed"){
$TestResultID = $Run.id
#Write-Host $TestResultID
#Add attachment via test run ID and test result ID
$TestResultAttachmentURL = "https://dev.azure.com/{organization}/{project}/_apis/test/Runs/$($runId)/Results/$($TestResultID)/attachments?api-version=6.0-preview.1"
$body =#"
{
"stream": "iVBORw0KGgoAAAANSUhEUgAAABgAAAAYCAIAAABvFaqvAAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAAJcEhZcwAADsMAAA7DAcdvqGQAAABlSURBVDhP7cxBCsAgDERR739pG/CnGJI0FopQ8O2cjNP6R85QbeNQU7wT1dkijaQ3vkZoWElaoTeJojW01cYh0jwfgiFBV/lEjOZtacijN/nLkOBHhIaVDgn+Wdycp6FXzlCl9wt0Y0cAzHo/zgAAAABJRU5ErkJggg==",
"fileName": "imageAsFileAttachment.png",
"comment": "Test attachment upload",
"attachmentType": "GeneralAttachment"
}
"#
$TestResultAttachmentResult = Invoke-RestMethod -Uri $TestResultAttachmentURL -ContentType "application/json" -Body $body -Headers #{Authorization=("Basic {0}" -f $base64AuthInfo)} -Method POST
}
}
If I click on .png file, it shows nothing.

how to pass performance counters to SCOM Retrieve Performance Data Rest API?

I'm trying SCOM Retrieve Performance Data Rest API but I'm unable to pass performance counters as it's unclear in their official documentation.
https://learn.microsoft.com/en-us/rest/api/operationsmanager/data/retrieve%20performance%20data
Example :-
## Get performances for the given server
$myObject = [PSCustomObject]#{
objectname = 'Memory'
countername = 'Available MBytes'
}
$uriPerf = "http://$MS/OperationsManager/data/performance"
$perfFilter = #"
{
"duration": 100,
"id": "103c21b9-1dd1-e6ff-ede8-1091e0111abc",
"peformanceCounters":[$myObject]
}"#
$PerfResponse = Invoke-WebRequest `
-Method POST `
-Uri $uriPerf `
-Body $perfFilter `
-Headers $scomHeaders `
-Credential $cred `
-WebSession $websession `
#-AllowUnencryptedAuthentication
$perf = ConvertFrom-Json -InputObject $PerfResponse.Content
Write-Host "Timestamp after get all get performance data for $($server.id) call $(Get-Date)"
Write-Host "----------------"
This is the error I'm getting
Invoke-WebRequest : {"errorMessage":"Passed parameter cannot be null","errorTrace":""}

Azure Advisor API Possible Limitations and Filter Issues

I manage many subscriptions so the current Azure Advisor while interesting requires you to go through too many screens and there is no way to download the CSV recommendations and compile them using PowerShell. This led to attempt to interface with the API. The issue I am having is that it appears that it limits you to 200 records so changing top does nothing. Many of the records are a generic security warning with a risk of none. I attempted to filter them out but my knowledge of API filters is poor and the documentation Microsoft provides could be better. Below is the PowerShell command I am using:
Call:
$Response = Invoke-RestMethod -Uri "https://management.azure.com/subscriptions/${SubscriptionId}/providers/Microsoft.Advisor/Recommendations?api-version=2017-04-19&`$top=999&`$filter=risk -ne None" -Method GET -Headers #{"Authorization" = "$AccessToken"} -Verbose
Response:
Without Filter Parameter
https://management.azure.com/subscriptions/<SubID>/providers/microsoft.Advisor/recommendations?api-version=2017-04-19&$top=200&$s
kiptoken=<Token>
With Filter Parameter
Invoke-RestMethod -Uri "https://management.azure.com/subscriptions/${SubscriptionId}/providers/Microsoft.Advisor/Recommendations?api-version=2017-04-19&`$top=999&`$filter=risk -eq 'None'" -Method GET -Headers #{"Authorization" = "$AccessToken"} -Verbose
VERBOSE: GET https://management.azure.com/subscriptions/<sub ID>/providers/Microsoft.Advisor/Recommendations?api-version=2017-04-1
9&$top=999&$filter=risk -eq 'None' with 0-byte payload
Invoke-RestMethod : {"message":"Invalid $filter param"}
At line:1 char:13
+ $Response = Invoke-RestMethod -Uri "https://management.azure.com/subs ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-RestMethod], WebException
+ FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeRestMethodCommand
I have been struggling with exactly the same today, hence I found your article.
I managed to resolve my issue by using a later version of the API, and slightly changing the filter
For example:
$filter="Category eq 'Cost'"
$url = "https://management.azure.com/subscriptions/$Subscriptionid/providers/microsoft.Advisor/recommendations?api-version=2017-04-19&`$top=999&`$filter=$filter"
There is a mistake in your API. The right API grammar is &$filter not $filter, you could see your API, you lose a &.
The following script works for me.
##get token
$TENANTID=""
$APPID=""
$PASSWORD=""
$result=Invoke-RestMethod -Uri https://login.microsoftonline.com/$TENANTID/oauth2/token?api-version=1.0 -Method Post -Body #{"grant_type" = "client_credentials"; "resource" = "https://management.core.windows.net/"; "client_id" = "$APPID"; "client_secret" = "$PASSWORD" }
$token=$result.access_token
##set subscriptionId
$subscriptionId=""
$Headers=#{
'authorization'="Bearer $token"
'host'="management.azure.com"
'contentype'='application/json'
}
$url="https://management.azure.com/subscriptions/$subscriptionID/providers/Microsoft.Advisor/Recommendations?api-version=2017-03-31&`$top=999`&$filter=risk -eq 'None'"
Invoke-RestMethod -Uri $url -Headers $Headers -Method GET

TFS2015 REST API Build definition update

I'm trying to update a build definition through REST API with PowerShell.
The script used is:
$url = "http://tfs:8080/tfs/collection/project/_apis/build/definitions/$($buildId)?api-version=2.0"
$obj = Invoke-RestMethod -Uri $url2 -Method Get -ContentType "application/json" -UseDefaultCredentials
$json = ConvertTo-Json $obj
Invoke-RestMethod -Uri $url -Method Put -Body $json -ContentType "application/json" -UseDefaultCredentials
First I've tried with a new empty definition and I've got the below error:
The collection must contain at least one element.Parameter name:
definition.Options.Inputs
So I added an extra code to remove the "options" part from the returned json:
if($obj.options -ne $null){
$obj.options = $null }
and the update worked. But when I'm using the code on an "real" existing build definition that is in production then I get another error:
The collection must contain at least one element.
Parameter name: definition.RetentionRules.Rule.Branches.Filter
I'm using TFS2015 Update 3.
Why is not working a simple update (without any modification) of a build definition via REST API?
The line $json = ConvertTo-Json $obj needs changed to include the -Depth argument with a minimum value of 3. The default is 2 and because of the nesting the values are lost when converted from an object to Json. More specifically what happens is the values get converted from an array to a simple string.
How to tell this is happening in the Json
Without the depth parameter
"retentionRules": [
{
"branches": "+refs/heads/*",
"artifacts": "build.SourceLabel",
"daysToKeep": 10,
"minimumToKeep": 1,
"deleteBuildRecord": true,
"deleteTestResults": true
}
]
With the depth parameter
"retentionRules": [
{
"branches": [
"+refs/heads/*"
],
"artifacts": [
"build.SourceLabel"
],
"daysToKeep": 10,
"minimumToKeep": 1,
"deleteBuildRecord": true,
"deleteTestResults": true
}
]
You will see that the the branches and artifacts values change from a string to an array with the proper depth value.
What your example code should be
$url = "http://tfs:8080/tfs/collection/project/_apis/build/definitions/$($buildId)?api-version=2.0"
$obj = Invoke-RestMethod -Uri $url2 -Method Get -ContentType "application/json" -UseDefaultCredentials
$json = ConvertTo-Json $obj -Depth 3
Invoke-RestMethod -Uri $url -Method Put -Body $json -ContentType "application/json" -UseDefaultCredentials