I searched quite a bit, found several threads here (e.g. this, that, and more), but I could not find the answer to the following task:
Use the Azure DevOps API to retrieve the content changes (basically the file before and after) of all the files of a specific PR.
I can find a PR, can loop through changes, iterations, commits (in various combinations), but I have not been able to download both the first and the last version of each or the files (and there should be a way as I can view the before and after in a PR in DevOps).
Any hints where/how I can retrieve both versions of a file of a certain commit/change/iteration?
Many thanks in advance!
Cheers,
Udo
Thanks for all the hints. Looks like I managed to find a wat to pull it. Please feel free to correct my approach.
Here's the complete PowerShell file:
[CmdletBinding()]
param (
[int]
$pullRequestId,
[string]
$repoName
)
# --- your own values ---
$pat = 'your-personal access token for Azure DevOps'
$urlOrganization = 'your Azure DevOps organization'
$urlProject = 'your Azure DevOps project'
$basePath = "$($env:TEMP)/pullRequest/" # a location to store all the data
# --- your own values ---
if (!$repoName)
{
$userInput = Read-Host "Please enter the repository name"
if (!$userInput)
{
Write-Error "No repository name given."
exit
}
$repoName = $userInput
}
if (!$pullRequestId)
{
$userInput = Read-Host "Please enter the PullRequest ID for $($repoName)"
if (!$userInput)
{
Write-Error "No PullRequest ID given."
exit
}
$pullRequestId = $userInput
}
$prPath = "$($basePath)$($pullRequestId)"
$sourcePath = "$($basePath)$($pullRequestId)/before"
$targetPath = "$($basePath)$($pullRequestId)/after"
# --- helper methods ---
function CleanLocation([string]$toBeCreated)
{
RemoveLocation $toBeCreated
CreateLocation $toBeCreated
if (!(Test-Path $toBeCreated))
{
Write-Error "Path '$toBeCreated' could not be created"
}
}
function CreateLocation([string]$toBeCreated)
{
if (!(Test-Path $toBeCreated)) { New-Item -ErrorAction Ignore -ItemType directory -Path $toBeCreated > $null }
}
function RemoveLocation([string]$toBeDeleted)
{
if (Test-Path $toBeDeleted) { Remove-Item -Path $toBeDeleted -Recurse -Force }
}
function DeleteFile([string]$toBeDeleted)
{
if (Test-Path $toBeDeleted) { Remove-Item -Path $toBeDeleted -Force }
}
# --- helper methods ---
# --- Azure DevOps helper methods ---
function GetFromDevOps($url)
{
$patToken = [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes(":$($pat)"))
$repoHeader = #{ "Authorization" = "Basic $patToken" }
$response = $(Invoke-WebRequest $url -Headers $repoHeader)
if ($response.StatusCode -ne 200)
{
Write-Error "FAILED: $($response.StatusDescription)"
}
return $response
}
function JsonFromDevOps($url)
{
$response = GetFromDevOps $url
return ConvertFrom-Json -InputObject $response.Content
}
function DownloadFromDevOps($url, $filename)
{
$patToken = [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes(":$($pat)"))
$repoHeader = #{ "Authorization" = "Basic $patToken" }
$ProgressPreference = 'SilentlyContinue'
Invoke-WebRequest $url -Headers $repoHeader -OutFile $filename
$ProgressPreference = 'Continue'
}
function DownloadAndSaveFile($itemUrl, $outputPath, $path, $overwrite)
{
$outFile = "$outputPath$($path)"
$fileExists = Test-Path $outFile
if (!$fileExists -or $overwrite)
{
$outPath = [System.IO.Path]::GetDirectoryName($outFile)
CreateLocation $outPath
if ($fileExists)
{
Write-Host " overwriting to $outputPath"
}
else
{
Write-Host " downloading to $outputPath"
}
DownloadFromDevOps $itemUrl $outFile
}
}
function DownloadFiles($changes, $beforePath, $beforeCommitId, $afterPath, $afterCommitId)
{
foreach ($change in $changes)
{
$item = $change.item
if ($item.isFolder)
{
continue;
}
$path = $item.path
$originalPath = $change.originalPath
if (!$path)
{
$path = $change.originalPath
}
$displayPath = $path ?? $originalPath
$changeType = $change.changeType
Write-Host "[$($changeType)] $($displayPath)"
if (($changeType -eq "edit, rename"))
{
$itemUrl = "$($urlRepository)/items?path=$($originalPath)&versionDescriptor.version=$($beforeCommitId)&versionDescriptor.versionOptions=0&versionDescriptor.versionType=2&download=true"
DownloadAndSaveFile $itemUrl $beforePath $originalPath
}
# just get the source/before version
if ($changeType -eq "delete")
{
$itemUrl = "$($urlRepository)/items?path=$($originalPath)&versionDescriptor.version=$($beforeCommitId)&versionDescriptor.versionOptions=0&versionDescriptor.versionType=2&download=true"
DownloadAndSaveFile $itemUrl $beforePath $originalPath
DeleteFile "$($afterPath)$($originalPath)"
}
if ($changeType -eq "edit")
{
$itemUrl = "$($urlRepository)/items?path=$($path)&versionDescriptor.version=$($beforeCommitId)&versionDescriptor.versionOptions=0&versionDescriptor.versionType=2&download=true"
DownloadAndSaveFile $itemUrl $beforePath $path
}
if (($changeType -eq "add") -or ($changeType -eq "edit") -or ($changeType -eq "edit, rename"))
{
$itemUrl = "$($urlRepository)/items?path=$($path)&versionDescriptor.version=$($afterCommitId)&versionDescriptor.versionOptions=0&versionDescriptor.versionType=2&download=true"
DownloadAndSaveFile $itemUrl $afterPath $path $true
}
$validChangeTypes = #('add', 'delete', 'edit', 'edit, rename')
if (!($validChangeTypes.Contains($changeType)))
{
Write-Warning "Unknown change type $($changeType)"
}
}
}
# --- Azure DevOps helper methods ---
$urlBase = "https://dev.azure.com/$($urlOrganization)/$($urlProject)/_apis"
$urlRepository = "$($urlBase)/git/repositories/$($repoName)"
$urlPullRequests = "$($urlRepository)/pullRequests"
$urlPullRequest = "$($urlPullRequests)/$($pullRequestId)"
$urlIterations = "$($urlPullRequest)/iterations"
CleanLocation $prPath
$iterations = JsonFromDevOps $urlIterations
foreach ($iteration in $iterations.value)
{
# the modified file
$srcId = $iteration.sourceRefCommit.commitId
# the original file
$comId = $iteration.commonRefCommit.commitId
$comId = $iteration.targetRefCommit.commitId
$urlIterationChanges = "$($urlIterations)/$($iteration.id)/changes"
$iterationChanges = JsonFromDevOps $urlIterationChanges
DownloadFiles $iterationChanges.changeEntries $sourcePath $comId $targetPath $srcId
}
I am trying to to use Azure Powershell Runbooks to delete files out of the Azure Fileshare. There are no errors returned, but the file is not deleted. The Automation Account has a Run As account setup that is not expired or anything and the script works if I run it from my local machine. Looking for some advise on this.
$ctx = New-AzureStorageContext -StorageAccountName "" -StorageAccountKey ""
$shareName = ""
$directoryPath = ".cloudconsole"
$DirIndex = 0
$day = 1
$startdate = (Get-Date).AddDays(-180)
$endDate = (Get-date).AddDays(-32)
$dirsToList = New-Object System.Collections.Generic.List[System.Object]
$shareroot = Get-AzureStorageFile -ShareName $shareName -Path $directoryPath -context $ctx
$dirsToList += $shareroot
While ($dirsToList.Count -gt $DirIndex)
{
$dir = $dirsToList[$DirIndex]
$DirIndex ++
$fileListItems = $dir | Get-AzureStorageFile
$dirsListOut = $fileListItems | where {$_.GetType().Name -eq "AzureStorageFileDirectory"}
$dirsToList += $dirsListOut
$files = $fileListItems | where {$_.GetType().Name -eq "AzureStorageFile"}
foreach($file in $files)
{
$task = $file.CloudFile.FetchAttributesAsync()
$task.Wait()
if ($file.CloudFile.Properties.LastModified -ge $startdate -and $file.CloudFile.Properties.LastModified -ge $endDate )
{
if ($file.CloudFile.Properties.LastModified.day -ne '01' )
{
$file | Remove-AzureStorageFile
}
}
if ($file.CloudFile.Properties.LastModified -lt $startdate)
{
$file | Remove-AzureStorageFile
}
}
}
BoxJumper: As kavyasaraboju-MT referred, below piece of script should help you.
$ctx = New-AzStorageContext -StorageAccountName $accountName -StorageAccountKey $key
$shareName = <shareName>
$startdate = (Get-Date).AddDays(-180)
$endDate = (Get-date).AddDays(-32)
$DirIndex = 0
$dirsToList = New-Object System.Collections.Generic.List[System.Object]
# Get share root Dir
$shareroot = Get-AzStorageFile -ShareName $shareName -Path . -context $ctx
$dirsToList += $shareroot
# List files recursively and remove file older than 14 days
While ($dirsToList.Count -gt $DirIndex)
{
$dir = $dirsToList[$DirIndex]
$DirIndex ++
$fileListItems = $dir | Get-AzStorageFile
$dirsListOut = $fileListItems | where {$_.GetType().Name -eq "AzureStorageFileDirectory"}
$dirsToList += $dirsListOut
$files = $fileListItems | where {$_.GetType().Name -eq "AzureStorageFile"}
foreach($file in $files)
{
# Fetch Attributes of each file and output
$task = $file.CloudFile.FetchAttributesAsync()
$task.Wait()
# remove file if it's modified between after last 180 days and before last 30 Days
if ($file.CloudFile.Properties.LastModified -lt (Get-Date).AddDays(-32) -and $file.CloudFile.Properties.LastModified -ge (Get-Date).AddDays(-180))
{
## print the file LMT
# $file | Select #{ Name = "Uri"; Expression = { $_.CloudFile.SnapshotQualifiedUri} }, #{ Name = "LastModified"; Expression = { $_.CloudFile.Properties.LastModified } }
# remove file
$file | Remove-AzStorageFile
}
}
#Debug log
# Write-Host $DirIndex $dirsToList.Length $dir.CloudFileDirectory.SnapshotQualifiedUri.ToString()
}
I'm pretty new working with Powershell and i have some working code but I'm not sure how to get it into an efficient routine to return all of the extended file properties of some video files i have.
I have:
# The basic setup for the next steps
$path = 'C:\test\videotocheck.mp4'
$shell = New-Object -COMObject Shell.Application
$folder = Split-Path $path
$file = Split-Path $path -Leaf
$shellfolder = $shell.Namespace($folder)
$shellfile = $shellfolder.ParseName($file)
# This command gets a list of all the extended attributes available for this file
0..500 | Foreach-Object { '{0} = {1}' -f $_, $shellfolder.GetDetailsOf($null, $_) }
# These commands get the individual attributes picked out of the list above
$shellfolder.GetDetailsOf($shellfile, 314)
$shellfolder.GetDetailsOf($shellfile, 316)
All i want to do is provide a filename and have it give me a list back of all of the attributes and their values (if they have one.)
Example:
I intend to use this in a SQL stored procedure. I can work with different types of output if that's easier.
I'm mostly interested in dimensions
Any guidance would be appreciated.
To get all of this extended metadate you could use the function below.
You can give it the path to a single file, or the path to a folder where the files are.
function Get-MetaData {
[CmdletBinding()]
[OutputType([Psobject[]])]
Param (
# Path can be the path to a folder or the full path and filename of a single file
[Parameter(Mandatory = $true, ValueFromPipeline = $true, Position = 0)]
[string]$Path,
# Pattern is unused if Path is pointing to a single file
[Alias('Filter')]
[string]$Pattern = '*.*',
[Alias('Indices')]
[int[]]$Properties = 1..500,
# Recurse is unused if Path is pointing to a single file
[switch]$Recurse,
[switch]$IncludeEmptyProperties
)
$item = Get-Item -Path $Path -ErrorAction SilentlyContinue
if (!$item) { Write-Error "$Path could not be found."; return }
if (!$item.PSIsContainer) {
# it's a file
$files = #($item)
$Path = $item.DirectoryName
}
else {
# it's a folder
$files = Get-ChildItem -Path $Path -Filter $Pattern -File -Recurse:$Recurse
}
$shell = New-Object -ComObject "Shell.Application"
$objDir = $shell.NameSpace($Path)
foreach($file in $files) {
$objFile = $objDir.ParseName($file.Name)
$mediaFile = $objDir.Items()
foreach($index in $Properties) {
$name = $objDir.GetDetailsOf($mediaFile, $index)
if (![string]::IsNullOrWhiteSpace($name)) {
$value = $objDir.GetDetailsOf($objFile, $index)
if (![string]::IsNullOrWhiteSpace($value) -or $IncludeEmptyProperties) {
[PsCustomObject]#{
Path = $file.FullName
Index = $index
Property = $name
Value = $value
}
}
}
}
}
# clean-up Com objects
$null = [System.Runtime.Interopservices.Marshal]::ReleaseComObject($objFile)
$null = [System.Runtime.Interopservices.Marshal]::ReleaseComObject($objDir)
$null = [System.Runtime.Interopservices.Marshal]::ReleaseComObject($shell)
[System.GC]::Collect()
[System.GC]::WaitForPendingFinalizers()
}
You can of course play around with the different parameters the function can take like -Pattern '*.mp4' to only list properties for mp4 files or add switch -IncludeEmptyProperties to also list properties that exist for that file type, but have no value for the specified file.
With parameter Properties you can give the function an array of int32 values of the property indices to return. If you leave that open, the function tries to get all properties from index 1 to index 500 (if available).
Most of the interesting property indices can be found at:
audio/video files: 0,1,2,3,4,5,9,11,12,13,14,15,16,17,18,19,20,21,22,26,27,28,36,164,165,194,213,220,223,237,243
font files:
0,1,2,3,4,5,20,21,25,33,34,164,165,166,196,310
image files:
0,1,2,3,4,5,9,11,31,164,165,174,175,176,177,178,194,196
Use it like this:
$result = Get-MetaData -Path '<pathToTheFile_OR_pathToTheFolder>'
# output to GridView
$result | Out-GridView
# output to CSV file
$result | Export-Csv -Path '<pathToTheOutput.csv>' -NoTypeInformation
Here's a simplified version similar to Theo's. It will accept file/folder paths or objects via pipeline or as parameter.
Function Get-FileMetaData {
[cmdletbinding()]
Param
(
[parameter(valuefrompipeline,ValueFromPipelineByPropertyName,Position=1,Mandatory)]
$InputObject
)
begin
{
$shell = New-Object -ComObject Shell.Application
}
process
{
foreach($object in $InputObject)
{
if($object -is [string])
{
try
{
$object = Get-Item $object -ErrorAction Stop
}
catch
{
Write-Warning "Error while processing $object : $($_.exception.message)"
break
}
}
try
{
Test-Path $object -ErrorAction Stop
}
catch
{
Write-Warning "Error while processing $($object.fullname) : $($_.exception.message)"
break
}
switch ($object)
{
{$_ -is [System.IO.DirectoryInfo]}{
write-host Processing folder $object.FullName -ForegroundColor Cyan
$currentfolder = $shell.namespace($object.FullName)
$items = $currentfolder.items()
}
{$_ -is [System.IO.FileInfo]}{
write-host Processing file $object.FullName -ForegroundColor Cyan
$parent = Split-Path $object
$currentfolder = $shell.namespace($parent)
$items = $currentfolder.ParseName((Split-Path $object -Leaf))
}
}
try
{
foreach($item in $items)
{
0..512 | ForEach-Object -Begin {$ht = [ordered]#{}}{
if($value = $currentfolder.GetDetailsOf($item,$_))
{
if($propname = $currentfolder.GetDetailsOf($null,$_))
{
$ht.Add($propname,$value)
}
}
} -End {[PSCustomObject]$ht}
}
}
catch
{
Write-Warning "Error while processing $($item.fullname) : $($_.exception.message)"
}
}
}
end
{
$shell = $null
}
}
Code to create a new user:
function New-DropBoxUser {
[CmdletBinding()]
Param(
[Parameter(Mandatory=$true, ValueFromPipeline=$true, Position=0)]
[string]$FullName
)
# Split name into username #
$FirstName, $LastName = $FullName.split(' ')
$UserName = ($FirstName[0]+$LastName).toLower()
$email = "$UserName#mycomp.com"
$Body = #{
"new_members" = #(#{
"member_email" = $email;
"member_given_name" = $FirstName;
"member_surname" = $LastName;
"send_welcome_email" = "true";
"role" = #{
".tag" = "member_only"
}
})
}
Write-Host ''
Write-Host 'Creating DropBox User...' -ForegroundColor 'Yellow' -BackgroundColor 'Black'
$AuthToken = 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX'
$token = "Bearer $AuthToken"
$response = Invoke-RestMethod `
-Method Post `
-Uri "https://api.dropboxapi.com/2/team/members/add" `
-Headers #{ Authorization = $token } `
-ContentType "application/json; charset=utf-8" `
-Body (ConvertTo-Json $Body)
}
Output:
Invoke-RestMethod : The remote server returned an error: (400) Bad Request.
Anyone, please tell me the fix. I tried the Endpoint "team/members/list" to fetch user list using the same $token and it works.
I am totally new to Dropbox API so my code is borrowed from Google, while being good with PowerShell I tried to patch up using many different codes but none worked.
When all else fails, RTFM, whose parameters say:
{
"new_members": [
{
"member_email": "tom.s#company.com",
"member_given_name": "Tom",
"member_surname": "Silverstone",
"member_external_id": "company_id:342432",
"send_welcome_email": true,
"role": "member_only"
}
],
"force_async": false
}
Try removing .tag from the role definition and replace "true" with $true
$Body = #{
"new_members" = #(#{
"member_email" = $email
"member_given_name" = $FirstName
"member_surname" = $LastName
"send_welcome_email" = $true
"role" = "member_only"
})
}
I am trying to invoke the API using this request. This request is working fine in Postman. But how can I convert it to PowerShell Invoke-RestMethod?
This is what I tried.
$token = "abcdef123456"
$url = "https://example.com/OVAV1/triggercalls/1"
$header = #{"Authorization"='abdcdef123456'}
$body =
{
"requestid": "<Echo back field>",
"templatename": "Line_Down",
"templateid": "abcde12345",
"noofrecords": 1,
"variableinfo": [
{
"<<MOBILE_NUMBER>>": "123456789",
"<<UNIQUE_CALL_IDENTIFIER>>": "A001",
"<<LinkType>>": "PRI",
"<<HostName>>": "HOSTNAME"
}
]
}
$request = Invoke-RestMethod -Uri $url -Headers $header -Body $body -ContentType "application/json" -Method Post
$request
I tried the JSON code in Postman. It's working fine. How can I use the same JSON code in PowerShell?
You can't put JSON data into PowerShell code like that. Put it in a here-string and it should work:
$body = #"
{
"requestid": "<Echo back field>",
"templatename": "Line_Down",
"templateid": "abcde12345",
"noofrecords": 1,
"variableinfo": [
{
"<<MOBILE_NUMBER>>": "123456789",
"<<UNIQUE_CALL_IDENTIFIER>>": "A001",
"<<LinkType>>": "PRI",
"<<HostName>>": "HOSTNAME"
}
]
}
"#
You could also define the data as a hashtable and convert it to a JSON string:
$data = #{
"requestid" = "<Echo back field>"
"templatename" = "Line_Down"
"templateid" = "abcde12345"
"noofrecords" = 1
"variableinfo" = #(
#{
"<<MOBILE_NUMBER>>" = "123456789"
"<<UNIQUE_CALL_IDENTIFIER>>" = "A001"
"<<LinkType>>" = "PRI"
"<<HostName>>" = "HOSTNAME"
}
)
}
$body = $data | ConvertTo-Json