Azure DevOps Server pipeline build fails when using self-signed SSL certificate with "unable to get local issuer certificate" during NuGet restore - ssl-certificate

After upgrading to Azure DevOps Server 2019, automated pipeline builds are failing at the NuGet restore step showing:
Error: Error: unable to get local issuer certificate
Packages failed to restore
Microsoft's documentation states that the build agent running on Windows uses the Windows certificate store, so I have checked that the required certificates are installed correctly on the build server, however it is still failing.
There are many questions with similar symptoms but not the same cause. After investigation, I have found the solution to this but I didn't spot anything on this exact issue so I will post an answer that will hopefully save somebody else some time!

It turns out that the Azure DevOps build agent is using a version of Node.js that doesn't use the Windows Certificate Store.
The solution required is to point Node.js at an exported copy (*.cer file) of your self-signed SSL certificate's root CA certificate, using either a system environment variable called NODE_EXTRA_CA_CERTS or by using a Task Variable called NODE.EXTRA.CA.CERTS, with a value pointing to the certificate.
Developer Community Issue Link

I use a PowerShell agent job with the following script. This effectively gives a "Use the Windows Machine Certificate Store" option to Node.JS for the pipeline.
Some notes:
Monitoring node.exe with ProcMon suggests that the file referenced in NODE_EXTRA_CA_CERTS is read every time the pipeline is run. However, others have suggested running Restart-Service vstsagent* -Force is required for the change to be picked up. This isn't my experience but perhaps something different between environments causes this behaviour.
This adds an additional ~1s pipeline execution time. Probably an acceptable price for a "set and forget certificate management for Node in Pipelines on Windows" but worth noting nonetheless.
# If running in a pipeline then use the Agent Home directory,
# otherwise use the machine temp folder which is useful for testing
if ($env:AGENT_HOMEDIRECTORY -ne $null) { $TargetFolder = $env:AGENT_HOMEDIRECTORY }
else { $TargetFolder = [System.Environment]::GetEnvironmentVariable('TEMP','Machine') }
# Loop through each CA in the machine store
Get-ChildItem -Path Cert:\LocalMachine\CA | ForEach-Object {
# Convert cert's bytes to Base64-encoded text and add begin/end markers
$Cert = "-----BEGIN CERTIFICATE-----`n"
$Cert+= $([System.Convert]::ToBase64String($_.export([System.Security.Cryptography.X509Certificates.X509ContentType]::Cert),'InsertLineBreaks'))
$Cert+= "`n-----END CERTIFICATE-----`n"
# Append cert to chain
$Chain+= $Cert
}
# Build target path
$CertFile = "$TargetFolder\TrustedRootCAs.pem"
# Write to file system
$Chain | Out-File $CertFile -Force -Encoding ASCII
# Clean-up
$Chain = $null
# Let Node (running later in the pipeline) know from where to read certs
Write-Host "##vso[task.setvariable variable=NODE.EXTRA.CA.CERTS]$CertFile"

I formatted the PowerShell script from #alifen. The script below can be executed on the build agent itself. It takes a parameter for the target path and sets the environment variable on the server.
Credit to #alifen
[CmdletBinding()]
param (
[Parameter()]
[string]
$TargetFolder = "$env:SystemDrive\Certs"
)
If (-not(Test-Path $TargetFolder))
{
$null = New-Item -ItemType Directory -Path $TargetFolder -Force
}
# Loop through each CA in the machine store
Get-ChildItem -Path Cert:\LocalMachine\CA | ForEach-Object {
# Convert cert's bytes to Base64-encoded text and add begin/end markers
$Cert = "-----BEGIN CERTIFICATE-----`n"
$Cert += $([System.Convert]::ToBase64String($_.export([System.Security.Cryptography.X509Certificates.X509ContentType]::Cert), 'InsertLineBreaks'))
$Cert += "`n-----END CERTIFICATE-----`n"
# Append cert to chain
$Chain += $Cert
}
# Build target path
$CertFile = "$TargetFolder\TrustedRootCAs.pem"
# Write to file system
Write-Host "[$($MyInvocation.MyCommand.Name)]: Exporting certs to: [$CertFile]"
$Chain | Out-File $CertFile -Force -Encoding ASCII
# Set Environment variable
Write-Host "[$($MyInvocation.MyCommand.Name)]: Setting environment variable [NODE_EXTRA_CA_CERTS] to [$CertFile]"
[Environment]::SetEnvironmentVariable("NODE_EXTRA_CA_CERTS", "$CertFile", "Machine")

Related

msbuild fails on Certificate could not be opened, network password not correct

I am trying to create a signed appx package as a test using a purchased code signing certificate. I cannot get it to build without installing the cert locally first (which I don't want to do given this will be done in a CI/CD environment).
I am executing the following on a solution containing an empty WPF project and WAP project.
msbuild $Solution_Path /p:Platform=x64 /p:Configuration=Release
/p:UapAppxPackageBuildMode=SideLoadOnly /p:AppxBundlePlatforms="x64"
/p:AppxPackageDir=$App_Packages_Directory /p:AppxBundle=Never
/p:AppxPackageSigningEnabled=true /p:PackageCertificateThumbprint=$myThumbprint
/p:PackageCertificateKeyFile=$myCert /p:PackageCertificatePassword=$myPassword
error: Certificate could not be opened
error: The specified network password is not correct
I have confirmed the password of $myPassword and thumbprint is $myThumprint by importing the cert and verifying it. I have also tried assigning "" to $myThumprint. I have confirmed the location of $myCert
It will build if I assign AppxPackageSigningEnable=false, but it will be unusable as it is not signed.
In appxmanifest, I have assigned Identity/Publisher to the publisher id of the cert (e.g., Publisher="CN=John Doe, O=Acme, L=TheMoon, S=OuterSpace, C=Universe") and Properties/PublisherDisplayName = the cert's CN (=John Doe)
I have tried exporting the pfx into a cer and using that, but that fails on the cert is not usable as it doesn't include a private key.
I have tried exporting the pfx into a base64 string and then creating a pfx from that - still fails (desperate measures).
Any tips greatly appreciated!
I read that a password protected cert needs to be stored in a cert store for msbuild to use it. Therefore, I ignored the cert on build and added it later by doing the following:
Remove all signing parameters from msbuild as follows
msbuild $Solution_Path /p:Platform=x64 /p:Configuration=Release
/p:UapAppxPackageBuildMode=SideLoadOnly /p:AppxBundlePlatforms="x64"
/p:AppxPackageDir=$App_Packages_Directory /p:AppxBundle=Never
/p:AppxPackageSigningEnabled=false
Given the name of the appx will change based on version and I couldn't find a way to pass wildcards to the SignTool, I used this to grab the built appx:
$Packages_2Sign = (Get-ChildItem -Recurse -Path $currentDirectory -Include *.appx).fullname
Finally, use the SignTool to sign the appx built from the prior step
SignTool sign /fd sha256 /a
/f $certificatePath /p $certificatePwd $Packages_2Sign

Create keyvault secret - Operation returned an invalid status code 'Conflict'

I want to create multiple secrets in keyvault. Assign dynamic values of Blobstorage account, Batch account.
I tried below code to create secrets:
Function CreateKeyvaultSecrets
{
Param
(
[Parameter(Mandatory=$true, Position=0)]
[string] $keyvaultName,
[Parameter(Mandatory=$true, Position=1)]
[string] $blobStorageAccountName,
[Parameter(Mandatory=$true, Position=2)]
[string] $batchaccountName,
[Parameter(Mandatory=$true, Position=3)]
[string] $logRgName
)
#Get Storagekey
$blobStorageKeyObject = (Get-AzStorageAccountKey -ResourceGroupName $logRgName -AccountName $blobStorageAccountName)| Where-Object {$_.KeyName -eq "key1"}
$blobStorageKey = $blobStorageKeyObject.Value
$blobStorageConnectionString = "DefaultEndpointsProtocol=https;AccountName=$blobStorageAccountName;AccountKey=$blobStorageKey;EndpointSuffix=core.windows.net"
#Create blobstorage key secret
$blobSecretkey = ConvertTo-SecureString -String $blobStorageKey -AsPlainText -Force
Set-AzKeyVaultSecret -VaultName $keyvaultName -Name 'blobstorageaccesskey' -SecretValue $blobSecretkey
#Create blobstorage connectionstring key secret
$blobconnectionstringSecret = ConvertTo-SecureString -String $blobStorageConnectionString -AsPlainText -Force
Set-AzKeyVaultSecret -VaultName $keyvaultName -Name 'blobstorageconnectionstring' -SecretValue $blobconnectionstringSecret
Write-host "Blob Storage Account connection string added to Keyvault secret"
}
CreateKeyvaultSecrets 'kvtevalmock' 'steval' 'abtaeval' 'rg-eval'
I am trying to execute above code from Azure DevOps Powershell task. Azure powershell version is 5.
Secrets are not getting creating. Below error is thrown:
WARNING: Upcoming breaking changes in the cmdlet 'Set-AzKeyVaultSecret' :
- The output type 'Microsoft.Azure.Commands.KeyVault.Models.PSKeyVaultSecret' is changing
- The following properties in the output type are being deprecated : 'SecretValueText'
- The change is expected to take effect from the version : '3.0.0'
Note : Go to https://aka.ms/azps-changewarnings for steps to suppress this breaking change warning, and other
information on breaking changes in Azure PowerShell.
##[error]Operation returned an invalid status code 'Conflict'
##[error]PowerShell exited with code '1'.
I test your script on my side, it works fine.
From the error message, looks your Az.KeyVault powershell module version is too old, my version is 3.4.0, try to update it with the command below.
Update-Module -Name Az.KeyVault -Force
After the update, close all the powershell sessions and open a new one to try again, it should work.

Copying ssh key from windows machine to windows server 2019

I've been trying to get access to Windows Server 2019 without password through OpenSSH protocol.
So I've created new key which I need it to be copied to the Windows Server, I've tried this:
ssh-copy-id -i ~/.ssh/id_rsa user#server
But I get this after entering correct password:
'exec' is not recognized as an internal or external command,
operable program or batch file.
The system cannot find the path specified.
The system cannot find the path specified.
My issue is how to transfer key from one windows machine(using gitbash, WSL, powershell or whatever)
to Windows Server 2019 location of authorized keys if I am not mistaken.
I am desperate enough to do it manually but location of those keys is mystery to me, do I need to set something on Windows Server first so that it can accept keys for authentication ?
What is the alternative on ssh-copy-id from Windows machine to Windows Server 2019 ?
Found solution:
Followed this helpful youtube guide, props to the
https://www.youtube.com/watch?v=Cs3wBl_mMH0&ab_channel=IT%2FOpsTalk-Deprecated-SeeChannelDescription
Also, installing OpenSSHUtils worked with:
Install-Module -Name OpenSSHUtils -RequiredVersion 0.0.2.0 -Scope AllUsers
Also this guide helped:
https://www.cloudsma.com/2018/03/installing-powershell-modules-on/
My server didn't have access so I manually copied file from:
C:\Program Files\WindowsPowerShell\Modules to the server's:
Server:\Program Files\WindowsPowerShell\Modules
First, this error message is followed by microsoft/vscode-remote-release issue 25
Current workaround (the context is VSCode, but should apply also for regular SSH connection):
Also, for anyone else here that loves their bash on windows but still wants to be able to use VSCode remote, the workaround I have currently setup is to use an autorun.cmd deployed on the servers that detects when an SSH connection is coming in and has a terminal allocated:
#echo off
if defined SSH_CLIENT (
:: check if we've got a terminal hooked up; if not, don't run bash.exe
C:\cygwin\bin\bash.exe -c "if [ -t 1 ]; then exit 1; fi"
if errorlevel 1 (
C:\cygwin\bin\bash.exe --login
exit
)
)
This is known to work with Cygwin bash, unsure about bash that ships with windows; I imagine it's very sensitive to how the TTY code works internally.
This way, launching cmd.exe works normally, using VSCode (because it does not allocate a PTY) works normally, but SSH'ing into the machine launches bash.exe.
I suspect it would also work using the bash.exe which comes with Git for Windows, should it be installed on the target server.
The destination file should be on the server:
%USERPROFILE%\.ssh\authorized_keys
If you can do it manually, simply try and scp it instead of using ssh-copy-id
scp user#server:C:/Users/<user>/.ssh/authorized_key authorized_key
# manual and local edit to add the public key
scp authorized_key user#server:C:/Users/<user>/.ssh/authorized_key
(again, I would use the scp.exe coming with Git For Windows, installed this time locally)
Found solution:
Followed this helpful youtube guide, props to the
https://www.youtube.com/watch?v=Cs3wBl_mMH0&ab_channel=IT%2FOpsTalk-Deprecated-SeeChannelDescription
Also, installing OpenSSHUtils worked with:
Install-Module -Name OpenSSHUtils -RequiredVersion 0.0.2.0 -Scope AllUsers
Also this guide helped:
https://www.cloudsma.com/2018/03/installing-powershell-modules-on/
My server didn't have access so I manually copied file from:
C:\Program Files\WindowsPowerShell\Modules to the server's:
Server:\Program Files\WindowsPowerShell\Modules

Error MSB3325: Cannot import the following key file

I have a project hosted in Azure DevOps and there the build is failing with the error message:
Error MSB3325: Cannot import the following key file: xxxx.pfx. The key
file may be password protected. To correct this, try to import the
certificate again or manually install the certificate to the Strong
Name CSP with the following key container name: VS_KEY_xxxx
This happens after a project has been changed to sign the assembly with a newly generated password protected pfx signing certificate.
I have tried various fixes given in other SO posts and nothing seems to work.
Can anyone with azure-devops expertise help me with this situation.
You can use the SnInstallPfx.exe and add this in your pipeline as a powershell task
- task: PowerShell#2
env:
SN_INSTALL_PFX: $(snInstallPfx.secureFilePath)
MYCERTIFICATE_PFX: $(myCertificatePfx.secureFilePath)
MYCERTIFICATE_PFX_PASSWORD: $(myCertificatePfxPassword)
inputs:
targetType: 'inline'
script: '&"$($ENV:SN_INSTALL_PFX)" "$($ENV:MYCERTIFICATE_PFX)" "$($ENV:MYCERTIFICATE_PFX_PASSWORD)"'
The pfx, exe and password are stored in the Pipeline library as secure files and variables.
For more information, see the following blog article.
Error MSB3325: Cannot import the following key file
You can create a PowerShell script and add a PowerShell Script step in your build definition to import the new certificate file before the VSBuild step:
The PowerShell script I used to use:
$pfxpath = 'pathtoees.pfx'
$password = 'password'
Add-Type -AssemblyName System.Security
$cert = New-Object System.Security.Cryptography.X509Certificates.X509Certificate2
$cert.Import($pfxpath, $password, [System.Security.Cryptography.X509Certificates.X509KeyStorageFlags]"PersistKeySet")
$store = new-object system.security.cryptography.X509Certificates.X509Store -argumentlist "MY", CurrentUser
$store.Open([System.Security.Cryptography.X509Certificates.OpenFlags]"ReadWrite")
$store.Add($cert)
$store.Close()
And it works fine on my side.
You can check the similar thread for some more details.
Hope this helps.

VSTS build fails with MSB3325, Cannot import PFX key file

I had created a build definition to build a desktop application online on visualstudio.com which fail at task Build Solution (Visual Studio build) with following error,
[error]C:\Program Files (x86)\Microsoft Visual
Studio\2017\Enterprise\MSBuild\15.0\Bin\Microsoft.Common.CurrentVersion.targets(3156,5):
Error MSB3325: Cannot import the following key file:
Sixmod5Certificate.pfx. The key file may be password protected. To
correct this, try to import the certificate again or manually install
the certificate to the Strong Name CSP with the following key
container name: VS_KEY_3B2BCC84AE4E26F1
I followed solution specified at, https://developercommunity.visualstudio.com/content/problem/156086/vsts-build-msb3325-cannot-import-the-following-key.html
then as specified at, https://stackoverflow.com/a/48698229/3531672
I had added a powershell script task before build task, as follows,
[CmdletBinding()]
param(
[Parameter(Mandatory)][string] $pfxpath,
[Parameter(Mandatory)][string] $password
)
Add-Type -AssemblyName System.Security
$cert = New-Object System.Security.Cryptography.X509Certificates.X509Certificate2
$cert.Import($pfxpath, $password, [System.Security.Cryptography.X509Certificates.X509KeyStorageFlags]"PersistKeySet")
$store = new-object system.security.cryptography.X509Certificates.X509Store -argumentlist "MY", CurrentUser
$store.Open([System.Security.Cryptography.X509Certificates.OpenFlags]"ReadWrite")
$store.Add($cert)
$store.Close()
but no luck yet,
There are different SO post similar to this specifying solution to build from Admin user, or installing pfx certificate manually, but as they are related to personal computer and I am trying to configure Continuous integration on visualstudio.com, they don't seem useful to me.
Please note I am able to successfully build on my local machine.
If you wish to regenerate this problem at your end, follow these steps,
STEP 1: Create a new VSTO Addin Project (Any Excel/Word/Powerpoint).
STEP 2: Attach this to VSTS.
STEP 3: In signing tab of Application properties, instead of using temperory certificate, create a new password protected certificate (PFX - Personal Information Exchange in my case) and use this to sign ClickOnce Manifest
STEP 4: Try to build on local machine, it will succeed.
STEP 5: Push it over and try to build on VSTS, you will get the same error as above.
I unchecked the "Sign the assembly" checkbox from the "project properties -> Signing" page and everything worked like a charm. The build was signed successfully through VSTS. Somehow I missed this solution provided in many SO threads related to the problem.