Azure Powershell New-AzSynapseManagedPrivateEndpoint Definition File format? - azure-powershell

I'm trying to deploy a managed private endpoint to my synapse instance to connect it to a key Vault in the same resource Group. The documentation for the same deployment in Bicep is very incomplete so I thought to try my luck with PowerShell using the following command :
New-AzSynapseManagedPrivateEndpoint `
-WorkspaceName "synapse-test-joao" `
-Name "managedPrivateEndpointKeyVault" `
-DefinitionFile file.json
However, I cannot seem to find the format for the definition file anywhere on the internet, so was wondering if anyone had any information regarding the json format and what parameters need to go inside.
This would be immensely helpful,
Joao

The sample -DefinitionFile should be in below format:
{
# here i am using sample blob storage
"name": "testblob", #ManagePrivate Endpoint Name
"properties": {
"privateLinkResourceId": "/subscriptions/<subscription ID>/resourceGroups/<ResourceGroup name>/providers/Microsoft.Storage/storageAccounts/<Storage Account Name>",
"groupId": "blob"
}
}
To get the -DefinitionFile Sample file to follow the below ways:
Way 1:
Go to your Azure Synapse Workspace -> Managed Private Endpoints
Drag your cursor into your private endpoint you can see {} - (Code) from there you can get the sample -DefinitionFile.
Way 2:
This way you can get the PrivateLinkResourceId of ManagedPrivateEndpoint
$ds = Get-AzSynapseWorkspace -Name <Synapse workspaces Name>
$ws | Get-AzSynapseManagedPrivateEndpoint -Name <Your Created Managed Private Endpoint Name>
# Private Link Resource Id
/subscriptions/<subscription ID>/resourceGroups/<ResourceGroup name>/providers/Microsoft.Storage/storageAccounts/<Storage Account Name>
Refer here

The code to deploy the managed private endpoint for Synapse in a devOps pipeline can be as follows:
param(
$environment,
$synapseWorkspaceVar,
$subscriptionId,
$resourceGroupNameVar,
$keyVaultNameVar
)
$createPrivateEndpointJsonString = #"
{
"properties": {
"privateLinkResourceId": "/subscriptions/$subscriptionId/resourceGroups/$resourceGroupNameVar/providers/Microsoft.KeyVault/vaults/$keyVaultNameVar",
"groupId": "vault"
}
}
"#
write-output $createPrivateEndpointJsonString
$tempFolderPath = ".\temp"
if (!(Test-path -path $tempFolderPath)) {
echo "new file"
New-Item -ItemType directory -path $tempFolderPath
}
write-output "creating the PrivateEndpoint Definition Json file...."
$jsonpath = ".\$tempFolderPath\createprivateendpoint.json"
Set-Content -Path $jsonpath -value $createPrivateEndpointJsonString
echo "Creating new Managed Private Endpoint from Synapse to KeyVault..."
New-AzSynapseManagedPrivateEndpoint `
-WorkspaceName $synapseWorkspaceVar `
-Name "managedPrivateEndpointKeyVaultACEP" `
-DefinitionFile $jsonpath
Get-AzSynapseManagedPrivateEndpoint -WorkspaceName $synapseWorkspaceVar

Related

Azure Powershell Workflow : Failed to convert 'System.Activities.InArgument`1[System.String]' to type 'System.String'

I have a couple of Azure Powershell Workflow runbooks that copy a production database and then run cleanup scripts against it and scale it down to generate dev and stage environemnts. I started getting the error "Failed to convert 'System.Activities.InArgument`1[System.String]' to type 'System.String'" from the parent workbook when I was editing it. I recreated the problem in a set of test runbooks: Test_Child and Test_Parent.
Test_Child is just an empty shell meant to receive a single input string.
workflow Test_Child
{
param(
[parameter(Mandatory=$True)]
[String] $Input
)
}
Test_Parent connects to Azure automation and executes the child workbook. If I run both processes in this order it fails. If I run either one separately or reverse the order of execution it succeeds.
workflow Test_Parent
{
$servicePrincipalConnection=Get-AutomationConnection -Name 'AzureRunAsConnection'
Write-Output "Logging into Azure AD ..."
Connect-AzAccount -Tenant $servicePrincipalConnection.TenantID `
-ApplicationID $servicePrincipalConnection.ApplicationID `
-CertificateThumbprint $servicePrincipalConnection.CertificateThumbprint `
-ServicePrincipal
try
{
Test_Child -Input "input"
}
catch
{
exit
}
}
If I execute the parent runbook I get an error.
Failed
Failed to convert 'System.Activities.InArgument`1[System.String]' to type 'System.String'.
If I remove the runbook call the process completes.
Test_Parent with runbook call removed:
workflow Test_Parent
{
$servicePrincipalConnection=Get-AutomationConnection -Name 'AzureRunAsConnection'
Write-Output "Logging into Azure AD ..."
Connect-AzAccount -Tenant $servicePrincipalConnection.TenantID `
-ApplicationID $servicePrincipalConnection.ApplicationID `
-CertificateThumbprint $servicePrincipalConnection.CertificateThumbprint `
-ServicePrincipal
}
Here is the output:
Logging into Azure AD ...
PSComputerName : localhost
PSSourceJobInstanceId : [My PSSourceJobInstanceId]
Environments : {AzureCloud, AzureUSGovernment, AzureChinaCloud, AzureGermanCloud}
Context : Microsoft.Azure.Commands.Profile.Models.Core.PSAzureContext
If I only call the workbook the process completes as well.
Here is the runbook Test_Parent with the Azure automation connection call removed:
workflow Test_Parent
{
try
{
Test_Child -Input "input"
}
catch
{
exit
}
}
The output
Completed
If I reverse the order of execution it completes.
workflow Test_Parent
{
try
{
Test_Child -Input "input"
}
catch
{
exit
}
$servicePrincipalConnection=Get-AutomationConnection -Name 'AzureRunAsConnection'
Write-Output "Logging into Azure AD ..."
Connect-AzAccount -Tenant $servicePrincipalConnection.TenantID `
-ApplicationID $servicePrincipalConnection.ApplicationID `
-CertificateThumbprint $servicePrincipalConnection.CertificateThumbprint `
-ServicePrincipal
}
Output
Completed
Logging into Azure AD ...
PSComputerName : localhost
PSSourceJobInstanceId : [My PSSourceJobInstanceId]
Environments : {AzureCloud, AzureUSGovernment, AzureChinaCloud, AzureGermanCloud}
Context : Microsoft.Azure.Commands.Profile.Models.Core.PSAzureContext
Any idea why making the automation connection and then executing the workbook creates a data type mismatch error?
I had the exact same problem. In our Test environment the runbooks ran without problem. In our UAT environment, which was set up at a later time, we had the same error as you.
After some investigation I noticed that both environments used a different version of the Az.Accounts module. (2.5.2 in Test, 2.6 in UAT).
After upgrading the Az.Accounts module to 2.7.2 the issue seems to have dissapeared, so i guess it was a bug in the 2.6 version.

Azure automation runbook Completed before running the code

I have a situation where in the Azure automation runbook results in 'Completed' state and does not run the 'actual' code. I have pasted the code below. It creates a Event Hub inside a Namespace. The code works perfectly executing in local machine but it does not execute in Runbook.
I have written a 'write-output "Declaring local variables for use in script"' --> to check if the printing is working. However, the code is not going beyond that. I am sure, I am missing some thing. Kindly help me.
Param(
[Parameter(Mandatory=$true)]
[string] $NameSpaceNameName,
[Parameter(Mandatory=$true)]
[string[]] $EventhubNames,
[Parameter(Mandatory=$true)]
[string] $ProjectId,
[Parameter(Mandatory=$true)]
[int] $PartitionCount,
[Parameter(Mandatory=$true)]
[string]$Requested_for,
[Parameter(Mandatory=$true)]
[string]$Comments
)
## Login to Azure using RunAsAccount
$servicePrincipalConnection = Get-AutomationConnection -Name 'AzureRunAsConnection'
Write-Output ("Logging in to Az Account...")
Login-AzAccount `
-ServicePrincipal `
-TenantId $servicePrincipalConnection.TenantId `
-ApplicationId $servicePrincipalConnection.ApplicationId `
-CertificateThumbprint $servicePrincipalConnection.CertificateThumbprint
write-output "Declaring local variables for use in script"
## Declaring local variables for use in script
$Creation_date = [System.Collections.ArrayList]#()
$ResourceGroups = Get-AzResourceGroup
$provided_name_space_exists = $false
## Change context to Platform subscription
select-azsubscription -subscription "GC302_Sub-platform_Dev"
## Create Event Hub
foreach($Resourcegroup in $ResourceGroups){
Write-Host("Processing the Resource Group: {0} " -f $Resourcegroup.ResourceGroupName)
$EventhubNameSpaces = Get-AzEventHubNamespace -ResourceGroupName $ResourceGroup.ResourceGroupName
# Iterate over each Namespace. Fetch the Resource Group that contains the provided Namespace
foreach($EHNameSpace in $EventhubNameSpaces){
if($EHNameSpace.Name -eq $NameSpaceName){
$provided_name_space_exists = $true
Write-Host ("Found the provided Namespace in resource group: {0}" -f $Resourcegroup.ResourceGroupName)
Write-Output ("Found the provided Namespace in resource group: {0}" -f $Resourcegroup.ResourceGroupName)
$nameSpace_resource_group = $ResourceGroup.ResourceGroupName
# Fetch the existing Event Hubs in the Namespace
$existing_event_hubs_list = Get-AzEventHub -Namespace $EHNameSpace.Name -ResourceGroupName $nameSpace_resource_group
## Check provided EH for uniqueness
if($existing_event_hubs_list.Count -le 1000){
for($i = 0;$i -lt $EventhubNames.Count;$i++){
if($existing_event_hubs_list.name -notcontains $EventhubNames[$i]){
$EventHub = New-AzEventHub -ResourceGroupName $nameSpace_resource_group -Namespace $EHNameSpace.Name -Name $EventhubNames[$i] -PartitionCount $PartitionCount
$date = $EventHub.CreatedAt
$Creation_date+= $date.GetDateTimeFormats()[46]
}else{
Write-Host ("Event hub: '{0}' already exists in the NameSpace: {1}. skipping this Event hub creation" -f $EventhubNames[$i], $EHNameSpace.Name)
}
}
}else{
Write-Host ("The Namespace - {0} has Event Hubs count greater or equal to 1000." -f $EHNameSpace.Name)
Write-Host ("Please refer the Link for Eevent Hubs quota/limit: 'https://learn.microsoft.com/en-us/azure/event-hubs/event-hubs-quotas#event-hubs-dedicated---quotas-and-limits'")
exit
}
}
}
}
# Print a message that Namespace does not exist
if($provided_name_space_exists -eq $false){
Write-Host ("Provided NameSpace: {0} does not exist." -f $NameSpaceName)
exit
}
Screenshot:
You have $NameSpaceNameName in the parameters section of the runbook but later in the runbook at 50th line you have $NameSpaceName which is not the same as mentioned in parameters section. Correct it and then it should work as expected. One suggestion is to always have an else block wherever you have if block to overcome such issues in future.

Azure automation - credentials delivered by Get-PSAutomationCredential don't work with Add-AzureAccount?

I'm modifying a gallery runbook that copies a live database to a test database on a schedule. It's failing at the first hurdle; authenticating and selecting the relevatn azure subscription
The runbook looks like this:
$Cred = Get-AutomationPSCredential -Name 'automationCredential'
Write-Output "UN: $($Cred.Username)"
Add-AzureAccount -Credential $Cred
I've used the portal credentials blade to create a credential named "automationCredential". For the username and password I supplied the username/pw that I log into the azure portal with. Note: this is NOT a school/work microsoft account, but a personal one
I can tell the call to Get-PSAutomationCredential is working out, because the Write-Ouput call shows the correct value
Add-AzureAccount however, delivers the following error:
Add-AzureAccount : unknown_user_type: Unknown User Type At
Set-DailyDatabaseRestore:22 char:22 CategoryInfo :
CloseError: (:) [Add-AzureAccount], AadAuthenticationFailedException
FullyQualifiedErrorId :
Microsoft.WindowsAzure.Commands.Profile.AddAzureAccount
Any pointers how to get a working credential?
According to your description, it seems that your account is a Microsoft account(such as *#outlook.com, *#hotmail.com). Microsoft does not support non-interactive login. It is also unsafe for you to use your account to login your subscription directly. For a runbook, you could use the following codes to logon.
$connectionName = "AzureRunAsConnection"
try
{
# Get the connection "AzureRunAsConnection "
$servicePrincipalConnection=Get-AutomationConnection -Name $connectionName
"Logging in to Azure..."
Add-AzureRmAccount `
-ServicePrincipal `
-TenantId $servicePrincipalConnection.TenantId `
-ApplicationId $servicePrincipalConnection.ApplicationId `
-CertificateThumbprint $servicePrincipalConnection.CertificateThumbprint
}
catch {
if (!$servicePrincipalConnection)
{
$ErrorMessage = "Connection $connectionName not found."
throw $ErrorMessage
} else{
Write-Error -Message $_.Exception
throw $_.Exception
}
}
In above code, you need use connection AzureRunAsConnection, it is created by Azure default, you could use it directly, you could check this connection, it includes your subscription information.
Also, you could create a new connection, please refer to this link.
Have you tried using the resource manager version off the login cmdlet (Add-AzureRmAccount)?

How to run a SQL batch within an Azure runbook?

I need to execute a batch to perform some maintenance tasks in my database but all the examples on Azure Automation I see are dealing with a single SQL command.
How do I do it if creating an SP is not an option? I think I need to either somehow embed my script.sql file into a runbook script or reference it (like here, for example)?
You could store the .sql file in Azure Blob Storage, and within the runbook download the .sql file, read its contents, and pass that to the SqlCommand object.
Something like:
try {
# Connect to Azure using service principal auth
$ServicePrincipalConnection = Get-AutomationConnection -Name $AzureConnectionAssetName
Write-Output "Logging in to Azure..."
$Null = Add-AzureRmAccount `
-ServicePrincipal `
-TenantId $ServicePrincipalConnection.TenantId `
-ApplicationId $ServicePrincipalConnection.ApplicationId `
-CertificateThumbprint $ServicePrincipalConnection.CertificateThumbprint
}
catch {
if(!$ServicePrincipalConnection) {
throw "Connection $AzureConnectionAssetName not found."
}
else {
throw $_.Exception
}
}
$Path = "C:\abc.sql"
Set-AzureRmCurrentStorageAccount -ResourceGroupName $ResourceGroupName -Name $StorageAccountName
Get-AzureStorageBlobContent -Container $Container -Blob $Blob -Destination $Path
$Content = Get-Content $Path
$Cmd = New-Object System.Data.SqlClient.SqlCommand($Content, $Conn)

Azure database backup to blob using Powershell

We need to backup the azure database and store it on blob so that it can be restored. I've seen this blog but it uses the third party cmdlets.
http://weblogs.thinktecture.com/cweyer/2011/01/automating-backup-of-a-sql-azure-database-to-azure-blob-storage-with-the-help-of-powershell-and-task-scheduler.html
Could someone please guide/help how above can be achieved using powershell.
Backing up to WA Blob Store is not supported from Azure DB, rather the service does automatic backups for you with PITR capability. You'll find the following documentation useful:
http://msdn.microsoft.com/en-us/library/azure/hh852669.aspx
http://msdn.microsoft.com/en-us/library/azure/jj650016.aspx
Hope this helps.
Here is my powershell script
https://gist.github.com/voxon2/be29a3fd6dabbb9155ca
Here is an article describing many different approaches other than powershell
http://blogs.msdn.com/b/mast/archive/2013/03/04/different-ways-to-backup-your-windows-azure-sql-database.aspx
First get your Azure Automation Settings done (see here).
Edit the blow script and save it as .ps1 file. When you run it for
the first time, it will ask you both your azure automation account and
your database credentials. During this process, it will save your
credentials in a local file securely (see here how it is done). After this time on wards, it uses the saved credentials.
The .psl file and the encrypted credential files should be stored in one
directory
Once you are happy you can schedule it to run in task scheduler.
function Get-MyCredential
{
param(
$CredPath,
[switch]$Help
)
$HelpText = #"
Get-MyCredential
Usage:
Get-MyCredential -CredPath `$CredPath
If a credential is stored in $CredPath, it will be used.
If no credential is found, Export-Credential will start and offer to
Store a credential at the location specified.
"#
if($Help -or (!($CredPath))){write-host $Helptext; Break}
if (!(Test-Path -Path $CredPath -PathType Leaf)) {
Export-Credential (Get-Credential) $CredPath
}
$cred = Import-Clixml $CredPath
$cred.Password = $cred.Password | ConvertTo-SecureString
$Credential = New-Object System.Management.Automation.PsCredential($cred.UserName, $cred.Password)
Return $Credential
}
function Export-Credential($cred, $path) {
$cred = $cred | Select-Object *
$cred.password = $cred.Password | ConvertFrom-SecureString
$cred | Export-Clixml $path
}
#Create a directory with you azure server name to isolate configurations
$FileRootPath = "C:\PowerShellScripts\AzureServerName"
Write-Host "Getting Azure credentials"
$AzureCred = Get-MyCredential ($FileRootPath + "AzureSyncred.txt")
#Use Azure Automation Account
#(If You do not have it will not work with other accounts)
Add-AzureAccount -Credential $AzureCred
Select-AzureSubscription -SubscriptionId "myAzureSubscriptionId"
#DO NOT use tcp:myServerName.database.windows.net,1433 but only myServerName
$ServerName = "myServerName"
$Date = Get-Date -format "yyyy-MM-dd-HH-mm"
$DatabaseName = "myTargetDatabaseName"
$BlobName = $Date + "-" + $DatabaseName.bacpac"
$StorageName = "myStorageAccountName"
$ContainerName = "myContainerNameToStoreBacpacFiles"
$StorageKey = "myStorageAccountKey"
Write-Host "Getting database user credential"
#DO NOT use myDatabaseUsername#myServerName but only myDatabaseUsername
$credential = Get-MyCredential ($FileRootPath + "DbSyncred.xml")
Write-Host "Connecting to Azure database"
$SqlCtx = New-AzureSqlDatabaseServerContext -ServerName $ServerName -Credential $credential
Write-Host "Connecting to Blob storage"
$StorageCtx = New-AzureStorageContext -StorageAccountName $StorageName -StorageAccountKey $StorageKey
$Container = Get-AzureStorageContainer -Name $ContainerName -Context $StorageCtx
Write-Host "Exporting data to blob"
$exportRequest = Start-AzureSqlDatabaseExport -SqlConnectionContext $SqlCtx -StorageContainer $Container -DatabaseName $DatabaseName -BlobName $BlobName
Get-AzureSqlDatabaseImportExportStatus -Request $exportRequest
# use the below script in powershell to execute the script
# powershell -ExecutionPolicy ByPass –File C:\PowerShellScripts\AzureServerName\mySavedScript.ps1 –noexit