Databricks: cannot get personal access token. Error 400 Invalid resource ID - api

$ADB_WORKSPACE_ID = "5555555555555"
$ADB_WORKSPACE_URL = "adb-5555555555555.5.azuredatabricks.net"
$adbGlobalToken = (az account get-access-token --resource 2ff814a6-3304-4ab8-85cb-cd0e6f879c1d | ConvertFrom-Json).accessToken
$azureApiToken= (az account get-access-token --resource "https://management.core.windows.net/" | ConvertFrom-Json).accessToken
$headers = #{
"Authorization" = "Bearer $adbGlobalToken";
"X-Databricks-Azure-SP-Management-Token" = $azureApiToken;
"X-Databricks-Azure-Workspace-Resource-Id" = $ADB_WORKSPACE_ID;
}
$body = #{
"comment" = "This is an example token";
"lifetime_seconds" = 300;
}
$uri = "https://${ADB_WORKSPACE_URL}/api/2.0/token/create"
write-host $uri
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
$json = Invoke-RestMethod -Method Post -Uri $uri -Headers $headers -ContentType "application/json" -Body $body
write-host $json
Above code gives me the following error:
Invoke-RestMethod :
Error 400 Invalid resource ID.
HTTP ERROR 400
Problem accessing /api/2.0/token/create. Reason:
Invalid resource ID.
(FictiveDatabricks URL & ID )

Like the error stated, I provided the wrong resource id.
The correct way to get it is the following:
$ADB_WORKSPACE_ID = "/subscriptions/$SubscriptionId/resourceGroups/$ResourceGroupName/providers/Microsoft.Databricks/workspaces/$WorkspaceName"
ResourceId is not the same as the WorkSpaceId.
ResourceId can also be found in the Databricks resource json in the "id" field.

Related

Create Azure Purview collection using API and Powershell

This is the first time I am working with API.
I am trying to create a purview subcollection using API and powershell.
$tenantID = "XXXXXXXXXXXXXXXXXXXXXXX"
$url = "https://login.microsoftonline.com/$tenantID/oauth2/token"
$params = #{ client_id = "XXXXXXXXXXXXXXXXXXXXXXX"; client_secret = "XXXXXXXXXXXXXXXXXXXXXXXXX"; grant_type = "client_credentials"; resource = ‘https://purview.azure.net’ }
$bearertoken = Invoke-WebRequest $url -Method Post -Body $params -UseBasicParsing | ConvertFrom-Json
$accesstoken = ConvertTo-SecureString $bearertoken.access_token -AsPlainText -Force
$purviewendpoint = "https://testpurview.purview.azure.com/account"
$url = "$purviewendpoint/collections/newcollection1?api-version=2019-11-01-preview"
$childcollection = #"
{
"parentCollection": {
"referenceName": "**testpurview**"
}
}
"#
Invoke-RestMethod -Method PUT -Uri $url -Body $childcollection -Token $accesstoken
Steps I tried:
Created a bearer token.
Created a variable for access_token from bearer token.
newcollection1: new subcollection which I want to create.
testpurview : This is the root collection of my Purview account.
Can someone help me if this is the correct way to create collection?

Getting Invalid JWT when creating project

When I get the token from the itwin-developer-console, the below PowerShell works. However, when I generate my own token, which appears to be valid per all the parsing I do of it, I get 401 Invalid JWT. I can use the token I generate to query and manipulate work area connections with no problems as long as I provide the product-settings-service scope. I've limited the scopes to just be projects:read projects:modify like the token I get from the console, but no joy. I notice that my token does not populate entitlements. Could that be it?
$url = "https://api.bentley.com/projects/"
$authVal = "Bearer $($oidcClientToken.access_token)"
$today = Get-Date -Format "yyyy-MM-dd"
$projectName = $today + " " + (Get-RandomString -Length 12 -Characters "ABCDEFGHIJKLMNOPQRSTUVWXYZ")
$bodyCreate = ConvertTo-Json -Depth 4 #{
project = #{
displayName = $projectName
projectNumber = $projectName
industry = "Oil and Gas"
projectType = "Offshore Structures"
billingCountry = "US"
status = "active"
allowExternalTeamMembers = $true
}
}
$resp5 = Invoke-RestMethod -ContentType "Application/Json" -Method Post -Uri $url -Body $bodyCreate -Headers #{'Authorization' = $authVal; 'Content-Type' = 'application/json'; 'Accept' = 'application/vnd.bentley.itwin-platform.v1+json' }
You need a different token for calling iTwin Platform APIs, please read the Authorization documentation on this.
Specifically in this case, the token needs to be issued by https://ims.bentley.com instead of https://imsoidc.bentley.com, which is currently used by itwin.js applications. The same client will work.

ARM Template deploymentScript PowerShell try catch ignored?

I am using an ARM Template of type "Microsoft.Resources/deploymentScripts". That uses a PowerShell script that should add an identity to a role in Azure AD. When I run that a second time the post requests, the last Invoke-RestMethod, to add the member fails. That intentional and okay because the member is already there. I wrapped that in a try catch and look for the specific error. This works fine locally but not when deployed on Azure. It still stops at the line of the invoke, 41, and does not seem to respect the try catch around that. What am I doing wrong?
param([string] $spObjectId, $roleName, $tenantId, $clientId, $clientSecret)
Write-Host $ErrorActionPreference
$ErrorActionPreference = "Continue"
$Body = #{
'tenant' = $tenantId
'client_id' = $clientId
'scope' = 'https://graph.microsoft.com/.default'
'client_secret' = $clientSecret
'grant_type' = 'client_credentials'
}
$url = "https://login.microsoftonline.com/$tenantId/oauth2/v2.0/token"
Write-Host $url
$Params = #{
'Uri' = $url
'Method' = 'Post'
'Body' = $Body
'ContentType' = 'application/x-www-form-urlencoded'
}
$AuthResponse = Invoke-RestMethod #Params
$Headers = #{
'Authorization' = "Bearer $($AuthResponse.access_token)"
}
$roles = Invoke-RestMethod -Uri 'https://graph.microsoft.com/v1.0/directoryRoles' -Headers $Headers
$role = $roles.value | Where-Object { $_.displayName -eq "$roleName" }
$roleId = $role.id
$Body = #{
"#odata.id"= "https://graph.microsoft.com/v1.0/directoryObjects/$spObjectId"
}
$json = $Body | ConvertTo-Json
$url = "https://graph.microsoft.com/v1.0/directoryRoles/$roleId/members/`$ref"
try {
Invoke-RestMethod -Method 'Post' -Uri $url -Body $json -Headers $Headers -ContentType 'application/json'
}
catch {
$errordetails = $_.ErrorDetails.Message | ConvertFrom-Json
if($errordetails.error.message -ne "One or more added object references already exist for the following modified properties: 'members'.")
{
throw $_.Exception
}
}
Write-Host "Done"
The error in the Azure Portal
The provided script failed with the following error:
Microsoft.PowerShell.Commands.HttpResponseException: Response status
code does not indicate success: 400 (Bad Request). at
System.Management.Automation.MshCommandRuntime.ThrowTerminatingError(ErrorRecord
errorRecord) at ,
/mnt/azscripts/azscriptinput/AddMemberToRole.ps1: line 41 at
, : line 1 at ,
/mnt/azscripts/azscriptinput/DeploymentScript.ps1: line 192. Please
refer to https://aka.ms/DeploymentScriptsTroubleshoot for more
deployment script information. (Code: DeploymentScriptError)
Ideally, Invoke-RestMethod's error is captured in the try catch block. Your code looks correct and similar to what has to be done:
try{ $restp=Invoke-RestMethod (...) } catch {$err=$_.Exception}
$err | Get-Member -MemberType Property
TypeName: System.Net.WebException
Name MemberType Definition
---- ---------- ----------
Message Property string Message {get;}
Response Property System.Net.WebResponse Response {get;}
Status Property System.Net.WebExceptionStatus Status {get;}
I suspect two reasons for this behavior:
The ARM template is having issues such that try catch block is neglected. Make sure the syntax is correct and expected (ARM works in local).
An old deployed script is messing with this script. Try changing the resource name and redeploy.

EDIT the Azure databrics cluster's SPARK configuration using PowerShell and REST API

I am trying to EDIT the Azure databrics cluster's SPARK configuration using PowerShell and REST API. However I am getting an error which I am unable to understand/fix. I have provided the 'required' fields as parameters, however, the error states that I haven't passed them
CODE:
$DBAPIRootUrl = "dec" # example: https://uksouth.azuredatabricks.net
$DBAPIKey = "abc" # Example dapi601e67891a9d1f7886e40916479aaa
[System.Net.ServicePointManager]::SecurityProtocol = [System.Net.SecurityProtocolType]::Tls12
$ClustersAPIListUrl = $DBAPIRootUrl.Trim('/') + "/api/2.0/clusters/list"
$ClustersAPIEditUrl = $DBAPIRootUrl.Trim('/') + "/api/2.0/clusters/edit"
$headers = #{
Authorization = "Bearer $DBAPIKey"
"Content-Type" = "application/json"
}
$response = Invoke-WebRequest -Uri $ClustersAPIListUrl -Method GET -Headers $headers #-Body $parameters
$json_response = ($response.Content | ConvertFrom-Json)
$jsonDoc = [pscustomobject]#{
cluster_id = $json_response.clusters.cluster_id
spark_version = $json_response.clusters.spark_version
node_type_id = $json_response.clusters.node_type_id
spark_conf = "
javax.jdo.option.ConnectionPassword
datanucleus.fixedDatastore false
javax.jdo.option.ConnectionURL jdbc:sqlserver://metadatasrvr.database.windows.net:1433;database=emptydb
datanucleus.schema.autoCreateAll true
spark.hadoop.hive.metastore.schema.verification false
datanucleus.autoCreateSchema true
spark.sql.hive.metastore.jars maven
javax.jdo.option.ConnectionDriverName com.microsoft.sqlserver.jdbc.SQLServerDriver
spark.sql.hive.metastore.version 1.2.0
javax.jdo.option.ConnectionUserName"
}
$jsonDoc | ConvertTo-Json
#$parameters | ConvertTo-Json
$response = Invoke-WebRequest -Uri $ClustersAPIEditUrl -Method POST -Headers $headers -Body $jsonDoc
ERROR:
Invoke-WebRequest : {"error_code":"INVALID_PARAMETER_VALUE","message":"Missing required fields: cluster_id, size"}
At line:21 char:13
+ $response = Invoke-WebRequest -Uri $ClustersAPIEditUrl -Method POST - ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-WebRequest], WebException
+ FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeWebRequestCommand
The error message clearly explains ""error_code":"INVALID_PARAMETER_VALUE","message":"Missing required fields: cluster_id, size"}".
Note: While editing Databricks cluster, make sure to pass the "cluster_id" and "node_type_id" as a mandatory expecting fields.
To Edit the configuration of a cluster to match the provided attributes and size.
An example request:
{
"cluster_id": "1202-211320-brick1",
"num_workers": 10,
"spark_version": "5.3.x-scala2.11",
"node_type_id": "Standard_D3_v2"
}
Reference: Databricks - REST API EDIT clusters
Hope this helps.

API: Trying to set name of an appveyor project

I am trying to create an appveyor project using powershell and set my own name.
function Invoke-AppveyorCreateProject {
param (
[System.String] $appveyorProjectName,
[System.String] $repositoryName
)
# step 1, create project
$project = #{
repositoryProvider="gitHub"
repositoryName="esskar/$repositoryName"
isPrivate=$true
}
$json = $project | ConvertTo-Json
$project = Invoke-RestMethod -Method Post -Uri "$AppveyorApiUrl/projects" -Headers $AppveyorRestHeaders -Body $json -ContentType "application/json"
# step 2, update project
$project.name = $appveyorProjectName
$json = $project | ConvertTo-Json
$project = Invoke-RestMethod -Method Put -Uri "$AppveyorApiUrl/projects" -Headers $AppveyorRestHeaders -Body $json -ContentType "application/json"
}
# Globals
$AppveyorApiUrl = 'https://ci.appveyor.com/api'
$AppveyorApiToken = $env:AppveyorApiToken
$AppveyorRestHeaders = #{
"Authorization" = "Bearer $AppveyorApiToken"
"Content-type" = "application/json"
}
$AppveyorAccountName = $env:AppveyorAccountName
Invoke-AppveyorCreateProject "foo.bar" "repo1"
Step 1 works, the project is created, but when I try to change the name, i get an exception
Invoke-RestMethod : {"message":"Object reference not set to an instance of an object."}
At appveyortest.ps1:20 char:16
+ ... $project = Invoke-RestMethod -Method Put -Uri "$AppveyorApiUrl/proje ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-RestMethod], WebExc
eption
+ FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeRestMethodCommand
when i try to set the name during creation in step 1, then name is not used, and project has the same name as without setting the name.
Use the following code for the Step 2:
$settings = Invoke-RestMethod -Uri "$AppveyorApiUrl/projects/$($project.accountName)/$($project.slug)/settings" -Headers $AppveyorRestHeaders -Method Get
$settings.settings.name = $appveyorProjectName
Invoke-RestMethod -Uri "$AppveyorApiUrl/projects" -Headers $AppveyorRestHeaders -Body ($settings.settings | ConvertTo-Json -Depth 10) -Method Put