I am debugging a client application calling REST API embedded with AZURE storage SAS-shared access signature to access azure storage resource. However, it is not getting through. The azure threw out an error stating that the mandatory header is missing, and abort the operation.
The REST API is fairly simple, although it is embedded with the SAS token generated by the azure storage account. The client application uses the REST API to write data into an azure blob.
is there anywhere I can find a good example showing how to generate the header for the REST API (SAS)? I need to find out the exact layout of the header (such as the type of information that needs to be embedded in the header.
Also, do I need to register my client application with the Azure AD?
I didn't think my client application needs to be registered with the AZURE since that is why we have client-side SAS. But, I could be wrong. Therefore, any input will be appreciated.
Thanks in advance.
If you use sas token to call Azure blob rest api, the request URL should be like
https://myaccount.blob.core.windows.net/<cantianer>/<blob>?<sastoken>
For example
$accountName=""
$accountKey=""
$containerName="output"
$blobName="test.txt"
$context= New-AzStorageContext -StorageAccountName $accountName -StorageAccountKey $accountKey
$sas = New-AzStorageAccountSASToken -Service Blob -ResourceType Service,Container,Object -Permission "rwdlacx" -Context $context
$body = "Hello"
$headers=#{"x-ms-blob-type"="BlockBlob"; "Content-Type"="text/plain"}
$url="https://$accountName.blob.core.windows.net/$containerName/$blobName$sas"
Invoke-WebRequest -Uri $url -Method Put -Headers $headers -Body $body -UseBasicParsing
Related
I am getting following error while running DataFlow pipeline
Error reporting inventory checksum: code: "Unauthenticated", message: "Request is missing required authentication credential.
Expected OAuth 2 access token, login cookie or other valid authentication credential.
We have created service account dataflow#12345678.iam.gserviceaccount.com with following roles
BigQuery Data Editor
Cloud KMS CryptoKey Decrypter
Dataflow Worker
Logs Writer
Monitoring Metric Writer
Pub/Sub Subscriber
Pub/Sub Viewer
Storage Object Creator
And in our python code we are using import google.auth
Any idea what am I missing here ?
I do not believe I need to create key for SA , however I am not sure if "OAuth 2 access token" for SA need to be created ? If yes how ?
This was the issue in my case https://cloud.google.com/dataflow/docs/guides/common-errors#lookup-policies
If you are trying to access a service through HTTP, with a custom request (not using a client library), you can obtain a OAuth2 token for that service account using the metadata server of the worker VM. See this example for Cloud Run, you can use the same code snippet in Dataflow to get a token and use it with your custom HTTP request:
https://cloud.google.com/run/docs/authenticating/service-to-service#acquire-token
I am trying to get response back from Azure management api in Data Factory for instance use this api (https://learn.microsoft.com/en-us/rest/api/resources/resource-groups/list) to get the resource group name in my pipeline. I am able to get the response back in Postman but not sure how to replicate it in Data Factory. I tried using the POST method by passing in the Client ID & Client Credentials like below and no luck. But, Not sure how to achieve this is Data Factory. What am I missing here.
$cmd = { .\curl.exe -X POST https://login.microsoftonline.com/common/oauth2/authorize -F grant_type=client_credentials -F resource=https://management.core.windows.net/ -F client_id=$client_id -F client_secret=$client_secret };
$responseToken = Invoke-Command -scriptblock $cmd;
$accessToken = (ConvertFrom-Json $responseToken).access_token;
(ConvertFrom-Json $responseToken)
Thank you.
I am confused what is the API you are refering to ? Can you please share the Azure docs about that ? . I see that there is a subscriptionid which needs to be put forward as part of the API call .
Here is how the normal flow goes , get the bearer token ( make sure that the you pass the right SCOPE ) , once you have the a bearer token you can use that to call the azure API , bearer token should also be part of the parameter with other parameter( if any ) .
I am trying to upload a local PDF file into share point via using Graph API but the PUT method is just letting me to create a new file in the SharePoint but not letting me upload a local existing C:/ Drive PDF file
Here is the API that I am using
PUT https://graph.microsoft.com/v1.0/drives/{drive-id}/root:/SharePointFilePath/file-name:/content
As per here, that is the correct endpoint. however, you need to send ithe file as a binary stream in the body. https://learn.microsoft.com/en-us/graph/api/driveitem-put-content?view=graph-rest-1.0&tabs=http
This means that you need to read your local pdf file into a binary stream and send it to the endpoint in the body of the put request. as a test for example, you could create a regular text file with the body as string and contenttype as text.
also if it's larger than 4MB, then you need to use a different method. https://developer.microsoft.com/en-us/graph/docs/api-reference/v1.0/api/driveitem_createuploadsession mainly an uploadsession.
Previous test script to upload file to SharePoint by PnP PowerShell.
https://social.technet.microsoft.com/Forums/msonline/en-US/25191b51-41dd-4f4b-93aa-a46594c9a184/uploading-a-file-to-sharepoint-online-using-microsoft-graph-api?forum=onlineservicessharepoint
Connect-PnPOnline -AppId 'yourAzure app id' -AppSecret "yourAzure app secret" -AADDomain 'xxx.onmicrosoft.com'
$accessToken= Get-PnPAccessToken
$apiUrl = "https://graph.microsoft.com/v1.0/sites/xxx.sharepoint.com,x,x/drives/driveid/items/01AKXHS4ELSEOTLPZHZZDYYBOW57WR6HK6:/test.xlsx:/content"
$path="C:\Lee\Script\testdata.xlsx"
$file=[IO.File]::ReadAllBytes($path)
Invoke-RestMethod -Headers #{Authorization = "Bearer $accessToken"} -Uri $apiUrl -Method Put -Body $file -ContentType "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"
I was trying to push some data to PUB/SUB as mentioned here
I assume, the URL "POST https://pubsub.googleapis.com/v1/projects/myproject-id/topics/mytopic-name:publish" with needed data in a json format as described , would publish the data to the pubsub. Is my assumption correct here? or should I write my own handler to publish the data which i get from a webpage using java/php/python etc?
I could successfully push the data through google play ground. But this requires my username/password to be given to authenticate.
Can I use API_KEY alone here to send the data from my webpage to google pubsub? Or should I go for OAuth2.0 login? If OAuth is used, is there any way to provide the client_id and the API_KEY to do the authentication without prompting the user to enter credentials?
Any help on the above topics would be really appreciated.
Thanks
You might be better off using one of the client libraries with a service account. This will make authentication straightforward, by setting the GOOGLE_APPLICATION_CREDENTIALS environment variable. If you cannot do this for some reason, you can set the Authorization: Bearer header with an OAuth token as shown here.
I am building a program that will list albums/folders and images from my Amazon Cloud Drive. I am able to authenticate and get a token by following the instructions here at amazon public apis.
I am having trouble with the next call to get my endpoints described here under Account API.
Using get
GET : {{metadataUrl}}/account/endpoint Gets the customer end-point against
which REST queries will be executed.
So my request looks just like this in fiddler and on my site just like it is described on the Amazon site here under Get Endpoint. The only differences is the token for obvious reasons.
Sample Request:
GET /drive/v1/account/endpoint HTTP/1.1
Host: drive.amazonaws.com
Authorization: Bearer
Atza|IQEBLjAsAhQ5zx7pKp9PCgCy6T1JkQjHHOEzpwIUQM
My response is:
{
"message":"Authorization header requires 'Credential' parameter.
Authorization header requires 'Signature' parameter.
Authorization header requires 'SignedHeaders' parameter.
Authorization header requires existence of either a 'X-Amz-Date'
or a 'Date' header. Authorization=Bearer"
}
What I am expecting:
{
"customerExists": true,
"contentUrl": "https://content-na.drive.amazonaws.com/cdproxy/",
"metadataUrl": "https://cdws.us-east-1.amazonaws.com/drive/v1/"
}
I did find some information about building the objects in my response from amazon, however, it's not listed in the documentation on the amazon developer site. Is this something new that has not been updated in the docs or am I doing something wrong?
I had a new line between Authorization: Bearer and my token Atza|IQEBLjAsAhQ5zx7pKp9PCgCy6T1JkQjHHOEzpwIUQM. After removing the new line my issue was fixed..