How to pass SSL Certificate in Copy Data Activity in Azure Data Factory to retrieve data from Rest Api? - ssl-certificate

I want to copy data from Rest API using data factory. I have passed Access token in Authorization but it showing error that "No required SSL certificate was sent". How can I pass SSL certificate while copying data from Rest API?

REST connector doesn't support this. REST connector only supports these properties:
More details about REST connector, you can refer to this doc https://learn.microsoft.com/en-us/azure/data-factory/connector-rest.

Related

Secure APIs in Choreo

I wrote a sample ballerina code in choreo API to return x-jwt-header header value to the caller.
when I turn off the security for that API, the header value is not there in the request headers. Does it mean the API is unsecured?
I want to crate an unsecured API. I want to know whether the header values will be there or not?
When you disable the security, basically API is exposed without security to the consumers. Hence the API is unsecured. x-jwt-header is used to pass the enduser data to the backend services. If you don't provide an access token when invoking API, then it can't send the enduser information to the backend services. That's why you are getting the response without that header.

How to securely invoke a Azure Logic App from Azure Data Factory

I have a Data Factory pipeline that calls a Logic App. The URL provided by the logic app designer includes a signature in the URL. In the Data Factory pipeline I have a Web activity that uses POST method to call the Logic App. This works, the app is called.
I would either prefer to separate this signature from the URL and store it in Azure Key Vault, or use authentication. Authentication options in Data Factory include; Basic, Managed Identity or Client Certificate. I tried to follow this article to enable authentication via Azure Active Directory OAuth but do not currently see how this would work with Azure Data Factory.
I know it is possible to enable API Management Gateway to enable more sophisticated authentication to APIs including Logic Apps but am not currently convinced that I should need to do this.
The other thing I noticed in the Logic App interface is that you cannot issue authorizations to execute. This differs from other Azure services like Data Lake for example where you can issue 'use-like' authorization to read a container. There is only the Authorization pane that can be used to define claims. I had a little go but I am missing key information.
Ultimately I would not want the signature exposed to developers including myself.
I believe you could pair the Managed Identity of the Web Activity with the Azure AD OAuth feature for Logic Apps.
The required claims need to be setup on the Authorization blade for validation of the tokens.
I personally like storing the PostURL in the Azure KeyVault then doing an Azure KeyVault GET API Call in ADF, then passing the output securely in ADF to the request to trigger the logic app. It's two activities which is like nothing and it's a secure method that can just use managed identity authentication that's Native in ADF.
Here is example of how you get a value from AKV:
Getting value from AKV
Here is the URL for Azure KeyVault example secret:
https://kv-ccok-ops-#{pipeline().globalParameters.Environment}.vault.azure.net//secrets/FILESERVER-UserName?api-version=7.1
Using value from AKV
This example should be super easy to follow.

How getting the client id to specify for a Google BigQuery connector

I'm trying to use a Google BigQuery connector, to read GA data, specifying the User Authentication. Now how can I obtain the unique client id to pass to teh Google BigQuery connector?
Thanks
When configuring the BigQuery connector in Azure Data Factory, you must supply the client ID and client secret. These can be obtained by creating new OAuth client credentials in the GCP console.
When creating the credentials, it will ask you for "Authorized redirect URIs". You can enter anything you want there. It just has to be in a valid URL format. Be sure to make note of what you entered because you will have to use the exact same value when doing the OAuth2 dance.
For more details, you can read my article on how to set up User Authentication for BigQuery in ADF v2: https://github.com/AnalyticJeremy/ADF_BigQuery

How to get a Signature for a Blob Storage in Windows Azure ?

https://configuat.blob.core.windows.net/b2c/Motorist/uat/files/ar/es/config_AR_es.json
This is my web service address.
As a read here there is a way to update Blob storage and I could not get the Signature.
My account is: configuat.
When using Shared Key authentication, the Authorization header must be in a specific format and must be able to authenticate your request with a matching signature. For more information on this topic, please see our MSDN article Authentication for the Windows Azure Storage Services.

What is an Endpoint?

I have been reading about OAuth and it keeps talking about endpoints. What is exactly an endpoint?
Come on guys :) We could do it simpler, by examples:
/this-is-an-endpoint
/another/endpoint
/some/other/endpoint
/login
/accounts
/cart/items
and when put under a domain, it would look like:
https://example.com/this-is-an-endpoint
https://example.com/another/endpoint
https://example.com/some/other/endpoint
https://example.com/login
https://example.com/accounts
https://example.com/cart/items
Can be either http or https, we use https in the example.
Also endpoint can be different for different HTTP methods, for example:
GET /item/{id}
PUT /item/{id}
would be two different endpoints - one for retrieving (as in "cRud" abbreviation), and the other for updating (as in "crUd")
And that's all, really that simple!
All of the answers posted so far are correct, an endpoint is simply one end of a communication channel. In the case of OAuth, there are three endpoints you need to be concerned with:
Temporary Credential Request URI (called the Request Token URL in the OAuth 1.0a community spec). This is a URI that you send a request to in order to obtain an unauthorized Request Token from the server / service provider.
Resource Owner Authorization URI (called the User Authorization URL in the OAuth 1.0a community spec). This is a URI that you direct the user to to authorize a Request Token obtained from the Temporary Credential Request URI.
Token Request URI (called the Access Token URL in the OAuth 1.0a community spec). This is a URI that you send a request to in order to exchange an authorized Request Token for an Access Token which can then be used to obtain access to a Protected Resource.
It's one end of a communication channel, so often this would be represented as the URL of a server or service.
An endpoint is a URL pattern used to communicate with an API.
An endpoint is the 'connection point' of a service, tool, or application accessed over a network. In the world of software, any software application that is running and "listening" for connections uses an endpoint as the "front door." When you want to connect to the application/service/tool to exchange data you connect to its endpoint
Endpoint, in the OpenID authentication lingo, is the URL to which you send (POST) the authentication request.
Excerpts from Google authentication API
To get the Google OpenID endpoint, perform discovery by sending either a GET or HEAD HTTP request to https://www.google.com/accounts/o8/id. When using a GET, we recommend setting the Accept header to "application/xrds+xml". Google returns an XRDS document containing an OpenID provider endpoint URL.The endpoint address is annotated as:
<Service priority="0">
<Type>http://specs.openid.net/auth/2.0/server</Type>
<URI>{Google's login endpoint URI}</URI>
</Service>
Once you've acquired the Google endpoint, you can send authentication requests to it, specifying the appropriate parameters (available at the linked page). You connect to the endpoint by sending a request to the URL or by making an HTTP POST request.
Short answer: "an endpoint is an abstraction that models the end of a message channel through which a system can send or receive messages" (Ibsen, 2010).
Endpoint vs URI (disambiguation)
The endpoint is not the same as a URI. One reason is because a URI can drive to different endpoints like an endpoint to GET, another to POST, and so on. Example:
#GET /api/agents/{agent_id} //Returns data from the agent identified by *agent_id*
#PUT /api/agents/{agent_id} //Update data of the agent identified by *agent_id*
Endpoint vs resource (disambiguation)
The endpoint is not the same as a resource. One reason is because different endpoints can drive to the same resource. Example:
#GET /api/agents/{agent_id} #Produces("application/xml") //Returns data in XML format
#GET /api/agents/{agent_id} #Produces("application/json") //Returns data in JSON format
The term Endpoint was initially used for WCF services. Later even though this word is being used synonymous to API resources, REST recommends to call these URI (URI[s] which understand HTTP verbs and follow REST architecture) as "Resource".
In a nutshell, a Resource or Endpoint is kind of an entry point to a remotely hosted application which lets the users to communicate to it via HTTP protocol.
The endpoint of the term is the URL that is focused on creating a request.
Take a look at the following examples from different points:
/api/groups/6/workings/1
/api/v2/groups/5/workings/2
/api/workings/3
They can clearly access the same source in a given API.
API stands for Application Programming Interface. It is a way for your application to interact with other applications via an endpoint. Conversely, you can build out an API for your application that is available for other developers to utilize/connect to via HTTP methods, which are RESTful. Representational State Transfer (REST):
GET: Retrieve data from an API endpoint.
PUT: Update data via an API - similar to POST but more about updating info.
POST: Send data to an API.
DELETE: Remove data from given API.
PATCH: Update data.