What are the best options for ingesting data from SaaS Application to Synapse through API with Authentication? We have a couple of SaaS Applications such as NetSuite, EyeShare & SAP SaaS Applications, and planning to ingest the data to Synapse but not finding enough information on the connectors supported by Synapse or ADF. Kindly help to suggest the tool or the method to get the SaaS data via. APIs.
As wBob said, I think we can use Web Hook tasks in pipelines to call API and ingest the data to Synapse.
Related
I have created instance in Azure API management service and want to import 200+ APIs. Is there any way to automatically import the APIs in portal by writing script or any other way also want to create a product and group the imported APIs?
You can automate deployment of API's by using Azure APIM REST API. You will find everything there. Basically whatever you do in Azure Portal on APIM resource by clicking you can do by calling this REST API.
If you don't feel like calling REST API, you can always use Az.ApiManagement PowerShell Module or az apim Azure CLI - they have more or less the same functionality.
I guess that is it, the question is too broad to give you more specifics.
I am working with a customer who is using Google BigQuery, and we are attempting to reverse-engineer their model. The problem is that the connection requires the Project Id in a FQDN form, which usually comes to mydomain.com:myProject
I am not familiar at all with BigQuery, is there a place in the users dashboard i can find this? Everything we have browsed through only seems to give us the Project ID.
GCP Bigquery represents a concept of Enterprise data warehouse, which is isolated and GCP fully-managed environment for high-scalable operations addressing data analytics tasks. Although GCP Bigquery conceptually built as serverless platform Software as a Service (SaaS) providing an execution environment for customer analytics purposes, infrastructure and hardware components are self managed by Google cloud computing inventory, that means that Bigquery service doesn't offer any regular hosting endpoint for external visitors.
The interaction with GCP Bigquery can be only established in the following ways:
BigQuery web UI in the Cloud Console;
Using BigQuery command-line tool;
Bigquery REST API through service endpoint:
https://bigquery.googleapis.com;
Using programming language specific Cloud Client Libraries for
the BigQuery API.
I am trying to port my Android application into a Cloud based one. Where I wanted the data stored in cloud. I am a .NET guy so looking into Azure. I see Azure Mobile services which allows me to create tables and ability to authenticate. I also Looked into Azure SQL database.
I would like to know what is the difference between these two services and what are the scenarios using mobile services gives values than using Azure SQL database
Actually, the tables you see listed in Azure Mobile Services are actually tables that are in an Azure SQL Database instance. The Mobile Services provides an abstraction layer on top of the tables in SQL Server. If you use the current JavaScript back-ended mobile services you'll see the table in the management portal and it uses the dynamic nature of JavaScript to provide what appears to be a NoSQL like experience with Azure SQL DB in that you can send in an object and properties it doesn't recognize will be added to the table schema (you turn this off when you ship to production :) ). The big thing to remember is that under the hood the database is an Azure SQL DB, so you should have access and do anything via the one created by Azure Mobile Services as you can with the raw Azure SQL DB.
By using Azure Mobile Services you get an API hosting layer that you can use to do direct access to the tables (based on the permissions you hinted at) but also to just about anything utilizing the custom API feature.
If you chose to bypass the Azure Mobile Services you'd likely want to have some API layer between your Android app and the data for a variety of reasons (security, abstraction of data location, etc.). In which case you'd have to write and host that API layer somewhere. Other options within Azure would be a Web API site in Azure Web Sites, an API hosted in an Azure Cloud Service or all the way up to an Azure VM; however, as you slide up that scale you're taking on more and more responsibility and work.
The scenario that is useful to use Azure Mobile Services over the Azure SQL DB is pretty much the scenario you have. You don't have to maintain your own hosting API layer, just the code the API layer executes. You can scale an necessary (to a very large scale indeed) and also get features like the push notifications and web jobs (though those are also things you can do via other services in Azure if you'd like). Thinks of Azure Mobile Services as a higher level grouping of Azure services that are helpful to mobile developers.
Mobile services provide you with built-in push notification features and other such mobile friendly features too. They are more mobile oriented in a way.
I have SOAP services with data. I want to download all data from that service and upload it to my own server and then use these data from my server (don't worry I have permission for that). I want to do it because now I don't have function from that service which I need.
I want to use Windows Azure for this and I think SQL Database scenario would be best. Now I have classes for previous SOAP service so I think EF Code first would help me with creating database and I upload data somehow. But what about API? How can I access my data from windows phone or tablet? Is azure database enought or I must create more? Is there any good article for that?
I think what you are saying, is that you are aggregating data from several sources and storing the information in your own database. And, you would like your database to be Azure Database. Then, you want to build an API to expose the data you retrieved.
If this is indeed your goal, then yes, Azure will do everything you need. I'd recommend checking out Web API in conjunction with your Azure deployment. I've used this scheme with some success over the past year.
Warning: You should know that Azure Database does not have an SLA which means that Microsoft does not guarantee any level of performance including transactions/second. This means that if your API has a high load, you could end up getting throttled heavily in an unpredictable way. I've been bitten by this before and ended up moving my data to Azure Table Storage instead.
Windows Azure gives you a few options to expose an API to your mobile clients:
You could build an API yourself with the ASP.NET Web API (and use SQL Azure as backend): Mobile-friendly REST service using ASP.NET Web API and SQL Database
You can use Windows Azure Mobile Services, this does all the heavy lifting of building a backend for you
I am to implement a service such the user can upload files to azure storage, and also store the meta data in a azure sql database.
Lets say the user want to upload 1gb of data, is it then needed to be send to the webservice first and then from service to azure storage? Is there a way to initiate the upload and then the user sends the data directly to the server with azure storage?
I think i read something like that but cant find it now and not sure what to search for on google.
Yes this is possible but you'll need to use a client side technology which can talk to the REST API, like Silverlight for example. Steve Marx did a series of blog posts explaining how you can leverage Silverlight and the REST API to upload files to blob storage (even very large files): Uploading Windows Azure Blobs from Silverlight – Part 3: Handling Big Files by Using the Block APIs. And a Silverlight control is very easy to integrate in an existing website.