I want to copy data from Web tables to SQL in Azure Data Factory.
I have created a linked service of type web table and provided a URL.
The page contains 2 web tables.
Both web tables are accessible using indexes 0 and 1
I have also parameterized that index.
Questions:
How to identify the total number of web tables on a given page?
How to identify web table names or relative paths on a given page?
How to configure multiple URLs in a single web table linked service?
Related
I am trying to establish a way of ingesting multiple company data with Azure Data Factory using HTTP Link Service.
The below image shows how to ingest a single data source or a single entry using the HTTP link
https://duedil.io/v4/company/gb/06999618.json
The above would give me the financials for a single company. If we want to ingest the financials of another company with company id of 07803944 we would have to replace the above link with
https://duedil.io/v4/company/gb/07803944.json
For obvious reasons it would not be practical to have a copy activity or pipeline for every company we need the financials for.
Therefore, can someone let me know if's possible to parameterize the link service to ingest multiple companies in the same copy activity? Alternatively, simply add individual company ids to the link service?
I tried the following, but it didn't work.
I want to access multiple storage accounts. I want one sql table to be uploaded to one blob and other sql table to the other. for this, i have to use just one pipeline. How can I create a dynamic blob?
Each storage account requires its own connection string.
One way to do this would be to create a service (for example an azure function) that you pipeline calls. In the function there is logic to place the the correct table in the correct storage account.
We are trying to create QVD by loading data through API. We are using QVSource web-connector.. Our custom API currently provides 2 tables (Customers & Orders)
When we connect QV Source and chose, JSON TO TABLE.. it is only showing first table and only creates load script for the first table.
How can I make QV Source produce load script for more then one table?
I am also using Fiddler web debugger which shows that the API in-deed is fetching 2 tables..
Screenshot attached.
I'm trying to move some tables from SQL to Azure Table Storage.
I created an MVC Website with the default authentication. I successfully connected it to my Azure SQL database. Now I want to use the table storage for authentication too, instead of the SQL database.
The problem is, I cannot find my storage account's unique namespace. What, where is that namespace?
Thanks!
Looking at a table URL, for example 'http://myaccountname.blob.core.windows.net/mytable', the 'myaccountname' will be the name of your account. Storage account names must be between 3 and 24 characters in length and may contain numbers and lowercase letters only. The storage account name must be unique on the Azure service. A list of storage accounts your own and more information about them can be found in the Azure Portal.
More information on authentication for tables can be found here and here. Manipulating and authenticating access to your tables are features built into the storage client libraries which are available in a variety of languages. Since you mention MVC, you might want to check out the .Net storage library.
I'm currently creating an SQL database that will eventually be accessed by web applications, both mobile and desktop platforms.
Setting up the SQL database is easy.
What I'm currently having trouble with is how I'm going to allow clients to access to information in the database without having direct access to the database, for security purposes.
For example, let's say I have a list of employees, and I'm creating employee profiles for a mobile application. I would like for the mobile application to be able to use a function such as get(employee_name), which will call the database and retrieve the information about the employee in the form of a JSON.
How would I go about doing this?
To see a very nice solution on using JSON on sql server tables please look at Phil Factors article Consuming JSON Strings in SQL Server at:
https://www.simple-talk.com/sql/t-sql-programming/consuming-json-strings-in-sql-server/