Web Activity endless running In Azure Data Factory - azure-data-factory-2

Currently it seems that web activity is broken.
When using simple pipeline
{
"name": "pipeline1",
"properties": {
"activities": [
{
"name": "Webactivity",
"type": "WebActivity",
"dependsOn": [],
"policy": {
"timeout": "7.00:00:00",
"retry": 0,
"retryIntervalInSeconds": 30,
"secureOutput": false,
"secureInput": false
},
"userProperties": [],
"typeProperties": {
"url": "https://www.microsoft.com/",
"connectVia": {
"referenceName": "AutoResolveIntegrationRuntime",
"type": "IntegrationRuntimeReference"
},
"method": "GET",
"body": ""
}
}
],
"annotations": []
}
}
When debugging it never finishes. There is "in progress" for several minutes.
I tried Web hook and it works.
Is there something else I could try?

A quick note on the "never finishes" issue: one of my pet peeves with Data Factory is that the default timeout for all activities is 7 DAYS. While I've had a few activities that needed to run for 7 hours, a WEEK is a ridiculous default timeout value. One of the first things I do in any production scenario is address the timeout values of all the activities.
As to the Web activity question: I set up a quick example in my test bed and it returned just fine:
Looking at the generated code, the only real difference I see is the absence of the "connectVia" property that was in your example:

Ok I've found it.
The default AutoResolveIntegrationRuntime only had managed private network which I couldn't change. So I created a new Integration Runtime with public network setting.
This is a litte bit strange as I started today with a brand new Azure Data Factory.
I wonder why I cannot change the default Integration Runtime to disable virtual network:

Related

How do I automate adding a custom Iot Hub Endpoint (and route to it)?

In order to receive Azure IotHub Device Twin change notifications, it appears that it's necessary to create a custom endpoint and create a route to send notifications to that endpoint. This seems straightforward enough on the Azure Portal, but as one might expect we want to automate it.
I haven't been able to find any documentation for the the az cli or even the REST API, though I might have missed something. I didn't find anything promising looking in the SDKs either.
How do I automate adding a custom endpoint and then setting up the route for device twin notifications?
You can check IotHubs template to see if it helps.
Route:
"routing": {
"endpoints": {
"serviceBusQueues": [
{
"connectionString": "string",
"name": "string",
"subscriptionId": "string",
"resourceGroup": "string"
}
]
},
"routes": [
{
"name": "string",
"source": "string",
"condition": "string",
"endpointNames": [
"string"
],
"isEnabled": boolean
}
],
Consumer group:
{
"apiVersion": "2016-02-03",
"type": "Microsoft.Devices/IotHubs/eventhubEndpoints/ConsumerGroups",
"name": "[concat(parameters('hubName'), '/events/cg1')]",
"dependsOn": [
"[concat('Microsoft.Devices/Iothubs/', parameters('hubName'))]"
]
},
For more detailed information you can reference:
Microsoft.Devices/IotHubs template reference
Create an IoT hub using Azure Resource Manager template (PowerShell)

How do I create an event using the SocialTables API?

I'm trying to use the /4.0/legacyvm3/teams/{team}/events endpoint to create an event. I'm running into some trouble with spaces.
I used the /4.0/legacyvm3/teams/{team}/venues endpoint to get a list of venues. I chose one to include in the spaces section and posted this:
{
"name": "Event via API Test 04",
"category": "athletic event",
"public": true,
"attendee_management": true,
"start_time": "2017-04-05T16:13:54.217Z",
"end_time": "2017-04-05T16:13:54.217Z",
"uses_metric": false,
"venue_mapper_version": 0,
"spaces": [
{
"venue_id": 128379,
"name": "Snurrrggggg"
}
]
}
The endpoint returns a 400 code and this error:
{
"code": 400,
"message": "Cannot read property 'toLowerCase' of undefined"
}
I tried including the wizard section, but each time it would return this error:
{
"message": "Access Denied to this feature"
}
After some experimentation, this body succeeded:
{
"name": "Event via API Test 03",
"category": "athletic event",
"public": true,
"attendee_management": true,
"start_time": "2017-04-05T16:13:54.217Z",
"end_time": "2017-04-05T16:13:54.217Z",
"uses_metric": false,
"venue_mapper_version": 0,
"spaces": [
{
"name": "Fake News Room"
}
]
}
But the application itself would not display the diagram, and the newly created room did not show up in my list of venues. Perhaps it did not assign permissions to it?
In any case, I don't actually want to create a new venue/space. I want to pass in an existing venue/space. How do I do that?
The short answer is to create a working diagram in 4.0 you will need to POST some data to the /4.0/diagrams endpoint.
The room you create doesn't map to the same concept as venues. When you create an event as you did, it creates a new space entity. The spaces endpoints can return information on those.

Dropbox API V2 list_file_members/batch empty results

I'm currently trying to work with the Dropbox list_file_members API endpoint, as it appears to me to be the only place to find out who owns a file (
see follow example result taken from the documentation page )
{
"users": [
{
"access_type": {
".tag": "owner"
},
"user": {
"account_id": "dbid:AAH4f99T0taONIb-OurWxbNQ6ywGRopQngc",
"same_team": true,
"team_member_id": "dbmid:abcd1234"
},
"permissions": [],
"is_inherited": false
}
],
"groups":[...]
...
}
However, when I call the API on a single file I get the follow
{
"users": [],
"groups": [
{
"access_type": {
".tag": "editor"
},
"permissions": [],
"is_inherited": true,
"group": {
"group_name": "Everyone at TEAM_NAME_HERE",
"group_id": "g:GROUP_ID_HERE",
"member_count": 6,
"group_management_type": {
".tag": "company_managed"
},
"group_type": {
".tag": "team"
},
"is_owner": false,
"same_team": true
}
}
],
"invitees": []
}
This result contains no owner information, so I'm assuming this is because everyone has the same access levels ??
The problem worsens when I try to call files in batches using the sharing_list_file_members/batch endpoint, I get the following result
[
{
"file": "id:THIS_IS_MY_FILE_ID",
"result": {
".tag": "result",
"members": {
"users": [],
"groups": [],
"invitees": []
},
"member_count": 0
}
}
]
Obviously this is even less helpful, this is the same when I access the API via my own PHP, as well as the API explorer, could anyone tell me where I'm going wrong and why I'm getting no results from users and even groups when done in batches ?
The /2/sharing/list_file_members endpoint is documented as:
Use to obtain the members who have been invited to a file, both inherited and uninherited members.
The /2/sharing/list_file_members/batch endpoint is documented as:
Get members of multiple files at once. The arguments to this route are more limited, and the limit on query result size per file is more strict. To customize the results more, use the individual file endpoint.
Inherited users are not included in the result, and permissions are not returned for this endpoint.
It sounds like the file for your example is in a team folder, and so the group listed for your non-batch example is the team group, i.e., an inherited group. The documentation indicates that this group isn't expected when using the batch endpoint.

ARM - How can I get the access key from a storage account to use in AppSettings later in the template?

I'm creating an Azure Resource Manager template that instantiates multiple resources, including an Azure storage account and an Azure App Service with a Web App.
I'd like to be able to capture the primary access key (or the full connection string, either way is fine) from the newly-created storage account, and use that as a value for one of the AppSettings for the Web App.
Is that possible?
Use the listkeys helper function.
"appSettings": [
{
"name": "STORAGE_KEY",
"value": "[listKeys(resourceId('Microsoft.Storage/storageAccounts', parameters('storageAccountName')), providers('Microsoft.Storage', 'storageAccounts').apiVersions[0]).keys[0].value]"
}
]
This quickstart does something similar:
https://azure.microsoft.com/en-us/documentation/articles/cache-web-app-arm-with-redis-cache-provision/
The syntax has changed since the other answer was accepted. The error you will now hit is 'Template language expression property 'key1' doesn't exist, available properties are 'keys'
Keys are now represented as an array of keys, and the syntax is now:
"StorageAccount": "[Concat('DefaultEndpointsProtocol=https;AccountName=',variables('StorageAccountName'),';AccountKey=',listKeys(resourceId('Microsoft.Storage/storageAccounts', variables('StorageAccountName')), providers('Microsoft.Storage', 'storageAccounts').apiVersions[0]).keys[0].value)]",
See: http://samcogan.com/retrieve-azure-storage-key-in-arm-script/
I faced with this issue two times. First in the 2015 and last today in May of 2017.
I need to add connection strings to the WebApp - I want to add strings automatically from generated resources during deployment from the ARM template. It can help later to not add manually this values.
First time I used old version of the function listKeys (it looks like old version returns result not as object but as value):
"AzureWebJobsStorage": {
"type": "Custom",
"value": "[concat(variables('storageConnectionString'), listKeys(resourceId('Microsoft.Storage/storageAccounts', parameters('storageAccountName')), '2015-05-01-preview').key1)]"
},
Today last version of the working template is:
"resources": [
{
"apiVersion": "2015-08-01",
"type": "config",
"name": "connectionstrings",
"dependsOn": [
"[resourceId('Microsoft.Web/Sites/', parameters('webSiteName'))]"
],
"properties": {
"DefaultConnection": {
"value": "[concat('Data Source=tcp:', reference(resourceId('Microsoft.Sql/servers/', parameters('sqlserverName'))).fullyQualifiedDomainName, ',1433;Initial Catalog=', parameters('databaseName'), ';User Id=', parameters('administratorLogin'), '#', parameters('sqlserverName'), ';Password=', parameters('administratorLoginPassword'), ';')]",
"type": "SQLServer"
},
"AzureWebJobsStorage": {
"type": "Custom",
"value": "[concat(variables('storageConnectionString'), listKeys(resourceId('Microsoft.Storage/storageAccounts', parameters('storageName')), '2016-01-01').keys[0].value)]"
},
"AzureWebJobsDashboard": {
"type": "Custom",
"value": "[concat(variables('storageConnectionString'), listKeys(resourceId('Microsoft.Storage/storageAccounts', parameters('storageName')), '2016-01-01').keys[0].value)]"
}
}
},
Thanks.
below is example for adding storage account to ADLA
"storageAccounts": [
{
"name": "[parameters('DataLakeAnalyticsStorageAccountname')]",
"properties": {
"accessKey": "[listKeys(variables('storageAccountid'),'2015-05-01-preview').key1]"
}
}
],
in variable you can keep
"variables": {
"apiVersion": "[providers('Microsoft.Storage', 'storageAccounts').apiVersions[0]]",
"storageAccountid": "[concat(resourceGroup().id,'/providers/','Microsoft.Storage/storageAccounts/', parameters('DataLakeAnalyticsStorageAccountname'))]"
},

Xbmc Database Path

I am working with XBMC. I have installed XBMC in my system(Windows 7, 32 bit). Xbmc is working fine in my system. I have developed an application in order to control the Xbmc remotely from Ipad. In order to retrieve the music files or video files from Xbmc, I am unable to. By searching the forums of xbmc, I found that we can write an sql query to get them out. But, the thing is I am unable to make out where the database is located in my system. Someone help me out where I can find it.
Regards,
Sushma.
The database itself
By default the location of the database is that described on the wiki page XBMC databases
but the actual location can be changed by the user, or a different database technology can be used entirely.
The settings that would affect this are located in advancedsettings.xml.
But in general it is advised by the XBMC developers to never access the database directly.
JSONRPC
In order to help with interacting with the database XBMC has supported the JSONRPC queries, the one downside of these is that XBMC needs to be running at the time to respond to these queries. The major advantage is that it XBMC will find the database for you and expose access to it with a common interface.
JSONRPC support was first added to XBMC in "Darhma" (v10), became really useful in "Eden" (v11) and will support almost everything possible in "Frodo" (v12). Information about the use of JSONRPC can be found in the wiki.
An example
In this example I'm assuming that you are targeting "Eden", the current stable release of XBMC. Also I have formatted the following with new lines, these are not required and are not present in the response from XBMC.
Request
If you were to use JSONRPC the request you would need to send would look something like:
{
"jsonrpc": "2.0",
"method": "VideoLibrary.GetMovies",
"params": {
"properties": [
"title",
"year",
"file"
],
"limits": {
"start": 0,
"end": 2
}
},
"id": 1
}
Note: If you wanted different information about each movie you could use other properties listed here.
*Note: You probably want to omit the "limits" part to get all movies.*
Responce
The response to this would be something like:
{
"id": 1,
"jsonrpc": "2.0",
"result": {
"limits": {
"end": 2,
"start": 0,
"total": 47
},
"movies": [
{
"label": "Label for movie",
"movieid": 1,
"title": "Title of movie",
"year": 2012
},
{
"label": "Label for another movie",
"movieid": 2,
"title": "Title of another movie",
"year": 2010
},
{
"label": "Label for a third movie",
"movieid": 3,
"title": "Title of a third movie",
"year": 2012
}
]
}
}
What to do now?
You have a choice at this point, you can either:
Add "file" to the list of properties, this will return the "file" property, the location of the video file.
Use JSONRPC to tell xbmc to play a movie.
Using this method is best when you don't want to play the file locally (on the iPad) but instead on XBMC.
Playing a movie on XBMC via JSONRPC
This is quite simple, use the "movieid" you received earlier in the following request:
{
"jsonrpc": "2.0",
"method": "Player.Open",
"params": {
"item": {
"movieid": 2
}
},
"id": 1
}
Lastly I would note that there are equivalent commands for TV episodes as shown for movies.