BigQuery API Job AllowLargeResults - google-bigquery
I am currently passing allowLargeResults:True to the BigQuery API via a Job, and I am getting the following error:
Reason:invalidQuery. Message:Unclosed string literal at: 1:779
The strange thing is that if I set it to False, it works fine.
My full job:
{u'status': {u'state': u'RUNNING'}, u'kind': u'bigquery#job',
u'statistics': {u'creationTime': u'1442029780703', u'startTime':
u'1442029780902'}, u'jobReference': {u'projectId': u'api-project-617799710969',
u'jobId': u'job_1UBxC6nXSBRRerkGNUNr6_DdyPU'},
u'etag': u'"m-xx"', u'user_email': u'xx#google.com', u'configuration':
{u'query': {u'query': u'SELECT * FROM table ', u'writeDisposition':
u'WRITE_TRUNCATE', u'allowLargeResults': True, u'destinationTable':
{u'projectId': u'api-project-617799710969', u'tableId': u'table',
u'datasetId': u'dataset'}}}, u'id': u'api-project-
617799710969:job_1UBxC6nXSBRRerkGNUNr6_DdyPU', u'selfLink':
u'https://www.googleapis.com/bigquery/v2/projects/api-project-617799710969/jobs/job_1UBxC6nXSBRRerkGNUNr6_DdyPU'}
Related
Not able to put tag column value on monday item
I am trying to use python to automation common Monday tasks. I am able to create an item in the board but the column (type=tag) is not updating.
I used this tutorial:
https://support.monday.com/hc/en-us/articles/360013483119-API-Quickstart-Tutorial-Python#
Here is my graphql code that I am executing:
query = 'mutation ($device: String!, $columnVals: JSON!) { create_item (board_id:<myboardid>, item_name:$device, column_values:$columnVals) { id } }'
vars = {'device': device,
'columnVals': json.dumps({
'cloud_name6': {'text': cloudname} # this is where i want to add a tag. cloud_name6 is id of the column.
})
}
data = {'query' : query, 'variables' : vars}
r = requests.post(url=apiUrl, json=data, headers=headers) print(r.json())
I have tried changing id to title as key in the Json string but no luck. I fetched the existing item and tried to add exact json string but still no luck. I also tried below json data without any luck
'columnVals': json.dumps({
'cloud_name6': cloudname
})
Any idea what's wrong with the query?
When creating or mutating tag columns via item queries, you need to send an array of ids of the tags ("tag_ids") that are relating to this item. You don't set or alter tag names via an item query.
Corrected Code
'columnVals': json.dumps({
'cloud_name6': {'tag_ids': [295026,295064]}
})
https://developer.monday.com/api-reference/docs/tags
Data Science Experience responds with an empty Hive table
From my Data Science Experience, I am able to make a connection to the Hive database in BigInsights and read the table schema. But Data Science Experience does not seem to be able to read the table contents as I get a count of zero! Here are some of my settings:
conf = (SparkConf().set("com.ibm.analytics.metadata.enabled","false"))
spark = SparkSession.builder.enableHiveSupport().getOrCreate()
dash = {
'jdbcurl': 'jdbc:hive2://nnnnnnnnnnn:10000/;ssl=true;',
'user': 'xxxxxxxxxx',
'password': 'xxxxxxxxx',
}
spark.conf
offers = spark.read.jdbc(dash['jdbcurl'],
table='offers',
properties={"user" : dash["user"],
"password" : dash["password"]})
offers.count() returns: 0
offers.show()
returns:
+-----------+----------+
|offers.name|offers.age|
+-----------+----------+
+-----------+----------+
Thanks.
Yes i was able to see same behaviour with hive jdbc connector.
I tried this python connector and it returned correct count.
https://datascience.ibm.com/docs/content/analyze-data/python_load.html
from ingest.Connectors import Connectors
`HiveloadOptions = { Connectors.Hive.HOST : 'bi-hadoop-prod-4222.bi.services.us-south.bluemix.net',
Connectors.Hive.PORT : '10000',
Connectors.Hive.SSL : True,
Connectors.Hive.DATABASE : 'default',
Connectors.Hive.USERNAME : 'charles',
Connectors.Hive.PASSWORD : 'march14march',
Connectors.Hive.SOURCE_TABLE_NAME : 'student'}
`
`HiveDF = sqlContext.read.format("com.ibm.spark.discover").options(**HiveloadOptions).load()`
HiveDF.printSchema()
HiveDF.show()
HiveDF.count()
Thanks,
Charles.
How to get top 10 videos from your channel with the views count included in the response
So I have a ouath web app that connects to youtube. I use the youtube analytics calls to get information like number of subscribers from my channel. But right now I try to make a top 10 videos from my channel with the views count for every video included in the response. I use this documentation:
Top videos for subscribed or unsubscribed viewers
My call looks like this:
$command = 'curl -H "Authorization: Bearer ' . $access_token . '" "https://www.googleapis.com/youtube/analytics/v1/reports?ids=channel%3D%3DMINE&start-date=' . date('Y-m-d', strtotime('-31 days')) . '&end-date=' . date('Y-m-d', strtotime('today')). '&metrics=views&dimensions=video&sort=views"';
But I get an error message as a response:
"The query is not supported. Check the documentation at https://developers.google.com/youtube/analytics/v1/available_reports for a list of supported queries."
I also tried this call with YoTube data API:
$videos_url = 'https://www.googleapis.com/youtube/v3/search?part=snippet&forMine=true&order=viewCount&type=video&access_token=' . $access_token;
But it provides this response:
["kind"]=>
string(26) "youtube#searchListResponse"
["etag"]=>
string(57) ""######""
["nextPageToken"]=>
string(112) "#######"
["pageInfo"]=>
object(stdClass)#267 (2) {
["totalResults"]=>
int(1)
["resultsPerPage"]=>
int(5)
}
["items"]=>
array(1) {
[0]=>
object(stdClass)#270 (4) {
["kind"]=>
string(20) "youtube#searchResult"
["etag"]=>
string(57) ""#####""
["id"]=>
object(stdClass)#269 (2) {
["kind"]=>
string(13) "youtube#video"
["videoId"]=>
string(11) "####"
}
["snippet"]=>
object(stdClass)#273 (6) {
["publishedAt"]=>
string(24) "2016-09-14T14:49:49.000Z"
["channelId"]=>
string(24) "#####"
["title"]=>
string(12) "My Slideshow"
["description"]=>
string(87) "I created this video with the YouTube Slideshow Creator (http://www.youtube.com/upload)"
This response provides no views count. I need for every video to get the views count as well.
Any ideas on how to accomplish this? Any help is welcomed! Thank you all for your time!
The issue with your youtube analytics api request is sort parameter. Try using value -views instead of views. You can double check the API link you provided. Also set the max-results
Example URL :
https://www.googleapis.com/youtube/analytics/v1/reports?ids=channel%3D%3Dmine&start-date=2016-05-01&end-date=2016-09-01&metrics=views&dimensions=video&max-results=10&sort=-views&key={YOUR_API_KEY}
Yodlee REST API - addItemForContentService1
Could you please provide a JSON sample of how you would post an addItemForContentService1 for a bank with a FieldInfoMultiFixed field. An example is Sallie Mae, content service ID #3805 Thanks
I wrote this in python. Replace some of the values such as cobrand_session_token, content_service_id with the actual values:
Most of the below is directly from this documentation.
payload = {"cobSessionToken": cobrand_session_token, "userSessionToken": session["user_session_token"], "contentServiceId": content_service_id, "shareCredentialsWithinSite": True, "startRefreshItemOnAddition": True, "credentialFields.enclosedType": "com.yodlee.common.FieldInfoMultiFixed",\
"credentialFields[0].valueIdentifier":"LOGIN",\
"credentialFields[0].displayName":"UserID",\
"credentialFields[0].isEditable":True,\
"credentialFields[0].isEscaped":False,\
"credentialFields[0].name":"LOGIN",\
"credentialFields[0].helpText":5112,\
"credentialFields[0].fieldType":"TEXT",\
"credentialFields[0].valueMask":"LOGIN_FIELD",\
"credentialFields[0].maxlength":40,\
"credentialFields[0].size":20,\
"credentialFields[1].valueIdentifier":"PASSWORD",\
"credentialFields[1].displayName":"Password",\
"credentialFields[1].isEditable":True,\
"credentialFields[1].isEscaped":False,\
"credentialFields[1].name":"PASSWORD",\
"credentialFields[1].helpText":4530,\
"credentialFields[1].fieldType":"IF_PASSWORD",\
"credentialFields[1].valueMask":"LOGIN_FIELD",\
"credentialFields[1].maxlength":40,\
"credentialFields[1].size":20, "credentialFields[0].value":"content_account_username", "credentialFields[1].value":"content_account_password"}
Error creating geography element using "STGeomFromText( 'LINESTRING(..." - SQL 2008
I am getting the following error when trying to create geography elements in sql 2008: NET Framework error occurred during execution of user-defined routine or aggregate "geography": System.ArgumentException: 24200: The specified input does not represent a valid geography instance. System.ArgumentException: Some example queries are: select geography::STGeomFromText( 'LINESTRING( 53.4707 -2.24758,53.4708 -2.25079,53.4733 -2.2597,53.4743 -2.24812,53.4747 -2.24119,53.4756 -2.26244,53.4765 -2.24278,53.4805 -2.28457,53.4909 -2.37892,53.5272 -2.36252,53.7159 -2.56899,53.7165 -2.56208,53.7183 -2.62195,53.7212 -2.55478,53.7439 -2.48239,53.7443 -2.48351,53.7446 -2.48767,53.7455 -2.48894,53.7464 -2.48376,51.5126 -0.22443,51.5126 -0.22455,51.5129 -0.22522,51.5148 -0.22595,51.515 -0.22565,51.5152 -0.22007,51.5153 -0.21982,51.5153 -0.22207,51.516 -0.21593,51.5207 -0.16548,51.5274 -0.13044,51.5287 -0.1195,51.5294 -0.11581,51.5314 -0.114,51.5319 -0.10629,51.546 -0.10225,51.5468 -0.09851,51.5615 -0.09948,51.5618 -0.09859,51.5619 -0.09874,51.5621 -0.09774,51.5622 -0.09793,51.5629 -0.074843,51.5635 -0.07319,51.5638 -0.07423,51.5655 -0.07308,51.5065 -0.23453,51.5098 -0.2346,51.5125 -0.22443,51.5126 -0.22455,51.5141 -0.30077,51.3824 0.219232,51.3832 0.22189,51.3871 0.19381,51.3924 0.19631,51.4505 0.2384,51.4561 0.24682,51.4905 0.26753,51.6112 -0.13568,51.614 -0.1423,51.6149 -0.1279,51.6159 -0.0959,51.6166 -0.08625,51.6827 -0.04306,51.6827 -0.0485,50.8295 -0.135709,50.8336 -0.1389,51.0842 -0.20043,51.2584 -0.12584,51.4889 -0.49628,51.4928 -0.41087,51.493 -0.41909,51.5134 -0.40288,51.5158 -0.40192,51.5169 -0.41292,51.517 -0.41706,51.5171 -0.41794,51.5175 -0.41693,51.5189 -0.41131,51.506 -3.5765,51.5071 -3.57572,51.508 -3.57682,51.5089 -3.57687,51.5094 -3.57552,51.5268 -3.57319,51.5319 -3.57145,51.533 -3.567,51.5537 -2.54326,52.2083 -2.15995,52.2133 -2.15969,52.3653 -2.23896,52.3797 -2.25068,52.3856 -2.24433,52.3856 -2.24637,51.3017 0.14978,51.3168 -0.55901,51.317 -0.55852,51.3179 -0.5572,51.318 -0.55734,51.3714 -0.5076,51.3876 0.50573,51.3881 0.50679,51.3884 0.50491,51.3885 0.505716,51.3951 0.49271,51.3957 0.49308,51.3984 0.45032,51.3998 0.43212,51.423 0.2361,51.4259 0.23944,51.4269 0.24394,51.3017 0.14978,51.3168 -0.55901,51.317 -0.55852,51.3179 -0.5572,51.318 -0.55734,51.3714 -0.5076,51.3876 0.50573,51.3881 0.50679,51.3884 0.50491,51.3885 0.505716,51.3951 0.49271,51.3957 0.49308,51.3984 0.45032,51.3998 0.43212,51.423 0.2361,51.4259 0.23944,51.4269 0.24394,52.2027 0.13072,52.2027 0.13139,52.2041 0.12733,52.2048 0.12558,52.2052 0.124187,52.2083 0.13738,52.2139 0.1106,52.2139 0.1237,52.2146 0.12279,52.2163 0.13467,52.2332 0.07004,52.2374 0.06243,52.3776 -0.25321,52.5366 -0.3223,53.3912 -1.06914,53.5818 -1.21172,53.6173 -1.23025,53.809 -1.34127,54.1533 -1.43958,54.4485 -1.66852,54.7766 -1.57715,54.7777 -1.57547,54.7777 -1.57631,54.7906 -1.53038,54.7934 -1.52982 )' , 4326) or select geography::STGeomFromText( 'LINESTRING( 53.4707 -2.24758,53.4708 -2.25079,53.4733 -2.2597,53.4743 -2.24812,53.4747 -2.24119,53.4756 -2.26244,53.4765 -2.24278,53.4805 -2.28457,53.4909 -2.37892,53.5272 -2.36252,53.7159 -2.56899,53.7165 -2.56208,53.7183 -2.62195,53.7212 -2.55478,53.7439 -2.48239,53.7443 -2.48351,53.7446 -2.48767,53.7455 -2.48894,53.7464 -2.48376,51.5126 -0.22443,51.5126 -0.22455,51.5129 -0.22522,51.5148 -0.22595,51.515 -0.22565,51.5152 -0.22007,51.5153 -0.21982,51.5153 -0.22207,51.516 -0.21593,51.5207 -0.16548,51.5274 -0.13044,51.5287 -0.1195,51.5294 -0.11581,51.5314 -0.114,51.5319 -0.10629,51.546 -0.10225,51.5468 -0.09851,51.5615 -0.09948,51.5618 -0.09859,51.5619 -0.09874,51.5621 -0.09774,51.5622 -0.09793,51.5629 -0.074843,51.5635 -0.07319,51.5638 -0.07423,51.5655 -0.07308,51.5065 -0.23453,51.5098 -0.2346,51.5125 -0.22443,51.5126 -0.22455,51.5141 -0.30077,51.3824 0.219232,51.3832 0.22189,51.3871 0.19381,51.3924 0.19631,51.4505 0.2384,51.4561 0.24682,51.4905 0.26753,51.6112 -0.13568,51.614 -0.1423,51.6149 -0.1279,51.6159 -0.0959,51.6166 -0.08625,51.6827 -0.04306,51.6827 -0.0485,50.8295 -0.135709,50.8336 -0.1389,51.0842 -0.20043,51.2584 -0.12584,51.4889 -0.49628,51.4928 -0.41087,51.493 -0.41909,51.5134 -0.40288,51.5158 -0.40192,51.5169 -0.41292,51.517 -0.41706,51.5171 -0.41794,51.5175 -0.41693,51.5189 -0.41131,51.506 -3.5765,51.5071 -3.57572,51.508 -3.57682,51.5089 -3.57687,51.5094 -3.57552,51.5268 -3.57319,51.5319 -3.57145,51.533 -3.567,51.5537 -2.54326,52.2083 -2.15995,52.2133 -2.15969,52.3653 -2.23896,52.3797 -2.25068,52.3856 -2.24433,52.3856 -2.24637,51.3017 0.14978,51.3168 -0.55901,51.317 -0.55852,51.3179 -0.5572,51.318 -0.55734,51.3714 -0.5076,51.3876 0.50573,51.3881 0.50679,51.3884 0.50491,51.3885 0.505716,51.3951 0.49271,51.3957 0.49308,51.3984 0.45032,51.3998 0.43212,51.423 0.2361,51.4259 0.23944,51.4269 0.24394,51.3017 0.14978,51.3168 -0.55901,51.317 -0.55852,51.3179 -0.5572,51.318 -0.55734,51.3714 -0.5076,51.3876 0.50573,51.3881 0.50679,51.3884 0.50491,51.3885 0.505716,51.3951 0.49271,51.3957 0.49308,51.3984 0.45032,51.3998 0.43212,51.423 0.2361,51.4259 0.23944,51.4269 0.24394,52.2027 0.13072,52.2027 0.13139,52.2041 0.12733,52.2048 0.12558,52.2052 0.124187,52.2083 0.13738,52.2139 0.1106,52.2139 0.1237,52.2146 0.12279,52.2163 0.13467,52.2332 0.07004,52.2374 0.06243,52.3776 -0.25321,52.5366 -0.3223,53.3912 -1.06914,53.5818 -1.21172,53.6173 -1.23025,53.809 -1.34127,54.1533 -1.43958,54.4485 -1.66852,54.7766 -1.57715,54.7777 -1.57547,54.7777 -1.57631,54.7906 -1.53038,54.7934 -1.52982,53.3615 -2.2943,53.3616 -2.29438,53.3619 -2.29431,53.3626 -2.29606,53.363 -2.29748,53.3683 -2.31356,53.3687 -2.31296,53.3967 -2.26766,53.3984 -2.22994,53.481 -2.05105,53.4821 -2.05214,53.4821 -2.11473,53.4827 -2.10573,53.4828 -2.05008,53.4858 -2.05878,53.7633 -2.70157,53.7636 -2.70038,53.7672 -2.70313,53.7952 -2.98557,53.8002 -2.72228,53.8011 -2.71739,53.804 -3.02696,53.8139 -3.05028,53.815 -3.04473,53.8163 -3.05226,53.4707 -2.24758,53.4708 -2.25079,53.4733 -2.2597,53.4743 -2.24812,53.4747 -2.24119,53.4756 -2.26244,53.4765 -2.24278,53.4805 -2.28457,53.4909 -2.37892,53.5272 -2.36252,53.7364 -2.64886,53.7952 -2.98557,53.804 -3.02696,53.806 -2.70331,53.8139 -3.05028,53.815 -3.04473,53.8163 -3.05226,51.401 -0.5297,51.4052 -0.34152,51.4111 -0.300524,51.4126 -0.30176,51.4164 -0.42419,51.4191 -0.41833,51.7152 -0.3928,52.8399 -1.29605,52.8429 -1.29658,52.9216 -1.16451,52.9265 -1.16722,52.9488 -1.1511,52.9489 -1.15012,52.9493 -1.14992,52.9504 -1.14983,52.9509 -1.15128,52.9513 -1.1503,52.9514 -1.15141,52.9517 -1.15273,52.9521 -1.15199,51.502 -0.15932,51.5024 -0.1555,51.5033 -0.14984,51.5033 -0.15075,51.5042 -0.1511,51.5132 -0.16003,51.5134 -0.15871,51.5227 -0.17393,51.5685 -0.2291,51.5712 -0.23129,51.5719 -0.22982,51.573 -0.228,51.5731 -0.22961,51.5763 -0.23571,52.3953 -1.17295,52.6728 -2.07963,53.4061 -2.8963,53.407 -2.90128,53.4075 -2.90136,53.4078 -2.97954,53.4078 -2.9802,53.4078 -2.98228,53.408 -2.98106,53.41 -2.97854,53.4103 -2.90155,53.4103 -2.97767,53.4112 -2.97723,53.4133 -2.96398,53.4143 -2.96216,53.4237 -2.57435,53.4273 -2.91804,51.5033 -0.149834,51.5042 -0.1511,51.5044 -0.14749,51.5051 -0.15035,51.5051 -0.15089,51.5052 -0.15032,51.5056 -0.14872,51.5126 -0.15886,51.5132 -0.1607,51.5179 -0.16725,51.5226 -0.17378,51.5685 -0.22934,51.5686 -0.22953,51.5709 -0.23185,51.5719 -0.23053,52.4003 -1.17515,52.4768 -1.68081,52.6655 -2.06869,53.4062 -2.95647,53.4064 -2.89868,53.4064 -2.95565,53.407 -2.95622,53.4072 -2.95609,53.4075 -2.97961,53.4076 -2.98199,53.4076 -2.98408,53.4084 -2.97404,53.4087 -2.97886,53.4087 -2.98279,53.4092 -2.97452,53.4098 -2.91679,53.4213 -2.54907,51.4871 0.2647,51.4878 0.26815,51.4901 0.26826,51.4931 0.26921,51.4986 0.26975,51.4988 0.25643,51.499 0.25929,51.5001 0.25941,51.5002 0.25852,51.5019 0.26817,51.6053 0.26623,51.6087 0.26389,51.6108 0.26624,51.8917 0.8989,51.8918 0.89627,51.8918 0.89984,51.8919 0.89838,51.892 0.900363,51.8924 0.89627,51.8926 0.86479,51.8926 0.89413,51.8926 0.89605,51.8953 0.8574,51.8966 0.89092,51.5042 -0.08496,51.5051 -0.0844,51.5061 -0.08801,51.5063 -0.08826,51.5108 -0.08656,51.5114 -0.07264,51.5115 -0.07297,51.5119 -0.06908,51.5138 -0.07593,51.5152 -0.07174,51.5161 -0.08162,51.5266 -0.02797,51.5288 -0.01608,51.53 -0.01417,51.5311 -0.01715,51.5927 0.21329,51.5929 0.20981,51.8917 0.8989,51.8918 0.89627,51.8918 0.89984,51.8919 0.89838,51.892 0.900363,51.8924 0.89627,51.8926 0.86479,51.8926 0.89413,51.8926 0.89605,51.8953 0.8574,51.8966 0.89092 )' , 4326) I have many similar queries than run just fine. Can anyone tell me why these particular ones fail? I'm at a loss. Thanks in advance.
I cannot see this from your data, but it seems that you are running into this problem: The LineString instance cannot overlap itself over an interval. It might be useful for you to use the MakeValid() function to correct this.
It was indeed the line overlapping itself. I managed to create a valid geography object using http://sqlspatialtools.codeplex.com/ Great article here: http://blogs.msdn.com/b/edkatibah/archive/2009/06/05/working-with-invalid-data-and-the-sql-server-2008-geography-data-type-part-2.aspx