Can BigQuery report mismatched the schema field? - google-bigquery

When I upsert a row that mismatches schema I get a PartialFailureError along with a message, e.g.:
[ { errors:
[ { message: 'Repeated record added outside of an array.',
reason: 'invalid' } ],
...
]
However for large rows this isn't sufficient, because I have no idea which field is the one creating the error. The bq command does report the malformed field.
Is there either a way to configure or access name of the offending field, or can this be added to the API endpoint?

Please see this Github Issue: https://github.com/googleapis/nodejs-bigquery/issues/70 . Apparently node.js client library is not getting the location field from the API so it's not able to return it to the caller.
Workaround that worked for me: I copied the JSON payload to my Postman client and manually sent a request to REST API (let me know if you need more details of how to do it).

Related

UPDATE and REVISE rest api is not working in vTiger CRM cloud service

So basically I need to use the update/revise Rest API to update the fields in the vTiger CRM.
But when I am using the rest API (link : https://help.vtiger.com/article/147111249-Rest-API-Manual) to update the fields, I am getting the error as "400 Unsupported operations: The request cannot be fulfilled due to bad syntax."
My api : endpoint/reviseelement=convert_into_json_string({id:5x369, potentialname:'demo2'})
Also apart from this, I had used the SQL query Rest API, to update the record in the modules, but it is also giving me the same error as: "400 Unsupported operations: The request cannot be fulfilled due to bad syntax."
My Api query : endpoint/query?query=UPDATE Potentials SET potentialname = 'demo2 where id = 5x369;
Also by using the webservice(https://community.vtiger.com/help/vtigercrm/developers/third-party-app-integration.html) rest api I am getting error like : "Permission to perform the operations is denied for id: ".
So how can I use the update api. Can anyone please help?
For web services API (https://community.vtiger.com/help/vtigercrm/developers/third-party-app-integration.html),
When you pass the payload data, make sure you're passing the x-www-form-urlencoded form of data.
Also, make sure you're passing 'element' argument with value like this:
{"id": "10x11471458", "leadsource": "Facebook"}.
if you're still facing this issue, you can share more details and I'll be happy to help you.
If you still need to do this the following should work.
Method: POST
End Point: /revise
Headers: "Authorization: Basic YOUR_TOKEN"
Body:
{
"element": {
"id":"5x369",
"potentialname": "demo2"
}
}
reference: https://www.vtiger.com/docs/rest-api-for-vtiger#/Revise

How do we filter the entities which is not start with "msdn" using MSDynamics Web API

I want to get all entities which are the name not start the prefix as 'msdn' from ms dynamics.
I tried the below APIs, got the error.
GET /api/data/v9.1/EntityDefinitions?$select=LogicalName&$filter=not startswith(LogicalName,%27msdn%27)
Response :
{
"error":
{"code":"0x0",
"message":"The \"startswith\" function isn't supported for Metadata Entities."
}
}
I referred https://learn.microsoft.com/en-us/powerapps/developer/common-data-service/webapi/query-data-web-api#standard-query-functions
I have checked that in one of my environment as well. What you require is not possible.
You will have to go 2 steps.
Retrieve all entities and then filter them out in your local program may it be JavaScript/C# or Json filtering/Power automate or something.

Invalid request body error when sending json string as data to an external api using CL_HTTP_CLIENT

We are facing an issue while sending json data to an external api using CL_HTTP_CLIENT.
The JSON data is produced using '/ui2/cl_json=>serialize( data = ls_body compress = abap_true pretty_name = /ui2/cl_json=>pretty_mode-camel_case )' .
when sending this JSON as data the the external api returns status 400 with response as
{ "errorCode": "INVALID_REQUEST_BODY", "message": "The request body is missing or improperly formatted. Unexpected character encountered while parsing value: \u001f. Path '', line 0, position 0." } .
we also stringyfied this JSON Data in backend as it might be due to parsing error but it didnt work.
The same stringyfied data tried to send through browser console using ajax and it did worked without any issue.
could any any one tell us how to handle this json object and send this to external api using CL_HTTP_CLIENT.
Note : JSON STRING is deeply nested .
Thanks in advance..
You can use request catcher service for getting SAP output.
Then check your output has valid json.
Check external api with rest tool like postman or SoapUI. Every developer not track guidliness may be external api has limitations.
The issue was with the unicodes in the string.
these were not accepted by the external api so removed from the string and sent to api and it did worked.
Thanks for You suggestion.

How to return both data field and errors field with Graphql-kotlin query

I am working on one server that resolve GraphQL queries. I used the graphql-kotlin library: https://github.com/ExpediaDotCom/graphql-kotlin.
I defined three resolve functions(could be viewed as three fields): getxxx(arguments...), getTarget(arguments...) and getSource(arguments...).
The problem is if one of my queried field failed, I will only get the 'errors' field. All other successful executed results(data) are dropped.
If I try to catch the exception for the failed field, then I will not get the 'error' field at last.
This is the image that had exception and dropped all fetched data:
The objective is returning both the successful fetched data in 'data' field and error message for failed field in 'errors' field.
I have checked this: How to return both error and data in a graphql resolver?
If I set a field that throw one error or exception intentionally, I will only get 'errors' field at last(Like the picture above).
In addition, many websites like this:https://itnext.io/the-definitive-guide-to-handling-graphql-errors-e0c58b52b5e1 suggest we should return partial result and error message, but in my case if I meet exception, only error message would be returned.
In our example application we have two top level queries
{
onlyCake(msg: "cake")
generateNumber
}
If I run this code this will return a random number and onlyCake returns the string <3. However if I modify the input of onlyCake to something that is not the string cake I will get an error. This is expected because the field onlyCake has a directive to implement this behavior.
https://github.com/ExpediaDotCom/graphql-kotlin/blob/2961b64d6e4cceb4034aec198d667e5f965decd2/example/src/main/kotlin/com/expedia/graphql/sample/directives/CakeOnlyDirectiveWiring.kt#L18-L19
The question is though, if I want to return the data still for generateNumber can I do that and have both errors and data in the response?
The answer is it's possible but not with the current schema. The issue is that our schema for onlyCake and generateNumber are both non-nullable fields. So as a client if I see that there is a data field I should expect to see both there otherwise I would have a parsing issue in my code. This is why we can't have this behaviour with the schema as is. If you want to implement this behaviour, the schema developer needs to decide where they can return null for some response and modify the errors field appropriately with a DataFetcherExceptionHandler
Or the other option is that we support the behaviour in graphql-java to return a DataFetcherResult<T> instead of just T from the kotlin functions. See the section Returning data and errors here: https://www.graphql-java.com/documentation/v13/execution/
I have created an issue to continue this discussion with other team members: https://github.com/ExpediaDotCom/graphql-kotlin/issues/244

BigQuery Command Line Tool: get error details

one of my jobs keeps failing and when I looked into why (by requesting job details) I get the following output:
status": {
"errorResult": {
"location": "gs://sf_auto/Datastore Mapper modules.models.userData/15716706166748C8426AD/output-46",
"message": "JSON table encountered too many errors, giving up. Rows: 1; errors: 1.",
"reason": "invalid"
},
"errors": [
{
"location": "gs://sf_auto/Datastore Mapper modules.models.userData/15716706166748C8426AD/output-46",
"message": "JSON table encountered too many errors, giving up. Rows: 1; errors: 1.",
"reason": "invalid"
}
],
"state": "DONE"
Problem is, it doesn't help at all, and I need more details. Is there anyway to understand which column or attribute caused the failings? Is there any way to get more information?
Edit Additional Details
We're running a map reduce job on appengine to transfer our datastore from appengine to BigQuery
The files are stored on Google Cloud Store
It's creating a brand new table instead of adding to an existing one
Update #2
I played around with the query trying lots of things as well as adjusting the scheme and i've narrowed down the problem to the uuid. For some reason this type of data messes everything up:
"uuid": "XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX"
The schema defines it as a String
OK, after loads of debugging I found the error... in the JSON Newline file we had two attributes that were similar:
uuid: "XXX..."
uuId: "XXX..."
This has been there for a while so I think some change within bigquery started to require that keys be unique regardless of capitalization. Will test so more and confirm!
A recent change made loads of JSON data case insensitive in field names similar to be consistent with how SQL queries treat field names. I have opened a work item to track the improvement of error message for this case.