We use terraform to deploy BigQuery objects (datasets, tables, routines etc..) to region europe-west2 in GCP. We do this many many times a day and all of a sudden at "2021-08-18T21:15:44.033910202Z" our deployments starting failing when attempting to deploy BigQuery routines. They are all failing with errors of the form:
status: {
code: 3
message: "Unknown option: description"
}
Here is the first log message I can find pertaining to this error (I have redacted project names):
"protoPayload": {
"#type": "type.googleapis.com/google.cloud.audit.AuditLog",
"status": {
"code": 3,
"message": "Unknown option: description"
},
"authenticationInfo": {
"principalEmail": "deployer-dev#myadminproject.iam.gserviceaccount.com",
"serviceAccountDelegationInfo": [
{
"firstPartyPrincipal": {
"principalEmail": "deployer-dev#myadminproject.iam.gserviceaccount.com"
}
}
]
},
"requestMetadata": {
"callerIp": "10.51.0.116",
"callerSuppliedUserAgent": "Terraform/0.14.7 (+https://www.terraform.io) Terraform-Plugin-SDK/2.5.0 terraform-provider-google/3.69.0,gzip(gfe)",
"callerNetwork": "//compute.googleapis.com/projects/myadminproject/global/networks/__unknown__",
"requestAttributes": {},
"destinationAttributes": {}
},
"serviceName": "bigquery.googleapis.com",
"methodName": "google.cloud.bigquery.v2.RoutineService.InsertRoutine",
"authorizationInfo": [
{
"resource": "projects/myproject/datasets/p00003818_dp_model",
"permission": "bigquery.routines.create",
"granted": true,
"resourceAttributes": {}
}
],
"resourceName": "projects/myproject/datasets/p00003818_dp_model/routines/UserProfile_Events_AllCarData_Deployment",
"metadata": {
"routineCreation": {
"routine": {
"routineName": "projects/myproject/datasets/p00003818_dp_model/routines/UserProfile_Events_AllCarData_Deployment"
},
"reason": "ROUTINE_INSERT_REQUEST"
},
"#type": "type.googleapis.com/google.cloud.audit.BigQueryAuditMetadata"
}
},
"insertId": "ak27xdbke",
"resource": {
"type": "bigquery_dataset",
"labels": {
"dataset_id": "p00003818_dp_model",
"project_id": "myproject"
}
},
"timestamp": "2021-08-18T21:15:43.109609Z",
"severity": "ERROR",
"logName": "projects/myproject/logs/cloudaudit.googleapis.com%2Factivity",
"receiveTimestamp": "2021-08-18T21:15:44.033910202Z"
}
The fact that this occurred without any changes by ourselves indicates that this is a problem at the Google end. I also observe that whilst we witnessed this in a few projects it occurred first in one project and then a few minutes later in another - that may or may not be helpful information.
Posting here in case anyone else hits this problem and also hoping it might catch the attention of a googler.
UPDATE! I have reproduced the pproblem using the REST API https://cloud.google.com/bigquery/docs/reference/rest/v2/routines/insert
I have entered a payload that does not include a description and that successfully creates a routine:
However, if I include a description which, as this screenshot indicates, is a valid parameter:
then the request fails:
I'm setting up a new asp.net core 3.1 project, exposing a RESTful API in FHIR format, which is a format extending JSON.
I'd like to make a simple UI available with SwaggerUI. That's where I've got an issue that errors are being shown.
When launching SwaggerUI, there are no errors. When expanding the single available POST request, these errors show up:
Full list:
Resolver error at paths./Bundle/$analyze.post.requestBody.content.text/json.schema.properties.parameter.items.properties.children.items.properties.children.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.requestBody.content.text/json.schema.properties.parameter.items.properties.extension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.requestBody.content.text/json.schema.properties.idElement.properties.extension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.requestBody.content.text/json.schema.properties.idElement.properties.children.items.properties.children.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.requestBody.content.text/json.schema.properties.meta.properties.children.items.properties.children.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.requestBody.content.text/json.schema.properties.meta.properties.extension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.requestBody.content.text/json.schema.properties.implicitRulesElement.properties.extension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.requestBody.content.text/json.schema.properties.implicitRulesElement.properties.children.items.properties.children.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.requestBody.content.text/json.schema.properties.languageElement.properties.extension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.requestBody.content.text/json.schema.properties.languageElement.properties.children.items.properties.children.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.text/plain.schema.properties.children.items.properties.children.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.application/json.schema.properties.identifier.properties.assigner.properties.identifier.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.application/json.schema.properties.identifier.properties.children.items.properties.children.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.application/json.schema.properties.identifier.properties.extension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.application/json.schema.properties.typeElement.properties.extension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.application/json.schema.properties.typeElement.properties.children.items.properties.children.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.application/json.schema.properties.timestampElement.properties.extension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.application/json.schema.properties.timestampElement.properties.children.items.properties.children.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.application/json.schema.properties.totalElement.properties.extension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.application/json.schema.properties.totalElement.properties.children.items.properties.children.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.application/json.schema.properties.link.items.properties.children.items.properties.children.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.application/json.schema.properties.link.items.properties.modifierExtension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.application/json.schema.properties.link.items.properties.extension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.application/json.schema.properties.entry.items.properties.children.items.properties.children.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.application/json.schema.properties.entry.items.properties.modifierExtension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.application/json.schema.properties.entry.items.properties.extension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.application/json.schema.properties.signature.properties.who.properties.identifier.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.application/json.schema.properties.signature.properties.onBehalfOf.properties.identifier.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.application/json.schema.properties.signature.properties.children.items.properties.children.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.application/json.schema.properties.signature.properties.extension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.requestBody.content.application/json.schema.properties.parameter.items.properties.nameElement.properties.extension.items.properties.children.items.properties.children.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.requestBody.content.application/json.schema.properties.parameter.items.properties.nameElement.properties.extension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.requestBody.content.application/json.schema.properties.parameter.items.properties.nameElement.properties.children.items.properties.children.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.requestBody.content.application/json.schema.properties.parameter.items.properties.value.properties.children.items.properties.children.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.requestBody.content.application/json.schema.properties.parameter.items.properties.value.properties.extension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.requestBody.content.application/json.schema.properties.parameter.items.properties.resource.properties.children.items.properties.children.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.requestBody.content.application/json.schema.properties.parameter.items.properties.resource.properties.idElement.properties.extension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.requestBody.content.application/json.schema.properties.parameter.items.properties.part.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.requestBody.content.application/json.schema.properties.parameter.items.properties.modifierExtension.items.properties.children.items.properties.children.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.requestBody.content.application/json.schema.properties.children.items.properties.children.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.requestBody.content.application/json.schema.properties.meta.properties.profileElement.items.properties.extension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.requestBody.content.application/json.schema.properties.meta.properties.profileElement.items.properties.children.items.properties.children.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.requestBody.content.application/json.schema.properties.meta.properties.security.items.properties.children.items.properties.children.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.requestBody.content.application/json.schema.properties.meta.properties.security.items.properties.extension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.requestBody.content.text/json.schema.properties.parameter.items.properties.modifierExtension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.requestBody.content.text/json.schema.properties.meta.properties.security.items.properties.userSelectedElement.properties.extension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.requestBody.content.text/json.schema.properties.meta.properties.security.items.properties.userSelectedElement.properties.children.items.properties.children.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.text/plain.schema.properties.identifier.properties.useElement.properties.extension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.text/plain.schema.properties.identifier.properties.useElement.properties.children.items.properties.children.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.text/plain.schema.properties.identifier.properties.type.properties.children.items.properties.children.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.text/plain.schema.properties.identifier.properties.type.properties.extension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.text/plain.schema.properties.identifier.properties.period.properties.children.items.properties.children.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.text/plain.schema.properties.identifier.properties.period.properties.extension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.text/plain.schema.properties.identifier.properties.assigner.properties.identifier.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.text/plain.schema.properties.identifier.properties.assigner.properties.children.items.properties.children.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.text/plain.schema.properties.identifier.properties.assigner.properties.extension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.text/plain.schema.properties.entry.items.properties.search.properties.children.items.properties.children.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.text/plain.schema.properties.entry.items.properties.search.properties.modifierExtension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.text/plain.schema.properties.entry.items.properties.search.properties.extension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.text/plain.schema.properties.entry.items.properties.request.properties.children.items.properties.children.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.text/plain.schema.properties.entry.items.properties.request.properties.modifierExtension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.text/plain.schema.properties.entry.items.properties.request.properties.extension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.text/plain.schema.properties.entry.items.properties.response.properties.children.items.properties.children.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.text/plain.schema.properties.entry.items.properties.response.properties.modifierExtension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.text/plain.schema.properties.entry.items.properties.response.properties.extension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.text/plain.schema.properties.signature.properties.dataElement.properties.extension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.text/plain.schema.properties.signature.properties.dataElement.properties.children.items.properties.children.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.application/json.schema.properties.identifier.properties.period.properties.startElement.properties.extension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.application/json.schema.properties.identifier.properties.period.properties.startElement.properties.children.items.properties.children.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.application/json.schema.properties.entry.items.properties.search.properties.modeElement.properties.extension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.application/json.schema.properties.entry.items.properties.search.properties.modeElement.properties.children.items.properties.children.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.application/json.schema.properties.entry.items.properties.search.properties.scoreElement.properties.extension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.application/json.schema.properties.entry.items.properties.search.properties.scoreElement.properties.children.items.properties.children.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.application/json.schema.properties.entry.items.properties.request.properties.methodElement.properties.extension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.application/json.schema.properties.entry.items.properties.request.properties.methodElement.properties.children.items.properties.children.items.$ref
Could not resolve reference:
The controller method:
[HttpPost("$analyze")]
public async Task<ActionResult<Bundle>> Analyze([FromBody]Parameters parameters)
{
[...]
return Ok();
}
The input (Parameters) and output (Bundle) parameters are objects contained in Hl7.Fhir.R4 (nuget, source)
Following packages are being used:
swagger.json (truncated, as it was too long to be posted):
{
"openapi": "3.0.1",
"info": {
"title": "Analyzer",
"version": "v1"
},
"paths": {
"/Bundle/$analyze": {
"post": {
"tags": [
"Bundle"
],
"parameters": [],
"requestBody": {
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/Parameters"
}
},
"text/json": {
"schema": {
"$ref": "#/components/schemas/Parameters"
}
},
"application/*+json": {
"schema": {
"$ref": "#/components/schemas/Parameters"
}
}
}
},
"responses": {
"200": {
"description": "Success",
"content": {
"text/plain": {
"schema": {
"$ref": "#/components/schemas/Bundle"
}
},
"application/json": {
"schema": {
"$ref": "#/components/schemas/Bundle"
}
},
"text/json": {
"schema": {
"$ref": "#/components/schemas/Bundle"
}
}
}
}
}
}
}
},
"components": {
"schemas": {
"ResourceType": {
"enum": [
"Account",
"ActivityDefinition",
"AdverseEvent",
"AllergyIntolerance",
"Appointment",
"AppointmentResponse",
"AuditEvent",
"Basic",
"Binary",
"BiologicallyDerivedProduct",
"BodyStructure",
"Bundle",
"CapabilityStatement",
"CarePlan",
"CareTeam",
"CatalogEntry",
"ChargeItem",
"ChargeItemDefinition",
"Claim",
"ClaimResponse",
"ClinicalImpression",
"CodeSystem",
"Communication",
"CommunicationRequest",
"CompartmentDefinition",
"Composition",
"ConceptMap",
"Condition",
"Consent",
"Contract",
"Coverage",
"CoverageEligibilityRequest",
"CoverageEligibilityResponse",
"DetectedIssue",
"Device",
"DeviceDefinition",
"DeviceMetric",
"DeviceRequest",
"DeviceUseStatement",
"DiagnosticReport",
"DocumentManifest",
"DocumentReference",
"DomainResource",
"EffectEvidenceSynthesis",
"Encounter",
[...]
],
"type": "string"
},
"ElementValue": {
"type": "object"
},
"Base": {
"type": "object",
"properties": {
"typeName": {
"type": "string",
"nullable": true,
"readOnly": true
},
"children": {
"type": "array",
"items": {
"$ref": "#/components/schemas/Base"
},
"nullable": true,
"readOnly": true
},
"namedChildren": {
"type": "array",
"items": {
"$ref": "#/components/schemas/ElementValue"
},
"nullable": true,
"readOnly": true
},
"userData": {
"type": "object",
"additionalProperties": {
"type": "object"
},
"nullable": true,
"readOnly": true,
"deprecated": true
}
}
},
"Element": {
"type": "object",
"properties": {
"typeName": {
"type": "string",
"nullable": true,
"readOnly": true
},
"children": {
"type": "array",
"items": {
"$ref": "#/components/schemas/Base"
},
"nullable": true,
"readOnly": true
},
"namedChildren": {
"type": "array",
"items": {
"$ref": "#/components/schemas/ElementValue"
},
"nullable": true,
"readOnly": true
},
"elementId": {
"type": "string",
"nullable": true
},
"extension": {
"type": "array",
"items": {
"$ref": "#/components/schemas/Extension"
},
"nullable": true
},
"userData": {
"type": "object",
"additionalProperties": {
"type": "object"
},
"nullable": true,
"readOnly": true,
"deprecated": true
}
}
},
"Extension": {
"type": "object",
"properties": {
"typeName": {
"type": "string",
"nullable": true,
"readOnly": true
},
"url": {
"type": "string",
"nullable": true
},
"value": {
"$ref": "#/components/schemas/Element"
},
"children": {
"type": "array",
"items": {
"$ref": "#/components/schemas/Base"
},
"nullable": true,
"readOnly": true
},
"namedChildren": {
"type": "array",
"items": {
"$ref": "#/components/schemas/ElementValue"
},
"nullable": true,
"readOnly": true
},
"elementId": {
"type": "string",
"nullable": true
},
"extension": {
"type": "array",
"items": {
"$ref": "#/components/schemas/Extension"
},
"nullable": true
},
"userData": {
"type": "object",
"additionalProperties": {
"type": "object"
},
"nullable": true,
"readOnly": true,
"deprecated": true
}
}
},
[...]
When testing the function, everything works fine.
What is SwaggerUI trying to tell me and how can I fix these errors? Any hint would be appreciated
This issue resolved itself with the latest swashbuckle version
The call of UseInlineDefinitionsForEnums helped to eliminate similar Resolver error at paths...Could not resolve reference:... in my case.
The error was:
Errors
Hide
Resolver error at paths./api/v1/Account/IsTenantAvailable.post.responses.200.content.text/plain.schema.properties.state.$ref
Could not resolve reference: Could not resolve pointer: /components/schemas/TenantAvailabilityState does not exist in document
Resolver error at paths./api/v1/Account/IsTenantAvailable.post.responses.200.content.application/json.schema.properties.state.$ref
Could not resolve reference: Could not resolve pointer: /components/schemas/TenantAvailabilityState does not exist in document
Resolver error at paths./api/v1/Account/IsTenantAvailable.post.responses.200.content.text/json.schema.properties.state.$ref
Could not resolve reference: Could not resolve pointer: /components/schemas/TenantAvailabilityState does not exist in document
The introduced call was:
using Microsoft.Extensions.DependencyInjection;
public class Startup
{
// ...
public IServiceProvider ConfigureServices(IServiceCollection services)
{
// ...
services.AddSwaggerGen(options =>
{
// ...
// calling SwaggerGenOptionsExtensions.UseInlineDefinitionsForEnums(Swashbuckle.AspNetCore.SwaggerGen.SwaggerGenOptions) instead of some UseReferencedDefinitionsForEnums
options.UseInlineDefinitionsForEnums(); // options.UseReferencedDefinitionsForEnums();
// ...
}
// ...
}
}
The call was inserted instead of commented out call of UseReferencedDefinitionsForEnums. It was commented out because of upgrading aspnetcore to 3.1 in my solution. The version that worked with 3.1 just did not contain this method any more, without any note about what should be called instead.
It is clear that both calls are related to enums. But I have no idea what exactly and why do they do, if they are for the same purpose and if they have any undesired side effects. But it just worked in context of eliminating this red plate in the screenshot on pressing "Try it out" on many of my endpoints.
I faced the issue on the version 5.6.3 of Swashbuckle.AspNetCore. Upgrading to the latest 6.1.1 did not make any help.
I tried to call https://api.ebay.com/sell/inventory/v1/bulk_migrate_listing, then response code is 2003
My request is :
{
"requests": [
{
"listingId": "160009220563"
}
]
}
Response body like this :
{
"errors": [
{
"errorId": 2003,
"domain": "ACCESS",
"category": "APPLICATION",
"message": "Internal error",
"longMessage": "There was a problem with an eBay internal system or process. Contact eBay developer support for assistance",
"parameters": [
{
"name": "reason",
"value": "Failed to transform underlying error response, see logs."
}
]
}
]
}
I am trying to insert a job through HTTP Post request, but i am getting Invalid path error.
My request body is as follows:
{
"configuration": {
"load": {
"sourceUris": [
"gs://onianalytics/PersData.csv"
],
"schema": {
"fields": [
{
"name": "Name",
"type": "STRING"
},
{
"name": "Age",
"type": "INTEGER"
}
]
},
"destinationTable": {
"datasetId": "Test_Dataset",
"projectId": "lithe-anvil-404",
"tableId": "tb_test_Pers"
}
}
},
"jobReference": {
"jobId": "10",
"projectId": "lithe-anvil-404"
}
}
For the sourceuri parameter, I am passing "gs://onianalytics/PersData.csv", where onianalytics is my bucket name and PersData.csv is my csv file (from which I want to upload data into google bigquery).
I am getting below response:
"status": {
"state": "DONE",
"errorResult": {
"reason": "invalid",
"message": "Invalid path: gs://onianalytics/PersData.csv"
},
"errors": [
{
"reason": "invalid",
"message": "Invalid path: gs://onianalytics/PersData.csv"
}
]
},
"statistics": {
"creationTime": "1387276603674",
"startTime": "1387276603751",
"endTime": "1387276603751"
}
}
My bucket is under the same projectid which has the BigQuery service activated. Also, I have Google Cloud Storage enabled under APIs and Auth. Following scopes are added while authenticating:
googleapis.com/auth/bigquery, googleapis.com/auth/cloud-platform, googleapis.com/auth/devstorage.full_control,googleapis.com/auth/devstorage.read_only,googleapis.com/auth/devstorage.read_write
I am inserting this job by "Try it!" link which is available on developers.google.com/bigquery/docs/reference/v2/jobs/insert.
In fact I am able to create buckets and objects in goggle cloud storage through APIs. But when i try to insert job from the uploaded object (which is a csv file), i got "Invalid Path" error. Can anyone please help me to identify why this error is occurring?
The error I get when trying the code above is "Not found: URI gs://onianalytics/PersData.csv".
I'm wondering if instead of /onianalytics/ you had a different path with invalid characters?
I am trying to insert a job through HTTP Post request, but i am getting Invalid path error.
My request body is as follows:
{
"configuration": {
"load": {
"sourceUris": [
"gs://onianalytics/PersData.csv"
],
"schema": {
"fields": [
{
"name": "Name",
"type": "STRING"
},
{
"name": "Age",
"type": "INTEGER"
}
]
},
"destinationTable": {
"datasetId": "Test_Dataset",
"projectId": "lithe-anvil-404",
"tableId": "tb_test_Pers"
}
}
},
"jobReference": {
"jobId": "10",
"projectId": "lithe-anvil-404"
}
}
For the sourceuri parameter, I am passing "gs://onianalytics/PersData.csv", where onianalytics is my bucket name and PersData.csv is my csv file (from which I want to upload data into google bigquery).
I am getting below response:
"status": {
"state": "DONE",
"errorResult": {
"reason": "invalid",
"message": "Invalid path: gs://onianalytics/PersData.csv"
},
"errors": [
{
"reason": "invalid",
"message": "Invalid path: gs://onianalytics/PersData.csv"
}
]
},
"statistics": {
"creationTime": "1387276603674",
"startTime": "1387276603751",
"endTime": "1387276603751"
}
}
Please explain why this error is occurring?
Is your bucket under the same projectId that has the BigQuery service activated and you requested tokens with? If not, have you tried enabling read/write access for that project?