Swagger Resolver error, Could not resolve reference: - asp.net-core

I'm setting up a new asp.net core 3.1 project, exposing a RESTful API in FHIR format, which is a format extending JSON.
I'd like to make a simple UI available with SwaggerUI. That's where I've got an issue that errors are being shown.
When launching SwaggerUI, there are no errors. When expanding the single available POST request, these errors show up:
Full list:
Resolver error at paths./Bundle/$analyze.post.requestBody.content.text/json.schema.properties.parameter.items.properties.children.items.properties.children.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.requestBody.content.text/json.schema.properties.parameter.items.properties.extension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.requestBody.content.text/json.schema.properties.idElement.properties.extension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.requestBody.content.text/json.schema.properties.idElement.properties.children.items.properties.children.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.requestBody.content.text/json.schema.properties.meta.properties.children.items.properties.children.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.requestBody.content.text/json.schema.properties.meta.properties.extension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.requestBody.content.text/json.schema.properties.implicitRulesElement.properties.extension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.requestBody.content.text/json.schema.properties.implicitRulesElement.properties.children.items.properties.children.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.requestBody.content.text/json.schema.properties.languageElement.properties.extension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.requestBody.content.text/json.schema.properties.languageElement.properties.children.items.properties.children.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.text/plain.schema.properties.children.items.properties.children.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.application/json.schema.properties.identifier.properties.assigner.properties.identifier.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.application/json.schema.properties.identifier.properties.children.items.properties.children.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.application/json.schema.properties.identifier.properties.extension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.application/json.schema.properties.typeElement.properties.extension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.application/json.schema.properties.typeElement.properties.children.items.properties.children.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.application/json.schema.properties.timestampElement.properties.extension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.application/json.schema.properties.timestampElement.properties.children.items.properties.children.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.application/json.schema.properties.totalElement.properties.extension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.application/json.schema.properties.totalElement.properties.children.items.properties.children.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.application/json.schema.properties.link.items.properties.children.items.properties.children.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.application/json.schema.properties.link.items.properties.modifierExtension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.application/json.schema.properties.link.items.properties.extension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.application/json.schema.properties.entry.items.properties.children.items.properties.children.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.application/json.schema.properties.entry.items.properties.modifierExtension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.application/json.schema.properties.entry.items.properties.extension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.application/json.schema.properties.signature.properties.who.properties.identifier.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.application/json.schema.properties.signature.properties.onBehalfOf.properties.identifier.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.application/json.schema.properties.signature.properties.children.items.properties.children.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.application/json.schema.properties.signature.properties.extension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.requestBody.content.application/json.schema.properties.parameter.items.properties.nameElement.properties.extension.items.properties.children.items.properties.children.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.requestBody.content.application/json.schema.properties.parameter.items.properties.nameElement.properties.extension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.requestBody.content.application/json.schema.properties.parameter.items.properties.nameElement.properties.children.items.properties.children.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.requestBody.content.application/json.schema.properties.parameter.items.properties.value.properties.children.items.properties.children.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.requestBody.content.application/json.schema.properties.parameter.items.properties.value.properties.extension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.requestBody.content.application/json.schema.properties.parameter.items.properties.resource.properties.children.items.properties.children.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.requestBody.content.application/json.schema.properties.parameter.items.properties.resource.properties.idElement.properties.extension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.requestBody.content.application/json.schema.properties.parameter.items.properties.part.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.requestBody.content.application/json.schema.properties.parameter.items.properties.modifierExtension.items.properties.children.items.properties.children.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.requestBody.content.application/json.schema.properties.children.items.properties.children.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.requestBody.content.application/json.schema.properties.meta.properties.profileElement.items.properties.extension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.requestBody.content.application/json.schema.properties.meta.properties.profileElement.items.properties.children.items.properties.children.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.requestBody.content.application/json.schema.properties.meta.properties.security.items.properties.children.items.properties.children.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.requestBody.content.application/json.schema.properties.meta.properties.security.items.properties.extension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.requestBody.content.text/json.schema.properties.parameter.items.properties.modifierExtension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.requestBody.content.text/json.schema.properties.meta.properties.security.items.properties.userSelectedElement.properties.extension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.requestBody.content.text/json.schema.properties.meta.properties.security.items.properties.userSelectedElement.properties.children.items.properties.children.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.text/plain.schema.properties.identifier.properties.useElement.properties.extension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.text/plain.schema.properties.identifier.properties.useElement.properties.children.items.properties.children.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.text/plain.schema.properties.identifier.properties.type.properties.children.items.properties.children.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.text/plain.schema.properties.identifier.properties.type.properties.extension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.text/plain.schema.properties.identifier.properties.period.properties.children.items.properties.children.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.text/plain.schema.properties.identifier.properties.period.properties.extension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.text/plain.schema.properties.identifier.properties.assigner.properties.identifier.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.text/plain.schema.properties.identifier.properties.assigner.properties.children.items.properties.children.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.text/plain.schema.properties.identifier.properties.assigner.properties.extension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.text/plain.schema.properties.entry.items.properties.search.properties.children.items.properties.children.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.text/plain.schema.properties.entry.items.properties.search.properties.modifierExtension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.text/plain.schema.properties.entry.items.properties.search.properties.extension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.text/plain.schema.properties.entry.items.properties.request.properties.children.items.properties.children.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.text/plain.schema.properties.entry.items.properties.request.properties.modifierExtension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.text/plain.schema.properties.entry.items.properties.request.properties.extension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.text/plain.schema.properties.entry.items.properties.response.properties.children.items.properties.children.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.text/plain.schema.properties.entry.items.properties.response.properties.modifierExtension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.text/plain.schema.properties.entry.items.properties.response.properties.extension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.text/plain.schema.properties.signature.properties.dataElement.properties.extension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.text/plain.schema.properties.signature.properties.dataElement.properties.children.items.properties.children.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.application/json.schema.properties.identifier.properties.period.properties.startElement.properties.extension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.application/json.schema.properties.identifier.properties.period.properties.startElement.properties.children.items.properties.children.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.application/json.schema.properties.entry.items.properties.search.properties.modeElement.properties.extension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.application/json.schema.properties.entry.items.properties.search.properties.modeElement.properties.children.items.properties.children.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.application/json.schema.properties.entry.items.properties.search.properties.scoreElement.properties.extension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.application/json.schema.properties.entry.items.properties.search.properties.scoreElement.properties.children.items.properties.children.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.application/json.schema.properties.entry.items.properties.request.properties.methodElement.properties.extension.items.properties.extension.items.$ref
Could not resolve reference:
Resolver error at paths./Bundle/$analyze.post.responses.200.content.application/json.schema.properties.entry.items.properties.request.properties.methodElement.properties.children.items.properties.children.items.$ref
Could not resolve reference:
The controller method:
[HttpPost("$analyze")]
public async Task<ActionResult<Bundle>> Analyze([FromBody]Parameters parameters)
{
[...]
return Ok();
}
The input (Parameters) and output (Bundle) parameters are objects contained in Hl7.Fhir.R4 (nuget, source)
Following packages are being used:
swagger.json (truncated, as it was too long to be posted):
{
"openapi": "3.0.1",
"info": {
"title": "Analyzer",
"version": "v1"
},
"paths": {
"/Bundle/$analyze": {
"post": {
"tags": [
"Bundle"
],
"parameters": [],
"requestBody": {
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/Parameters"
}
},
"text/json": {
"schema": {
"$ref": "#/components/schemas/Parameters"
}
},
"application/*+json": {
"schema": {
"$ref": "#/components/schemas/Parameters"
}
}
}
},
"responses": {
"200": {
"description": "Success",
"content": {
"text/plain": {
"schema": {
"$ref": "#/components/schemas/Bundle"
}
},
"application/json": {
"schema": {
"$ref": "#/components/schemas/Bundle"
}
},
"text/json": {
"schema": {
"$ref": "#/components/schemas/Bundle"
}
}
}
}
}
}
}
},
"components": {
"schemas": {
"ResourceType": {
"enum": [
"Account",
"ActivityDefinition",
"AdverseEvent",
"AllergyIntolerance",
"Appointment",
"AppointmentResponse",
"AuditEvent",
"Basic",
"Binary",
"BiologicallyDerivedProduct",
"BodyStructure",
"Bundle",
"CapabilityStatement",
"CarePlan",
"CareTeam",
"CatalogEntry",
"ChargeItem",
"ChargeItemDefinition",
"Claim",
"ClaimResponse",
"ClinicalImpression",
"CodeSystem",
"Communication",
"CommunicationRequest",
"CompartmentDefinition",
"Composition",
"ConceptMap",
"Condition",
"Consent",
"Contract",
"Coverage",
"CoverageEligibilityRequest",
"CoverageEligibilityResponse",
"DetectedIssue",
"Device",
"DeviceDefinition",
"DeviceMetric",
"DeviceRequest",
"DeviceUseStatement",
"DiagnosticReport",
"DocumentManifest",
"DocumentReference",
"DomainResource",
"EffectEvidenceSynthesis",
"Encounter",
[...]
],
"type": "string"
},
"ElementValue": {
"type": "object"
},
"Base": {
"type": "object",
"properties": {
"typeName": {
"type": "string",
"nullable": true,
"readOnly": true
},
"children": {
"type": "array",
"items": {
"$ref": "#/components/schemas/Base"
},
"nullable": true,
"readOnly": true
},
"namedChildren": {
"type": "array",
"items": {
"$ref": "#/components/schemas/ElementValue"
},
"nullable": true,
"readOnly": true
},
"userData": {
"type": "object",
"additionalProperties": {
"type": "object"
},
"nullable": true,
"readOnly": true,
"deprecated": true
}
}
},
"Element": {
"type": "object",
"properties": {
"typeName": {
"type": "string",
"nullable": true,
"readOnly": true
},
"children": {
"type": "array",
"items": {
"$ref": "#/components/schemas/Base"
},
"nullable": true,
"readOnly": true
},
"namedChildren": {
"type": "array",
"items": {
"$ref": "#/components/schemas/ElementValue"
},
"nullable": true,
"readOnly": true
},
"elementId": {
"type": "string",
"nullable": true
},
"extension": {
"type": "array",
"items": {
"$ref": "#/components/schemas/Extension"
},
"nullable": true
},
"userData": {
"type": "object",
"additionalProperties": {
"type": "object"
},
"nullable": true,
"readOnly": true,
"deprecated": true
}
}
},
"Extension": {
"type": "object",
"properties": {
"typeName": {
"type": "string",
"nullable": true,
"readOnly": true
},
"url": {
"type": "string",
"nullable": true
},
"value": {
"$ref": "#/components/schemas/Element"
},
"children": {
"type": "array",
"items": {
"$ref": "#/components/schemas/Base"
},
"nullable": true,
"readOnly": true
},
"namedChildren": {
"type": "array",
"items": {
"$ref": "#/components/schemas/ElementValue"
},
"nullable": true,
"readOnly": true
},
"elementId": {
"type": "string",
"nullable": true
},
"extension": {
"type": "array",
"items": {
"$ref": "#/components/schemas/Extension"
},
"nullable": true
},
"userData": {
"type": "object",
"additionalProperties": {
"type": "object"
},
"nullable": true,
"readOnly": true,
"deprecated": true
}
}
},
[...]
When testing the function, everything works fine.
What is SwaggerUI trying to tell me and how can I fix these errors? Any hint would be appreciated

This issue resolved itself with the latest swashbuckle version

The call of UseInlineDefinitionsForEnums helped to eliminate similar Resolver error at paths...Could not resolve reference:... in my case.
The error was:
Errors
Hide
Resolver error at paths./api/v1/Account/IsTenantAvailable.post.responses.200.content.text/plain.schema.properties.state.$ref
Could not resolve reference: Could not resolve pointer: /components/schemas/TenantAvailabilityState does not exist in document
Resolver error at paths./api/v1/Account/IsTenantAvailable.post.responses.200.content.application/json.schema.properties.state.$ref
Could not resolve reference: Could not resolve pointer: /components/schemas/TenantAvailabilityState does not exist in document
Resolver error at paths./api/v1/Account/IsTenantAvailable.post.responses.200.content.text/json.schema.properties.state.$ref
Could not resolve reference: Could not resolve pointer: /components/schemas/TenantAvailabilityState does not exist in document
The introduced call was:
using Microsoft.Extensions.DependencyInjection;
public class Startup
{
// ...
public IServiceProvider ConfigureServices(IServiceCollection services)
{
// ...
services.AddSwaggerGen(options =>
{
// ...
// calling SwaggerGenOptionsExtensions.UseInlineDefinitionsForEnums(Swashbuckle.AspNetCore.SwaggerGen.SwaggerGenOptions) instead of some UseReferencedDefinitionsForEnums
options.UseInlineDefinitionsForEnums(); // options.UseReferencedDefinitionsForEnums();
// ...
}
// ...
}
}
The call was inserted instead of commented out call of UseReferencedDefinitionsForEnums. It was commented out because of upgrading aspnetcore to 3.1 in my solution. The version that worked with 3.1 just did not contain this method any more, without any note about what should be called instead.
It is clear that both calls are related to enums. But I have no idea what exactly and why do they do, if they are for the same purpose and if they have any undesired side effects. But it just worked in context of eliminating this red plate in the screenshot on pressing "Try it out" on many of my endpoints.
I faced the issue on the version 5.6.3 of Swashbuckle.AspNetCore. Upgrading to the latest 6.1.1 did not make any help.

Related

how can I solve error 'The handle is invalid' in serilog

I have an Asp.net core web api using .net core 6.
I use Serilog to store the app log in a text file.
sometimes I get this error:
Caught exception while emitting to sink Serilog.Sinks.SystemConsole.ConsoleSink: System.IO.IOException: The handle is invalid.
at System.ConsolePal.WindowsConsoleStream.Write(ReadOnlySpan`1 buffer)
at System.IO.StreamWriter.Flush(Boolean flushStream, Boolean flushEncoder)
at System.IO.StreamWriter.Write(String value)
at System.IO.TextWriter.SyncTextWriter.Write(String value)
at Serilog.Sinks.SystemConsole.ConsoleSink.Emit(LogEvent logEvent)
at Serilog.Core.Sinks.SafeAggregateSink.Emit(LogEvent logEvent)
could you please help me how can I solve this issue?
this is my appsetting:
"Serilog": {
"Using": [
"Serilog.Sinks.Console",
"Serilog.Sinks.File"
],
"MinimumLevel": "Information",
"WriteTo": [
{
"Name": "File",
"Args": {
"path": "AppLogs/bckLog-.log",
"rollingInterval": "Day",
"restrictedToMinimumLevel": "Warning"
}
},
{
"Name": "Console"
}
],
"Enrich": [
"FromLogContext"
]
}
and this is my program.cs
Serilog.Debugging.SelfLog.Enable(message => { Log.Logger?.Error(message); Console.WriteLine(message); });
Log.Logger = new LoggerConfiguration().ReadFrom.Configuration(builder.Configuration).CreateLogger();
builder.Host.UseSerilog();

AppSync request mapping template errors not logged in CloudWatch

Crosspost from: https://repost.aws/questions/QUp5jDZ6bsRkeXhIwHgQaWkg/app-sync-request-mapping-template-errors-not-logged-in-cloud-watch
I have a simple resolver that has a simple Lambda function as a data source. This function always throws an error (to test out logging).
The resolver has request mapping template enabled and it is configured as follows:
$util.error("request mapping error 1")
The API has logging configured to be as verbose as possible yet I cannot see this request mapping error 1 from my CloudWatch logs in RequestMapping log type:
{
"logType": "RequestMapping",
"path": [
"singlePost"
],
"fieldName": "singlePost",
"resolverArn": "xxx",
"requestId": "bab942c6-9ae7-4771-ba45-7911afd262ac",
"context": {
"arguments": {
"id": "123"
},
"stash": {},
"outErrors": []
},
"fieldInError": false,
"errors": [],
"parentType": "Query",
"graphQLAPIId": "xxx"
}
The error is not completely lost because I can see this error in the query response:
{
"data": {
"singlePost": null
},
"errors": [
{
"path": [
"singlePost"
],
"data": null,
"errorType": null,
"errorInfo": null,
"locations": [
{
"line": 2,
"column": 3,
"sourceName": null
}
],
"message": "request mapping error 1"
}
]
}
When I add $util.appendError("append request mapping error 1") to the request mapping template so it looks like this:
$util.appendError("append request mapping error 1")
$util.error("request mapping error 1")
Then the appended error appears in the RequestMapping log type but the errors array is still empty:
{
"logType": "RequestMapping",
"path": [
"singlePost"
],
"fieldName": "singlePost",
"resolverArn": "xxx",
"requestId": "f8eecff9-b211-44b7-8753-6cc6e269c938",
"context": {
"arguments": {
"id": "123"
},
"stash": {},
"outErrors": [
{
"message": "append request mapping error 1"
}
]
},
"fieldInError": false,
"errors": [],
"parentType": "Query",
"graphQLAPIId": "xxx"
}
When I do the same thing with response mapping template then everything works as expected (errors array contains $util.error(message) and outErrors array contains $util.appendError(message) messages.
Is this working as expected so the $util.error(message) will never show up in RequestMapping type CloudWatch logs?
Under what conditions will errors array in RequestMapping log type be populated?
Bonus question: can the errors array contain more than 1 item for either RequestMapping or ResponseMapping log types?

BigQuery: Routine deployment failing with error "Unknown option: description"

We use terraform to deploy BigQuery objects (datasets, tables, routines etc..) to region europe-west2 in GCP. We do this many many times a day and all of a sudden at "2021-08-18T21:15:44.033910202Z" our deployments starting failing when attempting to deploy BigQuery routines. They are all failing with errors of the form:
status: {
code: 3
message: "Unknown option: description"
}
Here is the first log message I can find pertaining to this error (I have redacted project names):
"protoPayload": {
"#type": "type.googleapis.com/google.cloud.audit.AuditLog",
"status": {
"code": 3,
"message": "Unknown option: description"
},
"authenticationInfo": {
"principalEmail": "deployer-dev#myadminproject.iam.gserviceaccount.com",
"serviceAccountDelegationInfo": [
{
"firstPartyPrincipal": {
"principalEmail": "deployer-dev#myadminproject.iam.gserviceaccount.com"
}
}
]
},
"requestMetadata": {
"callerIp": "10.51.0.116",
"callerSuppliedUserAgent": "Terraform/0.14.7 (+https://www.terraform.io) Terraform-Plugin-SDK/2.5.0 terraform-provider-google/3.69.0,gzip(gfe)",
"callerNetwork": "//compute.googleapis.com/projects/myadminproject/global/networks/__unknown__",
"requestAttributes": {},
"destinationAttributes": {}
},
"serviceName": "bigquery.googleapis.com",
"methodName": "google.cloud.bigquery.v2.RoutineService.InsertRoutine",
"authorizationInfo": [
{
"resource": "projects/myproject/datasets/p00003818_dp_model",
"permission": "bigquery.routines.create",
"granted": true,
"resourceAttributes": {}
}
],
"resourceName": "projects/myproject/datasets/p00003818_dp_model/routines/UserProfile_Events_AllCarData_Deployment",
"metadata": {
"routineCreation": {
"routine": {
"routineName": "projects/myproject/datasets/p00003818_dp_model/routines/UserProfile_Events_AllCarData_Deployment"
},
"reason": "ROUTINE_INSERT_REQUEST"
},
"#type": "type.googleapis.com/google.cloud.audit.BigQueryAuditMetadata"
}
},
"insertId": "ak27xdbke",
"resource": {
"type": "bigquery_dataset",
"labels": {
"dataset_id": "p00003818_dp_model",
"project_id": "myproject"
}
},
"timestamp": "2021-08-18T21:15:43.109609Z",
"severity": "ERROR",
"logName": "projects/myproject/logs/cloudaudit.googleapis.com%2Factivity",
"receiveTimestamp": "2021-08-18T21:15:44.033910202Z"
}
The fact that this occurred without any changes by ourselves indicates that this is a problem at the Google end. I also observe that whilst we witnessed this in a few projects it occurred first in one project and then a few minutes later in another - that may or may not be helpful information.
Posting here in case anyone else hits this problem and also hoping it might catch the attention of a googler.
UPDATE! I have reproduced the pproblem using the REST API https://cloud.google.com/bigquery/docs/reference/rest/v2/routines/insert
I have entered a payload that does not include a description and that successfully creates a routine:
However, if I include a description which, as this screenshot indicates, is a valid parameter:
then the request fails:

How to pass AWS Lambda error in AWS SNS notification through AWS Step Functions?

I have created an AWS Step Function which triggers a Lambda python code, terminates without error if Lambda succeeds, otherwise calls an SNS topic to message the subscribed users if the Lambda fails. It is running, but the message was fixed. The Step Function JSON is as follows:
{
"StartAt": "Lambda Trigger",
"States": {
"Lambda Trigger": {
"Type": "Task",
"Resource": "arn:aws:lambda:us-east-2:xxxxxxxxxxxx:function:helloworldTest",
"End": true,
"Catch": [
{
"ErrorEquals": [
"States.ALL"
],
"ResultPath": "$.error",
"Next": "Notify Failure"
}
]
},
"Notify Failure": {
"Type": "Task",
"Resource": "arn:aws:states:::sns:publish",
"Parameters": {
"Message": "Batch job submitted through Step Functions failed with the following error, $.error",
"TopicArn": "arn:aws:sns:us-east-2:xxxxxxxxxxxx:lambda-execution-failure"
},
"End": true
}
}
}
Only thing is, I want to append the failure error message to my message string, which I tried, but is not working as expected.
But I get a mail as follows:
How to go about it?
I could solve the problem using "Error.$": "$.Cause".
The following is a working example of the failure portion of state machine:
"Job Failure": {
"Type": "Task",
"Resource": "arn:aws:states:::sns:publish",
"Parameters": {
"Subject": "Lambda Job Failed",
"Message": {
"Alarm": "Lambda Job Failed",
"Error.$": "$.Cause"
},
"TopicArn": "arn:aws:sns:us-east-2:xxxxxxxxxxxx:Job-Run-Notification"
},
"End": true
}
Hope this helps!
Here is the full version of the code
{
"Comment": "A Hello World example of the Amazon States Language using an AWS Lambda function",
"StartAt": "HelloWorld",
"States": {
"HelloWorld": {
"Type": "Task",
"Resource": "arn:aws:lambda:us-east-1:XXXXXXXXXXXXX:function:StepFunctionTest",
"End": true,
"Catch": [
{
"ErrorEquals": [
"States.ALL"
],
"Next": "NotifyFailure"
}
]
},
"NotifyFailure": {
"Type": "Task",
"Resource": "arn:aws:states:::sns:publish",
"Parameters": {
"Subject": "[ERROR]: Task failed",
"Message": {
"Alarm": "Batch job submitted through Step Functions failed with the following error",
"Error.$": "$.Cause"
},
"TopicArn": "arn:aws:sns:us-east-1:XXXXXXXXXXXXX:Notificaiton"
},
"End": true
}
}
}
This line is already appending exception object to 'error' path.
"ResultPath": "$.error"
We just need pass '$' to Message.$ to SNS task, both input and error details will be sent to SNS.
{
"TopicArn":"${SnsTopic}",
"Message.$":"$"
}
if we don't want input to Lambda to be appended in email, we should skip ResultPath or have just '$' as ResultPath, input object is ignored.
"ResultPath": "$"

How can resolve SSL handshake issue on reindex from remote server

I am trying to reindex from a remote server to my local es index. remote is a https host, I am getting SSL handshake exception as following error, please advise, thank you
{
"source": {
"remote": {
"host": "https://otherhost:9200"
},
"index": "twitter",
"query": {
"match_all": {
}
}
},
"dest": {
"index": "new_twitter"
}
}
this is response:
{
"error": {
"root_cause": [
{
"type": "s_s_l_handshake_exception",
"reason": "General SSLEngine problem"
}
],
"type": "s_s_l_handshake_exception",
"reason": "General SSLEngine problem",
"caused_by": {
"type": "s_s_l_handshake_exception",
"reason": "General SSLEngine problem",
"caused_by": {
"type": "validator_exception",
"reason": "PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target",
"caused_by": {
"type": "sun_cert_path_builder_exception",
"reason": "unable to find valid certification path to requested target"
}
}
}
},
"status": 500
}