FlatFile schema creation "out of order segment" error in mule 4 - mule
I am trying to parse a huge flat file into flat file schema, from a file if 3 rows contains tag value "2" then it should pick all three rows at a time but in my case, am unable to pick those rows whose tag values are similar but if it is in sequentially manner then i am able to pick if it is in different rows then it is unable to pick the rows and while parsing it throws error.
I want to pick all data along with all the data tag value starts with =2
all the tag value starts with= G
Please help me how can i achieve it?
1220612WEBL230PROD2206080606CA01
200000162608361 FFVV220606D910142122982635 4TKTT0140MAZUR/JESSICA APRIL
500000100000071069CAD2CA 00000001425XG 00000003384SQ 00000
G000001YXY YVR AC K 03JUL22 06SEPKZ2HZCFL
2000001CAD 657.60 CAD 710.690QYXY AC YVR Q3.00Q27.00 197.1
G000001 00000000000 CAD2 CA
2000001AC ONLY -BG AC
Schema
form: FLATFILE
structures:
- id: 'ID'
name: ID
data:
- { idRef: 'IT01', count: 1 }
- { idRef: 'IT02', count: '>1' }
- { idRef: 'IT05', count: '>1' }
- { idRef: 'IT0G', count: '>1' }
segments:
- id: 'IT01'
name: IFG RET file header
values:
- { name: 'recIdentifier', usage: M, type: String, length: 1, tagValue: '1' }
- { name: 'sate', usage: M, type: String, length: 6 }
- { name: 'fier', usage: M, type: String, length: 4 }
- { name: 'Number', usage: M, type: String, length: 3 }
- { name: 'nt', usage: M, type: String, length: 4 }
- { name: 'ate', usage: M, type: String, length: 6 }
- { name: 'ime', usage: M, type: String, length: 4 }
- { name: 'Code', usage: M, type: String, length: 2 }
- { name: 'Number', usage: M, type: String, length: 2 }
- { name: 'notUsed1', usage: M, type: String, length: 41 }
- id: 'IT02'
name: IFG RET file IT02 record
values:
- { name: 'notUsed1', usage: M, type: String, length: 1, tagValue: '2' }
- { name: 'sate', usage: M, type: String, length: 6 }
- { name: 'fier', usage: M, type: String, length: 4 }
- { name: 'Number', usage: M, type: String, length: 3 }
- { name: 'nt', usage: M, type: String, length: 4 }
- { name: 'ate', usage: M, type: String, length: 6 }
- { name: 'ime', usage: M, type: String, length: 4 }
- { name: 'Code', usage: M, type: String, length: 2 }
- { name: 'Number', usage: M, type: String, length: 2 }
- { name: 'notUsed1', usage: M, type: String, length: 41 }
- id: 'IT05'
name: IFG RET file IT05 record
values:
- { name: 'notUsed1', usage: M, type: String, length: 1, tagValue: '5' }
- { name: 'sate', usage: M, type: String, length: 6 }
- { name: 'fier', usage: M, type: String, length: 4 }
- { name: 'Number', usage: M, type: String, length: 3 }
- { name: 'nt', usage: M, type: String, length: 4 }
- { name: 'ate', usage: M, type: String, length: 6 }
- { name: 'ime', usage: M, type: String, length: 4 }
- { name: 'Code', usage: M, type: String, length: 2 }
- { name: 'Number', usage: M, type: String, length: 2 }
- { name: 'notUsed1', usage: M, type: String, length: 41 }
- id: 'IT0G'
name: IFG RET file IT0G record
values:
- { name: 'recIdentifier', usage: M, type: String, length: 1, tagValue: 'G' }
- { name: 'sate', usage: M, type: String, length: 6 }
- { name: 'fier', usage: M, type: String, length: 4 }
- { name: 'Number', usage: M, type: String, length: 3 }
- { name: 'nt', usage: M, type: String, length: 4 }
- { name: 'ate', usage: M, type: String, length: 6 }
- { name: 'ime', usage: M, type: String, length: 4 }
- { name: 'Code', usage: M, type: String, length: 2 }
- { name: 'Number', usage: M, type: String, length: 2 }
- { name: 'notUsed1', usage: M, type: String, length: 41 }
Expected output-
{
"actions": [
{
"5": [
{
"Records": " 0000000",
"hello3": " 00000000000"
}
],
"2": [
{
"After": "16DEC",
"hello": "YYZ"
}
],
"Header": { //how to generate this field in flat file schema please help me
"1": {
"reportingSystemIdentifier": "WEBL",
"recIdentifier": "1"
}
}
]
}
``````````````
The error is caused because the structure expects the order defined in the flat file schema (records 2, records 5, records G) but in the file line 5 has a record 2 after a record G.
If the records are in ordered groups of 2, 5, G you could try using a group with repetitions.
However if the records are unordered, then the flat file format may not support such a structure. You may need to implement an alternative with code (Java or scripts).
Updating the structure with a group that contains records 2, 5 and G in that order may work. Just as an example:
...
structures:
- id: 'IFGRET'
name: IFGRET
data:
- { idRef: 'IT01', count: 1 }
- groupId: 'Details'
count: '>1'
items:
- { idRef: 'IT02', count: '>1' }
- { idRef: 'IT05', count: '>1' }
- { idRef: 'IT0G', count: '>1' }
...
Related
How to perform tensorboard performance metrics visualization while using transfer learning?
I have a colab notebook with which I intend to train a object detection model and perform performance metric visualization using tensorboard Colab notebook link I am unable to display the charts, following is the content of driving_label_map.pbtxt which is required to be uploaded.
item {
id: 1
name: 'car'
}
item {
id: 2
name: 'pedestrian'
}
item {
id: 3
name: 'trafficLight-GreenLeft'
}
item {
id: 4
name: 'trafficLight-Green'
}
item {
id: 5
name: 'trafficLight-Red'
}
item {
id: 6
name: 'trafficLight-RedLeft'
}
item {
id: 7
name: 'trafficLight'
}
item {
id: 8
name: 'truck'
}
item {
id: 9
name: 'biker'
}
item {
id: 10
name: 'trafficLight-Yellow'
}
item {
id: 11
name: 'trafficLight-YellowLeft'
}
How to compare two arrays in Standard SQL (BigQuery)?
I have two tables with the exact same schema, one of the fields is a record and I try to compare this field between the two tables.
For example:
Table A:
{ Name: 'Mary',
DOB: '06.06.1970',
Children:[
{ Name: John, Age: 6 },
{ Name: Agatha, Age: 10}
]
},
{ Name: 'Bob',
DOB: '30.03.1982',
Children:[
{ Name: Anthony, Age: 9 },
{ Name: William, Age: 4 },
{ Name: Rose, Age: 2 }
]
},
{ Name: 'Adam',
DOB: '312.10.1980',
Children:[
{ Name: Andrew, Age: 9 },
{ Name: Kate, Age: 4 }
]
}
Table B:
{ Name: 'Mary',
DOB: '06.06.1970',
Children:[
{ Name: John, Age: 6 },
{ Name: Agatha, Age: 10}
]
},
{ Name: 'Bob',
DOB: '30.03.1982',
Children:[
{ Name: Anthony, Age: 9 },
{ Name: George, Age: 4 },
{ Name: Frank, Age: 2 }
]
},
{ Name: 'Adam',
DOB: '312.10.1980',
Children:[
{ Name: Andrew, Age: 9 },
{ Name: Kate, Age: 4 },
{ Name: Jonathan, Age: 2 }
]
}
The output I'd like to get is:
Name
----------
Bob
Adam
Since the Children details are not the same for Bob or Adam in table A and table B (whenever it's the size of the record (num of children) or the details (children names etc...))
I tried to use SELECT UNION DISTINCT but it doesn't work on an array, I also tried != and <> but it doesn't work on arrays...
It would be ideal to compare two arrays like you can compare two lists in Python (comparing the structure and the content).
Is there a way to implement something similar ?
Thank you
select name
from `project.dataset.tableA` a
join `project.dataset.tableB` b
using(name)
where a.dob != b.dob
or (select string_agg(format('%t', s) order by name) from a.children s)
!= (select string_agg(format('%t', s) order by name) from b.children s)
Deep assertion on JSON array with contains in Karate
Scenario:
Given def cat =
"""
{
name: 'Billie',
kittens: [
{ id: 23, name: 'Bob', age: 35 },
{ id: 42, name: 'Wild', age: 25 }
]
}
"""
Then match cat.kittens contains [{ id: 42, name: 'Wild' }, { id: 23, name: 'Bob' }]
As seen in the example, I am not worried about 'age'. I understand that I can use '#ignore'. Is there any other way, if I have lot of such nodes that I have to ignore.
I am just thinking, can it work with JSON arrays as it does with JSON Objects. asserting only specified nodes.
error:
assert.feature:24 - path: $.kittens[*], actual: [{"id":23,"name":"Bob","age":35},{"id":42,"name":"Wild","age":25}], expected: {id=42, name=Wild}, reason: actual value does not contain expected
EDIT:
I tried something suggested in Karate - how to check if array contains values?
but that did not help me. I am not looking for schema-validation instead I am looking for functional validation where every object may have different values for keys.
Below one fails
Scenario:
Given def cat =
"""
{
name: 'Billie',
kittens: [
{ id: 23, name: 'Bob', age: 35 },
{ id: 42, name: 'Wild', age: 25 }
]
}
"""
* def expected = [{ id: 42, name: 'Wild' }, { id: 23, name: 'Bob' }]
Then match cat.kittens contains '#(^expected)'
This one works fine but this is not helping me.
Scenario:
Given def cat =
"""
{
name: 'Billie',
kittens: [
{ id: 23, name: 'Bob', age: 35 },
{ id: 42, name: 'Wild', age: 25 }
]
}
"""
* def expected = { id: 42, name: 'Wild' }
Then match cat.kittens contains '#(^expected)'
Currently, I am reading arrays separately and asserting them with loop.
Just adding the word deep works in 0.9.6.RC4
Then match cat.kittens contains deep [{ id: 42, name: 'Wild' }, { id: 23, name: 'Bob' }]
I don't understand why the linked answer when I closed your first question did not help you. Maybe I don't understand, and someone else can provide a better answer.
OpenApi 3 with Kotlin does not generates validation
I'm trying to generate requests with OpenApi generator, but all constraints like min, minvalue, maxlength etc are ignored. Example:
data class SomeClassNameGen (
#JsonProperty("queryText") val queryText: kotlin.String,
#JsonProperty("test") val test: kotlin.Int? = null
}
code from openapi.yaml
SomeClassName:
title: desc
description: desc
type: desc
properties:
test:
type: integer
minimum: 1
maximum: 20
queryText:
description: Текст запроса
type: string
minLength: 10,
exclusiveMaximum: true
required:
- queryText
JSON schema - conditionally apply a $ref to a value based on the other value?
I am wanting to conditionally validate against a $ref based on another value in my schema.
items: {
type: 'object',
properties: {
kind: { type: 'string', enum: ['foo', 'bar'] },
//parameters: { $ref: 'foo.json#' } // This works
parameters: {
if: {
kind: 'foo'
},
then: {
$ref: 'foo.json#'
}
}
// also tried
if: {
kind: 'foo'
},
then: {
parameters: { $ref: 'foo.json#' }
}
I would like parameters to be validated against the foo.json reference whenever the value of kind is equal to foo (same with bar and bar.json). However the above is not working. Uncommenting out the commented section works so they are not equivalent.
How can I format this to conditionally apply a $ref to a value based on the other value?
I actually have about 10 different values for the type enum so if there is a cleaner way do this than if/else I am open.
Ah got it....
items: {
type: 'object',
properties: {
kind: { type: 'string', enum: ['foo', 'bar'] }
},
required: ['parameters'],
if: {
properties: { kind: { const: 'foo' } },
required: ['kind']
},
then: {
properties: {
parameters: {
$ref: 'fpp.json#'
}
}
}
It's helpful for me to think that whatever is in if and else are essentially merged with the main schema.