I am working on a project where we receive a flat file but the first and last lines have information that does not fit the fixed width pattern. Is there a way to dataweave all of this information correctly and if possible put the header and footer into variables and just have the contents in the payload.
Example File
HDMTFSBEUP00000220170209130400 MT HD07
DT01870977 FSFSS F3749261 CR00469002017020820170225 0000
DT01870978 FSFSS F3749262 CR00062002017020820170125 0000
TRMTFSBEUP00000220170209130400 000000020000002000000000000043330000000000000 0000
I know for CSV you can skip a line but dont see it with fixed width and also the header and footer will both start with the first 2 letters every time so maybe they can be filtered by dataweave?
Please refer to the DataWeave Flatfile Schemas documentation. There are several examples for processing several different types of data.
In this case, I tried to simplify your example data, and apply a custom schema as follow:
Example data:
HDMTFSBEUP00000220170209130400
DT01870977
DT01870978
TRMTFSBEUP00000220170209130400
Schema/Flat File Definition:
form: FLATFILE
structures:
- id: 'test'
name: test
tagStart: 0
tagLength: 2
data:
- { idRef: 'header' }
- { idRef: 'data', count: '>1' }
- { idRef: 'footer' }
segments:
- id: 'header'
name: header
tag: 'HD'
values:
- { name: 'header', type: String, length: 39 }
- id: 'data'
name: data
tag: 'DT'
values:
- { name: 'code', type: String, length: 17 }
- id: 'footer'
name: footer
tag: 'TR'
values:
- { name: 'footer', type: String, length: 30 }
The schema will validate the example data and identify based on the tag, the first 2 letters. The output will be grouped accordingly.
{
"header": {},
"data": [{}, {}],
"footer": {}
}
Since the expected result is only the data, then just select it: payload.data.
Use range selector to skip header and footer.
payload[1..-2] map {
field1: $[0..15],
field2: $[16..31]
...,
...
}
[1..-2] will select from 2nd line till the second last line in the payload.
$[0..15] will select from 1st column index to 16th index. $[16..31] select from 17th column index to 32nd index.
I was facing the same issue and the answer #sulthony h wrote needs a little tweak. I used these lines instead and it worked for me.
data:
- { idRef: 'header', count: 1 }
- { idRef: 'data', count: '>1' }
- { idRef: 'footer', count: 1 }
"count" was missing from header and footer, and that was throwing an exception. Hope this helps.
Related
I'm adding a list called 'tourlocation' to my Keystone 5 project. In my mongo database my tourlocations collection has an object called 'coordinates', with two values: 'lat' and 'long'. Example:
"coordinates" : {
"lat" : 53.343761,
"long" : -6.24953
},
In the previous version of keystone, I could define my tourlocation list coordinates object like this:
coordinates: {
lat: {
type: Number,
noedit: true
},
long: {
type: Number,
noedit: true
}
Now unfortunately, when I try to define the list this way it gives the error: The 'tourlocation.coordinates' field doesn't specify a valid type. (tourlocation.coordinates.type is undefined)'
Is there any way to represent objects in keystone 5?
#Alex Hughes I believe your error says "type" which you may need to add it like this
keystone.createList('User', {
fields: {
name: { type: Text }, // Look at the type "Text" even in the MongoDB you can choose the type but it will be better to choose it here from the beginning.
email: { type: Text },
},
});
Note that noedit: true is not supported in version 5 of KeystoneJS.
For more info look at this page https://www.keystonejs.com/blog/field-types#core-field-types
I can't find anything about how to do this type of query in FaunaDB. I need to select only specifics fields from a document, not all fields. I can select one field using Select function, like below:
serverClient.query(
q.Map(
q.Paginate(q.Documents(q.Collection('products')), {
size: 12,
}),
q.Lambda('X', q.Select(['data', 'title'], q.Get(q.Var('X'))))
)
)
Forget the selectAll function, it's deprecated.
You can also return an object literal like this:
serverClient.query(
q.Map(
q.Paginate(q.Documents(q.Collection('products')), {
size: 12,
}),
q.Lambda(
'X',
{
title: q.Select(['data', 'title'], q.Get(q.Var('X')),
otherField: q.Select(['data', 'other'], q.Get(q.Var('X'))
}
)
)
)
Also you are missing the end and beginning quotation marks in your question at ['data, title']
One way to achieve this would be to create an index that returns the values required. For example, if using the shell:
CreateIndex({
name: "<name of index>",
source: Collection("products"),
values: [
{ field: ["data", "title"] },
{ field: ["data", "<another field name>"] }
]
})
Then querying that index would return you the fields defined in the values of the index.
Map(
Paginate(
Match(Index("<name of index>"))
),
Lambda("product", Var("product"))
)
Although these examples are to be used in the shell, they can easily be used in code by adding a q. in front of each built-in function.
Request Json Looks like the below :-
{
"eNumber": 8506493,
"details": [
{
"id":12345,
"name": xyz123
}
]
}
As part of requirement, I need to check the "details" array that either "id" or "name" field must present. if "id" field is present, "name" is non-mandatory. if "name" field is present, "id" is non-mandatory. Throws error if not met.
I tried few options using filter the details array and check the size of the filtered array in the validation component. It does not seems to be working well. if anyone has got better solutions.please share it here.
This example code will return true or false if it passes your condition
%dw 2.0
import * from dw::core::Arrays
output application/json
---
payload.details every ((item) -> item.id? or item.name?)
The function I'm using is every that checks that all elements in an array passes the given criteria.
Later you can use a choice and a use the raise error or you can use the fail dw function with an if else.
You can restrict it at RAML level.
Sample RAML -
#%RAML 1.0
title: api
types:
test1:
type: object
properties:
id:
type: string
example: 123a
test2:
type: object
properties:
name:
type: string
example: xyz124
test3:
type: object
properties:
enumber:
type: string
example: 8506493a
details :
type: [test1 | test2]
/test:
post:
body:
application/json:
type: test3
I resolve most of my problem only few left out of which this one is preventing me to submit the form. I am using Nativescript + vue and without Typescript. how to display the Valueproviders with array list? Here is the code
https://play.nativescript.org/?template=play-vue&id=2oWObE
There was the problem with your data type. As per the documentation Array should have key and label properties. But still if you want id and name then you should try like below.
'valuesProvider': {
key: 'id',
label: 'name',
items: [
{ id: 1121, name: 'Shanghai' },
{ id: 3651, name: 'Lagos' },
{ id: 5213, name: 'Moscow' },
{ id: 6214, name: 'São Paulo' },
{ id: 6985, name: 'Sydney' }
]
};
https://docs.nativescript.org/vuejs/ns-ui/DataForm/dataform-editors-providers
Anyway, I tried that and that was not working for me either then searched for it and relaised that there is an open feature request to support the valuesProvider for picker from JSON metadata. You can vote to support the same feature.
https://github.com/NativeScript/nativescript-ui-feedback/issues/369
Solution
Just get you cityList out of vue data and map your data
https://play.nativescript.org/?template=play-vue&id=2oWObE&v=6
more detailed version with groups
https://play.nativescript.org/?template=play-vue&id=rqK7wO&v=3
I'm trying to use a match on a json object by only checking for some elements. The Json object contains a nested array which seem to be the root cause for the test failing. It seems like using the 'match contains' assertion on key:values within the nested array does not work as expected and throws an error.
I've tried to use 'match contains' on first level keys and that works fine but when attempting to match keys within a nested array I get the following error message: reason: actual value has 1 more key(s) than expected
* def json = { id: 1, priority: 1, compRound: { id: 1, comp_id: 89 } }
* match json contains
"""
{
id: '#number',
priority: '#number',
compRound: {
id: '#number'
}
}
"""
As I'm using the match contains assertion, I would expect for the test to pass but instead it looks like Karate expects all key:values within the nested array to be present.
Please read this section of the docs carefully: https://github.com/intuit/karate#contains-short-cuts
* def json = { id: 1, priority: 1, compRound: { id: 1, comp_id: 89 } }
* def compRound = { id: '#number' }
* match json == { id: '#number', priority: '#number', compRound: '#(^compRound)' }