Request Json Looks like the below :-
{
"eNumber": 8506493,
"details": [
{
"id":12345,
"name": xyz123
}
]
}
As part of requirement, I need to check the "details" array that either "id" or "name" field must present. if "id" field is present, "name" is non-mandatory. if "name" field is present, "id" is non-mandatory. Throws error if not met.
I tried few options using filter the details array and check the size of the filtered array in the validation component. It does not seems to be working well. if anyone has got better solutions.please share it here.
This example code will return true or false if it passes your condition
%dw 2.0
import * from dw::core::Arrays
output application/json
---
payload.details every ((item) -> item.id? or item.name?)
The function I'm using is every that checks that all elements in an array passes the given criteria.
Later you can use a choice and a use the raise error or you can use the fail dw function with an if else.
You can restrict it at RAML level.
Sample RAML -
#%RAML 1.0
title: api
types:
test1:
type: object
properties:
id:
type: string
example: 123a
test2:
type: object
properties:
name:
type: string
example: xyz124
test3:
type: object
properties:
enumber:
type: string
example: 8506493a
details :
type: [test1 | test2]
/test:
post:
body:
application/json:
type: test3
I stumbled on the idea of limiting my tables and associations by using arrays in my models like so.
I'm working on a task assignment app where the user will, in essence, construct a sentence to perform an action. I'm using keywords to help structure the boundaries of the sentence.
Examples include (keywords in brackets):
"[I will] paint the fence" <- makes a new task, assigned to current_user
"[Ask] Huck [to] paint the fence" <- find_or_create task, assign to find_or_create user
"[Archive] painting the fence" <- soft-delete task
So here's my migration:
class CreateKeywords < ActiveRecord::Migration[5.1]
def change
create_table :keywords do |t|
t.string :name, null: false
t.text :pre, array: true, default: []
t.text :post, array: true, default: []
t.string :method, null: false, default: "read" # a CRUD indicator
end
end
end
keyword.post indicates what models could follow the keyword
keyword.pre indicates what models could preceed the keyword
My seed data looks like this:
Keyword.create([
{ name: "i will", post: ["task"] },
{ name: "ask", post: ["user"] },
{ name: "assign", post: ["user", "task"] },
{ name: "find a", post: ["user", "task"] },
{ name: "make a new", post: ["user", "task"], method: "create" },
{ name: "finalize", post: ["task"] },
{ name: "archive", post: ["user", "task"], method: "delete" },
{ name: "update", post: ["user", "task"], method: "update" },
{ name: "for", post: ["user", "task"], pre: ["user", "task"] },
{ name: "to", post: ["user", "task"], pre: ["user", "task"] },
{ name: "and repeat", pre: ["task"] },
{ name: "before", pre: ["task"] },
{ name: "after", pre: ["task"] },
{ name: "on", pre: ["task"] }
])
Now I want to do things like:
key = Keyword.third
Keyword.where(pre: key.post)
But this returns exact matches and I want to do something like:
"Return all keywords where any value of key.post can be found in Keyword.pre"
I haven't had any luck along these lines:
Keyword.where(pre.include? key.post)
I can iterate over all the Keywords and use AND:
results = []
Keyword.all.each do |k|
comb = k.pre & key.post
if comb.present?
results << k
end
end
results.map { |k| k.name }
But this feels bad.
And I'm a bit out of my depth on the SQL necessary to do this.
Any pointers?
There are two things you want to know about:
PostgreSQL's "array constructor" syntax which looks like array['a', 'b', 'c'] rather than the more common '{a,b,c}' syntax.
PostgreSQL's array operators.
Array constructor syntax is convenient because when ActiveRecord sees an array as the value for a placeholder, it expands the array as a comma delimited list which works equally well with x in (?) and x && array[?].
For the operator to use, you want:
all keywords where any value of key.post can be found in Keyword.pre
which is another way of saying that key.post and Keyword.pre overlap and the operator for that is &&. There are also subset (<#) and superset (#>) operators if you need slightly different logic.
Putting that together:
Keyword.where('pre && array[?]', key.post)
or:
Keyword.where('pre && array[:post]', post: key.post)
lacostenycoder, in the comments, is right to be concerned empty arrays. ActiveRecord expands empty arrays to a single null when filling in a placeholder so you could end up doing SQL like:
pre && array[null]
and PostgreSQL won't be able to determine the type of array[null]. You can include a cast:
Keyword.where('pre && array[:post]::text[]', post: key.post)
# --------------------------------^^^^^^^^ tell PostgreSQL that this is an array of text
But, since pre && array[null]::text[] will never be true, you could just skip the whole thing if key.post.empty?.
Empty arrays don't overlap with any other array (not even another empty array) so you don't need to worry about empty arrays beyond what ActiveRecord will do with them. Similarly for pre is null, null && any_array will never be true (it will actually evaluate to null) so there won't be any overlaps with such things.
This is actually much easier that you would expect:
Keyword.where("pre && post")
# testing vs your seed data plus a few I added returns:
#Keyword Load (1.0ms) SELECT "keywords".* FROM "keywords" WHERE (pre && post)
#[<Keyword:0x007fc03c0438b0 id: 9, name: "for", pre: ["user", "task"], post: ["user", "task"], method: "read">,
#<Keyword:0x007fc03c0435b8 id: 10, name: "to", pre: ["user", "task"], post: ["user", "task"], method: "read">,
#<Keyword:0x007fc03c043248 id: 15, name: "foo", pre: ["bar", "baz"], post: ["baz", "boo"], method: "read">,
#<Keyword:0x007fc03c042f28 id: 16, name: "rock", pre: ["anthem"], post: ["ballad", "anthem"], method: "read">,
#<Keyword:0x007fc03c042cf8 id: 17, name: "disco", pre: ["mood"], post: ["ballad", "anthem", "mood"], method: "read">]
I currently have two JSON files with data.
The first has a list of actions by a user. However, the userID is a UID string instead of their user name.
The second is a JSON list of UID strings with their corresponding user names. I stored this JSON in a Flow Variable named UserPayload
I'd like to merge the user name into the first list so that my output has actual users names instead of the UID.
Here is what I had so far...but the userId part doesn't work since my flowVar is an list.
%dw 1.0
%output application/csv
---
payload map ((payload01, indexOfPayload01) -> {
user: payload.01.userId,
actions: payload01.action,
FileName: payload01.payload.properties.name default "N/A",
userId: flowVars.UserPayload.objectName
})
Any help would be appreciated. Thanks!
you can access the right object from the UserPayload flowVar by either filtering the content of UserPayload based on userId or creating a associative array from UserPayload.
filtering UserPayload
%dw 1.0
%output application/json
---
payload map (action, index) -> {
user: action.user,
action: action.action,
FileName: action.FileName,
userId: (flowVars.UserPayload filter $.objectId == action.user)[0].objectName
}
associative array
%dw 1.0
%output application/json
%var users = flowVars.UserPayload groupBy $.objectId
---
payload map (action, index) -> {
user: action.user,
action: action.action,
FileName: action.FileName,
userId: users[action.user][0].objectName
}
for my examples i used following sample data derived from your question.
flowVars.UserPayload
[{
"objectId": "0000-0000-0001",
"objectName": "first user"
}, {
"objectId": "0000-0000-0002",
"objectName": "second user"
}]
payload
[{
"user": "0000-0000-0001",
"action": "some crazy action",
"FileName": "some-crazy-action.mov"
}]
I am working on a project where we receive a flat file but the first and last lines have information that does not fit the fixed width pattern. Is there a way to dataweave all of this information correctly and if possible put the header and footer into variables and just have the contents in the payload.
Example File
HDMTFSBEUP00000220170209130400 MT HD07
DT01870977 FSFSS F3749261 CR00469002017020820170225 0000
DT01870978 FSFSS F3749262 CR00062002017020820170125 0000
TRMTFSBEUP00000220170209130400 000000020000002000000000000043330000000000000 0000
I know for CSV you can skip a line but dont see it with fixed width and also the header and footer will both start with the first 2 letters every time so maybe they can be filtered by dataweave?
Please refer to the DataWeave Flatfile Schemas documentation. There are several examples for processing several different types of data.
In this case, I tried to simplify your example data, and apply a custom schema as follow:
Example data:
HDMTFSBEUP00000220170209130400
DT01870977
DT01870978
TRMTFSBEUP00000220170209130400
Schema/Flat File Definition:
form: FLATFILE
structures:
- id: 'test'
name: test
tagStart: 0
tagLength: 2
data:
- { idRef: 'header' }
- { idRef: 'data', count: '>1' }
- { idRef: 'footer' }
segments:
- id: 'header'
name: header
tag: 'HD'
values:
- { name: 'header', type: String, length: 39 }
- id: 'data'
name: data
tag: 'DT'
values:
- { name: 'code', type: String, length: 17 }
- id: 'footer'
name: footer
tag: 'TR'
values:
- { name: 'footer', type: String, length: 30 }
The schema will validate the example data and identify based on the tag, the first 2 letters. The output will be grouped accordingly.
{
"header": {},
"data": [{}, {}],
"footer": {}
}
Since the expected result is only the data, then just select it: payload.data.
Use range selector to skip header and footer.
payload[1..-2] map {
field1: $[0..15],
field2: $[16..31]
...,
...
}
[1..-2] will select from 2nd line till the second last line in the payload.
$[0..15] will select from 1st column index to 16th index. $[16..31] select from 17th column index to 32nd index.
I was facing the same issue and the answer #sulthony h wrote needs a little tweak. I used these lines instead and it worked for me.
data:
- { idRef: 'header', count: 1 }
- { idRef: 'data', count: '>1' }
- { idRef: 'footer', count: 1 }
"count" was missing from header and footer, and that was throwing an exception. Hope this helps.
here is my html file :
...
<span data-dojo-id="staffStore" data-dojo-type="dojo.data.ItemFileReadStore" data-dojo-props='data:../../staff.json'></span>
<input data-dojo-type="dijit.form.ComboBox"
data-dojo-props="store: staffStore,
keyAttr: 'id',
searchAttr: 'staff_name',
autoComplete: true,
id: 'staff_name',
name:'staff_name',
value: '' "/>
...
and the json data goes as follows:
{
identifier: "id";,
label: "id",
items: [{id: 982483700, staff_name: "guanyu";},{id: 582057769, staff_name: "zhangfei";},{id: 166802994, staff_name: "zhaoyun";}]
}
here is my problem:
when i use post method i have got 'staff_name' in the searchAttr: 'staff_name' passed to the background-appication ,but i want to have the 'id' in the keyAttr: 'id' passed to background-application.in a word,i have passed made a wrong post action.can someone help me get out of this problem?
Use dijit/form/FilteringSelect not dijit/form/ComboBox.
You can enter any text into a ComboBox therefore it cannot return id which is the reason it returns text (or label or searchAttr). FilteringSelect allows only to chose one of options and therefore it can return id, which it does.
See it in action and experiment at jsFiddle: http://jsfiddle.net/phusick/QzQ38/