Case
I have a json object (as imported from excell file, and it needs to be formatted like one as well afterwards). So i can not mess with the structure at all.
The input consists of a json formatted object with sheets, every sheet is an array of objects.
I want to add another object (the var 'append') to the array (as last item) when the sheetnumber equals the kwadrantcode in the 'append' object
input 1
The payload
{
"(sheet)1": [
{
"kwadrantId": 1.0,
"Team": "blauw1",
"timestamp": "2019-09-26T16:37:54",
"controlecode": 12431
}
],
"(sheet)2": [
{
"kwadrantId": 2.0,
"Team": "rood1",
"timestamp": "2019-09-26T16:37:54",
"controlecode": 81243
},
{
"kwadrantId": 2.0,
"Team": "blauw2",
"timestamp": "2019-09-26T18:00:54",
"controlecode": 67676
}
]
}
the code so far
%dw 2.0
output application/json
var append = {"kwadrant": "2",
"controlecode": "22222",
"kleur": "blauw",
"subteam": "1",
"form_id": "83177",
"submit": "Claim"
}
---
payload pluck($) map (sheet, sheetnumber)-> (sheet map (claim, claimcounter) -> claim) ++ [append]
This code succesfully appends the object to the array at the right location.
The Problem here is that this last part (the map target) does not seem to allow a conditional.
So it will always insert it at the end of every list, not just the one where i want it to end up.
[
[
{
"kwadrantId": 1.0,
"Team": "blauw1",
"timestamp": "2019-09-26T16:37:54",
"controlecode": "12431"
},
{
"kwadrant": "2",
"controlecode": "22222",
"kleur": "blauw",
"subteam": "1",
"form_id": "83177",
"submit": "Claim"
}
],
[
{
"kwadrantId": 2.0,
"Team": "rood1",
"timestamp": "2019-09-26T16:37:54",
"controlecode": "12431"
},
{
"kwadrantId": 2.0,
"Team": "blauw2",
"timestamp": "2019-09-26T18:00:54",
"controlecode": 67676.0
},
{
"kwadrant": "2",
"controlecode": "22222",
"kleur": "blauw",
"subteam": "1",
"form_id": "83177",
"submit": "Claim"
}
]
]
A few things i have tried
honestly, i think i've tried everything the past two hours, but here are a few i can think of out of the top of my head.
++ [append] if (true)
(if true )++ [append]
payload pluck($) map (sheet, sheetnumber)-> (sheet map (claim, claimcounter) -> claim)((if true) ++ [append])
and most variations i can think of with brackets
I will worry about formatting the 'append' var to the right structure later, just getting it in the right spot is the issue i can't seem to get working.
The simplest way that I can think of is to use the mapObject function
%dw 2.0
output application/json
var append =
{
"kwadrant": "2",
"controlecode": "22222",
"kleur": "blauw",
"subteam": "1",
"form_id": "83177",
"submit": "Claim"
}
fun extractNumber(pageName: Key) =
(pageName as String match /\(sheet\)([0-9]+)/)[1]
---
payload mapObject ((value, key, index) -> do {
if(extractNumber(key) == append.kwadrant)
{(key): value << append}
else
{(key): value}
})
So basically I will go for every sheet in the root and when it matches the one I need to add append it to the value using the <<
Related
I need to compare two fields in a array and replace their values with different data and output in separate objects
Here is my sample input array:
{
"data": [
{
"house": "house1",
"condition": "bad",
"age": "old",
},
{
"house": "house2",
"condition": "good",
"age": "new",
}
]
}
Output should be
{
"data": [
{
"house": "house1",
"condition": "repair"
},
{
"house": "house1",
"age": "over50",
},
{
"house": "house2",
"condition": "No repair",
},
{
"house": "house2",
"age": "recent"
}
]
}
If condition is "bad" I need to replace with "repair" else if condition is "good" I need to replace with "No repair".
Same type of logic. for the age field. If age is "old" I need to replace with "Over50" and if age is "new" I need to replace with "recent". The two fields(age and condition) are going to be in every iteration
As aled suggested, Map would do this along with OrderBy.
%dw 2.0
output application/json
var a=payload.data
---
"data":(a map{
"house": $.house,
"condition":if (($.condition)=="bad") "repair" else "No repair"
} ++ (a map{
"house": $.house,
"age":if (($.age)=="old") "over50" else "recent"
})orderBy $.house)
Output
{
"data": [
{
"house": "house1",
"condition": "repair"
},
{
"house": "house1",
"age": "over50"
},
{
"house": "house2",
"condition": "No repair"
},
{
"house": "house2",
"age": "recent"
}
]
}
Alternative solution with a single map (you map to an array of the 2 objects required and then flatten the result):
%dw 2.0
output application/json
fun mapCondition(cond : String) = if (cond == "bad") "repair" else "No repair"
fun mapAge(age : String) = if (age == "old") "over50" else "recent"
---
{
data : flatten(payload.data
map ((item, index) ->
[
{
house: item.house,
condition: mapCondition(item.condition)
},
{
house: item.house,
age: mapAge(item.age)
}
]))
}
Use map() over data array and use if conditions to transform the field of each element inside the map. Seems pretty direct.
You can also apply map and flatten with a single call to flatMap.
%dw 2.0
output application/json
---
data: payload.data flatMap [
{
house: $.house,
condition: $.condition match {
case 'bad' -> 'Repair'
else -> 'No Repair'
}
},
{
house: $.house,
age: $.age match {
case 'old' -> 'over50'
else -> 'new'
}
}
]
In the below array there can be multiple objects, and each object can have more than below mentioned key values. I need only the uniqueID and name fields to be displayed.
[
{
"uniqueID": "1",
"Name": "Annie",
"Standard": "3",
"School" : "ABC School"
},
{
"uniqueID": "2",
"Name": "Apoo",
"Standard": "4",
"School" : "PQR School"
},
{
"uniqueID": "3",
"Name": "Xavier",
"Standard": "5",
"School" : "MNO School"
}
]
Desired output:
{
"errors": [
{
"uniqueID": "1",
"Name": "Annie"
},
{
"uniqueID": "2",
"Name": "Apoo"
},
{
"uniqueID": "3",
"Name": "Xavier"
}
]
}
Couple of approaches.
Script
%dw 2.0
output application/json
---
errors: payload map {
uniqueID: $.uniqueID,
Name: $.Name
}
%dw 2.0
output application/json
---
errors: payload map {
($ - "Standard" - "School")
}
In addition to Salim Khan's answer, another way is to use filterObject() to let only the desired attributes in the output. The advantage is that it is somewhat more generic if other attributes are added or changed.
%dw 2.0
output application/json
var allowedKeydNames=["uniqueID", "Name"]
---
{
errors: payload map (
$ filterObject ((value, key, index) -> (allowedKeydNames contains (key as String)))
)
}
I am using dataweave diff function to compare two json objects. Diff is functioning correctly and returning the items that have changed, but it isn't returning anything but the differences. It is returning the path of the item that has changed, but I am not finding a good way to navigate to a string based path. I am only showing one group in the example but there could be many groups by different unique names.
{
"email": "someone#somewhere.com",
"frequency": 1,
"group1": [],
"group2": [],
etc...
)
So here is an example of what I am trying to describe:
Payload 1
{
"email": "someone#somewhere.com",
"frequency": 1,
"group1": [
{
"name": "item1",
"enabled": true
},
{
"name": "item2",
"enabled": true
},
{
"name": "item3",
"enabled": true
}
]
}
Payload 2:
{
"email": "someone#somewhere.com",
"frequency": 1,
"group1": [
{
"name": "item1",
"enabled": true
},
{
"name": "item2",
"enabled": true
},
{
"name": "item3",
"enabled": false
}
]
}
In this case diff between payload 1 and payload 2 would return
{
"matches": false,
"diffs": [
{
"expected": "true",
"actual": "false",
"path": "(root).group1[2].enabled"
}
]
}
I need to reach back into the original payload and navigate to (root).order[2] to get the name key/value pair. What is the best way to do this?
So at the end of the day I need the payload to be:
[
{
name: "item3",
enabled: false,
level: 1
}
]
I appreciate any assistance you can offer. I am also doing a map of the payload level so I know which piece of data to update when I traverse the results. So in this case the level would be 1 based on the fact that is an array off the main object.
This is my current dataweave I just need the first element to be dynamic based on the path that the diff returns
%dw 2.0
output application/json
---
payload.diffs map (item, index) -> {
name: initialPayload.group1[2].name, <-needs to be dynamic based on path
value: (item.actual) replace "\"" with "",
level: sizeOf(item.path splitBy ".")-1
}
I was able to resolve the issue with this weave.
%dw 2.0
fun GetValue(group: String, level: Number, itemkey: String) =
GetKeyValue((((group) splitBy /\./ reduce (e, acc = vars.initialPayload) -> acc[e])[level]), itemkey)
fun GetKeyValue(obj: Object, itemKey: String) =
(entriesOf(obj) filter (item) -> item.key as String == itemKey).value[0]
output application/json skipNullOn="everywhere"
---
{
changes: payload.diffs map (item, index) -> {
name:
if (sizeOf(item.path splitBy ".") - 1 == 1)
((item.path splitBy ".")[1])
else
GetValue((((item.path splitBy ".")[1]) splitBy "[")[0], (((item.path splitBy ".")[1]) splitBy "[")[1] replace "]" with "", "name"),
value: item.actual replace "\"" with "",
level: (sizeOf(item.path splitBy ".") - 1),
}
}
I'm trying to use UpdateRecord 1.9.0 processor to modify a JSON but it does not replace the values as I want.
this is the source message
{
"type": "A",
"ids": [{
"id": "1",
"value": "abc"
}, {
"id": "2",
"value": "def"
}, {
"id": "3",
"value": "ghi"
}
]
}
and the wanted output
{
"ids": [{
"userId": "1",
}, {
"userId": "2",
}, {
"userId": "3",
}
]
}
I have configured the processor as follows
processor config
Reader:
reader
Schema registry:
schema
writer:
writer
And it works, the output is a JSON without the field 'type' and the ids have the field 'userId' instead 'id' and 'value'.
To fill the value of userId, I defined the replace strategy and the property to replace:
strategy
But the output is wrong. The userId is always filled with the id of the last element in the array:
{
"ids": [{
"userId": "3"
}, {
"userId": "3"
}, {
"userId": "3"
}
]
}
I think the value of the expression is ok because if I try to replace only one record it works fine (/ids[0]/userId, ..id)
Nifi docs has a really similar example (example 3):
https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.7.1/org.apache.nifi.processors.standard.UpdateRecord/additionalDetails.html
But it does not work for me.
What am I doing wrong?
thanks
Finally I have used JoltJSONTransform processor instead UpdateRecord
JoltJSONTransform
template:
[
{
"operation": "shift",
"spec": {
"ids":{
"*":{
"id": "ids[&1].userId"
}
}
}
}
]
Easier than UpdateRecord
My Input
{
"Root": {
"order": [
{
"locale": "en-US",
"orderItems": [
{
"product": {
"partNumber": "23853864"
},
"itemSpecifics": {
"options": {
"color": "Olive",
"size": "S"
},
"actualPrice": "7",
"customItemData": {
"TEMP_8401": "8.95",
"TEMP_150207": "3.00"
}
}
}
]
}
... Large amount of JSON Data ...
]
}
}
Expected output
{
"Root": {
"order": [
{
"locale": "en-US",
"orderItems": [
{
"product": {
"partNumber": "23853864"
},
"itemSpecifics": {
"options": {
"color": "Olive",
"size": "S"
},
"actualPrice": "7",
"customItemData": {
"8401": "8.95",
"150207": "3.00"
}
}
}
]
}
... Large amount of JSON Data ...
]
}
}
I want to remove "TEMP_" in the "customItemData" object keys, but I don't want to manually remap the entire JSON object again, assigning properties one by one. Is there any alternative? Any shorter logic in DataWeave? I'm using Mule 3.9.0.
This uses recursion and pattern matching to walk through the data structure and modify the keys. If the key doesn't contain the string "TEMP" it leaves it as is, if it does, it modifies it according to your requirements.
%dw 1.0
%output application/json
%function applyToKeys(e, fn)
e match {
:array -> $ map ((v) -> applyToKeys(v, fn)),
:object -> $ mapObject ((v, k) -> {(fn(k)): applyToKeys(v, fn)}),
default -> $
}
---
applyToKeys(payload,
((key) -> ((key as :string) splitBy "_" )[1]
when ((key as :string) contains "TEMP")
otherwise key))
Please keep in mind the tradeoffs when using a solution like this. The code is certainly less verbose, but it requires a solid understanding of advanced concepts like recursion, pattern matching, lambdas, and higher order functions.