Mule Nested JSON data issue - If Child does not exists in data mule output parent and child columns data NULL. How to capture parent columns data if child data not exists.
JSON Data :
{
"Report_Entry": [{
"p_sname": "TEST_1",
"p_sno": "1234",
"SUBJECT_DATA_GROUP": [
{
"sub_name": "social",
"sub_marks: "80"
},
{
"sub_name": "science",
"sub_marks: "60"
},
{
"sub_name": "maths",
"sub_marks: "90"
}
],
"p_s_start_date": "10-10-1991",
"p_s_status": "A"
},
{
"p_sname": "TEST_2",
"p_sno": "9999",
"SUBJECT_DATA_GROUP": [
{
"sub_name": "social",
"sub_marks: "30"
},
{
"sub_name": "science",
"sub_marks: "40"
},
{
"sub_name": "maths",
"sub_marks: "50"
}
],
"p_s_start_date": "10-10-1999",
"p_s_status": "A"
},
{
"p_sname": "TEST_3",
"p_sno": "8888",
"p_s_start_date": "10-10-2000",
"p_s_status": "A"
},
{
"p_sname": "TEST_4",
"p_sno": "6666",
"p_s_start_date": "10-10-2020",
"p_s_status": "A"
}
]
}
%dw 2.0
output application/java
---
payload.Report_Entry flatMap ((item, order) ->
item.SUBJECT_DATA_GROUP map ((line, lineOrder) -> {
//Main
"p_sname": item.p_sname,
"p_sno": item.p_sno,
"Hours": item.Hours,
//Line
"sub_name": line.sub_name,
"sub_marks": line.sub_marks,
//Main
"p_s_start_date": item.p_s_start_date,
"p_s_status": item.p_s_status
}
)
)
Result :
p_sname|p_sno|sub_name|sub_marks|p_s_start_date|p_s_status
TEST_1|1234|social|80|10-10-1991|A
TEST_1|1234|science|60|10-10-1991|A
TEST_1|1234|maths|90|10-10-1991|A
TEST_2|9999|maths|30|10-10-1999|A
TEST_2|9999|maths|40|10-10-1999|A
TEST_2|9999|maths|50|10-10-1999|A
null|null|null|null|null|null
null|null|||null|null
Expected Result :
p_sname|p_sno|sub_name|sub_marks|p_s_start_date|p_s_status
TEST_1|1234|social|80|10-10-1991|A
TEST_1|1234|science|60|10-10-1991|A
TEST_1|1234|maths|90|10-10-1991|A
TEST_2|9999|maths|30|10-10-1999|A
TEST_2|9999|maths|40|10-10-1999|A
TEST_2|9999|maths|50|10-10-1999|A
TEST_3|8888|||10-10-2000|A
TEST_4|6666|||10-10-2020|A
You can use default keyword to set array with an empty object to handle scenarios where SUBJECT_DATA_GROUP doesn't exists.
I have used application/csv as the mimetype to match the given output.
%dw 2.0
output application/csv separator="|"
---
payload.Report_Entry flatMap ((item) ->
item.SUBJECT_DATA_GROUP default [{}] map ((line) -> {
p_sname: item.p_sname,
p_sno: item.p_sno,
Hours: item.Hours,
sub_name: line.sub_name,
sub_marks: line.sub_marks,
p_s_start_date: item.p_s_start_date,
p_s_status: item.p_s_status
})
)
Related
I have a json structure as follows:
[
{
"apiId": 18211158,
"policyName0": "cors"
},
{
"apiId": 18211158,
"policyName1": "client-id-enforcement"
},
{
"apiId": 18211150,
"policyName0": "client-id-enforcement"
},
{
"apiId": 18211162,
"policyName0": "client-id-enforcement"
},
{
"apiId": 18211162,
"policyName1": "cors"
},
{
"apiId": 18211162,
"policyName2": "oauth"
}
]
I need to transform this into following using dataweave 2.0
[
{
"apiId": 18211158
"policyName0": "cors",
"policyName1": "client-id-enforcement",
},
{
"apiId": 18211150
"policyName0": "client-id-enforcement",
},
{
"apiId": 18211162
"policyName0": "client-id-enforcement",
"policyName1": "cors",
"policyName2": "oAuth",
}
]
Note that each apiID can have multiple policies and the policyName json attribute is generated and cal have values ranging from 1 to 9.
How can I condense the json based on apiID as the key in dataweave?
The best way to do this is by using a groupBy with a pluck to reshape the groups into objects
%dw 2.0
output application/json
---
payload
groupBy ((value, key) -> value.apiId)
pluck ((value, key, index) -> {
"apiId": key,
(value map ((item, index) -> item - "apiId"))
})
Try the below code
%dw 2.0
output application/json skipNullOn="everywhere"
---
payload groupBy $."apiId" pluck $ map {
apiId: $."apiId"[0],
policyName0: $.policyName0[0],
policyName1: $.policyName1[0],
policyName1: $.policyName2[0]
}
Output As below
[
{
"apiId": 18211158,
"policyName0": "cors",
"policyName1": "client-id-enforcement"
},
{
"apiId": 18211150,
"policyName0": "client-id-enforcement"
},
{
"apiId": 18211162,
"policyName0": "client-id-enforcement",
"policyName1": "cors",
"policyName1": "oauth"
}
]
How to write Dataweave transformation in Anytime Studio for given input and output of Json array.
Input:
{
"result": [{
"Labels": [{
"value": [{
"fieldName": "firstName",
"value": "John"
},
{
"fieldName": "lastName",
"value": "Doe"
},
{
"fieldName": "fullName",
"value": "John Doe"
}
]
}]
}]
}
Output:
{
"result": [{
"Labels": [{
"value": [{
"firstName": "John",
"lastName": "Doe",
"fullName": "John Doe"
}]
}]
}]
}
https://docs.mulesoft.com/dataweave/2.4/dw-core-functions-reduce Reduce function might be the one should be used
Thank you in advance
You can just use map to map all the arrays to required format. For the value part you can map the values as fieldName: value array and deconstruct them to an object by wrapping the array around parentheses
%dw 2.0
output application/json
---
{
result: payload.result map ((item) -> {
Labels: item.Labels map ((label) -> {
value: [
{
(label.value map ((field) ->
(field.fieldName): field.value
)) //wrap the array, i.e. lavel.value map ... in parentheses so that it will give you individual key pair.
}
]
})
})
}
You can try below if you are aware that the keyNames will not change:
%dw 2.0
output application/json
---
payload update {
case res at .result -> res map (res, resIndex) -> (res update {
case lbl at .Labels -> lbl map (lbl, lblIndex) -> (lbl update {
case val at .value -> [
(val reduce ((item, acc = {}) -> acc ++ {
(item.fieldName): (item.value)
}))
]
}
)
}
)
}
Here's 2 caveats and a solution. Your input and output files, both are not valid JSON.
Input file, in your "result" object, "Labels" need curly braces {} since they are objects. Key-value pairs should look like this {key:value} not like that key:value
Output file, inside your "value" arrays, key-value pairs need to have the curlies {key:value}
So here's a valid JSON version of your input
{
"result": [
{"Labels": [
{
"value": [
{"fieldName": "firstName","value": "John"},
{"fieldName": "lastName","value": "Doe"},
{"fieldName": "fullName","value": "John Doe"}
]
}
]},
{"Labels": [
{
"value": [
{"fieldName": "firstName","value": "John"}
]
}
]}
]}
Here's a solution
%dw 2.0
import keySet from dw::core::Objects
// this is "result"
var layer1key = keySet(payload)[0]
// this is "Labels" and grabs the first Labels, so assumes Labels doesn't change
var layer2 = payload[layer1key]
var layer2key = keySet(layer2[0])[0]
// this is "value"
var layer3 = layer2[layer2key]
var layer3key = keySet(layer3[0][0])[0]
// this is "fieldName" and "value"
var layer4 = layer3 map (x) -> x['value']
var data1 = ((layer1key) : layer4 map (x) -> {
(layer2key): x map (y) -> {
(layer3key): y map (z) -> {
(z['fieldName']):z['value']
}
}
})
output application/json
---
data1
And a valid JSON version of your output
{
"result": [
{
"Labels": [
{
"value": [
{
"firstName": "John"
},
{
"lastName": "Doe"
},
{
"fullName": "John Doe"
}
]
}
]
},
{
"Labels": [
{
"value": [
{
"firstName": "John"
}
]
}
]
}
]
}
I need to compare two fields in a array and replace their values with different data and output in separate objects
Here is my sample input array:
{
"data": [
{
"house": "house1",
"condition": "bad",
"age": "old",
},
{
"house": "house2",
"condition": "good",
"age": "new",
}
]
}
Output should be
{
"data": [
{
"house": "house1",
"condition": "repair"
},
{
"house": "house1",
"age": "over50",
},
{
"house": "house2",
"condition": "No repair",
},
{
"house": "house2",
"age": "recent"
}
]
}
If condition is "bad" I need to replace with "repair" else if condition is "good" I need to replace with "No repair".
Same type of logic. for the age field. If age is "old" I need to replace with "Over50" and if age is "new" I need to replace with "recent". The two fields(age and condition) are going to be in every iteration
As aled suggested, Map would do this along with OrderBy.
%dw 2.0
output application/json
var a=payload.data
---
"data":(a map{
"house": $.house,
"condition":if (($.condition)=="bad") "repair" else "No repair"
} ++ (a map{
"house": $.house,
"age":if (($.age)=="old") "over50" else "recent"
})orderBy $.house)
Output
{
"data": [
{
"house": "house1",
"condition": "repair"
},
{
"house": "house1",
"age": "over50"
},
{
"house": "house2",
"condition": "No repair"
},
{
"house": "house2",
"age": "recent"
}
]
}
Alternative solution with a single map (you map to an array of the 2 objects required and then flatten the result):
%dw 2.0
output application/json
fun mapCondition(cond : String) = if (cond == "bad") "repair" else "No repair"
fun mapAge(age : String) = if (age == "old") "over50" else "recent"
---
{
data : flatten(payload.data
map ((item, index) ->
[
{
house: item.house,
condition: mapCondition(item.condition)
},
{
house: item.house,
age: mapAge(item.age)
}
]))
}
Use map() over data array and use if conditions to transform the field of each element inside the map. Seems pretty direct.
You can also apply map and flatten with a single call to flatMap.
%dw 2.0
output application/json
---
data: payload.data flatMap [
{
house: $.house,
condition: $.condition match {
case 'bad' -> 'Repair'
else -> 'No Repair'
}
},
{
house: $.house,
age: $.age match {
case 'old' -> 'over50'
else -> 'new'
}
}
]
I am getting multiple json files in a for loop and each json in each loop are in a format :
{
"myproperties": {
"http.port": "8088",
"http.base": "/abc",
"http.path": "test"
},
"information": [{
"abc": {
"key1": "ghghghghgh"
},
"efg": {
"key1": "value1"
}
}]
}
and
{
"myproperties": {
"http.port": "6789",
"db.base": "tat",
"db.path": "ghghghg"
},
"information": [{
"efg": {
"key1": "ghghghghgh"
},
"ijk": {
"key1": "value1"
}
}]
}
and so on ……….
I manage to combine all the json in a list out side the for loop and the combine json list looks like:
[{
"myproperties": {
"http.port": "8088",
"http.base": "/abc",
"http.path": "test"
},
"information": [{
"abc": {
"key1": "ghghghghgh"
},
"efg": {
"key1": "value1"
}
}]
},
{
"myproperties": {
"http.port": "6789",
"db.base": "tat",
"db.path": "ghghghg"
},
"information": [{
"efg": {
"key1": "ghghghghgh"
},
"ijk": {
"key1": "value1"
}
}]
}]
Now I want to make **single** json output out of this combine json something in a following format:
{
"myproperties": {
"http.port": "6789",
"db.base": "tat",
"db.path": "ghghghg",
"http.base": "/abc",
"http.path": "test"
},
"information": [{
"efg": {
"key1": "ghghghghgh"
},
"ijk": {
"key1": "value1"
}
},
{
"abc": {
"key1": "ghghghghgh"
},
"efg": {
"key1": "value1"
}
}
]
}
please note in the myproperties section only unique and distinct node is there.
I am not sure how to begin this with dataweave… any pointer will be appreciated,
I tried the following :
%dw 1.0
%output application/json skipNullOn="everywhere",encoding="UTF-8"
---
payload map {
myproperties: $.myproperties,
information: $.information
}
But not working
Thanks
Try mapObject for combining properties and flatten for changing information array to single array like
%dw 1.0
%output application/json
---
{
myproperties : payload.myproperties mapObject $ ,
information : (flatten payload.information)
}
HTH
Update:-
For distinct properties you have to get distinct properties first and then map all distinct properties. But you may loose duplicate properties. refer below
%dw 1.0
%output application/json
%var distinctProperties =((payload.myproperties mapObject $) pluck $$) distinctBy $
%var aggregatedPropertiesMap = (payload.myproperties mapObject $)
---
{
myproperties : {( distinctProperties map { // interate over distinct properties
($) : aggregatedPropertiesMap[$] // map property as key and fetch its value from aggregatedPropertiesMap
})} ,
information : (flatten payload.information)
}
Hi I need to transform the following JSON object:
{
"products": [
{
"itemno": "123131",
"description" : "Big Widget",
"attributes": [
{
"color": [
{
"value": "Red",
"codeValue": "RED_NO2"
},
{
"value": "Blue Licorice",
"codeValue": "BLUE-355"
}
]
},
{
"chemicals": [
{
"value": "Red Phosphorous",
"codeValue": "RED_PHOS"
},
{
"value": "Chlorine Bleach",
"codeValue": "CHLRN_BLCH"
}
]
}
]
}
]
}
I am trying to transform this with each attribute having an array of values where their value is the codeValue and it's an array of those string values.
This is the desired output:
{
"products": [
{
"itemno": "123131",
"description: : "Big Widget",
"attributes": [
{
"color": ["RED_NO2", "BLUE-355"]
},
{
"chemicals": ["RED_PHOS", "CHLRN_BLCH"]
}
]
}
]
}
This is the Dataweave. I cannot determine how to get the attribute names (i.e. color, chemicals as keys with the desired data.
There is not a lot of good examples on transforming data with Dataweave and I have spent a lot of time trying to figure this out.
This is one of the dataweaves that got there somewhat, but isn't the solution:
%dw 1.0
%output application/json
---
payload.products map
{
"ItemNo" : $.sku,
"Desc" : $.description,
"Test" : "Value",
"Attributes" : $.attributes map
{
'$$' : $ pluck $.value
}
}
Your help is greatly appreciated.
You can do something like this:
%dw 1.0
%output application/json
%function attributeCodeValues(attributes)
attributes map ((attr) ->
attr mapObject ((values, descriptor) ->
{
(descriptor): values map $.codeValue
}
)
)
---
payload.products map {
"ItemNo" : $.sku,
"Desc" : $.description,
"Test" : "Value",
"Attributes" : attributeCodeValues($.attributes)
}