Creating a single json from a combined json list - mule

I am getting multiple json files in a for loop and each json in each loop are in a format :
{
"myproperties": {
"http.port": "8088",
"http.base": "/abc",
"http.path": "test"
},
"information": [{
"abc": {
"key1": "ghghghghgh"
},
"efg": {
"key1": "value1"
}
}]
}
and
{
"myproperties": {
"http.port": "6789",
"db.base": "tat",
"db.path": "ghghghg"
},
"information": [{
"efg": {
"key1": "ghghghghgh"
},
"ijk": {
"key1": "value1"
}
}]
}
and so on ……….
I manage to combine all the json in a list out side the for loop and the combine json list looks like:
[{
"myproperties": {
"http.port": "8088",
"http.base": "/abc",
"http.path": "test"
},
"information": [{
"abc": {
"key1": "ghghghghgh"
},
"efg": {
"key1": "value1"
}
}]
},
{
"myproperties": {
"http.port": "6789",
"db.base": "tat",
"db.path": "ghghghg"
},
"information": [{
"efg": {
"key1": "ghghghghgh"
},
"ijk": {
"key1": "value1"
}
}]
}]
Now I want to make **single** json output out of this combine json something in a following format:
{
"myproperties": {
"http.port": "6789",
"db.base": "tat",
"db.path": "ghghghg",
"http.base": "/abc",
"http.path": "test"
},
"information": [{
"efg": {
"key1": "ghghghghgh"
},
"ijk": {
"key1": "value1"
}
},
{
"abc": {
"key1": "ghghghghgh"
},
"efg": {
"key1": "value1"
}
}
]
}
please note in the myproperties section only unique and distinct node is there.
I am not sure how to begin this with dataweave… any pointer will be appreciated,
I tried the following :
%dw 1.0
%output application/json skipNullOn="everywhere",encoding="UTF-8"
---
payload map {
myproperties: $.myproperties,
information: $.information
}
But not working
Thanks

Try mapObject for combining properties and flatten for changing information array to single array like
%dw 1.0
%output application/json
---
{
myproperties : payload.myproperties mapObject $ ,
information : (flatten payload.information)
}
HTH
Update:-
For distinct properties you have to get distinct properties first and then map all distinct properties. But you may loose duplicate properties. refer below
%dw 1.0
%output application/json
%var distinctProperties =((payload.myproperties mapObject $) pluck $$) distinctBy $
%var aggregatedPropertiesMap = (payload.myproperties mapObject $)
---
{
myproperties : {( distinctProperties map { // interate over distinct properties
($) : aggregatedPropertiesMap[$] // map property as key and fetch its value from aggregatedPropertiesMap
})} ,
information : (flatten payload.information)
}

Related

Mule Nested JSON data issue - If Child does not exists in data mule output parent and child columns data NULL

Mule Nested JSON data issue - If Child does not exists in data mule output parent and child columns data NULL. How to capture parent columns data if child data not exists.
JSON Data :
{
"Report_Entry": [{
"p_sname": "TEST_1",
"p_sno": "1234",
"SUBJECT_DATA_GROUP": [
{
"sub_name": "social",
"sub_marks: "80"
},
{
"sub_name": "science",
"sub_marks: "60"
},
{
"sub_name": "maths",
"sub_marks: "90"
}
],
"p_s_start_date": "10-10-1991",
"p_s_status": "A"
},
{
"p_sname": "TEST_2",
"p_sno": "9999",
"SUBJECT_DATA_GROUP": [
{
"sub_name": "social",
"sub_marks: "30"
},
{
"sub_name": "science",
"sub_marks: "40"
},
{
"sub_name": "maths",
"sub_marks: "50"
}
],
"p_s_start_date": "10-10-1999",
"p_s_status": "A"
},
{
"p_sname": "TEST_3",
"p_sno": "8888",
"p_s_start_date": "10-10-2000",
"p_s_status": "A"
},
{
"p_sname": "TEST_4",
"p_sno": "6666",
"p_s_start_date": "10-10-2020",
"p_s_status": "A"
}
]
}
%dw 2.0
output application/java
---
payload.Report_Entry flatMap ((item, order) ->
item.SUBJECT_DATA_GROUP map ((line, lineOrder) -> {
//Main
"p_sname": item.p_sname,
"p_sno": item.p_sno,
"Hours": item.Hours,
//Line
"sub_name": line.sub_name,
"sub_marks": line.sub_marks,
//Main
"p_s_start_date": item.p_s_start_date,
"p_s_status": item.p_s_status
}
)
)
Result :
p_sname|p_sno|sub_name|sub_marks|p_s_start_date|p_s_status
TEST_1|1234|social|80|10-10-1991|A
TEST_1|1234|science|60|10-10-1991|A
TEST_1|1234|maths|90|10-10-1991|A
TEST_2|9999|maths|30|10-10-1999|A
TEST_2|9999|maths|40|10-10-1999|A
TEST_2|9999|maths|50|10-10-1999|A
null|null|null|null|null|null
null|null|||null|null
Expected Result :
p_sname|p_sno|sub_name|sub_marks|p_s_start_date|p_s_status
TEST_1|1234|social|80|10-10-1991|A
TEST_1|1234|science|60|10-10-1991|A
TEST_1|1234|maths|90|10-10-1991|A
TEST_2|9999|maths|30|10-10-1999|A
TEST_2|9999|maths|40|10-10-1999|A
TEST_2|9999|maths|50|10-10-1999|A
TEST_3|8888|||10-10-2000|A
TEST_4|6666|||10-10-2020|A
You can use default keyword to set array with an empty object to handle scenarios where SUBJECT_DATA_GROUP doesn't exists.
I have used application/csv as the mimetype to match the given output.
%dw 2.0
output application/csv separator="|"
---
payload.Report_Entry flatMap ((item) ->
item.SUBJECT_DATA_GROUP default [{}] map ((line) -> {
p_sname: item.p_sname,
p_sno: item.p_sno,
Hours: item.Hours,
sub_name: line.sub_name,
sub_marks: line.sub_marks,
p_s_start_date: item.p_s_start_date,
p_s_status: item.p_s_status
})
)

Transformation of Json array in Dataweave

How to write Dataweave transformation in Anytime Studio for given input and output of Json array.
Input:
{
"result": [{
"Labels": [{
"value": [{
"fieldName": "firstName",
"value": "John"
},
{
"fieldName": "lastName",
"value": "Doe"
},
{
"fieldName": "fullName",
"value": "John Doe"
}
]
}]
}]
}
Output:
{
"result": [{
"Labels": [{
"value": [{
"firstName": "John",
"lastName": "Doe",
"fullName": "John Doe"
}]
}]
}]
}
https://docs.mulesoft.com/dataweave/2.4/dw-core-functions-reduce Reduce function might be the one should be used
Thank you in advance
You can just use map to map all the arrays to required format. For the value part you can map the values as fieldName: value array and deconstruct them to an object by wrapping the array around parentheses
%dw 2.0
output application/json
---
{
result: payload.result map ((item) -> {
Labels: item.Labels map ((label) -> {
value: [
{
(label.value map ((field) ->
(field.fieldName): field.value
)) //wrap the array, i.e. lavel.value map ... in parentheses so that it will give you individual key pair.
}
]
})
})
}
You can try below if you are aware that the keyNames will not change:
%dw 2.0
output application/json
---
payload update {
case res at .result -> res map (res, resIndex) -> (res update {
case lbl at .Labels -> lbl map (lbl, lblIndex) -> (lbl update {
case val at .value -> [
(val reduce ((item, acc = {}) -> acc ++ {
(item.fieldName): (item.value)
}))
]
}
)
}
)
}
Here's 2 caveats and a solution. Your input and output files, both are not valid JSON.
Input file, in your "result" object, "Labels" need curly braces {} since they are objects. Key-value pairs should look like this {key:value} not like that key:value
Output file, inside your "value" arrays, key-value pairs need to have the curlies {key:value}
So here's a valid JSON version of your input
{
"result": [
{"Labels": [
{
"value": [
{"fieldName": "firstName","value": "John"},
{"fieldName": "lastName","value": "Doe"},
{"fieldName": "fullName","value": "John Doe"}
]
}
]},
{"Labels": [
{
"value": [
{"fieldName": "firstName","value": "John"}
]
}
]}
]}
Here's a solution
%dw 2.0
import keySet from dw::core::Objects
// this is "result"
var layer1key = keySet(payload)[0]
// this is "Labels" and grabs the first Labels, so assumes Labels doesn't change
var layer2 = payload[layer1key]
var layer2key = keySet(layer2[0])[0]
// this is "value"
var layer3 = layer2[layer2key]
var layer3key = keySet(layer3[0][0])[0]
// this is "fieldName" and "value"
var layer4 = layer3 map (x) -> x['value']
var data1 = ((layer1key) : layer4 map (x) -> {
(layer2key): x map (y) -> {
(layer3key): y map (z) -> {
(z['fieldName']):z['value']
}
}
})
output application/json
---
data1
And a valid JSON version of your output
{
"result": [
{
"Labels": [
{
"value": [
{
"firstName": "John"
},
{
"lastName": "Doe"
},
{
"fullName": "John Doe"
}
]
}
]
},
{
"Labels": [
{
"value": [
{
"firstName": "John"
}
]
}
]
}
]
}

How can I transform an array of objects to an array of strings and not lose the key in Dataweave?

Hi I need to transform the following JSON object:
{
"products": [
{
"itemno": "123131",
"description" : "Big Widget",
"attributes": [
{
"color": [
{
"value": "Red",
"codeValue": "RED_NO2"
},
{
"value": "Blue Licorice",
"codeValue": "BLUE-355"
}
]
},
{
"chemicals": [
{
"value": "Red Phosphorous",
"codeValue": "RED_PHOS"
},
{
"value": "Chlorine Bleach",
"codeValue": "CHLRN_BLCH"
}
]
}
]
}
]
}
I am trying to transform this with each attribute having an array of values where their value is the codeValue and it's an array of those string values.
This is the desired output:
{
"products": [
{
"itemno": "123131",
"description: : "Big Widget",
"attributes": [
{
"color": ["RED_NO2", "BLUE-355"]
},
{
"chemicals": ["RED_PHOS", "CHLRN_BLCH"]
}
]
}
]
}
This is the Dataweave. I cannot determine how to get the attribute names (i.e. color, chemicals as keys with the desired data.
There is not a lot of good examples on transforming data with Dataweave and I have spent a lot of time trying to figure this out.
This is one of the dataweaves that got there somewhat, but isn't the solution:
%dw 1.0
%output application/json
---
payload.products map
{
"ItemNo" : $.sku,
"Desc" : $.description,
"Test" : "Value",
"Attributes" : $.attributes map
{
'$$' : $ pluck $.value
}
}
Your help is greatly appreciated.
You can do something like this:
%dw 1.0
%output application/json
%function attributeCodeValues(attributes)
attributes map ((attr) ->
attr mapObject ((values, descriptor) ->
{
(descriptor): values map $.codeValue
}
)
)
---
payload.products map {
"ItemNo" : $.sku,
"Desc" : $.description,
"Test" : "Value",
"Attributes" : attributeCodeValues($.attributes)
}

MuleSoft transform gives error while transforming using DWL for JSON payload

Im am new to MuleSoft.I am trying to transform a JSON payload,using transform.
I want to transform my payload as below
Input:
{
"ResponseStatus": {
"Status": "SUCCESS",
"StatusText": "SUCCESS"
},
"Processes": {
"Process": [
{
"ProcessId": "1234567",
"ProcessProperties": {
"Property": [
{
"Name": "XXXXXXXXXXX",
"Value": "11111111",
"Desc": "YYYYYYYY"
},
{
"Name": "AAAAAAAAA",
"Value": "2222222",
"Desc": "BBBBBBBB"
},
{
"Name": "QQQQQQQQQ",
"Value": "#######",
"Desc": "CCCCCCCC"
},
{
"Name": "NNNNNNN",
"Value": "IIIIIIII",
"Desc": "UYUYUYUY"
}
]
},
"EditMode": "CCCCCC",
"ProcessType": "ABCD",
"AppName": "VFVFVGBG",
"StatusHistory": {
"STS": [
{
"Sts": "COMPLETED"
}
]
}
}
]
}
}
Output:
[
{
"ProcessId": "1234567",
"AAAAAAAAA": "2222222",
"QQQQQQQQQ": "#######"
}
]
I have read DWL reference from below Mulesoft link.Also reffered this SO link.
Below is what I have tried so far,
%dw 1.0
%output application/json
---
{
"ProcessId": (payload.Processes.Process.ProcessId)[0],
AAAAAAAAA: {
(payload.Processes.Process.ProcessProperties.Property mapObject {
($.Name):$.Value when $.Name =="AAAAAAAAA" otherwise ""
})
},
QQQQQQQQQ: {
(payload.Processes.Process.ProcessProperties.Property mapObject {
($.Name):$.Value when $.Name =="QQQQQQQQQ" otherwise ""
})
}
}
I am still not able to get the desired output.
It gives me "Cannot coerce a :array to a :key"
Can anyone please help me?
The "property" json element in your input json is "Array",which it is not able to parse to a single value.
Please try below snippet and let me know if that gives your deisred o/p.
payload.Processes.Process map (
(val , index) ->
{"ProcessId":(payload.Processes.Process.ProcessId)[0]
,
(val.ProcessProperties.Property map {
(($.Name) : $.Value) when $.Name =='AAAAAAAAA' }
),
(val.ProcessProperties.Property map {
(($.Name) : $.Value) when $.Name =='QQQQQQQQQ' }
)
}
)

Combine records into a single JSON object per Category using Dataweave

I am using Anypoint Studio 6.1 and Mule 3.8.1 and in Dataweave I have a CSV file as input and JSON list output.
Multiple records are coming in on the CSV file with the same Product Category Id but will have a different product details associated with it. I want my output to be aggregated for each of these records so I have one object for each Product Category Id which contains a list of the product details for each product under that Product Category id. How can I do this?
Current output:
[
{
"Products": {
"ProductDetails": [
{
"ProductDetail": {
"ProductSubDetails": {
"ProductAmount": 7.50,
"ProductReplenFrequency": "monthly",
"ProductsNotes": "Product Notes 1",
"pick": false
},
"ProductType": "PS4 Game"
},
"ProductSubType": "Game"
}
]
},
"ProductsCategoryId": "ProductS001",
"ProductSubType": "Computers and Games"
},
{
"Products": {
"ProductDetails": [
{
"ProductDetail": {
"ProductSubDetails": {
"ProductAmount": 7.50,
"ProductReplenFrequency": "fortnightly",
"ProductsNotes": "Products Notes 2",
"pick": false
},
"ProductType": "X Box One Game"
},
"ProductSubType": "Game"
}
]
},
"ProductsCategoryId": "ProductS001",
"ProductSubType": "Computers and Games"
}
]
Expected output:
[
{
"Products": {
"ProductDetails": [
{
"ProductDetail": {
"ProductSubDetails": {
"ProductAmount": 7.50,
"ProductReplenFrequency": "monthly",
"ProductsNotes": "Product Notes 1",
"pick": false
},
"ProductType": "PS4 Game"
},
"ProductSubType": "Game"
},
{
"ProductDetail": {
"ProductSubDetails": {
"ProductAmount": 7.50,
"ProductReplenFrequency": "fortnightly",
"ProductsNotes": "Products Notes 2",
"pick": false
},
"ProductType": "X Box One Game"
},
"ProductSubType": "Game"
}
]
},
"ProductsIdentifier": "ProductS001",
"ProductSubType": "Computers and Games"
}
]
Dataweave code:
%dw 1.0
%input payload application/csv
%output application/json skipNullOn = "everywhere" , encoding = "UTF-8"
%var dataLookup = {(payload."ProductsCategoryId" map {($.Id): $.Value})}
---
(payload filter $$ > 2) map ((payload01 , indexOfPayload01) -> {
Products: {
ProductsDetails: [{
ProductsDetail: {
ProductsubDetails: {
ProductsAmount: payload01."Products Amount" as :number,
ProductsFrequency: payload01."Products Frequency"
},
ProductsType: payload01."Products Type"
}
}]
},
ProductsCategoryId: payload01."ProductsCategoryId"
})
Sample Data:
ProductsCategoryId,Product Type,Product Frequency,Product Amount
ProductS001,PS4 Game,Monthly,7.5
ProductS001,X Box One Game,Fortnightly,7.5
ProductS002,Lego,Daily,7
Thanks
You have to use look up for achieving desired output. Please refer the following answer for more details
Lookup list of Maps variable in data weave script
Updated answer as per comments
Try with following mapping. Also I am not sure why you have use filter $$ > 2.
For body starts at line 5 you can add input reader property "bodyStartLineNumber" Please refer link for more details
%dw 1.0
%input payload application/csv
%output application/json skipNullOn = "everywhere" , encoding = "UTF-8"
---
payload groupBy $.ProductsCategoryId map {
Products: {
ProductsDetails: $ map (product , indexOfProduct) -> {
ProductsDetail: {
ProductsubDetails: {
ProductsAmount: product."Product Amount" as :number,
ProductsFrequency: product."Product Frequency"
},
ProductsType: product."Product Type"
}
}
},
ProductsCategoryId: $."ProductsCategoryId"[0]
}
Output gnerated:-
[
{
"Products": {
"ProductsDetails": [
{
"ProductsDetail": {
"ProductsubDetails": {
"ProductsAmount": 7
},
"ProductsType": "Lego"
}
}
]
},
"ProductsCategoryId": "ProductS002"
},
{
"Products": {
"ProductsDetails": [
{
"ProductsDetail": {
"ProductsubDetails": {
"ProductsAmount": 7.5
},
"ProductsType": "PS4 Game"
}
},
{
"ProductsDetail": {
"ProductsubDetails": {
"ProductsAmount": 7.5
},
"ProductsType": "X Box One Game"
}
}
]
},
"ProductsCategoryId": "ProductS001"
}
]
HTH