Derive parent-child hierarchy in JSON from an array using DataWeave - mule

Hopefully this is the last of my questions related to this and this
The earlier questions were related to generating XML payload while this query is around generating JSON o/p in a specific format...
Thanks to #Harshank and #sudhish_s was able to make some progress with XML o/p. However I am not able to figure out how to go about json o/p
Here is the input payload ( have limited it to three rows / json objects but could be in thousands)
[
{
"gp": "S1",
"gp_eye_colour": "blue",
"gp_name" : "John",
"parent": "S1",
"parent_eye_colour" : "blue",
"parent_name" : "Sam",
"child": "C1",
"child_eye_colour" : "black",
"child_name" : "C1-name"
},
{
"gp": "S1",
"gp_eye_colour": "blue",
"gp_name" : "John",
"parent": "P1",
"parent_eye_colour" : "blue",
"parent_name" : "Don",
"child": "C1",
"child_eye_colour" : "brown",
"child_name" : "C1-name"
} ,
{
"gp": "S2",
"gp_eye_colour": "blue",
"gp_name" : "David",
"parent": "P2",
"parent_eye_colour" : "blue",
"parent_name" : "Martha",
"child": "C1",
"child_eye_colour" : "brown",
"child_name" : "C1-name"
}
]
Desired o/p is as below :
{
"S2_blue": {
"code": "S2",
"eyeColour": "blue",
"name": "David",
"hierarchy": [
{
"code": "P2",
"eyeColour": "blue",
"name": "Martha",
"hierarchy": [
{
"code": "C1",
"eyeColour": "brown",
"name": "C1-name",
"hierarchy": []
}
]
}
]
},
"S1_blue": {
"code": "S1",
"eyeColour": "blue",
"name": "John",
"hierarchy": [
{
"code": "P1",
"eyeColour": "blue",
"name": "Don",
"hierarchy": [
{
"code": "C1",
"eyeColour": "brown",
"name": "C1-name",
"hierarchy": []
}
]
},
{
"code": "C1",
"eyeColour": "black",
"name": "C1-name",
"hierarchy": []
}
]
}
}
Similar rules need to apply :
Its a Grandparent >> Parent >> Child hierarchy
If Grandparent and Parent share same characteristics (gp == parent and gp_eye_colour == parent_eye_colour then skip parent and directly include child(ren)
( as an example parent Sam is excluded since John (gp) has the same value for gp and eye_colour
Don is not exluded ( same rule as above )
Using the solutions provided to the earlier asked questions - I came up with the following code:
%dw 2.0
output application/json
var hierarchy = ["gp", "parent", "child"]
fun getDirectGeanologies(records, level) = do {
var hLevel = hierarchy[level]
var xyz = if(level == 0)
(records groupBy ((item, index) -> item.gp ++ "_" ++ item.gp_eye_colour) )
else if(level == 1)
(records groupBy ((item, index) -> item.parent ++ "_" ++ item.parent_eye_colour))
else
(records groupBy ((item, index) -> item.child ++ "_" ++ item.child_eye_colour))
---
xyz mapObject ((children, code) ->
(code): {
code: children[0][hLevel],
eyeColour: children[0][hLevel ++ "_eye_colour"],
name: children[0][hLevel ++ "_name"],
hierarchy:
if (level == sizeOf(hierarchy) - 1) [] // if level = 2 ( child stop recursion)
else do {
var nextLevel = level + 1
var nextGen = if(nextLevel == 1)(children groupBy ((item, index) -> item.parent ++ "_" ++ item.parent_eye_colour)) else (children groupBy ((item, index) -> item.child ++ "_" ++ item.child_eye_colour))
---
nextGen mapObject ((nextGenChildren, nextGenCode) ->
if (nextGenCode == code)
getDirectGeanologies (nextGenChildren, nextLevel + 1 ) // skip parent
else
getDirectGeanologies (nextGenChildren, nextLevel)
)
}
}
)
}
---
getDirectGeanologies(payload,0)
However it is not producing desired o/p :
{
"S1_blue": {
"code": "S1",
"eyeColour": "blue",
"name": "John",
"hierarchy": {
"C1_black": {
"code": "C1",
"eyeColour": "black",
"name": "C1-name",
"hierarchy": [
]
},
"P1_blue": {
"code": "P1",
"eyeColour": "blue",
"name": "Don",
"hierarchy": {
"C1_brown": {
"code": "C1",
"eyeColour": "brown",
"name": "C1-name",
"hierarchy": [
]
}
}
}
}
},
"S2_blue": {
"code": "S2",
"eyeColour": "blue",
"name": "David",
"hierarchy": {
"P2_blue": {
"code": "P2",
"eyeColour": "blue",
"name": "Martha",
"hierarchy": {
"C1_brown": {
"code": "C1",
"eyeColour": "brown",
"name": "C1-name",
"hierarchy": [
]
}
}
}
}
}
}
The issues here are :
hierarchy inside gp should be an array but I am getting a object
Not 100% sure but because of above problem rather than having objects inside an array against hierarchy I am getting an object with key as parent/child and eye colour ( Ex : C1_black , P1_blue etc )
So not really sure how to get this part and I am having a tough time to visualise the recursive behaviour
Thanks in advance and once again really appreciate the help already provided by #Harshank and #sudhish_s

I have answered this in mulesoft help forum. LInk MuleSoft Help Forum link

Related

Mule transform JSON to Map [duplicate]

This question already has answers here:
Convert JSON to Map
(19 answers)
Closed 3 months ago.
I have this JSON object
"actos-evaluacion" : [
{
   "subject" : < id-subject>,
   "name" : < name>,
   "marks" : ["<student_id>:< mark>","< student_id>:< mark>",
     "<student_id>:< mark>"]
},
...
{
   "subject" : < id-subject>,
   "name" : < name>,
   "marks" : ["<student_id>:< mark>","< student_id>:< mark>",
     "<student_id>:< mark>"]
},
]
}
And I want to convert it into this Map, so there will be more Maps than JSON, because each JSON will split into different maps depending on the number of students it has.
{
id_subject :
name :
student_id :
mark :
}
I've tried putting a foreach(json) where subject_id and name where stored in variables. Then set_payload with the list and another foreach for the elements of the list but I don't know how to recover the variables into a Map operator to join them all
A DataWeave script can generate the output. I had to make some assumptions of what the input and output actually look to illustrate the method to loop over the internal array inside each element of the first array. You can adapt the method if the samples are not exactly what you use. My script generates an array of DataWeave objects that are equivalent to a Java Map.
Input:
{
"actos-evaluacion" : [
{
"subject" : "id1",
"name": "name1",
"marks": [ "student1:mark1", "student2:mark2", "student1:mark3"]
},
{
"subject" : "id2",
"name" : "name2",
"marks": [ "student1:mark1", "student2:mark2", "student1:mark3"]
}
]
}
DataWeave script:
%dw 2.0
output application/json
---
payload."actos-evaluacion" flatMap ((item, index) ->
item.marks map ((subject, index2) -> {
id_subject: item.subject,
name: item.name,
student_id: (subject splitBy ":") [0],
mark: (subject splitBy ":")[1]
})
)
Output:
[
{
"id_subject": "id1",
"name": "name1",
"student_id": "student1",
"mark": "mark1"
},
{
"id_subject": "id1",
"name": "name1",
"student_id": "student2",
"mark": "mark2"
},
{
"id_subject": "id1",
"name": "name1",
"student_id": "student1",
"mark": "mark3"
},
{
"id_subject": "id2",
"name": "name2",
"student_id": "student1",
"mark": "mark1"
},
{
"id_subject": "id2",
"name": "name2",
"student_id": "student2",
"mark": "mark2"
},
{
"id_subject": "id2",
"name": "name2",
"student_id": "student1",
"mark": "mark3"
}
]

Transformation of Json array in Dataweave

How to write Dataweave transformation in Anytime Studio for given input and output of Json array.
Input:
{
"result": [{
"Labels": [{
"value": [{
"fieldName": "firstName",
"value": "John"
},
{
"fieldName": "lastName",
"value": "Doe"
},
{
"fieldName": "fullName",
"value": "John Doe"
}
]
}]
}]
}
Output:
{
"result": [{
"Labels": [{
"value": [{
"firstName": "John",
"lastName": "Doe",
"fullName": "John Doe"
}]
}]
}]
}
https://docs.mulesoft.com/dataweave/2.4/dw-core-functions-reduce Reduce function might be the one should be used
Thank you in advance
You can just use map to map all the arrays to required format. For the value part you can map the values as fieldName: value array and deconstruct them to an object by wrapping the array around parentheses
%dw 2.0
output application/json
---
{
result: payload.result map ((item) -> {
Labels: item.Labels map ((label) -> {
value: [
{
(label.value map ((field) ->
(field.fieldName): field.value
)) //wrap the array, i.e. lavel.value map ... in parentheses so that it will give you individual key pair.
}
]
})
})
}
You can try below if you are aware that the keyNames will not change:
%dw 2.0
output application/json
---
payload update {
case res at .result -> res map (res, resIndex) -> (res update {
case lbl at .Labels -> lbl map (lbl, lblIndex) -> (lbl update {
case val at .value -> [
(val reduce ((item, acc = {}) -> acc ++ {
(item.fieldName): (item.value)
}))
]
}
)
}
)
}
Here's 2 caveats and a solution. Your input and output files, both are not valid JSON.
Input file, in your "result" object, "Labels" need curly braces {} since they are objects. Key-value pairs should look like this {key:value} not like that key:value
Output file, inside your "value" arrays, key-value pairs need to have the curlies {key:value}
So here's a valid JSON version of your input
{
"result": [
{"Labels": [
{
"value": [
{"fieldName": "firstName","value": "John"},
{"fieldName": "lastName","value": "Doe"},
{"fieldName": "fullName","value": "John Doe"}
]
}
]},
{"Labels": [
{
"value": [
{"fieldName": "firstName","value": "John"}
]
}
]}
]}
Here's a solution
%dw 2.0
import keySet from dw::core::Objects
// this is "result"
var layer1key = keySet(payload)[0]
// this is "Labels" and grabs the first Labels, so assumes Labels doesn't change
var layer2 = payload[layer1key]
var layer2key = keySet(layer2[0])[0]
// this is "value"
var layer3 = layer2[layer2key]
var layer3key = keySet(layer3[0][0])[0]
// this is "fieldName" and "value"
var layer4 = layer3 map (x) -> x['value']
var data1 = ((layer1key) : layer4 map (x) -> {
(layer2key): x map (y) -> {
(layer3key): y map (z) -> {
(z['fieldName']):z['value']
}
}
})
output application/json
---
data1
And a valid JSON version of your output
{
"result": [
{
"Labels": [
{
"value": [
{
"firstName": "John"
},
{
"lastName": "Doe"
},
{
"fullName": "John Doe"
}
]
}
]
},
{
"Labels": [
{
"value": [
{
"firstName": "John"
}
]
}
]
}
]
}

How to inherit parent elements for each existing child Mule 4

There is an array of parent elements that you need to duplicate for each child sku element you have.
Input:
{
"items": [
{
"order": "ASD51247",
"reference": "271559410",
"date": "2022-04-01T22:58:19.077Z",
"lines": [
{
"number": "6578523423489",
"description": "ITEM 1"
}
],
"received": "2022-03-31T09:46:13.260Z",
"created": "2022-03-31T09:46:13.109Z"
},{
"order": "AST927353",
"reference": "271944522",
"date": "2022-04-01T22:58:19.038Z",
"lines": [
{
"number": "4563252696546",
"description": "ITEM 2"
},
{
"number": "890456234326",
"description": "ITEM 3"
}
],
"received": "2022-03-31T10:25:08.508Z",
"created": "2022-03-31T10:25:08.353Z"
}
]}
Expected Output:
{ "item":[
{
"order":"ASD51247",
"reference":"271559410",
"date":"2022-04-01T22:58:19.077Z",
"number":"6578523423489",
"description":"ITEM 1",
"receivedAt":"2022-03-31T09:46:13.260Z",
"createdAt":"2022-03-31T09:46:13.109Z"
},
{
"order":"AST927353",
"reference":"271944522",
"date":"2022-04-01T22:58:19.038Z",
"number":"4563252696546",
"description":"ITEM 2",
"received":"2022-03-31T10:25:08.508Z",
"created":"2022-03-31T10:25:08.353Z"
},
{
"order":"AST927353",
"reference":"271944522",
"date":"2022-04-01T22:58:19.038Z",
"number":"890456234326",
"description":"ITEM 3",
"received":"2022-03-31T10:25:08.508Z",
"created":"2022-03-31T10:25:08.353Z"
}]}
In output, must be inherit the parent elements for each child in the "lines" array. Any help would be appreciated. Thank you.
%dw 2.0
output application/json
---
item: payload.items flatMap ((item, index) ->
(item.lines map {
order: item.order,
reference:item.reference,
date: item.date,
number: $.number,
description: $.description,
received: item.received,
created:item.created
})
)
Modifying Imtiyaz's answer a bit
%dw 2.0
output application/json
---
{
items: (payload.items ) flatMap ((item, index) -> (
item.lines map ((line) -> (item - 'lines' - "received" - "created ") ++ {
description: line.description,
number: line.number,
created: item.created,
received: item.received
})
))
}
try below script.
%dw 2.0
output application/json
---
{
items: payload.items flatMap ((item, index) -> (
item.lines map ((line) -> (item - 'lines') ++ {
description: line.description
})
))
}

How to get all values where a certain Key matches in Dataweave 2.0?

Payload :
[
{
"Contacts": "123456,098765",
"Emails" : ""
},
{
"Contacts": "ABC123",
"Emails" : ""
}
]
How can I get a list of all emails from the below array of objects where the contact Id matches from each row in the payload? (Expected output below)
Variable accConts
{
"queryResponse": [
{
"Email": "test123#test.com",
"SalesforceId": "123456"
},
{
"Email": "test#test.com",
"SalesforceId": "098765"
},
{
"Email": "ABC#test.com",
"SalesforceId": "ABC123"
}
]
}
Expected Output:
[
{
"Contacts": "123456,098765",
"Emails" : "test123#test.com, test#test.com"
},
{
"Contacts": "ABC123",
"Emails" : "ABC#test.com"
}
]
HTH..
%dw 2.0
output application/json
var qResp ={
"queryResponse": [
{
"Email": "test123#test.com",
"SalesforceId": "123456"
},
{
"Email": "test#test.com",
"SalesforceId": "098765"
},
{
"Email": "ABC#test.com",
"SalesforceId": "ABC123"
}
]
}
---
payload filter ($.Contacts != null) map using (iter = $$) {
"Contacts" : $.Contacts,
"Emails": (qResp.queryResponse filter (payload[iter].Contacts contains $.SalesforceId)) reduce ((item,acc = "") -> (acc ++ "," ++ item.Email)[1 to -1]
)
}
I accepted Salim Khan's answer as he guided me in the right direction and the logic to get emails worked. I just needed to rework the map logic,
payload map (row, index) -> {
"Contacts" : row."Contacts",
"Emails" : (qResp.queryResponse filter (row."Contacts" contains $.SalesforceId)) reduce ((item,acc = "") -> (acc ++ "," ++ item.Email)[1 to -1]
),
}
Hopefully this comaprision helps
Wanted to add a slightly more succinct solution to show another approach.
%dw 2.0
output application/json
var qResp =
{
"queryResponse": [
{
"Email": "test123#test.com",
"SalesforceId": "123456"
},
{
"Email": "test#test.com",
"SalesforceId": "098765"
},
{
"Email": "ABC#test.com",
"SalesforceId": "ABC123"
}
]
}
---
payload map (value) ->
{
'Contacts':value.Contacts,
'Emails': qResp.queryResponse[?(value.Contacts contains $.SalesforceId)]..Email joinBy ", "
}

Transform JSON response with lodash

I'm new in lodash (v3.10.1), and having a hard time understanding.
Hope someone can help.
I have an input something like this:
{
{"id":1,"name":"Matthew","company":{"id":1,"name":"abc","industry":{"id":5,"name":"Medical"}}},
{"id":2,"name":"Mark","company":{"id":1,"name":"abc","industry":{"id":5,"name":"Medical"}}},
{"id":3,"name":"Luke","company":{"id":1,"name":"abc","industry":{"id":5,"name":"Medical"}}},
{"id":4,"name":"John","company":{"id":1,"name":"abc","industry":{"id":5,"name":"Medical"}}},
{"id":5,"name":"Paul","company":{"id":1,"name":"abc","industry":{"id":5,"name":"Medical"}}}
];
I would like to output this or close to this:
{
"industries": [
{
"industry":{
"id":5,
"name":"Medical",
"companies": [
{
"company":{
"id":1,
"name":"abc",
"employees": [
{"id":1,"name":"Matthew"},
{"id":2,"name":"Mark"},
{"id":3,"name":"Luke"},
{"id":4,"name":"John"},
{"id":5,"name":"Paul"}
]
}
}
]
}
}
]
}
Here's something that gets you close to what you want. I structured the output to be an object instead of an array. You don't need the industries or industry properties in your example output. The output structure looks like this:
{
"industry name": {
"id": "id of industry",
"companies": [
{
"company name": "name of company",
"id": "id of company",
"employees": [
{
"id": "id of company",
"name": "name of employee"
}
]
}
]
}
}
I use the _.chain function to wrap the collection with a lodash wrapper object. This enables me to explicitly chain lodash functions.
From there, I use the _.groupBy function to group elements of the collection by their industry name. Since I'm chaining, I don't have to pass in the array again to the function. It's implicitly passed via the lodash wrapper. The second argument of the _.groupBy is the path to the value I want to group elements by. In this case, it's the path to the industry name: company.industry.name. _.groupBy returns an object with each employee grouped by their industry (industries are keys for this object).
I then do use _.transform to transform each industry object. _.transform is essentially _.reduce except that the results returned from the _.transform function is always an object.
The function passed to the _.transform function gets executed against each key/value pair in the object. In the function, I use _.groupBy again to group employees by company. Based off the results of _.groupBy, I map the values to the final structure I want for each employee object.
I then call the _.value function because I want to unwrap the output collection from the lodash wrapper object.
I hope this made sense. If it doesn't, I highly recommend reading Lo-Dash Essentials. After reading the book, I finally got why lodash is so useful.
"use strict";
var _ = require('lodash');
var emps = [
{ "id": 1, "name": "Matthew", "company": { "id": 1, "name": "abc", "industry": { "id": 5, "name": "Medical" } } },
{ "id": 2, "name": "Mark", "company": { "id": 1, "name": "abc", "industry": { "id": 5, "name": "Medical" } } },
{ "id": 3, "name": "Luke", "company": { "id": 1, "name": "abc", "industry": { "id": 5, "name": "Medical" } } },
{ "id": 4, "name": "John", "company": { "id": 1, "name": "abc", "industry": { "id": 5, "name": "Medical" } } },
{ "id": 5, "name": "Paul", "company": { "id": 1, "name": "abc", "industry": { "id": 5, "name": "Medical" } } }
];
var result = _.chain(emps)
.groupBy("company.industry.name")
.transform(function(result, employees, industry) {
result[industry] = {};
result[industry].id = _.get(employees[0], "company.industry.id");
result[ industry ][ 'companies' ] = _.map(_.groupBy(employees, "company.name"), function( employees, company ) {
return {
company: company,
id: _.get(employees[ 0 ], 'company.id'),
employees: _.map(employees, _.partialRight(_.pick, [ 'id', 'name' ]))
};
});
return result;
})
.value();
Results from your example are as follows:
{
"Medical": {
"id": 5,
"companies": [
{
"company": "abc",
"id": 1,
"employees": [
{
"id": 1,
"name": "Matthew"
},
{
"id": 2,
"name": "Mark"
},
{
"id": 3,
"name": "Luke"
},
{
"id": 4,
"name": "John"
},
{
"id": 5,
"name": "Paul"
}
]
}
]
}
}
If you ever wanted the exact same structure as in the questions, I solved it using the jsonata library:
(
/* lets flatten it out for ease of accessing the properties*/
$step1 := $ ~> | $ |
{
"employee_id": id,
"employee_name": name,
"company_id": company.id,
"company_name": company.name,
"industry_id": company.industry.id,
"industry_name": company.industry.name
},
["company", "id", "name"] |;
/* now the magic begins*/
$step2 := {
"industries":
[($step1{
"industry" & $string(industry_id): ${
"id": $distinct(industry_id)#$I,
"name": $distinct(industry_name),
"companies": [({
"company" & $string(company_id): {
"id": $distinct(company_id),
"name": $distinct(company_name),
"employees": [$.{
"id": $distinct(employee_id),
"name": $distinct(employee_name)
}]
}
} ~> $each(function($v){ {"company": $v} }))]
}
} ~> $each(function($v){ {"industry": $v} }))]
};
)
You can see it in action on the live demo site: https://try.jsonata.org/VvW4uTRz_