Merge two json payload with dataweave - mule

I have a Mule flow that calls two REST API's resulting in two json list payloads. I need to obtain a merged json list from these two payloads.
I managed to achieve this with a scatter-gather flow with a custom aggregator strategy. However the code I had to write for this looks rather messy and it's slow.
Is there any way to achieve this using dataweave?`
input Json list 1:
[{
"ID": "1",
"firstName": "JANE""familyName": "DOE",
"Entities": [{
"ID": "515785",
"name": "UNKOWN UNITED",
"callSign": "UU"
}]
},
{
"ID": "2",
"firstName": "JOHN",
"familyName": "DOE",
"Entities": [{
"Entities_ID": "515785",
"name": "UNKOWN UNITED",
"callSign": "UU"
}]
}]
input Json list 2:
[{
"ID": "1",
"firstName": "JANE",
"familyName": "DOE",
"Entities": [{
"Entities_ID": "8916514",
"name": "UNKOWN INCORPORATED",
"callSign": "UI"
}]
},
{
"ID": "4",
"firstName": "JAKE",
"familyName": "DOE",
"Entities": [{
"Entities_ID": "8916514",
"name": "UNKOWN INCORPORATED",
"callSign": "UI"
}]
}]
desired output is a merged list without duplicate IDs:
[{
"ID": "1",
"firstName": "JANE",
"familyName": "DOE",
"Entities": [{
"Entities_ID": "515785",
"name": "UNKOWN UNITED",
"callSign": "UU"
},
{
"Entities_ID": "8916514",
"name": "UNKOWN INCORPORATED",
"callSign": "UI"
}]
},
{
"ID": "2",
"firstName": "JOHN",
"familyName": "DOE",
"Entities": [{
"Entities_ID": "515785",
"name": "UNKOWN UNITED",
"callSign": "UU"
}]
},
{
"ID": "4",
"firstName": "JAKE",
"familyName": "DOE",
"Entities": [{
"Entities_ID": "8916514",
"name": "UNKOWN INCORPORATED",
"callSign": "UI"
}]
}]

This works for me with dataweave where payload is list1 and flowVars.list2 is list2
%dw 1.0
%output application/json
%var mergeddata = flowVars.list2 groupBy $.ID
---
payload map ((data,index) -> {
ID: data.ID,
firstName : data.firstName,
familyName : data.familyName,
Entities : data.Entities ++ (mergeddata[data.ID].Entities default [])
}) ++
(flowVars.list2 filter (not (payload.ID contains $.ID)))
Hope this helps

Related

lodash find object by id across an entire nested object

I would like to find name value at whatever level when searching by ID
I assume Lodash would be the best method, I have been searching and testing but no luck, is this possible?
{
"delegate": [
{
"id": "2",
"name": "David",
"balances": [],
"responsibility": [
{
"id": "18",
"name": "Lilly",
"balances": []
}
]
},
{
"id": "12",
"name": "Barb",
"balances": [],
"responsibility": [
{
"id": "3",
"name": "Amy",
"balances": []
},
{
"id": "9",
"name": "John",
"balances": []
}
]
}
],
"responsibility": [
{
"id": "14",
"name": "Shawn",
"balances": []
}
]
}

Passing a JSON to Scenario Outline

https://github.com/intuit/karate/issues/554
I have a multirow JSON, I would like to iterate for each row using Scenario outline
It is working if I call by the row number, i am using a dynamic JSON file
Data.json
[
{
"address": {
"addressLine1": "ttes",
"addressLine2": "Test"
},
"name": {
"firstName": "TEST",
"lastName": "TEST"
},
"phone": {
"phoneExtension": "1234",
"phoneNumber": "999999999"
},
"email": {
"emailAddress": "TEST#TEST.com"
}
},
{
"address": {
"addressLine1": "ttes1",
"addressLine2": "Test1"
},
"name": {
"firstName": "TEST1",
"lastName": "TEST1"
},
"phone": {
"phoneExtension": "1234",
"phoneNumber": "999999999"
},
"email": {
"emailAddress": "TEST#TEST1.com"
}
}]
Working if
Feature: Read from File
Background:
* def Testdata = read('Data.json')
Scenario Outline: [1] Test with multiple records on JSON
Given url_stg
And path 'test','test'
And request Testdata[0]
When method POST
Examples:
|Testdata |
Working for Testdata[0]. I have dynamic json, I am looking for a way to iterate through the json autoamtically.
My request body should be one row from JSON like :
{
"address": {
"addressLine1": "ttes1",
"addressLine2": "Test1",
},
"name": {
"firstName": "TEST1",
"lastName": "TEST1",
},
"phone": {
"phoneExtension": "1234",
"phoneNumber": "999999999",
},
"email": {
"emailAddress": "TEST#TEST1.com"
}
This will be easier in the next version, see this ticket: https://github.com/intuit/karate/issues/717#issuecomment-489339287
So if you can build from source and test that would help.
For now you have to do this:
And request { address: <address>, name: <name>, phone: <phone>, email: <email> }

Scenario Outline in Karate for multi-line JSON

How to use scenario outline to iterate for a JSON which is more than single cell
https://github.com/intuit/karate#the-cucumber-way
Data.json
[{
"address": {
"addressLine1": "ttes",
"addressLine2": "Test",
"addressLine3": "Test",
"addressType": "business",
"city": "TEST",
"company": "TEST",
"country": "TEST",
"state": "TEST",
"postalCode": "XXXX"
},
"name": {
"firstName": "TEST",
"lastName": "TEST",
"middleInit": "T",
"title": "Mr."
},
"phone": {
"phoneExtension": "1234",
"phoneNumber": "999999999",
"phoneType": "mobile"
},
"email": {
"emailAddress": "TEST#TEST.com"
}
},
{
"address": {
"addressLine1": "ttes1",
"addressLine2": "Test1",
"addressLine3": "Test1",
"addressType": "business",
"city": "TEST1",
"company": "TEST1",
"country": "TEST1",
"state": "TEST1",
"postalCode": "XXXX"
},
"name": {
"firstName": "TEST1",
"lastName": "TEST1",
"middleInit": "T",
"title": "Mr."
},
"phone": {
"phoneExtension": "1234",
"phoneNumber": "999999999",
"phoneType": "mobile"
},
"email": {
"emailAddress": "TEST#TEST1.com"
}
}]
====
Feature: Read from File
Background:
* def Testdata = read('Data.json')
Scenario Outline: [1] Test with multiple records on JSON
Given url_stg
And path 'test','test'
And request Testdata
When method POST
Examples:
|Testdata |
Please refer it in this documentation as it very straight forward for your case.
dynamic-scenario-outline
refer to this answer for different ways of providing data to Scenario Outline:
https://stackoverflow.com/a/56135752/8615449

Is there a way to create more flexible loops with Dataweave maps?

I have a problem that would take me ten seconds in any other language but here I am unable to solve it with Dataweave.
I have JSON that looks like this:
{
"100": [{
"courseID": "100",
"course": "physics",
"instructor": "John S",
"section": "A",
"date": "01-01-1900"
}],
"200": [{
"courseID": "200",
"course": "math",
"instructor": "Jim R",
"section": "A",
"date": "01-03-1900"
},
{
"courseID": "200",
"course": "math",
"instructor": "Jim R",
"section": "B",
"date": "01-07-1900"
},
{
"courseID": "200",
"course": "math",
"instructor": "Jim R",
"section": "C",
"date": "01-08-1900"
}
]}
}
I know for certain that the objects in the arrays contain duplicate information, except for section and date. So I need to condense it and transform it into this:
{
"courses": [{
"courseID": "100",
"course": "physics",
"instructor": "John S",
"sections": [{
"section": "A",
"date": "01-01-1900"
}]
},
{
"courseID": "200",
"course": "math",
"instructor": "Jim R",
"sections": [{
"section": "A",
"date": "01-03-1900"
},
{
"section": "B",
"date": "01-07-1900"
},
{
"section": "C",
"date": "01-08-1900"
}]
]}
I can't get this to happen. I have tried every way I can see to iterate over the document, using map(), mapObject(), and pluck(). They need to be nested I would think, and I only want to pull ID, course, and instructor from the first course it iterates over, then continue to build out the object with the section data. The nesting seems to cause problems because there is a mix of objects and arrays, and map() and mapObject() complain if a nested function is returning the wrong type. This is the kind of thing I've been trying without success:
%dw 2.0
output application/json
---
"courses": payload.pluck(courseValue, courseKey, courseIndex) ->
courseValue map (classValue, classKey, classIndex) ->
{ courseID: classValue.courseID,
instructor: classValue.instructor
etc...
}
I hope that's all clear. I'm pretty burnt out from messing with it for so long. Is it possible what I want to do is just too complex for DW to handle and I should use another method?
This solution should work in all DataWeave versions
courses: (payload pluck $$) map using (c = payload[$]) {
courseID : $,
course: c[0].course,
instructor: c[0].instructor,
sections: c map {
section: $.section,
date: $.date
}
}
This solution worked in Mule 3.x but not in 4.x, which is explained in this blog post under 'Automatic Coercion from Object to Array': https://blogs.mulesoft.com/dev/mule-dev/dataweave-mule-4-beta-syntax-changes-part-1/
courses: payload map {
courseID: $[0].courseID,
course: $[0].course,
instructor: $[0].instructor,
sections: $ map {
section: $.section,
date: $.date
}
}
I see that you are using DataWeave 2.0. Here is the simple DataWeave transformer that you can use.
Transformer:
<ee:transform doc:name="Transform Message" >
<ee:message >
<ee:set-payload ><![CDATA[%dw 2.0
output application/json
---
{
courses: (payload pluck $$) map ((courseValue, courseKey) -> {
courseID: payload[courseValue][0].courseID,
course: payload[courseValue][0].course,
instructor: payload[courseValue][0].instructor,
sections: payload[courseValue] map ((sectionValue, sectionKey) -> {
section: sectionValue.section,
date: sectionValue.date
})
})
}
]]></ee:set-payload>
</ee:message>
</ee:transform>
Input:
{
"100": [
{
"courseID": "100",
"course": "physics",
"instructor": "John S",
"section": "A",
"date": "01-01-1900"
}
],
"200": [
{
"courseID": "200",
"course": "math",
"instructor": "Jim R",
"section": "A",
"date": "01-03-1900"
},
{
"courseID": "200",
"course": "math",
"instructor": "Jim R",
"section": "B",
"date": "01-07-1900"
},
{
"courseID": "200",
"course": "math",
"instructor": "Jim R",
"section": "C",
"date": "01-08-1900"
}
]
}
Output:
{
"courses": [
{
"courseID": "100",
"course": "physics",
"instructor": "John S",
"sections": [
{
"section": "A",
"date": "01-01-1900"
}
]
},
{
"courseID": "200",
"course": "math",
"instructor": "Jim R",
"sections": [
{
"section": "A",
"date": "01-03-1900"
},
{
"section": "B",
"date": "01-07-1900"
},
{
"section": "C",
"date": "01-08-1900"
}
]
}
]
}

dojo how to get all tree node of specific type?

I wrote the following code to create a dojo tree.
store = new dojo.data.ItemFileWriteStore({url: link});
treeModel = new dijit.tree.TreeStoreModel({
store: store,
query: {
"type": "ROOT"
},
rootId: "newRoot",
childrenAttrs: ["children"]
});
tree= new dijit.Tree({model: treeModel},"treeOne");
Following is my JSON file structure :
{
identifier: "id",
label: "name",
items: [
{id: "ROOT",name: "Change Windows",type: "ROOT"},
]}
I want to get all the nodes (basically their 'id' part)of specific 'type',lets say type= "ROOT". Is there anyway to get all those node? I thought of doing this using tree._itemNodeMap, but don't know any way to iterate through this whole item map,because it need a id as a input to return any specific node.
If you're talking about obtaining the data items programatically, you can get them straight from the store using fetch.
Sample JSON for ItemFile*Store:
{
"identifier": "id",
"label": "name",
"items": [{
"id": "ROOT",
"name": "Root",
"type": "ROOT",
"children": [{
"id": "P1",
"name": "StackExchange",
"type": "website",
"children": [{
"id": "C1",
"name": "StackOverflow",
"type": "website"
},
{
"id": "C2",
"name": "ServerFault",
"type": "website"
}]
},
{
"id": "P2",
"name": "Sandwich",
"type": "food",
"children": [{
"id": "C3",
"name": "Ham",
"type": "food"
},
{
"id": "C4",
"name": "Cheese",
"type": "food"
}]
},
{
"id": "P3",
"name": "Potluck",
"type": "mixed",
"children": [{
"id": "C5",
"name": "Google",
"type": "website"
},
{
"id": "C6",
"name": "Banana",
"type": "food"
}]
}]
}]
}
Sample code:
dojo.require('dojo.data.ItemFileReadStore');
dojo.ready(function() {
var store = new dojo.data.ItemFileReadStore({
url: 'so-data.json'
});
store.fetch({
query: {
type: 'food'
},
queryOptions: {
deep: true
},
onItem: function(item) {
console.log(store.getLabel(item));
}
});
});
This will log Sandwich, Ham, Cheese, and Banana.