Mule transform JSON to Map [duplicate] - mule

This question already has answers here:
Convert JSON to Map
(19 answers)
Closed 3 months ago.
I have this JSON object
"actos-evaluacion" : [
{
   "subject" : < id-subject>,
   "name" : < name>,
   "marks" : ["<student_id>:< mark>","< student_id>:< mark>",
     "<student_id>:< mark>"]
},
...
{
   "subject" : < id-subject>,
   "name" : < name>,
   "marks" : ["<student_id>:< mark>","< student_id>:< mark>",
     "<student_id>:< mark>"]
},
]
}
And I want to convert it into this Map, so there will be more Maps than JSON, because each JSON will split into different maps depending on the number of students it has.
{
id_subject :
name :
student_id :
mark :
}
I've tried putting a foreach(json) where subject_id and name where stored in variables. Then set_payload with the list and another foreach for the elements of the list but I don't know how to recover the variables into a Map operator to join them all

A DataWeave script can generate the output. I had to make some assumptions of what the input and output actually look to illustrate the method to loop over the internal array inside each element of the first array. You can adapt the method if the samples are not exactly what you use. My script generates an array of DataWeave objects that are equivalent to a Java Map.
Input:
{
"actos-evaluacion" : [
{
"subject" : "id1",
"name": "name1",
"marks": [ "student1:mark1", "student2:mark2", "student1:mark3"]
},
{
"subject" : "id2",
"name" : "name2",
"marks": [ "student1:mark1", "student2:mark2", "student1:mark3"]
}
]
}
DataWeave script:
%dw 2.0
output application/json
---
payload."actos-evaluacion" flatMap ((item, index) ->
item.marks map ((subject, index2) -> {
id_subject: item.subject,
name: item.name,
student_id: (subject splitBy ":") [0],
mark: (subject splitBy ":")[1]
})
)
Output:
[
{
"id_subject": "id1",
"name": "name1",
"student_id": "student1",
"mark": "mark1"
},
{
"id_subject": "id1",
"name": "name1",
"student_id": "student2",
"mark": "mark2"
},
{
"id_subject": "id1",
"name": "name1",
"student_id": "student1",
"mark": "mark3"
},
{
"id_subject": "id2",
"name": "name2",
"student_id": "student1",
"mark": "mark1"
},
{
"id_subject": "id2",
"name": "name2",
"student_id": "student2",
"mark": "mark2"
},
{
"id_subject": "id2",
"name": "name2",
"student_id": "student1",
"mark": "mark3"
}
]

Related

Derive parent-child hierarchy in JSON from an array using DataWeave

Hopefully this is the last of my questions related to this and this
The earlier questions were related to generating XML payload while this query is around generating JSON o/p in a specific format...
Thanks to #Harshank and #sudhish_s was able to make some progress with XML o/p. However I am not able to figure out how to go about json o/p
Here is the input payload ( have limited it to three rows / json objects but could be in thousands)
[
{
"gp": "S1",
"gp_eye_colour": "blue",
"gp_name" : "John",
"parent": "S1",
"parent_eye_colour" : "blue",
"parent_name" : "Sam",
"child": "C1",
"child_eye_colour" : "black",
"child_name" : "C1-name"
},
{
"gp": "S1",
"gp_eye_colour": "blue",
"gp_name" : "John",
"parent": "P1",
"parent_eye_colour" : "blue",
"parent_name" : "Don",
"child": "C1",
"child_eye_colour" : "brown",
"child_name" : "C1-name"
} ,
{
"gp": "S2",
"gp_eye_colour": "blue",
"gp_name" : "David",
"parent": "P2",
"parent_eye_colour" : "blue",
"parent_name" : "Martha",
"child": "C1",
"child_eye_colour" : "brown",
"child_name" : "C1-name"
}
]
Desired o/p is as below :
{
"S2_blue": {
"code": "S2",
"eyeColour": "blue",
"name": "David",
"hierarchy": [
{
"code": "P2",
"eyeColour": "blue",
"name": "Martha",
"hierarchy": [
{
"code": "C1",
"eyeColour": "brown",
"name": "C1-name",
"hierarchy": []
}
]
}
]
},
"S1_blue": {
"code": "S1",
"eyeColour": "blue",
"name": "John",
"hierarchy": [
{
"code": "P1",
"eyeColour": "blue",
"name": "Don",
"hierarchy": [
{
"code": "C1",
"eyeColour": "brown",
"name": "C1-name",
"hierarchy": []
}
]
},
{
"code": "C1",
"eyeColour": "black",
"name": "C1-name",
"hierarchy": []
}
]
}
}
Similar rules need to apply :
Its a Grandparent >> Parent >> Child hierarchy
If Grandparent and Parent share same characteristics (gp == parent and gp_eye_colour == parent_eye_colour then skip parent and directly include child(ren)
( as an example parent Sam is excluded since John (gp) has the same value for gp and eye_colour
Don is not exluded ( same rule as above )
Using the solutions provided to the earlier asked questions - I came up with the following code:
%dw 2.0
output application/json
var hierarchy = ["gp", "parent", "child"]
fun getDirectGeanologies(records, level) = do {
var hLevel = hierarchy[level]
var xyz = if(level == 0)
(records groupBy ((item, index) -> item.gp ++ "_" ++ item.gp_eye_colour) )
else if(level == 1)
(records groupBy ((item, index) -> item.parent ++ "_" ++ item.parent_eye_colour))
else
(records groupBy ((item, index) -> item.child ++ "_" ++ item.child_eye_colour))
---
xyz mapObject ((children, code) ->
(code): {
code: children[0][hLevel],
eyeColour: children[0][hLevel ++ "_eye_colour"],
name: children[0][hLevel ++ "_name"],
hierarchy:
if (level == sizeOf(hierarchy) - 1) [] // if level = 2 ( child stop recursion)
else do {
var nextLevel = level + 1
var nextGen = if(nextLevel == 1)(children groupBy ((item, index) -> item.parent ++ "_" ++ item.parent_eye_colour)) else (children groupBy ((item, index) -> item.child ++ "_" ++ item.child_eye_colour))
---
nextGen mapObject ((nextGenChildren, nextGenCode) ->
if (nextGenCode == code)
getDirectGeanologies (nextGenChildren, nextLevel + 1 ) // skip parent
else
getDirectGeanologies (nextGenChildren, nextLevel)
)
}
}
)
}
---
getDirectGeanologies(payload,0)
However it is not producing desired o/p :
{
"S1_blue": {
"code": "S1",
"eyeColour": "blue",
"name": "John",
"hierarchy": {
"C1_black": {
"code": "C1",
"eyeColour": "black",
"name": "C1-name",
"hierarchy": [
]
},
"P1_blue": {
"code": "P1",
"eyeColour": "blue",
"name": "Don",
"hierarchy": {
"C1_brown": {
"code": "C1",
"eyeColour": "brown",
"name": "C1-name",
"hierarchy": [
]
}
}
}
}
},
"S2_blue": {
"code": "S2",
"eyeColour": "blue",
"name": "David",
"hierarchy": {
"P2_blue": {
"code": "P2",
"eyeColour": "blue",
"name": "Martha",
"hierarchy": {
"C1_brown": {
"code": "C1",
"eyeColour": "brown",
"name": "C1-name",
"hierarchy": [
]
}
}
}
}
}
}
The issues here are :
hierarchy inside gp should be an array but I am getting a object
Not 100% sure but because of above problem rather than having objects inside an array against hierarchy I am getting an object with key as parent/child and eye colour ( Ex : C1_black , P1_blue etc )
So not really sure how to get this part and I am having a tough time to visualise the recursive behaviour
Thanks in advance and once again really appreciate the help already provided by #Harshank and #sudhish_s
I have answered this in mulesoft help forum. LInk MuleSoft Help Forum link

Nested array data transformation with Mule 3

I am trying to transform data in Mule 3 (DataWeave 1.0) but I am not getting the desire result.
This is the input payload:
[
"https://scrum-caller.com/repo/v1/inside#Changes",
[
{
"id": "8db55441-6255-4d24-8d39-658536985214",
"number": "0w-30",
"Desc": "maintain"
}
],
"https://scrum-caller.com/repo/v1/inside#Changes",
[
{
"id": "11111111-6666-2222-3g3g-854712547412",
"number": "5w-40",
"Desc": "on prod"
}
],
"https://scrum-caller.com/repo/v1/inside#Changes",
[
{
"id": "1ab32c5b-ffs3-3243-74fv-3376218042bb",
"number": "5w-30",
"Desc": "on test"
}
]
]
And my desire output need to be like the one below
{
"#odata.context": "https://scrum-caller.com/repo/v1/inside#Changes",
"value": [
{
"id": "8db55441-6255-4d24-8d39-658536985214",
"number": "0w-30",
"Desc": "maintain"
},
{
"id": "11111111-6666-2222-3g3g-854712547412",
"number": "5w-40",
"Desc": "on prod"
},
{
"id": "1ab32c5b-ffs3-3243-74fv-3376218042bb",
"number": "5w-30",
"Desc": "on test"
}
]
}
Thanks for helping guys.
Assuming all the URLs are the same, since the question doesn't provide details only an example, I just take the first element as the value of the input for "#odata.context", the the value is just filtering out the non-array elements and use the reduce operator to get a single array of the other elements.
%dw 1.0
%output application/json
---
{
"#odata.context" : payload[0],
value : payload filter ($ is :array) reduce ($ ++ $$)
}

Not able to match same json which has mismatch in indexes of array of objects (Karate) [duplicate]

This question already has an answer here:
Is there a simple match for objects containing array where the array content order doesn't matter?
(1 answer)
Closed 1 year ago.
Trying to match two jsons, but getting test fails. Well, both jsons are the same but objects indexes inside the array are not same. I think should not make any difference. Following are two jsons:
This is the code line: And match response contains ScenarioModelResponse where
**response : **
{
"relationships": [
{
"sourceId": "36",
"targetId": "149",
"type": "Reid Enright"
}
],
"modelId": "027f93d1-ef9e-4f1e-b2c4-684436c5b18a",
"elements": [
{
"externalRefId": "36",
"attributes": {
"jsonPbject": "Reid Enright"
},
"id": "057f7b7e-11b9-4779-97c0-67485153c285",
"type": "Rocky Shore"
},
{
"externalRefId": "149",
"attributes": {
"jsonPbject": "Ben Lyon"
},
"id": "325b989e-b299-4cfc-86b5-0813106da38e",
"type": "Claire Voyance"
}
]
}
ScenarioModelResponse :
{
"relationships": [
{
"sourceId": "36",
"targetId": "149",
"type": "Reid Enright"
}
],
"modelId": "027f93d1-ef9e-4f1e-b2c4-684436c5b18a",
"elements": [
{
"externalRefId": "149",
"attributes": {
"jsonPbject": "Ben Lyon"
},
"id": "325b989e-b299-4cfc-86b5-0813106da38e",
"type": "Claire Voyance"
},
{
"externalRefId": "36",
"attributes": {
"jsonPbject": "Reid Enright"
},
"id": "057f7b7e-11b9-4779-97c0-67485153c285",
"type": "Rocky Shore"
}
]
}
This the error I am getting after execution :
$.elements[0].externalRefId | not equal (STRING:STRING)
'149'
'36'
The arrays are NOT the same. This can be solved in 2 lines:
* match response.relationships == expected.relationships
* match response.elements contains only expected.elements
For a detailed explanation, refer:
https://stackoverflow.com/a/65939070/143475
https://stackoverflow.com/a/55710769/143475

Apache Nifi: UpdateRecord replace child values

I'm trying to use UpdateRecord 1.9.0 processor to modify a JSON but it does not replace the values as I want.
this is the source message
{
"type": "A",
"ids": [{
"id": "1",
"value": "abc"
}, {
"id": "2",
"value": "def"
}, {
"id": "3",
"value": "ghi"
}
]
}
and the wanted output
{
"ids": [{
"userId": "1",
}, {
"userId": "2",
}, {
"userId": "3",
}
]
}
I have configured the processor as follows
processor config
Reader:
reader
Schema registry:
schema
writer:
writer
And it works, the output is a JSON without the field 'type' and the ids have the field 'userId' instead 'id' and 'value'.
To fill the value of userId, I defined the replace strategy and the property to replace:
strategy
But the output is wrong. The userId is always filled with the id of the last element in the array:
{
"ids": [{
"userId": "3"
}, {
"userId": "3"
}, {
"userId": "3"
}
]
}
I think the value of the expression is ok because if I try to replace only one record it works fine (/ids[0]/userId, ..id)
Nifi docs has a really similar example (example 3):
https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.7.1/org.apache.nifi.processors.standard.UpdateRecord/additionalDetails.html
But it does not work for me.
What am I doing wrong?
thanks
Finally I have used JoltJSONTransform processor instead UpdateRecord
JoltJSONTransform
template:
[
{
"operation": "shift",
"spec": {
"ids":{
"*":{
"id": "ids[&1].userId"
}
}
}
}
]
Easier than UpdateRecord

ElasticSearch - how to give priority to the matching from the same row

I have the following documents in ElasticSearch 0.19.11, using:
{ "title": "dogs species",
"col_names": [ "name", "description", "country_of_origin" ],
"rows": [
{ "row": [ "Boxer", "good dog", "Germany" ] },
{ "row": [ "Irish Setter", "great dog", "Ireland" ] }
]
}
{ "title": "Misc stuff",
"col_names": [ "foo" ],
"rows": [
{ "row": [ "Setter is impotant" ] },
{ "row": [ "Ireland is green" ] }
]
}
The mapping is as follows:
{
"table" : {
"properties" : {
"title" : {"type" : "string"},
"col_names" : {"type" : "string"},
"rows" : {
"properties" : {
"row" : {"type" : "string"}
}
}
}
}
}
Question: I'm now searching for "Ireland Setter" and I need to have a higher score for documents that have search terms in the same row.
Currently the second document gets score of 0.22, while the first one - 0.14.
I want the first document to get a higher score in this case, since it has both "Ireland" and "Setter" in the same row. How can it be done?
With great cooperation from ElasticSearch google-group members, the solution is found.
Here is the link to the discussion: https://groups.google.com/forum/?fromgroups#!topic/elasticsearch/4O9dff2SNhg