I'm fairly new to mulesoft, kindly help me with this one.
I'm having a http component in a for each loop,
for the first iteration it gives a payload like:
[
{
"Value1": "xxxx",
"Value2": "yyyy",
"Value3": "zzzz"
},
{
"Value1": "qqqq",
"Value2": "eeee",
"Value3": "rrrr"
}
]
and I'm storing this payload in a variable "var1"
for the second iteration it gives a payload like:
[
{
"Value1": "1234",
"Value2": "5678",
"Value3": "0987"
},
{
"Value1": "qqqq",
"Value2": "5yy4",
"Value3": "9jf7"
}
]
I need to combine them in a way that the result should be
[
{
"Value1": "xxxx",
"Value2": "yyyy",
"Value3": "zzzz"
},
{
"Value1": "qqqq",
"Value2": "eeee",
"Value3": "rrrr"
},
{
"Value1": "1234",
"Value2": "5678",
"Value3": "0987"
},
{
"Value1": "qqqq",
"Value2": "5yy4",
"Value3": "9jf7"
}
]
and store it in a variable, so that I can call it to set payload outside the for each loop.
Thank you for the help !!
You can make use of expression transformer to concat the two Vars.
If you want to do any more operations make use of Dataweave like below
flatten <listOfLists> => <mergedList>
Related
I have entered the payload as shown below, but I have to replace the values of "labelnumber" with the similar number from the main line array with the matches "payload.lines.number = payload.notification.body.lines.number" that are inside the line array inside the notification element, can anyone help me how can we replace it.
input payload
{
"date": "2022-11-15T19:24:36.871Z",
"lines": [
{
"number": "123",
"labelnumber":"ABC",
"received": "2022-11-15T19:30:17.955Z"
},
{
"number": "456",
"labelnumber":"DFG",
"received": "2022-11-15T19:30:57.426Z"
},
{
"number": "789",
"labelnumber":"HIJ",
"received": "2022-11-15T19:31:49.042Z"
}
],
"notification":{
"body":{
"date": "2022-11-15T19:24:36.871Z",
"lines": [
{
"number": "123",
"labelnumber":"",
"received": "2022-11-15T19:30:17.955Z"
},
{
"number": "123",
"labelnumber":"",
"received": "2022-11-15T19:30:57.426Z"
},
{
"number": "456",
"labelnumber":"",
"received": "2022-11-15T19:31:49.042Z"
},
{
"number": "789",
"labelnumber":"",
"received": "2022-11-15T19:31:49.042Z"
},
{
"number": "789",
"labelnumber":"",
"received": "2022-11-15T19:31:49.042Z"
}
]
}
}
}
expected payload
{
"date": "2022-11-15T19:24:36.871Z",
"lines": [
{
"number": "123",
"labelnumber":"ABC",
"received": "2022-11-15T19:30:17.955Z"
},
{
"number": "456",
"labelnumber":"DFG",
"received": "2022-11-15T19:30:57.426Z"
},
{
"number": "789",
"labelnumber":"HIJ",
"received": "2022-11-15T19:31:49.042Z"
}
],
"notification":{
"body":{
"date": "2022-11-15T19:24:36.871Z",
"lines": [
{
"number": "123",
"labelnumber":"ABC",
"received": "2022-11-15T19:30:17.955Z"
},
{
"number": "123",
"labelnumber":"ABC",
"received": "2022-11-15T19:30:57.426Z"
},
{
"number": "456",
"labelnumber":"DFG",
"received": "2022-11-15T19:31:49.042Z"
},
{
"number": "789",
"labelnumber":"HIJ",
"received": "2022-11-15T19:31:49.042Z"
},
{
"number": "789",
"labelnumber":"HIJ",
"received": "2022-11-15T19:31:49.042Z"
}
]
}
}
}
As you can see in this example, I need to loop through the main row array and find the matches within the notification array and assign the corresponding value.
NOTE: maybe in the notification.body has many elements and I don't need to change them and I don't know the name of its elements, in this case it only has "date" but I need to keep them as they are.
With the update operator you can update only the keys that you want. This solution assumes that all the number values are defined in payload.lines.
%dw 2.0
output application/json
---
payload update {
case lines at .notification.body.lines -> lines map ((item, index) ->
item update {
case .labelnumber -> (payload.lines filter ($.number == item.number))[0].labelnumber
}
)
}
Converting reference line Array to Hashmap will help replace values efficiently instead of looping every time. Following code will work
%dw 2.0
output application/json
var lineMap = {(payload.lines map {
($.number) : $.labelnumber
})}
var modifiedLines = (payload.notification.body.lines map ((item, index) -> {
data : item mapObject ((value, key, dataindex) -> {
(key) : if (key ~= 'labelnumber') lineMap[item.number] else value
})
})).data
---
(payload - 'notification') ++
{
"notification":{
"body":{
"lines": modifiedLines
}
}
}
Update : with updated requirement body can have any fields
%dw 2.0
import * from dw::util::Values
output application/json
var lineMap = {(payload.lines map {
($.number) : $.labelnumber
})}
var modifiedLines = payload.notification.body.lines map ((item, index) -> item update 'labelnumber' with lineMap[item.number])
var body = (payload.notification.body - 'lines') ++ {lines : modifiedLines}
---
(payload - 'notification') ++
{
"notification":{
"body": body
}
}
A table I called raw_data with three columns: ID, timestamp, payload, the column paylod is a json type having values such as:
{
"data": {
"author_id": "1461871206425108480",
"created_at": "2022-08-17T23:19:14.000Z",
"geo": {
"coordinates": {
"type": "Point",
"coordinates": [
-0.1094,
51.5141
]
},
"place_id": "3eb2c704fe8a50cb"
},
"id": "1560043605762392066",
"text": " ALWAYS # London, United Kingdom"
},
"matching_rules": [
{
"id": "1560042248007458817",
"tag": "london-paris"
}
]
}
From this I want to select rows where the coordinates is available, such as [-0.1094,51.5141]in this case.
SELECT *
FROM raw_data, json_each(payload)
WHERE json_extract(json_each.value, '$.data.geo.') IS NOT NULL
LIMIT 20;
Nothing was returned.
EDIT
NOT ALL json objects have the coordinates node. For example this value:
{
"data": {
"author_id": "1556031969062010881",
"created_at": "2022-08-18T01:42:21.000Z",
"geo": {
"place_id": "006c6743642cb09c"
},
"id": "1560079621017796609",
"text": "Dear Desperate sister say husband no dey oo."
},
"matching_rules": [
{
"id": "1560077018183630848",
"tag": "kaduna-kano-katsina-dutse-zaria"
}
]
}
The correct path is '$.data.geo.coordinates.coordinates' and there is no need for json_each():
SELECT *
FROM raw_data
WHERE json_extract(payload, '$.data.geo.coordinates.coordinates') IS NOT NULL;
See the demo.
[
{
"key": "test1",
"category": "test",
"name": "test1",
"translations":
{
"english": "eng"
}
},
{
"key": "test2",
"category": "test",
"name": "test1",
"translations":
{
"english": "eng2",
"german": "German"
}
},
{
"key": "test3",
"category": "power",
"name": "test1",
"translations":
{
"EN_lang": "jik"
}
}
]
Here, we have multiple field's are with different values and we have to match value in translations (field position will change on every call)
You have to be clear about what you want to assert. Hint, the new contains deep (available in 0.9.6.RC4) can help:
* match response contains deep { key: 'test2', translations: { english: 'eng2' } }
Else you should look at transforming the JSON into a shape where it is easier to do the assertions you want: https://github.com/intuit/karate#json-transforms
I'm new in lodash (v3.10.1), and having a hard time understanding.
Hope someone can help.
I have an input something like this:
{
{"id":1,"name":"Matthew","company":{"id":1,"name":"abc","industry":{"id":5,"name":"Medical"}}},
{"id":2,"name":"Mark","company":{"id":1,"name":"abc","industry":{"id":5,"name":"Medical"}}},
{"id":3,"name":"Luke","company":{"id":1,"name":"abc","industry":{"id":5,"name":"Medical"}}},
{"id":4,"name":"John","company":{"id":1,"name":"abc","industry":{"id":5,"name":"Medical"}}},
{"id":5,"name":"Paul","company":{"id":1,"name":"abc","industry":{"id":5,"name":"Medical"}}}
];
I would like to output this or close to this:
{
"industries": [
{
"industry":{
"id":5,
"name":"Medical",
"companies": [
{
"company":{
"id":1,
"name":"abc",
"employees": [
{"id":1,"name":"Matthew"},
{"id":2,"name":"Mark"},
{"id":3,"name":"Luke"},
{"id":4,"name":"John"},
{"id":5,"name":"Paul"}
]
}
}
]
}
}
]
}
Here's something that gets you close to what you want. I structured the output to be an object instead of an array. You don't need the industries or industry properties in your example output. The output structure looks like this:
{
"industry name": {
"id": "id of industry",
"companies": [
{
"company name": "name of company",
"id": "id of company",
"employees": [
{
"id": "id of company",
"name": "name of employee"
}
]
}
]
}
}
I use the _.chain function to wrap the collection with a lodash wrapper object. This enables me to explicitly chain lodash functions.
From there, I use the _.groupBy function to group elements of the collection by their industry name. Since I'm chaining, I don't have to pass in the array again to the function. It's implicitly passed via the lodash wrapper. The second argument of the _.groupBy is the path to the value I want to group elements by. In this case, it's the path to the industry name: company.industry.name. _.groupBy returns an object with each employee grouped by their industry (industries are keys for this object).
I then do use _.transform to transform each industry object. _.transform is essentially _.reduce except that the results returned from the _.transform function is always an object.
The function passed to the _.transform function gets executed against each key/value pair in the object. In the function, I use _.groupBy again to group employees by company. Based off the results of _.groupBy, I map the values to the final structure I want for each employee object.
I then call the _.value function because I want to unwrap the output collection from the lodash wrapper object.
I hope this made sense. If it doesn't, I highly recommend reading Lo-Dash Essentials. After reading the book, I finally got why lodash is so useful.
"use strict";
var _ = require('lodash');
var emps = [
{ "id": 1, "name": "Matthew", "company": { "id": 1, "name": "abc", "industry": { "id": 5, "name": "Medical" } } },
{ "id": 2, "name": "Mark", "company": { "id": 1, "name": "abc", "industry": { "id": 5, "name": "Medical" } } },
{ "id": 3, "name": "Luke", "company": { "id": 1, "name": "abc", "industry": { "id": 5, "name": "Medical" } } },
{ "id": 4, "name": "John", "company": { "id": 1, "name": "abc", "industry": { "id": 5, "name": "Medical" } } },
{ "id": 5, "name": "Paul", "company": { "id": 1, "name": "abc", "industry": { "id": 5, "name": "Medical" } } }
];
var result = _.chain(emps)
.groupBy("company.industry.name")
.transform(function(result, employees, industry) {
result[industry] = {};
result[industry].id = _.get(employees[0], "company.industry.id");
result[ industry ][ 'companies' ] = _.map(_.groupBy(employees, "company.name"), function( employees, company ) {
return {
company: company,
id: _.get(employees[ 0 ], 'company.id'),
employees: _.map(employees, _.partialRight(_.pick, [ 'id', 'name' ]))
};
});
return result;
})
.value();
Results from your example are as follows:
{
"Medical": {
"id": 5,
"companies": [
{
"company": "abc",
"id": 1,
"employees": [
{
"id": 1,
"name": "Matthew"
},
{
"id": 2,
"name": "Mark"
},
{
"id": 3,
"name": "Luke"
},
{
"id": 4,
"name": "John"
},
{
"id": 5,
"name": "Paul"
}
]
}
]
}
}
If you ever wanted the exact same structure as in the questions, I solved it using the jsonata library:
(
/* lets flatten it out for ease of accessing the properties*/
$step1 := $ ~> | $ |
{
"employee_id": id,
"employee_name": name,
"company_id": company.id,
"company_name": company.name,
"industry_id": company.industry.id,
"industry_name": company.industry.name
},
["company", "id", "name"] |;
/* now the magic begins*/
$step2 := {
"industries":
[($step1{
"industry" & $string(industry_id): ${
"id": $distinct(industry_id)#$I,
"name": $distinct(industry_name),
"companies": [({
"company" & $string(company_id): {
"id": $distinct(company_id),
"name": $distinct(company_name),
"employees": [$.{
"id": $distinct(employee_id),
"name": $distinct(employee_name)
}]
}
} ~> $each(function($v){ {"company": $v} }))]
}
} ~> $each(function($v){ {"industry": $v} }))]
};
)
You can see it in action on the live demo site: https://try.jsonata.org/VvW4uTRz_
I can't find very much documentation on how to properly define the index function such that I can do a full text search on the information that I need.
I've used the Alchemy API to add "entities" json to my documents.
For instance, I have a document with the following:
"_id": "redacted",
"_rev": "redacted",
"session": "20152016",
"entities": [
{
"relevance": "0.797773",
"count": "3",
"type": "Organization",
"text": "California Constitution"
},
{
"relevance": "0.690092",
"count": "1",
"type": "Organization",
"text": "Governors Highway Safety Association"
}
]
I haven't been able to find any code snippets showing how to construct a search index function that looks at nested json.
My stab at indexing the whole object appears to be incorrect.
This is the full design document:
{
"_id": "_design/entities",
"_rev": "redacted",
"views": {},
"language": "javascript",
"indexes": {
"entities": {
"analyzer": "standard",
"index": "function (doc) {\n if (doc.entities.relevance > 0.5){\n index(\"default\", doc.entities.text, {\"store\":\"yes\"});\n }\n\n}"
}
}
}
And the search index formatted a little bit more clearly is
function (doc) {
if (doc.entities.relevance > 0.5){
index("default", doc.entities.text, {"store":"yes"});
}
}
Adding the for loop as suggested below makes a lot of sense.
However, I still am not able to return any results.
My query is
"https://user.cloudant.com/calbills/_design/entities/_search/entities?q=Governors"
Server response is:
{"total_rows":0,"bookmark":"g2o","rows":[]}
The "for..in" style loop doesn't seem to work.
However, I do get results using the more standard for loop loops.
function (doc) {
if(doc.entities){
var arrayLength = doc.entities.length;
for (var i = 0; i < arrayLength; i++) {
if (parseFloat(doc.entities[i].relevance) > 0.5)
index("default", doc.entities[i].text);
}
}
}
Cheers!
Your need to loop on the elements in the doc.entities array.
function (doc) {
for(entity in doc.entities){
if (parseFloat(entity.relevance) > 0.5){
index("default", entity.text, {"store":"yes"});
}
}
}
This is what I tried :
function(doc){
if(doc.entities){
for( var p in doc.entities ){
if (doc.entities[p].relevance > 0.5)
{
index("entitiestext", doc.entities[p].text, {"store":"yes"});
}
}
}
}
Query String used :"q=entitiestext:California Constitution&include_docs=true"
Result:
{
"total_rows": 1,
"bookmark": "xxxx",
"rows": [
{
"id": "redacted",
"order": [
0.03693288564682007,
1
],
"fields": {
"entitiestext": [
"Governors Highway Safety Association",
"California Constitution"
]
},
"doc": {
"_id": "redacted",
"_rev": "4-7f6e6db246abcf2f884dc0b91451272a",
"session": "20152016",
"entities": [
{
"relevance": "0.797773",
"count": "3",
"type": "Organization",
"text": "California Constitution"
},
{
"relevance": "0.690092",
"count": "1",
"type": "Organization",
"text": "Governors Highway Safety Association"
}
]
}
}
]
}
Query String used: q=entitiestext:California Constitution
Result:
{
"total_rows": 1,
"bookmark": "xxxx",
"rows": [
{
"id": "redacted",
"order": [
0.03693288564682007,
1
],
"fields": {
"entitiestext": [
"Governors Highway Safety Association",
"California Constitution"
]
}
}
]
}