I am trying to filter same level object values in the payload in Dataweave. I was able to loop through but it does not produce the expected output.
Sample Payload:
{
"root": {
"createItem": {
"itemInfo": {
"lines": [{
"lineIdentifier": "4",
"Attributes": "Test1",
"partNumber": "QZRB"
}, {
"lineIdentifier": "10",
"Attributes": "Test3",
"partNumber": "QPR1"
}, {
"lineIdentifier": "12",
"Attributes": "Test4",
"partNumber": "QHT2"
}]
}
},
"ItemResponse": {
"lines": [{
"lineIdentifier": 4,
"itemName": "QZRB",
"status": "FAILED"
}, {
"lineIdentifier": 10,
"itemName": "QPR1",
"status": "COMPLETE"
}, {
"lineIdentifier": 12,
"itemName": "QHT2",
"status": "COMPLETE"
}]
}
}
}
Expected Output:
{
"root": {
"createItem": {
"itemInfo": {
"lines": [ {
"lineIdentifier": "10",
"Attributes": "Test3",
"partNumber": "QPR1"
}, {
"lineIdentifier": "12",
"Attributes": "Test4",
"partNumber": "QHT2"
}]
}
}
}
}
Here's what I am doing:
{
root: {
(payload.root.createItem.itemInfo.lines map ((respLines, indexOfRespLines) -> {
items:payload.root.ItemResponse.lines filter ($.itemName == respLines.partNumber and $.status =='COMPLETE') map
{
item: $.itemName,
attributes: respLines.Attributes
}
}
)
)
}
}
How do I achieve this?
Thanks,
ROA
try this:
%dw 1.0
%output application/json
%var completedLines = payload.root.ItemResponse.lines filter $.status == 'COMPLETE' map $.lineIdentifier as :string
---
{
root: {
createItem: {
itemInfo: {
lines: payload.root.createItem.itemInfo.lines filter (completedLines contains $.lineIdentifier)
}
}
}
}
pay attention to as :string in completedLines, because the lineIdentifier in ItemResponse is a number, while in itemInfo it is a string.
Related
I tried to filter below json payload characters[] array which is having result as 'valid' and data[] array name as 'WBB' and priority as '1'
I tried below code but not working can some one help me ?.
flatten(payload.offers.category.characters) filter ((item, index) -> item.result=='valid' and flatten(item.data) filter ((item, index) -> item.name=='WBB' and item.priority==1))
Json payload
{
"offers": [
{
"id": 100,
"name": "Test1",
"category": {
"characters": [
{
"result": "valid",
"data": [
{
"name": "WBB",
"priority": 1
},
{
"name": "ILL",
"priority": 2
}
]
}
]
}
},
{
"id": 200,
"name": "Test2",
"category": {
"characters": [
{
"data": [
{
"name": "ISS",
"priority": 1
},
{
"name": "ILL",
"priority": 2
}
]
}
]
}
},
{
"id": 300,
"name": "Test3",
"category": {
"characters": [
{
"data": [
{
"name": "WSS",
"priority": 1
},
{
"name": "ILL",
"priority": 2
}
]
}
]
}
}
]
}
Expected payload
[
{
"name": "WBB",
"priority": 1
}
]
flatten((flatten(payload.offers..characters) filter $.result == "valid").data) filter ($.name=="WBB" and $.priority == 1)
flatten(payload..data) filter ((item, index) -> item.name == 'WBB' )
I'm pretty sure the below can be done, I'm struggling to understand how to do it in MongoDB.
My data is structured like this (demo data):
db={
"recipes": [
{
"id": 1,
"name": "flatbread pizza",
"ingredients": {
"1010": 1,
"1020": 2,
"1030": 200
}
},
{
"id": 2,
"name": "cheese sandwich",
"ingredients": {
"1040": 1,
"1050": 2
}
}
],
"ingredients": [
{
"id": 1010,
"name": "flatbread",
"unit": "pieces"
},
{
"id": 1020,
"name": "garlic",
"unit": "clove"
},
{
"id": 1030,
"name": "tomato sauce",
"unit": "ml"
},
{
"id": 1040,
"name": "bread",
"unit": "slices"
},
{
"id": 1050,
"name": "cheese",
"unit": "slices"
}
]
}
The output I'm trying to achieve would look like this:
[
{
"id": 1,
"name": "flatbread pizza",
“flatbread”: “1 pieces”,
“garlic”: “2 cloves”,
“tomato sauce”: “200 ml”
},
{
"id": 2,
"name": "cheese sandwich",
“bread”: “1 slices”,
“cheese”: “2 slices”
}
]
I've tried several approaches, and I get stuck at the bit where I need to do a lookup based on the ingredient name (which actually is the id). I tried using $objectToArray to turn it into a k-v document, but then I get stuck in how to construct the lookup pipeline.
This is not a simple solution, and probably can be improved:
db.recipes.aggregate([
{
"$addFields": {
ingredientsParts: {
"$objectToArray": "$ingredients"
}
}
},
{
$unwind: "$ingredientsParts"
},
{
"$group": {
_id: "$id",
name: {
$first: "$name"
},
ingredientsParts: {
$push: {
v: "$ingredientsParts.v",
id: {
$toInt: "$ingredientsParts.k"
}
}
}
}
},
{
"$lookup": {
"from": "ingredients",
"localField": "ingredientsParts.id",
"foreignField": "id",
"as": "ingredients"
}
},
{
$unwind: "$ingredients"
},
{
"$addFields": {
"ingredientsPart": {
"$filter": {
input: "$ingredientsParts",
as: "item",
cond: {
$eq: [
"$$item.id",
"$ingredients.id"
]
}
}
}
}
},
{
$project: {
ingredients: 1,
ingredientsPart: {
"$arrayElemAt": [
"$ingredientsPart",
0
]
},
name: 1
}
},
{
"$addFields": {
units: {
k: "$ingredients.name",
v: {
"$concat": [
{
$toString: "$ingredientsPart.v"
},
" ",
"$ingredients.unit"
]
}
}
}
},
{
$group: {
_id: "$_id",
name: {
$first: "$name"
},
units: {
$push: "$units"
}
}
},
{
"$addFields": {
"data": {
"$arrayToObject": "$units"
}
}
},
{
"$addFields": {
"data.id": "$_id",
"data.name": "$name"
}
},
{
"$replaceRoot": {
"newRoot": "$data"
}
}
])
You can see it works here
As rickhg12hs said, it can be modeled better.
Problem with using mapObject function properly.
Trying to retain existing array structure but calculate number of vehicles and properties and update the existing array that contains the value.
GENERAL data comes from one source, VEHICLE data comes from another source, PROPERTY data comes from another source. So when merging, I have to update GENERAL data with count of other source data.
Also GENERAL is an array object, it will always have 1. So using GENERAL[0] is safe and fine.
Original Payload
[
{
"commId": "1",
"GENERAL": [
{
"ID": "G1",
"VEHICLE_COUNT": "TODO",
"PROPERTY_COUNT": "TODO"
}
],
"VEHICLE": [
{
"ID": "V1-1"
},
{
"ID": "V1-2"
}
],
"PROPERTY": [
{
"ID": "P1-1"
}
]
},
{
"commId": "2",
"GENERAL": [
{
"ID": "G2",
"VEHICLE_COUNT": "TODO",
"PROPERTY_COUNT": "TODO"
}
],
"VEHICLE": [
{
"ID": "V2-1"
}
],
"PROPERTY": [
{
"ID": "P2-1"
},
{
"ID": "P2-2"
}
]
},
{
"commId": "3",
"GENERAL": [
{
"ID": "G3",
"VEHICLE_COUNT": "TODO",
"PROPERTY_COUNT": "TODO"
}
],
"VEHICLE": [
{
"ID": "V3-1"
},
{
"ID": "V3-2"
},
{
"ID": "V3-3"
}
]
}
]
Tried using map to loop through the payload and tried modifying 2 attribute but only managed to map one but even that is showing wrong output.
test map (item, index) -> {
(item.GENERAL[0] mapObject (value, key) -> {
(key): (value == sizeOf (item.VEHICLE)
when (key as :string) == "VEHICLE_COUNT"
otherwise value)
})
}
Expected output:
[
{
"commId": "1",
"GENERAL": [
{
"ID": "G1",
"VEHICLE_COUNT": "2",
"PROPERTY_COUNT": "1"
}
],
"VEHICLE": [
{
"ID": "V1-1"
},
{
"ID": "V1-2"
}
],
"PROPERTY": [
{
"ID": "P1-1"
}
]
},
{
"commId": "2",
"GENERAL": [
{
"ID": "G2",
"VEHICLE_COUNT": "1",
"PROPERTY_COUNT": "2"
}
],
"VEHICLE": [
{
"ID": "V2-1"
}
],
"PROPERTY": [
{
"ID": "P2-1"
},
{
"ID": "P2-2"
}
]
},
{
"commId": "3",
"GENERAL": [
{
"ID": "G3",
"VEHICLE_COUNT": "3",
"PROPERTY_COUNT": "0"
}
],
"VEHICLE": [
{
"ID": "V3-1"
},
{
"ID": "V3-2"
},
{
"ID": "V3-3"
}
]
}
]
Getting totally wrong output so far:
[
{
"ID": "G1",
"VEHICLE_COUNT": false,
"PROPERTY_COUNT": "TODO"
},
{
"ID": "G2",
"VEHICLE_COUNT": false,
"PROPERTY_COUNT": "TODO"
},
{
"ID": "G3",
"VEHICLE_COUNT": false,
"PROPERTY_COUNT": "TODO"
}
]
Edited: Update for dynamic transform
The below dataweave transform is not particularly attractive, but it might work for you.
Thanks to Christian Chibana for helping me find a dynmaic answer by answering this question: Why does Mule DataWeave array map strip top level objects?
%dw 1.0
%output application/json
---
payload map ((item) ->
(item - "GENERAL") ++
GENERAL: item.GENERAL map (
$ - "VEHICLE_COUNT"
- "PROPERTY_COUNT"
++ { VEHICLE_COUNT: sizeOf (item.VEHICLE default []) }
++ { PROPERTY_COUNT: sizeOf (item.PROPERTY default []) }
)
)
It is dynamic, so everything should be copied across as it comes in, with only the two fields you want being updated.
The output for this transform with the input you supplied is below. Only difference from your desired is that the counts are shown as numbers rather than strings. If you really need them as strings you can cast them like (sizeOf (comm.VEHICLE default [])) as :string,
[
{
"commId": "1",
"VEHICLE": [
{
"ID": "V1-1"
},
{
"ID": "V1-2"
}
],
"PROPERTY": [
{
"ID": "P1-1"
}
],
"GENERAL": [
{
"ID": "G1",
"VEHICLE_COUNT": 2,
"PROPERTY_COUNT": 1
}
]
},
{
"commId": "2",
"VEHICLE": [
{
"ID": "V2-1"
}
],
"PROPERTY": [
{
"ID": "P2-1"
},
{
"ID": "P2-2"
}
],
"GENERAL": [
{
"ID": "G2",
"VEHICLE_COUNT": 1,
"PROPERTY_COUNT": 2
}
]
},
{
"commId": "3",
"VEHICLE": [
{
"ID": "V3-1"
},
{
"ID": "V3-2"
},
{
"ID": "V3-3"
}
],
"GENERAL": [
{
"ID": "G3",
"VEHICLE_COUNT": 3,
"PROPERTY_COUNT": 0
}
]
}
]
My docs represents an order in the given format:
{
"name", // the client
"sku" // the product
}
So, suppose the follow data exists:
{ "name": "rudolph", "sku": "apple" }
{ "name": "rudolph", "sku": "apple" }
{ "name": "rudolph", "sku": "apple" }
{ "name": "john", "sku": "banana" }
{ "name": "john", "sku": "banana" }
{ "name": "paul", "sku": "banana" }
{ "name": "paul", "sku": "apple" }
{ "name": "peter", "sku": "banana" }
I can get the clients who bought only 1 kind of item with the query:
{
"aggs": {
"clients": {
"terms": {
"field": "name"
},
"aggs": {
"distinct_sku": {
"cardinality": {
"field": "sku"
}
},
"unique_sku": {
"bucket_selector": {
"buckets_path": {
"qty": "distinct_sku"
},
"script": "params.qty == 1"
}
},
"aggs": {
"terms": {
"field": "sku"
}
}
}
}
},
"size": 0
}
which results
{
...
"aggregations": {
"clients": {
...
"buckets": [
{
"key": "rudolph",
"doc_count": 3,
...
"aggs": {
...
"buckets": [
{
"key": "apple",
"doc_count": 3
}
]
}
},
{
"key": "john",
"doc_count": 2,
...
"aggs": {
...
"buckets": [
{
"key": "banana",
"doc_count": 2
}
]
}
},
{
"key": "peter",
"doc_count": 1,
...
"aggs": {
...
"buckets": [
{
"key": "banana",
"doc_count": 1
}
]
}
}
]
}
}
}
It's possible to manage the query to return how much each item appears in the result?
Something like this:
{
"apple" : 1,
"banana" : 2
}
Thanks in advance.
EDIT
My base has a huge amount of clients and a small quantity of products, so:
Iterate over the above aggregation result to build the wanted result is not an option.
If I have to send a query for each product, It's Ok.
I need to extract key and values from json input object to form different json output.
I went through the documentation and other questions asked similar to this where I could found out that $$ gives the key but, in my case it is giving me index but not the key name.
The input json looks like this :{
"key2": "val2",
"key3": "val3",
"key4": "val4",
"key5": "val5",
"key6": "val6"
}
The dataweave code I have written is :
{
"someOtherKey": "val",
properties: {
entry: payload map
{
key:$$,
value:$
}
}
}
After transformation I am getting :
{
"someOtherKey": "val",
"properties": {
"entry": [
{
"key": 0,
"value": "val2"
},
{
"key": 1,
"value": "val3"
},
{
"key": 2,
"value": "val4"
},
{
"key": 3,
"value": "val5"
},
{
"key": 4,
"value": "val6"
}
]
}
}
Here I am expecting output with key name as value for Key
Expected output :
{
"someOtherKey": "val",
"properties": {
"entry": [{
"key": "key2",
"value": "val2"
},
{
"key": "key3",
"value": "val3"
},
{
"key": "key4",
"value": "val4"
},
{
"key": "key5",
"value": "val5"
},
{
"key": "key6",
"value": "val6"
}
]
}
}
The tag pluck worked for me. Here is the example :
{
"someOtherKey": "val",
properties: {
entry: payload pluck
{
key:$$,
value:$
}
}
}
Use mapObject instead of map
%dw 1.0
%output application/json
---
{
key: "val",
key1: "val1",
properties: {
entry: payload mapObject {
key:$$,
value:$
}
}
}
Hope this helps.