How to map certain keys of an array to make another one in mule 4 - mule

I have an array -
[
{
"Number": "12345",
"abc": {
"group": "abc",
"operation": "Create"
},
"def": {
"group": "def",
"operation": "Create"
}
},
{
"Number": "45678",
"xyz": {
"group": "xyz",
"operation": "Update"
},
"sdf": {
"group": "sfd",
"operation": "Delete"
}
}
]
and need to convert into this form -
[
{
"Number": "12345",
"group": "abc",
"operation": "Create"
},
{
"Number": "12345",
"group": "def",
"operation": "Create"
},
{
"Number": "45678",
"group": "xyz",
"operation": "Update"
},
{
"Number": "45678",
"group": "sfd",
"operation": "Delete"
}
]
Trying to write dataweave expression for the same. The issue is that abc, def, xyz and all are objects which may or maynot come and can have different values.

Another way to handle this:
%dw 2.0
output application/json
---
payload flatMap ((item, index) ->
(item - "Number") pluck {
"Number": item.Number,
($)
}
)
The approach is mostly the same, but here is the explanation: we use map to iterate, but with flatMap instead since we know we will be returning multiple items from each instance. Then the first thing we do is remove the key Number from the item since we only want to build a new object for each key that isn't Number. Then we can pluck, which gives us access to each key and value; from here we build a new object with our item's number value, and expand the entire object we plucked into that object. When using an anonymous function like this, the $, $$, $$$, etc represent the functions parameters - in pluck's case value, key, index. The parentheses we put around $ means to expand the entire object into our object; in javascript this is similar to { ...props, anotherKey: 'value' }. This means we don't really need to know or care about the structure of that object, which is useful if we have a potentially flexible schema.

You need to map each element, then filter each object to eliminate the attribute Number, and use pluck to convert each remaining key into an array. I used flatMap to concatenate each resulting array from each pluck into the response.
%dw 2.0
output application/json
---
payload flatMap ((item, index) ->
item
filterObject ((value, key, index) -> !(key ~= "Number"))
pluck ((value, key, index) -> {Number: item.Number, group: value.group, operation: value.operation})
)

Related

How to use try function with map in dataweave 2.0

Hi everybody i hope you are well, i have a doubt how can i do to use the try inside map in dataweave, i explain about my issue, i receive a csv file with multiple rows, that rows are group by order first and the second, i use map to transform the data in json ( to join all the rows with the same order) format using a couple colums that colums coming from the csv file, if any colum has empty or null the map fail and broke all the content file, how can i use the try function in dataweave if any group of orders fail only get the order and put in another part of the json and follow with the next order without broke the loop.
Part of the CSV File - Demo:
number,date,upc,quantity,price
1234556,2022-08-04,4015,1,
1234556,2022-08-04,4019,1,2.00
1234556,2022-08-04,4016,1,3.00
1234557,2022-08-04,4015,1,3.00
Dataweave:
%dw 2.0
output application/json
---
payload groupBy ($.number) pluck $ map ( () -> {
"number": $[0].number,
"date": $[0].date,
"items": $ map {
"upc": $.upc,
"price": $.price as Number {format: "##,###.##"} as String {format: "##,###.00"},
"quantity": $.quantity
}
})
Error Message:
Unable to coerce `` as Number using `##,###.##` as format.
NOTE: if put the data in the position "price "= something the the issue are solve in the first row, but i need use the function try or what do you recomend, i cant validate all the elements in csv because this is a demo the complete file has a many columns... if you would has another coment to better my code i'd apreciated.
Expected Result: (I don't know if this is possible)
[
{
"data": [
{
"number":"1234557",
"date":"2022-08-04",
"items":[
{
"upc":"4015",
"price":"3.00",
"quantity":"1"
}
]
}
]
},
{
"Error":[
{
"number":"1234556",
"message":"Unable to coerce `` as Number using `##,###.##` as format."
}
]
}
]
best regards!!
Hi The closer I got from what you asked for was
%dw 2.0
output application/json
import * from dw::Runtime
fun safeMap<T, R>(items: Array<T>, callback: (item:T) -> R ): Array<R | {message: String}> =
items map ((item) -> try(() -> callback(item)) match {
case is {success: false} -> {message: $.error.message as String}
case is {success: true, result: R} -> $.result
})
---
payload
groupBy ($.number)
pluck $
safeMap ((item) -> {
"number": item[0].number,
"date": item[0].date,
"items": item safeMap {
"upc": $.upc,
"price": $.price as Number {format: "##,###.##"} as String {format: "##,###.00"},
"quantity": $.quantity
}
})
This uses a combination of map and try function.
And it outputs
[
{
"number": "1234556",
"date": "2022-08-04",
"items": [
{
"message": "Unable to coerce `` as Number using `##,###.##` as format."
},
{
"upc": "4019",
"price": "2.00",
"quantity": "1"
},
{
"upc": "4016",
"price": "3.00",
"quantity": "1"
}
]
},
{
"number": "1234557",
"date": "2022-08-04",
"items": [
{
"upc": "4015",
"price": "3.00",
"quantity": "1"
}
]
}
]
If you are looking to resolve the value of price if price is null/empty value in the input csv and get rid of the error(which is happening because it cannot format Null values to Number) , try adding default in case of empty/null values and formatting to String only when value exists, like below:
%dw 2.0
output application/json
---
payload groupBy ($.number) pluck $ map ( () -> {
"number": $[0].number,
"date": $[0].date,
"items": $ map {
"upc": $.upc,
"price": ($.price as String {format: "##,###.00"}) default $.price,
"quantity": $.quantity
}
})
Note:
For price, you don't need to convert to Number at all if you want your output as formatted string ultimately.

Merge fields from multiple nested arrays in JSON input into new payload array using Dataweave

I am using Anypoint Studio 7.3 and Mule 4.2.
Using the JSON input below, I would like to create new payloads to write to database tables including a payload which builds a list of customers from each record entry in the array to create a list entry looking like this:
[{
recordId,
customerId
}]
and the same for a list of transactions with each list entry looking like this:
[{
recordId
customerId
transactionId
}]
but when I try to transform the data the field values show either as null or as a list instead of a single field in the object like this:
{
"customers": [{
"record": "1234",
"customerId": [
"5435e1cd-146d-4aac-9164-4a2d80d5eccd"
]
}]
}
instead of this:
{
"customers": [{
"recordId": "1234",
"customerId": "5435e1cd-146d-4aac-9164-4a2d80d5eccd"
}]
}
JSON INPUT:
{
"records": [{
"recordId": "1234",
"customers": [{
"customerId": "1234",
"transactions": [{
"transactionId": "1234",
"prices": [{
"priceId": "1234",
"price": 1.00
}]
}]
}]
}]
}
Thanks for any help
In order to return the list of customers from each record
%dw 2.0
output application/java
---
payload.records flatMap
((record, index) ->
record.customers map ((customer, index) ->
{
recordId: record.recordId,
customerId: customer.customerId
}
)
)
And for returning list of transactions with each list entry
%dw 2.0
output application/json
---
payload.records flatMap
((record, index) ->
record.customers flatMap ((customer, index) ->
customer.transactions map ((transaction, index) ->
{
recordId: record.recordId,
customerId: customer.customerId,
transactionId: transaction.transactionId
}
)
)
)
The key part here is using flatMap for flattening nesting levels of arrays into one.

Dataweave (nested) map conditional mapping

Case
I have a json object (as imported from excell file, and it needs to be formatted like one as well afterwards). So i can not mess with the structure at all.
The input consists of a json formatted object with sheets, every sheet is an array of objects.
I want to add another object (the var 'append') to the array (as last item) when the sheetnumber equals the kwadrantcode in the 'append' object
input 1
The payload
{
"(sheet)1": [
{
"kwadrantId": 1.0,
"Team": "blauw1",
"timestamp": "2019-09-26T16:37:54",
"controlecode": 12431
}
],
"(sheet)2": [
{
"kwadrantId": 2.0,
"Team": "rood1",
"timestamp": "2019-09-26T16:37:54",
"controlecode": 81243
},
{
"kwadrantId": 2.0,
"Team": "blauw2",
"timestamp": "2019-09-26T18:00:54",
"controlecode": 67676
}
]
}
the code so far
%dw 2.0
output application/json
var append = {"kwadrant": "2",
"controlecode": "22222",
"kleur": "blauw",
"subteam": "1",
"form_id": "83177",
"submit": "Claim"
}
---
payload pluck($) map (sheet, sheetnumber)-> (sheet map (claim, claimcounter) -> claim) ++ [append]
This code succesfully appends the object to the array at the right location.
The Problem here is that this last part (the map target) does not seem to allow a conditional.
So it will always insert it at the end of every list, not just the one where i want it to end up.
[
[
{
"kwadrantId": 1.0,
"Team": "blauw1",
"timestamp": "2019-09-26T16:37:54",
"controlecode": "12431"
},
{
"kwadrant": "2",
"controlecode": "22222",
"kleur": "blauw",
"subteam": "1",
"form_id": "83177",
"submit": "Claim"
}
],
[
{
"kwadrantId": 2.0,
"Team": "rood1",
"timestamp": "2019-09-26T16:37:54",
"controlecode": "12431"
},
{
"kwadrantId": 2.0,
"Team": "blauw2",
"timestamp": "2019-09-26T18:00:54",
"controlecode": 67676.0
},
{
"kwadrant": "2",
"controlecode": "22222",
"kleur": "blauw",
"subteam": "1",
"form_id": "83177",
"submit": "Claim"
}
]
]
A few things i have tried
honestly, i think i've tried everything the past two hours, but here are a few i can think of out of the top of my head.
++ [append] if (true)
(if true )++ [append]
payload pluck($) map (sheet, sheetnumber)-> (sheet map (claim, claimcounter) -> claim)((if true) ++ [append])
and most variations i can think of with brackets
I will worry about formatting the 'append' var to the right structure later, just getting it in the right spot is the issue i can't seem to get working.
The simplest way that I can think of is to use the mapObject function
%dw 2.0
output application/json
var append =
{
"kwadrant": "2",
"controlecode": "22222",
"kleur": "blauw",
"subteam": "1",
"form_id": "83177",
"submit": "Claim"
}
fun extractNumber(pageName: Key) =
(pageName as String match /\(sheet\)([0-9]+)/)[1]
---
payload mapObject ((value, key, index) -> do {
if(extractNumber(key) == append.kwadrant)
{(key): value << append}
else
{(key): value}
})
So basically I will go for every sheet in the root and when it matches the one I need to add append it to the value using the <<

csv to json conversion where some tags may not come in random

I have a csv input like below: Notice that the tag Edible is not coming for the second set of values. Also notice that the data for one object is coming in columns as well as three rows:
Key|Value|ConcatenatedString
Name|Fruit|"apple,orange,pineapple"
Status|Good|"apple,orange,pineapple"
Edible|Yes|"apple,orange,pineapple"
Name|Furniture|"chair,table,bed"
Status|Good|"chair,table,bed"
I need it in the below json format:
{
Name:"Fruit",
Status:"Good",
Edible:"Yes"
ConcatenatedString:"apple,orange,pineapple"
},
{
Name:"Furniture",
Status:"Good",
Edible:null
ConcatenatedString:"chair,table,bed"
}
I was using the below code when all the tags were coming for all objects. But now that some tags may not come at all I am not sure how to handle this as I was using a position based approach:
%dw 2.0
input payload application/csv separator='|'
output application/json
---
payload map
{
Name:payload[(($$)*4)+0].Value,
Status:payload[(($$)*4)+1].Value,
Edible:payload[(($$)*4)+2].Value,
ConcatenatedString:payload[(($$)*4)+0]."ConcatenatedString"
}
filter ($.Name!= null)
Thanks in advance,
Anoop
here is my answer.
%dw 2.0
input payload application/csv separator="|"
output application/json
---
payload
groupBy ((item, index) -> item.ConcatenatedString)
pluck ((value, key, index) -> {
Name: (value filter ((item, index) -> item.Key == "Name")).Value[0],
Status: (value filter ((item, index) -> item.Key == "Status")).Value[0],
Edible: (value filter ((item, index) -> item.Key == "Edible")).Value[0],
ConcatenatedString: key
})
Basically first you need to group by the criteria you want to group by. In your case ConcatenatedString. This returns
{
"chair,table,bed": [
{
"Key": "Name",
"Value": "Furniture",
"ConcatenatedString": "chair,table,bed"
},
{
"Key": "Status",
"Value": "Good",
"ConcatenatedString": "chair,table,bed"
}
],
"apple,orange,pineapple": [
{
"Key": "Name",
"Value": "Fruit",
"ConcatenatedString": "apple,orange,pineapple"
},
{
"Key": "Status",
"Value": "Good",
"ConcatenatedString": "apple,orange,pineapple"
},
{
"Key": "Edible",
"Value": "Yes",
"ConcatenatedString": "apple,orange,pineapple"
}
]
}
And then you iterate with pluck by every key value pair and filter the elements you want to map.

Modify Existing Json key : Mule

My Input
{
"Root": {
"order": [
{
"locale": "en-US",
"orderItems": [
{
"product": {
"partNumber": "23853864"
},
"itemSpecifics": {
"options": {
"color": "Olive",
"size": "S"
},
"actualPrice": "7",
"customItemData": {
"TEMP_8401": "8.95",
"TEMP_150207": "3.00"
}
}
}
]
}
... Large amount of JSON Data ...
]
}
}
Expected output
{
"Root": {
"order": [
{
"locale": "en-US",
"orderItems": [
{
"product": {
"partNumber": "23853864"
},
"itemSpecifics": {
"options": {
"color": "Olive",
"size": "S"
},
"actualPrice": "7",
"customItemData": {
"8401": "8.95",
"150207": "3.00"
}
}
}
]
}
... Large amount of JSON Data ...
]
}
}
I want to remove "TEMP_" in the "customItemData" object keys, but I don't want to manually remap the entire JSON object again, assigning properties one by one. Is there any alternative? Any shorter logic in DataWeave? I'm using Mule 3.9.0.
This uses recursion and pattern matching to walk through the data structure and modify the keys. If the key doesn't contain the string "TEMP" it leaves it as is, if it does, it modifies it according to your requirements.
%dw 1.0
%output application/json
%function applyToKeys(e, fn)
e match {
:array -> $ map ((v) -> applyToKeys(v, fn)),
:object -> $ mapObject ((v, k) -> {(fn(k)): applyToKeys(v, fn)}),
default -> $
}
---
applyToKeys(payload,
((key) -> ((key as :string) splitBy "_" )[1]
when ((key as :string) contains "TEMP")
otherwise key))
Please keep in mind the tradeoffs when using a solution like this. The code is certainly less verbose, but it requires a solid understanding of advanced concepts like recursion, pattern matching, lambdas, and higher order functions.