Hi everybody i hope you are well, i have a doubt how can i do to use the try inside map in dataweave, i explain about my issue, i receive a csv file with multiple rows, that rows are group by order first and the second, i use map to transform the data in json ( to join all the rows with the same order) format using a couple colums that colums coming from the csv file, if any colum has empty or null the map fail and broke all the content file, how can i use the try function in dataweave if any group of orders fail only get the order and put in another part of the json and follow with the next order without broke the loop.
Part of the CSV File - Demo:
number,date,upc,quantity,price
1234556,2022-08-04,4015,1,
1234556,2022-08-04,4019,1,2.00
1234556,2022-08-04,4016,1,3.00
1234557,2022-08-04,4015,1,3.00
Dataweave:
%dw 2.0
output application/json
---
payload groupBy ($.number) pluck $ map ( () -> {
"number": $[0].number,
"date": $[0].date,
"items": $ map {
"upc": $.upc,
"price": $.price as Number {format: "##,###.##"} as String {format: "##,###.00"},
"quantity": $.quantity
}
})
Error Message:
Unable to coerce `` as Number using `##,###.##` as format.
NOTE: if put the data in the position "price "= something the the issue are solve in the first row, but i need use the function try or what do you recomend, i cant validate all the elements in csv because this is a demo the complete file has a many columns... if you would has another coment to better my code i'd apreciated.
Expected Result: (I don't know if this is possible)
[
{
"data": [
{
"number":"1234557",
"date":"2022-08-04",
"items":[
{
"upc":"4015",
"price":"3.00",
"quantity":"1"
}
]
}
]
},
{
"Error":[
{
"number":"1234556",
"message":"Unable to coerce `` as Number using `##,###.##` as format."
}
]
}
]
best regards!!
Hi The closer I got from what you asked for was
%dw 2.0
output application/json
import * from dw::Runtime
fun safeMap<T, R>(items: Array<T>, callback: (item:T) -> R ): Array<R | {message: String}> =
items map ((item) -> try(() -> callback(item)) match {
case is {success: false} -> {message: $.error.message as String}
case is {success: true, result: R} -> $.result
})
---
payload
groupBy ($.number)
pluck $
safeMap ((item) -> {
"number": item[0].number,
"date": item[0].date,
"items": item safeMap {
"upc": $.upc,
"price": $.price as Number {format: "##,###.##"} as String {format: "##,###.00"},
"quantity": $.quantity
}
})
This uses a combination of map and try function.
And it outputs
[
{
"number": "1234556",
"date": "2022-08-04",
"items": [
{
"message": "Unable to coerce `` as Number using `##,###.##` as format."
},
{
"upc": "4019",
"price": "2.00",
"quantity": "1"
},
{
"upc": "4016",
"price": "3.00",
"quantity": "1"
}
]
},
{
"number": "1234557",
"date": "2022-08-04",
"items": [
{
"upc": "4015",
"price": "3.00",
"quantity": "1"
}
]
}
]
If you are looking to resolve the value of price if price is null/empty value in the input csv and get rid of the error(which is happening because it cannot format Null values to Number) , try adding default in case of empty/null values and formatting to String only when value exists, like below:
%dw 2.0
output application/json
---
payload groupBy ($.number) pluck $ map ( () -> {
"number": $[0].number,
"date": $[0].date,
"items": $ map {
"upc": $.upc,
"price": ($.price as String {format: "##,###.00"}) default $.price,
"quantity": $.quantity
}
})
Note:
For price, you don't need to convert to Number at all if you want your output as formatted string ultimately.
Related
I have the following scenario and problem, I recive by CSV File and mapping with DW, groupping by column "PON", i need to get the total of the order multiply this column ( Qty * Price ), I don't have the correct result, I will show you:
CSV Data:
PON,Item,Qty,Price
PON1000,2015,2,38.08
PON1000,2016,1,33.37
PON1001,2015,2,38.08
DW:
%dw 2.0
output application/json
---
payload groupBy ($.PON) pluck $ map ( () -> {
"order": $[0].PON default "",
"total": (sum( $.Price filter ($ != "") ) as Number) as String {format: "##,###.00"},
"products": $ map {
"product": $.Item,
"price": ($.Price as Number) as String {format: "##,###.00"},
"quantity": $.Qty
}
})
Obtained Result:
[
{
"order": "PON1000",
"total": "71.45",
"products": [
{
"product": "2015",
"price": "38.08",
"quantity": "2"
},
{
"product": "2016",
"price": "33.37",
"quantity": "1"
}
]
},
{
"order": "PON1001",
"total": "38.08",
"products": [
{
"product": "2015",
"price": "38.08",
"quantity": "2"
}
]
}
]
I NEED MULTIPLY BY ORDER THE "price" * "quantity" CORRESPONDENT AND FINALLY SUM THAT VALUE AND PUT IN THE COLUMN total by ORDER
Expected Result:
[
{
"order": "PON1000",
"total": "109.53",
"products": [
{
"product": "2015",
"price": "38.08",
"quantity": "2"
},
{
"product": "2016",
"price": "33.37",
"quantity": "1"
}
]
},
{
"order": "PON1001",
"total": "76.16",
"products": [
{
"product": "2015",
"price": "38.08",
"quantity": "2"
}
]
}
]
Any help would be appreciated. Thank you.
Best Regards!!!
I just added two things:
For the total I used Arrays::sumBy which let's you do an operation on each item and the it sums all the results.
I infered from your code you expect the price to be empty, so I created safeNumber() to check that and return 0 in case it's empty (default did not work).
This is the code I've got so far:
%dw 2.0
output application/json
import * from dw::core::Arrays
fun safeNumber(str) = if(isEmpty(str)) 0 else str as Number
---
payload groupBy ($.PON) pluck $ map ( () -> {
"order": $[0].PON default "",
"total": ($ sumBy( (i) -> safeNumber(i.Qty) * safeNumber(i.Price))) as String {format: "##,##0.00"},
"products": $ map {
"product": $.Item,
"price": safeNumber($.Price) as String {format: "##,##0.00"},
"quantity": $.Qty
}
})
Below script will help you.
%dw 2.0
output application/json
import divideBy from dw::core::Objects
---
payload groupBy $.PON mapObject ((value, key, index) ->
({
order: (key),
total: sum(value map ($.Qty * $.Price)),
value : value
})
) divideBy 3
You can use the following DataWeave expression:
%dw 2.0
output application/json
---
payload groupBy $.PON pluck $ map (() -> {
"order": $[0].PON,
"total": sum($ map ($.Qty as Number * $.Price as Number)),
"products": $ map {
"product": $.Item,
"price": $.Price as Number,
"quantity": $.Qty as Number
}
})
I am trying to achieve the below output from the given input. I have tried several ways of making a generic function so that when any data is passed with similar structure I get the similar output. Please help me to achieve this.
Input
[{
"name": "Thinker",
"details": [
{
"num": 1
},
{
"num": 2
}
]
},
{
"name": "Blinker",
"details": [
{
"num": 3
},
{
"num": 4
}
]
}]
Output
[{
"name": "Thinker",
"num": 1
},
{
"name": "Thinker",
"num": 2
},
{
"name": "Blinker",
"num": 3
},
{
"name": "Blinker",
"num": 4
}]
The key is mapping through both outer and inner arrays and then flattening the result:
output application/json
---
flatten (payload map ((item, index) -> item.details map ((detail, index) -> {
name: item.name,
num: detail.num
})))
As Aled has mentioned in his comments to one of the answers, this could be another way to solve this.
%dw 2.0
output application/json
---
payload flatMap (item,index) -> (
item.details map {
name: item.name,
num: $.num
}
)
Try this DW
1st Map to Map on main array
2nd Map to Map on details
reduce to convert nested array to single array
%dw 2.0
output application/json
---
payload map(
($.details map(item,index)->{
"name":($.name),
"num":item[0]
})
)reduce($$++$)
Output
[
{
"name": "Thinker",
"num": 1
},
{
"name": "Thinker",
"num": 2
},
{
"name": "Blinker",
"num": 3
},
{
"name": "Blinker",
"num": 4
}
]
DW
%dw 2.0
output application/json
---
payload map(
($.details map(item,index)->{
(keysOf($)[0]):($.name),
(keysOf(item)[0]):item[0]
})
)reduce($$++$)
If your request structure will always remain this way(regardelss of field names), then you can try this script as below, that is more genric and will work on same type of strucure
%dw 2.0
output application/json
---
payload flatMap ((item, index) ->
(item filterObject ($ is Array))[0] map
($ ++ (item filterObject ($ is String | Number | Null)) )
)
Note: it can work on this type of structure always and not depend on field names, and gives expected output
[{
"anyname": "xyz",
"anydetails": [
{
"abc": 1
},
{
"ijk": 2
}
]
}]
I have an array -
[
{
"Number": "12345",
"abc": {
"group": "abc",
"operation": "Create"
},
"def": {
"group": "def",
"operation": "Create"
}
},
{
"Number": "45678",
"xyz": {
"group": "xyz",
"operation": "Update"
},
"sdf": {
"group": "sfd",
"operation": "Delete"
}
}
]
and need to convert into this form -
[
{
"Number": "12345",
"group": "abc",
"operation": "Create"
},
{
"Number": "12345",
"group": "def",
"operation": "Create"
},
{
"Number": "45678",
"group": "xyz",
"operation": "Update"
},
{
"Number": "45678",
"group": "sfd",
"operation": "Delete"
}
]
Trying to write dataweave expression for the same. The issue is that abc, def, xyz and all are objects which may or maynot come and can have different values.
Another way to handle this:
%dw 2.0
output application/json
---
payload flatMap ((item, index) ->
(item - "Number") pluck {
"Number": item.Number,
($)
}
)
The approach is mostly the same, but here is the explanation: we use map to iterate, but with flatMap instead since we know we will be returning multiple items from each instance. Then the first thing we do is remove the key Number from the item since we only want to build a new object for each key that isn't Number. Then we can pluck, which gives us access to each key and value; from here we build a new object with our item's number value, and expand the entire object we plucked into that object. When using an anonymous function like this, the $, $$, $$$, etc represent the functions parameters - in pluck's case value, key, index. The parentheses we put around $ means to expand the entire object into our object; in javascript this is similar to { ...props, anotherKey: 'value' }. This means we don't really need to know or care about the structure of that object, which is useful if we have a potentially flexible schema.
You need to map each element, then filter each object to eliminate the attribute Number, and use pluck to convert each remaining key into an array. I used flatMap to concatenate each resulting array from each pluck into the response.
%dw 2.0
output application/json
---
payload flatMap ((item, index) ->
item
filterObject ((value, key, index) -> !(key ~= "Number"))
pluck ((value, key, index) -> {Number: item.Number, group: value.group, operation: value.operation})
)
I have a requirement wherein I have to convert JSON data from one format to other.
I have to fetch corresponding values of JSON array and make them a key value pair.
Below are the required details:
Input:
"Headers": {
"Header": [
{
"Key": "SellerOrganization",
"Value": "XYZ"
},
{
"Key": "SellerType",
"Value": "B2C"
},
{
"Key": "Region",
"Value": "SOUTH"
},
{
"Key": "OrderType",
"Value": "RETURN"
},
{
"Key": "InvoiceType",
"Value": ""
},
{
"Key": "EventType",
"Value": "Created"
},
{
"Key": "EntryType",
"Value": "Call Center"
}
]
}
Expected Output:
{
SellerOrganization:XYZ,
SellerType: B2C,
Region:SOUTH,
OrderType:RETURN,
InvoiceType:"",
EventType:Created,
EntryType:Call Center
}
You can use the dynamic object that it will basically do what you want.
%dw 2.0
output application/json
---
{
(payload.Headers.Header map ((item, index) -> {
(item.Key): item.Value
})
)
}
You can take advantage of reduce function here which will let you convert your array to an key, value pair object
%dw 2.0
output application/json
---
payload.Header reduce ((item, acc = {}) -> acc ++ {
(item.Key): item.Value
})
I want to create a json message from 2 different Json message. Both input messages are coming from flow variables. How can it be done using Dataweave.
Message 1:
[
{
"userId": 11,
"name": "ABC",
"age": 30
},
{
"userId": 44,
"name": "XYZ",
"age": 30
}
]
Message 2:
[
{
"userId": 11,
"Address": "BLR"
},
{
"userId": 44,
"Address": "CCU"
}
]
Expected output:
[
{
"userId": 11,
"name": "ABC",
"Address": "BLR"
},
{
"userId": 44,
"name": "XYZ",
"Address": "CCU"
}
]
Thank You in advance,
Nitesh
Refer this article. Seems the perfect solution for your usecase.
You can use lookup instead of filter as using filter is performance overhead.
Try this
%dw 1.0
%output application/json
%var adressLookup = {( flowVars.data map {
($.userId ++ "") : $
})}
---
payload map {
userId : $.userId,
name : $.name,
Address : adressLookup[$.userId ++ ""].Address
}
where payload is message1 and flowVars.data is message2
P.S : I have used $.userId ++ "" as key in string format for hashmap, some how numeric key isn't work with weave hashmap.
also refer Merge two json payload with dataweave
Hope this helps.