When the location and gl fields contains the same values we need to perform the sum operation for the amount and the input is as follows:
location,gl,amt
1,11,1
1,11,2
1,12,1
1,12,2
2,11,1
2,12,2
The required output is as follows:
[
{
"loc": "1",
"gl": "11",
"amt": 3
},
{
"loc": "1",
"gl": "12",
"amt": 3
},
{
"loc": "2",
"gl": "11",
"amt": "1"
},
{
"loc": "2",
"gl": "12",
"amt": "2"
}
]
Because the grouping really requires more than one level (location and gl) I have this convoluted answer. There may some way to simplify it.
%dw 2.0
output application/json
---
flatten((payload groupBy ($.location) mapObject (value, key) -> (
(key): value groupBy ($.gl) pluck
($$): $.amt map ($ as Number)
)) pluck ((value1, key1, index1) -> value1 map ($ mapObject ((value, key, index) -> {
location: key1,
gl: key,
amt: sum(value)
}))))
Output:
[
{
"location": "1",
"gl": "11",
"amt": 3
},
{
"location": "1",
"gl": "12",
"amt": 3
},
{
"location": "2",
"gl": "11",
"amt": 1
},
{
"location": "2",
"gl": "12",
"amt": 2
}
]
You can perform the groupBy with two fields by using string interpolation, then use pluck to create an array of array's that contain grouped records. Then I flatMap through the collection, I set loc and gl using the first records in that collection since those values aren't expected to change, then use sum() to add together the amount array
%dw 2.0
output application/json
var groupedRecords = payload groupBy "$($.location)-$($.gl)" pluck $
---
groupedRecords flatMap (item) -> do {
var firstItem = item[0]
var amtArr = item.amt
---
{
loc: firstItem.location,
gl: firstItem.gl,
amt: sum(amtArr)
}
}
Related
I have the following scenario and problem, I recive by CSV File and mapping with DW, groupping by column "PON", i need to get the total of the order multiply this column ( Qty * Price ), I don't have the correct result, I will show you:
CSV Data:
PON,Item,Qty,Price
PON1000,2015,2,38.08
PON1000,2016,1,33.37
PON1001,2015,2,38.08
DW:
%dw 2.0
output application/json
---
payload groupBy ($.PON) pluck $ map ( () -> {
"order": $[0].PON default "",
"total": (sum( $.Price filter ($ != "") ) as Number) as String {format: "##,###.00"},
"products": $ map {
"product": $.Item,
"price": ($.Price as Number) as String {format: "##,###.00"},
"quantity": $.Qty
}
})
Obtained Result:
[
{
"order": "PON1000",
"total": "71.45",
"products": [
{
"product": "2015",
"price": "38.08",
"quantity": "2"
},
{
"product": "2016",
"price": "33.37",
"quantity": "1"
}
]
},
{
"order": "PON1001",
"total": "38.08",
"products": [
{
"product": "2015",
"price": "38.08",
"quantity": "2"
}
]
}
]
I NEED MULTIPLY BY ORDER THE "price" * "quantity" CORRESPONDENT AND FINALLY SUM THAT VALUE AND PUT IN THE COLUMN total by ORDER
Expected Result:
[
{
"order": "PON1000",
"total": "109.53",
"products": [
{
"product": "2015",
"price": "38.08",
"quantity": "2"
},
{
"product": "2016",
"price": "33.37",
"quantity": "1"
}
]
},
{
"order": "PON1001",
"total": "76.16",
"products": [
{
"product": "2015",
"price": "38.08",
"quantity": "2"
}
]
}
]
Any help would be appreciated. Thank you.
Best Regards!!!
I just added two things:
For the total I used Arrays::sumBy which let's you do an operation on each item and the it sums all the results.
I infered from your code you expect the price to be empty, so I created safeNumber() to check that and return 0 in case it's empty (default did not work).
This is the code I've got so far:
%dw 2.0
output application/json
import * from dw::core::Arrays
fun safeNumber(str) = if(isEmpty(str)) 0 else str as Number
---
payload groupBy ($.PON) pluck $ map ( () -> {
"order": $[0].PON default "",
"total": ($ sumBy( (i) -> safeNumber(i.Qty) * safeNumber(i.Price))) as String {format: "##,##0.00"},
"products": $ map {
"product": $.Item,
"price": safeNumber($.Price) as String {format: "##,##0.00"},
"quantity": $.Qty
}
})
Below script will help you.
%dw 2.0
output application/json
import divideBy from dw::core::Objects
---
payload groupBy $.PON mapObject ((value, key, index) ->
({
order: (key),
total: sum(value map ($.Qty * $.Price)),
value : value
})
) divideBy 3
You can use the following DataWeave expression:
%dw 2.0
output application/json
---
payload groupBy $.PON pluck $ map (() -> {
"order": $[0].PON,
"total": sum($ map ($.Qty as Number * $.Price as Number)),
"products": $ map {
"product": $.Item,
"price": $.Price as Number,
"quantity": $.Qty as Number
}
})
I'm looking for an output similar to this one below where i want to groupBy costomer and orderid.
Input:
[
{
"item": 621,
"orderid": "ON22",
"qty": 45.0,
"customer": "610",
"date": "1988-08-13"
},
{
"item": 63,
"orderid": "ON2234",
"qty": 7,
"customer": "813",
"date": "2001-08-13"
}
]
Desired output:
[
{
"customer":"813",
"data":[
{
"item":63,
"qty":7,
"orderid":"ON2234",
"date":"2001-08-13"
}
]
},
{
"customer":"610",
"data":[
{
"item": 621,
"qty": 45.0,
"orderid": "ON22",
"date": "1988-08-13"
}
]
}
]
You can simply map the default output of your groupBy result since your output does not require any additional logic.
%dw 2.0
output application/json
---
payload groupBy $.customer pluck ((customerOrders, customerId) -> {
customer: customerId as String,
data: customerOrders
})
There is an array of parent elements that you need filter and remove if the sky are empty and get the number and remove all the coincidences of the principal array.
Input:
[{
"number": "7358",
"sku": "301-01"
}, {
"number": "7358",
"sku": "301-02"
}, {
"number": "7359",
"sku": ""
}, {
"number": "7359",
"sku": "301-04"
}, {
"number": "7356",
"sku": ""
}, {
"number": "7356",
"sku": "301-05"
}, {
"number": "7356",
"sku": "301-07"
}]
Output:
[{
"number": "7358",
"sku": "301-01"
}, {
"number": "7358",
"sku": "301-02"
}]
In Output we only find the elements that complied with having their sku with content.
Try with this script:
The idea is to collect all number(s) where the sku is "" and create an array out of these. After that you can iterate through the payload and filter out objects where the number in the object is present in the array created in the previous step.
%dw 2.0
output application/json
var atleastOneEmptySku = (payload filter ($.sku == ""))..number
---
payload filter (!(atleastOneEmptySku contains $.number))
In the below array there can be multiple objects, and each object can have more than below mentioned key values. I need only the uniqueID and name fields to be displayed.
[
{
"uniqueID": "1",
"Name": "Annie",
"Standard": "3",
"School" : "ABC School"
},
{
"uniqueID": "2",
"Name": "Apoo",
"Standard": "4",
"School" : "PQR School"
},
{
"uniqueID": "3",
"Name": "Xavier",
"Standard": "5",
"School" : "MNO School"
}
]
Desired output:
{
"errors": [
{
"uniqueID": "1",
"Name": "Annie"
},
{
"uniqueID": "2",
"Name": "Apoo"
},
{
"uniqueID": "3",
"Name": "Xavier"
}
]
}
Couple of approaches.
Script
%dw 2.0
output application/json
---
errors: payload map {
uniqueID: $.uniqueID,
Name: $.Name
}
%dw 2.0
output application/json
---
errors: payload map {
($ - "Standard" - "School")
}
In addition to Salim Khan's answer, another way is to use filterObject() to let only the desired attributes in the output. The advantage is that it is somewhat more generic if other attributes are added or changed.
%dw 2.0
output application/json
var allowedKeydNames=["uniqueID", "Name"]
---
{
errors: payload map (
$ filterObject ((value, key, index) -> (allowedKeydNames contains (key as String)))
)
}
I have a JSON structure with two arrays saved in a JSONB column. A bit simplified it looks like this
{
"prop1": "abc",
"prop2": "xyz",
"items": [
{
"itemId": "123",
"price": "10.00"
},
{
"itemId": "124",
"price": "9.00"
},
{
"itemId": "125",
"price": "8.00"
}
],
"groups": [
{
"groupId": "A",
"discount": "20",
"discountId": "1"
},
{
"groupId": "B",
"discount": "30",
"discountId": "2"
},
{
"groupId": "B",
"discount": "20",
"discountId": "3"
},
{
"groupId": "C",
"discount": "40",
"discountId": "4"
}
]
}
Schema:
CREATE TABLE campaign
(
id TEXT PRIMARY KEY,
data JSONB
);
Since each row (data column) can be fairly large, I'm trying to filter out matching item objects and group objects from the items and groups arrays.
My current query is this
SELECT * FROM campaign
WHERE
(data -> 'items' #> '[{"productId": "123"}]') OR
(data -> 'groups' #> '[{"groupId": "B"}]')
which returns rows containing either the matching group or the matching item. However, depending on the row, the data column can be a fairly large JSON object (there may be hundreds of objects in items and tens in groups and I've omitted several keys/properties for brevity in this example) which is affecting query performance (I've added GIN indexes on the items and groups arrays, so missing indices is not why it's slow).
How can I filter out the items and groups arrays to only contain matching elements?
Given this matching row
{
"prop1": "abc",
"prop2": "xyz",
"items": [
{
"itemId": "123",
"price": "10.00"
},
{
"itemId": "124",
"price": "9.00"
},
{
"itemId": "125",
"price": "8.00"
}
],
"groups": [
{
"groupId": "A",
"discount": "20",
"discountId": "1"
},
{
"groupId": "B",
"discount": "30",
"discountId": "2"
},
{
"groupId": "B",
"discount": "20",
"discountId": "3"
},
{
"groupId": "C",
"discount": "40",
"discountId": "4"
}
]
}
I'd like the result to be something like this (the matching item/group could be in different columns from the rest of the data column - doesn't have to be returned in a single JSON object with two arrays like this, but I would prefer it if doesn't affect performance or lead to a really hairy query):
{
"prop1": "abc",
"prop2": "xyz",
"items": [
{
"itemId": "123",
"price": "10.00"
}
],
"groups": [
{
"groupId": "B"
"discount": "20",
"discountId": "3"
}
]
}
What I've managed to do so far is unwrap and match an object in the items array using this query, which removes the 'items' array from the data column and filters out the matching item object to a separate column, but I'm struggling to join this with matches in the groups array.
SELECT data - 'items', o.obj
FROM campaign c
CROSS JOIN LATERAL jsonb_array_elements(c.data #> '{items}') o(obj)
WHERE o.obj ->> 'productId' = '124'
How can I filter both arrays in one query?
Bonus question: For the groups array I also want to return the object with the lowest discount value if possible. Or else the result would need to be an array of matching group objects instead of a single matching group.
Related questions: How to filter jsonb array elements and How to join jsonb array elements in Postgres?
If your postgres version is 12 or more, you can use the jsonpath language and functions. The query below returns the expected result with the subset of items and groups which match the given criteria. Then you can adapt this query within a sql function so that the search criteria is an input parameter.
SELECT jsonb_set(jsonb_set( data
, '{items}'
, jsonb_path_query_array(data, '$.items[*] ? (#.itemId == "123" && #.price == "10.00")'))
, '{groups}'
, jsonb_path_query_array(data, '$.groups[*] ? (#.groupId == "B" && #.discount == "20" && #.discountId == "3")'))
FROM (SELECT
'{
"prop1": "abc",
"prop2": "xyz",
"items": [
{
"itemId": "123",
"price": "10.00"
},
{
"itemId": "124",
"price": "9.00"
},
{
"itemId": "125",
"price": "8.00"
}
],
"groups": [
{
"groupId": "A",
"discount": "20",
"discountId": "1"
},
{
"groupId": "B",
"discount": "30",
"discountId": "2"
},
{
"groupId": "B",
"discount": "20",
"discountId": "3"
},
{
"groupId": "C",
"discount": "40",
"discountId": "4"
}
]
}' :: jsonb) AS d(data)
WHERE jsonb_path_exists(data, '$.items[*] ? (#.itemId == "123" && #.price == "10.00")')
AND jsonb_path_exists(data, '$.groups[*] ? (#.groupId == "B" && #.discount == "20" && #.discountId == "3")')