I have data like this
{
"ref": Ref(Collection("categories"), "1"),
"data": {
"title": "Game"
}
}
And
[
{
"ref": Ref(Collection("courses"), "1"),
"data": {
"categoryRef": Ref(Collection("categories"), "1"),
"courseTitle": "101",
"description": "Some1"
}
},
{
"ref": Ref(Collection("courses"), "2"),
"data": {
"categoryRef": Ref(Collection("categories"), "1"),
"courseTitle": "102",
"description": "Some2"
}
},
{
"ref": Ref(Collection("courses"), "3"),
"data": {
"categoryRef": Ref(Collection("categories"), "2"),
"courseTitle": "103",
"description": "Some3"
}
}
]
I just want to get all courses that belong to the selected category, in this case game
So result will be these.
[
{
"ref": Ref(Collection("courses"), "1"),
"data": {
"categoryRef": Ref(Collection("categories"), "1"),
"courseTitle": "101",
"description": "Some1"
}
},
{
"ref": Ref(Collection("courses"), "2"),
"data": {
"categoryRef": Ref(Collection("categories"), "1"),
"courseTitle": "102",
"description": "Some2"
}
}
]
Is this just a simple index on categoryRef and search in it? should I pass ref as FQL like this? q.Ref(q.Collection('categories'), '1')
Or maybe something like this which I don't know how to finish it.
q.Map(
q.Paginate(q.Documents(q.Collection('courses'))),
q.Lambda('courseDoc',
q.Let({
course: q.Get(q.Var('courseDoc')),
categoryRef: q.Get(q.Select(['data', 'categoryRef'], q.Var('course'))),
You need an index to perform searching on collections of arbitrary size. Only small collections are amenable to the "table scan" approach.
If create an index, like this:
> CreateIndex({
name: "courses_by_category",
source: Collection("courses"),
terms: [
{ field: ["data", "categoryRef"] },
],
})
Then you can search for course documents with a query like this:
> Paginate(
Match(
Index("courses_by_category"),
Ref(Collection("categories"), "1")
)
)
{
data: [ Ref(Collection("courses"), "1"), Ref(Collection("courses"), "2") ]
}
To fetch the entire document for matching entries:
> Map(
Paginate(
Match(
Index("courses_by_category"),
Ref(Collection("categories"), "1")
)
),
Lambda("ref", Get(Var("ref")))
)
{
data: [
{
ref: Ref(Collection("courses"), "1"),
ts: 1663619044380000,
data: {
categoryRef: Ref(Collection("categories"), "1"),
courseTitle: '101',
description: 'Some1'
}
},
{
ref: Ref(Collection("courses"), "2"),
ts: 1663619067090000,
data: {
categoryRef: Ref(Collection("categories"), "1"),
courseTitle: '102',
description: 'Some2'
}
}
]
}
Related
I'm trying to filter an array based on some values nested in objects.
My data pertains to offers (array), customers(array) with tickets(array) and other child arrays.
I want to orderBy to get all the customers information ordered by the latest timeStamp (attribute in tickets array)
From the example, offer 1 has customer 50001 with tickets 1001, 1002 and customer 50002 with tickets 1003, 1004. I want the customer which has latest timestamp in all the tickets available to be displayed first: (Desc order) with all the other passengers ordered accordingly.
Request Payload:
{
"count": 1,
"offers": [{
"offerInfo": {
"orderNumber": "1",
"orderCreationDtTime": "2023-01-10 00:00:00"
},
"customers": [{
"customerInfo": {
"name": {
"frstNm": "JOHN",
"lstNm": "DOE"
}
},
"customerNum": "50001",
"tickets": [{
"timestamp": "2023-01-07 00:38:00.167000",
"ticketService": {
"ticketNum": "1001",
"ticketType": "3"
},
"ticketReps": [{
"seq": "1",
"comment": "1st",
"location": "US"
},
{
"seq": "2",
"comment": "2nd",
"location": "US"
}
]
},
{
"timestamp": "2023-01-11 00:38:00.167000",
"ticketService": {
"ticketNum": "1002",
"ticketType": "3"
},
"ticketReps": [{
"seq": "3",
"comment": "1st",
"location": "US"
},
{
"seq": "4",
"comment": "2nd",
"location": "US"
}
]
}
]
},
{
"customerInfo": {
"name": {
"frstNm": "FAN",
"lstNm": "SING"
}
},
"customerNum": "50002",
"tickets": [{
"timestamp": "2023-01-10 00:38:00.167000",
"ticketService": {
"ticketNum": "1003",
"ticketType": "3"
},
"ticketReps": [{
"seq": "1",
"comment": "1st",
"location": "US"
},
{
"seq": "2",
"comment": "2nd",
"location": "US"
}
]
},
{
"timestamp": "2023-01-19 00:38:00.167000",
"ticketService": {
"ticketNum": "1004",
"ticketType": "3"
},
"ticketReps": [{
"seq": "1",
"comment": "1st",
"location": "US"
},
{
"seq": "2",
"comment": "2nd",
"location": "US"
}
]
}
]
}
]
}]
}
Expecting Payload after transform message:
{
"count": 1,
"offers": [{
"offerInfo": {
"orderNumber": "1",
"orderCreationDtTime": "2023-01-10 00:00:00"
},
"customers": [{
"customerInfo": {
"name": {
"frstNm": "FAN",
"lstNm": "SING"
}
},
"customerNum": "50002",
"tickets": [{
"timestamp": "2023-01-19 00:38:00.167000",
"ticketService": {
"ticketNum": "1004",
"ticketType": "3"
},
"ticketReps": [{
"seq": "1",
"comment": "1st",
"location": "US"
},
{
"seq": "2",
"comment": "2nd",
"location": "US"
}
]
},
{
"timestamp": "2023-01-10 00:38:00.167000",
"ticketService": {
"ticketNum": "1003",
"ticketType": "3"
},
"ticketReps": [{
"seq": "1",
"comment": "1st",
"location": "US"
},
{
"seq": "2",
"comment": "2nd",
"location": "US"
}
]
}
]
},
{
"customerInfo": {
"name": {
"frstNm": "JOHN",
"lstNm": "DOE"
}
},
"customerNum": "50001",
"tickets": [{
"timestamp": "2023-01-11 00:38:00.167000",
"ticketService": {
"ticketNum": "1002",
"ticketType": "3"
},
"ticketReps": [{
"seq": "3",
"comment": "1st",
"location": "US"
},
{
"seq": "4",
"comment": "2nd",
"location": "US"
}
]
},
{
"timestamp": "2023-01-07 00:38:00.167000",
"ticketService": {
"ticketNum": "1001",
"ticketType": "3"
},
"ticketReps": [{
"seq": "1",
"comment": "1st",
"location": "US"
},
{
"seq": "2",
"comment": "2nd",
"location": "US"
}
]
}
]
}
]
}]
}
Below script will help you.
%dw 2.0
output application/json
import * from dw::util::Values
---
payload update ["offers","customers"] with (
(($ map (
$ update {
case .tickets -> ($ orderBy $.timestamp as LocalDateTime {format : "yyyy-MM-dd HH:mm:ss.SSSSSS"}) [-1 to 0]
}
)) orderBy $.tickets[0].timestamp as LocalDateTime {format : "yyyy-MM-dd HH:mm:ss.SSSSSS"}) [-1 to 0]
)
A solution using the update operator for each step and auxiliary functions for clarity.
%dw 2.0
output application/json
fun convertTimestampToNumber(t)=t as LocalDateTime {format: "yyyy-MM-dd HH:mm:ss.SSSSSS"} as String {format: "yyyyMMddHHmmssSSSSSS"} as Number
fun getMaxTimestamp(t)=max(t map convertTimestampToNumber($.timestamp))
---
payload update {
case offers at .offers ->
offers map ($ update {
case customers at .customers ->
customers map ($ update {
case tickets at .tickets -> tickets orderBy ( -convertTimestampToNumber($.timestamp) )
})
orderBy ( -getMaxTimestamp($.tickets) )
}
)
}
Output:
{
"count": 1,
"offers": [
{
"offerInfo": {
"orderNumber": "1",
"orderCreationDtTime": "2023-01-10 00:00:00"
},
"customers": [
{
"customerInfo": {
"name": {
"frstNm": "FAN",
"lstNm": "SING"
}
},
"customerNum": "50002",
"tickets": [
{
"timestamp": "2023-01-19 00:38:00.167000",
"ticketService": {
"ticketNum": "1004",
"ticketType": "3"
},
"ticketReps": [
{
"seq": "1",
"comment": "1st",
"location": "US"
},
{
"seq": "2",
"comment": "2nd",
"location": "US"
}
]
},
{
"timestamp": "2023-01-10 00:38:00.167000",
"ticketService": {
"ticketNum": "1003",
"ticketType": "3"
},
"ticketReps": [
{
"seq": "1",
"comment": "1st",
"location": "US"
},
{
"seq": "2",
"comment": "2nd",
"location": "US"
}
]
}
]
},
{
"customerInfo": {
"name": {
"frstNm": "JOHN",
"lstNm": "DOE"
}
},
"customerNum": "50001",
"tickets": [
{
"timestamp": "2023-01-11 00:38:00.167000",
"ticketService": {
"ticketNum": "1002",
"ticketType": "3"
},
"ticketReps": [
{
"seq": "3",
"comment": "1st",
"location": "US"
},
{
"seq": "4",
"comment": "2nd",
"location": "US"
}
]
},
{
"timestamp": "2023-01-07 00:38:00.167000",
"ticketService": {
"ticketNum": "1001",
"ticketType": "3"
},
"ticketReps": [
{
"seq": "1",
"comment": "1st",
"location": "US"
},
{
"seq": "2",
"comment": "2nd",
"location": "US"
}
]
}
]
}
]
}
]
}
I have three collections
posts=[
{
"id": "p1",
"title": "title 1"
},
{
"id": "p2",
"title": "title 2"
}]
users = [
{
"id": "u1",
"name": "name1"
},
{
"id": "u2",
"name": "name2"
}]
comments = [
{
"userId": "u1",
"postId": "p1",
"comment": "comment 1"
}]
I want to get all collection posts and comments in each post by userId(u1) as:
posts=[
{
"id": "p1",
"title": "title 1",
"comments":[
"userId": "u1",
"comment": "comment 1"
]
},
{
"id": "p2",
"title": "title 2",
"comments":[]
}]
I used aggregate function and $lookup operator but I don't know using the $match operator to filter userId. I used aggregate bellow:
self.db.posts.aggregate([
{
"$lookup":{
"from": "comments",
"localField": "id",
"foreignField": "postId",
"as": "comments",
}
},
{
"$match":{
"comments.userId": {"$eq": param.objectUserId}
},
},
{"$skip": (param.page - 1) * param.pageSize},
{"$limit": param.pageSize},
{"$sort": {"unixDate": pymongo.DESCENDING}}
])
It only return one post in array corresponding with userId="u1"
Please help me!
Thank all!
You have to make use of the pipeline option of $lookup stage and pass the additional conditions that you want to apply.
db.posts.aggregate([
{
"$lookup": {
"from": "comments",
"let": {
"pId": "$id"
},
"pipeline": [
{
"$match": {
"$expr": {
"$eq": [
"$postId",
"$$pId"
],
},
"userId": "u1",
},
},
{
"$project": {
"_id": 0,
"userId": 1,
"comment": 1,
},
},
],
"as": "comments"
}
}
])
Mongo Playground Sample Execution
self.db.posts.aggregate([
{
"$lookup": {
"from": "comments",
"let": {
"pId": "$id"
},
"pipeline": [
{
"$match": {
"$expr": {
"$eq": [
"$postId",
"$$pId"
],
},
"userId": param.objectUserId,
},
},
{
"$project": {
"_id": 0,
"userId": 1,
"comment": 1,
},
},
],
"as": "comments"
}
},
{"$skip": (param.page - 1) * param.pageSize},
{"$limit": param.pageSize},
{"$sort": {"unixDate": pymongo.DESCENDING}}
])
how can I fetch the name and team_name keys in this API data?
condition: here 18,1,17, etc are subject codes that change according to the subject and not fix this subject available in the next API call.
{
"18": {
"detail": {
"id": "18",
"name": "Hindi"
},
"list": [
{
"id": "5",
"team_name": "Gurpreet",
},
{
"id": "2",
"team_name": "Test1",
}
]
},
"17": {
"detail": {
"id": "17",
"name": "Punjabi"
},
"list": [
{
"id": "6",
"team_name": "Guru",
},
{
"id": "3",
"team_name": "Test",
}
]
},
"1": {
"detail": {
"id": "1",
"name": "History"
},
"list": [
{
"id": "7",
"team_name": "Gurpreet",
}
]
},
"19": {
"detail": {
"id": "19",
"name": "Math"
},
"list": [
{
"id": "4",
"team_name": "Gurpreet",
}
]
},
"status": true
}
Use this code. You can check keys getter to check dynamics key.
import 'dart:convert';
void main() async {
var f = {
"18": {
"detail": {"id": "18", "name": "Hindi"},
"list": [
{
"id": "5",
"team_name": "Gurpreet",
},
{
"id": "2",
"team_name": "Test1",
}
]
},
"17": {
"detail": {"id": "17", "name": "Punjabi"},
"list": [
{
"id": "6",
"team_name": "Guru",
},
{
"id": "3",
"team_name": "Test",
}
]
},
"1": {
"detail": {"id": "1", "name": "History"},
"list": [
{
"id": "7",
"team_name": "Gurpreet",
}
]
},
"19": {
"detail": {"id": "19", "name": "Math"},
"list": [
{
"id": "4",
"team_name": "Gurpreet",
}
]
},
"status": true
};
for (var o in f.keys) {
print(o);
if (f[o] is bool) {
print(f[o]);
} else { // check it is Map. I consider it always is Map
if ((f[o] as Map)['detail'] != null) {
print((f[o] as Map)['detail']['name']);
}
if ((f[o] as Map)['list'] != null) {
print((f[o] as Map)['list'][0]['team_name']); // you can use for here. please check array is not null
}
}
}
}
Problem with using mapObject function properly.
Trying to retain existing array structure but calculate number of vehicles and properties and update the existing array that contains the value.
GENERAL data comes from one source, VEHICLE data comes from another source, PROPERTY data comes from another source. So when merging, I have to update GENERAL data with count of other source data.
Also GENERAL is an array object, it will always have 1. So using GENERAL[0] is safe and fine.
Original Payload
[
{
"commId": "1",
"GENERAL": [
{
"ID": "G1",
"VEHICLE_COUNT": "TODO",
"PROPERTY_COUNT": "TODO"
}
],
"VEHICLE": [
{
"ID": "V1-1"
},
{
"ID": "V1-2"
}
],
"PROPERTY": [
{
"ID": "P1-1"
}
]
},
{
"commId": "2",
"GENERAL": [
{
"ID": "G2",
"VEHICLE_COUNT": "TODO",
"PROPERTY_COUNT": "TODO"
}
],
"VEHICLE": [
{
"ID": "V2-1"
}
],
"PROPERTY": [
{
"ID": "P2-1"
},
{
"ID": "P2-2"
}
]
},
{
"commId": "3",
"GENERAL": [
{
"ID": "G3",
"VEHICLE_COUNT": "TODO",
"PROPERTY_COUNT": "TODO"
}
],
"VEHICLE": [
{
"ID": "V3-1"
},
{
"ID": "V3-2"
},
{
"ID": "V3-3"
}
]
}
]
Tried using map to loop through the payload and tried modifying 2 attribute but only managed to map one but even that is showing wrong output.
test map (item, index) -> {
(item.GENERAL[0] mapObject (value, key) -> {
(key): (value == sizeOf (item.VEHICLE)
when (key as :string) == "VEHICLE_COUNT"
otherwise value)
})
}
Expected output:
[
{
"commId": "1",
"GENERAL": [
{
"ID": "G1",
"VEHICLE_COUNT": "2",
"PROPERTY_COUNT": "1"
}
],
"VEHICLE": [
{
"ID": "V1-1"
},
{
"ID": "V1-2"
}
],
"PROPERTY": [
{
"ID": "P1-1"
}
]
},
{
"commId": "2",
"GENERAL": [
{
"ID": "G2",
"VEHICLE_COUNT": "1",
"PROPERTY_COUNT": "2"
}
],
"VEHICLE": [
{
"ID": "V2-1"
}
],
"PROPERTY": [
{
"ID": "P2-1"
},
{
"ID": "P2-2"
}
]
},
{
"commId": "3",
"GENERAL": [
{
"ID": "G3",
"VEHICLE_COUNT": "3",
"PROPERTY_COUNT": "0"
}
],
"VEHICLE": [
{
"ID": "V3-1"
},
{
"ID": "V3-2"
},
{
"ID": "V3-3"
}
]
}
]
Getting totally wrong output so far:
[
{
"ID": "G1",
"VEHICLE_COUNT": false,
"PROPERTY_COUNT": "TODO"
},
{
"ID": "G2",
"VEHICLE_COUNT": false,
"PROPERTY_COUNT": "TODO"
},
{
"ID": "G3",
"VEHICLE_COUNT": false,
"PROPERTY_COUNT": "TODO"
}
]
Edited: Update for dynamic transform
The below dataweave transform is not particularly attractive, but it might work for you.
Thanks to Christian Chibana for helping me find a dynmaic answer by answering this question: Why does Mule DataWeave array map strip top level objects?
%dw 1.0
%output application/json
---
payload map ((item) ->
(item - "GENERAL") ++
GENERAL: item.GENERAL map (
$ - "VEHICLE_COUNT"
- "PROPERTY_COUNT"
++ { VEHICLE_COUNT: sizeOf (item.VEHICLE default []) }
++ { PROPERTY_COUNT: sizeOf (item.PROPERTY default []) }
)
)
It is dynamic, so everything should be copied across as it comes in, with only the two fields you want being updated.
The output for this transform with the input you supplied is below. Only difference from your desired is that the counts are shown as numbers rather than strings. If you really need them as strings you can cast them like (sizeOf (comm.VEHICLE default [])) as :string,
[
{
"commId": "1",
"VEHICLE": [
{
"ID": "V1-1"
},
{
"ID": "V1-2"
}
],
"PROPERTY": [
{
"ID": "P1-1"
}
],
"GENERAL": [
{
"ID": "G1",
"VEHICLE_COUNT": 2,
"PROPERTY_COUNT": 1
}
]
},
{
"commId": "2",
"VEHICLE": [
{
"ID": "V2-1"
}
],
"PROPERTY": [
{
"ID": "P2-1"
},
{
"ID": "P2-2"
}
],
"GENERAL": [
{
"ID": "G2",
"VEHICLE_COUNT": 1,
"PROPERTY_COUNT": 2
}
]
},
{
"commId": "3",
"VEHICLE": [
{
"ID": "V3-1"
},
{
"ID": "V3-2"
},
{
"ID": "V3-3"
}
],
"GENERAL": [
{
"ID": "G3",
"VEHICLE_COUNT": 3,
"PROPERTY_COUNT": 0
}
]
}
]
I am trying to filter same level object values in the payload in Dataweave. I was able to loop through but it does not produce the expected output.
Sample Payload:
{
"root": {
"createItem": {
"itemInfo": {
"lines": [{
"lineIdentifier": "4",
"Attributes": "Test1",
"partNumber": "QZRB"
}, {
"lineIdentifier": "10",
"Attributes": "Test3",
"partNumber": "QPR1"
}, {
"lineIdentifier": "12",
"Attributes": "Test4",
"partNumber": "QHT2"
}]
}
},
"ItemResponse": {
"lines": [{
"lineIdentifier": 4,
"itemName": "QZRB",
"status": "FAILED"
}, {
"lineIdentifier": 10,
"itemName": "QPR1",
"status": "COMPLETE"
}, {
"lineIdentifier": 12,
"itemName": "QHT2",
"status": "COMPLETE"
}]
}
}
}
Expected Output:
{
"root": {
"createItem": {
"itemInfo": {
"lines": [ {
"lineIdentifier": "10",
"Attributes": "Test3",
"partNumber": "QPR1"
}, {
"lineIdentifier": "12",
"Attributes": "Test4",
"partNumber": "QHT2"
}]
}
}
}
}
Here's what I am doing:
{
root: {
(payload.root.createItem.itemInfo.lines map ((respLines, indexOfRespLines) -> {
items:payload.root.ItemResponse.lines filter ($.itemName == respLines.partNumber and $.status =='COMPLETE') map
{
item: $.itemName,
attributes: respLines.Attributes
}
}
)
)
}
}
How do I achieve this?
Thanks,
ROA
try this:
%dw 1.0
%output application/json
%var completedLines = payload.root.ItemResponse.lines filter $.status == 'COMPLETE' map $.lineIdentifier as :string
---
{
root: {
createItem: {
itemInfo: {
lines: payload.root.createItem.itemInfo.lines filter (completedLines contains $.lineIdentifier)
}
}
}
}
pay attention to as :string in completedLines, because the lineIdentifier in ItemResponse is a number, while in itemInfo it is a string.