My requirement is to generate a line number for every new line generated in the json message. The input message is having array inside array, i.e, parent and child array.
Input message
[
{
id:"1",
Details:[
{
Name:"RAM",
LastName:"Manohar",
DOB:"20-10-1990",
Report:[
{
DateOfJoin:"03-03-2019",
Dept:"HR",
BillCode:"acx-12s",
EffectiveDate:"03-03-2019"
},
{
DateOfJoin:"03-04-2019",
Dept:"HR",
BillCode:"abc-12s",
EffectiveDate:"03-04-2019"
},
{
Name:"Alex",
LastName:"Ham",
DOB:"20-11-1980",
Report:[
{
DateOfJoin:"03-03-2019",
Dept:"HR",
BillCode:"acx-12s",
EffectiveDate:"03-03-2019"
},
{
DateOfJoin:"03-04-2019",
Dept:"HR",
BillCode:"abc-12s",
EffectiveDate:"03-04-2019"
}
]
}
]
},
{
id:"2",
Details:[
{
Name:"Kiran",
LastName:"Kurella",
DOB:"20-10-1980",
Report:[
{
DateOfJoin:"03-03-2019",
Dept:"DC",
BillCode:"acx-12s",
EffectiveDate:"03-03-2019"
},
{
DateOfJoin:"03-04-2019",
Dept:"DC",
BillCode:"abc-12s",
EffectiveDate:"03-04-2019"
},
{
Name:"Sunil",
LastName:"Kumar",
DOB:"20-11-1980",
Report:[
{
DateOfJoin:"03-01-2019",
Dept:"DC",
BillCode:"acx-12s",
EffectiveDate:"03-03-2019"
},
{
DateOfJoin:"03-04-2019",
Dept:"DC",
BillCode:"abc-12s",
EffectiveDate:"03-04-2019"
}
]
}
]
}
]
}
]
}
]
expected output:
[{LineNumber:1,
Dept:"HR",
Name: "Ram"},
{LineNumber:2,
Dept:"HR",
Name: "Alex"},
{LineNumber:3,
Dept:"HR",
Name: "Kiran"},
{LineNumber:4,
Dept:"HR",
Name: "Sunil"}]
Linenumber needs to be generated sequentially and irrespective of parent array or sub array. any help on this will be very appreciated. I have the logic in which i can generate the number using java function but in that case i need to set the variable value (flow variable) inside data weave which can be used in the java function to call recursively.
Use:
payload map {
count: $$
}
Related
I have a collection "Owners" and I want to return a list of "Owner" matching a filter (any filter), plus the count of "Pet" from the "Pets" collection for that owner, except I don't want the dead pets. (made up example)
I need the returned documents to look exactly like an "Owner" document with the addition of the "petCount" field because I'm using Java Pojos with the Mongo Java driver.
I'm using AWS DocumentDB that does not support $lookup with filters yet. If it did I would use this and I'd be done:
db.Owners.aggregate( [
{ $match: {_id: UUID("b13e733d-2686-4266-a686-d3dae6501887")} },
{ $lookup: { from: 'Pets', as: 'pets', 'let': { ownerId: '$_id' }, pipeline: [ { $match: { $expr: { $ne: ['$state', 'DEAD'] } } } ] } },
{ $addFields: { petCount: { $size: '$pets' } } },
{ $project: { pets: 0 } }
]).pretty()
But since it doesn't this is what I got so far:
db.Owners.aggregate( [
{ $match: {_id: { $in: [ UUID("cbb921f6-50f8-4b0c-833f-934998e5fbff") ] } } },
{ $lookup: { from: 'Pets', localField: '_id', foreignField: 'ownerId', as: 'pets' } },
{ $unwind: { path: '$pets', preserveNullAndEmptyArrays: true } },
{ $match: { 'pets.state': { $ne: 'DEAD' } } },
{ "$group": {
"_id": "$_id",
"doc": { "$first": "$$ROOT" },
"pets": { "$push": "$pets" }
}
},
{ $addFields: { "doc.petCount": { $size: '$pets' } } },
{ $replaceRoot: { "newRoot": "$doc" } },
{ $project: { pets: 0 } }
]).pretty()
This works perfectly, except if an Owner only has "DEAD" pets, then the owner doesn't get returned because all the "document copies" got filtered out by the $match. I'd need the parent document to be returned with petCount = 0 when ALL of them are "DEAD". I cannot figure out how to do this.
Any ideas?
These are the supported operations for DocDB 4.0 https://docs.amazonaws.cn/en_us/documentdb/latest/developerguide/mongo-apis.html
EDIT: update to use $filter as $reduce not supported by aws document DB
You can use $filter to keep only not DEAD pets in the lookup array, then count the size of the remaining array.
Here is the Mongo playground for your reference.
$reduce version
You can use $reduce in your aggregation pipeline to to a conditional sum for the state.
Here is Mongo playground for your reference.
As of January 2022, Amazon DocumentDB added support for $reduce, the solution posted above should work for you.
Reference.
Am getting objects in a s3 buckets using following
s3 = boto3.resource(
service_name='s3',
aws_access_key_id=key_id,
aws_secret_access_key=secret
)
for summary_obj in s3.Bucket(bucket_name).objects.all():
print(summary_obj.key)
Its giving me all object like this
'sub1/sub1_1/file1.zip',
'sub1/sub1_2/file2.zip',
'sub2/sub2_1/file3.zip',
'sub3/file4.zip',
'sub4/sub4_1/file5.zip',
'sub5/sub5_1/file6.zip',
'sub5/sub5_2/file7.zip',
'sub5/sub5_3/file8.zip',
'sub6/'
But i want to have a list of json of all objects with proper directory structure like this to show in my app
[
{'sub1': [
{
'sub1_1': ['file1.zip'] // All files in sub1_1 folder
},
{
'sub1_2': ['file2.zip'] // All files in sub1_2 folder
},
]},
{'sub2': [
{
'sub2_1': [
'file3.zip'
]
}
]},
{'sub3': [
'file4.zip'
]},
{'sub4': [
{
'sub4_1': [
'file5.zip'
]
}
]},
{'sub5': [
{
'sub5_1': [
'file6.zip'
]
},
{
'sub5_2': [
'file7.zip'
]
},
{
'sub5_3': [
'file8.zip'
]
}
]},
{'sub6': []}
]
what is the best way to do this in python3.8?
I give it a try and the closest I could get to your json was through recursion which works with any level of sub-folders and folders:
from collections import defaultdict
objects=['sub1/sub1_1/file1.zip',
'sub1/sub1_2/file2.zip',
'sub2/sub2_1/file3.zip',
'sub3/file4.zip',
'sub4/sub4_1/file5.zip',
'sub5/sub5_1/file6.zip',
'sub5/sub5_2/file7.zip',
'sub5/sub5_3/file8.zip',
'sub5/sub5_3/file9.zip',
'sub5/sub5_3/sub5_4/file1.zip',
'sub5/sub5_3/sub5_4/file2.zip',
'sub6/']
#print(objects)
def construct_dict(in_list, accumulator):
if not in_list:
return
else:
if in_list[0] not in accumulator:
accumulator[in_list[0]] = defaultdict(list)
return construct_dict(in_list[1::], accumulator[in_list[0]])
accumulator = defaultdict(list)
for obj in objects:
construct_dict(obj.split('/'), accumulator)
print(json.dumps(accumulator))
Which gives (the content is same, but structure a bit different):
{
"sub1": {
"sub1_1": {
"file1.zip": {}
},
"sub1_2": {
"file2.zip": {}
}
},
"sub2": {
"sub2_1": {
"file3.zip": {}
}
},
"sub3": {
"file4.zip": {}
},
"sub4": {
"sub4_1": {
"file5.zip": {}
}
},
"sub5": {
"sub5_1": {
"file6.zip": {}
},
"sub5_2": {
"file7.zip": {}
},
"sub5_3": {
"file8.zip": {},
"file9.zip": {},
"sub5_4": {
"file1.zip": {},
"file2.zip": {}
}
}
},
"sub6": {
"": {}
}
}
I'm trying to fetch data from a single collection type of my Strapi backend into a Vue.js project using Apollo. It works well with a single alias, but I'm having troubles making it work with multiple aliases.
I'm getting my data from a collection type of "campaigns" which has a boolean field of "archive". I want to create an array of "campaigns" that contains all of the campaigns that haven't been archived (archive = false) as well as an array of "archive" that contains all of the archived ones (archive = true).
This is my code:
import gql from "graphql-tag";
export default {
name: "Campaigns",
data() {
return {
campaigns: [],
archive: []
};
},
apollo: {
campaigns: gql`
query getCampaigns {
campaigns: campaigns(where: { archive: "false" }, sort: "order:DESC") {
name
url
}
archive: campaigns(where: { archive: "true" }, sort: "order:DESC") {
name
url
}
}
`
}
The query returns an array of "campaigns", but the array of "archive" is still empty.
I've tried switching things up (put the archive alias first, switched the boolean values to make sure I can generally access the data of the archived campaigns etc.). The problem apparently lies with the "archive"-alias.
When I use the same query with Strapi's GraphQL playground I get the desired result:
{
campaigns: campaigns(where: { archive: "false" }, sort: "order:DESC") {
name
}
archive: campaigns(where: { archive: "true" }, sort: "order:DESC") {
name
}
}
... returns ...
{
"data": {
"campaigns": [
{
"name": "2020"
},
{
"name": "2019"
},
{
"name": "2018"
},
{
"name": "2017"
}
],
"archive": [
{
"name": "2016"
},
{
"name": "2015"
}
]
}
}
How can I make the query work in Vue.js with Apollo?
I think I've found a solution. Technically speaking I guess these are separate queries (which sort of defeats the purpose of aliases if I'm correct) but it does what I want:
apollo: {
campaigns: {
query: gql`
query {
campaigns: campaigns(
where: { archive: "false" }
sort: "order:desc"
) {
name
url
}
}
`
},
archive: {
query: gql`
query {
archive: campaigns(where: { archive: "true" }, sort: "order:desc") {
name
url
}
}
`
}
}
Apparently under some circumstance the initialization "apollo: { XYZ:" and the alias "query { XYZ:" have to match. I've seen in the docs that they don't necessarily have to match, but I don't fully understand when and why.
I guess I can't really tell what the initial parameter does.
You're using campaigns as the key for your entire query, so you need to initialize your data like this:
data() {
return {
campaigns: {
campaigns: [],
archive: [],
},
};
},
Then you can access each list through the key (i.e. campaigns.campaigns and campaigns.archive).
I believe the best way to do this is to use the update property: https://apollo.vuejs.org/guide/apollo/queries.html#name-matching
apollo: {
campaigns: {
query: gql`
query {
campaigns: campaigns(
where: { archive: "false" }
sort: "order:desc"
) {
name
url
}
}
`
},
archive: {
update: data => data.campaigns,
query: gql`
query {
campaigns(where: { archive: "true" }, sort: "order:desc") {
name
url
}
}
`
}
}
I want to do a replace in projection. Like a SQL Server REPLACE. I'm pretty sure we can handle that in code but looking for some shell commands.
Here is what I have
db.OrderHistoryHeader.aggregate([
{
$project:{
"_id":0,
"OrderNo":1 // I want to do Replace(OrderNo,'XYZ','ABC')
}
}
],
{
allowDiskUse:true
}).pretty();
There's no built-in operator for that currently but you can use $indexOfBytes combined with $substr and $concat.
db.OrderHistoryHeader.aggregate([
{
$addFields:
{
index: { $indexOfBytes: [ "$OrderNo", "XYZ" ] },
}
},
{
$project: {
OrderNo: {
$concat: [
{ $substr: [ "$OrderNo", 0, "$index" ] },
"ABC",
{ $substr: [ "$OrderNo", { $add: [3, "$index"] }, -1 ] }
]
}
}
},
{
$project: {
index: 0
}
}
])
Where 3 is the length of text being replaced.
You can use the replaceOne method
db.collection.replaceOne(filter, replacement, options)
From documentation:
Behavior
replaceOne() replaces the first matching document in the collection that matches the filter, using the replacement document.
upsert
If upsert: true and no documents match the filter, db.collection.replaceOne() creates a new document based on the replacement document.
I have this query that provides me the join I want to:
db.summoners.aggregate([
{ "$match": { "nick":"Luispfj" } },
{ "$unwind": "$matches" },
{
"$lookup": {
"from":"matches",
"localField":"matches.gameId",
"foreignField":"gameId",
"as":"fullMatches"
}
},
{ "$unwind": "$fullMatches" },
{
"$group": {
"_id": null,
"matches": { "$push":"$fullMatches" }
}
}
])
But when I run the unwind function the null entries are gone. How do I retrieve them (with their respective "gameId"s, if possible?
Also, is there a way to retrieve only the matches array, instead of it being a subproperty of the "null-id-object" it creates?
$unwind takes an optional field preserveNullAndEmptyArrays which by default is false. If you set it to true, unwind will output the documents that are null. Read more about $unwind
{
"$unwind": {
path: "$fullMatches",
preserveNullAndEmptyArrays: true
}
},