provided that I have following statement in sequelize
return models.VehicleModel
.findAll({
include: [
{
attributes: ['id'],
model: models.Unit
},
{
model: models.myPattern,
where: {
VehicleModelId: { $in: data.VehicleModelIds }
},
include: [
{
model: models.Calendar,
include: [
{
model: models.Restrictions,
include: [
{
model: models.RestrictionType
}],
where: {
//THIS IS THE LINE I NEED TO CHANGE SOMEHOW
UnitId: { $in: [models.Unit.id] }
},
required: false
}
]
}
]
}
],
where: {
id: { $in: data.VehicleModelIds }
}
})
This gives me NULL in the query like so:
[myPattern.Calendar.Restrictions].UnitId IN (NULL) AND ....
I want to be able to get the following instead:
[myPattern.Calendar.Restrictions].UnitId IN (Units.id) AND ....
This is because they are not mandatory to be there but should of course be linked to the proper unit corresponding to the vehicle model.
The myPattern is linked to the Calendar who is in turn linked to the Restrictions and so to the Units.
This all using id column.
Any help would be appreciated!
Found it
So for people looking for the "how to":
//...
where: {
UnitId: { $in: [models.sequelize.literal('Units.id')] },
}
//...and so on
This will output in the query: [myPattern.Calendar.Restrictions].UnitId IN (Units.id)
Related
I have a collection "Owners" and I want to return a list of "Owner" matching a filter (any filter), plus the count of "Pet" from the "Pets" collection for that owner, except I don't want the dead pets. (made up example)
I need the returned documents to look exactly like an "Owner" document with the addition of the "petCount" field because I'm using Java Pojos with the Mongo Java driver.
I'm using AWS DocumentDB that does not support $lookup with filters yet. If it did I would use this and I'd be done:
db.Owners.aggregate( [
{ $match: {_id: UUID("b13e733d-2686-4266-a686-d3dae6501887")} },
{ $lookup: { from: 'Pets', as: 'pets', 'let': { ownerId: '$_id' }, pipeline: [ { $match: { $expr: { $ne: ['$state', 'DEAD'] } } } ] } },
{ $addFields: { petCount: { $size: '$pets' } } },
{ $project: { pets: 0 } }
]).pretty()
But since it doesn't this is what I got so far:
db.Owners.aggregate( [
{ $match: {_id: { $in: [ UUID("cbb921f6-50f8-4b0c-833f-934998e5fbff") ] } } },
{ $lookup: { from: 'Pets', localField: '_id', foreignField: 'ownerId', as: 'pets' } },
{ $unwind: { path: '$pets', preserveNullAndEmptyArrays: true } },
{ $match: { 'pets.state': { $ne: 'DEAD' } } },
{ "$group": {
"_id": "$_id",
"doc": { "$first": "$$ROOT" },
"pets": { "$push": "$pets" }
}
},
{ $addFields: { "doc.petCount": { $size: '$pets' } } },
{ $replaceRoot: { "newRoot": "$doc" } },
{ $project: { pets: 0 } }
]).pretty()
This works perfectly, except if an Owner only has "DEAD" pets, then the owner doesn't get returned because all the "document copies" got filtered out by the $match. I'd need the parent document to be returned with petCount = 0 when ALL of them are "DEAD". I cannot figure out how to do this.
Any ideas?
These are the supported operations for DocDB 4.0 https://docs.amazonaws.cn/en_us/documentdb/latest/developerguide/mongo-apis.html
EDIT: update to use $filter as $reduce not supported by aws document DB
You can use $filter to keep only not DEAD pets in the lookup array, then count the size of the remaining array.
Here is the Mongo playground for your reference.
$reduce version
You can use $reduce in your aggregation pipeline to to a conditional sum for the state.
Here is Mongo playground for your reference.
As of January 2022, Amazon DocumentDB added support for $reduce, the solution posted above should work for you.
Reference.
In my application, I am using sequelize ORM. There are several entities: A Tool can have Tags and Categories.
Now I want to search for all Tools, that have a specific Tag, but I want to include all relating Tags of that tool (not just the specific one). If I now place a where statement into the include, only specified Tags are included into the result set (see [2]). I tried to limit the Tags in the outer where statement (see [1]), but this does not help either.
Example
Tool A has Tags t1, t2 and t3. Now I want to search all Tools that have the Tag t3, but the result set shall contain all three tags.
Expected result:
Tool A
\
- Tag t1
- Tag t2
- Tag t3
db.Tool.scope('published').findAll({
where: { '$tool.tag.name$': filter.tag }, // [1] Does not work
include: [
{
model: db.User,
attributes: ['id', 'username']
},
{
model: db.Tag,
attributes: ['name'],
through: { attributes: [] },
// [2] Would limit the result specified tag
// where: {
// name: {
// [Op.and]: filter.tag
// }
// }
},
{
model: db.Category,
attributes: ['id', 'name', 'views'],
through: { attributes: ['relevance'] },
where: {
id: {
[Op.and]: filter.category
}
}
}
],
where: {
title: {
[Op.like]: `%${filter.term}%`,
}
},
attributes: ['id', 'title', 'description', 'slug', 'docLink', 'vendor', 'vendorLink', 'views', 'status', 'createdAt'],
order: [['title', 'ASC'], [db.Tag, 'name', 'ASC']]
})
I know I could perform this by performing a select via the Tag in the first place (db.Tag.findAll() instead of db.Tool.findAll(); I've already done this elsewhere in my project), but at the same time I also want to be able to filter by another entity (Category) the same way. So the Tool.findAll() should be the starting point.
Any help appreciated!
First off, you have two where clauses in your top-level query:
where: { '$tool.tag.name$': filter.tag }, // [1] Does not work
// ...
where: {
title: {
[Op.like]: `%${filter.term}%`,
}
},
I think your best approach is going to be with a literal subquery in the WHERE clause. Basically we want to find the ids of all of the tools that have the right tag and that contain the filter.term.
The subquery part for the WHERE looks something like...
SELECT ToolId FROM ToolTags WHERE TagId='t2';
Inspired by the subquery solution from this post Sequelize - subquery in where clause
// assuming your join table is named 'ToolTags' in the database--we need the real table name not the model name
const tempSQL = sequelize.dialect.QueryGenerator.selectQuery('ToolTags',{
attributes: ['ToolId'],
where: {
TagId: filter.tag
}})
.slice(0,-1); // to remove the ';' from the end of the SQL
db.Tool.scope('published').findAll({
where: {
title: {
[Op.like]: `%${filter.term}%`,
},
id: {
[Op.In]: sequelize.literal(`(${tempSQL})`)
}
},
include: [
{
model: db.User,
attributes: ['id', 'username']
},
{
model: db.Tag,
attributes: ['name'],
through: { attributes: [] },
},
// {
// model: db.Category,
// attributes: ['id', 'name', 'views'],
// through: { attributes: ['relevance'] },
// where: {
// id: {
// [Op.and]: filter.category
// }
// }
// }
],
attributes: ['id', 'title', 'description', 'slug', 'docLink', 'vendor', 'vendorLink', 'views', 'status', 'createdAt'],
order: [['title', 'ASC'], [db.Tag, 'name', 'ASC']]
})
I commented out your category join for now. I think you should try to isolate the solution for the tags before adding more onto the query.
What is the optimize Query for this situation
So the Situation is a User is following many XY user and these XY have got events, So what will the best and optimize query to get all the events from his followers XY in sorted form (sort by Date). I have got create Date in my schema
This is my User Schema
var userSchema = new Schema({
followers:[{
type: Schema.Types.ObjectId,
ref: 'XY'
}]
});
var User = mongoose.model('User',userSchema);
module.exports = User;
Here is My XY schema
var XY= new Schema({
events:[{
type: Schema.Types.ObjectId,
ref: 'Event'
}],
});
var XY= mongoose.model('XY',XY);
module.exports = XY;
Try this.
User.aggregate([
{ $match: { _id: mongoose.Types.ObjectId(<userId here>) } },
{ $unwind: { path: "$followers" } },
{ $lookup: { from: 'XY', localField: 'followers', foreignField: '_id', as: 'followers' } },
{ $unwind: { path: "$followers" } },
{ $unwind: { path: "$followers.events" } },
{ $lookup: { from: 'Event', localField: 'followers.events', foreignField: '_id', as: 'followers.events' } },
{ $unwind: { path: "$followers.events" } },
{ $sort: { "followers.events.createdDate": **-1** } }, // -1 -> desc, 1 -> asc
{
"$project":
{
"_id": "$followers.events._id",
"createdDate": "$followers.events.createdDate",
// populate other event details here accordingly
}
}
], function (err, events, next) {
});
$lookup lets you populate a sub-document from a different schema.
After populating, the resultant documents will be an array, so $unwind is used before working on them.
note that $unwind is also used before doing a $lookup here as the field we are trying to populate is itself an array.
I have this query that provides me the join I want to:
db.summoners.aggregate([
{ "$match": { "nick":"Luispfj" } },
{ "$unwind": "$matches" },
{
"$lookup": {
"from":"matches",
"localField":"matches.gameId",
"foreignField":"gameId",
"as":"fullMatches"
}
},
{ "$unwind": "$fullMatches" },
{
"$group": {
"_id": null,
"matches": { "$push":"$fullMatches" }
}
}
])
But when I run the unwind function the null entries are gone. How do I retrieve them (with their respective "gameId"s, if possible?
Also, is there a way to retrieve only the matches array, instead of it being a subproperty of the "null-id-object" it creates?
$unwind takes an optional field preserveNullAndEmptyArrays which by default is false. If you set it to true, unwind will output the documents that are null. Read more about $unwind
{
"$unwind": {
path: "$fullMatches",
preserveNullAndEmptyArrays: true
}
},
I have two tables on a PostgreSQL database, contracts and payments. One contract has multiple payments done.
I'm having the two following models:
module.exports = function(sequelize, DataTypes) {
var contracts = sequelize.define('contracts', {
id: {
type: DataTypes.INTEGER,
autoIncrement: true
}
}, {
createdAt: false,
updatedAt: false,
classMethods: {
associate: function(models) {
contracts.hasMany(models.payments, {
foreignKey: 'contract_id'
});
}
}
});
return contracts;
};
module.exports = function(sequelize, DataTypes) {
var payments = sequelize.define('payments', {
id: {
type: DataTypes.INTEGER,
autoIncrement: true
},
contract_id: {
type: DataTypes.INTEGER,
},
payment_amount: DataTypes.INTEGER,
}, {
classMethods: {
associate: function(models) {
payments.belongsTo(models.contracts, {
foreignKey: 'contract_id'
});
}
}
});
return payments;
};
I would like to sum all the payments made for every contract, and used this function:
models.contracts.findAll({
attributes: [
'id'
],
include: [
{
model: models.payments,
attributes: [[models.sequelize.fn('sum', models.sequelize.col('payments.payment_amount')), 'total_cost']]
}
],
group: ['contracts.id']
})
But it generates the following query:
SELECT "contracts"."id", "payments"."id" AS "payments.id", sum("payments"."payment_amount") AS "payments.total_cost"
FROM "contracts" AS "contracts"
LEFT OUTER JOIN "payments" AS "payments" ON "contracts"."id" = "payments"."contract_id" GROUP BY "contracts"."id";
I do not ask to select payments.id, because I would have to include it in my aggregation or group by functions, as said in the error I have:
Possibly unhandled SequelizeDatabaseError: error: column "payments.id"
must appear in the GROUP BY clause or be used in an aggregate function
Am I missing something here ? I'm following this answer but even there I don't understand how the SQL request can be valid.
This issue has been fixed on Sequelize 3.0.1, the primary key of the included models must be excluded with
attributes: []
and the aggregation must be done on the main model (infos in this github issue).
Thus for my use case, the code is the following
models.contracts.findAll({
attributes: ['id', [models.sequelize.fn('sum', models.sequelize.col('payments.payment_amount')), 'total_cost']],
include: [
{
model: models.payments,
attributes: []
}
],
group: ['contracts.id']
})
Try
group: ['contracts.id', 'payments.id']
Can you write your function as
models.contracts.findAll({
attributes: [
'models.contracts.id'
],
include: [
{
model: models.payments,
attributes: [[models.sequelize.fn('sum', models.sequelize.col('payments.payment_amount')), 'total_cost']]
}
],
group: ['contracts.id']
})
Is the issue that you might want to be selecting from payments and joining contracts rather than the other way around?