How to convert this sql query to mongodb - sql

Considering this query written in sql server how would I efficiently convert it to mongodb:
select * from thetable where column1 = column2 * 2

You can use below aggregation.
You project a new field comp to calculate the expression value followed by $match to keep the docs with eq(0) value and $project with exclusion to drop comp field.
db.collection.aggregate([
{ $addFields: {"comp": {$cmp: ["$column1", {$multiply: [ 2, "$column2" ]} ]}}},
{ $match: {"comp":0}},
{ $project:{"comp":0}}
])

If you want to run your query in mongo Shell,
try below code,
db.thetable .find({}).forEach(function(tt){
var ttcol2 = tt.column2 * 2
var comapreCurrent = db.thetable.findOne({_id : tt._id,column1 : ttcol2});
if(comapreCurrent){
printjson(comapreCurrent);
}
});

I liked the answer posted by #Veeram but it would also be possible to achieve this using $project and $match pipeline operation.
This is just for understanding the flow
Assume we have the below 2 documents stored in a math collection
Mongo Documents
{
"_id" : ObjectId("58a055b52f67a312c3993553"),
"num1" : 2,
"num2" : 4
}
{
"_id" : ObjectId("58a055be2f67a312c3993555"),
"num1" : 2,
"num2" : 6
}
Now we need to find if num1 = 2 times of num2 (In our case the document with _id ObjectId("58a055b52f67a312c3993553") will be matching this condition)
Query:
db.math.aggregate([
{
"$project": {
"num2": {
"$multiply": ["$num2",1]
},
"total": {
"$multiply": ["$num1",2]
},
"doc": "$$ROOT"
}
},
{
"$project": {
"areEqual": {"$eq": ["$num2","$total"]
},
doc: 1
}
},
{
"$match": {
"areEqual": true
}
},
{
"$project": {
"_id": 1,
"num1": "$doc.num1",
"num2": "$doc.num2"
}
}
])
Pipeline operation steps:-
The 1st pipeline operation $project calculates the total
The 2nd pipeline operation $project is used to check if the total
matches the num2. This is needed as we cannot use the comparision
operation of num2 with total in the $match pipeline operation
The 3rd pipeline operation matches if areEqual is true
The 4th pipeline operation $project is just used for projecting the fields
Note:-
In the 1st pipeline operation I have multiplied num2 with 1 as num1 and num2 are stored as integers and $multiply returns double value. So incase I do not use $mutiply for num2, then it tries to match 4 equals 4.0 which will not match the document.

Certainly no need for multiple pipeline stages when a single $redact pipeline will suffice as it beautifully incorporates the functionality of $project and $match pipeline steps. Consider running the following pipeline for an efficient query:
db.collection.aggregate([
{
"$redact": {
"$cond": [
{
"$eq": [
"$column1",
{ "$multiply": ["$column2", 2] }
]
},
"$$KEEP",
"$$PRUNE"
]
}
}
])
In the above, $redact will return all documents that match the condition using $$KEEP and discards those that don't match using the $$PRUNE system variable.

Related

Mongodb Query to filter documents based on the length of a field value

I am writing a AWS lambda code in python. My database is AWS DocumentDB. And I use pymongo.
This code snippet works fine
query = {"media_id": {"$exists": True}} collection.find(query)
But it returns a lot of records, so I want to fetch the records where the length of media_id field is less than 3.
For that I tried this query query = { "media_id": {"$exists": True}, "$expr: {"$lt": [{"$strLenCP": "$media_id"}, 3]},}, but I get
Feature not supported Error
because $expr is not supported in DocumentDB.
I am looking for the query which works in DocumentDB.
The solution might seems a bit tedious, but all the operations inside should be supported according to the official doc
Use an aggregation to project an auxiliary field to store the length of media_id then match on your criteria.
db.collection.aggregate([
{
$addFields: {
length: {
"$strLenCP": "$media_id"
}
}
},
{
$match: {
media_id: {
$exists: true
},
length: {
$gte: 3
}
}
},
{
"$project": {
length: false
}
}
])

How to convert sql query with exist into mongodb query

I have two documents on mongodb, these are percentages and items. I'm good at SQL, I can write PLSql query as follows but i can not convert to mongodb query. Because my mongodb level of knowledge is at the beginning. Actually I know I have to use $gt for the and condition. But I don't know how I can say not exists or union keyword for mongodb. How can I write mongodb query? which keywords should i search for?
select p.*, "to_top" as list
from percentages p
where p.percentage > 5
and p.updatetime > sysdate - 1/24
and not exists (select 1
from items i
where i.id = p.p_id
and i.seller = p.seller)
order by p.percentage desc
union
select p2.*, "to_bottom" as list
from percentages p2
where p2.percentage > 5
and p2.updatetime > sysdate - 1/24
and exists (select 1
from items i2
where i2.id = p2.p_id
and i2.seller = p2.seller)
order by p2.percentage desc
There is no UNION for MongoDB. Luckely, each query is performed on the same collection and have very close condition, so we can implement "Mongo way" query.
Explanation
Normally, alsmost all complex SQL queries are done with the MongoDB aggregation framework.
We filter document by percentage / updatetime. Explanation why we need to use $expr
SQL JOIN / Subquery is done with the $lookup operator.
SQL SYSDATE in MongoDB way can be NOW or CLUSTER_TIME variable.
db.percentages.aggregate([
{
$match: {
percentage: { $gt: 5 },
$expr: {
$gt: [
"$updatetime",
{
$subtract: [
ISODate("2020-06-14T13:00:00Z"), //Change to $$NOW or $$CLUSTER_TIME
3600000
]
}
]
}
}
},
{
$lookup: {
from: "items",
let: {
p_id: "$p_id",
seller: "$seller"
},
pipeline: [
{
$match: {
$expr: {
$and: [
{
$eq: [ "$$p_id", "$id"]
},
{
$eq: [ "$$seller", "$seller"]
}
]
}
}
},
{
$limit: 1
}
],
as: "items"
}
},
{
$addFields: {
list: {
$cond: [
{
$eq: [{$size: "$items"}, 0]
},
"$to_top",
"$to_bottom"
]
},
items: "$$REMOVE"
}
},
{
$sort: { percentage: -1 }
}
])
MongoPlayground
Note: The MongoDB aggregation has the $facet operator that allows to perform different queries on the same collection.
SCHEMA:
db.percentages.aggregate([
{$facet:{
q1:[...],
q2:[...],
}},
//We apply "UNION" the result documents for each pipeline into single array
{$project:{
data:{$concatArrays:["$q1","$q2"]}
}},
//Flatten array into single object
{$unwind:"$data"}
//Replace top-level document
{$replaceWith:"$data"}
])
MongoPlayground
why you don't import your mangoDB data into oracle and use sql(that is more easy and powerful than mango.)

How to implement the follow RMDB query in MongoDB

My team started to use MongoDB now and wanna migrate some sql to Mongo.
For example, I have an order table and has the fields price and quanty.I want to query the price*quanty greater than 100. sql is like below
select * from Order where price * quanty > 100;
How to use "price * quanty" this kind query in Mongo?
Thanks.
You can do this by using the $expr operator to use aggregation expressions within your query:
db.orders.find({
$expr: {
$gt: [
{ $multiply: ["$price", "$quantity"] },
100
]
}
})
As JohnnyHK points out you can use $expr, but as an alternative you can also use aggregation to first create a new field that is the product of two other fields:
db.orders.aggregate([
{ $set: { product: { $multiply: [ "$price", "$quantity" ] } } }
])
Note: $set is new in 4.2 and just an alias for $addFields
Then add a $match stage that only matches documents with the new product field meeting your condition:
db.orders.aggregate([
{ $set: { product: { $multiply: [ "$price", "$quantity" ] } } },
{ $match: { product: { $gt: 100 } } }
])

Nest Elastic - Building Dynamic Nested Query

I have to query a nested object using Nest, however the query is built in dynamic way. Below is code that demonstrate using query on nested "books" in a static way
QueryContainer qry;
qry = new QueryStringQuery()
{
DefaultField = "name",
DefaultOperator = Operator.And,
Query = "salman"
};
QueryContainer qry1 = null;
qry1 = new RangeQuery() // used to search for range ( from , to)
{
Field = "modified",
GreaterThanOrEqualTo = Convert.ToDateTime("21/12/2015").ToString("dd/MM/yyyy"),
};
QueryContainer all = qry && qry1;
var results = elastic.Search<Document>(s => s
.Query(q => q
.Bool(qb => qb
.Must(all)))
.Filter(f =>
f.Nested(n => n
.Path("books")
.Filter(f3 => f3.And(
f1 => f1.Term("book.isbn", "122"),
f2 => f2.Term("book.author", "X"))
)
)
)
);
The problem is that i need to combine multiple queries (using And,OR operators) for "books" in dynamic fashion. For example, get the books that satisfy these set of conditions:
Condition 1: Books that has Author "X" and isbn "1"
Condition 2: Books that has Author "X" and isbn "2"
Condition 3: Books that has Author "Z" and isbn "3"
Other Condtions: .....
Now, the filter in the nested Query should retrieve books if:
Condition 1 AND Condition 2 Or Condition 3
Suppose that i have class name FilterOptions that contains the following attributes:
FieldName
Value
Operator (which will combine the next filter)
I am going to loop on the given FilterOptions array to build the query.
Question:
What should i use to build the nested query? Is it a FilterDesciptor and how to combine them add the nested query to the Search Method?
Please, recommend any valuable link or example?
I agree with paweloque, it seems your first two conditions are contradictory and wouldn't work if AND-ed together. Ignoring that, here's my solution. I've implemented this in such a way that allows for more than the three specific conditions you have. I too feel it would fit better in a bool statement.
QueryContainer andQuery = null;
QueryContainer orQuery = null;
foreach(var authorFilter in FilterOptions.Where(f=>f.Operator==Operator.And))
{
andQuery &= new TermQuery
{
Field = authorFilter.FieldName,
Value = authorFilter.Value
};
}
foreach(var authorFilter in FilterOptions.Where(f=>f.Operator==Operator.Or))
{
orQuery |= new TermQuery
{
Field = authorFilter.FieldName,
Value = authorFilter.Value
};
}
After that, in the .Nested call I would put:
.Path("books")
.Query(q=>q
.Bool(bq=>bq
.Must(m=>m.MatchAll() && andQuery)
.Should(orQuery)
))
In the specific case of the Condition 1 and Condition 2 you'd probably not get any results because these are exclusive conditions. But I assume now, that you want to get results which match either of those conditions. You've chosen nested which is definitely the way to go. With the nested type you can combine parameters for a single book.
Combining nested queries
For your use case I'd use bool query type with must or should clauses.
A query to get books for either Condition 1 or Condition 2 would be:
POST /books/_search
{
"query": {
"bool": {
"should": [
{
"nested": {
"path": "books",
"query": {
"bool": {
"must": [
{
"match": {
"books.isbn": "2"
}
},
{
"match": {
"books.author": "X"
}
}
]
}
}
}
},
{
"nested": {
"path": "books",
"query": {
"bool": {
"must": [
{
"match": {
"books.isbn": "1"
}
},
{
"match": {
"books.author": "X"
}
}
]
}
}
}
}
]
}
}
}
Can you explain, why are your books nested? Without nesting them in a top structure but indexing directly as a top level object in an index/type you could simplify your queries.
Not-Analyzed
There is another caveat that you have to remind: If you want to have an exact match on the author and the ISBN you have to make sure that the ISBN and author fields are set to not_analyzed. Otherwise they get analyzed and splitted into parts and your match would'n work very well.
E.g. if you have a ISBN Number with dashes, then it would get split into parts:
978-3-16-148410-0
would become indexed as:
978
3
16
148410
0
And a search with exactly the same ISBN number would give you all the books which have one of the sub-numbers in their ISBN number. If you want to prevent this, use the not_analyzed index-type and Multi-fields:
"isbn": {
"type": "string",
"fields": {
"raw": {
"type": "string",
"index": "not_analyzed"
}
}
}
Then to address the not_analyzed isbn field you'd have to call it:
books.isbn.raw
Hope this helps.

How to query mongodb with “like” for number data type? [duplicate]

I want to regex search an integer value in MongoDB. Is this possible?
I'm building a CRUD type interface that allows * for wildcards on the various fields. I'm trying to keep the UI consistent for a few fields that are integers.
Consider:
> db.seDemo.insert({ "example" : 1234 });
> db.seDemo.find({ "example" : 1234 });
{ "_id" : ObjectId("4bfc2bfea2004adae015220a"), "example" : 1234 }
> db.seDemo.find({ "example" : /^123.*/ });
>
As you can see, I insert an object and I'm able to find it by the value. If I try a simple regex, I can't actually find the object.
Thanks!
If you are wanting to do a pattern match on numbers, the way to do it in mongo is use the $where expression and pass in a pattern match.
> db.test.find({ $where: "/^123.*/.test(this.example)" })
{ "_id" : ObjectId("4bfc3187fec861325f34b132"), "example" : 1234 }
I am not a big fan of using the $where query operator because of the way it evaluates the query expression, it doesn't use indexes and the security risk if the query uses user input data.
Starting from MongoDB 4.2 you can use the $regexMatch|$regexFind|$regexFindAll available in MongoDB 4.1.9+ and the $expr to do this.
let regex = /123/;
$regexMatch and $regexFind
db.col.find({
"$expr": {
"$regexMatch": {
"input": {"$toString": "$name"},
"regex": /123/
}
}
})
$regexFinAll
db.col.find({
"$expr": {
"$gt": [
{
"$size": {
"$regexFindAll": {
"input": {"$toString": "$name"},
"regex": "123"
}
}
},
0
]
}
})
From MongoDB 4.0 you can use the $toString operator which is a wrapper around the $convert operator to stringify integers.
db.seDemo.aggregate([
{ "$redact": {
"$cond": [
{ "$gt": [
{ "$indexOfCP": [
{ "$toString": "$example" },
"123"
] },
-1
] },
"$$KEEP",
"$$PRUNE"
]
}}
])
If what you want is retrieve all the document which contain a particular substring, starting from release 3.4, you can use the $redact operator which allows a $conditional logic processing.$indexOfCP.
db.seDemo.aggregate([
{ "$redact": {
"$cond": [
{ "$gt": [
{ "$indexOfCP": [
{ "$toLower": "$example" },
"123"
] },
-1
] },
"$$KEEP",
"$$PRUNE"
]
}}
])
which produces:
{
"_id" : ObjectId("579c668c1c52188b56a235b7"),
"example" : 1234
}
{
"_id" : ObjectId("579c66971c52188b56a235b9"),
"example" : 12334
}
Prior to MongoDB 3.4, you need to $project your document and add another computed field which is the string value of your number.
The $toLower and his sibling $toUpper operators respectively convert a string to lowercase and uppercase but they have a little unknown feature which is that they can be used to convert an integer to string.
The $match operator returns all those documents that match your pattern using the $regex operator.
db.seDemo.aggregate(
[
{ "$project": {
"stringifyExample": { "$toLower": "$example" },
"example": 1
}},
{ "$match": { "stringifyExample": /^123.*/ } }
]
)
which yields:
{
"_id" : ObjectId("579c668c1c52188b56a235b7"),
"example" : 1234,
"stringifyExample" : "1234"
}
{
"_id" : ObjectId("579c66971c52188b56a235b9"),
"example" : 12334,
"stringifyExample" : "12334"
}
Now, if what you want is retrieve all the document which contain a particular substring, the easier and better way to do this is in the upcoming release of MongoDB (as of this writing) using the $redact operator which allows a $conditional logic processing.$indexOfCP.
db.seDemo.aggregate([
{ "$redact": {
"$cond": [
{ "$gt": [
{ "$indexOfCP": [
{ "$toLower": "$example" },
"123"
] },
-1
] },
"$$KEEP",
"$$PRUNE"
]
}}
])