I want to get years and apply group by on it to get the counts of record. For this my SQL query is -
select substr(lastModified,0,4) , count(*) from EOM group by substr(lastModified,0,4).
I created query for above is -
{
"size":0,
"aggs": {
"profit": {
"scripted_metric": {
"init_script" : "_agg.transactions = []",
"map_script" : "_agg.transactions.add(doc.timeModified.date.getYear())",
"combine_script" : "return _agg.transactions"
}
},
"aggs":{
"terms":{
"field":"profit"
}
}
}
}
RESULT -
{
"aggregations": {
"profit": {
"value": [
[
2014,
2015,
2016,
2015,
2017
],
[
2015,
2015,
2016,
2016,
2017
]
]
},
"aggs": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": []
}
}
}
It give me years but not apply aggrgation on profit and its bucket is blank.
I want sense query for the above sql query.
Why not use a Date Range aggregation instead of a scripted metric ?
https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-daterange-aggregation.html
(the only drawback is that you will have to manually provide the years in ranges section)
Related
I want to convert the following SQL query to MongoDB query:
SELECT count(invoiceNo), year, month, manager
FROM battle
WHERE year=2021 AND month='Dec' OR year=2022 AND month='Jan' AND manager = 'name#test.com'
GROUP BY year,month;
I've tried to do so, but it seems to be incorrect:
const getNoOfOrders = await BattlefieldInfo.aggregate([
{
$match: {
$and: [
{
year: periodDate[0]['year']
},
{ month: periodDate[0]['month'] }
],
$or: [
{
$and: [
{
year: prevYear
},
{ month: prevMonth }
]
}
],
$and: [{ manager: email }]
}
},
{
$group: {
_id: '$month'
}
},
{
$project: {
// noOfOrders: { $count: '$invoiceNo' },
month: 1,
year: 1,
manager: 1
}
}
]);
Because I am getting an empty array. But it should be something like this:
| count(invoiceNo) | manager | year | month |
+------------------+---------------+------+-------+
2 name#test.com 2021 Dec
3 name#test.com 2022 Jan
From my point of view, I think parenthesis (bracket) is important to group the conditions together such as month and year.
SELECT count(invoiceNo), `year`, month, manager
FROM battle
WHERE (`year` = 2021 AND month = 'Dec')
OR (`year` = 2022 AND month = 'Jan')
AND manager = 'abc#email.com'
GROUP BY month, `year`
Sample DBFiddle
Same goes for your MongoDB query. While to search with month and year, you can do without $and as below:
{
year: 2021,
month: "Dec"
}
Instead of:
$and: [
{
year: 2021
},
{
month: "Dec"
}
]
And make sure that $group stage need an accumulator operator:
noOfOrders: {
$count: {}
}
Or
noOfOrders: {
$sum: 1
}
Complete MongoDB query
db.collection.aggregate([
{
$match: {
$or: [
{
year: 2021,
month: "Dec"
},
{
year: 2022,
month: "Jan"
}
],
manager: "abc#email.com"
}
},
{
$group: {
_id: {
month: "$month",
year: "$year"
},
noOfOrders: {
$count: {}
},
manager: {
$first: "$manager"
}
}
},
{
$project: {
_id: 0,
noOfOrders: 1,
month: "$_id.month",
year: "$_id.year",
manager: "$manager"
}
}
])
Sample Mongo Playground
Note:
Would be great for both queries to add manager as one of the group keys. Since you are filtering for the specific (only one) manager's record(s), it's fine. But without filtering for specific manager, your query will result in the wrong output.
I have data stored in a Mongo collection that is structured like this:
{
"numberAtPending" : 3,
"numberAtInProgress" : 5,
"numberAtCancelled" : 1,
"numberAtShipped" : 50,
"timeOfRequest" : ISODate("2022-01-10T12:52:15.813Z"),
"requestingSupplier" : "SUPPLIER_1",
},
{
"numberAtPending" : 5,
"numberAtInProgress" : 3,
"numberAtCancelled" : 4,
"numberAtShipped" : 35,
"timeOfRequest" : ISODate("2022-01-15T09:11:02.992Z"),
"requestingSupplier" : "SUPPLIER_1",
},
{
"numberAtPending" : 12,
"numberAtInProgress" : 3,
"numberAtCancelled" : 1,
"numberAtShipped" : 21,
"timeOfRequest" : ISODate("2022-01-10T14:21:55.221Z"),
"requestingSupplier" : "SUPPLIER_2",
}
I wish to construct a query that would let me sum up each count in each entry and group by requestingSupplier.
For example, I would like to answer the question, for the month of January '22, what was the sum of each entity and get a response similar to:-
"TotalNumberAtPending": 300
"TotalNumberAtInProgress" : 150,
"TotalNumberAtCancelled" : 70,
"TotalNumberAtShipped" : 400
"Supplier" : "SUPPLIER_1",
"TotalNumberAtPending": 230
"TotalNumberAtInProgress" : 110,
"TotalNumberAtCancelled" : 40,
"TotalNumberAtShipped" : 300
"Supplier" : "SUPPLIER_2",
Any help most appreciated!
thanks and regards
You can try this query (also I'm assuming the output you show is an example and not the real values because I don't know from where can you get 300, 150, 400...)
So, try this:
You have two options to match values in the range of two dates. If you want to input the name of the month and the year you can try something like:
Use $expr and $eq with $year and $month. And then you can use as input exaclty the number of the desired month or year.
{
"$match": {
"$expr": {
"$and": [
{
"$eq": [
{
"$month": "$timeOfRequest"
},
1
]
},
{
"$eq": [
{
"$year": "$timeOfRequest"
},
2022
]
}
]
}
}
}
Or you can match by the date range. If you want to get all documents from 01-2022 you can use this $match stage where the range is from the first second of January (equal) to the first second of February (not equal, so i.e. is the last second of January).
{
"$match": {
"timeOfRequest": {
"$gte": ISODate("2022-01-01T00:00:00Z"),
"$lt": ISODate("2022-02-01T00:00:00Z")
}
}
}
So, with the filter done you only need to use $group like this to generate the desired fields values.
{
"$group": {
"_id": "$requestingSupplier",
"TotalNumberAtPending": {
"$sum": "$numberAtPending"
},
"TotalNumberAtInProgress": {
"$sum": "$numberAtInProgress"
},
"TotalNumberAtCancelled": {
"$sum": "$numberAtCancelled"
},
"TotalNumberAtShipped": {
"$sum": "$numberAtShipped"
},
"Supplier": {
"$first": "$requestingSupplier"
}
}
}
Example here and here
I have the following query to be converted to DSL and executed on ES. I could not find a suitable aggregation and filter over results of aggregation available out-of-the-box in ES. As an alternative, I am fetching the 'group by count' for each id from ES and filtering the result as a part of my application logic, which is not efficient. Can you suggest any more suitable solution?
select distinct id from index where colA = "something" group by id having count(*) > 10;
index mapping
id : (string)
colA: (string)
Terms aggregation: to get distinct Ids.
Bucket selector: to return ids with doc count more than 10
{
"query": {
"bool": {
"filter": [
{
"term": {
"colA.keyword": "something" --> where clause
}
}
]
}
},
"aggs": {
"distinct_id": {
"terms": { --> group by
"field": "id.keyword",
"size": 10
},
"aggs": {
"ids_having_count_morethan_10": {
"bucket_selector": { --> having
"buckets_path": {
"count": "_count"
},
"script": "params.count>10"
}
}
}
}
}
}
I have a question about "INTERSECT" query in the MongoDB query.
I want to query like the following SQL query in MongoDB.
(select course_id from section where semester = 'fall' and year = '2009')
intersect
(select course_id from section where semester = 'spring' and year = '2010');
In my MongoDB, Section collection data structure is as follows.
{
"_id" : {
"course_id" : "486",
"sec_id" : "1",
"semester" : "Fall",
"year" : 2009.0
},
"course_id" : "486",
"sec_id" : "1",
"semester" : "Fall",
"year" : 2009.0,
"building" : "Whitman",
"room_number" : "134",
"time_slot_id" : "K"
}
How to query to get the same result as SQL language?
It's not exactly sexy but you can do something like this:
db.collection.aggregate([
{
$match: {
$or: [
{
$and: [
{
year: 2009
},
{
semester: "fall"
}
]
},
{
$and: [
{
year: 2010
},
{
semester: "spring"
}
]
}
]
}
},
{
$group: {
_id: "$course_id",
years_x_semester: {$addToSet: {year: "$year", semester: "$semester"}},
}
},
{
$match: {
"years_x_semester.1": {$exists: true}
}
}
])
I have a linq to sql query and I am pulling data from it fine but unsure how to add a group or count.
Linq Code
public IEnumerable<COMPLAINT> GetAllCount()
{
try
{
return from m in _context.Comp
join p in _context.Checklists on m.ID equals p.C_ID
orderby m.Received_DT descending
select m;
}
catch (Exception ex)
{
_logger.LogError("Could not get complaint", ex);
return null;
}
}
results
[
{
"comP_ID": 909,
"received_DT": "2018-01-22T00:00:00",
"Outcome": "Unsatisfactory",
"RecFinding": "Decline"
},
{
"comP_ID": 909,
"received_DT": "2018-01-22T00:00:00",
"Outcome": "Unsatisfactory",
"RecFinding": "Sustained"
},
{
"comP_ID": 909,
"received_DT": "2018-01-22T00:00:00",
"Outcome": "satisfactory",
"RecFinding": "Decline"
},
{
"comP_ID": 909,
"received_DT": "2018-01-22T00:00:00",
"Outcome": "satisfactory",
"RecFinding": "Sustained"
},
]
Question
I am fairly new to the LINQ To SQL and also writing Fluent API's to return the data but I am unsure how do I get the return of only one variable grouped by results or and count to look like this:
[
{
"Decline": 2,
"Sustained": 2,
},
]
Or
[
{
"Unsatisfactory": 2,
"satisfactory": 2,
},
]
I have tried a few group into g then select new methods, but I am really confused on how to incorporate it into the linq code. Thank you.