Its been observed that lte and gte both take lower bound values. how do I make it to take upper bound for lte and lower bound for gte??
This is how my query looks
{
"from": 0,
"size": 40,
"query": {
"filtered": {
"filter": {
"bool": {
"should": [
{
"range": {
"CreatedOn": {
"lte": "201505",
"gte": "201404",
"format": "yyyyMM"
}
}
}
]
}
}
}
}
}
The above query does not return me valid documents such as "2015-05-06T12:55:34.44", "2015-05-26T14:42:24.963" etc. It only returns "2015-05-01T11:42:24.963" from lte 201505
201404 means April 1st 2014
201505 means May 1st 2015
This is the reason why you get that result.
Elasticsearch "translates" in the background your query into a range one like 1396310400000 TO 1430524799999, meaning 01 Apr 2014 00:00:00 GMT TO 01 May 2015 23:59:59 GMT.
If you want everything between entire month of April 2014 and entire month of May 2015, then use this:
"range": {
"createdOn": {
"lte": "201506||-1d/d",
"gte": "201404",
"format": "yyyyMM"
}
}
Related
Mongoose saves all dates in UTC format that's OK.
Each user has a specific timezone. It stores the in user's document.
I would like to send any documents to the client with converted dates in the user's timezone.
I know how to format one value, but I'm looking for a solution to avoid transformation one-by-one.
The workaround solution will be a pass-through transform function on the client for each value or recursive modification response.
Any thoughts?
You can do this in aggregation.
Lets say you have this data:
[
{
"_id": {
"$oid": "5f18b5c87f9f9c0fd8322b60"
},
"createdAt": {
"$date": "2020-07-22T21:55:20.575Z"
},
},
{
"_id": {
"$oid": "5f19efac5cfa75483865eaa2"
},
"createdAt": {
"$date": "2020-07-23T20:14:36.108Z"
},
}
]
you can do this:
const timezone = "America/Chicago"
Model.aggregate([
{
$set: {
localTime: {
$dateToString: {
date: "$createdAt",
timezone
}
}
},
}
]);
The result of the aggregation will be:
[
{
"_id": ObjectId("5f18b5c87f9f9c0fd8322b60"),
"createdAt": ISODate("2020-07-22T21:55:20.575Z"),
"localTime": "2020-07-22T16:55:20.575Z"
},
{
"_id": ObjectId("5f19efac5cfa75483865eaa2"),
"createdAt": ISODate("2020-07-23T20:14:36.108Z"),
"localTime": "2020-07-23T15:14:36.108Z"
}
]
Demo example: https://mongoplayground.net/p/7IOGMrC2sf5
I want to group by start_time with hour between 20190701 and 20190710,but not each day each hour is a bucket, I want the data Divided into 24 buckets,for example: 20190701,20190801,20190901... fall into the 01 bucket,20190702,20190802,20190902... fall into the 02 bucket and so on.
this is each day each hour is a bucket,the result is not what I want,how to solve this problem?
start_time field type as follows:
"start_time":
{
"type": "date",
"format": "yyyy-MM-dd HH:mm:ss||epoch_second"
}
My code as followed:
GET qd_analysis/kw/_search
{
"size": 0,
"query": {
"bool": {
"must": [
{
"term": {
"uin": {
"value": "111"
}
}
},
{
"range": {
"imp_date": {
"gte": "20190701",
"lte": "20190710"
}
}
}
]
}
},
"aggs": {
"result": {
"date_histogram": {
"field": "start_time",
"time_zone": "+08:00",
"interval": "hour",
"format": "HH",
"order": {
"_count": "desc"
}
}
}
}
}
I want to group by start_time with hour between 20190701 and 20190710,but not each day each hour is a bucket, I want the data Divided into 24 buckets,for example: 20190701,20190801,20190901... fall into the 01 bucket,20190702,20190802,20190902... fall into the 02 bucket and so on.
You will need to use terms aggregation, with script to extract the hour-of-day:
{
"aggs": {
"hour_of_day": {
"terms": {
"script": "doc['#timestamp'].date.hourOfDay"
}
}
}
}
So I have 3 types of logs in my Elasticsearch index-
CA, CT, And Acc
I am trying to query Elasticsearch to get a count of each for the 24 hours before the call but I'm not having much luck combining them.
Calling
10.10.23.45:9200/filebeat-*/_count
With
{
"query":{
"term": {"type":"ct"}
}
}
Gets me the count, but trying to add the time-range has proved to be fruitless. When I try to add a range to the same query -- it doesn't work
I tried using:
{
"query":{
"term": {"type":"ct"},
"range":{
"date":{
"gte": "now-1d/d",
"lt" : "now"
}
}
}
}
But was returned
{
"error": {
"root_cause": [
{
"type": "parsing_exception",
"reason": "[term] malformed query, expected [END_OBJECT] but found [FIELD_NAME]",
"line": 5,
"col": 3
}
],
"type": "parsing_exception",
"reason": "[term] malformed query, expected [END_OBJECT] but found [FIELD_NAME]",
"line": 5,
"col": 3
},
"status": 400
}
You need to use Bool Query to combine two types of queries into one. Try this instead.
POST _search
{
"query": {
"bool" : {
"must" : {
"term": {"type":"ct"}
},
"must" : {
"range":{
"date":{
"gte": "now-1d/d",
"lt" : "now"
}
}
}
}
}
}
The following worked for me (note -- this is a post sent to elasticsearch:9200/index/_search )
{"query":{"bool":{"must":[{"query_string":{"analyze_wildcard":true,"query":"type:\"acc\""}},{"range":{"#timestamp":{"gte":"now-1h","lte":"now","format":"epoch_millis"}}}]}}}
I want to get years and apply group by on it to get the counts of record. For this my SQL query is -
select substr(lastModified,0,4) , count(*) from EOM group by substr(lastModified,0,4).
I created query for above is -
{
"size":0,
"aggs": {
"profit": {
"scripted_metric": {
"init_script" : "_agg.transactions = []",
"map_script" : "_agg.transactions.add(doc.timeModified.date.getYear())",
"combine_script" : "return _agg.transactions"
}
},
"aggs":{
"terms":{
"field":"profit"
}
}
}
}
RESULT -
{
"aggregations": {
"profit": {
"value": [
[
2014,
2015,
2016,
2015,
2017
],
[
2015,
2015,
2016,
2016,
2017
]
]
},
"aggs": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": []
}
}
}
It give me years but not apply aggrgation on profit and its bucket is blank.
I want sense query for the above sql query.
Why not use a Date Range aggregation instead of a scripted metric ?
https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-daterange-aggregation.html
(the only drawback is that you will have to manually provide the years in ranges section)
Suppose there are several documents per person that contain values:
{
"name": "John",
"value": 1,
"timestamp": 2014-06-15
}
{
"name": "John",
"value": 2,
"timestamp": 2014-06-16
}
{
"name": "Sam",
"value": 2,
"timestamp": 2014-06-15
}
{
"name": "Sam",
"value": 3,
"timestamp": 2014-06-16
}
How do I get a list of the most recent documents for each person?
How do I get an average of the values for the list of the most recent documents for each person? Given the sample data, this would be 2.5, not 2.
Is there some combination of buckets and metrics that could achieve this result? Will I need to implement a custom aggregator as part of a plugin, or must this sort of computation be performed in memory?
If you only need to find the most recent persons try something like this:
"aggs": {
"personName": {
"terms": {
"field": "name",
"size": 5,
"order": {"timeCreated": "desc"}
},
"aggs": {
"timeCreated": {
"max": {"field": "timestamp"}
}
}
}
}
The second operation is just an aggregation, and to get the average of the value field you could try something like:
curl -XPOST "http://DOMAIN:9200/your/data/_search" -d'
{
"size": 0,
"aggregations": {
"the_name": {
"terms": {
"field": "name",
"order": {
"value_avg": "desc"
}
},
"aggregations": {
"value_avg": {
"avg": {
"field": "value"
}
}
}
}
}
}'
To achieve a solution for your first issue I would recommend you to order the response by date, and then in your project ignore a term when you have another with the same name (meaning filter the data after the response of ES)