I have to convert the following query in elasticsearch :
select * from index where observable not in (select observable from index where tags = 'whitelist')
I read that I should use a Filter in a Not Filter but I don't understand how to do.
Can anyone help me?
Thanks
EDIT:
I have to get all except those that have 'whitelist' tag but I need to check also that nothing of the blacklist element is contained into the whitelist.
Your SQL query can be simplified to this:
select * from index where tags not in ('whitelist')
As a result the "corresponding" ES query would be
curl -XPOST localhost:9200/index/_search -d '{
"query": {
"filtered": {
"filter": {
"bool": {
"must_not": {
"terms": {
"tags": [
"whitelist"
]
}
}
}
}
}
}
}'
or another using the not filter instead of bool/must_not:
curl -XPOST localhost:9200/index/_search -d '{
"query": {
"filtered": {
"filter": {
"not": {
"terms": {
"tags": [
"whitelist"
]
}
}
}
}
}
}'
Related
I was working on a query that I found. It's a little bit more complex than I thought. This is part of each data document which is distinguished by an event name.
"eventTime" : "2021-07-11T08:29:00-0800",
"userId" : "P9QuPERPURPC3swJpyBb4",
"eventName" : "mko", // mko and mkp are two possible values
"eventData" : {}
The target is: userIds who have eventName('mko') AND does not have eventName('mkp')
I could not precisely understand what is the best way to handle 'not exist' in Elasticsearch queries. I'd appreciate any help.
I think the below queries might help you.
Get a user with the name mko:
{
"query": {
"bool": {
"must": [
{
"term": {
"eventName": "mko"
}
}
]
}
}
}
Get a user with a name other than mko:
{
"query": {
"bool": {
"must_not": [
{
"term": {
"eventName": "mko"
}
}
]
}
}
}
Get a user with the name mkp:
{
"query": {
"bool": {
"must": [
{
"term": {
"eventName": "mkp"
}
}
]
}
}
}
Get a user with a name other than mkp:
{
"query": {
"bool": {
"must_not": [
{
"term": {
"eventName": "mkp"
}
}
]
}
}
}
To get the selected fields only you can use the _source field in query: https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-source-field.html.
And to check whether the field eventName is exists in a document. You can use the exists query within the must queries: https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-exists-query.html
'''select count(*) as count from activity where project_id in (61,129) and (entry_device_id in (1068,1069) or exit_device_id in (1068,1069) );'''
I tried with should in elastic query and match but not getting the desired results.
Got some idea from elasticsearch bool query combine must with OR
And tried but not getting the correct results.
Need help in this
Depending on your index's mapping, a combination of terms queries should get you started:
GET your_activity_index/_count
{
"query": {
"bool": {
"must": [
{
"terms": {
"project_id": [ 61, 129 ]
}
},
{
"bool": {
"should": [
{
"terms": {
"entry_device_id": [ 1068, 1069 ]
}
},
{
"terms": {
"exit_device_id": [ 1068, 1069 ]
}
}
]
}
}
]
}
}
}
I read couple of similar problems on SO and suggest solution not work..
I want to find all fields where word is shorter than 8
my database screen:
I tried to do this using this query
{
"query": {
"match_all": {}
},
"filter": {
"script": {
"script": "doc['word'].length < 5"
}
}
}
what I doing wrong? I miss something?
Any field used in a script is loaded entirely into memory (http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-scripting.html#_document_fields), so you may want to consider an alternative approach.
You can e.g. use the regexp-filter to just find terms of a certain length, with a pattern like .{0,4}.
Here's a runnable example you can play with: https://www.found.no/play/gist/2dcac474797b0b2b952a
#!/bin/bash
export ELASTICSEARCH_ENDPOINT="http://localhost:9200"
# Index documents
curl -XPOST "$ELASTICSEARCH_ENDPOINT/_bulk?refresh=true" -d '
{"index":{"_index":"play","_type":"type"}}
{"word":"bar"}
{"index":{"_index":"play","_type":"type"}}
{"word":"barf"}
{"index":{"_index":"play","_type":"type"}}
{"word":"zip"}
'
# Do searches
# This will not match barf
curl -XPOST "$ELASTICSEARCH_ENDPOINT/_search?pretty" -d '
{
"query": {
"filtered": {
"filter": {
"regexp": {
"word": {
"value": ".{0,3}"
}
}
}
}
}
}
'
it's the first time I use the 'missing' parameter and I am not sure if I am doing something wrong as i am not getting what i expect.
Can someone please tell me if the missing condition is correctly integrated in this query? it should created 5 facets, counting for each one only the occurrences for which decimallatitude field is 'not set in the index' or its value is null.
curl -XGET http://my_url:9200/idx_occurrence/Occurrene/_search?pretty=true -d '{
"filter": {
"missing": {
"field": "decimallatitude",
"existence": true,
"null_value": true
}
},
"query": {
"query_string": {
"fields": ["dataset"],
"query": "3",
"default_operator": "AND"
}
},
"facets": {
"test": {
"terms": {
"field": ["kingdom_interpreted"],
"size": 5
}
}
}
}
'
As you can see on the Search API - Filter page, the filter is applied to your query results but not to the facets. To make it work for facets, try using the Filtered Query instead
curl -XGET http://my_url:9200/idx_occurrence/Occurrene/_search?pretty=true -d '{
"query": {
"filtered": {
"filter": {
"missing": {
"field": "decimallatitude",
"existence": true,
"null_value": true
}
},
"query": {
"query_string": {
"fields": ["dataset"],
"query": "3",
"default_operator": "AND"
}
}
}
},
"facets": {
"test": {
"terms": {
"field": ["kingdom_interpreted"],
"size": 5
}
}
}
}
'
I am issuing a query to elasticsearch and I am getting multiple record types. How do I limit the results to one type?
The following query will limit results to records with the type "your_type":
curl - XGET 'http://localhost:9200/_all/your_type/_search?q=your_query'
See http://www.elasticsearch.org/guide/reference/api/search/indices-types.html for more details.
You can also use query dsl to filter out results for specific type like this:
$ curl -XGET 'http://localhost:9200/_search' -d '{
"query": {
"filtered" : {
"filter" : {
"type" : { "value" : "my_type" }
}
}
}
}
'
Update for version 6.1:
Type filter is now replaced by Type Query: https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-type-query.html
You can use that in both Query and Filter contexts.
{
"query" : {
"filtered" : {
"filter" : {
"bool" : {
"must" :[{"term":{"_type":"UserAudit"}}, {"term" : {"eventType": "REGISTRATION"}}]
}
}
}
},
"aggs":{
"monthly":{
"date_histogram":{
"field":"timestamp",
"interval":"1y"
},
"aggs":{
"existing_visitor":{
"terms":{
"field":"existingGuest"
}
}
}
}
}
}
"_type":"UserAudit" condition will look the records only specific to type
On version 2.3 you can query _type field like:
{
"query": {
"terms": {
"_type": [ "type_1", "type_2" ]
}
}
}
Or if you want to exclude a type:
{
"query": {
"bool" : {
"must_not" : {
"term" : {
"_type" : "Hassan"
}
}
}
}
}