I have a document like below, the "tags" field is a nested document, and I want to make all child field for tags document to be index = not_analyzed. The problem is that field in tags will be dynamic. any tag could possible.
So how I can define dynamic mapping for this.
{
strong text'level': 'info',
'tags': {
'content': u'Nov 6 11:07:10 ja10 Keepalived_healthcheckers: Adding service [172.16.08.105:80] to VS [172.16.1.21:80]',
'id': 1755360087,
'kid': '2012121316',
'mailto': 'yanping3,chunying,pengjie',
'route': 15,
'service': 'LVS',
'subject': 'LVS_RS',
'upgrade': 'no upgrade configuration for this alert'
},
'timestamp': 1383707282.500464
}
I think you can use dynamic templates for this. For example following shell script creates dynamic_mapping_test index with dynamic template set when indexing field tags.*, mapping is set to type:string and index:not_analyzed.
echo "Delete dynamic_mapping_test"
curl -s -X DELETE http://localhost:9200/dynamic_mapping_test?pretty ; echo ""
echo "Create dynamic_mapping_test with nested tags and dynamic_template"
curl -s -X POST http://localhost:9200/dynamic_mapping_test?pretty -d '{
"mappings": {
"document": {
"dynamic_templates": [
{
"string_template": {
"path_match": "tags.*",
"mapping": {
"type": "string",
"index": "not_analyzed"
}
}
}
],
"properties": {
"tags": {
"type": "nested"
}
}
}
}
}' ; echo ""
echo "Display mapping"
curl -s "http://localhost:9200/dynamic_mapping_test/_mapping?pretty" ; echo ""
echo "Index document with new property tags.content"
curl -s -X POST "http://localhost:9200/dynamic_mapping_test/document?pretty" -d '{
"tags": {
"content": "this CONTENT should not be analyzed"
}
}' ; echo ""
echo "Refresh index"
curl -s -X POST "http://localhost:9200/dynamic_mapping_test/_refresh"
echo "Display mapping again"
curl -s "http://localhost:9200/dynamic_mapping_test/_mapping?pretty" ; echo ""
echo "Index document with new property tags.title"
curl -s -X POST "http://localhost:9200/dynamic_mapping_test/document?pretty" -d '{
"tags": {
"title": "this TITLE should not be analyzed"
}
}' ; echo ""
echo "Refresh index"
curl -s -X POST "http://localhost:9200/dynamic_mapping_test/_refresh"; echo ""
echo "Display mapping again"
curl -s "http://localhost:9200/dynamic_mapping_test/_mapping?pretty" ; echo ""
I suggest, all string "not_analyzed", and all numbers to long and "not_analyzed".
Because default string analyzed have more memory and file size.
I have reduced size and search fields' full word
range search long type.
{
"mappings": {
"_default_": {
"_source": {
"enabled": true
},
"_all": {
"enabled": false
},
"_type": {
"index": "no",
"store": false
},
"dynamic_templates": [
{
"el": {
"match": "*",
"match_mapping_type": "long",
"mapping": {
"type": "long",
"index": "not_analyzed"
}
}
},
{
"es": {
"match": "*",
"match_mapping_type": "string",
"mapping": {
"type": "string",
"index": "not_analyzed"
}
}
}
]
}
}
}
I don't think there is any way to specify mapping while indexing the data. So, as an alternative, you can modify your tags document to have the following mapping:
{ tags: {
properties: {
tag_type: {type: 'string', index: 'not_analyzed'}
tag_value: {type: 'string', index: 'not_analyzed'}
}
}
}
Here, tag_type can contain the any of the values (content, id, kid, mailto, etc.), and tag_values can contain the actual value of the field that is named in tag_type.
Related
Im trying to make this api POST request to view.publish endpoint on slack api
As the documentation explains, im using the token and user_id in params, but i dont know what do i need to do with the view param
i set "application/json;charset=UTF-8" as content-type in the headers and on the body the JSON of the payload i want to publish:
{
"blocks": [
{
"type": "section",
"text": {
"type": "mrkdwn",
"text": "This is a section block with a button."
},
"accessory": {
"type": "button",
"text": {
"type": "plain_text",
"text": "Click Me",
"emoji": true
},
"value": "click_me_123",
"url": "http://scoreboard-azureslackbot-salvosoftware.s3-website.us-east-2.amazonaws.com/",
"action_id": "button-action"
}
}
]
}
This error is displayed when i make that api call:
{
"ok": false,
"error": "invalid_arguments",
"response_metadata": {
"messages": [
"[ERROR] failed to match all allowed schemas [json-pointer:/view]",
"[ERROR] must provide an object [json-pointer:/view]",
"[ERROR] must provide an object [json-pointer:/view]"
]
}
}
Documentation says that view must be a JSON-encoded string
Also this warning is displayed in the body tab:
This answer is from Aubrey, support team in slack.
No params needed.
In the Authorization tab you need beared authorization.
On the body you need to set is as raw -> JSON and this would be your JSON:
{
"user_id": {{YOUR_USER_ID}},
"view": {
"type": "home",
"blocks": [
{
{{YOUR_PAYLOAD}}
}
]
}
}
It helped me basing on this CURL command
curl -L -X POST 'https://slack.com/api/views.publish' \
-H 'Content-type: application/json' \
-H 'Authorization: Bearer {{YOUR TOKEN}}' \
--data-raw '{
"user_id": {{YOUR USER ID}},
"view": {
"type": "home",
"blocks": [{
{{YOUR PAYLOAD}}
}]
}
}'
I have an indice named users
When I make a request on http://localhost:9200/users/_search?pretty=true with the following query:
curl -X GET "localhost:9200/users/_search?pretty=true" -H 'Content-Type: application/json' -d'
{
"query": {
"query_string": {
"query" : "firstName: Daulet"
}
}
}'
the query returns two users with the following names:
firstName: Daulet
firstName: Daulet Nurlanuly
How do I make the query string query return a the document with firstName: Daulet ?
I've looked up that Elasticsearch uses Apache Lucene's request syntax and that for the strict search I would need to do the following by enclosing request in quotes as followes:
firstName: "Daulet"
But it is already enclosed within quotes
How do I do that using only Query String Query?
** UPDATE **
The response I get when I make a GET request at http://localhost:9200/users:
{
"users": {
"aliases": {},
"mappings": {
"userentity": {
"properties": {
"firstName": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"id": {
"type": "long"
},
"language": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"lastName": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
}
},
"settings": {
"index": {
"refresh_interval": "1s",
"number_of_shards": "5",
"provided_name": "users",
"creation_date": "1530245236170",
"store": {
"type": "fs"
},
"number_of_replicas": "1",
"uuid": "IlE1Ynv2Q462LBttptVaTg",
"version": {
"created": "5060999"
}
}
}
}
}
You're correct that you need to surround the value with double quotes. You're on the right path and you simply need to escape the double quotes and use the firstName.keyword field instead of firstName, basically like this:
curl -X GET "localhost:9200/users/_search?pretty=true" -H 'Content-Type: application/json' -d'
{
"query": {
"query_string": {
"query" : "firstName.keyword:\"Daulet\""
}
}
}'
curl -XPUT "localhost:9200/products" -d '{
"settings": {
"index": {
"number_of_replicas" : 0,
"number_of_shards": 1
}
},
"mappings": {
"products": {
"properties": {
"location" : {
"type" : "geo_point"
}
}
}
}
}'
I currently have a bash script that creates my index. Code is above.
How do I add stemming to it?
The most generic way to do it is by replacing default analyzer with snowball analyzer. This will enable stemming for all dynamically-mapped string fields. This is how you can enable english stemmer:
curl -XPUT "localhost:9200/products" -d '{
"settings": {
"index": {
"number_of_replicas" : 0,
"number_of_shards": 1,
"analysis" :{
"analyzer": {
"default": {
"type" : "snowball",
"language" : "English"
}
}
}
}
},
"mappings": {
"products": {
"properties": {
"location" : {
"type" : "geo_point"
}
}
}
}
}'
I'm trying to create an index within a set under a specific namespace, but am unsure how to do it.
My resources have this as an HTTP example:
POST /example/v1/index/{namespace}/{set}/{indexName}
and for an example input:
{
"fields": [
{ "indexField": "firstName", "indexReverseOrder": true },
{ "indexField": "lastName" }
],
"options": {
"isUnique": true
}
}
this consumes
application/json;charset=UTF-8
but when I write this out as
curl -X POST exampleurl.com/example/v1/index/example_namespace/example_set/example
set -d " {
"fields": [
{ "indexField": "firstName", "indexReverseOrder": true },
{ "indexField": "lastName" }
],
"options": {
"isUnique": true
} }" -H "Content-type : application/json;charset=UTF-8"
I get the following HTTP status code
HTTP/1.1 415 Unsupported Media Type
Can anyone explain to me what's going on and how I might fix this? Also, let me know if you don't have enough information about the API to understand it, thanks!
EDIT:
As some sort of a reference, for this API when I create a set in a namespace I do:
curl -X POST http://exampleurl.com/example/v1/store/example_namespace -d "example_set" -H "Content-type: application/json;charset=UTF-8"
and this is successful. I thought indexes would be similar to this, but apparently not.
The error is due to the bash shell misinterpreting the double quotes before the json
curl -X POST exampleurl.com/example/v1/index/example_namespace/example_set/example
set -d " {
"fields": [
{ "indexField": "firstName", "indexReverseOrder": true },
{ "indexField": "lastName" }
],
"options": {
"isUnique": true
} }" -H "Content-type : application/json;charset=UTF-8"
should be:
curl -X POST exampleurl.com/example/v1/index/example_namespace/example_set/example
set -d ' {
"fields": [
{ "indexField": "firstName", "indexReverseOrder": true },
{ "indexField": "lastName" }
],
"options": {
"isUnique": true
} }' -H "Content-type : application/json;charset=UTF-8"
The difference is the single quotes encapsulating the json. The bash shell will give an error in trying to execute the command.
You have a typo in your media type:
application/json;charset=UTF=8
Should be:
application/json;charset=UTF-8
it's the first time I use the 'missing' parameter and I am not sure if I am doing something wrong as i am not getting what i expect.
Can someone please tell me if the missing condition is correctly integrated in this query? it should created 5 facets, counting for each one only the occurrences for which decimallatitude field is 'not set in the index' or its value is null.
curl -XGET http://my_url:9200/idx_occurrence/Occurrene/_search?pretty=true -d '{
"filter": {
"missing": {
"field": "decimallatitude",
"existence": true,
"null_value": true
}
},
"query": {
"query_string": {
"fields": ["dataset"],
"query": "3",
"default_operator": "AND"
}
},
"facets": {
"test": {
"terms": {
"field": ["kingdom_interpreted"],
"size": 5
}
}
}
}
'
As you can see on the Search API - Filter page, the filter is applied to your query results but not to the facets. To make it work for facets, try using the Filtered Query instead
curl -XGET http://my_url:9200/idx_occurrence/Occurrene/_search?pretty=true -d '{
"query": {
"filtered": {
"filter": {
"missing": {
"field": "decimallatitude",
"existence": true,
"null_value": true
}
},
"query": {
"query_string": {
"fields": ["dataset"],
"query": "3",
"default_operator": "AND"
}
}
}
},
"facets": {
"test": {
"terms": {
"field": ["kingdom_interpreted"],
"size": 5
}
}
}
}
'