I'd like to retrieve
document with _id of 1
OR
document with value === 13 AND anotherValue === 56
Error:
There is no index available for this selector.
This is my query:
{
"selector": {
"$or": [
{
"_id": "1"
},
{
"value": "13",
"anotherValue": "56"
}
]
}
}
Indexes setup:
Your available Indexes:
special: _id
json: value, anotherValue
json: _id, value, anotherValue
For this query you need to add a selector to get all the IDs like so:
{
"selector": {
"_id": {"$gt":null},
"$or": [
{
"_id": "1"
},
{
"value": "13",
"anotherValue": "56"
}
]
}
}
You can learn more here:
https://cloudant.com/blog/mango-json-vs-text-indexes/
And this SO post:
index and query items in an array with mango query for cloudant and couchdb 2.0
Alternatively, you can add a text index on all fields:
{
"index": {},
"type": "text"
}
And then your original selector should work.
Related
This question already has an answer here:
Asserting and using conditions for an array response in Karate
(1 answer)
Closed 2 years ago.
I want to loop through below nested json structure and want to update all the required fields, however I could achieve this through typescript but want to do this in karate JS, I do not see any examples how to nested for each works.
I want to update 26 periods data(here for readability i used 3), based on index I want to update period field, i.e. if(index == key), these 26 periods are under each car attribute.(NOTe: you again have multiple cars and multiple car attributes and each car attribute you have 26 periods data)
I cannot use this Karate - Match two dynamic responses only when you have single array list and have less data
[
{
"cars": [
{
"name": "car 1",
"periodsData": [
{
"period": "5ed73ed31a775d1ab0c9fb5c",
"index": 1
},
{
"period": "5ed73ed31a775d1ab0c9fb5d",
"index": 2
},
{
"period": "5ed73ed31a775d1ab0c9fb5e",
"index": 3
}
]
},
{
"name": "car 2",
"periodsData": [
{
"period": "5ed73ed31a775d1ab0c9fb5c",
"index": 1
},
{
"period": "5ed73ed31a775d1ab0c9fb5d",
"index": 2
},
{
"period": "5ed73ed31a775d1ab0c9fb5e",
"index": 3
}
]
},
{
"name": "car 3",
"periodsData": [
{
"period": "5ed73ed31a775d1ab0c9fb5c",
"index": 1
},
{
"period": "5ed73ed31a775d1ab0c9fb5d",
"index": 2
},
{
"period": "5ed73ed31a775d1ab0c9fb5e",
"index": 3
}
]
}
],
"totalPeriodEprps": [
{
"period": "5ed73ed31a775d1ab0c9fb5c",
"index": 1
},
{
"period": "5ed73ed31a775d1ab0c9fb5d",
"index": 2
},
{
"period": "5ed73ed31a775d1ab0c9fb5e",
"index": 3
}
]
}
carId ="dfd"
]
This above array repeats
Type script code
//periods is a map of index and values
async modifyCarsData(mid, id, periods, campaignData) {
//carData is a json file
carData.forEach(element => {
element.carId= id;
// Update all egrp periods
element.totalPeriodEGRPs.forEach(eGrpPeriod => {
// egrprd.period =
if (periods.size === element.totalPeriodEGRPs.length) {
periods.forEach((value, key) => {
if (key === eGrpPeriod.index.toString()) {
eGrpPeriod.period = value;
return true;
}
});
}
});
element.cars.forEach(carCell => {
// Logic for updating periods data
carCell .periodsData.forEach(periodAttribute => {
if (periods.size === carCell.periodsData.length) {
periods.forEach((value, key) => {
if (key === periodAttribute.index.toString()) {
periodAttribute.period = value;
return true;
}
});
}
});
});
});
Don't think of this as an update, but as a transform. I'm not using your example because it is un-necessarily complicated. Here is a simpler example that gives you all the concepts you need:
* def data = [{ name: 'one', periods: [{ index: 1, value: 'a' },{ index: 2, value: 'b' }]}, { name: 'two', periods: [{ index: 1, value: 'c' },{ index: 2, value: 'd' }]}]
* def fnPeriod = function(x){ x.value = x.value + x.index; return x }
* def fnData = function(x){ return { name: x.name, periods: karate.map(x.periods, fnPeriod) } }
* def converted = karate.map(data, fnData)
* print converted
Which prints:
[
{
"name": "one",
"periods": [
{
"index": 1,
"value": "a1"
},
{
"index": 2,
"value": "b2"
}
]
},
{
"name": "two",
"periods": [
{
"index": 1,
"value": "c1"
},
{
"index": 2,
"value": "d2"
}
]
}
]
If this doesn't work for you, please look for another tool. Karate is designed for testing and assertions, not doing what you normally do in programming languages. And I suspect that you have fallen into the trap of writing "over-smart tests", so please read this: https://stackoverflow.com/a/54126724/143475
Also refer: https://stackoverflow.com/a/53120851/143475
I have following index:
+-----+-----+-------+
| oid | tag | value |
+-----+-----+-------+
| 1 | t1 | aaa |
| 1 | t2 | bbb |
| 2 | t1 | aaa |
| 2 | t2 | ddd |
| 2 | t3 | eee |
+-----+-----+-------+
where: oid - object ID, tag - property name, value - property value.
Mappings:
"mappings": {
"document": {
"_all": { "enabled": false },
"properties": {
"oid": { "type": "integer" },
"tag": { "type": "text" }
"value": { "type": "text" },
}
}
}
This simple structure allows store any number of object properties and it is a quite simple to search by one property or by more using OR logical operator.
E.g. get object oid's where:
(tag='t1' AND value='aaa') OR (tag='t2' AND value='ddd')
ES query:
{
"_source": { "includes":["oid"] },
"query": {
"bool": {
"should": [
{
"bool": {
"must": [
{ "term": { "tag": "t1" } },
{ "term": { "value": "aaa" } }
]
}
},
{
"bool": {
"must": [
{ "term": { "tag": "t2" } },
{ "term": { "value": "ddd" } }
]
}
}
],
"minimum_should_match": "1"
}
}
}
But it is hard to search by two or more properties using AND logical operator. So the question is how to join two sub-queries to two different records through the AND operator. E.g. get object oid's where:
(tag='t1' AND value='aaa') AND (tag='t2' AND value='ddd')
In this case result must be: { "oid": "2" }
Searching data contains in two different records and applying MUST instead of SHOULD from the previous example returns nothing in this case.
I have two equivalents in SQL of what I need:
SELECT i1.[oid]
FROM [index] i1 INNER JOIN [index] i2 ON i1.oid = i2.oid
WHERE
(i1.tag='t1' AND i1.value='aaa')
AND
(i2.tag='t2' AND i2.value='ddd')
---------
SELECT [oid] FROM [index] WHERE tag='t1' AND value='aaa'
INTERSECT
SELECT [oid] FROM [index] WHERE tag='t2' AND value='ddd'
Do the two requests and merge them on the client is not the option.
Elastic Search version is 6.1.1
In order to achieve what you want, you need to use the nested type, i.e. your mapping should look like this:
PUT my-index
{
"mappings": {
"doc": {
"properties": {
"oid": {
"type": "keyword"
},
"data": {
"type": "nested",
"properties": {
"tag": {
"type": "keyword"
},
"value": {
"type": "text"
}
}
}
}
}
}
}
The documents would be indexed like this:
PUT /my-index/doc/_bulk
{ "index": {"_id": 1}}
{ "oid": 1, "data": [ {"tag": "t1", "value": "aaa"}, {"tag": "t2", "value": "bbb"}] }
{ "index": {"_id": 2}}
{ "oid": 2, "data": [ {"tag": "t1", "value": "aaa"}, {"tag": "t2", "value": "ddd"}, {"tag": "t3", "value": "eee"}] }
Then you can make your query work like this:
POST my-index/_search
{
"query": {
"bool": {
"filter": [
{
"nested": {
"path": "data",
"query": {
"bool": {
"filter": [
{
"term": {
"data.tag": "t1"
}
},
{
"term": {
"data.value": "aaa"
}
}
]
}
}
}
},
{
"nested": {
"path": "data",
"query": {
"bool": {
"filter": [
{
"term": {
"data.tag": "t2"
}
},
{
"term": {
"data.value": "ddd"
}
}
]
}
}
}
}
]
}
}
}
There might be one way, which is a little ugly: adding terms aggregations to your query body.
{
"query": {
"bool": {
"should": [
{
"bool": {
"must": [
{ "term": { "tag": "t1" } },
{ "term": { "value": "aaa" } }
]
}
},
{
"bool": {
"must": [
{ "term": { "tag": "t2" } },
{ "term": { "value": "ddd" } }
]
}
}
],
"minimum_should_match": "1"
}
},
"size": 0,
"aggs": {
"find_joined_oid": {
"terms": {
"field": "oid.keyword"
}
}
}
}
If everything goes right, this will output something like
{
"took": 123,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 123,
"max_score": 0,
"hits": []
},
"aggregations": {
"find_joined_oid": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"key": "1",
"doc_count": 1
},
{
"key": "2",
"doc_count": 2
}
}
}
}
Here, in the "aggregations" part,
"key": "1"
means your "oid":"1", and
"doc_counts": 1
means there is 1 hit in query with "oid":"1".
As you know how many tags you are querying to match, say N, in the aggregations result body, only those "key"s with "doc_count" equal to N are the result you're pursuing. In this example, you are querying tag:t1 (with value aaa) and tag:t2 (with value ddd), thus N=2. You can iterate in the result bucket list to find out those "key"s who have "doc_count" equal to 2.
However, there should be a better way. If you would alter your mapping to a document like style, ie. store all fields of one oid in one doc, life will be much easier.
{
"properties": {
"oid": { "type": "integer" },
"tag-1": { "type": "text" }
"value-1": { "type": "text" },
"tag-2": { "type": "text" }
"value-2": { "type": "text" }
}
}
When you want to add new tag-value pairs, just get the original doc with oid concerned, put new tag-pair into the doc, and put the whole new doc back into Elasticsearch with the same _id which you get from the original one. Most of the time dynamic mapping will work properly in your case, which means you don't need to assert mapping for new fields explicitly.
No-SQL databases like Elasticsearch and others are not designed to handle such SQL style query you are asking.
I have a Cloudant DB on Bluemix with an index defined as:
{
"index": {
"fields": [
{ "typ": "asc" },
{ "sen": "asc" },
{ "tim": "asc" }
]
},
"type": "json"
}
WHen I have a query of the form
{
"selector": {
"tim": {"$gt": millisecs},
"typ": "H"
},
"fields": ["sen","val","tim"],
"sort": [
{ "typ": "asc" },
{ "sen": "asc" },
{ "tim": "asc" }
],
"limit": readCount
}
it works perfectly. If I want to get everything, i.e. remove the condition typ="H", I get the error
"error":"no_usable_index","reason":"There is no index available for this selector."
I get the same response if I have "typ" : { "$in": ["H", "T"] }. I would have expected that the more generic query would work better than the one with extra selectors.
I just don't understand how this could be!
"typ" is the first field of your index, so is the basis of the ordering.
"tim", if it's the only element of the query, doesn't take advantage of the index, so it would trigger a full table scan if that query was allowed.
However you can ask explicitely for a full table scan if you add:
"_id": { "$gt": null }
See the doc, your case is not really described, but I think it's implied.
Did you try to create separate indexes for these fields and run the same query?
Here is my data
[
{
"properties": {
"key": {
"data": "companya data",
"company": "Company A"
}
},
"uniqueId" : 1
},
{
"properties": {
"key": {
"data": "companyb data",
"company": "Company B"
}
},
"uniqueId" : 2
},
{
"properties": {
"key": {
"data": "companyc data",
"company": "Company C"
}
},
"uniqueId" : 3
}
]
The format I need for my typeahead directive is below. I was trying to figure out the other post I made but still couldn't make it work. The best was to just make the nested collection as a simple collection of object.
[
{
"uniqueId" : 1,
"data": "companya data"
},
{
"uniqueId" : 2,
"data": "companyb data"
},
{
"uniqueId" : 3,
"data": "companyc data"
}
]
I got it!
console.log(
_(jsonData).map(function(obj) {
return {
d : obj.properties.key.data,
id : obj.uniqueId
}
})
.value()
);
You do not have to use the chaining feature of lodash as long as you are only performing one operation. You can simply use:
_.map(jsonData, function(obj) {
return {
d : obj.properties.key.data,
id : obj.uniqueId
}
});
I changed the elasticsearch mapping field type from:
"articles": {
"properties": {
"id": {
"type": "long"
}}}
to
"articles": {
"properties": {
"id": {
"type": "string",
"index": "not_analyzed"
}
After that I did the following steps:
Create the index with new mapping
Reindex the mapping to the new index
After the mapping update my previous query filter doesn't work anymore and I have no results:
GET /art/_search
{
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"bool": {
"must": [
{
"type": {
"value": "articles"
}
},
{
"term": {
"id": "123467679"
}
}
]
}
}
}
},
"size": 1,
"sort": [
{
"_score": "desc"
}
]
}
If I check with this query the result is what I expect:
GET /art/articles/_search
{
"query": {
"match_all": {}
}
}
I would appreciate if somebody have some idea why after the field type change the query is no longer working.
Thanks!
The problem in the query was with ID filter.
The query works correctly changing the filter from:
"term": {
"id": "123467679"
}
in:
"term": {
"_id": "123467679"
}
I'm still a beginner with elasticsearch to figure out why the mapping change broke the query although I did the reindex, but "_id" fixed my query.
You can find more informations in the :
elasticsearch mapping reference documentation.