I am trying to make a Mongodb query in Mule with the $in function, but mule says Invalid input '$', expected Namespace or NameIdentifier
have a collection that stores user authorization
{
"_id" : ObjectId("584a0dea073d4c3e976140a9"),
"partnerDataAccess" : [
{
"factoryID" : "Fac-1",
"partnerID" : "Part-1"
}
],
"userID" : "z12",
}
{
"_id" : ObjectId("584f5eba073d4c3e976140ab"),
"partnerDataAccess" : [
{
"factoryID" : "Fac-1",
"partnerID" : "Part-2"
},
{
"factoryID" : "Fac-2",
"partnerID" : "Part-2"
}
],
"userID" : "w12",
}
the flow will submit a userID and partnerID and query the database to see if authorization exist
when I query from Robo 3T, I write queries like this
e.g. user w12 and partner Part-2
db.getCollection('user').find({
userID:"w12", "partnerDataAccess.partnerID": {$in : ["Part-2", "ALL"]}
})
The $in was used because there is the "ALL" setting for admins
but while I try to put the find part into the Mongodb connector, Mule gives error during development and runtime
Hardcoded:
<mongo:find-one-document collectionName="user" doc:name="Find one document" doc:id="a03a6689-6b9d-473c-b8a6-3b8d1e989e38" config-ref="MongoDB_Config">
<mongo:find-query ><![CDATA[#[{
userID:"w12",
"partnerDataAccess.partnerID": {$in : ["Part-2", "ALL"]}
}]]]></mongo:find-query>
</mongo:find-one-document>
parametized
<mongo:find-one-document collectionName="user" doc:name="Find one document" doc:id="a03a6689-6b9d-473c-b8a6-3b8d1e989e38" config-ref="MongoDB_Config">
<mongo:find-query ><![CDATA[#[{
userID: payload.User,
"partnerDataAccess.partnerID": {$in : [ payload.partner, "ALL"]}
}]]]></mongo:find-query>
</mongo:find-one-document>
Error:
during development:
Invalid input '$', expected } or ~ or , (line 3, column 38):
Runtime:
Message : "Script '{
userID:"w12",
"partnerDataAccess.partnerID": {$in : ["Part-2", "ALL"]}
} ' has errors:
Invalid input '$', expected Namespace or NameIdentifier (line 3, column 38):
at 3 : 3" evaluating expression:
I have tried removing the $ or escaping the $ with backslash but it does not work
I know my query is not actually complex, welcome any help
seems to have found the correct way
><![CDATA[#[{
userID:"w12",
"partnerDataAccess.partnerID": {"\$in" : ["Part-2", "ALL"]}
}]]]>
Related
I am just wondering how can I do conditional schema validation. The API response is dynamic based on customerType key. If customerType is person then, person details will be included and if the customerType is org organization details will be included in the JSON response. So the response can be in either of the following forms
{
"customerType" : "person",
"person" : {
"fistName" : "A",
"lastName" : "B"
},
"id" : 1,
"requestDate" : "2021-11-11"
}
{
"customerType" : "org",
"organization" : {
"orgName" : "A",
"orgAddress" : "B"
},
"id" : 2,
"requestDate" : "2021-11-11"
}
The schema I created to validate above 2 scenario is as follows
{
"customerType" : "#string",
"organization" : "#? response.customerType=='org' ? karate.match(_,personSchema) : karate.match(_,null)",
"person" : "#? response.customerType=='person' ? karate.match(_,orgSchema) : karate.match(_,null)",
"id" : "#number",
"requestDate" : "#string"
}
but the schema fails to match with the actual response. What changes should I make in the schema to make it work?
Note : I am planning to reuse the schema in multiple tests so I will be keeping the schema in separate files, independent of the feature file
Can you refer to this answer which I think is the better approach: https://stackoverflow.com/a/47336682/143475
That said, I think you missed that the JS karate.match() API doesn't return a boolean, but a JSON that contains a pass boolean property.
So you have to do things like this:
* def someVar = karate.match(actual, expected).pass ? {} : {}
I am working on search functionality and I need to execute a simple query that checks if there is anything matching the search string converted to lowercase. In simpler terms, user searches "SiteName", and I query if there is anything matching "sitename".
However, I get an error when I use LOWER() function in the query.
This is what I tried:
POST /_sql?format=json
{
"query":"SELECT siteid, sitename FROM zones WHERE
sitename LIKE LOWER('SiteFirst') ", "fetch_size" : 90
}
and I get this error:
{
"error" : {
"root_cause" : [
{
"type" : "parsing_exception",
"reason" : "line 1:70: mismatched input 'LOWER' expecting {'?',
STRING}"
}
],
"type" : "parsing_exception",
"reason" : "line 1:70: mismatched input 'LOWER' expecting {'?', STRING}",
"caused_by" : {
"type" : "input_mismatch_exception",
"reason" : null
}
},
"status" : 400
}
This same query works without LOWER().
Any suggestions about how to fix this error?
Thanks!
I'm pretty sure the LOWER is called LCASE in ES SQL.
More importantly, LIKE works only on exact fields, plus it's recommended to use MATCH instead of LIKE.
So try this:
POST /_sql?format=json
{
"query": "SELECT siteid, sitename FROM zones WHERE MATCH(sitename, 'SiteFirst')",
"fetch_size": 90
}
Thank you for first.
MongoDB Version:4.2.11
I have a piece of data like this:
{
"name":...,
...
"administration" : [
{"name":...,"job":...},
{"name":...,"job":...}
],
"shareholder" : [
{"name":...,"proportion":...},
{"name":...,"proportion":...},
]
}
I want to match some specified data through regular expressions:
For a example：
db.collection.aggregate([
{"$match" :
{
"$or" :
[
{"name" : {"$regex": "Keyword"}}
{"administration.name": {"$regex": "Keyword"}},
{"shareholder.name": {"$regex": "Keyword"}},
]
}
},
])
I want to set a flag when the $or operator successfully matches any condition, which is represented by a custom field, for example:{"name" : {"$regex": "Keyword"}}Execute on success:
{"$project" :
{
"_id":false,
"name" : true,
"__regex_type__" : "name"
}
},
{"administration.name" : {"$regex": "Keyword"}}Execute on success:"__regex_type__" : "administration.name"
I try do this:
{"$project" :
{
"_id":false,
"name" : true,
"__regex_type__" :
{
"$switch":
{
"branches":
[
{"case": {"$regexMatch":{"input":"$name","regex": "Keyword"}},"then" : "name"},
{"case": {"$regexMatch":{"input":"$administration.name","regex": "Keyword"}},"then" : "administration.name"},
{"case": {"$regexMatch":{"input":"$shareholder.name","regex": "Keyword"}},"then" : "shareholder.name"},
],
"default" : "Other matches"
}
}
}
},
But $regexMatch cannot match the array,I tried to use $unwind again, but returned the number of many array members, which did not meet my starting point.
I want to implement the same function as mysql this SQL statement in mongodb, like this:
SELECT name,administration.name,shareholder.name,(
CASE
WHEN name REGEXP("Keyword") THEN "name"
WHEN administration.name REGEXP("Keyword") THEN "administration.name"
WHEN shareholder.name REGEXP("Keyword") THEN "shareholder.name"
END
)AS __regex_type__ FROM db.mytable WHERE
name REGEXP("Keyword") OR
shareholder.name REGEXP("Keyword") OR
administration.name REGEXP("Keyword");
Maybe this method is stupid, but I don’t have a better solution.
If you have a better solution, I would appreciate it!!!
Thank you!!!
Since $regexMatch does not handle arrays, use $filter to filter individual array elements with $regexMatch, then use $size to see how many elements matched.
[{"$match"=>{"$or"=>[{"a"=>"test"}, {"arr.a"=>"test"}]}},
{"$project"=>
{"a"=>1,
"arr"=>1,
"src"=>
{"$switch"=>
{"branches"=>
[{"case"=>{"$regexMatch"=>{"input"=>"$a", "regex"=>"test"}},
"then"=>"a"},
{"case"=>
{"$gte"=>
[{"$size"=>
{"$filter"=>
{"input"=>"$arr.a",
"cond"=>
{"$regexMatch"=>{"input"=>"$$this", "regex"=>"test"}}}}},
1]},
"then"=>"arr.a"}],
"default"=>"def"}}}}]
[{"_id"=>BSON::ObjectId('5ffb2df748966813f82f15ad'), "a"=>"test", "src"=>"a"},
{"_id"=>BSON::ObjectId('5ffb2df748966813f82f15ae'),
"arr"=>[{"a"=>"test"}],
"src"=>"arr.a"}]
Is there any way to use fully qualified class name to search from Artifactory(similar to class-searth in Artifactory web UI). Based on this Documentation , i know i can use wildcard(*) and .class file extension like this:-
GET /api/search/archive?name=*Logger.class&repos=third-party-releases-local,repo1-cache
But i am looking for a way to use fully qualified class name similar to this:-
GET /api/search/archive?name=org.apache.log4j.Logger&repos=third-party-releases-local,repo1-cache
but this is not working.
You can use the Artifactory query language for this.
For example, a query for searching an archive item called org/apache/log4j/Logger.class in the jcenter-cache repository would be
items.find({
"repo" : "jcenter-cache",
"archive.entry.name":{"$eq":"Logger.class "},
"archive.entry.path":{"$eq":"org/apache/log4j"}
})
The response would be
{
"results" : [ {
"repo" : "jcenter-cache",
"path" : "org/apache/log4j/com.springsource.org.apache.log4j/1.2.16",
"name" : "com.springsource.org.apache.log4j-1.2.16.jar",
"type" : "file",
"size" : 481202,
"created" : "2015-12-30T20:57:36.305Z",
"created_by" : "admin",
"modified" : "2010-08-04T13:18:06.000Z",
"modified_by" : "admin",
"updated" : "2015-12-30T20:57:36.354Z"
} ],
"range" : {
"start_pos" : 0,
"end_pos" : 1,
"total" : 1
}
}
To run such a query using curl use the following when the query is inside a file names aql.txt
curl -H "content-type: text/plain" -uuser:password --data #aql.txt http://my-artifactory-host/api/search/aql
I had much success building my own little search with elasticsearch in the background. But there is one thing I couldn't find in the documentation.
I'm indexing the names of musicians and bands. There is one band called "The The" and due to the stop words list this band is never indexed.
I know I can ignore the stop words list completely but this is not what I want since the results searching for other bands like "the who" would explode.
So, is it possible to save "The The" in the index but not disabling the stop words at all?
You can use the synonym filter to convert The The into a single token eg thethe which won't be removed by the stopwords filter.
First, configure the analyzer:
curl -XPUT 'http://127.0.0.1:9200/test/?pretty=1' -d '
{
"settings" : {
"analysis" : {
"filter" : {
"syn" : {
"synonyms" : [
"the the => thethe"
],
"type" : "synonym"
}
},
"analyzer" : {
"syn" : {
"filter" : [
"lowercase",
"syn",
"stop"
],
"type" : "custom",
"tokenizer" : "standard"
}
}
}
}
}
'
Then test it with the string "The The The Who".
curl -XGET 'http://127.0.0.1:9200/test/_analyze?pretty=1&text=The+The+The+Who&analyzer=syn'
{
"tokens" : [
{
"end_offset" : 7,
"position" : 1,
"start_offset" : 0,
"type" : "SYNONYM",
"token" : "thethe"
},
{
"end_offset" : 15,
"position" : 3,
"start_offset" : 12,
"type" : "<ALPHANUM>",
"token" : "who"
}
]
}
"The The" has been tokenized as "the the", and "The Who" as "who" because the preceding "the" was removed by the stopwords filter.
To stop or not to stop
Which brings us back to whether we should include stopwords or not? You said:
I know I can ignore the stop words list completely
but this is not what I want since the results searching
for other bands like "the who" would explode.
What do you mean by that? Explode how? Index size? Performance?
Stopwords were originally introduced to improve search engine performance by removing common words which are likely to have little effect on the relevance of a query. However, we've come a long way since then. Our servers are capable of much more than they were back in the 80s.
Indexing stopwords won't have a huge impact on index size. For instance, to index the word the means adding a single term to the index. You already have thousands of terms - indexing the stopwords as well won't make much difference to size or to performance.
Actually, the bigger problem is that the is very common and thus will have a low impact on relevance, so a search for "The The concert Madrid" will prefer Madrid over the other terms.
This can be mitigated by using a shingle filter, which would result in these tokens:
['the the','the concert','concert madrid']
While the may be common, the the isn't and so will rank higher.
You wouldn't query the shingled field by itself, but you could combine a query against a field tokenized by the standard analyzer (without stopwords) with a query against the shingled field.
We can use a multi-field to analyze the text field in two different ways:
curl -XPUT 'http://127.0.0.1:9200/test/?pretty=1' -d '
{
"mappings" : {
"test" : {
"properties" : {
"text" : {
"fields" : {
"shingle" : {
"type" : "string",
"analyzer" : "shingle"
},
"text" : {
"type" : "string",
"analyzer" : "no_stop"
}
},
"type" : "multi_field"
}
}
}
},
"settings" : {
"analysis" : {
"analyzer" : {
"no_stop" : {
"stopwords" : "",
"type" : "standard"
},
"shingle" : {
"filter" : [
"standard",
"lowercase",
"shingle"
],
"type" : "custom",
"tokenizer" : "standard"
}
}
}
}
}
'
Then use a multi_match query to query both versions of the field, giving the shingled version more "boost"/relevance. In this example the text.shingle^2 means that we want to boost that field by 2:
curl -XGET 'http://127.0.0.1:9200/test/test/_search?pretty=1' -d '
{
"query" : {
"multi_match" : {
"fields" : [
"text",
"text.shingle^2"
],
"query" : "the the concert madrid"
}
}
}
'