I have used Node to retrieve a set of results from SQL and they're returned like this;
[
{
"event_id": 111111,
"date_time": "2012-11-16T01:59:07.000Z",
"agent_addr": "127.0.0.1",
"priority": 6,
"message": "aaaaaaaaa",
"up_time": 9015040,
"hostname": "bbbbbbb",
"context": "ccccccc"
},
{
"event_id": 111112,
"date_time": "2012-11-16T01:59:07.000Z",
"agent_addr": "127.0.0.1",
"priority": 6,
"message": "aaaaaaaaa",
"up_time": 9015040,
"hostname": "bbbbbbb",
"context": "ddddddd"
},
]
There are usually a lot of entries in the array and I need to efficiently filter the array to show only the entries that have a context of "ccccccc". I've tried a for loop, but it's incredibly slow.
Any suggesstions?
There is a very simple way of doing that if you want to do that in node and don't want to use sql for that you can user javascript built-in Array.filter function.
var output = arr.filter(function(x){return x.context=="ccccccc"}); //arr here is you result array
The ouput array will contains only objects having context "ccccccc".
Another way of doing what Khurrum said, is with the arrow function. It has the same result but some people prefer that notation.
var output = arr.filter(x => x.context == "ccccccc" );
As suggested by Matt, why not include WHERE context = "ccccccc" in yout SQL query?
Else if you must keep all in maybe use one of the following to filter the results
// Place all "ccccccc" context row in an array
var ccccccc = [];
for (var i = results.length - 1; i >= 0; i--) {
if(results[i] == 'ccccccc')
ccccccc.push(results[i]);
};
// Place any context in an named array within an object
var contexts = {};
for (var i = results.length - 1; i >= 0; i--) {
if(contexts[results[i]] == 'undefined')
contexts[results[i]]
contexts[results[i]].push(results[i]);
};
or use the underscore (or similar) filter function.
Related
I've been stuck for a while trying to optimise my groq query.
I have page content that contains an array objects (different languages).
I've been playing around in Sanity Vision to see how I can filter the output so that I only get the content in the correct language.
//query
*[_type == "home"]{
content[]{
"language": metaData.language ->.language,
},
}
// query result
"result":[
0:{
"content":[
0:{
"language":"en-AU"
}
1:{
"language":"th-TH"
}
]
}
]
I wanted to get just the 1 content that matches the language.
I tried this but it didn't work
*[_type == "home"]{
content[]{
...,
"language": metaData.language ->.language,
},
}[0][content[].language == "en-AU"]
Does anyone know how?
Thank you!
Finally found an answer
I realise where there is an array inside the return data, you can filter it further using another [], in this case [metaData.language->.language match $language]
*[_type == "home"]{
content[metaData.language->.language == $language]{
...,
metaData {
...,
language->
}
}[0]
}[0]
Here is my scenario. I have data in a Cosmos DB and I want to return c.this, c.that etc as the indexer for Azure Cognitive Search. One field I want to return is JSON of an unknown structure. The one thing I do know about it is that it is flat. However it is my understanding that the return value for an indexer needs to be known. How, using SQL in a SELECT, would I return all JSON elements in the flat object? Here is an example value I would be querying:
{
"BusinessKey": "SomeKey",
"Source": "flat",
"id": "SomeId",
"attributes": {
"Source": "flat",
"Element": "element",
"SomeOtherElement": "someOtherElement"
}
}
So I would want my select to be maybe something like:
SELECT
c.BusinessKey,
c.Source,
c.id,
-- SOMETHING HERE TO LIST OUT ALL ATTRIBUTES IN THE JSON AS FIELDS IN THE RESULT
And I would want the result to be:
{
"BusinessKey": "SomeKey",
"Source": "flat",
"id": "SomeId",
"attributes": [{"Source":"flat"},{"Element":"element"},{"SomeOtherElement":"someotherelement"}]
}
Currently we are calling ToString on the c.attributes, which is the JSON of unknown structure but it is adding all the escape characters. When we want to search the index, we have to add all those escape characters and it's getting really unruly.
Is there a way to do this using SQL?
Thanks for any help!
You could use UDF in cosmos db sql.
UDF code:
function userDefinedFunction(object){
var returnArray = [];
for (var key in object) {
var map = {};
map[key] = object[key];
returnArray.push(map);
}
return returnArray;
}
Sql:
SELECT
c.BusinessKey,
c.Source,
c.id,
udf.test(c.attributes) as attributes
from c
Output:
Using Microsoft CosmoDBs SQL like syntax. I have a collection of entries that follow a schema like this (simplified for this post)
{"id":"123456",
"activities": {
"activityA": {
"loginType": "siteA",
"lastLogin": "2018-02-06T19:42:22.205Z"
},
"activityB": {
"loginType": "siteB",
"lastLogin": "2018-03-07T11:39:50.346Z"
},
"activityC": {
"loginType": "siteC",
"lastLogin": "2018-04-08T15:21:15.312Z"
}
}
}
Without knowing the exact index into the activities entry activities list/sub collection, how can I query to get back all items in the Cosmo db collection that have a "lastLogin" matching a date range?
If I only wanted to search on the first item in the activities list, I could do something like this using index 0.
SELECT * FROM c where (c.activities[0].lastLogin > '2018-01-01T00:00:00') and (c.activities[0].lastLogin <= '2019-02-15T00:00:00')
But I want to search all entries in the list. Would be nice if there was something like this:
SELECT * FROM c where (c.activities[?].lastLogin > '2018-01-01T00:00:00') and (c.activities[?].lastLogin <= '2019-02-15T00:00:00')
But that doesn't exist.
The answer is that you can not iterate over a non list collection. Had the collection item been structured like this
{"id":"123456",
"activities": [
{ "label": "activityA",
"loginType": "siteA",
"lastLogin": "2018-02-06T19:42:22.205Z"
},
{
"label": "activtyB",
"loginType": "siteB",
"lastLogin": "2018-03-07T11:39:50.346Z"
},
etc...
It would be easy to crease a UDF to iterate over with something like this
UDF: filterActivityList
function(activityList, targetDateTimeStart, targetDateTimeEnd) {
var s, _i, _len;
for (_i = 0, _len = activityList.length; _i < _len; _i++) {
s = activityList[_i];
if ((s.lastLogin >= targetDateTimeStart) && (s.lastLogin < targetDateTimeEnd))
{
return true;
}
}
return false;
}
Then to query:
select * from c WHERE udf.filterActivityList(c.activities, '2018-01-01T00:00:00', '2018-02-01T00:00:00');
If I were to leave the structure as a JSON hierarchy instead of converting it to a JSON list then I would have to write another udf to accept the top level node of the hierarchy as an input parameter and have it convert the notes under it to a list, then apply the udf.filterActivityList UDF to the result. From my experience this approach is resource intensive and takes a very long time for Cosmo to process.
As a beginner I have few questions. I am using the Get request, which would populate json below.
https://reqres.in/api/users
{
"total": 12,
"total_pages": 4,
"data": [{
"id": 1,
"first_name": "George",
"last_name": "Bluth",
"avatar": "https://s3.amazonaws.com/uifaces/faces/twitter/calebogden/128.jpg"
}]
}
for the 2 tests below while the 1st one passes the 2nd test fails with the message:
AssertionError: expected false to be truthy
//Verify Page number total is 12
var jsonData = JSON.parse(responseBody);
tests["Checking total page number-Manual"] = jsonData.total === 12;
//verify is exists and is 1
var jsonData = JSON.parse(responseBody);
tests["Checking ID exists and is 1"] = jsonData.id === 1;
Question 1:
A github post that I found says there may be an error and suggests to use
the new pm.* equivalent instead. However I do not see any difference between the 1st and the 2nd. So why does the 2nd test fail?
Question 2:
Is it possible to write test to verify that for ID:1 the firstname is George?
Thanks in advance for your time.
The reason that your 2nd test fails is because data is an array and in this case you must access the first element. You would want to do something like this (new syntax):
pm.test("Verify id is equal to 1", function() {
var jsonData = pm.response.json();
pm.expect(jsonData.data[0].id).to.equal(1);
});
Similarly for testing first name is George:
pm.test("Verify id is equal to 1", function() {
var jsonData = pm.response.json();
pm.expect(jsonData.data[0].first_name).to.equal("George");
});
If you always expect it to only be a single element in the array then you're safe to use index 0 i.e. data[0]. However if you expect there to be more elements in the data array then you would have to iterate through them to look for the correct element.
Here's a good reference for the API:
https://learning.getpostman.com/docs/postman/scripts/postman_sandbox_api_reference/
With SQL we can do the following :
select * from x where concat(x.y ," ",x.z) like "%find m%"
when x.y = "find" and x.z = "me".
How do I do the same thing with MongoDB, When I use a JSON structure similar to this:
{
data:
[
{
id:1,
value : "find"
},
{
id:2,
value : "me"
}
]
}
The comparison to SQL here is not valid since no relational database has the same concept of embedded arrays that MongoDB has, and is provided in your example. You can only "concat" between "fields in a row" of a table. Basically not the same thing.
You can do this with the JavaScript evaluation of $where, which is not optimal, but it's a start. And you can add some extra "smarts" to the match as well with caution:
db.collection.find({
"$or": [
{ "data.value": /^f/ },
{ "data.value": /^m/ }
],
"$where": function() {
var items = [];
this.data.forEach(function(item) {
items.push(item.value);
});
var myString = items.join(" ");
if ( myString.match(/find m/) != null )
return 1;
}
})
So there you go. We optimized this a bit by taking the first characters from your "test string" in each word and compared the tokens to each element of the array in the document.
The next part "concatenates" the array elements into a string and then does a "regex" comparison ( same as "like" ) on the concatenated result to see if it matches. Where it does then the document is considered a match and returned.
Not optimal, but these are the options available to MongoDB on a structure like this. Perhaps the structure should be different. But you don't specify why you want this so we can't advise a better solution to what you want to achieve.