I have a collection containing multiple clients, months, etc.
Each month I need to run a series of queries against the collection to retrieve metrics - basically a bunch of counts.
Most of those queries have the same 4 or 5 filters, with an additional 1-2 filters that change for each query.
//standard filters:
{ client: "ABC Corp",
environment: "Production",
device: "true",
registered: "true"
}
//special filters:
{ type: "typeA",
screen: "screen1" }
{ type: "typeA",
screen: "screen2" }
In MSSQL I would create a view containing the 4 standard filters, and query that view repeatedly, applying the additional 1-2 filters to retrieve the needed metrics.
Any suggestions about what MongoDB or basic JS approach I could use for this? My goal is to avoid hitting the collection over and over again with those same standard filters.
Thanks
V
You could use $out operator in aggregation framework to achieve this.
$out stage can output results into a new collection
For example:
db.collection.aggregate([
{
$match:
{
client: "ABC Corp",
environment: "Production",
device: "true",
registered: "true"
}
},
{
$out: "new_collection"
}
])
Then you can run your additional queries on the new_collection.
Your "view" collection will be recreated every time when you use it in $out stage but any indexes you defined on that collection will be left intact.
Related
I'm struggling to get my head around this one and I know the way to do this is through a custom index. Essentially, I have several collections that share some common properties, one of which is "system id" which describes a many-to-one relationship, e.g.
// Id() = "component:a"
{
"Name": "Component A"
"SystemId": "system:foo"
}
// Id() = "resource:a"
{
"Name": "Resource A",
"SystemId": "system:foo"
}
So these are two example objects which live in two different collections, Components and Resources, respectively.
I have another collection called "Notifications" and they have a RecipientID which refers to the Id of one of the entities described above. e.g.
// Id() = "Notifications/84-A"
{
"RecipientID": "component:a",
"Message": "hello component"
}
// Id() = "Notifications/85-A"
{
"RecipientID": "resource:a",
"Message": "hello resource"
}
The query that I want to be able to perform is pretty straight forward -- "Give me all notifications that are addressed to entities which have a system of ''" but I also want to make sure I have some other bits from the entities themselves such as their name, so a result object something like this:
{
"System": "system:foo",
"Notifications": [{
"Entity": "component:a",
"EntityName": "Component A",
"Notifications": [{
"Id": "Notifications/84-A",
"Message": "hello component"
}]
}, {
"Entity": "resource:a",
"EntityName": "Resource A",
"Notifications": [{
"Id": "Notifications/85-A",
"Message": "hello resource"
}]
}]
}
Where I am with it right now is that I'm creating a AbstractMultiMapIndexCreationTask which has maps for all of these different entities and then I'm trying to use the reduce function to mimic the relationship.
Where I'm struggling is with how these map reduces work. They require that the shape is identical between maps as well as the reduce output. I've tried some hacky things like including all of the notifications as part of the map and putting dummy values where the properties don't match, but besides it making it really hard to read/understand, I still couldn't figure out the reduce function to make it work properly.
How can I achieve this? I've been digging for examples, but all of the examples I've come across make some assumptions about how references are arranged. For example, I don't think I can use Include because of the different collections (I don't know how Raven would know what to query), and coming from the other direction, the entities don't have enough info to include or load notifications (I haven't found any 'load by query' function). The map portion of the index seems fine, I can say 'give me all the entities that share this system, but the next part of grabbing the notifications and creating that response above has eluded me. I can't really do this in multiple steps either, because I also need to be able to page. I've gone in circles a lot and I could use some help.
How about indexing the related docs ?
Something like this (a javascript index):
map("Notifications", (n) => {
let c = load(n.RecipientID, 'Components');
let r = load(n.RecipientID, 'Resources');
return {
Message: n.Message,
System: c.SystemId || r.SystemId,
Name: c.Name || r.Name,
RelatedDocId: id(c) || id(r)
};
})
Then you can query on this index, filter by the system value, and get all matching notifications docs.
i.e. sample query:
from index 'yourIndexName'
where System == "system:foo"
Info about related documents is here:
RavenDB Demo
https://demo.ravendb.net/demos/csharp/related-documents/index-related-documents
Documentation
https://ravendb.net/docs/article-page/5.4/csharp/indexes/indexing-related-documents
I have a simple singleton document schema defined in my Sanity/NextJS project, to model my "Colophon" page (richText is a custom block field type):
export default {
title: 'Colophon',
name: 'colophon',
type: 'document',
__experimental_actions: ['update', 'publish'],
fields: [
{
title: 'Body',
name: 'body',
type: 'richText',
validation: Rule => Rule.required(),
},
],
};
I retrieve this document with a simple query in my NextJS application:
export async function getStaticProps() {
const colophon = await client.fetch(`
*[_type == "colophon"][0]
`);
// ...
};
Is it possible to write a GROQ query to retrieve the meta title defined in the schema, i.e. Colophon? Although this is a singleton document, I would like to avoid repeating this string in my project if possible. At the moment, I can only see the fields on the document in my results, i.e. body.
Thanks for reading!
No, I don't believe there is.
As far as I understand what you're after; The schema is defined in the studio-instance and not the datastore. Those two are not hard coupled. I have several studio-instances with small variations on the schemas using one single project/datastore. The API you query to get data does not care which studio and schema was used and cant answer for the actual schema details.
I'm trying to query tasks by ObjectID to get their most recent snapshots. I'm trying the API out, and am not getting the expected results that I was hoping for after reading the docs. I don't get a results field in the response object. Here's my code:
_loadTaskSnapshot: function() {
let snapshot = Ext.create('Rally.data.lookback.SnapshotStore', {
context: {
workspace: '/workspace/2290039850'
},
"find": {
"ObjectID": 34858774310,
"_ValidFrom": {
"$gte": "2016",
"$lt": "2017"
}
},
"fields": ["Name", "Estimate", "ToDo", "TimeSpent"],
});
return snapshot.load();
}
This is the store with 18 snapshots for the specified task. The first snapshot is opened. You can see there is no results field where I could find the Name, Estimate, ToDo, and TimeSpent:
Alejandro, you are asking for the changes in the fields, not the values of the fields. This is a common misconception with the lookback api. There is a special way to get the current values shown in the help pages available inside Agile Central.
Any information returned is actually held within the object underneath 'raw' and 'data'. Each of those may not contain any values if there has been no 'changes' to those fields at the time the snapshot was taken.
I'm trying to query a collection of data with entities that contain the "related" property:
...,{ related :
{ global: [{name: "foo"}, {name: "bar"}] },
{ local: [{name: "bar"}] },
{ random: [{name: "foo"}] },
{ dingbat: [{name: "baz"}] },
}
I want to write a query which selects all entities which have name="foo" anywhere within the related property.
I can do this just fine:
select * where related.global.name='foo'
However there could be any number of keys within the "related" property so I can't just AND them all into a single query. Trying to do something like this (which doesn't work)
select * where related.*.name='foo'
Is there any way to achieve this?
Not at this time. However, it is something we could consider for the future. We have ElasticSearch for indexing in our 2.1 platform and we can use the '_all' functionality from ES but we have not exposed that in Usergrid yet.
I have inherited code that displays a graph of defects on a project. I have now took my project and split it into two projects, so that now there is a parent project and two children. The code (below) just accumulates the defects from the parent and does not include the data from the children.
snippet:
storeType: "Rally.data.lookback.SnapshotStore",
storeConfig: {
find: { _TypeHierarchy: "Defect", Children: null },
fetch: ["Severity", "State"], hydrate: ["Severity", "State"],
sort: { _ValidFrom: 1 },
filters: [{ property: "Project", value: context.getProject().ObjectID }, { property: "_TypeHierarchy", value: "Defect" }, { property: "Children", value: null}] },
So I'm pretty sure the problem is in this part "value: context.getProject().ObjectID" as it says to get the data from the current project (and not its children). How can I accomplish what I need?
Not sure if you're intending to get lookback (time series) data or current (WSAPI) data from Rally. Your code implies lookback so I will answer with that in mind.
You could try adding to your find clause (and removing the current Project filter):
"_ProjectHierarchy": { $in : [123] }
where 123 is the object id of your parent project. That should get defects from any projects that include you parent project in the hierarchy.
So Igor basically got it above, just writing as a post and not in a commet, this is the code that works - you can see why in the comments above. Please note - I did make one change over what he wrote, as when I used the "__At" it caused not all bugs to be counted - it looked like it only counted bugs since their last update time.
storeConfig: {
find: { _TypeHierarchy: "Defect", _ProjectHierarchy: context.getProject().ObjectID},
fetch: ["Severity", "State"], hydrate: ["Severity", "State"],
sort: { _ValidFrom: 1 },
},