FaunaDB get entries by date range with index binding not working - faunadb

I am struggling to get an Index by Date to work with a Range.
I have this collection called orders:
CreateCollection({name: "orders"})
And I have these sample entries, with one attribute called mydate. As you see it is just a string. And I do need to create the date as a string since in my DB we already have around 12K records with dates like that so I cant just start using the Date() to create them.
Create(Collection("orders"), {data: {"mydate": "2020-07-10"}})
Create(Collection("orders"), {data: {"mydate": "2020-07-11"}})
Create(Collection("orders"), {data: {"mydate": "2020-07-12"}})
I have created this index that computes the date to and actual Date object
CreateIndex({
name: "orders_by_my_date",
source: [
{
collection: Collection("orders"),
fields: {
date: Query(Lambda("order", Date(Select(["data", "mydate"], Var("order"))))),
},
},
],
terms: [
{
binding: "date",
},
],
});
If I try to fetch a single date the index works.
// this works
Paginate(
Match(Index("orders_by_my_date"), Date("2020-07-10"))
);
// ---
{
data: [Ref(Collection("orders"), "278496072502870530")]
}
But when I try to get a Range it never finds data.
// This does NOT work :(
Paginate(
Range(Match(Index("orders_by_my_date")), Date("2020-07-09"), Date("2020-07-15"))
);
// ---
{
data: []
}
Why the index does not work with a Range?

Range operates on the values of an index, not on the terms.
See: https://docs.fauna.com/fauna/current/api/fql/functions/range?lang=javascript
You need to change your index definition to:
CreateIndex({
name: "orders_by_my_date",
source: [
{
collection: Collection("orders"),
fields: {
date: Query(Lambda("order", Date(Select(["data", "mydate"], Var("order"))))),
},
},
],
values: [
{ binding: "date" },
{ field: ["ref"] },
],
})
Then you can get the results that you expect:
> Paginate(Range(Match(Index('orders')), Date('2020-07-11'), Date('2020-07-15')))
{
data: [
[
Date("2020-07-11"),
Ref(Collection("orders"), "278586211497411072")
],
[
Date("2020-07-12"),
Ref(Collection("orders"), "278586213229658624")
],
[
Date("2020-07-13"),
Ref(Collection("orders"), "278586215000703488")
],
[
Date("2020-07-14"),
Ref(Collection("orders"), "278586216887091712")
],
[
Date("2020-07-15"),
Ref(Collection("orders"), "278586218585784832")
]
]
}

Another alternative is to use a filter with a lambda expression to validate which values you want
Filter(
Paginate(Documents(Collection('orders'))),
Lambda('order',
And(
GTE(Select(['data', 'mydate'], Var('order')), '2020-07-09'),
LTE(Select(['data', 'mydate'], Var('order')), '2020-07-15')
)
)
)
You can update the conditions as you need
I believe this will work with the strings you have already

There are some mistakes here, first of all, you have to create documents that way:
Create(Collection("orders"), {data: {"mydate": ToDate("2020-07-10")}})
The index has to be created like this:
CreateIndex(
{
name: "orders_by_my_date",
source: Collection("orders"),
values:[{field:['data','mydate']},{field:['ref']}]
}
)
and finally, you can query your index and range:
Paginate(Range(Match('orders_by_my_date'),[Date("2020-07-09")], [Date("2020-07-15")]))
{ data:
[ [ Date("2020-07-10"),
Ref(Collection("orders"), "278532030954734085") ],
[ Date("2020-07-11"),
Ref(Collection("orders"), "278532033804763655") ],
[ Date("2020-07-12"),
Ref(Collection("orders"), "278532036737630725") ] ] }
or if you want to get the full doc:
Map(Paginate(Range(Match('orders_by_my_date'),[Date("2020-07-09")], [Date("2020-07-15")])),Lambda(['date','ref'],Get(Var('ref'))))
{ data:
[ { ref: Ref(Collection("orders"), "278532030954734085"),
ts: 1601887694290000,
data: { mydate: Date("2020-07-10") } },
{ ref: Ref(Collection("orders"), "278532033804763655"),
ts: 1601887697015000,
data: { mydate: Date("2020-07-11") } },
{ ref: Ref(Collection("orders"), "278532036737630725"),
ts: 1601887699800000,
data: { mydate: Date("2020-07-12") } } ] }

Related

Faunadb create index on child item field

I'm trying to get an index on a field of a child item in my document. The data is this:
[
{
"ref": Ref(Collection("ColA"), "111111111111111111"),
"ts":1659241462840000,
"data":{
"name":"Test a",
"members":[
{
"userId":"1",
"roles":[
"admin"
]
}
]
}
},
{
"ref": Ref(Collection("ColA"), "111111111111111112"),
"ts":1659241462840000,
"data":{
"name":"Test b",
"members":[
{
"userId":"1",
"roles":[
"admin"
]
},
{
"userId":"2",
"roles":[
"read-only"
]
}
]
}
},
{
"ref": Ref(Collection("ColA"), "111111111111111113"),
"ts":1659241462840000,
"data":{
"name":"Test c",
"members":[
{
"userId":"2",
"roles":[
"admin"
]
}
]
}
}
]
Trying to using data.members.userId as term in the index. This only gives back one result when I use the index with the filter value '1'
Then I tried to create the index as following:
CreateIndex({
name: 'spaces_member_ids',
source: {
collection: Collection("ColA"),
fields: {
members: Query(
Lambda(
"ColA",
Select(["data", "members", "userId"], Var("ColA"), '')
)
),
},
},
terms: [
{ binding: "members" },
],
values: [
{ field: "data.name" },
{ field: "ref" },
]
})
But that gives no results when I use the index with the filter value '1' Both times I expect to get two items back (Test a and Test b).
Anyone knows how to create an index that gived back all the data of ColA filtered on field 'userId' in the 'members' array?
The problem is that there is no userId field as a direct descendant of the members array.
For background, Fauna index entries can only contain scalar values. Objects are not indexed at all. For arrays, one index entry is created per scalar value in the array. If you attempt to index multiple array fields, the number of index entries produced is the Cartesian product of the items in all indexed arrays.
If you create your index like so:
CreateIndex({
name: 'spaces_member_ids',
source: Collection("ColA"),
terms: [
{ field: ["data", "members", 0, "userId"] },
],
values: [
{ field: ["data", "name"] },
{ field: "ref" },
]
})
Then you'll be able to search for userId values that appear in the first item in the members array.
If you need to create index entries for all userId values from each ColA document, then your binding approach is close, but it needs to provide an array.
CreateIndex({
name: "spaces_member_ids",
source: {
collection: Collection("ColA"),
fields: {
members: Query(
Lambda(
"ColA",
Map(
Select(["data", "members"], Var("ColA"), []),
Lambda(
"member",
Select(["userId"], Var("member"), "")
)
)
)
),
},
},
terms: [
{ binding: "members" },
],
values: [
{ field: ["data", "name"] },
{ field: "ref" },
]
})
The notable changes that I made are:
Within the binding, Map is used to iterate on the members field in the document.
Simply returning the userId field value within a Map is sufficient to return an array of userId values.
Corrected the syntax in the values definition. Fauna indexes don't process dot notation.

How to get nested documents in FaunaDB with a filter?

The following query:
Paginate(Documents(Collection("backyard"))),
Lambda(
"f",
Let(
{
backyard: Get(Var("f")),
user: Get(Select(["data", "user"], Var("backyard")))
},
{
backyard: Var("backyard"),
user: Var("user")
}
)
)
)
results to:
{
data: [
{
backyard: {
ref: Ref(Collection("backyard"), "333719283470172352"),
ts: 1654518359560000,
data: {
user: Ref(Collection("user"), "333718599460978887"),
product: "15358",
date: "2022-06-06",
counter: "1"
}
},
user: {
ref: Ref(Collection("user"), "333718599460978887"),
ts: 1654517707220000,
data: {
email: "<email>",
name: "Paolo"
}
}
},
{
backyard: {
ref: Ref(Collection("backyard"), "333747850716381384"),
ts: 1654545603400000,
data: {
user: Ref(Collection("user"), "333718599460978887"),
product: "15358",
date: "2022-06-08",
counter: "4"
}
},
user: {
ref: Ref(Collection("user"), "333718599460978887"),
ts: 1654517707220000,
data: {
email: "<email>",
name: "Paolo"
}
}
}
]
}
How can I filter backyard by date without losing the nested users?
I tried:
Map(
Paginate(Range(Match(Index("backyard_by_date")), "2022-05-08", "2022-06-08")),
Lambda(
"f",
Let(
{
backyard: Get(Var("f")),
user: Get(Select(["data", "user"], Var("backyard")))
},
{
backyard: Var("backyard"),
user: Var("user")
}
)
)
)
However, the resultset is an empty array and the following already returns an empty array:
Paginate(Range(Match(Index("backyard_by_date")), "2022-05-08", "2022-06-08"))
My index:
{
name: "backyard_by_date",
unique: false,
serialized: true,
source: "backyard"
}
Maybe I have to adjust my index? The following helped me a lot:
How to get nested documents in FaunaDB?
How to Get Data from two collection in faunadb
how to join collections in faunadb?
Your index definition is missing details. Once that gets fixed, everything else you were doing is exactly right.
In your provided index, there are no terms or values specified, which makes the backyard_by_date index a "collection" index: it only records the references of every document in the collection. In this way, it is functionally equivalent to using the Documents function but incurs additional write operations as documents are created or updated within the backyard collection.
To make your query work, you should delete your existing index and (after 60 seconds) redefine it like this:
CreateIndex({
name: "backyard_by_date",
source: Collection("backyard"),
values: [
{field: ["data", "date"]},
{field: ["ref"]}
]
})
That definition configures the index to return the date field and the reference for every document.
Let's confirm that the index returns what we expect:
> Paginate(Match(Index("backyard_by_date")))
{
data: [
[ '2022-06-06', Ref(Collection("backyard"), "333719283470172352") ],
[ '2022-06-08', Ref(Collection("backyard"), "333747850716381384") ]
]
}
Placing the date field's value first means that we can use it effectively in Range:
> Paginate(Range(Match(Index("backyard_by_date")), "2022-05-08", "2022-06-08"))
{
data: [
[ '2022-06-06', Ref(Collection("backyard"), "333719283470172352") ],
[ '2022-06-08', Ref(Collection("backyard"), "333747850716381384") ]
]
}
And to verify that Range is working as expected:
> Paginate(Range(Match(Index("backyard_by_date")), "2022-06-07", "2022-06-08"))
{
data: [
[ '2022-06-08', Ref(Collection("backyard"), "333747850716381384") ]
]
}
Now that we know the index is working correctly, your filter query needs a few adjustments:
> Map(
Paginate(
Range(Match(Index("backyard_by_date")), "2022-05-08", "2022-06-08")
),
Lambda(
["date", "ref"],
Let(
{
backyard: Get(Var("ref")),
user: Get(Select(["data", "user"], Var("backyard")))
},
{
backyard: Var("backyard"),
user: Var("user")
}
)
)
)
{
data: [
{
backyard: {
ref: Ref(Collection("backyard"), "333719283470172352"),
ts: 1657918078190000,
data: {
user: Ref(Collection("user"), "333718599460978887"),
product: '15358',
date: '2022-06-06',
counter: '1'
}
},
user: {
ref: Ref(Collection("user"), "333718599460978887"),
ts: 1657918123870000,
data: { name: 'Paolo', email: '<email>' }
}
},
{
backyard: {
ref: Ref(Collection("backyard"), "333747850716381384"),
ts: 1657918172850000,
data: {
user: Ref(Collection("user"), "333718599460978887"),
product: '15358',
date: '2022-06-08',
counter: '4'
}
},
user: {
ref: Ref(Collection("user"), "333718599460978887"),
ts: 1657918123870000,
data: { name: 'Paolo', email: '<email>' }
}
}
]
}
Since the index returns a date string and a reference, the Lambda inside the Map has to accept those values as arguments. Aside from renaming f to ref, the rest of your query is unchanged.

How do I index a value which is a #relation in a document in FaunaDB?

Is it possible to create an index of a value from a #relation type in FaunaDB? Here is the schema but I just cannot figure out how to create an index for what would be the data.testing.status value.
type TestType {
testing: Testing!
}
type Testing {
status: PaymentStatus!
testType: [TestType!] #relation
}
enum PaymentStatus {
PAID
UNPAID
}
I don't know if the enum is causing an issue? I can't find any documentation on this.
Here is the query:
Map(
Paginate(Match(Index("certificate_remittance_by_remittance"), "UNPAID")),
Lambda("ref", Get(Var("ref")))
)
and the relevant document data:
"ref": Ref(Collection("Certificate"), "302119834927235593"),
"ts": 1624382777140000,
"data": {
"remittance": Ref(Collection("Remittance"), "302119834830766601"),
}
and remittance document:
{
"ref": Ref(Collection("Remittance"), "302119834830766601"),
"ts": 1624382777140000,
"data": {
"status": "UNPAID",
"chequeNumber": "",
"remittanceOwed": 245,
"remittanceAmount": 245
}
}
Certainly.
The enum portion of your schema isn't causing an issue. If it were, you would not have been able to import the schema successfully.
With your provided schema, note that the GraphQL API has already created a relationship index for you:
Get(Index("testType_testing_by_testing"))
{
ref: Index("testType_testing_by_testing"),
ts: 1624315929200000,
active: true,
serialized: true,
name: "testType_testing_by_testing",
source: Collection("TestType"),
data: {
gql: {
ts: Time("2021-06-21T22:52:08.969203Z")
}
},
terms: [
{
field: ["data", "testing"]
}
],
unique: false,
partitions: 1
}
To add an index for the TestType collection, on the status field, you'd run:
CreateIndex({
name: "TestType_by_status",
source: Collection("TestType"),
terms: [
{ field: ["data", "status"] },
]
})

SQLite script to MongoDB

I'm a newbie on MongoDB. And I need help. I have a small project with SQLite (7 tables and 1 view). And I need to make this project on MongoDB, I'm using Studio 3T, I'm already migrate SQLite tables to MongoDB collections, but now I need to make (VIEW/TEST) for test this project. Please help how to write this SQL script with MongoDB.
SQLITE:
MongoDB:
SQLIte script I want to make with MongoDB:
CREATE VIEW rezultatas AS
SELECT p.pavadinimas AS detales_pavadinimas,
SUM(d.pagamintas_kiekis) AS pagamintas_kiekis,
SUM(z.gamybos_islaidos) AS vidutine_kaina,
STRFTIME('%m', d.pagaminimo_data) AS menuo,
STRFTIME('%Y', d.pagaminimo_data) AS metai
FROM detales d,
zinynas z,
produktas p
WHERE (z.detale_id = p._id_) AND
(d.detale_id = z.detale_id) AND
(d.pagaminimo_data >= z.data_nuo) AND
NOT EXISTS (
SELECT *
FROM zinynas
WHERE (d.detale_id = detale_id) AND
(d.pagaminimo_data >= data_nuo) AND
(z.data_nuo < data_nuo)
)
GROUP BY p.pavadinimas,
STRFTIME('%m', d.pagaminimo_data),
STRFTIME('%Y', d.pagaminimo_data)
I had to guess some things due to the lack of schemes but the basic layout should work.
We're going to use $createView with these parameters as input:
db.createView('rezultatas', 'produktas', pipeline)
Meaning our pipeline creating the view starts with the produktas collection.
The pipeline to use:
[
{ // match the documents from the zinyas collection.
$lookup:
{
from: "zinynas",
let: { produktas_id: "$_id" }, // i'm guessing its _id
pipeline: [
{ $match:
{ $expr: { $eq: [ "$detale_id", "$$produktas_id" ] }}
},
],
as: "z"
}
},
{
$unwind: "$z"
},
{ // match the documents from the detales collection. only keep the one with maximum data_nuo value.
$lookup:
{
from: "detales",
let: { z_detale_id: "$z.detale_id", z_data_nuo: "$z.data_nuo" },
pipeline: [
{ $match:
{
$and: [
{ $expr: { $eq: [ "$detale_id", "$$z_detale_id" ] }},
{ $expr: { $gte: [ "$pagaminimo_data", "$$z_data_nuo"]}}
]
}
},
{
$sort: {
data_nuo: -1
}
},
{
$limit: 1
}
],
as: "d"
}
},
{
$unwind: "$d"
},
{ // end up saving the fields we want.
$group: {
_id: { pavadinimas : "$pavadinimas", month: {$month: "$d.pagaminimo_data"}, year: {$year: "$d.pagaminimo_data"}},
pagamintas_kiekis: {$sum: "$d.pagamintas_kiekis"},
vidutine_kaina: {$sum: "$z.gamybos_islaidos"},
month: {$first: {$month: "$d.pagaminimo_data"}},
year: {$first: {$year: "$d.pagaminimo_data"}},
detales_pavadinimas: {$first: "$pavadinimas"}
}
}
]

Using lodash to retrieve values from a complex array

I have the following complex array
[
{
label: "Country1",
metrics: [
{
label: "xyz",
metric: "xyz",
value: 234184
},
{
label: "abc",
metric: "abc",
value: 145678
}
]
},
{
label: "Country2",
metrics: [
{
label: "xyz",
metric: "xyz",
value: 123456
},
{
label: "abc",
metric: "abc",
value: 456789
}
]
},
{
label: "Country3",
metrics: [
{
label: "xyz",
metric: "xyz",
value: 62389
},
{
label: "abc",
metric: "abc",
value: 4964738
}
]
}
]
I need to convert it to the following simple array wherein from the metrics sub array the values for label and value becomes a key value pair.
[
{label: “Country1”, xyz: 234184, abc: 145678},
{label: “Country2”, xyz: 123456, abc: 456789},
{label: “Country3”, xyz: 62389, abc: 4964738}
]
Can this conversion happen using lodash?
There might be a better / cleaner way to do this, but this is what I could come up with in just a few minutes.
const inputData = [
{
label: "Country1",
metrics: [
{
label: "xyz",
metric: "xyz",
value: 234184
},
{
label: "abc",
metric: "abc",
value: 145678
}
]
},
{
label: "Country2",
metrics: [
{
label: "xyz",
metric: "xyz",
value: 123456
},
{
label: "abc",
metric: "abc",
value: 456789
}
]
},
{
label: "Country3",
metrics: [
{
label: "xyz",
metric: "xyz",
value: 62389
},
{
label: "abc",
metric: "abc",
value: 4964738
}
]
}
];
const newData = _.map(inputData, first => {
const additional = _.map(first.metrics, metric => [metric.label, metric.value]);
return _.assign({}, _.fromPairs(additional), {
label: first.label
});
});
console.log(newData);
<script src="https://cdnjs.cloudflare.com/ajax/libs/lodash.js/4.17.10/lodash.js"></script>
Here's a solution that doesn't require lodash, you simply have to take advantage of using Array#map to transform each item in the top-level array, and then use Array#reduce to transform the object and get the rest of the properties you need. The methodology used below takes advantage of the following ES6 features:
Destructuring assignment to extract the properties from the Array#map and Array#reduce callbacks.
Spread Syntax to combine properties of objects from different sources.
Computed property names for a dynamic property name in an object.
var result = data.map(({ label, metrics }) =>
metrics.reduce(
(result, { metric, value }) => ({ ...result, [metric]: value }),
{ label }
)
);
var data = [{label:"Country1",metrics:[{label:"xyz",metric:"xyz",value:234184},{label:"abc",metric:"abc",value:145678}]},{label:"Country2",metrics:[{label:"xyz",metric:"xyz",value:123456},{label:"abc",metric:"abc",value:456789}]},{label:"Country3",metrics:[{label:"xyz",metric:"xyz",value:62389},{label:"abc",metric:"abc",value:4964738}]}];
var result = data.map(({ label, metrics }) =>
metrics.reduce(
(result, { metric, value }) => ({ ...result, [metric]: value }),
{ label }
)
);
console.log(result);
.as-console-wrapper{min-height:100%;top:0}