How to select all documents where one date field is after another date field in FaunaDB - faunadb

I have a very simple collection with documents that look like this:
{
...
latestEdit: Time(...),
lastPublished: Time(...)
}
I would like to query all documents that have a latestEdit time that's after lastPublished time.
I find FQL to be very different to SQL and I'm finding the transition quite hard.
Any help much appreciated.

Fauna's FQL is not declarative, so you have to construct the appropriate indexes and queries to help you solve problems like this.
Fauna indexes have a feature called "bindings", which allow you to provide a user-defined function that can compute a value based on document values. The binding lets us index the computed value by itself (rather than having to index on latestEdit or lastPublished). Here's what that might look like:
CreateIndex({
name: "edit_after_published",
source: {
collection: Collection("test"),
fields: {
needsPublish: Query(
Lambda(
"doc",
Let(
{
latestEdit: Select(["data", "latestEdit"], Var("doc")),
lastPublished: Select(["data", "lastPublished"], Var("doc")),
},
If(
GT(Var("latestEdit"), Var("lastPublished")),
true,
false
)
)
)
)
}
},
terms: [ { binding: "needsPublish" } ]
})
You can see that we define a binding called needsPublish. The binding uses Let to define named values for the two document fields that we want to compare, and then the If statement checks to see if the latestEdit value is greather than lastPublished value: when it is we return true, otherwise we return false. Then, the binding is used in the index's terms definition, which defines the fields that we want to be able to search on.
I created sample documents in a collection called test, like so:
> Create(Collection("test"), { data: { name: "first", latestEdit: Now(), lastPublished: TimeSubtract(Now(), 1, "day") }})
{
ref: Ref(Collection("test"), "306026106743423488"),
ts: 1628108088190000,
data: {
name: 'first',
latestEdit: Time("2021-08-04T20:14:48.121Z"),
lastPublished: Time("2021-08-03T20:14:48.121Z")
}
}
> Create(Collection("test"), { data: { name: "second", lastPublished: Now(), latestEdit: TimeSubtract(Now(), 1, "day") }})
{
ref: Ref(Collection("test"), "306026150784664064"),
ts: 1628108130150000,
data: {
name: 'second',
lastPublished: Time("2021-08-04T20:15:30.148Z"),
latestEdit: Time("2021-08-03T20:15:30.148Z")
}
}
The first document subtracts one day from lastPublished and the second document subtracts one day from latestEdit, to test both conditions of the binding.
Then we can query for all documents where needsPublish results in true:
> Map(Paginate(Match(Index("edit_after_published"), true)), Lambda("X", Get(Var("X"))))
{
data: [
{
ref: Ref(Collection("test"), "306026106743423488"),
ts: 1628108088190000,
data: {
name: 'first',
latestEdit: Time("2021-08-04T20:14:48.121Z"),
lastPublished: Time("2021-08-03T20:14:48.121Z")
}
}
]
}
And we can also query for all documents where needsPublish is false:
> Map(Paginate(Match(Index("edit_after_published"), false)), Lambda("X", Get(Var("X"))))
{
data: [
{
ref: Ref(Collection("test"), "306026150784664064"),
ts: 1628108130150000,
data: {
name: 'second',
lastPublished: Time("2021-08-04T20:15:30.148Z"),
latestEdit: Time("2021-08-03T20:15:30.148Z")
}
}
]
}

Related

Indexes: Search by Boolean?

I'm having some trouble with FaunaDB Indexes. FQL is quite powerful but the docs seem to be limited (for now) to only a few examples/use cases. (Searching by String)
I have a collection of Orders, with a few fields: status, id, client, material and date.
My goal is to search/filter for orders depending on their Status, OPEN OR CLOSED (Boolean true/false).
Here is the Index I created:
CreateIndex({
name: "orders_all_by_open_asc",
unique: false,
serialized: true,
source: Collection("orders"),
terms: [{ field: ["data", "status"] }],
values: [
{ field: ["data", "unique_id"] },
{ field: ["data", "client"] },
{ field: ["data", "material"] },
{ field: ["data", "date"] }
]
}
So with this Index, I want to specify either TRUE or FALSE and get all corresponding orders, including their data (fields).
I'm having two problems:
When I pass TRUE OR FALSE using the Javascript Driver, nothing is returned :( Is it possible to search by Booleans at all, or only by String/Number?
Here is my Query (in FQL, using the Shell):
Match(Index("orders_all_by_open_asc"), true)
And unfortunately, nothing is returned. I'm probably doing this wrong.
Second (slightly unrelated) question. When I create an Index and specify a bunch of Values, it seems the data returned is in Array format, with only the values, not the Fields. An example:
[
1001,
"client1",
"concrete",
"2021-04-13T00:00:00.000Z",
],
[
1002,
"client2",
"wood",
"2021-04-13T00:00:00.000Z",
]
This format is bad for me, because my front-end expects receiving an Object with the Fields as a key and the Values as properties. Example:
data:
{
unique_id : 1001,
client : "client1",
material : "concrete",
date: "2021-04-13T00:00:00.000Z"
},
{
unique_id : 1002,
client : "client2",
material : "wood",
date: "2021-04-13T00:00:00.000Z"
},
etc..
Is there any way to get the Field as well as the Value when using Index values, or will it always return an Array (and not an object)?
Could I use a Lambda or something for this?
I do have another Query that uses Map and Lambda to good effect, and returns the entire document, including the Ref and Data fields:
Map(
Paginate(
Match(Index("orders_by_date"), date),
),
Lambda('item', Get(Var('item')))
)
This works very nicely but unfortunately, it also performs one Get request per Document returned and that seems very inefficient.
This new Index I'm wanting to build, to filter by Order Status, will be used to return hundreds of Orders, hundreds of times a day. So I'm trying to keep it as efficient as possible, but if it can only return an Array it won't be useful.
Thanks in advance!! Indexes are great but hard to grasp, so any insight will be appreciated.
You didn't show us exactly what you have done, so here's an example that shows that filtering on boolean values does work using the index you created as-is:
> CreateCollection({ name: "orders" })
{
ref: Collection("orders"),
ts: 1618350087320000,
history_days: 30,
name: 'orders'
}
> Create(Collection("orders"), { data: {
unique_id: 1,
client: "me",
material: "stone",
date: Now(),
status: true
}})
{
ref: Ref(Collection("orders"), "295794155241603584"),
ts: 1618350138800000,
data: {
unique_id: 1,
client: 'me',
material: 'stone',
date: Time("2021-04-13T21:42:18.784Z"),
status: true
}
}
> Create(Collection("orders"), { data: {
unique_id: 2,
client: "you",
material: "muslin",
date: Now(),
status: false
}})
{
ref: Ref(Collection("orders"), "295794180038328832"),
ts: 1618350162440000,
data: {
unique_id: 2,
client: 'you',
material: 'muslin',
date: Time("2021-04-13T21:42:42.437Z"),
status: false
}
}
> CreateIndex({
name: "orders_all_by_open_asc",
unique: false,
serialized: true,
source: Collection("orders"),
terms: [{ field: ["data", "status"] }],
values: [
{ field: ["data", "unique_id"] },
{ field: ["data", "client"] },
{ field: ["data", "material"] },
{ field: ["data", "date"] }
]
})
{
ref: Index("orders_all_by_open_asc"),
ts: 1618350185940000,
active: true,
serialized: true,
name: 'orders_all_by_open_asc',
unique: false,
source: Collection("orders"),
terms: [ { field: [ 'data', 'status' ] } ],
values: [
{ field: [ 'data', 'unique_id' ] },
{ field: [ 'data', 'client' ] },
{ field: [ 'data', 'material' ] },
{ field: [ 'data', 'date' ] }
],
partitions: 1
}
> Paginate(Match(Index("orders_all_by_open_asc"), true))
{ data: [ [ 1, 'me', 'stone', Time("2021-04-13T21:42:18.784Z") ] ] }
> Paginate(Match(Index("orders_all_by_open_asc"), false))
{ data: [ [ 2, 'you', 'muslin', Time("2021-04-13T21:42:42.437Z") ] ] }
It's a little more work, but you can compose whatever return format that you like:
> Map(
Paginate(Match(Index("orders_all_by_open_asc"), false)),
Lambda(
["unique_id", "client", "material", "date"],
{
unique_id: Var("unique_id"),
client: Var("client"),
material: Var("material"),
date: Var("date"),
}
)
)
{
data: [
{
unique_id: 2,
client: 'you',
material: 'muslin',
date: Time("2021-04-13T21:42:42.437Z")
}
]
}
It's still an array of results, but each result is now an object with the appropriate field names.
Not too familiar with FQL, but I am somewhat familiar with SQL languages. Essentially, database languages usually treat all of your values as strings until they don't need to anymore. Instead, your query should use the string definition that FQL is expecting. I believe it should be OPEN or CLOSED in your case. You can simply have an if statement in java to determine whether to search for "OPEN" or "CLOSED".
To answer your second question, I don't know for FQL, but if that is what is returned, then your approach with a lamda seems to be fine. Not much else you can do about it from your end other than hope that you get a different way to get entries in API form somewhere in the future. At the end of the day, an O(n) operation in this context is not too bad, and only having to return a hundred or so orders shouldn't be the most painful thing in the world.
If you are truly worried about this, you can break up the request into portions, so you return only the first 100, then when frontend wants the next set, you send the next 100. You can cache the results too to make it very fast from the front-end perspective.
Another suggestion, maybe I am wrong and failed at searching the docs, but I will post anyway just in case it's helpful.
My index was failing to return objects, example data here is the client field:
"data": {
"status": "LIVRAISON",
"open": true,
"unique_id": 1001,
"client": {
"name": "TEST1",
"contact_name": "Bob",
"email": "bob#client.com",
"phone": "555-555-5555"
Here, the client field returned as null even though it was specified in the Index.
From reading the docs, here: https://docs.fauna.com/fauna/current/api/fql/indexes?lang=javascript#value
In the Value Objects section, I was able to understand that for Objects, the Index Field must be defined as an Array, one for each Object key. Example for my data:
{ field: ['data', 'client', 'name'] },
{ field: ['data', 'client', 'contact_name'] },
{ field: ['data', 'client', 'email'] },
{ field: ['data', 'client', 'phone'] },
This was slightly confusing, because my beginner brain expected that defining the 'client' field would simply return the entire object, like so:
{ field: ['data', 'client'] },
The only part about this in the docs was this sentence: The field ["data", "address", "street"] refers to the street field contained in an address object within the document’s data object.
This is enough information, but maybe it would deserve its own section, with a longer example? Of course the simple sentence works, but with a sub-section called 'Adding Objects to Fields' or something, this would make it extra-clear.
Hoping my moments of confusion will help out. Loving FaunaDB so far, keep up the great work :)

how to join collections in faunadb?

I want to get nested ref's value within the query I'm executing, but by default response is returning the ref of other collection. consider this minimum example; here are user and coin models, within "users" and "coins" collections
user { // ref: 123456
name: foo
location: bar
}
coin { // ref: 124457
amount: 5457
awardedTo: Ref(Collection("users"), "123456")
}
when I run this query
q.Get(q.Ref(q.Collection("coins"), "124457"))
the response is something like this:
{
data: {
amount: 5457,
awardedTo: #ref: {id: "123456", collection: {…}}
},
ref: #ref: {id: "124457", collection: {…}},
ts: 1622547855525255
}
But how is it possible to get nested user's value in the same query to have a response like this:
{
data: {
amount: 5457,
awardedTo: {
name: foo,
location: bar
}
},
ref: #ref: {id: "124457", collection: {…}},
ts: 1622547855525255
}
I have read Join's documentation but it wasn't helpful in this case, and also tried this way, but it didn't work either:
q.Let({
coin: q.Get(q.Ref(q.Collection("coins"), '124457'))
},
q.Union(
q.Get(q.Select(["data","awaredTo"], q.Var("coin"))),
q.Var("coins")
)
)
you can use this FQL:
Let(
{
coin: Select(['data'],Get(Ref(Collection("coin"), "1"))),
user: Select(['data'],Get(Select(['awardedTo'],Var('coin'))))
},
Merge(Var('coin'),{awardedTo:Var('user')})
)
It retrieves data from coin, extracts the user ref and merge all together.
Luigi

useGetList fetched data sorted by id ignoring sort option

The problem is that data fetched by useGetList is sorted by ids of the objects and ignores the order by other keys given by my backend. This is my call example (try to order by name):
const { data, loading } = useGetList(
'my-controller-path',
{ page: 1, perPage: 10000 },
{ field: 'name', order: 'ASC' },
{}
)
console.log(data)
When I print data after this call, data is sorted by 'id' and not by 'name'. It's not a problem with the backend because I hardcoded returning data to eliminate this from equation. Data provided from backend:
data: [
{ id: 2, name: 'Ana'},
{ id: 3, name: 'Bea'},
{ id: 1, name: 'Cena'}
]
And when I print data on console after using useGetList:
[
{id: 1, name: "Cena"},
{id: 2, name: "Ana"},
{id: 3, name: "Bea"}
]
Any idea what is causing the problem? I read the source code of useGetList and don't quite understand if the problem lays there. The link to source code.
How does your API works ? In my case, I needed to map the sort properties like so:
getList: async (
resource,
{ pagination: { page, perPage }, sort: { field, order }, filter }
) => {
const query = {
...filter,
sort: order.toLowerCase(),
order: field,
page: JSON.stringify(page),
limit: JSON.stringify(perPage),
};
...
}
in my custom data provider to match my API sort and order properties.

How to search for/select by included entity but include all related entities into result set

In my application, I am using sequelize ORM. There are several entities: A Tool can have Tags and Categories.
Now I want to search for all Tools, that have a specific Tag, but I want to include all relating Tags of that tool (not just the specific one). If I now place a where statement into the include, only specified Tags are included into the result set (see [2]). I tried to limit the Tags in the outer where statement (see [1]), but this does not help either.
Example
Tool A has Tags t1, t2 and t3. Now I want to search all Tools that have the Tag t3, but the result set shall contain all three tags.
Expected result:
Tool A
\
- Tag t1
- Tag t2
- Tag t3
db.Tool.scope('published').findAll({
where: { '$tool.tag.name$': filter.tag }, // [1] Does not work
include: [
{
model: db.User,
attributes: ['id', 'username']
},
{
model: db.Tag,
attributes: ['name'],
through: { attributes: [] },
// [2] Would limit the result specified tag
// where: {
// name: {
// [Op.and]: filter.tag
// }
// }
},
{
model: db.Category,
attributes: ['id', 'name', 'views'],
through: { attributes: ['relevance'] },
where: {
id: {
[Op.and]: filter.category
}
}
}
],
where: {
title: {
[Op.like]: `%${filter.term}%`,
}
},
attributes: ['id', 'title', 'description', 'slug', 'docLink', 'vendor', 'vendorLink', 'views', 'status', 'createdAt'],
order: [['title', 'ASC'], [db.Tag, 'name', 'ASC']]
})
I know I could perform this by performing a select via the Tag in the first place (db.Tag.findAll() instead of db.Tool.findAll(); I've already done this elsewhere in my project), but at the same time I also want to be able to filter by another entity (Category) the same way. So the Tool.findAll() should be the starting point.
Any help appreciated!
First off, you have two where clauses in your top-level query:
where: { '$tool.tag.name$': filter.tag }, // [1] Does not work
// ...
where: {
title: {
[Op.like]: `%${filter.term}%`,
}
},
I think your best approach is going to be with a literal subquery in the WHERE clause. Basically we want to find the ids of all of the tools that have the right tag and that contain the filter.term.
The subquery part for the WHERE looks something like...
SELECT ToolId FROM ToolTags WHERE TagId='t2';
Inspired by the subquery solution from this post Sequelize - subquery in where clause
// assuming your join table is named 'ToolTags' in the database--we need the real table name not the model name
const tempSQL = sequelize.dialect.QueryGenerator.selectQuery('ToolTags',{
attributes: ['ToolId'],
where: {
TagId: filter.tag
}})
.slice(0,-1); // to remove the ';' from the end of the SQL
db.Tool.scope('published').findAll({
where: {
title: {
[Op.like]: `%${filter.term}%`,
},
id: {
[Op.In]: sequelize.literal(`(${tempSQL})`)
}
},
include: [
{
model: db.User,
attributes: ['id', 'username']
},
{
model: db.Tag,
attributes: ['name'],
through: { attributes: [] },
},
// {
// model: db.Category,
// attributes: ['id', 'name', 'views'],
// through: { attributes: ['relevance'] },
// where: {
// id: {
// [Op.and]: filter.category
// }
// }
// }
],
attributes: ['id', 'title', 'description', 'slug', 'docLink', 'vendor', 'vendorLink', 'views', 'status', 'createdAt'],
order: [['title', 'ASC'], [db.Tag, 'name', 'ASC']]
})
I commented out your category join for now. I think you should try to isolate the solution for the tags before adding more onto the query.

Filter an array of dates (datetime) with GROQ (sanity) (React App)

I have a list of movies that could be shown more than once. I decided to provide a user with an option to select multiple dates for a single movie (sanity studio interface).
The schema for movies is as follows:
export default {
name: 'movie',
title: 'Movie',
type: 'document',
fields: [
{
name: 'title',
title: 'Title',
type: 'string'
},
{
name: 'dates',
title: 'Dates',
type: 'array',
of: [
{
type: 'datetime',
options: {
dateFormat: 'YYYY-MM-DD',
timeFormat: 'HH:mm',
timeStep: 15,
calendarTodayLabel: 'Today'
}
}
]
},
{
name: 'poster',
title: 'Poster',
type: 'image',
options: {
hotspot: true
}
},
{
name: 'body',
title: 'Body',
type: 'blockContent'
}
],
preview: {
select: {
title: 'title',
date: 'date',
media: 'poster'
}
}
}
Current query:
const query = groq`*[_type == "movie"]{
title,
dates,
poster,
body
}`
I need to filter the movie that has today's date in the dates array with GROQ
Maybe I'm overcomplicating this and someone will come up with a better way.
The idea is to avoid duplicates in the database (1 movie can be shown 3-6 times). That's the only reason I used an array
The solution for this should be:
const query = '*[_type == "movie" && dates match $today]{title, dates, poster, body}'
const today = new Date().toISOString().split('T')[0]
client.fetch(query, {today}).then(result => {
// movies which are showing today
})
However, there is currently a bug in the string tokenizer which cripples date string matching. In the meantime, I'm afraid your only option is to fetch all movies and filter client side. We're hoping to get this fixed as soon as possible.