Related
I'm trying to get an index on a field of a child item in my document. The data is this:
[
{
"ref": Ref(Collection("ColA"), "111111111111111111"),
"ts":1659241462840000,
"data":{
"name":"Test a",
"members":[
{
"userId":"1",
"roles":[
"admin"
]
}
]
}
},
{
"ref": Ref(Collection("ColA"), "111111111111111112"),
"ts":1659241462840000,
"data":{
"name":"Test b",
"members":[
{
"userId":"1",
"roles":[
"admin"
]
},
{
"userId":"2",
"roles":[
"read-only"
]
}
]
}
},
{
"ref": Ref(Collection("ColA"), "111111111111111113"),
"ts":1659241462840000,
"data":{
"name":"Test c",
"members":[
{
"userId":"2",
"roles":[
"admin"
]
}
]
}
}
]
Trying to using data.members.userId as term in the index. This only gives back one result when I use the index with the filter value '1'
Then I tried to create the index as following:
CreateIndex({
name: 'spaces_member_ids',
source: {
collection: Collection("ColA"),
fields: {
members: Query(
Lambda(
"ColA",
Select(["data", "members", "userId"], Var("ColA"), '')
)
),
},
},
terms: [
{ binding: "members" },
],
values: [
{ field: "data.name" },
{ field: "ref" },
]
})
But that gives no results when I use the index with the filter value '1' Both times I expect to get two items back (Test a and Test b).
Anyone knows how to create an index that gived back all the data of ColA filtered on field 'userId' in the 'members' array?
The problem is that there is no userId field as a direct descendant of the members array.
For background, Fauna index entries can only contain scalar values. Objects are not indexed at all. For arrays, one index entry is created per scalar value in the array. If you attempt to index multiple array fields, the number of index entries produced is the Cartesian product of the items in all indexed arrays.
If you create your index like so:
CreateIndex({
name: 'spaces_member_ids',
source: Collection("ColA"),
terms: [
{ field: ["data", "members", 0, "userId"] },
],
values: [
{ field: ["data", "name"] },
{ field: "ref" },
]
})
Then you'll be able to search for userId values that appear in the first item in the members array.
If you need to create index entries for all userId values from each ColA document, then your binding approach is close, but it needs to provide an array.
CreateIndex({
name: "spaces_member_ids",
source: {
collection: Collection("ColA"),
fields: {
members: Query(
Lambda(
"ColA",
Map(
Select(["data", "members"], Var("ColA"), []),
Lambda(
"member",
Select(["userId"], Var("member"), "")
)
)
)
),
},
},
terms: [
{ binding: "members" },
],
values: [
{ field: ["data", "name"] },
{ field: "ref" },
]
})
The notable changes that I made are:
Within the binding, Map is used to iterate on the members field in the document.
Simply returning the userId field value within a Map is sufficient to return an array of userId values.
Corrected the syntax in the values definition. Fauna indexes don't process dot notation.
The following query:
Paginate(Documents(Collection("backyard"))),
Lambda(
"f",
Let(
{
backyard: Get(Var("f")),
user: Get(Select(["data", "user"], Var("backyard")))
},
{
backyard: Var("backyard"),
user: Var("user")
}
)
)
)
results to:
{
data: [
{
backyard: {
ref: Ref(Collection("backyard"), "333719283470172352"),
ts: 1654518359560000,
data: {
user: Ref(Collection("user"), "333718599460978887"),
product: "15358",
date: "2022-06-06",
counter: "1"
}
},
user: {
ref: Ref(Collection("user"), "333718599460978887"),
ts: 1654517707220000,
data: {
email: "<email>",
name: "Paolo"
}
}
},
{
backyard: {
ref: Ref(Collection("backyard"), "333747850716381384"),
ts: 1654545603400000,
data: {
user: Ref(Collection("user"), "333718599460978887"),
product: "15358",
date: "2022-06-08",
counter: "4"
}
},
user: {
ref: Ref(Collection("user"), "333718599460978887"),
ts: 1654517707220000,
data: {
email: "<email>",
name: "Paolo"
}
}
}
]
}
How can I filter backyard by date without losing the nested users?
I tried:
Map(
Paginate(Range(Match(Index("backyard_by_date")), "2022-05-08", "2022-06-08")),
Lambda(
"f",
Let(
{
backyard: Get(Var("f")),
user: Get(Select(["data", "user"], Var("backyard")))
},
{
backyard: Var("backyard"),
user: Var("user")
}
)
)
)
However, the resultset is an empty array and the following already returns an empty array:
Paginate(Range(Match(Index("backyard_by_date")), "2022-05-08", "2022-06-08"))
My index:
{
name: "backyard_by_date",
unique: false,
serialized: true,
source: "backyard"
}
Maybe I have to adjust my index? The following helped me a lot:
How to get nested documents in FaunaDB?
How to Get Data from two collection in faunadb
how to join collections in faunadb?
Your index definition is missing details. Once that gets fixed, everything else you were doing is exactly right.
In your provided index, there are no terms or values specified, which makes the backyard_by_date index a "collection" index: it only records the references of every document in the collection. In this way, it is functionally equivalent to using the Documents function but incurs additional write operations as documents are created or updated within the backyard collection.
To make your query work, you should delete your existing index and (after 60 seconds) redefine it like this:
CreateIndex({
name: "backyard_by_date",
source: Collection("backyard"),
values: [
{field: ["data", "date"]},
{field: ["ref"]}
]
})
That definition configures the index to return the date field and the reference for every document.
Let's confirm that the index returns what we expect:
> Paginate(Match(Index("backyard_by_date")))
{
data: [
[ '2022-06-06', Ref(Collection("backyard"), "333719283470172352") ],
[ '2022-06-08', Ref(Collection("backyard"), "333747850716381384") ]
]
}
Placing the date field's value first means that we can use it effectively in Range:
> Paginate(Range(Match(Index("backyard_by_date")), "2022-05-08", "2022-06-08"))
{
data: [
[ '2022-06-06', Ref(Collection("backyard"), "333719283470172352") ],
[ '2022-06-08', Ref(Collection("backyard"), "333747850716381384") ]
]
}
And to verify that Range is working as expected:
> Paginate(Range(Match(Index("backyard_by_date")), "2022-06-07", "2022-06-08"))
{
data: [
[ '2022-06-08', Ref(Collection("backyard"), "333747850716381384") ]
]
}
Now that we know the index is working correctly, your filter query needs a few adjustments:
> Map(
Paginate(
Range(Match(Index("backyard_by_date")), "2022-05-08", "2022-06-08")
),
Lambda(
["date", "ref"],
Let(
{
backyard: Get(Var("ref")),
user: Get(Select(["data", "user"], Var("backyard")))
},
{
backyard: Var("backyard"),
user: Var("user")
}
)
)
)
{
data: [
{
backyard: {
ref: Ref(Collection("backyard"), "333719283470172352"),
ts: 1657918078190000,
data: {
user: Ref(Collection("user"), "333718599460978887"),
product: '15358',
date: '2022-06-06',
counter: '1'
}
},
user: {
ref: Ref(Collection("user"), "333718599460978887"),
ts: 1657918123870000,
data: { name: 'Paolo', email: '<email>' }
}
},
{
backyard: {
ref: Ref(Collection("backyard"), "333747850716381384"),
ts: 1657918172850000,
data: {
user: Ref(Collection("user"), "333718599460978887"),
product: '15358',
date: '2022-06-08',
counter: '4'
}
},
user: {
ref: Ref(Collection("user"), "333718599460978887"),
ts: 1657918123870000,
data: { name: 'Paolo', email: '<email>' }
}
}
]
}
Since the index returns a date string and a reference, the Lambda inside the Map has to accept those values as arguments. Aside from renaming f to ref, the rest of your query is unchanged.
I'm having some trouble with FaunaDB Indexes. FQL is quite powerful but the docs seem to be limited (for now) to only a few examples/use cases. (Searching by String)
I have a collection of Orders, with a few fields: status, id, client, material and date.
My goal is to search/filter for orders depending on their Status, OPEN OR CLOSED (Boolean true/false).
Here is the Index I created:
CreateIndex({
name: "orders_all_by_open_asc",
unique: false,
serialized: true,
source: Collection("orders"),
terms: [{ field: ["data", "status"] }],
values: [
{ field: ["data", "unique_id"] },
{ field: ["data", "client"] },
{ field: ["data", "material"] },
{ field: ["data", "date"] }
]
}
So with this Index, I want to specify either TRUE or FALSE and get all corresponding orders, including their data (fields).
I'm having two problems:
When I pass TRUE OR FALSE using the Javascript Driver, nothing is returned :( Is it possible to search by Booleans at all, or only by String/Number?
Here is my Query (in FQL, using the Shell):
Match(Index("orders_all_by_open_asc"), true)
And unfortunately, nothing is returned. I'm probably doing this wrong.
Second (slightly unrelated) question. When I create an Index and specify a bunch of Values, it seems the data returned is in Array format, with only the values, not the Fields. An example:
[
1001,
"client1",
"concrete",
"2021-04-13T00:00:00.000Z",
],
[
1002,
"client2",
"wood",
"2021-04-13T00:00:00.000Z",
]
This format is bad for me, because my front-end expects receiving an Object with the Fields as a key and the Values as properties. Example:
data:
{
unique_id : 1001,
client : "client1",
material : "concrete",
date: "2021-04-13T00:00:00.000Z"
},
{
unique_id : 1002,
client : "client2",
material : "wood",
date: "2021-04-13T00:00:00.000Z"
},
etc..
Is there any way to get the Field as well as the Value when using Index values, or will it always return an Array (and not an object)?
Could I use a Lambda or something for this?
I do have another Query that uses Map and Lambda to good effect, and returns the entire document, including the Ref and Data fields:
Map(
Paginate(
Match(Index("orders_by_date"), date),
),
Lambda('item', Get(Var('item')))
)
This works very nicely but unfortunately, it also performs one Get request per Document returned and that seems very inefficient.
This new Index I'm wanting to build, to filter by Order Status, will be used to return hundreds of Orders, hundreds of times a day. So I'm trying to keep it as efficient as possible, but if it can only return an Array it won't be useful.
Thanks in advance!! Indexes are great but hard to grasp, so any insight will be appreciated.
You didn't show us exactly what you have done, so here's an example that shows that filtering on boolean values does work using the index you created as-is:
> CreateCollection({ name: "orders" })
{
ref: Collection("orders"),
ts: 1618350087320000,
history_days: 30,
name: 'orders'
}
> Create(Collection("orders"), { data: {
unique_id: 1,
client: "me",
material: "stone",
date: Now(),
status: true
}})
{
ref: Ref(Collection("orders"), "295794155241603584"),
ts: 1618350138800000,
data: {
unique_id: 1,
client: 'me',
material: 'stone',
date: Time("2021-04-13T21:42:18.784Z"),
status: true
}
}
> Create(Collection("orders"), { data: {
unique_id: 2,
client: "you",
material: "muslin",
date: Now(),
status: false
}})
{
ref: Ref(Collection("orders"), "295794180038328832"),
ts: 1618350162440000,
data: {
unique_id: 2,
client: 'you',
material: 'muslin',
date: Time("2021-04-13T21:42:42.437Z"),
status: false
}
}
> CreateIndex({
name: "orders_all_by_open_asc",
unique: false,
serialized: true,
source: Collection("orders"),
terms: [{ field: ["data", "status"] }],
values: [
{ field: ["data", "unique_id"] },
{ field: ["data", "client"] },
{ field: ["data", "material"] },
{ field: ["data", "date"] }
]
})
{
ref: Index("orders_all_by_open_asc"),
ts: 1618350185940000,
active: true,
serialized: true,
name: 'orders_all_by_open_asc',
unique: false,
source: Collection("orders"),
terms: [ { field: [ 'data', 'status' ] } ],
values: [
{ field: [ 'data', 'unique_id' ] },
{ field: [ 'data', 'client' ] },
{ field: [ 'data', 'material' ] },
{ field: [ 'data', 'date' ] }
],
partitions: 1
}
> Paginate(Match(Index("orders_all_by_open_asc"), true))
{ data: [ [ 1, 'me', 'stone', Time("2021-04-13T21:42:18.784Z") ] ] }
> Paginate(Match(Index("orders_all_by_open_asc"), false))
{ data: [ [ 2, 'you', 'muslin', Time("2021-04-13T21:42:42.437Z") ] ] }
It's a little more work, but you can compose whatever return format that you like:
> Map(
Paginate(Match(Index("orders_all_by_open_asc"), false)),
Lambda(
["unique_id", "client", "material", "date"],
{
unique_id: Var("unique_id"),
client: Var("client"),
material: Var("material"),
date: Var("date"),
}
)
)
{
data: [
{
unique_id: 2,
client: 'you',
material: 'muslin',
date: Time("2021-04-13T21:42:42.437Z")
}
]
}
It's still an array of results, but each result is now an object with the appropriate field names.
Not too familiar with FQL, but I am somewhat familiar with SQL languages. Essentially, database languages usually treat all of your values as strings until they don't need to anymore. Instead, your query should use the string definition that FQL is expecting. I believe it should be OPEN or CLOSED in your case. You can simply have an if statement in java to determine whether to search for "OPEN" or "CLOSED".
To answer your second question, I don't know for FQL, but if that is what is returned, then your approach with a lamda seems to be fine. Not much else you can do about it from your end other than hope that you get a different way to get entries in API form somewhere in the future. At the end of the day, an O(n) operation in this context is not too bad, and only having to return a hundred or so orders shouldn't be the most painful thing in the world.
If you are truly worried about this, you can break up the request into portions, so you return only the first 100, then when frontend wants the next set, you send the next 100. You can cache the results too to make it very fast from the front-end perspective.
Another suggestion, maybe I am wrong and failed at searching the docs, but I will post anyway just in case it's helpful.
My index was failing to return objects, example data here is the client field:
"data": {
"status": "LIVRAISON",
"open": true,
"unique_id": 1001,
"client": {
"name": "TEST1",
"contact_name": "Bob",
"email": "bob#client.com",
"phone": "555-555-5555"
Here, the client field returned as null even though it was specified in the Index.
From reading the docs, here: https://docs.fauna.com/fauna/current/api/fql/indexes?lang=javascript#value
In the Value Objects section, I was able to understand that for Objects, the Index Field must be defined as an Array, one for each Object key. Example for my data:
{ field: ['data', 'client', 'name'] },
{ field: ['data', 'client', 'contact_name'] },
{ field: ['data', 'client', 'email'] },
{ field: ['data', 'client', 'phone'] },
This was slightly confusing, because my beginner brain expected that defining the 'client' field would simply return the entire object, like so:
{ field: ['data', 'client'] },
The only part about this in the docs was this sentence: The field ["data", "address", "street"] refers to the street field contained in an address object within the document’s data object.
This is enough information, but maybe it would deserve its own section, with a longer example? Of course the simple sentence works, but with a sub-section called 'Adding Objects to Fields' or something, this would make it extra-clear.
Hoping my moments of confusion will help out. Loving FaunaDB so far, keep up the great work :)
I am struggling to get an Index by Date to work with a Range.
I have this collection called orders:
CreateCollection({name: "orders"})
And I have these sample entries, with one attribute called mydate. As you see it is just a string. And I do need to create the date as a string since in my DB we already have around 12K records with dates like that so I cant just start using the Date() to create them.
Create(Collection("orders"), {data: {"mydate": "2020-07-10"}})
Create(Collection("orders"), {data: {"mydate": "2020-07-11"}})
Create(Collection("orders"), {data: {"mydate": "2020-07-12"}})
I have created this index that computes the date to and actual Date object
CreateIndex({
name: "orders_by_my_date",
source: [
{
collection: Collection("orders"),
fields: {
date: Query(Lambda("order", Date(Select(["data", "mydate"], Var("order"))))),
},
},
],
terms: [
{
binding: "date",
},
],
});
If I try to fetch a single date the index works.
// this works
Paginate(
Match(Index("orders_by_my_date"), Date("2020-07-10"))
);
// ---
{
data: [Ref(Collection("orders"), "278496072502870530")]
}
But when I try to get a Range it never finds data.
// This does NOT work :(
Paginate(
Range(Match(Index("orders_by_my_date")), Date("2020-07-09"), Date("2020-07-15"))
);
// ---
{
data: []
}
Why the index does not work with a Range?
Range operates on the values of an index, not on the terms.
See: https://docs.fauna.com/fauna/current/api/fql/functions/range?lang=javascript
You need to change your index definition to:
CreateIndex({
name: "orders_by_my_date",
source: [
{
collection: Collection("orders"),
fields: {
date: Query(Lambda("order", Date(Select(["data", "mydate"], Var("order"))))),
},
},
],
values: [
{ binding: "date" },
{ field: ["ref"] },
],
})
Then you can get the results that you expect:
> Paginate(Range(Match(Index('orders')), Date('2020-07-11'), Date('2020-07-15')))
{
data: [
[
Date("2020-07-11"),
Ref(Collection("orders"), "278586211497411072")
],
[
Date("2020-07-12"),
Ref(Collection("orders"), "278586213229658624")
],
[
Date("2020-07-13"),
Ref(Collection("orders"), "278586215000703488")
],
[
Date("2020-07-14"),
Ref(Collection("orders"), "278586216887091712")
],
[
Date("2020-07-15"),
Ref(Collection("orders"), "278586218585784832")
]
]
}
Another alternative is to use a filter with a lambda expression to validate which values you want
Filter(
Paginate(Documents(Collection('orders'))),
Lambda('order',
And(
GTE(Select(['data', 'mydate'], Var('order')), '2020-07-09'),
LTE(Select(['data', 'mydate'], Var('order')), '2020-07-15')
)
)
)
You can update the conditions as you need
I believe this will work with the strings you have already
There are some mistakes here, first of all, you have to create documents that way:
Create(Collection("orders"), {data: {"mydate": ToDate("2020-07-10")}})
The index has to be created like this:
CreateIndex(
{
name: "orders_by_my_date",
source: Collection("orders"),
values:[{field:['data','mydate']},{field:['ref']}]
}
)
and finally, you can query your index and range:
Paginate(Range(Match('orders_by_my_date'),[Date("2020-07-09")], [Date("2020-07-15")]))
{ data:
[ [ Date("2020-07-10"),
Ref(Collection("orders"), "278532030954734085") ],
[ Date("2020-07-11"),
Ref(Collection("orders"), "278532033804763655") ],
[ Date("2020-07-12"),
Ref(Collection("orders"), "278532036737630725") ] ] }
or if you want to get the full doc:
Map(Paginate(Range(Match('orders_by_my_date'),[Date("2020-07-09")], [Date("2020-07-15")])),Lambda(['date','ref'],Get(Var('ref'))))
{ data:
[ { ref: Ref(Collection("orders"), "278532030954734085"),
ts: 1601887694290000,
data: { mydate: Date("2020-07-10") } },
{ ref: Ref(Collection("orders"), "278532033804763655"),
ts: 1601887697015000,
data: { mydate: Date("2020-07-11") } },
{ ref: Ref(Collection("orders"), "278532036737630725"),
ts: 1601887699800000,
data: { mydate: Date("2020-07-12") } } ] }
I have a dynamic table enhanced by jQuery DataTables, which display a custom object similar to this example.
JSON:
{
"data": [
{
"name": "Tiger Nixon",
"position": "System Architect",
"salary": "$320,800",
"start_date": {
"display": "SomeString",
"timestamp": 1303686000
},
"office": "Edinburgh",
"extn": "5421"
},
// ... skipped ...
]}
JavaScript:
$(document).ready(function() {
$('#example').DataTable( {
ajax: "data/orthogonal.txt",
columns: [
{ data: "name" },
{ data: "position" },
{ data: "office" },
{ data: "extn" },
{ data: {
_: "start_date.display",
sort: "start_date.timestamp"
} },
{ data: "salary" }
]
} );
} );
The difference is that I dynamically build the columns configuration, because the columns can be in any order, and others columns can be added or removed from the list. For this example (my case is very similar) the problem is that for some reason the timestamp property is ordered as a String instead of being ordered as a number.
I discovered that after setting the column "type" as "number" the ordering works perfectly. I'm presuming that DataTables is auto detecting the column as "String" for some reason (maybe because the display element is a string).
How does DataTables set the type of the columns, when is not explicitly declared?
Edit 1
I made a sample code to show the problem
http://jsfiddle.net/Teles/agrLjd2n/16/
jQuery DataTables has built-in mechanism for type detection. There are multiple predefined functions for various types with fallback to string data type.
It's also possible to use third-party plug-ins or write your own.
There are multiple ways to specify data type, below are just the few.
SOLUTION 1
Use type option.
$(document).ready(function() {
$('#example').DataTable( {
ajax: "data/orthogonal.txt",
columns: [
{ data: "name" },
{ data: "position" },
{ data: "office" },
{ data: "extn" },
{ data: "start_date.display", type: "date" },
{ data: "salary" }
]
} );
} );
SOLUTION 2
Use returned JSON data for type detection, see columns.data for more information.
$(document).ready(function() {
$('#example').DataTable( {
ajax: "data/orthogonal.txt",
columns: [
{ data: "name" },
{ data: "position" },
{ data: "office" },
{ data: "extn" },
{ data: {
_: "start_date.display",
sort: "start_date.timestamp",
type: "start_date.timestamp",
} },
{ data: "salary" }
]
} );
} );
DataTables always check the "type" property of the column "data" to auto detect the type of the column, if no "type" property is specified it will check the default value "_".
So if you want DataTables to auto detect the type of the column checking the type of your "sort" property you should set the "type" property of data to be equals to your "sort" value
Here is a sample code with different approchs to achieve what I was tryng to do. Thanks #Gyrocode.com and #davidkonrad.
var Cell = function(display, value) {
this.display = display;
this.value = value;
}
$(document).ready(function() {
var cells = [
new Cell("120 (10%)", 120),
new Cell("60 (5%)", 60),
new Cell("30 (2.5%)", 30)
];
$('#example').DataTable( {
data: cells,
columns: [
{
title : "Column NOT OK",
data: {
_: "display",
sort: "value"
}
}, {
type : "num",
title : "Column Ok setting column type",
data: {
_: "display",
sort: "value"
}
}, {
title : "Column Ok changing default value",
data: {
_: "value",
display: "display",
filter: "display"
}
}, {
title : "Column Ok setting data type",
data: {
_: "display",
sort: "value",
type: "value"
}
}, {
type : "num",
title : "Column Not OK",
data: "display"
}
]
} );
} );