I have a question about JSONStore searchFields.
If I use number as the searchFields key and try to find data by WL.JSONStore.find method with 0 as the query, It will hit all data (not filtered).
With the integer of the case above works fine.
What's the difference between number and integer?
JSONStore uses SQLite to persist data, you can read about SQLite Data Types here. The short answer is number will store data as REAL while integer will store data as INTEGER.
If you create a collection called nums with one searchField called num of type number
var nums = WL.JSONStore.initCollection('nums', {num: 'number'}, {});
and add some data:
var len = 5;
while (len--) {
nums.add({num: len});
}
then call find with the query: {num: 0}
nums.find({num: 0}, {onSuccess: function (res) {
console.log(JSON.stringify(res));
}})
you should get back:
[{"_id":1,"json":{"num":4}},{"_id":2,"json":{"num":3}},{"_id":3,"json":{"num":2}},{"_id":4,"json":{"num":1}},{"_id":5,"json":{"num":0}}]
Notice that you got back all the documents you stored (num = 4, 3, 2, 1, 0).
If you look at the .sqlite file:
$ cd ~/Library/Application Support/iPhone Simulator/6.1/Applications/[id]/Documents
$ sqlite3 jsonstore.sqlite
(The android file should be under /data/data/com.[app-name]/databases/)
sqlite> .schema
CREATE TABLE nums ( _id INTEGER primary key autoincrement, 'num' REAL, json BLOB, _dirty REAL default 0, _deleted INTEGER default 0, _operation TEXT);
Notice the data type for num is REAL.
Running a query the same query used in the find function:
sqlite> SELECT * FROM nums WHERE num LIKE '%0%';
1|4.0|{"num":4}|1363326259.80431|0|add
2|3.0|{"num":3}|1363326259.80748|0|add
3|2.0|{"num":2}|1363326259.81|0|add
4|1.0|{"num":1}|1363326259.81289|0|add
5|0.0|{"num":0}|1363326259.81519|0|add
Notice 4 is stored as 4.0 and JSONStore's queries always use LIKE, any num with a 0 will match the query.
If you use integer instead:
var nums = WL.JSONStore.initCollection('nums', {num: 'integer'}, {});
Find returns:
[{"_id":5,"json":{"num":0}}]
The schema shows that num has an INTEGER data type:
sqlite> .schema
CREATE TABLE nums ( _id INTEGER primary key autoincrement, 'num' INTEGER, json BLOB, _dirty REAL default 0, _deleted INTEGER default 0, _operation TEXT);
sqlite> SELECT * FROM nums WHERE num LIKE '%0%';
5|0|{"num":0}|1363326923.44466|0|add
I skipped some of the onSuccess and all the onFailure callbacks for brevity.
The actual difference between a JSON number and integer is
defining {age: 'number'} indexes 1 as 1.0,
while defining{age: 'integer'} indexes 1 as 1.
Hope you understand
Related
I'm trying to migrate our current ES to CrateDB and one of the issues I'm facing is searching for two specific values within the same object when this object is part of an array of objects.
CREATE TABLE test.artefact (
id INTEGER,
metadata ARRAY(OBJECT(STATIC) AS (
key_id INTEGER,
value TEXT
))
);
insert into test.artefact(id, metadata) values (
1,
[
{
"key_id" = 1,
"value" = 'TEST1'
},
{
"key_id" = 2,
"value" = 'TEST2'
}
]
);
So basically, I'm trying to search metadata providing key_id and value.
A select like this one finds artefact 1 as a match, even when key and value are in different objects:
select * from test.artefact where 1 = ANY(metadata['key_id']) AND 'TEST2' = ANY(metadata['value'])
I have tried other functions, like UNNEST, with no luck.
Copy from CrateDB Community:
One way that should work is
SELECT *
FROM test.artefact
WHERE {key_id = 1, value = 'TEST2'} = ANY(metadata)
however this is probably not the most performant way.
together with the queries on the fields it might be quick enough.
SELECT *
FROM test.artefact
WHERE
1 = ANY(metadata['key_id'])
AND 'TEST2' = ANY(metadata['value'])
AND {key_id = 1, value = 'TEST2'} = ANY(metadata)
The Gorm documentation for Struct & Map Conditions provides the following snippet for querying a table with a slice of primary keys
// Slice of primary keys
db.Where([]int64{20, 21, 22}).Find(&users)
// SELECT * FROM users WHERE id IN (20, 21, 22);
However, if the slice is empty then all records are returned. Looking at the source code for the Find function I can see conditions are only added if len(conds) > 0
// Find find records that match given conditions
func (db *DB) Find(dest interface{}, conds ...interface{}) (tx *DB) {
tx = db.getInstance()
if len(conds) > 0 {
if exprs := tx.Statement.BuildCondition(conds[0], conds[1:]...); len(exprs) > 0 {
tx.Statement.AddClause(clause.Where{Exprs: exprs})
}
}
tx.Statement.Dest = dest
return tx.callbacks.Query().Execute(tx)
}
This is the opposite of what my SQLite command line returns. If the condition is empty then no records are returned (because they all have a primary key)
-- no records returned
SELECT * FROM my_table WHERE id IN ();
Question
Is there a way to query a slice of primary keys using Gorm such that if the slice is empty, no records are returned?
Since primary keys increases from 1, 0 id could be used in a empty query.
ids := []int64{20, 21, 22}
db.Where(append([]int64{0}, ids...)).Find(&users)
I need to compute the factorial of a variable in Google BigQuery - is there a function for this? I cannot find one in the documentation here:
https://cloud.google.com/bigquery/query-reference#arithmeticoperators
My proposed solution at this point is to compute the factorial for numbers 1 through 100 and upload that as a table and join with that table. If you have something better, please advise.
As context may reveal a best solution, the factorial is used in the context of computing a Poisson probability of a random variable (number of events in a window of time). See the first equation here: https://en.wikipedia.org/wiki/Poisson_distribution
Try below. Quick & dirty example
select number, factorial
FROM js(
// input table
(select number from
(select 4 as number),
(select 6 as number),
(select 12 as number)
),
// input columns
number,
// output schema
"[{name: 'number', type: 'integer'},
{name: 'factorial', type: 'integer'}]",
// function
"function(r, emit){
function fact(num)
{
if(num<0)
return 0;
var fact=1;
for(var i=num;i>1;i--)
fact*=i;
return fact;
}
var factorial = fact(r.number)
emit({number: r.number, factorial: factorial});
}"
)
If the direct approach works for the values you need the Poisson distribution calculated on, then cool. If you reach the point where it blows up or gives you inaccurate results, then read on for numerical analysis fun times.
In general you'll get better range and numerical stability if you do the arithmetic on the logarithms, and then exp() as the final operation.
You want: c^k / k! exp(-c).
Compute its log, ln( c^k / k! exp(-c) ),
i.e. k ln(x) - ln(k!) - c
Take exp() of that.
Well but how do we get ln(k!) without computing k!? There's a function called the gamma function, whose practical point here is that its logarithm gammaln() can be approximated directly, and ln(k!) = gammaln(k+1).
There is a Javascript gammaln() in Phil Mainwaring's answer here, which I have not tested, but assuming it works it should fit into a UDF for you.
Extending Mikhail's answer to be general and correct for computing the factorial for all number 1 to n, where n < 500, the following solution holds and can be computed efficiently:
select number, factorial
FROM js(
// input table
(
SELECT
ROW_NUMBER() OVER() AS number,
some_thing_from_the_table
FROM
[any table with at least LIMIT many entries]
LIMIT
100 #Change this to any number to compute factorials from 1 to this number
),
// input columns
number,
// output schema
"[{name: 'number', type: 'integer'},
{name: 'factorial', type: 'float'}]",
// function
"function(r, emit){
function fact(num)
{
if(num<0)
return 0;
var fact=1;
for(var i=num;i>1;i--)
fact*=i;
return fact;
}
#Use toExponential and parseFloat to handle large integers in both Javascript and BigQuery
emit({number: r.number, factorial: parseFloat(fact(r.number).toExponential())});
}"
)
You can get up to 27! using SQL UDF. Above that value NUMERIC type gets overflow error.
CREATE OR REPLACE FUNCTION factorial(integer_expr INT64) AS ( (
SELECT
ARRAY<numeric>[
1,
1,
2,
6,
24,
120,
720,
5040,
40320,
362880,
3628800,
39916800,
479001600,
6227020800,
87178291200,
1307674368000,
20922789888000,
355687428096000,
6402373705728000,
121645100408832000,
2432902008176640000,
51090942171709440000.,
1124000727777607680000.,
25852016738884976640000.,
620448401733239439360000.,
15511210043330985984000000.,
403291461126605635584000000.,
10888869450418352160768000000.][
OFFSET
(integer_expr)] AS val ) );
select factorial(10);
Schema:
TABLE field (
field_id serial NOT NULL,
section_id integer,
title text,
type text,
default_val json,
rank integer,
tiny integer,
"values" json,
grp_id integer,
aggregate integer DEFAULT 1,
enabled integer,
deleted integer DEFAULT 0,
"desc" text
)
TABLE entry
(
entry_id serial NOT NULL,
section_id integer,
deleted integer DEFAULT 0,
grp_id integer,
data json,
last_edited bigint,
last_editor_id integer
)
The column field.values might look like this: {0: {"rank" : 2, "title" : "asdf"}, 1: {"rank" : 1}}
The column entry.data might look like this: {250: 1, 251: 0}
What I'd like to query:
SELECT entry.*
FROM entry
LEFT JOIN field ON field.field_id = 31
WHERE entry.deleted = 0 AND section_id = $1 AND grp_id = $2
ORDER BY cast(field.values#>>'{**entry.data->>250**, rank}' as numeric) ASC
This is the part that I don't know how to do: **entry.data->>250**
Is this even possible? Basically I'm trying to sort by the rank value of an index within a field that corresponds to an entry.
EDIT:
Attemped:
(field.values->(cast(coalesce(e.data->>'f4', '0') as numeric)))->>'rank'
Error: operator does not exist: json -> numeric
You don't try to write a string which some how resolves the embedded functions. You concatenate strings to build a larger one...
field.values#>>('{' || entry.data->>250 || ', rank}')
Or, maybe easier to read?
(field.values->(entry.data->>250))->>'rank'
I am wondering if there is some way to read/update a data structure into PostgreSQL table "at once" like is possible with files.
Public Structure myStuct
Dim p_id As Integer
Dim p_myInt As Integer
Dim p_myString As String
Dim p_myDecNumber As Double
End Structure
Dim st as myStruct
FileGet (ff, recordnum, st)
OR
st.p_id = 1234
st.myInt = 14
st.myString = "tarzan"
st.myDecNumber = 3.14
FilePut (ff, recordnum, st)
Question is if we have table with formed data which is of same type like members of structure "st" wouldn't be possible to insert/update whole structure at some index in the table instead of writing every single member one-by-one like I do now?
Have you tried postgresql composite types?
E.g.
CREATE TYPE myStruct AS (
p_Id integer,
myInt integer,
myString text,
myDecNumber double precision
);
CREATE TABLE myStructs (
value myStruct
);
INSERT INTO myStructs VALUES ( ROW(1234, 14, 'tarzan', 3.14) );
http://www.postgresql.org/docs/9.2/static/rowtypes.html#AEN7268