Search through jsonb column in ActiveRecord - sql

My User model have jsonb column which names is raw. It looks like this:
{"client"=>{"tokens"=>["asdasd"]}}
Now I want to find a user by a token which is in raw["client"]["tokens"]. How can I do that?

I usually first craft such queries in a SQL console and then convert it to ActiveRecord.
You can navigate hash keys in Postgres. The query
SELECT
raw #> '{client,tokens}'
FROM users
will return just the tokens array from that path. Now we will need to check if it contains the value we are looking for. The query
SELECT
raw #> '{client,tokens}' ? 'asdasd'
FROM users
will select t for those row that have a matching token. Now you can move this to the WHERE section:
SELECT
*
FROM users
WHERE raw #> '{client,tokens}' ? 'asdasd'
And if this selects what you expect, then you can convert it to AR:
User.where("config #> '{client, tokens}' ? :token", token: 'asdasd')
Note that I can not use ? for parameter substitution and use :token instead. Also note that this only works in JSONB (Postgres 9.4+ https://www.postgresql.org/docs/9.4/static/functions-json.html#FUNCTIONS-JSONB-OP-TABLE)
Update:
You should(tm) (I have not tested this) get along with:
CREATE INDEX index_tokens ON users USING GIN ((raw #> '{client, tokens}'));
See https://www.postgresql.org/docs/9.6/static/datatype-json.html#JSON-INDEXING for more details
Update2:
Another way to query would be:
raw #> '{"client": {"tokens": ["asdasd"]}}'
Which should be able to use a simple GIN index on the raw column (which uses more space than the expression index described above).
CREATE INDEX index_user_raw ON users USING GIN (raw)
Again: details see the JSON INDEXING link above. And use a Query Visualizer to see the differences. I like http://tatiyants.com/pev

Related

How to index name of an element in xml column in Postgres

I am trying to index name of elements and I keep running into this error
ERROR: set-returning functions are not allowed in index expressions
This is what I have tried so far.
Sample xml:
<book><title>Manual</title><chapter>1</chapter></book>
DDL:
CREATE INDEX test2_element_name_idx
ON test2 USING GIN(xpath('local-name(/*)',unnest(xpath('//book/*', xml_data))));
Is it possible to index on element names? At the end I want to index on all elements that are under <book> (i.e <title> <chapter>)
One of the sample usecase is, I wanna query (with xpath) to learn how many books have title. And I believe that indexing it would make the queries more efficient. Please correct me if I my understanding is incorrect.
A stated by a_horse_with_no_name, you can't use a function which returns multiple rows for indexing a table. What you can do instead is to build an array with the multiple rows returned by your function. I propose here after a solution that may need to be adjusted because I never used the xml data type and functions (json is better :-) :
CREATE OR REPLACE FUNCTION xml_data_agg(xml_data xml)
RETURNS xml[] LANGUAGE sql AS
$$
SELECT array(SELECT xpath('local-name(/*)',unnest(xpath('//book/*', xml_data)))) ;
$$ ;
CREATE INDEX test2_element_name_idx
ON test2 USING GIN(xml_data_agg(xml_data));
Then you can use this index in queries where you can put this type of where clause : WHERE xml_data_agg(xml_data) #> array[list_of_the_xlm_elements_to_be_searched]

PostgreSQL calculated index for JSON value doesn't work in peewee?

I am using peewee to create a PostgreSQL calculated index from a value in a JSON column.
Here is the code
idx = Tweet.index(Tweet.data['x'], name='x')
Tweet.add_index(idx)
This produces the following SQL which does not work.
CREATE INDEX IF NOT EXISTS "x" ON "tweet" ("data"->>'x')
According to the PostgreSQL docs, the JSON expression must be wrapped by two sets of parentheses, as follows:
CREATE INDEX IF NOT EXISTS "x" ON "tweet" (("data"->>'x'))
Am I doing something wrong, or is this a bug in Peewee? How can I fix that?
That seems like a postgres bug since it's arbitrary as all hell. In this case, just add the index explicitly using the SQL() helper:
Tweet.add_index(SQL('create index ...'))

Using Bookshelf to execute a query on Postgres JSONB array elements

I have a postgres table with jsonb array elements and I'm trying to do sql queries to extract the matching elements. I have the raw SQL query running from the postgres command line interface:
select * from movies where director #> any (array ['70', '45']::jsonb[])
This returns the results I'm looking for (all records from the movies table where the director jsonb elements contain any of the elements in the input element).
In the code, the value for ['70, '45'] would be a dynamic variable ie. fixArr and the length of the array is unknown.
I'm trying to build this into my Bookshelf code but haven't been able to find any examples that address the complexity of the use case. I've tried the following approaches but none of them work:
models.Movies.where('director', '#> any', '(array' + JSON.stringify(fixArr) + '::jsonb[])').fetchAll()
ERROR: The operator "#> any" is not permitted
db.knex.raw('select * from movies where director #> any(array'+[JSON.stringify(fixArr)]+'::jsonb[])')
ERROR: column "45" does not exist
models.Movies.query('where', 'director', '#>', 'any (array', JSON.stringify(fixArr) + '::jsonb[])').fetchAll()
ERROR: invalid input syntax for type json
Can anyone help with this?
As you have noticed, knex nor bookshelf doesn't bring any support for making jsonb queries easier. As far as I know the only knex based ORM that supports jsonb queries etc. nicely is Objection.js
In your case I suppose better operator to find if jsonb column contains any of the given values would be ?|, so query would be like:
const idsAsString = ids.map(val => `'${val}'`).join(',');
db.knex.raw(`select * from movies where director \\?| array[${idsAsString}]`);
More info how to deal with jsonb queries and indexing with knex can be found here https://www.vincit.fi/en/blog/objection-js-postgresql-power-json-queries/
No, you're just running into the limitations of that particular query builder and ORM.
The easiest way is using bookshelf.Model.query and knex.raw (whereRaw, etc.). Alias with AS and subclass your Bookshelf model to add these aliased attributes if you care about such things.
If you want things to look clean and abstracted through Bookshelf, you'll just need to denormalize the JSONB into flat tables. This might be the best approach if your JSONB is mostly flat and simple.
If you end up using lots of JSONB (it can be quite performant with appropriate indexes) then Bookshelf ORM is wasted effort. The knex query builder is only not a waste of time insofar as it handles escaping, quoting, etc.

Oracle string search related query to find user names that matches search?

I am bit new to SQL help me to write a simple query
suppose table name is "Users" and below column name is "DomainUser"
ac\jason_mirabello
mac\jason_thompson
nyc\jason_graham
jdb\jason_bates
i got user name that comes after '\'
so i need to search through user name to find if a user exist so i need to exclude domain part
help me writing a query to achieve above in oracle
You can use a LIKE operator:
SELECT * FROM users
WHERE domainuser LIKE '%\jason_graham';
but I like using a regular expression here in case the backslash \ doesn't exist:
SELECT * FROM users
WHERE REGEXP_SUBSTR(domainuser, '[^\\]+$') = 'jason_graham';
The difficulty with both of the above queries is that neither will use an index. What you can do is create a function-based index which will work with the latter query:
CREATE INDEX domainuser_un ON users (REGEXP_SUBSTR(domainuser, '[^\\]+$'));
It might also be helpful to use LOWER() so that the index is "case-insensitive" (just make sure to convert the search term to lowercase too!) - assuming your usernames are not case-sensitive:
CREATE INDEX domainuser_un ON users (LOWER(REGEXP_SUBSTR(domainuser, '[^\\]+$')));
Here is an explanation of the regular expression I used.
Try using like clause as below:
SELECT *
FROM Users
WHERE DomainUser LIKE '%jason_graham'

SQL LIKE operator to find words in stored JSON

I have this JSON stored in DB:
Column name: json
- '{"brand":"1","year":"2008","model":"2","price":"2001212","category":"Category Example"}'
- '{"brand":"1","year":"2008","model":"2","price":"2001212","category":"Category Example2"}'
I want to make a search using Like operator to find all categories with "Category" word:
At this moment Im doing it this way, but only return a complete phrase:
select * from table where json like '%"category":"Category Example"%';
How can I build a query that returns all categories with "Category word"?
Updated:
I'm using MySQL
Thanks
While undeclared this looks like a Postgres question.
There are hardly any JSON-processing tool in the current version 9.2.
But a whole set of tools will be shipped with the upcoming Postgres 9.3 currently in beta.
I interpret your question as:
Find all rows where the json column contains one or more fields named 'category' holding a value that contains the string 'Category'.
One ore more? Not sure if Postgres enforces uniqueness, I don't have a 9.3 installation at hand.
With Postgres 9.3, your query could look like this:
SELECT *
FROM tbl
WHERE json->>'category' LIKE '%Category%'
->> .. "Get JSON object field as text"
Use ILIKE for a case insensitive search.
More in this related answer:
How do I query using fields inside the new PostgreSQL JSON datatype?
Can you use a library? The "common schema" library offers a function that does just what you need:
http://common-schema.googlecode.com/svn/trunk/common_schema/doc/html/extract_json_value.html
Maybe I asked a really bad question, because I could make the search using Regexp.
I found this solution. Maybe this is not the fastest way, but does what I need:
select * from table where json regexp '"category":"([^"]*)Category([^"]*)"';
Thanks
I hope this helps.
select * from table where json #> '{"category":"Category Example"}';