I have this JSON(B) column in my postgres database. I thrown in this column all the data that I get from an API. Now i need to extract some key/value pairs from this array.e.g. for a JSON array like:
[{"myKey1": "school type", "myKey2": "primary"}, {"myKey1": "study coverage", "myKey2": "secondary"}]
i need to extract:
myKey1: school type myKey2: primary
myKey1: study coverage myKey22: secondary
Since i am new to JSON, to me it appears that i first need to break the array into objects (surrounded by curly brackets) and than there are some postgres functions for objects (given in the following link https://www.postgresql.org/docs/9.4/static/functions-json.html) that i can use to extract the required key/value pairs.
My database table is:
CREATE TABLE "Education" (
"id" SERIAL NOT NULL,
"json_text" JSONB NOT NULL,
);
I am using the following query to achieve this:
SELECT tmp_col->>'myKey1'
FROM (
SELECT json_text->> 0 AS temp_col
FROM education
) AS temp_table
I get the following error when i run this query.
Any clues to what is missing?
Also is there any better ways to get the information in need. I am working in Java language to display the information.
Thanks.
If you can then please try to process JSON in java. You can query the complete JSON from postgres and then by using JSON API for Java you can then parse your JSON.
Related
Is it possible to generate a json object using the column name as keys automatically?
I have a table with many columns and I need to dump it into a json object.
I know I can do this using the JSON_OBJECT function but I was looking for a more condensed syntax that would allow me to do this without having to specify the name of all the columns
SELECT JSON_OBJECT("col_a", m.col_a, "col_b", m.col_b, "col_c", m.col_c, ...)
FROM largetable AS m
Something like this?
SELECT JSON_OBJECT(m.*)
FROM largetable AS m
I'm using MariaDB version 10.8.2
Json objects make sense in other languages like javascript, C#... There are many libraries to convert the result of a MariaDB query into a list of json objects in most languages.
Also, a good practice is to make the database engine do as little effort as possible when performing queries and processing the result in the application.
This is of course not possible, since the parser would not accept an odd number of parameters for the JSON_OBJECT function.
To do that in pure SQL, you can't do that within a single statement, since you need to retrieve the column names from information_schema first:
select #statement:=concat("SELECT JSON_OBJECT(", group_concat(concat("\"",column_name,"\"", ",", column_name)),") FROM mytable") from information_schema.columns where table_name="mytable" and table_schema="test";
prepare my_statement from statement;
execute my;
Much easier and faster is to convert the result in your application, for example in Python:
import mariadb, json
conn= mariadb.connect(db="test")
cursor= conn.cursor(dictionary=True)
cursor.execute("select * from mytable")
json_row= json.dumps(cursor.fetchone())
I have a postgres table with jsonb array elements and I'm trying to do sql queries to extract the matching elements. I have the raw SQL query running from the postgres command line interface:
select * from movies where director #> any (array ['70', '45']::jsonb[])
This returns the results I'm looking for (all records from the movies table where the director jsonb elements contain any of the elements in the input element).
In the code, the value for ['70, '45'] would be a dynamic variable ie. fixArr and the length of the array is unknown.
I'm trying to build this into my Bookshelf code but haven't been able to find any examples that address the complexity of the use case. I've tried the following approaches but none of them work:
models.Movies.where('director', '#> any', '(array' + JSON.stringify(fixArr) + '::jsonb[])').fetchAll()
ERROR: The operator "#> any" is not permitted
db.knex.raw('select * from movies where director #> any(array'+[JSON.stringify(fixArr)]+'::jsonb[])')
ERROR: column "45" does not exist
models.Movies.query('where', 'director', '#>', 'any (array', JSON.stringify(fixArr) + '::jsonb[])').fetchAll()
ERROR: invalid input syntax for type json
Can anyone help with this?
As you have noticed, knex nor bookshelf doesn't bring any support for making jsonb queries easier. As far as I know the only knex based ORM that supports jsonb queries etc. nicely is Objection.js
In your case I suppose better operator to find if jsonb column contains any of the given values would be ?|, so query would be like:
const idsAsString = ids.map(val => `'${val}'`).join(',');
db.knex.raw(`select * from movies where director \\?| array[${idsAsString}]`);
More info how to deal with jsonb queries and indexing with knex can be found here https://www.vincit.fi/en/blog/objection-js-postgresql-power-json-queries/
No, you're just running into the limitations of that particular query builder and ORM.
The easiest way is using bookshelf.Model.query and knex.raw (whereRaw, etc.). Alias with AS and subclass your Bookshelf model to add these aliased attributes if you care about such things.
If you want things to look clean and abstracted through Bookshelf, you'll just need to denormalize the JSONB into flat tables. This might be the best approach if your JSONB is mostly flat and simple.
If you end up using lots of JSONB (it can be quite performant with appropriate indexes) then Bookshelf ORM is wasted effort. The knex query builder is only not a waste of time insofar as it handles escaping, quoting, etc.
I access array of composite values like this:
PG_GETARG_ARRAYTYPE_P(0)
/* Then I deconstruct it into C array */
deconstruct_array()
/* Later I iterate thru values and attempt to access columns of my composite type */
GetAttributeByName(input_data1[i], "keyColumnName", &isnull[0])
This is how it looks in SQL:
SELECT * FROM my_c_function(array[(44, 1)::comp_type, (43, 0)::comp_type], array[(42, 1)::comp_type, (43, 1)::comp_type]);
Expected result:
array[(44, 1)::comp_type, (42, 1)::comp_type, (43, 1)::comp_type] /*order doesn't matter*/
But this does not work, because GetAttributeByName() works only with HeapTupleHeader, sadly I have array of Datum.
Normally you get HeapTupleHeader by accessing function attribute like so: PG_GETARG_HEAPTUPLEHEADER(0) but that is not meant for arrays (or I'm wrong?).
So is there some function/makro to get columns from Datum that is composite type or to convert composite type Datum into HeapTuple? I have gone as deep as heap_getattr(), but can't really find anything useful. Can't remember if there is already some kind of function that would access composite array in similar fashion and would show me how to do it.
For the context:
I have 2 arrays of composite type and I want to write C function for fast concatenation of them. I however cannot simply add right argument to left, because they could share "key" column and in that case I would like result to have only values from right side.
This is simple task in plpgSQL (unnest, full join, array_agg) but is very slow. I have tested the same task in hstore and json and both are much faster than unnest+array_agg, but I cannot use those data types without extensive database structure changes, so I was looking for different solution.
I guess all you need is the DatumGetHeapTupleHeader macro defined in fmgr.h.
I have a table "move" with one column "move_doc" which is a CLOB. The json stored inside has the structure:
{
moveid : "123",
movedate : "xyz",
submoves: [
{
submoveid: "1",
...
},
{
submoveid : "2",
...
}
]
}
I know I can run an Oracle 12c query to access the submoves list with:
select move.move_doc.submoves from move move
How do I access particular submoves of the array? And the attributes inside a particular submove?
You have to use Oracle functions json_query and/or json_value like this:
SELECT json_value(move_doc, '$.submoves[0].submoveid' RETURNING NUMBER) FROM move;
returns 1.
SELECT json_query(move_doc, '$.submoves[1]') FROM move;
would return the second JSON element, i.e. something like
{
submoveid : "2",
...
}
json_value is used to retrieve a scalar value, json_query is used to retrieve JSON values. You might also want to have a look at json_table which returns an SQL result table and thus can be used in Joins.
See this Oracle Doc for more examples
Beda here from the Oracle JSON team.
We have added a new multi-value index in release 21c allowing you to index values from a JSON array. Obviously, 21c is brand new and you want to know how to do this in older releases: Functional indexes (using JSON_Value function) are limited to a single value per JSON document and therefore are not capable to index array values. But: there is a 'JSON search index' which indexes your entire JSON document and therefore also values in the array. Another solution is to use a materialized view usign JSON_Table. This will expand the array values into separate rows.Then you can add a regular B-Tree index on that column.
Sample code here:
JSON indexing with functional indexes and JSON search index
https://livesql.oracle.com/apex/livesql/file/content_HN507PELCEEJGVNW4Q61L34DS.html
JSON and materialized views
https://livesql.oracle.com/apex/livesql/file/content_HYMB1YBP4CPMG6T6MXY5G9X5L.html
From what I've looked, In Oracle, you can index the "whole array" as a single index entry, but not individual elements of an array.
NoSQL databases like MongoDB, Couchbase, Cassandra have "array/collection" indexes which can index individual elements or fields of objects within an array and query them.
I have this JSON stored in DB:
Column name: json
- '{"brand":"1","year":"2008","model":"2","price":"2001212","category":"Category Example"}'
- '{"brand":"1","year":"2008","model":"2","price":"2001212","category":"Category Example2"}'
I want to make a search using Like operator to find all categories with "Category" word:
At this moment Im doing it this way, but only return a complete phrase:
select * from table where json like '%"category":"Category Example"%';
How can I build a query that returns all categories with "Category word"?
Updated:
I'm using MySQL
Thanks
While undeclared this looks like a Postgres question.
There are hardly any JSON-processing tool in the current version 9.2.
But a whole set of tools will be shipped with the upcoming Postgres 9.3 currently in beta.
I interpret your question as:
Find all rows where the json column contains one or more fields named 'category' holding a value that contains the string 'Category'.
One ore more? Not sure if Postgres enforces uniqueness, I don't have a 9.3 installation at hand.
With Postgres 9.3, your query could look like this:
SELECT *
FROM tbl
WHERE json->>'category' LIKE '%Category%'
->> .. "Get JSON object field as text"
Use ILIKE for a case insensitive search.
More in this related answer:
How do I query using fields inside the new PostgreSQL JSON datatype?
Can you use a library? The "common schema" library offers a function that does just what you need:
http://common-schema.googlecode.com/svn/trunk/common_schema/doc/html/extract_json_value.html
Maybe I asked a really bad question, because I could make the search using Regexp.
I found this solution. Maybe this is not the fastest way, but does what I need:
select * from table where json regexp '"category":"([^"]*)Category([^"]*)"';
Thanks
I hope this helps.
select * from table where json #> '{"category":"Category Example"}';