Postgres: Expand JSON column into rows - sql

In a Postgres 9.3 database I have a table in which one column contains JSON, as in the test table shown in the example below.
test=# create table things (id serial PRIMARY KEY, details json, other_field text);
CREATE TABLE
test=# \d things
Table "public.things"
Column | Type | Modifiers
-------------+---------+-----------------------------------------------------
id | integer | not null default nextval('things_id_seq'::regclass)
details | json |
other_field | text |
Indexes:
"things_pkey" PRIMARY KEY, btree (id)
test=# insert into things (details, other_field)
values ('[{"json1": 123, "json2": 456},{"json1": 124, "json2": 457}]', 'nonsense');
INSERT 0 1
test=# insert into things (details, other_field)
values ('[{"json1": 234, "json2": 567}]', 'piffle');
INSERT 0 1
test=# select * from things;
id | details | other_field
----+-------------------------------------------------------------+-------------
1 | [{"json1": 123, "json2": 456},{"json1": 124, "json2": 457}] | nonsense
2 | [{"json1": 234, "json2": 567}] | piffle
(2 rows)
The JSON is always an array containing a variable number of hashes. Each hash always has the same set of keys. I am trying to write a query which returns a row for each entry in the JSON array, with columns for each hash key and the id from the things table. I'm hoping for output like the following:
thing_id | json1 | json2
----------+-------+-------
1 | 123 | 456
1 | 124 | 457
2 | 234 | 567
i.e. two rows for entries with two items in the JSON array. Is it possible to get Postgres to do this?
json_populate_recordset feels like an essential part of the answer, but I can't get it to work with more than one row at once.

select id,
(details ->> 'json1')::int as json1,
(details ->> 'json2')::int as json2
from (
select id, json_array_elements(details) as details
from things
) s
;
id | json1 | json2
----+-------+-------
1 | 123 | 456
1 | 124 | 457
2 | 234 | 567

Related

How to do a batch insert version of auto increment?

I have a temporary table that has columns id and value:
| id | value |
And I have a table that aggregates data with columns g_id and value:
| g_id | value |
The id in the temporary table is locally ordered by id, here's 2 examples:
temp_table 1
| id | value |
+----+--------+
| 1 | first |
| 2 | second |
| 3 | third |
temp_table 2
| id | value |
+----+-------+
| 2 | alpha |
| 3 | beta |
| 4 | gamma |
I want to insert all the rows from the temp table into the global table, while having the g_id ordered globally. If I insert temp table 1 before temp table 2, it should be like this:
global_table
| group_id | value |
+----------+--------+
| 1 | first |
| 2 | second |
| 3 | third |
| 4 | alpha |
| 5 | beta |
| 6 | gamma |
For my purposes, it is ok if there are jumps between consecutive numbers, and if 2 inserts are done at the same time, it can be interleaved. The requirement is basically that the id columns are always increasing.
e.g.
Let's say I have 2 sets of queries run at the same time, and the group_id in global_table is auto incremented:
Query 1:
INSERT INTO global_table (value) VALUES (first)
INSERT INTO global_table (value) VALUES (second)
INSERT INTO global_table (value) VALUES (third)
Query 2:
INSERT INTO global_table (value) VALUES (alpha)
INSERT INTO global_table (value) VALUES (beta)
INSERT INTO global_table (value) VALUES (gamma)
I can get something like this:
global_table
| group_id | value |
+----------+--------+
| 1 | first |
| 3 | alpha |
| 4 | second |
| 5 | third |
| 6 | beta |
| 7 | gamma |
How do I achieve something like this with inserting from a table? Like with
INSERT INTO global_table (value)
SELECT t.value
FROM temp_table t
Unfortunately, this may not result in a incrementing id all the time.
The requirement is basically that the id columns are always
increasing.
If I understood you correctly, you should use ORDER BY in your INSERT statement.
INSERT INTO global_table (value)
SELECT t.value
FROM temp_table t
ORDER BY t.id;
In SQL Server, if you include ORDER BY t.id, it will guarantee that the new IDs generated in the global_table table will be in the specified order.
I don't know about other databases, but SQL Server IDENTITY has such guarantee.
Using your sample data, it is guaranteed that the group_id generated for value second will be greater than the value generated for the value first. And group_id for value third will be greater than for value second.
They may be not consecutive, but if you specify ORDER BY, their relative order will be preserved.
Same for the second table, and even if you run two INSERT statements at the same time. Generated group_ids may interleave between two tables, but relative order within each statement will be guaranteed.
See my answer for a similar question with more technical details and references.

How can i insert into table on the basis of a column value is same or not in Postgresql

I am inserting data into a table looks like this
| num | name | value |
----------------------------------
| 1 | name1 | 1 |
| 2 | name2 | 1 |
| 3 | name3 | 1 |
| 4 | name4 | 2 |
| 5 | name5 | 3 |
I wanted to insert with where clause like insert into table (num, name, value) values(6,name,1) when (num and value together) not exist in any row together
I tried to select first and insert on basis of that result but I think that is not the best way I want it in a single query
tried like: select * from the table where name=$name and value= $value if I got result then not insert otherwise insert. It was done with two queries but i don't want it.
Any help will be appriciated.
Use a unique constraint to enforce uniqueness for (num, value):
alter table t add constraint unq_t_num_value unique (num, value);
Then the database ensures that the integrity of the table -- that these values are unique. You don't have to do it explicitly.
Note that if the unique constraint is violated, you get an error and the insert is aborted (along with other rows that might be inserted). If you want to ignore the error instead, you can use on conflict ignore.

INSERT ALL with ID using DEFAULT ON NULL fails PK Constraint

Using Oracle 12, I have a table defined similar to this:
CREATE TABLE example
( "ID" NUMBER(*,0) DEFAULT ON NULL ex_seq.nextval NOT NULL ENABLE,
"SIG_BOOK" NUMBER(10,0),
"SIG_LINE" NUMBER(10,0),
"TRANSFER" NUMBER(10,0) DEFAULT NULL,
CONSTRAINT "PK_EXAMPLE_ID" PRIMARY KEY ("ID")
-- snipped
)
When I do standard individual row inserts and leave off the ID, the sequence.nextval is called and the row is properly inserted. But we have to insert up to several thousand rows, so I am trying to use code like this:
INSERT ALL
INTO example (sig_book, sig_line, transfer) VALUES (1,22000006,3436440)
INTO example (sig_book, sig_line, transfer) VALUES (1,22000006,3184718)
SELECT * FROM dual
When using INSERT ALL then the Primary Key constraint is violated.
We can switch back to the standard Trigger/Sequence pair, but were hoping to gain additional performance from using INSERT ALL.
Is there something special I have to do to get this bulk insert to work on a table with the key defined using DEFAULT ON NULL, or do I need to return to the old Trigger/Sequence pair?
It gets a bit more clear when you look at the query plan:
-----------------------------------------------------------------------
| Id | Operation | Name | Rows | Cost (%CPU)| Time |
-----------------------------------------------------------------------
| 0 | INSERT STATEMENT | | 1 | 2 (0)| 00:00:01 |
| 1 | MULTI-TABLE INSERT | | | | |
| 2 | SEQUENCE | EX_SEQ | | | |
| 3 | FAST DUAL | | 1 | 2 (0)| 00:00:01 |
| 4 | INTO | EXAMPLE | | | |
| 5 | INTO | EXAMPLE | | | |
-----------------------------------------------------------------------
First Oracle runs the query (SELECT FROM dual), assigning the single id from the sequence and only after it goes through the results, inserting it to the tables. So this approach will not work.
You might try to use an interim table without the id:
CREATE TABLE example_without_id ("SIG_BOOK" NUMBER(10,0), ... );
INSERT ALL
INTO example_without_id (sig_book, sig_line, transfer) VALUES (1,22000006,3436440)
INTO example_without_id (sig_book, sig_line, transfer) VALUES (1,22000006,3184718)
SELECT * FROM dual;
INSERT INTO example (sig_book, sig_line, transfer) SELECT * FROM example_without_id;
Something as simple as putting these records in a file and loading them via SQL Loader could work as well.
How about using INSERT INTO . . . SELECT instead?
INSERT INTO example (sig_book, sig_line, transfer)
SELECT 1,22000006,3436440 FROM DUAL UNION ALL
SELECT 1,22000006,3184718 FROM DUAL;

Postgres Array Prefix Matching

I have an array search in Postgres hat matches at least one tag as this:
SELECT * FROM users WHERE tags && ['fun'];
| id | tags |
| 1 | [fun,day] |
| 2 | [fun,sun] |
It is possible to match on prefixes? Something like:
SELECT * FROM users WHERE tags LIKE 'f%';
| id | tags |
| 1 | [fun,day] |
| 2 | [fun,sun] |
| 3 | [far] |
| 4 | [fin] |
try this
create table users (id serial primary key, tags text[]);
insert into users (tags)
values
('{"fun", "day"}'),
('{"fun", "sun"}'),
('{"test"}'),
('{"fin"}');
select *
from users
where exists (select * from unnest(tags) as arr where arr like 'f%')
SQL FIDDLE EXAMPLE
Here's a working example that should get you more or less what you're after. Note that I am not claiming that this approach will scale...
create table users (
id serial primary key,
tags text[] not null
);
insert into users (tags) values
('{"aaaa","bbbb","cccc"}'::text[]),
('{"badc","dddd","eeee"}'::text[]),
('{"gggg","ffbb","attt"}'::text[]);
select *
from (select id,unnest(tags) arr from users) u
where u.arr like 'a%';
id | arr
----+------
1 | aaaa
3 | attt

Insert data into table where data in only two column varies

I have list of names and I need to insert into a table with a primary key which is auto generated and another three columns which will have the same data for each name. Is there any way to acheive this in single query?
| ID | Name | Age| Class|In-Charge|
|121 | Luc | 12 | Five | 47855 |
|122 | Wayne| 12 | Five | 47855 |
|123 | Lih | 12 | Five | 47855 |
You can use something like this where you SELECT the name from your list and the other values are just static values:
insert into yourtable (Name, Age, Class, [In-Charge])
select Name, 12, 'Five', 47855
from yourlist
See SQL Fiddle with Demo