I'm working on a project where the database could be an instance of oracle or Postgres.
I have the need to write a query with a like that work on both dbs.
The query works on a text column containing a JSON string, for example:
{"ruleName":"r2_an","divisionNameList":["div1"],"names":["name1"],"thirdTypeLabels":[],"secondTypeLabels":[],"firstTypeLabels":[]}
I need to select the lines with empty thirdTypeLabels.
select *
from my_table
where JSON like '%thirdTypeLabels%[]%';
On Oracle, for example, does not extract anything, even if in "my_table" there is more than one line matching.
The query is inside a Java software, using JDBC, because we need performace.
Have you any suggestion?
You should use a proper JSON parser otherwise there is no guarantee that %thirdTypeLabels%[]% will restrict the match of the empty array to the thirdTypeLabels key-value pair.
So for Oracle 18c you can use:
SELECT id,
thirdTypeLabelsCount
FROM mytable t
CROSS JOIN
JSON_TABLE(
t.json,
'$'
COLUMNS(
thirdTypeLabelsCount NUMBER PATH '$.thirdTypeLabels.size()'
)
)
WHERE thirdTypeLabelsCount = 0;
or
SELECT *
FROM mytable
WHERE JSON_EXISTS( json, '$ ? (#.thirdTypeLabels.size() == 0) ' )
db<>fiddle
For Postgres you have two choices to make this query work properly:
select *
from the_table
where jsonb_array_length(json::jsonb -> 'thirdTypeLabels') = 0;
Or - starting with Postgres 12 - using a JSON Path expression
select *
from the_table
where jsonb_path_exists(json::jsonb, '$.thirdTypeLabels.size() ? (# == 0)' );
Or use the same JSON path expression as in Oracle:
select *
from the_table
where jsonb_path_exists(json::jsonb, '$' ? (#.thirdTypeLabels.size() == 0)');
In Postgres you should also use a column defined as jsonb rather than text (or varchar)
Related
I have a table that has JSON data stored and I'm using json_exists functions in the query. Below is my sample data from the column for one of the rows.
{"fields":["query.metrics.metric1.field1",
"query.metrics.metric1.field2",
"query.metrics.metric1.field3",
"query.metrics.metric2.field1",
"query.metrics.metric2.field2"]}
I want all those rows which have a particular field. So, I'm trying below.
SELECT COUNT(*)
FROM my_table
WHERE JSON_EXISTS(fields, '$.fields[*]."query.metrics.metric1.field1"');
It does not give me any results back. Not sure what I'm missing here. Please help.
Thanks
You can use # operator which refers to an occurrence of the array fields such as
SELECT *
FROM my_table
WHERE JSON_EXISTS(fields, '$.fields?(#=="query.metrics.metric1.field1")')
Demo
Edit : The above case works for 12R2+, considering that it doesn't work for your version(12R1), try to use JSON_TABLE() such as
SELECT fields
FROM my_table,
JSON_TABLE(fields, '$.fields[*]' COLUMNS ( js VARCHAR2(90) PATH '$' ))
WHERE js = 'query.metrics.metric1.field1'
Demo
I have no idea how to "pattern match" on the array element, but just parsing the whole thing and filtering does the job.
with t(x, json) as (
select 1, q'|{"fields":["a", "b"]}|' from dual union all
select 2, q'|{"fields":["query.metrics.metric1.field1","query.metrics.metric1.field2","query.metrics.metric1.field3","query.metrics.metric2.field1","query.metrics.metric2.field2"]}|' from dual
)
select t.*
from t
where exists (
select null
from json_table(
t.json,
'$.fields[*]'
columns (
array_element varchar2(100) path '$'
)
)
where array_element = 'query.metrics.metric1.field1'
);
In your code, you are accessing the field "query.metrics.metric1.field1" of an object in the fields array, and there is no such object (the elements are strings)...
Mockup code for my problem:
SELECT Id FROM Tags WHERE TagName IN '<osx><keyboard><security><screen-lock>'
The problem in detail
I am trying to get tags used in 2011 from apple.stackexchange data. (this query)
As you can see, tags in tag changes are stored as plain text in the Text field.
<tag1><tag2><tag3>
<osx><keyboard><security><screen-lock>
How can I create a unique list of the tags, to look them up in the Tags table, instead of this hardcoded version:
SELECT * FROM Tags
WHERE TagName = 'osx'
OR TagName = 'keyboard'
OR TagName = 'security'
Here is a interactive example.
Stackexchange uses T-SQL, my local copy is running under postgresql using Postgres app version 9.4.5.0.
Assuming this table definition:
CREATE TABLE posthistory(post_id int PRIMARY KEY, tags text);
Depending on what you want exactly:
To convert the string to an array, trim leading and trailing '<>', then treat '><' as separator:
SELECT *, string_to_array(trim(tags, '><'), '><') AS tag_arr
FROM posthistory;
To get list of unique tags for whole table (I guess you want this):
SELECT DISTINCT tag
FROM posthistory, unnest(string_to_array(trim(tags, '><'), '><')) tag;
The implicit LATERAL join requires Postgres 9.3 or later.
This should be substantially faster than using regular expressions. If you want to try regexp, use regexp_split_to_table() instead of regexp_split_to_array() followed by unnest() like suggested in another answer:
SELECT DISTINCT tag
FROM posthistory, regexp_split_to_table(trim(tags, '><'), '><') tag;
Also with implicit LATERAL join. Related:
Split column into multiple rows in Postgres
What is the difference between LATERAL and a subquery in PostgreSQL?
To search for particular tags:
SELECT *
FROM posthistory
WHERE tags LIKE '%<security>%'
AND tags LIKE '%<osx>%';
SQL Fiddle.
Applied to your search in T-SQL in our data explorer:
SELECT TOP 100
PostId, UserId, Text AS Tags FROM PostHistory
WHERE year(CreationDate) = 2011
AND PostHistoryTypeId IN (3 -- initial tags
, 6 -- edit tags
, 9) -- rollback tags
AND Text LIKE ('%<' + ##TagName:String?postgresql## + '>%');
(T-SQL syntax uses the non-standard + instead of ||.)
https://data.stackexchange.com/apple/query/edit/417055
I've simplified the data to the relevant column only and called it tags to present the example.
Sample data
create table posthistory(tags text);
insert into posthistory values
('<lion><backup><time-machine>'),
('<spotlight><alfred><photo-booth>'),
('<lion><pdf><preview>'),
('<pdf>'),
('<asd>');
Query to get unique list of tags
SELECT DISTINCT
unnest(
regexp_split_to_array(
trim('><' from tags), '><'
)
)
FROM
posthistory
First we're removing all occurences of leading and trailing > and < signs from each row, then using regexp_split_to_array() function to get values into arrays, and then unnest() to expand an array to a set of rows. Finally DISTINCT eliminates duplicate values.
Presenting SQLFiddle to preview how it works.
I am trying to execute this query using Postgresql 9.2
WITH udetail as (select(regexp_split_to_array('user#domain', E'\\#+')))
select * from udetail[1];
Buy it gives me a syntax error near where the start of '['. This same query is working fine under version 9.3. So I'm wonder if there's an alternative way of writing the query to get the same result.
I think you are looking for something like this:
WITH udetail(x) as (
select(regexp_split_to_array('user#domain', E'\\#+')))
select x[1] from udetail;
I don't think you can index a table expression like udetail in your case. It is the column of the table expression that is of array type. Hence, you can use an array subscript number on the column, not on the table itself.
Demo here
You should use an alias either as the query parameter:
with udetail(arr) as (
select regexp_split_to_array('user#domain', E'\\#+')
)
select arr[1] from udetail;
or as column alias:
with udetail as (
select regexp_split_to_array('user#domain', E'\\#+') as arr
)
select arr[1] from udetail;
You can also do it without aliases:
with udetail as (
select regexp_split_to_array('user#domain', E'\\#+')
)
select regexp_split_to_array[1] from udetail;
In a PostgreSQL table I have a column which has values like
AX,B,C
A,BD
X,Y
J,K,L,M,N
In short , it will have a few comma separated strings in the column for each record. I wanted to get the last one in each record. I ended up with this.
select id, reverse(substr(reverse(mycolumn),1,position(',' in reverse(mycolumn)))) from mytable order by id ;
Is there an easier way?
I would do it this way:
select reverse(split_part(reverse(myColumn), ',', 1))
With regexp_replace:
select id, regexp_replace(mycolumn, '.*,', '')
from mytable
order by id;
Is there an easier way?
With your current data, Gordon's answer works best imo. Other options would be a regex (messy), or converting the column to a text[] array e.g. ('{' || col || '}')::text[] or variations thereof.
If you were using a text[] array instead of plain text for your column, you'd want to use array functions directly:
select col[array_length(col, 1)]
http://www.postgresql.org/docs/current/static/functions-array.html
Example with dummy data:
with bar as (
select '{a,b,c}'::text[] as foo
)
select foo[array_length(foo, 1)] from bar;
You could, of course, also create a parse_csv() function or get_last_csv_value() function to avoid writing the above.
Let say you have a SELECT id from table query (the real case is a complex query) that does return you several results.
The problem is how to get all id return in a single row, comma separated?
SELECT string_agg(id::text, ',') FROM table
Requires PostgreSQL 9.0 but that's not a problem.
You can use the array() and array_to_string() functions togetter with your query.
With SELECT array( SELECT id FROM table ); you will get a result like: {1,2,3,4,5,6}
Then, if you wish to remove the {} signs, you can just use the array_to_string() function and use comma as separator, so: SELECT array_to_string( array( SELECT id FROM table ), ',' ) will get a result like: 1,2,3,4,5,6
You can generate a CSV from any SQL query using psql:
$ psql
> \o myfile.csv
> \f ','
> \a
> SELECT col1 AS column1, col2 AS column2 ... FROM ...
The resulting myfile.csv will have the SQL resultset column names as CSV column headers, and the query tuples as CSV rows.
h/t http://pookey.co.uk/wordpress/archives/51-outputting-from-postgres-to-csv
use array_to_string() & array() function for the same.
select array_to_string(array(select column_name from table_name where id=5), ', ');
Use this below query it will work and gives the exact result.
SELECT array_to_string(array_agg(id), ',') FROM table
Output : {1,2,3,4,5}
SELECT array_agg(id, ',') FROM table
{1,2,3,4}
I am using Postgres 11 and EntityFramework is fetching it as array of integers.