How to flatten out nested array of strings in json column? - sql

I have the following table:
id
contents
123
{ blocks: [{ text: "abc" }, { text: "123" }] }
foo
{ blocks: [{ text: "bar" }, { text: "moretext" }, { text: "ok" }] }
I want to write a view of the above that looks like:
id
contents
raw_text
123
{blocks: [{text: "abc"}, {text: "123"}]}
abc, 123
foo
{blocks: [{text: "bar"}, {text: "moretext"}, { text: "ok"}]}
bar, moretext, ok
This was the query I tried running:
select post.id, array_to_string(array_agg(jsonb_array_elements(post.contents -> 'blocks') ->> 'text')) as paragraphs from post group by id
But it results in the error
aggregate function calls cannot contain set-returning function calls.

If a JSON array of all the values is also acceptable, you can use a JSON path query:
select id, contents,
jsonb_path_query_array(contents, '$.blocks[*].text')
from post;
As there is no simply cast from a JSON array to a native Postgres array, and you do need that as a CSV string, you need to unnest and aggregate with a scalar sub-query:
select id, contents,
(select string_agg(x.item ->> 'text', ', ')
from jsonb_array_elements(contents -> 'blocks') as x(item)) as raw_text
from post;
The reason for your error is, that you are mixing nesting multiple aggregate and set returning function which simply isn't supported.

Related

PostgreSQL / TypeORM: String array type, how to use LIKE in query?

My backend database is PostgreSQL
I have a TypeORM object simplified to:
#Entity()
#Index(['name'], {unique: true}
export class Foo extends BaseEntity
{
#PrimaryGeneratedColumn('uuid')
id: string;
#Column()
name: string;
#Column('varchar', { array: true })
bar: string[];
}
I'm creating an API query handler that can handle searches. I can easily do a LIKE query on the name like this:
let qs = Foo.createQueryBuilder('foo');
qs.andWhere('foo.name ILIKE :name', {
name:'%${name}%'
});
I'd like to also search for e.g. any "bar" LIKE %myqueryterm% but I can't seem to find anything on that.
I see a bunch of docs on how to exactly match a search term in bar, but no soft comparison.
What I essentially want to do is that I have a data set
[
{id: 1, name: 'whatever', bar: ['apple','bananna','yeti','woo']},
{id: 2, name: 'something else', bar: ['red','blue','green', 'boo']},
{id: 3, name: 'i dunno', bar: ['ford','chevy']},
]
and I'd like to let the user to be able to query e.g. "%oo% and return the first 2 records based on bar strings containing that substring.
Postgres provides array functions and operators that you can use to create any complex query.
In your case, a clean way of doing this would be to
Convert the array to a string and then
Perform the LIKE operation on that string
Something like this should work:
.createQueryBuilder('foo')
.where("array_to_string(foo.bar, ',') LIKE :bar", {
bar: '%aa%',
})
.getMany();
I don't know typeorm. but based on https://github.com/typeorm/typeorm/issues/881
The sql query would be like:
WITH cte (
id,
name,
bar
) AS (
VALUES (1, 'whatever', ARRAY['apple', 'bananna', 'yeti', 'woo']),
(2, 'something else', ARRAY['red', 'blue', 'green', 'boo']),
(3, 'i dunno', ARRAY['ford', 'chevy'])
),
cte1 AS (
SELECT
json_agg(row_to_json(cte.*)) AS json_all
FROM
cte
)
SELECT
value
FROM
cte1,
json_array_elements(json_all)
WHERE
value ->> 'bar' ~ 'oo';
Based on the github page, it would be like:
getConnection().query("
with cte(id,name,bar) as (values
(1,'whatever',array ['apple','bananna','yeti','woo'])
,(2,'something else',array ['red','blue','green', 'boo'])
,(3,'i dunno',array ['ford','chevy'])
),cte1 AS
(select json_agg(row_to_json(cte.*)) as json_all from cte)
select value
from cte1,json_array_elements(json_all)
where value->>'bar' ~ #0", ['oo']);
case insensitively match would be value->>'bar' ~* #0"

How to select specific fields on FaunaDB Query Language?

I can't find anything about how to do this type of query in FaunaDB. I need to select only specifics fields from a document, not all fields. I can select one field using Select function, like below:
serverClient.query(
q.Map(
q.Paginate(q.Documents(q.Collection('products')), {
size: 12,
}),
q.Lambda('X', q.Select(['data', 'title'], q.Get(q.Var('X'))))
)
)
Forget the selectAll function, it's deprecated.
You can also return an object literal like this:
serverClient.query(
q.Map(
q.Paginate(q.Documents(q.Collection('products')), {
size: 12,
}),
q.Lambda(
'X',
{
title: q.Select(['data', 'title'], q.Get(q.Var('X')),
otherField: q.Select(['data', 'other'], q.Get(q.Var('X'))
}
)
)
)
Also you are missing the end and beginning quotation marks in your question at ['data, title']
One way to achieve this would be to create an index that returns the values required. For example, if using the shell:
CreateIndex({
name: "<name of index>",
source: Collection("products"),
values: [
{ field: ["data", "title"] },
{ field: ["data", "<another field name>"] }
]
})
Then querying that index would return you the fields defined in the values of the index.
Map(
Paginate(
Match(Index("<name of index>"))
),
Lambda("product", Var("product"))
)
Although these examples are to be used in the shell, they can easily be used in code by adding a q. in front of each built-in function.

How to filter on a json column for a specific value?

I'm on postgres and have a table orders with a data column which is jsonb. Here's a condensed example of data in one of them - they have UUID keys and a value of { id, value }
{
'36462bd9-4ffa-4ee3-9a04-c2eb7575fe6c': {
id: '',
value: '2020-04-20T01:32:14.017Z',
},
'9baaed61-1275-4bbc-ae4f-2994ec9f7fda': { id: '4', value: 'Paper Towels' },
}
How can I do operations such as to find any orders where data has some UUID (ie. 9baaed61-1275-4bbc-ae4f-2994ec9f7fda) and { id: '4' }?
You can use the contains operator #>
select *
from the_table
where data #> '{"9baaed61-1275-4bbc-ae4f-2994ec9f7fda": {"id": "4"}}';
This assumes that the invalid JSON id: '4' from your question is really stored as "id":"4". If the value is stored as a number: "id": 4 then you need to use that in the comparison value.

Cannot pass input field of repeated record type into Bigquery UDF

When I pass an input field of repeated record type into Bigquery UDF, it keeps saying that the input field is not found.
This is my 2 rows of data:
{"name":"cynthia", "Persons":[ { "name":"john","age":1},{"name":"jane","age":2} ]}
{"name":"jim","Persons":[ { "name":"mary","age":1},{"name":"joe","age":2} ]}
This is the schema of the data:
[
{"name":"name","type":"string"},
{"name":"Persons","mode":"repeated","type":"RECORD",
"fields":
[
{"name": "name","type": "STRING"},
{"name": "age","type": "INTEGER"}
]
}
]
And this is the query:
SELECT
name,maxts
FROM
js
(
//input table
[dw_test.clokTest_bag],
//input columns
name, Persons,
//output schema
"[
{name: 'name', type:'string'},
{name: 'maxts', type:'string'}
]",
//function
"function(r, emit)
{
emit({name: r.name, maxts: '2'});
}"
)
LIMIT 10
Error I got when trying to run the query:
Error: 5.3 - 15.6: Undefined input field Persons
Job ID: ord2-us-dc:job_IPGQQEOo6NHGUsoVvhqLZ8pVLMQ
Would someone please help?
Thank you.
In your list of input columns, list the leaf fields directly:
//input columns
name, Persons.name, Persons.age,
They'll still appear in their proper structure when you get the records in your UDF.

Is there a way to get the Type for a Column using package database/sql in golang?

Basically, without knowing before hand what the resulting structure of a query might be, I'd like to query the database, and return a structure like this (json-y)
// Rows
[
// Row 1
[
{ ColumnName: "id", Value: 1, Type: int },
{ ColumnName: "name", Value: "batman", Type: string },
...
],
// Row 2
[
{ ColumnName: "id", Value: 2, Type: int },
{ ColumnName: "name", Value: "superman", Type: string },
...
]
]
Is there a way to get the Type for a Column using package database/sql in golang?
I'm suspecting that what I want to do is
make an array of interface{} the size of Column(),
then for each column determine it's type,
then fill the array with a pointer to that type,
and then pass the array to Scan()
Which is a little like this code example from sqlx, but without first knowing the Struct that the data would be populating.
You should be able to do it this way:
func printRows(rows *sql.Rows){
colTypes, err := rows.ColumnTypes()
for _,s := range colTypes {
log.Println("cols type:", s.DatabaseTypeName());
}
}
Using database/sql? No (as far as I know).
But you can use this code for arbitrary queries. And json.Marshall() from the json package will use reflection to determine the right way to print a value, so you could have a structure like this:
type Column struct {
ColumnName string
ColumnValue interface{}
ColumnType string
}
And then use reflect.TypeOf(someVariable).String() to get the type for ColumnType.