Create columns based on key-value pairs in a string in BigQuery - google-bigquery

I have a BigQuery table that looks like this, where response_group is a string
id
response_group
A1
[{"Answer":"blue","Question":"what's your favourite colour?"},{"Answer":"dogs","Question":"do you prefer dogs or cats?"},{"Answer":"coffee", "Question":"do you prefer tea or coffee?"}]
A2
[{"Answer":"green","Question":"what's your favourite colour?"},{"Answer":"Superman","Question":"who's your favourite superhero?"},{"Answer":"coffee", "Question":"do you prefer tea or coffee?"}]
As shown here, not all of the questions are the same for each ID.
What I'd like to create is a table like this
id
what's your favourite colour?
do you prefer dogs or cats?
do you prefer tea or coffee?
who's your favourite superhero?
A1
blue
dogs
coffee
A2
green
coffee
superman
I've made lots of attempts at a solution, but can't get anything to work.
Can anyone point me in the right direction?

Try below
create temp table tmp as (
select id,
translate(json_value(response, '$.Question'), " '?", "_") question,
json_value(response, '$.Answer') answer
from your_table, unnest(json_extract_array(response_group)) response
);
execute immediate (
select 'select * from tmp pivot (any_value(answer) for question in (' ||
string_agg(distinct '"' || question || '"') || '))'
from tmp
);
if applied to sample data in your question - output is
To test above I was using dummy data from your question
create temp table your_table as (
select 'A1' id, '''[{"Answer":"blue","Question":"what's your favourite colour?"},{"Answer":"dogs","Question":"do you prefer dogs or cats?"},{"Answer":"coffee", "Question":"do you prefer tea or coffee?"}]''' response_group union all
select 'A2', '''[{"Answer":"green","Question":"what's your favourite colour?"},{"Answer":"Superman","Question":"who's your favourite superhero?"},{"Answer":"coffee", "Question":"do you prefer tea or coffee?"}]'''
);

Related

Split not-atomar value into multiple rows with PostgreSQL

I have some not atomar data in a database like this:
ID
Component ID List
1
123, 456
2
123, 345
I need to transform those table into a view that provides the "Component ID List" in a way, that I can use joins. Expected result:
ID
Component ID List
1
123
1
456
2
123
2
345
Because I have this case in quite a few tables I look for the possibility to create a reusable way to perform this action, e.g. with a SQL-function. The tables have different column-names so the function would need a parameter, like this:
SELECT *, split_values("Component ID List") FROM xyz
I know the best way would be to fix the problem in the raw-data but that's not possible in this case.
Any suggestions how to solve this the best way possible?
You can use unnest(string_to_array(Component_ID_List, ', ')):
SELECT ID,
unnest(string_to_array(Component_ID_List, ', ')) as Component_ID_List
FROM table_name;
Fiddle

How can I implement a select list in apex oracle's interactive grid using only sql and what's available in oracle APEX?

ID
Material
Material_Num
User
Assign User
1
stick
1111
Billy Bob
"select list of users"
2
stone
1112
Jane Doe
"select list of users"
3
rock
1113
John Deer
"select list of users"
4
slab
1114
"select list of users"
5
brick
1115
"select list of users"
There will be a save button which will update the user column with the value in the assign user column. If the select list can be consolidated under the user column and keep the updated values that would be even better.
These are the steps I have taken so far:
Added an interactive grid to the page.
Created a query to populate the interactive grid.
SELECT DISTINCT
MD.ID,
MD.MATERIAL,
MD.MATERIAL_NUMBER,
FIRST_NAME || ' ' || LAST_NAME USER,
USER AS DISPLAY_VALUE
FROM MATERIAL_DATA MD
INNER JOIN MATERIAL_OWNER_DATA MOD
ON MD.USER = MOD.OWNER
Added a process called "save interactive grid data" to make the grid editable
Added an assign user column to the interactive grid.
Made the assign_user column editable and changed it to a select list.
SELECT
T1.DISPLAY_VALUE as DISPLAY_VALUE
FROM (
( SELECT
TM.FIRST_NAME || ' ' || TM.LAST_NAME as DISPLAY_VALUE,
upper(TM.NUMBER) as RETURN_VALUE,
1 as ORD
FROM
TEAM_MEMBERS TM
WHERE
TM.NUMBER IS NOT NULL
AND TM.BOOL = 'Y'
)
UNION ALL
( SELECT
'Not Required' as DISPLAY_VALUE,
'Not Required' as RETURN_VALUE,
0 as ORD
FROM DUAL
)
ORDER BY 3, 1
) T1
Added the assign_user column to my interactive grid's select statement.
So far as I have searched I have not seen a step-by-step way to add the select list to the grid. It's possible I've oversimplified my process but this is the best way I could describe it. If answering this is too cumbersome a link to a good source would also be appreciated.
My company won't allow the use of css or javascript so this has to be done strictly with sql and what's already available in apex_oracle.
The 2nd query you posted should be
SELECT t1.display_value, t1.return_value FROM (...) t1
(at least, looks like to me).

SQL query find rows that share the same tags

Let's see if someone can help me...
On a product page, I have to show related products. These products are related depending on their tags. For example my product (ID 23) has the following tags: cooking, noodles, healthy.
What I have to do, is a SQL query, that search for products that have the tags "cooking, noodles, healthy" but separately
First search for all the products that have "cooking" in their tags, then look for other products to have "noodles" , and finally "healthy".
The products shown have to be in order: first show all products that have "cooking" in the tags column, then "noodles" and lastly "healthy". Important, the products do not have to be repeated.
This is the order I hope to get: ID: 25, 25, 40, 43, 46
This is the database
ID
Tags
Name
23
cooking, noodles, healthy
Noodles Hot white pepper
25
cooking, noodles, healthy
Soap of noodles
35
cooking, noodles, healthy
Noodles with carrots
40
food, noodles, ketchup
New Noodles with ketchup
43
apple, cook, healthy
Apple with sugar
46
banana, cook, healthy
Banana with sugar
Thanks for the help!!
The way you store your data makes things unnecessarily suboptimal. You should not have multiple strings stored in a single column; instead, there should be another table to store the relation between products and tags, with each tuple should on a separate row.
For your current design, you can do:
select t.*
from mytable t
where
',' || tags || ',' like '%,cooking,%'
or ',' || tags || ',' like '%,noodles,%'
or ',' || tags || ',' like '%,healthy,%'
order by
case
when ',' || tags || ',' like '%,cooking,%' then 1
when ',' || tags || ',' like '%,noddles,%' then 2
else 3
end,
id
This uses standard string concatenation operator ||: you might need to adapt that to your actual database.
Some databases have built-in functions to search a CSV list. If, for example, you are running MySQL:
select t.*
from mytable t
where
find_in_set('cooking', tags)
or find_in_set('noodles', tags)
or find_in_set('healthy', tags)
order by
case
when find_in_set('cooking', tags) then 1
when find_in_set('noodles', tags) then 2
else 3
end,
id

Searching for partial text within a column

I have a search field in an admin system where a user can enter search criteria to find a product to edit within their shop. Some of the product names are long and similar. For example I have multiple product names with measurements in them. Here is an example:
Widget Brown Bull Edge Stone 400x500x600
Widget Brown Bull Snow Stone 700x500x300
There are about a hundred products like this, similar in name with only slight variations in either colour, size or description.
I've tried using regEx without much luck, I've also tried variations of LIKE without luck either unfortunately.
I want to be able to enter search criteria in my HTML form field and use SQL to limit the results based on the amount of search criteria I enter.
For example if I entered:
"Widget Brown 400" in the search field I can't use %like% in the where clause as it won't match that pattern.
My ideal result would be for me to enter Widget Brown 400 and the list of results would show those products that match that search criteria.
As commnented by Gordon Linoff, the best approach would be to settle for SQL-Server full text search.
Apart from that, another, less elegant and performant solution would be to use table valued function STRING_SPLIT() (available in SQL Server 2016), to turn the search words to a derived table, and then search each individual word in the text using LIKE. If all searched words are present in the text, then you have a match.
Consider:
SELECT *
FROM mytable t
WHERE NOT EXISTS (
SELECT 1
FROM STRING_SPLIT(#search, ' ') x
WHERE t.txt NOT LIKE '%' + x.value + '%'
);
Demo on DB Fiddle
CREATE TABLE mytable (id int, txt varchar(max));
INSERT INTO mytable VALUES
(1, 'Widget Brown Bull Edge Stone 400x500x600'),
(2, 'Widget Brown Bull Snow Stone 700x500x300');
DECLARE #search NVARCHAR(400) = 'Widget Brown 400';
SELECT *
FROM mytable t
WHERE NOT EXISTS (
SELECT 1
FROM STRING_SPLIT(#search, ' ') x
WHERE t.txt NOT LIKE '%' + x.value + '%'
);
id | txt
-: | :---------------------------------------
1 | Widget Brown Bull Edge Stone 400x500x600

How do you concat multiple rows into one column in SQL Server?

I've searched high and low for the answer to this, but I can't figure it out. I'm relatively new to SQL Server and don't quite have the syntax down yet. I have this datastructure (simplified):
Table "Users" | Table "Tags":
UserID UserName | TagID UserID PhotoID
1 Bob | 1 1 1
2 Bill | 2 2 1
3 Jane | 3 3 1
4 Sam | 4 2 2
-----------------------------------------------------
Table "Photos": | Table "Albums":
PhotoID UserID AlbumID | AlbumID UserID
1 1 1 | 1 1
2 1 1 | 2 3
3 1 1 | 3 2
4 3 2 |
5 3 2 |
I'm looking for a way to get the all the photo info (easy) plus all the tags for that photo concatenated like CONCAT(username, ', ') AS Tags of course with the last comma removed. I'm having a bear of a time trying to do this. I've tried the method in this article but I get an error when I try to run the query saying that I can't use DECLARE statements... do you guys have any idea how this can be done? I'm using VS08 and whatever DB is installed in it (I normally use MySQL so I don't know what flavor of DB this really is... it's an .mdf file?)
Ok, I feel like I need to jump in to comment about How do you concat multiple rows into one column in SQL Server? and provide a more preferred answer.
I'm really sorry, but using scalar-valued functions like this will kill performance. Just open SQL Profiler and have a look at what's going on when you use a scalar-function that calls a table.
Also, the "update a variable" technique for concatenation is not encouraged, as that functionality might not continue in future versions.
The preferred way of doing string concatenation to use FOR XML PATH instead.
select
stuff((select ', ' + t.tag from tags t where t.photoid = p.photoid order by tag for xml path('')),1,2,'') as taglist
,*
from photos
order by photoid;
For examples of how FOR XML PATH works, consider the following, imagining that you have a table with two fields called 'id' and 'name'
SELECT id, name
FROM table
order by name
FOR XML PATH('item'),root('itemlist')
;
Gives:
<itemlist><item><id>2</id><name>Aardvark</a></item><item><id>1</id><name>Zebra</name></item></itemlist>
But if you leave out the ROOT, you get something slightly different:
SELECT id, name
FROM table
order by name
FOR XML PATH('item')
;
<item><id>2</id><name>Aardvark</a></item><item><id>1</id><name>Zebra</name></item>
And if you put an empty PATH string, you get even closer to ordinary string concatenation:
SELECT id, name
FROM table
order by name
FOR XML PATH('')
;
<id>2</id><name>Aardvark</a><id>1</id><name>Zebra</name>
Now comes the really tricky bit... If you name a column starting with an # sign, it becomes an attribute, and if a column doesn't have a name (or you call it [*]), then it leaves out that tag too:
SELECT ',' + name
FROM table
order by name
FOR XML PATH('')
;
,Aardvark,Zebra
Now finally, to strip the leading comma, the STUFF command comes in. STUFF(s,x,n,s2) pulls out n characters of s, starting at position x. In their place, it puts s2. So:
SELECT STUFF('abcde',2,3,'123456');
gives:
a123456e
So now have a look at my query above for your taglist.
select
stuff((select ', ' + t.tag from tags t where t.photoid = p.photoid order by tag for xml path('')),1,2,'') as taglist
,*
from photos
order by photoid;
For each photo, I have a subquery which grabs the tags and concatenates them (in order) with a commma and a space. Then I surround that subquery in a stuff command to strip the leading comma and space.
I apologise for any typos - I haven't actually created the tables on my own machine to test this.
Rob
I'd create a UDF:
create function GetTags(PhotoID int) returns #tags varchar(max)
as
begin
declare #mytags varchar(max)
set #mytags = ''
select #mytags = #mytags + ', ' + tag from tags where photoid = #photoid
return substring(#mytags, 3, 8000)
end
Then, all you have to do is:
select GetTags(photoID) as tagList from photos
Street_Name ; Street_Code
west | 14
east | 7
west+east | 714
If want to show two different row concat itself , how can do it?
(I mean last row i want to show from select result. My table had first and secord record)