Related
I am using the code below to get the quarterly wages for individuals from 2010Q1-2020Q4. If an individual did not work in a particular quarter they do not have an observation for that quarter. Instead, I would like for there to be an observation but have the quarterly wage be 0. For example,
What is currently happening:
| MPI | Quarter| Wage|
|PersonA|2010Q1 | 100 |
|PersonA|2010Q2 | 100 |
|PersonA|2010Q3 | 100 |
|PersonB|2010Q1 | 100 |
Desired output
| MPI | Quarter| Wage|
|PersonA|2010Q1 | 100 |
|PersonA|2010Q2 | 100 |
|PersonA|2010Q3 | 100 |
|PersonA|2010Q4 | 0 |
|PersonB|2010Q1 | 100 |
|PersonB|2010Q2 | 0 |
|PersonB|2010Q3 | 0 |
|PersonB|2010Q4 | 0 |
ws_data AS (
SELECT
MASTER_PERSON_INDEX AS mpi
,SUBSTR(cast(wg.naics as string), 1, 2) AS NAICS_2
,SUBSTR(cast(wg.yrqtr as string), 0,5) AS quarter
,wg.yrqtr
,wg.employer
,wg.wages
,SUBSTR(cast(wg.yrqtr as string), 0,4) AS YEAR
FROM
( SELECT
*
FROM
`ws.ws_ui_wage_records_di` wsui
WHERE
wsui.MASTER_PERSON_INDEX IN (SELECT mpi FROM rc_table_ra16_all_grads_1b)
AND
wsui.yrqtr IN (20101, 20102, 20103, 20104,
20111, 20112, 20113, 20114,
20121, 20122, 20123, 20124,
20131, 20132, 20133, 20134,
20141, 20142, 20143, 20144,
20151, 20152, 20153, 20154,
20161, 20162, 20163, 20164,
20171, 20172, 20173, 20174,
20181, 20182, 20183, 20184,
20191, 20192, 20193, 20194,
20201, 20202, 20203, 20204)
)wg
),
ws_agg AS (
SELECT
mpi
-- ,STATS_MODE(NAICS_2) AS NAICS_2
-- ,STATS_MODE(NAICS_DESC) AS NAICS_DESC
,quarter
,SUM(wages) AS wages_quart
FROM
ws_data
GROUP BY
mpi, quarter
),
ws_annot AS (
SELECT
dagg.*
,row_number() OVER(PARTITION BY dagg.mpi, cast(wages_quart as string) ORDER BY dagg.wages_quart DESC)AS rn
FROM
ws_agg dagg
)
Try using this data to create a CTE at the top as a Quarter table and then using that as the starting point in your main from statement. You should be able to replace the original code I copied from (wg where statement) with that top CTE as well.
(20101, 20102, 20103, 20104,
20111, 20112, 20113, 20114,
20121, 20122, 20123, 20124,
20131, 20132, 20133, 20134,
20141, 20142, 20143, 20144,
20151, 20152, 20153, 20154,
20161, 20162, 20163, 20164,
20171, 20172, 20173, 20174,
20181, 20182, 20183, 20184,
20191, 20192, 20193, 20194,
20201, 20202, 20203, 20204)
Your db may have a DateDimension table with quarters in it that you could use as well.
Since you want all quarters, and all individuals, one way to achieve this is to start with building all individual-quarter combinations in your data and use that as a 'driver' in a left join; like this:
select
Pers.MID
, Qtr.Quarter
, coalesce(W.Wage,0) as Wage
, ...
from
(select distinct MPIfrom YourTable) Pers
cross join
(select distinct Quarter from DateDimensionTable) Qtr
left join
YourTable W
on w.MPI=Pers.MPI
and w.Quarter=Qtr.Quarter
If your table has all the periods you are interested in, you can use YourTable, instead of DateDimensionTable. But if it doesn't, and I guess it can't be guaranteed, then you can use a Date/Calendar table here , if you have any, or you can dynamically generate quarters between min and max quarter in YourTable; just search for these terms). You can also hardcode them as you have in your query (as JBontje recommended).
If a combination is missing from YourTable then the Wage for that combo will be null, you can use coalesce to treat it as zero.
I am working in Google BigQuery and have a dataset with two repeated records: one is the element name and the other is the element value. A simplified version of my dataset (with just 2 entries) looks like this:
globalId
Meta.name
Meta.value
9200000104
ViewsL7D
2877
OldPrice
33.47
NewPrice
33.21
9200000783
ViewsL7D
19231
OldPrice
14.27
NewPrice
12.11
What I want to get is a table like this:
globalId
ViewsL7D
OldPrice
NewPrice
9200000104
2877
33.47
33.21
9200000783
19231
14.27
12.11
I haven't worked with repeated records before so I don't really know how to tackle this, but I have tried using the following code so far:
SELECT * FROM table LEFT JOIN UNNEST(cloudmetadata)
But when doing this I keep the repeated records and just add six extra columns, alternating between the names and values. Can someone help?
Consider below approach
select globalId,
max(if(name = 'ViewsL7D', value, null)) as ViewsL7D,
max(if(name = 'OldPrice', value, null)) as OldPrice,
max(if(name = 'NewPrice', value, null)) as NewPrice
from your_table t, t.Meta
group by globalId
if applied to sample data in your question
output is
you can simply pivot the table
if you have more than these three meat name you have to use dynamic sql
CREATE TABLE table1 (
`globalId` varchar(10),
`Meta.name` VARCHAR(8),
`Meta.value` DECIMAL(10,2)
);
✓
INSERT INTO table1
(`globalId`, `Meta.name`, `Meta.value`)
VALUES
('9200000104', 'ViewsL7D', '2877'),
('9200000104', 'OldPrice', '33.47'),
('9200000104', 'NewPrice', '3321'),
('9200000783', 'ViewsL7D', '19231'),
('9200000783', 'OldPrice', '14.27'),
('9200000783', 'NewPrice', '12.11');
SELECT
`globalId`,
MAX(IF (`Meta.name` = 'ViewsL7D',`Meta.value`, NULL )) AS ViewsL7D,
MAX(IF (`Meta.name` = 'OldPrice',`Meta.value`, NULL )) AS OldPrice,
MAX(IF (`Meta.name` = 'NewPrice',`Meta.value`, NULL )) AS NewPrice
FROM table1
GROUP BY `globalId`
globalId | ViewsL7D | OldPrice | NewPrice
:--------- | -------: | -------: | -------:
9200000104 | 2877.00 | 33.47 | 3321.00
9200000783 | 19231.00 | 14.27 | 12.11
db<>fiddle here
Given the shape of your table, I suppose you have two arrays name and value kept in a Meta structure. I recreated the table with the following:
WITH
mydata AS (
SELECT
9200000104 AS globalid,
STRUCT(["ViewsL7D",
"OldPrice",
"NewPrice"] AS name,
[2877,
33.47,
33.21] AS value) AS Meta
UNION ALL
SELECT
9200000783 AS globalid,
STRUCT(["ViewsL7D",
"OldPrice",
"NewPrice"] AS name,
[19231,
14.27,
12.11] AS value) AS Meta )
From there you want to unnest your array and pivot it:
SELECT
globalid,
MAX(IF(name="ViewsL7D",value, NULL)) AS ViewsL7D,
MAX(IF(name="OldPrice",value, NULL)) AS OldPrice,
MAX(IF(name="NewPrice",value, NULL)) AS NewPrice
FROM
(SELECT
globalid,
Meta.name[OFFSET(offset_value)] name,
Meta.value[OFFSET(offset_value)] value
FROM mydata,
UNNEST(Meta.value) value WITH OFFSET offset_value ORDER BY offset_value)
GROUP BY globalid
ORDER BY
globalid
It gives this
I need to concatenate values of the column content. I tried to concatenate them using listagg, but it doesn't work. I don't know why because it doesn't return errors. Probably exists a better option, than listtagg to concatenate them
I had three essential columns:
ID, Step_id, timestamp and content. Content is split when its length exceed certain length.
When content is divided then id, step_id is the same but timestamp is other.
For example:
Id | step_id | timestamp | content|
1 | A | 15:21:21 | ABCDEFG|
1 | A | 15:21:22 | HIJK|
I try to connect this in:
Id | step_id | content |
1 | A | ABCDEFGHIJK |
I've tried to use row_number and join parts but it doesn't work and it is logical.
My code:
With dane as
(
select id,step_id,row_number() over (partition by id, step_id, order by timestamp) as rno, cast(content as varchar(15024)) as content from xya),
dane1 as (select id, content as con1,step_id from dane where rno=1)
dane2 as (select id, content as con2 ,step_id from dane where rno=2)
dane3 as (select id, content as con3 ,step_id from dane where rno=3)
dane4 as (select id, content as con4,step_id from dane where rno=4)
select dane3. id, con1||con2||con3||con4, dane1.step_id from
dane1 left join dane 2 on dane1.id=dane2.id and dane1.step_id=dane2.step_id
left join dane3 on dane3.id=dane2.id and dane3.step_id=dane2.step_id
Try this one:
select id, step_id, listagg(content)within group(order by timestamp) content
from xya
group by id,step_id;
Simple example with test data:
with xya( Id , step_id , timestamp , content) as (
select 1 , 'A' , '15:21:21' , 'ABCDEFG' from dual union all
select 1 , 'A' , '15:21:22' , 'HIJK' from dual
)
select id, step_id, listagg(content)within group(order by timestamp) content
from xya
group by id,step_id;
ID STEP_ID CONTENT
---------- ------- ------------------------------
1 A ABCDEFGHIJK
I try IT earlier but its dosent work. Propably due to format data = clob (9750 char). On normal data its work but not this case. Answers from program when i try use to-char: the the data type, length, od value od argument 1 of to-char is invalid SQL code=171 SQLstate=42815 driver 3.53.95.
I work on db2
When i run without to-char answers from program is attempt to use a function when the aplication compability settings is set for a previus level. SQLcode =4743 SQLstate =56038 driver 3.53.95
I have a table in postgresql with a text column that has values like this:
column
-----------
CA;TB;BA;CB
XA;VA
GA;BA;LA
I want to sort the elements that are in each value, so that the query results like this:
column
-----------
BA;CA;CB;TB
VA;XA
BA;GA;LA
I have tried with string_to_array, regexp_split_to_array, array_agg, but I don't seem to get close to it.
Thanks.
I hope this would be easy to understand:
WITH tab AS (
SELECT
*
FROM
unnest(ARRAY[
'CA;TB;BA;CB',
'XA;VA',
'GA;BA;LA']) AS txt
)
SELECT
string_agg(val, ';')
FROM (
SELECT
txt,
regexp_split_to_table(txt, ';') AS val
FROM
tab
ORDER BY
val
) AS sub
GROUP BY
txt;
First I split values to rows (regexp_split_to_table) and sort. Then group by original values and join again with string_agg.
Output:
BA;CA;CB;TB
BA;GA;LA
VA;XA
I'm probably overcomplicating it:
t=# with a(c)as (values('CA;TB;BA;CB')
,('XA;VA')
,('GA;BA;LA'))
, w as (select string_agg(v,';') over (partition by c order by v), row_number() over (partition by c),count(1) over(partition by c) from a,unnest(string_to_array(a.c,';')) v)
select * from w where row_number = count;
string_agg | row_number | count
-------------+------------+-------
BA;CA;CB;TB | 4 | 4
BA;GA;LA | 3 | 3
VA;XA | 2 | 2
(3 rows)
and here with a little ugly hack:
with a(c)as (values
('CA;TB;BA;CB')
,('XA;VA')
,('GA;BA;LA'))
select translate(array_agg(v order by v)::text,',{}',';') from a, unnest(string_to_array(a.c,';')) v group by c;
translate
-------------
BA;CA;CB;TB
BA;GA;LA
VA;XA
(3 rows)
Could someone assist me with writing SQL query to calculate/display the following example:
table - shipment
columns - product code / qty / unique code
sku a | 5 | nnnn
sku a | 5 | nn
sku a | 10 | (blank)
sku b | 2 | nnn
sku c | 2 | (blank)
sku c | 2 | (blank)
I'm looking for an output like this:
columns - product code / qty / unique code
sku a | 20 | nnnn, nn
sku b | 2 | nnn
sku c | 4 | (blank)
LISTAGG is your saviour here.
SELECT
product_code,
SUM(qty) as total_qty,
LISTAGG(unique_code, ',') WITHIN GROUP (ORDER BY unique_code)
FROM
shipment
GROUP BY
product_code
EDIT:
Putting the answer here for the better code output:
You have "i." in front of the fields in your listagg, but you have no table aliases. Also, you need to add pallet_id to the group by. Try this
SELECT
reference_id,
pallet_id,
SUM(update_qty) as total_qty,
LISTAGG(user_def_type_1, ',') WITHIN GROUP (ORDER BY user_def_type_1)
FROM
inventory_transaction
WHERE
code = 'Shipment' AND site_id = 'GBRUN2A' AND client_id = '021' AND dstamp >= current_date -21
GROUP BY
reference_id, pallet_id
If you are still getting an error, can you confirm you are on Oracle? I'm pretty sure it's an Oracle-only function.
Yes - Using Oracle Sql Developer version 4.0.1.14
ETA: Can you confirm which version of Oracle Database you are running. Listagg is only on Oracle 12c and Oracle 11g Release 2. If you are running a previous version, have a look here for some alternate ideas.
If you're using a version of Oracle that doesn't support LISTAGG() (e.g., Oracle 10g), then there are a couple of things you can do. The easiest is to use the undocumented WM_CONCAT() function (which returns either a VARCHAR or a CLOB depending on the release):
SELECT reference_id, pallet_id, SUM(update_qty) as total qty
, WM_CONCAT(user_def_type_1)
FROM inventory_transaction
GROUP BY reference_id
One difficulty with using WM_CONCAT() is that the results of the concatenation won't be ordered. You also have no choice about your delimiter. Another option, probably a better one, is to use XMLAGG() (this is actually a documented function):
SELECT reference_id, pallet_id, SUM(update_qty) as total qty
, XMLAGG(XMLELEMENT(e, user_def_type_1 || ',')).EXTRACT('//text()')
FROM inventory_transaction
GROUP BY reference_id;
Here you have your choice of delimiters, and XMLAGG() supports an ORDER BY clause:
SELECT reference_id, pallet_id, SUM(update_qty) as total qty
, XMLAGG(XMLELEMENT(e, user_def_type_1 || ',') ORDER BY user_def_type_1).EXTRACT('//text()')
FROM inventory_transaction
GROUP BY reference_id;
You can find other options at this Stack Overflow question.