Related
I have the following dataset in BigQuery: Dataset
When the type is V, count is always equal to zero.
When the type is V, I would like the column count to get the first value under this row with type T.
The rows are ordered according to group_id and position column.
This is the final result I would like to have: Desired dataset
I tried this
FIRST_VALUE( count )
OVER (
PARTITION BY id_group,id_person
ORDER BY
CASE WHEN type LIKE "T" THEN 1 ELSE 0 END DESC,
position
ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
) as NEW_count
but it always gives me the first count with a type T, when I want the first value below the row.
I don't think this scenario can be solved using navigation functions, since the closest T value position is not deterministic (lead, lag, first, last, nth value).
You need to query the same dataset to find the closest t_value using subqueries.
Here you have a working example:
WITH source_data AS (
SELECT 'A1' AS id_group, 'a' AS id_perso, 1 AS position, 'V' AS type, 0 AS count
UNION ALL
SELECT 'A1', 'b', 2, 'V', 0
UNION ALL
SELECT 'A1', 'c', 3, 'T', 13
UNION ALL
SELECT 'A1', 'd', 4, 'V', 0
UNION ALL
SELECT 'A1', 'e', 5, 'T', 5
UNION ALL
SELECT 'A1', 'f', 6, 'T', 7
UNION ALL
SELECT 'A1', 'g', 7, 'V', 0
UNION ALL
SELECT 'A1', 'h', 8, 'V', 0
UNION ALL
SELECT 'A1', 'i', 9, 'V', 0
UNION ALL
SELECT 'A1', 'j', 10,' 'T, 0
)
SELECT *,
(SELECT count FROM source_data counts WHERE counts.position =
(SELECT MIN(t_values.position) FROM source_data t_values WHERE t_values.type='T' and t_values.position > source.position))
FROM source_data source
You can coalesce the t_value if you need 0s instead of nulls
You can consider the below query for your requirement.
with cte as (
select 'A1' id_group, 'a' id_person, 1 position, 'V'type, 0 count union all
select 'A1','b',2,'V',0 union all
select 'A1','c',3,'T',13 union all
select 'A1','d',4,'V',0 union all
select 'A1','e',5,'T',5 union all
select 'A1','f',6,'T',7 union all
select 'A1','g',7,'V',0 union all
select 'A1','h',8,'V',0 union all
select 'A1','i',9,'V',0 union all
select 'A1','j',10,'T',0
)
select *,last_value(count_1 ignore nulls) over (order by position desc) new_count,
from (select *,case when type='V' and count=0 then null else count
end count_1
from cte
)
order by position
Is there a way to generate list of missing dates in table in Oracle?
Input
name,my_date
A,04-JAN-2000
A,05-JAN-2000
A,08-JAN-2000
A,08-JAN-2000 -- duplicates possible
A,10-JAN-2000
B,09-FEB-2001
B,10-FEB-2001
B,05-FEB-2001
Result
A,06-JAN-2000
A,07-JAN-2000
A,09-JAN-2000
B,06-FEB-2001
B,07-FEB-2001
B,08-FEB-2001
After suggestion from #diiN__________ to see Oracle: select missing dates, I managed to get it working for a specific name as follows:
WITH all_dates_wo_boundary_values as
(SELECT oldest + level my_date
FROM (SELECT MIN(my_date) oldest
,MAX(my_date) recent
FROM mytable my
WHERE my.name = 'A'
)
connect by level <= recent - oldest - 1
)
SELECT my_date
FROM all_dates_wo_boundary_values
MINUS
SELECT my_date
FROM mytable my
WHERE my.name = 'A'
How could it be done for multiple names at once?
For multiple names, you can use the LEAD analytic function to find the next date and then CROSS JOIN LATERAL (available from Oracle 12) a row-generator to generate the missing values:
SELECT t.name,
m.missing
FROM (
SELECT name,
dt,
LEAD(dt) OVER (PARTITION BY name ORDER BY dt) AS next_dt
FROM table_name
) t
CROSS JOIN LATERAL (
SELECT dt + LEVEL AS missing
FROM DUAL
WHERE dt + 1 < next_dt
CONNECT BY dt + LEVEL < next_dt
) m
Which, for the sample data:
CREATE TABLE table_name (Name,dt) AS
SELECT 'A', DATE '2000-01-04' FROM DUAL UNION ALL
SELECT 'A', DATE '2000-01-05' FROM DUAL UNION ALL
SELECT 'A', DATE '2000-01-08' FROM DUAL UNION ALL
SELECT 'A', DATE '2000-01-08' FROM DUAL UNION ALL
SELECT 'A', DATE '2000-01-10' FROM DUAL UNION ALL
SELECT 'B', DATE '2001-02-05' FROM DUAL UNION ALL
SELECT 'B', DATE '2001-02-09' FROM DUAL UNION ALL
SELECT 'B', DATE '2001-02-10' FROM DUAL;
Outputs:
NAME
MISSING
A
06-JAN-00
A
07-JAN-00
A
09-JAN-00
B
06-FEB-01
B
07-FEB-01
B
08-FEB-01
db<>fiddle here
In my database that represents a car service station, I am trying to figure out a SQL query that would give me a total average of how much does the customer pays for a single service but instead of getting AVG() of the price on all existing Invoices, I want to group the invoices by the same reservation_id. After that, I would like to get the total average of all of those grouped results.
I am using the two tables listed in the picture below. I want to get the value of a total average price by applying AVG() on all averages that are made by grouping prices by the same FK Reservation_reservation_id.
I tried to make this into a single query but I failed so I came looking for help from more experienced users. Also, I need to select (get) only the result of the total average. This result should give me an overview of how much each customer pays on average for one reservation.
Thanks for your time
You appear to want to aggregate twice:
SELECT AVG( avg_price ) avg_avg_price
FROM (
SELECT AVG( price ) AS avg_price
FROM invoice
GROUP BY reservation_reservation_id
)
Which, for the sample data:
CREATE TABLE invoice ( reservation_reservation_id, price ) AS
SELECT 1, 10 FROM DUAL UNION ALL
SELECT 1, 12 FROM DUAL UNION ALL
SELECT 1, 14 FROM DUAL UNION ALL
SELECT 1, 16 FROM DUAL UNION ALL
SELECT 2, 10 FROM DUAL UNION ALL
SELECT 2, 11 FROM DUAL UNION ALL
SELECT 2, 12 FROM DUAL;
Outputs:
AVG_AVG_PRICE
12
db<>fiddle here
If you want this per customer:
SELECT customer_customer_id, AVG(avg_reservation_price)
FROM (SELECT i.customer_customer_id, i.reservation_reservation_id,
AVG(i.price) as avg_reservation_price
FROM invoice i
GROUP BY i.customer_customer_id, i.reservation_reservation_id
) ir
GROUP BY customer_customer_id;
If you want this for a particular "checkout reason" -- which is the closest that I imagine that "service" means -- then join in the reservations table and filter:
SELECT customer_customer_id, AVG(avg_reservation_price)
FROM (SELECT i.customer_customer_id, i.reservation_reservation_id,
AVG(i.price) as avg_reservation_price
FROM invoice i JOIN
reservation r
ON i.reservation_reservation_id = r.reservation_id
WHERE r.checkup_type = ?
GROUP BY i.customer_customer_id, i.reservation_reservation_id
) ir
GROUP BY customer_customer_id;
You might want to try the below:
with aux (gr, subgr, val) as (
select 'a', 'a1', 1 from dual union all
select 'a', 'a2', 2 from dual union all
select 'a', 'a3', 3 from dual union all
select 'a', 'a4', 4 from dual union all
select 'b', 'b1', 5 from dual union all
select 'b', 'b2', 6 from dual union all
select 'b', 'b3', 7 from dual union all
select 'b', 'b4', 8 from dual)
SELECT
gr,
avg(val) average_gr,
avg(avg(val)) over () average_total
FROM
aux
group by gr;
Which, applied to your table, would result in:
SELECT
reservation_id,
avg(price) average_rn,
avg(avg(price)) over () average_total
FROM
invoices
group by reservation_id;
I have a table in BigQuery
SELECT 1 as big_id, 1 as temp_id, '101' as names
UNION ALL SELECT 1,1, 'z3Awwer',
UNION ALL SELECT 1,1, 'gA1sd03',
UNION ALL SELECT 1,2, 'z3Awwer',
UNION ALL SELECT 1,2, 'gA1sd03',
UNION ALL SELECT 1,3, 'gA1sd03',
UNION ALL SELECT 1,3, 'sAs10sdf4',
UNION ALL SELECT 1,4, 'sAs10sdf4',
UNION ALL SELECT 1,5, 'Adf105',
UNION ALL SELECT 2,1, 'A1sdf02',
UNION ALL SELECT 2,1, '345A103',
UNION ALL SELECT 2,2, '345A103',
UNION ALL SELECT 2,2, 'A1sd04',
UNION ALL SELECT 2,3, 'A1sd04',
UNION ALL SELECT 2,4, '6_0Awe105'
I want to filter it by temp_id if all names of one temp_id included in some another temp_id in partition by big_id window. For example I do not need to select all rows where temp_id = 2 because all names of temp_id = 2 included in temp_id = 1. As well as need to keep all rows of temp_id = 1 because this names range covers names range of temp_id = 2
So expected output:
SELECT 1 as big_id, 1 as temp_id, '101' as names
UNION ALL SELECT 1,1, 'z3Awwer',
UNION ALL SELECT 1,1, 'gA1sd03',
UNION ALL SELECT 1,3, 'gA1sd03',
UNION ALL SELECT 1,3, 'sAs10sdf4',
UNION ALL SELECT 1,5, 'Adf105',
UNION ALL SELECT 2,1, 'A1sdf02',
UNION ALL SELECT 2,1, '345A103',
UNION ALL SELECT 2,2, '345A103',
UNION ALL SELECT 2,2, 'A1sd04',
UNION ALL SELECT 2,4, '6_0Awe105'
How can I make it using BigQuery?
Below is for BigQuery Standard SQL
#standardsql
with temp as (
select big_id, temp_id, array_agg(names) names
from `project.dataset.table`
group by big_id, temp_id
)
select big_id, temp_id, names
from (
select big_id, temp_id, any_value(names) names
from (
select t1.*,
( select count(1)
from t1.names name
join t2.names name
using(name)
where t1.temp_id != t2.temp_id
) = array_length(t1.names) as flag
from temp t1
join temp t2
using (big_id)
)
group by big_id, temp_id
having countif(flag) = 0
), unnest(names) names
If to apply above to sample data from your question - the output is
I need to aggregate consecutive values in a table with BigQuery, as shown in the example
Segment can be only 'A' or 'B'. Value is a String.
Basically, for each id i need to consider only segment='A' taking into account the gaps.
It should be ORDER BY date_column ASC
Example
id, segment, value, date_column
1, A, 3, daytime
1, A, 2, daytime
1, A, x, daytime
1, B, 3, daytime
1, B, 3, daytime
1, B, 3, daytime
1, A, 7, daytime
1, A, 3, daytime
1, B, 3, daytime
1, A, 9, daytime
1, A, 9, daytime
2, A, 3, daytime
2, B, 3, daytime
2, A, 3, daytime
2, A, m, daytime
Expected result
id, agg_values_A_segment
1, ['32x', '73', '99']
2, ['3', '3m']
How can I achieve this result?
I'm struggling with the 'gap' between the segments.
Below options for BigQuery Standard SQL
Option 1 - using window analytics functions
#standardSQL
SELECT id, ARRAY_AGG(values_in_group ORDER BY grp) agg_values_A_segment
FROM (
SELECT id, grp, STRING_AGG(value, '' ORDER BY date_column) values_in_group
FROM (
SELECT id, segment, value, date_column, flag,
COUNTIF(flag) OVER(PARTITION BY id ORDER BY date_column) grp
FROM (
SELECT *, IFNULL(LAG(segment) OVER(PARTITION BY id ORDER BY date_column), segment) != segment flag
FROM `project.dataset.table`
)
)
WHERE segment = 'A'
GROUP BY id, grp
)
GROUP BY id
You can test, play with above using sample data from your question as in below example:
#standardSQL
WITH `project.dataset.table` AS (
SELECT 1 id, 'A' segment, '3' value, DATETIME '2019-01-07T18:46:21' date_column UNION ALL
SELECT 1, 'A', '2', '2019-01-07T18:46:22' UNION ALL
SELECT 1, 'A', 'x', '2019-01-07T18:46:23' UNION ALL
SELECT 1, 'B', '3', '2019-01-07T18:46:24' UNION ALL
SELECT 1, 'B', '3', '2019-01-07T18:46:25' UNION ALL
SELECT 1, 'B', '3', '2019-01-07T18:46:26' UNION ALL
SELECT 1, 'A', '7', '2019-01-07T18:46:27' UNION ALL
SELECT 1, 'A', '3', '2019-01-07T18:46:28' UNION ALL
SELECT 1, 'B', '3', '2019-01-07T18:46:29' UNION ALL
SELECT 1, 'A', '9', '2019-01-07T18:46:30' UNION ALL
SELECT 1, 'A', '9', '2019-01-07T18:46:31' UNION ALL
SELECT 2, 'A', '3', '2019-01-07T18:46:32' UNION ALL
SELECT 2, 'B', '3', '2019-01-07T18:46:33' UNION ALL
SELECT 2, 'A', '3', '2019-01-07T18:46:34' UNION ALL
SELECT 2, 'A', 'm', '2019-01-07T18:46:35'
)
SELECT id, ARRAY_AGG(values_in_group ORDER BY grp) agg_values_A_segment
FROM (
SELECT id, grp, STRING_AGG(value, '' ORDER BY date_column) values_in_group
FROM (
SELECT id, segment, value, date_column, flag,
COUNTIF(flag) OVER(PARTITION BY id ORDER BY date_column) grp
FROM (
SELECT *, IFNULL(LAG(segment) OVER(PARTITION BY id ORDER BY date_column), segment) != segment flag
FROM `project.dataset.table`
)
)
WHERE segment = 'A'
GROUP BY id, grp
)
GROUP BY id
-- ORDER BY id
with result
Row id agg_values_A_segment
1 1 32x
73
99
2 2 3
3m
Option 2 - above option should work for big volumes of rows per id, but looks a little heavy - so second option is more of simple option but assumes you have some character or sequence of chars that you sure will not be result from combining your values, for example pipe char or tab or as in below example I choose word 'delimiter' assuming it will not appear as a result of concatenation
#standardSQL
SELECT id,
ARRAY(SELECT part FROM UNNEST(parts) part WHERE part != '') agg_values_A_segment
FROM (
SELECT id,
SPLIT(STRING_AGG(IF(segment = 'A', value, 'delimiter'), ''), 'delimiter') parts
FROM `project.dataset.table`
GROUP BY id
)
You can test, play with above using same sample data:
#standardSQL
WITH `project.dataset.table` AS (
SELECT 1 id, 'A' segment, '3' value, DATETIME '2019-01-07T18:46:21' date_column UNION ALL
SELECT 1, 'A', '2', '2019-01-07T18:46:22' UNION ALL
SELECT 1, 'A', 'x', '2019-01-07T18:46:23' UNION ALL
SELECT 1, 'B', '3', '2019-01-07T18:46:24' UNION ALL
SELECT 1, 'B', '3', '2019-01-07T18:46:25' UNION ALL
SELECT 1, 'B', '3', '2019-01-07T18:46:26' UNION ALL
SELECT 1, 'A', '7', '2019-01-07T18:46:27' UNION ALL
SELECT 1, 'A', '3', '2019-01-07T18:46:28' UNION ALL
SELECT 1, 'B', '3', '2019-01-07T18:46:29' UNION ALL
SELECT 1, 'A', '9', '2019-01-07T18:46:30' UNION ALL
SELECT 1, 'A', '9', '2019-01-07T18:46:31' UNION ALL
SELECT 2, 'A', '3', '2019-01-07T18:46:32' UNION ALL
SELECT 2, 'B', '3', '2019-01-07T18:46:33' UNION ALL
SELECT 2, 'A', '3', '2019-01-07T18:46:34' UNION ALL
SELECT 2, 'A', 'm', '2019-01-07T18:46:35'
)
SELECT id,
ARRAY(SELECT part FROM UNNEST(parts) part WHERE part != '') agg_values_A_segment
FROM (
SELECT id,
SPLIT(STRING_AGG(IF(segment = 'A', value, 'delimiter'), ''), 'delimiter') parts
FROM `project.dataset.table`
GROUP BY id
)
-- ORDER BY id
obviously with same result
Row id agg_values_A_segment
1 1 32x
73
99
2 2 3
3m
note: second option can result with resources exceeded for case when you have too many rows per id - you just need to try it on your real data
SQL tables represent unordered sets. This is particularly true in a parallel, columnar database such as BigQuery. The rest of this answer assumes you have a column that specifies the ordering of the rows.
This is a gaps-and-islands problem. You can use the difference of row_number() to identify the adjacent groups . . . and then aggregation:
select id, array_agg(vals order by min_ordercol)
from (select id, segment, string_agg(value delimiter '' order by date_column) as vals,
min(<ordercol>) as min_ordercol
from (select t.*,
row_number() over (partition by id order by date_column) as seqnum,
row_number() over (partition by id, segment order by date_column) as seqnum_2,
from t
) t
group by id, segment, (seqnum - seqnum_2)
) x
group by id;