If I use LIMIT 10 in a SQL query (using BigQuery), is there a way to also return the total count?
For example, 100 rows exist. How can I query to return the first 10 but also display to users how many rows are available in total without doing a separate count(id) aggregate query?
To add to Mikhail's answer, you may want to do this to see the count of the unique values in a grouped query. In the following example, there are 10 unique values of R, but you only want to see the the first 4, along with the count of the unique rows. I also added showing the count for each group and overall count of every row. (Standard SQL below)
WITH YourTable AS (
SELECT 1 AS r UNION ALL
SELECT 3 AS r UNION ALL
SELECT 4 AS r UNION ALL
SELECT 4 AS r UNION ALL
SELECT 4 AS r UNION ALL
SELECT 5 AS r UNION ALL
SELECT 6 AS r UNION ALL
SELECT 7 AS r UNION ALL
SELECT 8 AS r UNION ALL
SELECT 9 AS r UNION ALL
SELECT 1 AS r UNION ALL
SELECT 2 AS r UNION ALL
SELECT 3 AS r UNION ALL
SELECT 4 AS r UNION ALL
SELECT 5 AS r UNION ALL
SELECT 6 AS r UNION ALL
SELECT 7 AS r UNION ALL
SELECT 8 AS r UNION ALL
SELECT 9 AS r UNION ALL
SELECT 1 AS r UNION ALL
SELECT 2 AS r UNION ALL
SELECT 3 AS r UNION ALL
SELECT 4 AS r UNION ALL
SELECT 5 AS r UNION ALL
SELECT 6 AS r UNION ALL
SELECT 7 AS r UNION ALL
SELECT 5 AS r UNION ALL
SELECT 6 AS r UNION ALL
SELECT 7 AS r UNION ALL
SELECT 8 AS r UNION ALL
SELECT 9 AS r UNION ALL
SELECT 1 AS r UNION ALL
SELECT 5 AS r UNION ALL
SELECT 6 AS r UNION ALL
SELECT 7 AS r UNION ALL
SELECT 8 AS r UNION ALL
SELECT 9 AS r UNION ALL
SELECT 1 AS r UNION ALL
SELECT 2 AS r UNION ALL
SELECT 3 AS r UNION ALL
SELECT 5 AS r UNION ALL
SELECT 6 AS r UNION ALL
SELECT 7 AS r UNION ALL
SELECT 8 AS r UNION ALL
SELECT 9 AS r UNION ALL
SELECT 1 AS r UNION ALL
SELECT 2 AS r UNION ALL
SELECT 3 AS r UNION ALL
SELECT 5 AS r UNION ALL
SELECT 6 AS r UNION ALL
SELECT 7 AS r UNION ALL
SELECT 8 AS r UNION ALL
SELECT 9 AS r UNION ALL
SELECT 1 AS r UNION ALL
SELECT 2 AS r UNION ALL
SELECT 3 AS r UNION ALL
SELECT 8 AS r UNION ALL
SELECT 9 AS r UNION ALL
SELECT 1 AS r UNION ALL
SELECT 2 AS r UNION ALL
SELECT 3 AS r UNION ALL
SELECT 4 AS r UNION ALL
SELECT 5 AS r UNION ALL
SELECT 6 AS r UNION ALL
SELECT 7 AS r UNION ALL
SELECT 8 AS r UNION ALL
SELECT 9 AS r UNION ALL
SELECT 10 AS r
)
SELECT
r,
SUM(1) OVER (ORDER BY r ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING) AS CountOfAllUniqueRows,
COUNT(r) AS CountOfEachR,
SUM(COUNT(R)) OVER (ORDER BY r ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING) AS CountOfAllRows
FROM YourTable
GROUP BY r
ORDER BY r
LIMIT 4
And giving the results of:
r CountOfAllUniqueRows CountOfEachR CountOfAllRows
1 10 8 68
2 10 6 68
3 10 7 68
4 10 6 68
Don't know why you would wanted to do this - maybe because of cost - so you avoid second scan - anyway - Below "trick' might work for you.
While getting only as many rows as you wish - you also getting total rows count but within each and every output row - So you need to handle this on your own when displaying this to user
With BigQuery Legacy SQL:
SELECT
r, cnt
FROM (
SELECT
r,
COUNT(r) OVER() AS cnt,
ROW_NUMBER() OVER() AS line
FROM
(SELECT 1 AS r),
(SELECT 2 AS r),
(SELECT 3 AS r),
(SELECT 4 AS r),
(SELECT 5 AS r),
(SELECT 6 AS r),
(SELECT 7 AS r),
(SELECT 8 AS r),
(SELECT 9 AS r),
(SELECT 10 AS r)
)
WHERE line <= 4
or
SELECT
r,
cnt
FROM (
SELECT r
FROM
(SELECT 1 AS r),
(SELECT 2 AS r),
(SELECT 3 AS r),
(SELECT 4 AS r),
(SELECT 5 AS r),
(SELECT 6 AS r),
(SELECT 7 AS r),
(SELECT 8 AS r),
(SELECT 9 AS r),
(SELECT 10 AS r)
) AS YourTable
CROSS JOIN (
SELECT COUNT(1) AS cnt
FROM
(SELECT 1 AS r),
(SELECT 2 AS r),
(SELECT 3 AS r),
(SELECT 4 AS r),
(SELECT 5 AS r),
(SELECT 6 AS r),
(SELECT 7 AS r),
(SELECT 8 AS r),
(SELECT 9 AS r),
(SELECT 10 AS r)
) rows
LIMIT 4
With BigQuery Standard SQL:
Don't forget to uncheck Use Legacy SQL checkbox under Show Options
WITH YourTable AS (
SELECT 1 AS r UNION ALL
SELECT 2 AS r UNION ALL
SELECT 3 AS r UNION ALL
SELECT 4 AS r UNION ALL
SELECT 5 AS r UNION ALL
SELECT 6 AS r UNION ALL
SELECT 7 AS r UNION ALL
SELECT 8 AS r UNION ALL
SELECT 9 AS r UNION ALL
SELECT 10 AS r
)
SELECT
r,
(SELECT COUNT(1) FROM YourTable) AS cnt
FROM YourTable
LIMIT 4
In all cases result is
r cnt
1 10
2 10
3 10
4 10
Related
in need Help with oracle SQL.
I have a table with
from to
F B
B R
R D
E X
X Q
and I need the list
F
B
R
D
E
X
Q
so my problem is the jump from R-->D to E-->X
Edit: It's a big list with from and to, seperatet with a annother column as citerium. Normaly there is every from in the to column, so i used
SELECT from,snr as Nr FROM list where StrAbsNr = 1
union all
SELECT to,snr + 1 as Nr FROM list
to create a ordered list. But there are gaps in some parts, in the example there is D-->E missing
has anybody an idea ?
for your example this work:
WITH ft AS
(SELECT 'f' vfrom, 'b' AS vto FROM dual UNION ALL
SELECT 'b' , 'r' FROM dual UNION ALL
SELECT 'r','d' FROM dual UNION ALL
SELECT 'e','x' FROM dual UNION ALL
SELECT 'x','q' FROM dual )
SELECT a.a, MAX(rn), MIN(ob)
FROM
( SELECT vfrom a , rownum rn, 1 ob FROM ft
UNION ALL
SELECT vto , rownum rn, 2 ob FROM ft
) a
GROUP BY a
ORDER BY MAX(rn), MIN(ob)
A MAX(RN) MIN(OB)
- ---------- ----------
f 1 1
b 2 1
r 3 1
d 3 2
e 4 1
x 5 1
q 5 2
7 rows selected
or analityc func row_number:
SELECT *
FROM
(SELECT a.a,
row_number() over (partition BY a order by rn, ob) rna,
ob,
rn
FROM
( SELECT vfrom a, rownum rn, 1 ob FROM ft
UNION ALL
SELECT vto , rownum rn, 2 ob FROM ft
) a
)
WHERE rna=1
ORDER BY rn,
ob
A RNA OB RN
- ---------- ---------- ----------
f 1 1 1
b 1 2 1
r 1 2 2
d 1 2 3
e 1 1 4
x 1 2 4
q 1 2 5
7 rows selected
select "from" as val from table
union
select to from table
And if you want to keep the order:
select distinct val
from (select "from" as val, rownum, 1 as valOrder from table
union
select to, rownum, 2 as valOrder from table)
order by rownum,valOrder
I have some time series data. For example look at the following values (Lets assume time here is minutes):
User Time Value
a 0 10
b 1 100
c 2 200
a 3 5
e 4 7
a 5 999
a 6 8
b 7 10
a 8 10
a 9 10
a 10 10
a 11 10
a 12 100
Now I want to find out if within any given 5 minute intervals a total SUM of more than 1000 is achieved.
For example in the above example I should get an output such as user a, minute 5,6,8,9.
That's an easy task for Window Function:
select *
from
(
select t.*
,sum("Value") -- cumulative sum over the previous five minutes
over (partition by "user"
order by "Time"
range 4 preceding) as sum_5_minutes
from Table1 t
) dt
where sum_5_minutes > 1000
See fiddle
Edit: SQLFiddle is offline again, but you can also search the next 5 minutes.
Edit2: SQLFiddle offline, but if the datatype is a TimeStamp or Date you must use intervals instead of integers:
select *
from
(
select t.*
,sum("Value")
over (partition by "User"
order by "Time"
range interval '4' minute preceding) as sum_prev5_minutes
,sum("Value")
over (partition by "User"
order by "Time"
range between interval '0' minute preceding -- or "current row" if there are no duplicate timestamps
and interval '4' minute following) as sum_next5_minutes
from Table1 t
) dt
where sum_prev5_minutes > 1000
or sum_next5_minutes > 1000
To illustrate my comment to dnoeth's post, and so don't take my answer as correct as he did the heavy lifting and deserves the green checkmark, the following shows how you can set the range at runtime...
WITH DAT AS (
SELECT 'a' u, 0 t, 10 v from dual union all
SELECT 'b' u, 1 t, 100 v from dual union all
SELECT 'c' u, 2 t, 200 v from dual union all
SELECT 'a' u, 3 t, 5 v from dual union all
SELECT 'e' u, 4 t, 7 v from dual union all
SELECT 'a' u, 5 t, 999 v from dual union all
SELECT 'a' u, 6 t, 8 v from dual union all
SELECT 'b' u, 7 t, 10 v from dual union all
SELECT 'a' u, 8 t, 10 v from dual union all
SELECT 'a' u, 9 t, 10 v from dual union all
SELECT 'a' u, 10 t, 10 v from dual union all
SELECT 'a' u, 11 t, 10 v from dual union all
SELECT 'a' u, 12 t, 100 v from dual )
-- imaging passing a variable in to this second query, setting it in a config table, or whatever.
-- This is just showing that you don't have to hard-code it into the actual select clause, and that the value can be determined at runtime.
, wind as (select 5 rng from dual)
select d.*
,sum(v) -- cumulative sum over the previous five minutes
over (partition by u order by t
range w.rng preceding) as sum_5_minutes
from dat d
join wind w on 1=1
order by u,t;
I also note that lad2025 is correct that this windowing WILL miss some rows in the set. To correct that you need to bring back all rows in the set over the range for a user where the preceeding five seconds exceed 1000. This works correctly for user Z below, but would have only brought back the second row as originally coded.
WITH DAT AS (
SELECT 'a' u, 0 t, 10 v from dual union all
SELECT 'b' u, 1 t, 100 v from dual union all
SELECT 'c' u, 2 t, 200 v from dual union all
SELECT 'a' u, 3 t, 5 v from dual union all
SELECT 'e' u, 4 t, 7 v from dual union all
SELECT 'a' u, 5 t, 999 v from dual union all
SELECT 'a' u, 6 t, 8 v from dual union all
SELECT 'b' u, 7 t, 10 v from dual union all
SELECT 'a' u, 8 t, 10 v from dual union all
SELECT 'a' u, 9 t, 10 v from dual union all
SELECT 'a' u, 10 t, 10 v from dual union all
SELECT 'a' u, 11 t, 10 v from dual union all
-- two Z rows added. In the initial version only the second row would be caught.
SELECT 'z' u, 10 t, 999 v from dual union all
SELECT 'z' u, 11 t, 10 v from dual union all
SELECT 'a' u, 12 t, 100 v from dual )
, wind as (select 3 rng from dual)
SELECT dd.*, sum_5_minutes
from dat dd
JOIN (
SELECT * FROM (
select d.*
,sum(v) -- cumulative sum over the previous five minutes
over (partition by u order by t
range w.rng preceding) as sum_5_minutes
,min(t) -- start point of the range that we are covering
over (partition by u order by t
range w.rng preceding) as rng_5_minutes
from dat d
join wind w on 1=1
) WHERE sum_5_minutes > 1000 ) fails
on dd.u = fails.u
and dd.t >= fails.rng_5_minutes
and dd.t <= fails.t
order by dd.u, dd.t;
Here is my attempt at this:
select
s1."user", s1."time", sum (s2."value") as five_minute_value
from
sample s1
left join sample s2 on
s1."user" = s2."user" and
s1."time" between s2."time" and s2."time" + 4
group by
s1."user", s1."time"
having
sum (s2."value") > 1000
Output on your data:
a 8 1017
a 9 1027
a 6 1012
a 5 1004
I have a table with 2 Columns:
'Employee' 'ReportsTo'
1
2 1
3 1
4 2
5 2
6 3
7 3
8 4
9 4
10 5
I would like to return like this:
'Employee' 'Reporting Officer'
1 2,3,4,5,6,7,8,9,10
2 4,5,8,9,10
3 6,7
4 8,9
5 10
6
7
8
9
10
This will work with your example table but not with tables like this (would need a more complex query):
Employee ReportsTo
-------- ---------
12 13
13 12
And it is not called "pivot after n iteration".
with cte as (
SELECT 1 Employee,0 ReportsTo UNION ALL
SELECT 2,1 UNION ALL
SELECT 3,1 UNION ALL
SELECT 4,2 UNION ALL
SELECT 5,2 UNION ALL
SELECT 6,3 UNION ALL
SELECT 7,3 UNION ALL
SELECT 8,4 UNION ALL
SELECT 9,4 UNION ALL
SELECT 10,5
),
cte2 as (
SELECT ReportsTo e,Employee ro FROM cte
UNION ALL
SELECT a.e,b.Employee FROM cte2 a JOIN cte b ON a.ro = b.ReportsTo
)
SELECT
Employee,
ISNULL(LEFT(ro,LEN(ro)-1),'') as "Reporting Officer"
FROM (
SELECT
Employee,
REPLACE(REPLACE((
SELECT ro FROM cte2 x WHERE x.e=cte.Employee ORDER BY 1 FOR XML PATH('')
),'<ro>',''),'</ro>',',') ro
FROM cte
) a
sqlfidle
I have a tableA (ID int, Match varchar, tot int)
ID Match Tot
1 123
2 123
3 12
4 12
5 4
6 12
7 8
Now, I want to calculate Tot which is total number of match exists in the table. for example 123 occured twice, 12 exist thrice and so on. Also note that I want the count only at first match. here is the expected result.:
ID Match Tot
1 123 2
2 123
3 12 3
4 12
5 4 1
6 12
7 8 1
Another case:
ID Match Count Tot
1 123 2
2 123 1
3 12 10
4 12 10
5 4 3
6 12 5
7 8 7
Now I want to add the count for the same match. expected result:
ID Match Count Tot
1 123 2 3
2 123 1
3 12 10 25
4 12 10
5 4 3 3
6 12 5
7 8 7 7
Thanks
WITH tableA(ID, Match) AS
(
SELECT 1,123 UNION ALL
SELECT 2,123 UNION ALL
SELECT 3,12 UNION ALL
SELECT 4,12 UNION ALL
SELECT 5,4 UNION ALL
SELECT 6,12 UNION ALL
SELECT 7,8
)
SELECT *,
CASE
WHEN ROW_NUMBER() OVER (PARTITION BY Match ORDER BY ID) = 1
THEN COUNT(*) OVER (PARTITION BY Match)
END AS Tot
FROM tableA
ORDER BY ID
SELECT match, COUNT(match ) as Tot
FROM tableA
GROUP BY match
Solution 1:
DECLARE #MyTable TABLE
(
ID INT PRIMARY KEY
,Match VARCHAR(10) NOT NULL
,Tot INT NULL
);
INSERT #MyTable(ID, Match)
SELECT 1, 123
UNION ALL
SELECT 2, 123
UNION ALL
SELECT 3, 12
UNION ALL
SELECT 4, 12
UNION ALL
SELECT 5, 4
UNION ALL
SELECT 6, 12
UNION ALL
SELECT 7, 8;
--SELECT
SELECT *
,CASE
WHEN ROW_NUMBER()OVER(PARTITION BY a.Match ORDER BY a.ID ASC)=1
THEN COUNT(*)OVER(PARTITION BY a.Match)
END TotCalculated
FROM #MyTable a;
--UPDATE
WITH MyCTE
AS
(
SELECT a.Tot
,CASE
WHEN ROW_NUMBER()OVER(PARTITION BY a.Match ORDER BY a.ID ASC)=1
THEN COUNT(*)OVER(PARTITION BY a.Match)
END TotCalculated
FROM #MyTable a
)
UPDATE MyCTE
SET Tot = TotCalculated;
SELECT *
FROM #MyTable;
Solution 2:
UPDATE #MyTable
SET Tot = NULL;
SELECT x.ID, y.Num
FROM
(
SELECT b.Match, MIN(b.ID) ID
FROM #MyTable b
GROUP BY b.Match
) x INNER JOIN
(
SELECT a.Match, COUNT(*) AS Num
FROM #MyTable a
GROUP BY a.Match
) y ON x.Match = y.Match
ORDER BY x.ID
UPDATE #MyTable
SET Tot = t.Num
FROM #MyTable z
INNER JOIN
(
SELECT x.ID, y.Num
FROM
(
SELECT b.Match, MIN(b.ID) ID
FROM #MyTable b
GROUP BY b.Match
) x INNER JOIN
(
SELECT a.Match, COUNT(*) AS Num
FROM #MyTable a
GROUP BY a.Match
) y ON x.Match = y.Match
) t ON z.ID = t.ID;
SELECT *
FROM #MyTable;
Are there any techniques that would allow a row set like this
WITH
base AS
(
SELECT 1 N FROM DUAL UNION ALL
SELECT 2 N FROM DUAL UNION ALL
SELECT 3 N FROM DUAL UNION ALL
SELECT 6 N FROM DUAL UNION ALL
SELECT 7 N FROM DUAL UNION ALL
SELECT 17 N FROM DUAL UNION ALL
SELECT 18 N FROM DUAL UNION ALL
SELECT 19 N FROM DUAL UNION ALL
SELECT 21 N FROM DUAL
)
SELECT a.N
FROM base a
to yield results
1 3
6 7
17 19
21 21
It is in effect a rows to ranges operation.
I'm playing in Oracle Land, and would appreciate any suggestions.
I feel like this can probably be improved on, but it works:
WITH base AS (
SELECT 1 N FROM DUAL UNION ALL
SELECT 2 N FROM DUAL UNION ALL
SELECT 3 N FROM DUAL UNION ALL
SELECT 6 N FROM DUAL UNION ALL
SELECT 7 N FROM DUAL UNION ALL
SELECT 17 N FROM DUAL UNION ALL
SELECT 18 N FROM DUAL UNION ALL
SELECT 19 N FROM DUAL UNION ALL
SELECT 21 N FROM DUAL
)
, lagged AS
(
SELECT n, LAG(n) OVER (ORDER BY n) lag_n FROM base
)
, groups AS
(
SELECT n, row_number() OVER (ORDER BY n) groupnum
FROM lagged
WHERE lag_n IS NULL OR lag_n < n-1
)
, grouped AS
(
SELECT n, (SELECT MAX(groupnum) FROM groups
WHERE groups.n <= base.n
) groupnum
FROM base
)
SELECT groupnum, MIN(n), MAX(n)
FROM grouped
GROUP BY groupnum
ORDER BY groupnum
Another way:
WITH base AS
(
SELECT 1 N FROM DUAL UNION ALL
SELECT 2 N FROM DUAL UNION ALL
SELECT 3 N FROM DUAL UNION ALL
SELECT 6 N FROM DUAL UNION ALL
SELECT 7 N FROM DUAL UNION ALL
SELECT 17 N FROM DUAL UNION ALL
SELECT 18 N FROM DUAL UNION ALL
SELECT 19 N FROM DUAL UNION ALL
SELECT 21 N FROM DUAL
)
select min(n), max(n) from
(
select n, connect_by_root n root from base
connect by prior n = n-1
start with n not in (select n from base b
where exists (select 1 from base b1 where b1.n = b.n-1)
)
)
group by root
order by root
Yet another way:
with base as (
select 1 n from dual union all
select 2 n from dual union all
select 3 n from dual union all
select 6 n from dual union all
select 7 n from dual union all
select 17 n from dual union all
select 18 n from dual union all
select 19 n from dual union all
select 21 n from dual)
select a,b
from (select a
,case when b is not null and a is not null
then b
else lead(n) over (order by n)
end b
from (select n
,a
,b
from (select n
,case n-1 when lag (n) over (order by n) then null else n end a
,case n+1 when lead (n) over (order by n) then null else n end b
from base)
where a is not null
or b is not null))
where a is not null
order by a