I have a table like this:
Now I wish to GROUP BY the Field1, which is not that hard. After this, I want to add a row number to each group. And finally.. this has to be done in Access which is slightly different of course. So this is the code I have:
SELECT A.*, (SELECT COUNT(*) FROM Tabel1 WHERE A.ID>=ID) AS RowNum
FROM Tabel1 AS A
ORDER BY A.ID;
So, this works well, but now I can't group it. How can I group it?
You can wrap your entire query in a subquery:
select
B.*
from
( SELECT
A.*,
(SELECT COUNT(*) FROM Tabel1 WHERE A.ID>=ID) AS RowNum
FROM Tabel1 AS A
) as B
ORDER BY B.ID;
From here, you can do joins as though the output of your query was a table. I have no idea what you actually want to group on, but here is an example:
select
B.Field1, count (*) as count, max (B.RowNum) as max_row
from
( SELECT
A.*,
(SELECT COUNT(*) FROM Tabel1 WHERE A.ID>=ID) AS RowNum
FROM Tabel1 AS A
) as B
group by
b.Field1
-- Edit 11/14/2016 --
I think I see now. In that case, first build a query to handle your grouping. This is just an example:
SELECT Field1, min (Id) as min_id, max (id) as max_id
FROM Table1
group by Field1
Name this query Table1_Summary for the purposes of our example.
Now, in your new query, you will refer to Table1_Summary in the exact same way you did with your example:
SELECT
t.*,
(select count (*) from Table1_Summary t2 where t.Field1 >= t2.Field1) as RowNum
FROM Table1_Summary t
You could theoretically do this in a single query, but for readability/maintainability sake, I'd recommend you keep them split. Here is an example of the output:
Related
I would like to write a query that returns the first row immediately after the last row with a given property (ordered by id). Id's may not be consecutive.
Ideally it would look something like this:
...
JOIN (select max(id) id from my_table where CONDITION) m
JOIN (select min(id) from my_table where id > m.id) n
However, I can not use identifier m in the second subselect.
It is possible to use nested queries in nested queries, but is there an easier way?
Thank you.
You could use lead() to get the next id before applying the condition:
select t.*
from my_table t join
(select max(next_id) as max_next_id
from (select t.*, lead(id) over (order by id) as next_id
from my_table t
) t
where <condition>
) tt
on t.id = tt.max_next_id;
You could also do:
select t.*
from my_table t
where t.id > (select max(t2.id) from my_table t2 where <condition>)
order by t2.id asc
fetch first 1 row only;
I am not sure how this is getting woven into the rest of your query, so I have used a CTE
WITH max_next AS (
SELECT r.id as max_id
,r.next_id
FROM (
SELECT m.id
,m.next_id
,ROW_NUMBER() OVER (ORDER BY m.id DESC) AS rn
FROM (
SELECT n.* -- to provide data to satisfy CONDITIONS
,LEAD(n.id) OVER(ORDER BY n.id) as next_id
FROM my_table AS n
) AS m
WHERE CONDITIONS
) AS r
WHERE r.rn = 1
)
I would also shrink the n.* to the columns needed by CONDITIONS to a, not be implicit as the * slows the compile time down (or historically has) as all meta data needs to be read to understand what columns is in the ANY, and the while the compile can also prune not used columns, it's faster if you just ask for what you want (in best case just a compile time savings, worse case, it read all the data when you only need x number of columns read)
And borrowing from Gordon solution, the ROW_NUMBER part could be simpler
WITH max_next AS (
SELECT m.id
,m.next_id
--, plus what ever other things you want from m
FROM (
SELECT n.* -- to satisfy CONDITIONS needs
,LEAD(n.id) OVER(ORDER BY n.id) as next_id
FROM my_table AS n
) AS m
WHERE CONDITIONS
ORDER BY m.id DESC LIMIT 1
)
So for an example for #PIG,
WITH my_table AS (
SELECT column1 AS id
,column2 AS con1
,column3 AS other
FROM VALUES (1,'a',123),(2,'b',234),(3,'a',345),(5,'b',456),(7,'a',567),(10,'c',678)
)
SELECT m.id
,m.next_id
,m.other
FROM (
SELECT n.* -- to satisfy CONDITIONS needs
,LEAD(n.id) OVER(ORDER BY n.id) as next_id
FROM my_table AS n
) AS m
WHERE m.con1 = 'b'
ORDER BY m.id DESC LIMIT 1;
gives 5, 7, 456 which is the last 'b' and the new row, and an extra value on my_table for entertainment purposes (and run on Snowflake to, which means I fixed the prior SQL also.)
This should work, it's pretty straightforward (easy), and it's good that you know records may not be stored in a ordered/consecutive fashion.
SELECT *
FROM my_table
WHERE id = (
SELECT min(id)
FROM my_table
WHERE id > (
SELECT max(id)
FROM my_table
WHERE CONDITION));
I have code where I'm sampling 50,000 random records. I.e.,
SELECT * FROM Table1
SAMPLE 50000;
That works. However, what I really want to do is sample the number of records that are in a different table. I.e.,
SELECT * FROM Table1
SAMPLE count(*) FROM Table2;
I get an error. What am I doing wrong?
This is not randomized like sample, so bear that in mind. But there also won't be an obvious pattern, I believe it's determined by disk location (don't quote me on that).
SELECT *
FROM Table1
QUALIFY ROW_NUMBER() OVER
( PARTITION BY 1
ORDER BY 1
) <=
( SELECT COUNT(*)
FROM Table2
);
Better way
SELECT TMP.* -- Or list the columns you want with "rnd"
FROM ( SELECT RANDOM(-10000000,10000000) rnd,
T1.*
FROM Table1 T1
) TMP
QUALIFY ROW_NUMBER() OVER
( ORDER BY rnd
) <=
( SELECT COUNT(*)
FROM Table2
);
SELECT TOP 50000 * FROM Table1 ORDER BY NEWID()
Query
select * from table1
where having count(reference)>1
I want to select * the data which have duplicate data,any idea why my query is not working?
Below are my expect result..
You can make use of window function count to find number of rows per id and reference and then filter to get those which have count more than 1.
;with cte as (
select t.*, count(*) over (partition by id, reference) cnt
from table1 t
)
select * from cte where cnt > 1;
Demo
In the above solution, I have made an assumption that name and id has one to one correspondence (which is true as per your given data). If that's not the case, add name too in the partition by clause:
;with cte as (
select t.*, count(*) over (partition by name, id, reference) cnt
from table1 t
)
select * from cte where cnt > 1;
I might actually approach this by using a subquery with GROUP BY:
SELECT t1.*
FROM table1 t1
INNER JOIN
(
SELECT Name, ID, reference
FROM table1
GROUP BY Name, ID, reference
HAVING COUNT(*) > 1
) t2
ON t1.Name = t2.Name AND
t1.ID = t2.ID AND
t1.reference = t2.reference
Demo here:
Rextester
Try this ), first i get count by partition, after that i get row with count > 1
select No, Name, ID, Reference
from (select count(*) over (partition by name, ID, reference) cnt, table1.* from table1)
where cnt>1
The easy way (although maybe not the best for performance) would be:
select * from table1 where reference in (
select reference from table1 group by reference having count(*)>1
)
In a subselect you have the duplicated data, and in the outter select you have all the data for these references.
I have a quite large table with a field ID and another field as collection_time. I want to select latest record for each ID. Unfortunately combination of (ID, collection_time) time is not unique together in my data. I want just one of records with the maximum collection time. I have tried two solutions but none of them has worked for me:
First: using query
SELECT * FROM
(SELECT *, ROW_NUMBER() OVER (PARTITION BY ID ORDER BY collection_time) as rn
FROM mytable) where rn=1
This results in Resources exceeded error that I guess is because of ORDER BY in the query.
Second
Using join between table and latest time:
(SELECT tab1.*
FROM mytable AS tab1
INNER JOIN EACH
(SELECT ID, MAX(collection_time) AS second_time
FROM mytable GROUP EACH BY ID) AS tab2
ON tab1.ID=tab2.ID AND tab1.collection_time=tab2.second_time)
this solution does not work for me because (ID, collection_time) are not unique together so in JOIN result there would be multiple rows for each ID.
I am wondering if there is a workaround for the resourcesExceeded error, or a different query that would work in my case?
SELECT
agg.table.*
FROM (
SELECT
id,
ARRAY_AGG(STRUCT(table)
ORDER BY
collection_time DESC)[SAFE_OFFSET(0)] agg
FROM
`dataset.table` table
GROUP BY
id)
This will do the job for you and is scalable considering the fact that the schema keeps changing, you won't have to change this
Short and scalable version:
select array_agg(t order by collection_time desc limit 1)[offset(0)].*
from mytable t
group by t.id;
Quick and dirty option - combine your both queries into one - first get all records with latest collection_time (using your second query) and then dedup them using your first query:
SELECT * FROM (
SELECT *, ROW_NUMBER() OVER (PARTITION BY tab1.ID) AS rn
FROM (
SELECT tab1.*
FROM mytable AS tab1
INNER JOIN (
SELECT ID, MAX(collection_time) AS second_time
FROM mytable GROUP BY ID
) AS tab2
ON tab1.ID=tab2.ID AND tab1.collection_time=tab2.second_time
)
)
WHERE rn = 1
And with Standard SQL (proposed by S.Mohsen sh)
WITH myTable AS (
SELECT 1 AS ID, 1 AS collection_time
),
tab1 AS (
SELECT ID,
MAX(collection_time) AS second_time
FROM myTable GROUP BY ID
),
tab2 AS (
SELECT * FROM myTable
),
joint AS (
SELECT tab2.*
FROM tab2 INNER JOIN tab1
ON tab2.ID=tab1.ID AND tab2.collection_time=tab1.second_time
)
SELECT * EXCEPT(rn)
FROM (
SELECT *, ROW_NUMBER() OVER (PARTITION BY ID) AS rn
FROM joint
)
WHERE rn=1
If you don't care about writing a piece of code for every column:
SELECT ID,
ARRAY_AGG(col1 ORDER BY collection_time DESC)[OFFSET(0)] AS col1,
ARRAY_AGG(col2 ORDER BY collection_time DESC)[OFFSET(0)] AS col2
FROM myTable
GROUP BY ID
I see no one has mentioned window functions with QUALIFY:
SELECT *, MAX(collection_time) OVER (PARTITION BY id) AS max_timestamp
FROM my_table
QUALIFY collection_time = max_timestamp
The window function adds a column max_timestamp that is accessible in the QUALIFY clause to filter on.
As per your comment, Considering you have a table with unique ID's for which you need to find latest collection_time. Here is another way to do it using Correlated Sub-Query. Give it a try.
SELECT id,
(SELECT Max(collection_time)
FROM mytable B
WHERE A.id = B.id) AS Max_collection_time
FROM id_table A
Another solution, which could be more scalable since it avoids multiple scans of the same table (which will happen with both self-join and correlated subquery in above answers). This solution only works with standard SQL (uncheck "Use Legacy SQL" option):
SELECT
ID,
(SELECT srow.*
FROM UNNEST(t.srows) srow
WHERE srow.collection_time = MAX(srow.collection_time))
FROM
(SELECT ID, ARRAY_AGG(STRUCT(col1, col2, col3, ...)) srows
FROM id_table
GROUP BY ID) t
Whats wrong with this SQL?
SELECT Id, (select SUM(VALUE) from SomeTable) AS SumValue, GETDATE()
FROM MyTable
WHERE SumValue > 0
You cannot use aliased columns in the SELECT clause in the same query, except in ORDER BY.
It needs to be subqueried
SELECT Id, SumValue, GETDATE()
FROM (
SELECT Id, (select SUM(VALUE) from TABLE) AS SumValue
FROM MyTable
) X
WHERE SumValue > 0
That is the general case. For your specific query, it doesn't make sense because the subquery is not correlated to the outer query, so either NO rows show, or ALL rows show (with the same SumValue). I will simply assume you have simplified the query a lot since a table name of "table" doesn't really work.
I would probably rewrite like this:
SELECT a.Id, b.SumValue, GETDATE() as [now]
FROM MyTable a
Join
(
select id, SUM(VALUE) as [SumValue]
from [TABLE]
Group by id
)b on a.Id = b.Id
WHERE b.SumValue > 0
This is assuming that the value you are totalling relates to the ID in your table?
right way is
SELECT Id, (select SUM(VALUE) from TABLE) AS SumValue, GETDATE()
FROM MyTable
WHERE (select SUM(VALUE) from TABLE) > 0