i'm querying a system that won't allow using DISTINCT, so my alternative is to do a GROUP BY to get near to a result
my desired query was meant to look like this,
SELECT
SUM(column1) AS column1,
SUM(column2) AS column2,
COUNT(DISTINCT(column3)) AS column3
FROM table
for the alternative, i would think i'd need some type of nested query along the lines of this,
SELECT
SUM(column1) AS column1,
SUM(column2) AS column2,
COUNT(SELECT column FROM table GROUP BY column) AS column3
FROM table
but it didn't work. Am i close?
You are using the wrong syntax for COUNT(DISTINCT). The DISTINCT part is a keyword, not a function. Based on the docs, this ought to work:
SELECT
SUM(column1) AS column1,
SUM(column2) AS column2,
COUNT(DISTINCT column3) AS column3
FROM table
Do, however, read the docs. BigQuery's implementation of COUNT(DISTINCT) is a bit unusual, apparently so as to scale better for big data. If you are trying to count a large number of distinct values then you may need to specify a second parameter (and you have an inherent scaling problem).
Update:
If you have a large number of distinct column3 values to count, and you want an exact count, then perhaps you can perform a join instead of putting a subquery in the select list (which BigQuery seems not to permit):
SELECT *
FROM (
SELECT
SUM(column1) AS column1,
SUM(column2) AS column2
FROM table
)
CROSS JOIN (
SELECT count(*) AS column3
FROM (
SELECT column3
FROM table
GROUP BY column3
)
)
Update 2:
Not that joining two one-row tables would be at all expensive, but #FelipeHoffa got me thinking more about this, and I realized I had missed a simpler solution:
SELECT
SUM(column1) AS column1,
SUM(column2) AS column2,
COUNT(*) AS column3
FROM (
SELECT
SUM(column1) AS column1,
SUM(column2) AS column2
FROM table
GROUP BY column3
)
This one computes a subtotal of column1 and column2 values, grouping by column3, then counts and totals all the subtotal rows. It feels right.
FWIW, the way you are trying to use DISTINCT isn't how its normally used, as its meant to show unique rows, not unique values for one column in a dataset. GROUP BY is more in line with what I believe you are ultimately trying to accomplish.
Depending upon what you need you could do one of a couple things. Using your second query, you would need to modify your subquery to get a count, not the actual values, like:
SELECT
SUM(column1) AS column1,
SUM(column2) AS column2,
(SELECT sum(1) FROM table GROUP BY column) AS column3
FROM table
Alternatively, you could do a query off your initial query, something like this:
SELECT sum(column1), sum(column2), sum(column4) from (
SELECT
SUM(column1) AS column1,
SUM(column2) AS column2,
1 AS column4
FROM table GROUP BY column3)
GROUP BY column4
Edit: The above is generic SQL, not too familiar with Google Big Query
You can probably use a CTE
WITH result as (select column from table group by column)
SELECT
SUM(column1) AS column1,
SUM(column2) AS column2,
Select Count(*) From result AS column3
FROM table
Instead of doing a COUNT(DISTINCT), you can get the same results by running a GROUP BY first, and then counting results.
For example, the number of different words that Shakespeare used by year:
SELECT corpus_date, COUNT(word) different_words
FROM (
SELECT word, corpus_date
FROM [publicdata:samples.shakespeare]
GROUP BY word, corpus_date
)
GROUP BY corpus_date
ORDER BY corpus_date
As a bonus, let's add a column that identifies which books were written during each year:
SELECT corpus_date, COUNT(word) different_words, GROUP_CONCAT(UNIQUE(corpus)) books
FROM (
SELECT word, corpus_date, UNIQUE(corpus) corpus
FROM [publicdata:samples.shakespeare]
GROUP BY word, corpus_date
)
GROUP BY corpus_date
ORDER BY corpus_date
Related
I am having 2 queries, in one of them I am using distinct and in other I am using group by. which is faster distinct or group by also if I have more than 10 columns then which one is faster?
select distinct column1, column2
from table
select column1, column2
from table
group by column1, column2
The table looks like
column1 column2 column3
400196 2021-07-06 33
400196 2021-07-06 33
400196 2021-08-16 33
I want to get the sum of column3 values based on grouping of column 1 but the duplicate values of date should not be added
The desired output is:
column1 column3
400196 66
The query I wrote is
select sum(column3)
from table_name
group by column1
But this gives me result 99
You can remove duplicate values in a subquery:
select t.column1, sum(t.column3)
from (select distinct t.column1, t.column2, t.column3
from t
) t
group by t.column1;
Note: This sort of problem can arise when you are joining tables together. Removing duplicates may not always be the right solution. Often it is better to do the calculation before joining, so you don't have duplicate values to deal with.
You could use a two step process here, first remove duplicates, then aggregate and sum:
SELECT column1, SUM(column3) AS column3
FROM (SELECT DISTINCT column1, column2, column3 FROM yourTable) t
GROUP BY column1;
Demo
How can I aggregate and arrive to these results?
SELECT *
FROM TABLE
GROUP BY Column1
Assuming the columns actually contain the literal string values 'TRUE' and 'FALSE', we could use:
SELECT
Column1,
MAX(Column2) AS Column2,
MAX(Column3) AS Column3
FROM yourTable
GROUP BY
Column1;
I'm facing with problem in one query. The easiest will be to explain step by step:
At first I'm searching a specific values in colum1 in table1 by using query like this:
Query #1:
select column1
from table1
where column1 in('xxx','yyy','zzz')
group by column1
having count(*) >3
So now I have a list on values from column1, which occurs more than 3 times.
Then I need to use that list in where condition in another query:
select column1, column2, column3
from table1
where column1 in (query 1)
Unfortunately when I'm using query 1 as subquery, execution is really slow and I need to find a different way to this. Any suggest how can I increase a performance ?
Best regards and thank you in advance
If they are the same table, then use window functions:
select t.*
from (select t.*, count(*) over (partition by column1) as cnt
from table1 t
where column1 in ('xxx', 'yyy', 'zzz')
) t
where cnt > 3;
Both this an your original query will benefit from h having an index on table1(column1).
1)First of all take a look if the query is correctly indexed.
Maybe you have to add an index on column1.
2) try with it:
select column1, column2, column3
from table1 as T1 inner join (
select column1, column2, column3
from table1
where column1 in (query 1)) as T2
on t1.column1 = t2.column1
Similar questions have been asked before and I tried all those solutions. Unfortunately none of them worked for me. Here is my requirement. There is a table with column1 (which is PK), column2, column3, column4, column5. Other than the first column (PK), all columns may have duplicate values in different rows. However, I want to pull a list of all rows with just one condition, column2 must not repeat. If there are multiple rows with duplicate values in column2, I just want any of the rows (say the first one, which can be done using min(column1)) and disregards the rows that has same values in column2.
I tried group by, didn't work because group by requires me to put all columns in group by and that results in a combination of all columns being unique.
EDIT
Thank you everybody for putting me on the right track. I tried many things and finally I think I found the right answer. Please comment if you see any issues with it.
select * from myTable where column1 in (select MIN(column1) from myTable group by column2)
To get the complete row with the minimum column1 per value of column2, you can use analytic functions to assign an order of the rows ordered by column1 partitioning by column2.
Then you can just pick the rows with row number 1 (the first row per partition) and you'll get the complete existing row with the smallest value of column1;
WITH cte AS (
SELECT column1, column2, column3, column4, column5,
ROW_NUMBER() OVER (PARTITION BY column2 ORDER BY column1) rn
FROM test
)
SELECT column1, column2, column3, column4, column5
FROM cte
WHERE rn=1;
An SQLfiddle to test with.
Just apply MIN() to all the other columns you want to retrieve, and use group by on the one column you want to be unique.
SELECT MIN(column1), column2, MIN(column3), MIN(column4), MIN(column5)
FROM your_table
GROUP BY column2
Update
As mentioned in the comments, the above solution does not return a complete row, but the minimum values across all columns in the group. To retrieve a complete row, based on the lowest value in column1:
SELECT column1, column2, column3, column4, column5
FROM your_table t1
WHERE t1.id = (
SELECT TOP 1 t2.id
FROM your_table t2
WHERE t2.column2 = t1.column2
ORDER BY t2.column1
)