I have a table form which I need to extract some information. This table has an oracle spatial (MDSYS.SDO_GEOMETRY) column, from which I also need some data.
I started out with a simple query like this:
select id, field1, field2
FROM my_table;
After that, I was able to loop over the result to extract the data that was in the spatial column:
SELECT *
FROM TABLE (SELECT a.POSITIONMAP.sdo_ordinates
FROM my_table
WHERE ID = 18742084);
The POSITIONMAP.sdo_ordinates seems to usually hold 4 values, like these:
100050,887
407294,948
0,577464740471056
-0,816415625470689
I need the last 2 values. I can achieve that by changing the query into this:
SELECT * FROM
(SELECT rownum AS num,
column_value AS orientatie
FROM TABLE (SELECT a.POSITIONMAP.sdo_ordinates
FROM my_table
WHERE ID = 18742084))
WHERE num IN (3,4)
Looping over every row from my first query to extract the data from the POSITIONMAP column is of course not very performance friendly, so my query becomes slow very quickly.
I would like to retrieve all information in one query, but there are a few things that prevent me from doing so.
Not every row in the table has data in POSITIONMAP
Some rows do have data in POSITIONMAP, but they only contain 2 values (so not the 3rd and 4th value that I am looking for.
I need the data in one row for every row in the table (using the previous query would result in duplicate rows
The closest I got is:
select
id,
field1,
field2
t.*
FROM my_table v,
table (v.POSITIONMAP.sdo_ordinates) t
This gives my 4 rows for every row in my_table.
As soon as I try to put the rownum condition into this query, I get an error: "invalid user.table.column, table.column, or column specification"
Is there any way to combine what I want to do into 1 query?
You can use sdo_util.getvertices as follows:
select t.x,t.y
from my_table mt
,table(sdo_util.getvertices(mt.positionmap)) t
where t.id = 2
I'm assuming that your geometries are lines (gtype=2002) and points (gtype= 2001). If you want X,Y values for lines and empty values for point you can filter on the sdo_gtype property of the geometry object.
select t.x,t.y
from my_table mt
,table(sdo_util.getvertices(mt.positionmap)) t
where t.id = 2
and mt.positionmap.sdo_gtype=2002
union all
select null as X,
null as Y
from my_table mt
where mt.positionmap.sdo_gtype=2001
One method is to use the ROW_NUMBER() analytic function:
SELECT *
FROM (
select id,
field1,
field2,
t.*,
ROW_NUMBER() OVER ( PARTITION BY v.id ORDER BY ROWNUM ) AS rn
FROM my_table v,
TABLE( v.POSITIONMAP.sdo_ordinates ) t
)
WHERE rn IN ( 3, 4 )
Related
I have 2 queries. The first is what I want to accomplish, however, it was taking much too long. Here's how the query looked like:
SELECT *
FROM old_table
WHERE id NOT IN (SELECT id FROM new_table)
UNION ALL
SELECT *
FROM new_table
Basically, I want everything that's in the old table, but not in the new table. I then union everything from the old and new. Again, this query was taking much too long on a bigger dataset. So, I optimized it like so:
WITH union_tbl AS (
(SELECT * FROM old_table)
UNION ALL
(SELECT * FROM new_table)
), row_tbl AS (
SELECT *, ROW_NUMBER() OVER (PARTITION BY id ORDER BY ordering) AS row
FROM union_tbl
)
SELECT *
FROM row_tbl
WHERE row = 1
In the old_table and new_table (in a separate query) I add a new column to each called ordering. The new_table gets 1 for all its rows. For the old table, it gets 2 for all its rows. In the end, I select where the row number is 1. So that means if there's 2 rows one for new_table and one for old_table then it should get the new_table row instead.
This is how I'm envisioning it, however, I'm not getting the same results as my previous query. I'd expect it to have the same exact results with the same rows. What am I getting wrong? Is my query logic incorrect?
Can you try the below code?
SELECT a.*
FROM old_table a left join new_table b on a.id=b.id
WHERE b.id IS NULL
UNION ALL
SELECT *
FROM new_table
This is a very optimized way
I want to get all rows in sequence from 2 tables in SQL Server.
Output should be 1st row from 1st table, then 1st row from 2nd table,
2nd row from 1st table, 2nd row from 2nd table....etc
What #eshirvana suggested will not get you the desired. Instead, it'll be table1.row1, table2.row1, table2.row2, table1.row2
You can use UNION to join data from two tables when the column names and types match. I'm making an assumption on how to order the data based on your desired outcome.
SELECT RowID, Row, z
FROM table1
UNION
SELECT *
FROM table2
ORDER BY z, RowID
Here's the working code:
https://dbfiddle.uk/?rdbms=sqlserver_2019&fiddle=068c0fd2056cc48718345e85b74b7bba
probably something like that :
select * from
(
select rowID,Row,z from table1
union all
select rowID,Row,z from table2
) alltables
order by z
You can try with below approach:
SELECT * FROM
(
SELECT RowId,Row,Z,1 AS TableOrder From Table1
UNION ALL
SELECT RowId,Row,z,2 AS TableOrder From Table2
)
ORDER BY Z,TableOrder
I have a Postgres table created with the following statement. This table is filled by as dump of data from another service.
CREATE TABLE data_table (
date date DEFAULT NULL,
dimension1 varchar(64) DEFAULT NULL,
dimension2 varchar(128) DEFAULT NULL
) TABLESPACE pg_default;
One of the steps in a ETL I'm building is extracting the unique values of dimension1 and inserting them in another intermediary table.
However, during some tests I found out that the 2 commands below do not return the same results. I would expect for both to return the same sum.
The first command returns more results compared with the second (1466 rows vs. 1504.
-- command 1
SELECT DISTINCT count(dimension1)
FROM data_table;
-- command 2
SELECT count(*)
FROM (SELECT DISTINCT ON (dimension1) dimension1
FROM data_table
GROUP BY dimension1) AS tmp_table;
Any obvious explanations for this? Alternatively to an explanation, is there any suggestion of any check on the data I should do?
EDIT: The following queries both return 1504 (same as the "simple" DISTINCT)
SELECT count(*)
FROM data_table WHERE dimension1 IS NOT NULL;
SELECT count(dimension1)
FROM data_table;
Thank you!
DISTINCT and DISTINCT ON have completely different semantics.
First the theory
DISTINCT applies to an entire tuple. Once the result of the query is computed, DISTINCT removes any duplicate tuples from the result.
For example, assume a table R with the following contents:
#table r;
a | b
---+---
1 | a
2 | b
3 | c
3 | d
2 | e
1 | a
(6 rows)
SELECT distinct * from R will result:
# select distinct * from r;
a | b
---+---
1 | a
3 | d
2 | e
2 | b
3 | c
(5 rows)
Note that distinct applies to the entire list of projected attributes: thus
select distinct * from R
is semantically equivalent to
select distinct a,b from R
You cannot issue
select a, distinct b From R
DISTINCT must follow SELECT. It applies to the entire tuple, not to an attribute of the result.
DISTINCT ON is a postgresql addition to the language. It is similar, but not identical, to group by.
Its syntax is:
SELECT DISTINCT ON (attributeList) <rest as any query>
For example:
SELECT DISTINCT ON (a) * from R
It semantics can be described as follows. Compute the as usual--without the DISTINCT ON (a)---but before the projection of the result, sort the current result and group it according to the attribute list in DISTINCT ON (similar to group by). Now, do the projection using the first tuple in each group and ignore the other tuples.
Example:
select * from r order by a;
a | b
---+---
1 | a
2 | e
2 | b
3 | c
3 | d
(5 rows)
Then for every different value of a (in this case, 1, 2 and 3), take the first tuple. Which is the same as:
SELECT DISTINCT on (a) * from r;
a | b
---+---
1 | a
2 | b
3 | c
(3 rows)
Some DBMS (most notably sqlite) will allow you to run this query:
SELECT a,b from R group by a;
And this give you a similar result.
Postgresql will allow this query, if and only if there is a functional dependency from a to b. In other words, this query will be valid if for any instance of the relation R, there is only one unique tuple for every value or a (thus selecting the first tuple is deterministic: there is only one tuple).
For instance, if the primary key of R is a, then a->b and:
SELECT a,b FROM R group by a
is identical to:
SELECT DISTINCT on (a) a, b from r;
Now, back to your problem:
First query:
SELECT DISTINCT count(dimension1)
FROM data_table;
computes the count of dimension1 (number of tuples in data_table that where dimension1 is not null). This query
returns one tuple, which is always unique (hence DISTINCT
is redundant).
Query 2:
SELECT count(*)
FROM (SELECT DISTINCT ON (dimension1) dimension1
FROM data_table
GROUP BY dimension1) AS tmp_table;
This is query in a query. Let me rewrite it for clarity:
WITH tmp_table AS (
SELECT DISTINCT ON (dimension1)
dimension1 FROM data_table
GROUP by dimension1)
SELECT count(*) from tmp_table
Let us compute first tmp_table. As I mentioned above,
let us first ignore the DISTINCT ON and do the rest of the
query. This is a group by by dimension1. Hence this part of the query
will result in one tuple per different value of dimension1.
Now, the DISTINCT ON. It uses dimension1 again. But dimension1 is unique already (due to the group by). Hence
this makes the DISTINCT ON superflouos (it does nothing).
The final count is simply a count of all the tuples in the group by.
As you can see, there is an equivalence in the following query (it applies to any relation with an attribute a):
SELECT (DISTINCT ON a) a
FROM R
and
SELECT a FROM R group by a
and
SELECT DISTINCT a FROM R
Warning
Using DISTINCT ON results in a query might be non-deterministic for a given instance of the database.
In other words, the query might return different results for the same tables.
One interesting aspect
Distinct ON emulates a bad behaviour of sqlite in a much cleaner way. Assume that R has two attributes a and b:
SELECT a, b FROM R group by a
is an illegal statement in SQL. Yet, it runs on sqlite. It simply takes a random value of b from any of the tuples in the group of same values of a.
In Postgresql this statement is illegal. Instead, you must use DISTINCT ON and write:
SELECT DISTINCT ON (a) a,b from R
Corollary
DISTINCT ON is useful in a group by when you want to access a value that is functionally dependent on the group by attributes. In other words, if you know that for every group of attributes they always have the same value of the third attribute, then use DISTINCT ON that group of attributes. Otherwise you would have to make a JOIN to retrieve that third attribute.
The first query gives the number of not null values of dimension1, while the second one returns the number of distinct values of the column. These numbers obviously are not equal if the column contains duplicates or nulls.
The word DISTINCT in
SELECT DISTINCT count(dimension1)
FROM data_table;
makes no sense, as the query returns a single row. Maybe you wanted
SELECT count(DISTINCT dimension1)
FROM data_table;
which returns the number of distinct not null values of dimension1. Note, that it is not the same as
SELECT count(*)
FROM (
SELECT DISTINCT ON (dimension1) dimension1
FROM data_table
-- GROUP BY dimension1 -- redundant
) AS tmp_table;
The last query yields the number of all (null or not) distinct values of the column.
To learn and understand what happens by visual example.
Here's a bit of SQL to execute on a PostgreSQL:
DROP TABLE IF EXISTS test_table;
CREATE TABLE test_table (
id int NOT NULL primary key,
col1 varchar(64) DEFAULT NULL
);
INSERT INTO test_table (id, col1) VALUES
(1,'foo'), (2,'foo'), (3,'bar'), (4,null);
select count(*) as total1 from test_table;
-- returns: 4
-- Because the table has 4 records.
select distinct count(*) as total2 from test_table;
-- returns: 4
-- The count(*) is just one value. Making 1 total unique can only result in 1 total.
-- So the distinct is useless here.
select col1, count(*) as total3 from test_table group by col1 order by col1;
-- returns 3 rows: ('bar',1),('foo',2),(NULL,1)
-- Since there are 3 unique col1 values. NULL's are included.
select distinct col1, count(*) as total4 from test_table group by col1 order by col1;
-- returns 3 rows: ('bar',1),('foo',2),(NULL,1)
-- The result is already grouped, and therefor already unique.
-- So again, the distinct does nothing extra here.
select count(distinct col1) as total5 from test_table;
-- returns 2
-- NULL's aren't counted in a count by value. So only 'foo' & 'bar' are counted
select distinct on (col1) id, col1 from test_table order by col1 asc, id desc;
-- returns 3 rows: (2,'a'),(3,'b'),(4,NULL)
-- So it gets the records with the maximum id per unique col1
-- Note that the "order by" matters here. Changing that DESC to ASC would get the minumum id.
select count(*) as total6 from (select distinct on (col1) id, col1 from test_table order by col1 asc, id desc) as q;
-- returns 3.
-- After seeing the previous query, what else would one expect?
select distinct col1 from test_table order by col1;
-- returns 3 unique values : ('bar'),('foo'),(null)
select distinct id, col1 from test_table order by col1;
-- returns all records.
-- Because id is the primary key and therefore makes each returned row unique
Here's a more direct summary that might useful for Googlers, answering the title but not the intricacies of the full post:
SELECT DISTINCT
availability: ISO
behaviour:
SELECT DISTINCT col1, col2, col3 FROM mytable
returns col1, col2 and col3 and omits any rows in which all of the tuple (col1, col2, col3) are the same. E.g. you could get a result like:
1 2 3
1 2 4
because those two rows are not identical due to the 4. But you could never get:
1 2 3
1 2 4
1 2 3
because 1 2 3 appears twice, and both rows are exactly the same. That is what DISTINCT prevents.
vs GROUP BY: SELECT DISTINCT is basically a subset of GROUP BY where you can't use aggregate functions: Is there any difference between GROUP BY and DISTINCT
SELECT DISTINCT ON
availability: PostgreSQL extension, WONTFIXED by SQLite
behavior: unlike DISTINCT, DISTINCT ON allows you to separate
what you want to be unique
from what you want to return
E.g.:
SELECT DISTINCT ON(col1) col2, col3 FROM mytable
returns col2 and col3, and does not return any two rows with the same col1. E.g.:
1 2 3
1 4 5
could not happen, because we have 1 twice on col1.
And e.g.:
SELECT DISTINCT ON(col1, col2) col2, col3 FROM mytable
would prevent any duplicated (col1, col2) tuples, e.g. you could get:
1 2 3
1 4 3
as it has different (1, 2) and (1, 4) tuples, but not:
1 2 3
1 2 4
where (1, 2) happens twice, only one of those two could appear.
We can uniquely determine which one of the possible rows will be selected with ORDER BY which guarantees that the first match is taken, e.g.:
SELECT DISTINCT ON(col1, col2) col2, col3 FROM mytable
ORDER BY col1 DESC, col2 DESC, col3 DESC
would ensure that among:
1 2 3
1 2 4
only 1 2 4 would be picked as it happens first on our DESC sorting.
vs GROUP BY: DISTINCT ON is not a subset of GROUP BY because it allows you to access extra rows not present in the GROUP BY, which is generally not allowed in GROUP BY, unless:
you group by primary key in Postgres (unique not null is a TODO for them)
or if that is allows as an ISO extension as in SQLite/MySQL
This makes DISTINCT ON extremely useful to fulfill the common use case of "find the full row that reaches the maximum/minimum of some column": Is there any difference between GROUP BY and DISTINCT
E.g. to find the city of each country that has the most sales:
SELECT DISTINCT ON ("country") "country", "city", "amount"
FROM "Sales"
ORDER BY "country" ASC, "amount" DESC, "city" ASC
or equivalently with * if we want all columns:
SELECT DISTINCT ON ("country") *
FROM "Sales"
ORDER BY "country" ASC, "amount" DESC, "city" ASC
Here each country appears only once, within each country we then sort by amount DESC and take the first, and therefore highest, amount.
RANK and ROW_NUMBER window functions
These can be used basically as supersets of DISTINCT ON, and implemented tested as of both SQLite 3.34 and PostgreSQL 14.3. I highly recommend also looking into them, see e.g.: How to SELECT DISTINCT of one column and get the others?
This is how the above "city with the highest amount of each country" query would look like with ROW_NUMBER:
SELECT *
FROM (
SELECT
ROW_NUMBER() OVER (
PARTITION BY "country"
ORDER BY "amount" DESC, "city" ASC
) AS "rnk",
*
FROM "Sales"
) sub
WHERE
"sub"."rnk" = 1
ORDER BY
"sub"."country" ASC
Try
SELECT count(dimension1a)
FROM (SELECT DISTINCT ON (dimension1) dimension1a
FROM data_table
ORDER BY dimension1) AS tmp_table;
DISTINCT ON appears to be synonymous with GROUP BY.
I have a table with 10 columns.
I want to return all rows for which Col006 is distinct, but return all columns...
How can I do this?
if column 6 appears like this:
| Column 6 |
| item1 |
| item1 |
| item2 |
| item1 |
I want to return two rows, one of the records with item1 and the other with item2, along with all other columns.
In SQL Server 2005 and above:
;WITH q AS
(
SELECT *, ROW_NUMBER() OVER (PARTITION BY col6 ORDER BY id) rn
FROM mytable
)
SELECT *
FROM q
WHERE rn = 1
In SQL Server 2000, provided that you have a primary key column:
SELECT mt.*
FROM (
SELECT DISTINCT col6
FROM mytable
) mto
JOIN mytable mt
ON mt.id =
(
SELECT TOP 1 id
FROM mytable mti
WHERE mti.col6 = mto.col6
-- ORDER BY
-- id
-- Uncomment the lines above if the order matters
)
Update:
Check your database version and compatibility level:
SELECT ##VERSION
SELECT COMPATIBILITY_LEVEL
FROM sys.databases
WHERE name = DB_NAME()
The key word "DISTINCT" in SQL has the meaning of "unique value". When applied to a column in a query it will return as many rows from the result set as there are unique, different values for that column. As a consequence it creates a grouped result set, and values of other columns are random unless defined by other functions (such as max, min, average, etc.)
If you meant to say you want to return all rows for which Col006 has a specific value, then use the "where Col006 = value" clause.
If you meant to say you want to return all rows for which Col006 is different from all other values of Col006, then you still need to specify what that value is => see above.
If you want to say that the value of Col006 can only be evaluated once all rows have been retrieved, then use the "having Col006 = value" clause. This has the same effect as the "where" clause, but "where" gets applied when rows are retrieved from the raw tables, whereas "having" is applied once all other calculations have been made (i.e. aggregation functions have been run etc.) and just before the result set is returned to the user.
UPDATE:
After having seen your edit, I have to point out that if you use any of the other suggestions, you will end up with random values in all other 9 columns for the row that contains the value "item1" in Col006, due to the constraint further up in my post.
You can group on Col006 to get the distinct values, but then you have to decide what to do with the multiple records in each group.
You can use aggregates to pick a value from the records. Example:
select Col006, min(Col001), max(Col002)
from TheTable
group by Col006
order by Col006
If you want the values to come from a specific record in each group, you have to identify it somehow. Example of using Col002 to identify the record in each group:
select Col006, Col001, Col002
from TheTable t
inner join (
select Col006, min(Col002)
from TheTable
group by Col006
) x on t.Col006 = x.Col006 and t.Col002 = x.Col002
order by Col006
SELECT *
FROM (SELECT DISTINCT YourDistinctField FROM YourTable) AS A
CROSS APPLY
( SELECT TOP 1 * FROM YourTable B
WHERE B.YourDistinctField = A.YourDistinctField ) AS NewTableName
I tried the answers posted above with no luck... but this does the trick!
select * from yourTable where column6 in (select distinct column6 from yourTable);
SELECT *
FROM harvest
GROUP BY estimated_total;
You can use GROUP BY and MIN() to get more specific result.
Lets say that you have id as the primary_key.
And we want to get all the DISTINCT values for a column lets say estimated_total, And you also need one sample of complete row with each distinct value in SQL. Following query should do the trick.
SELECT *, min(id)
FROM harvest
GROUP BY estimated_total;
create table #temp
(C1 TINYINT,
C2 TINYINT,
C3 TINYINT,
C4 TINYINT,
C5 TINYINT,
C6 TINYINT)
INSERT INTO #temp
SELECT 1,1,1,1,1,6
UNION ALL SELECT 1,1,1,1,1,6
UNION ALL SELECT 3,1,1,1,1,3
UNION ALL SELECT 4,2,1,1,1,6
SELECT * FROM #temp
SELECT *
FROM(
SELECT ROW_NUMBER() OVER (PARTITION BY C6 Order by C1) ID,* FROM #temp
)T
WHERE ID = 1
For this MySQL SELECT statement:
SELECT * FROM MY_TABLE WHERE ID IN(x,y,y,z):
I want 4 rows back - ie I WANT row duplication for the case where I pass duplicate IDs in the list.
Is this possible?
using the IN() construct, that's not possible.
the only way i can think to do this is with a UNION:
SELECT * FROM my_table WHERE id = x
UNION ALL
SELECT * FROM my_table WHERE id = y
UNION ALL
SELECT * FROM my_table WHERE id = y
UNION ALL
SELECT * FROM my_table WHERE id = z
but in all honesty, i would just do the IN() like you have it and make your app code duplicate the rows as needed.
Put your IDs, including dups in a temp table and join your results on that table. The join will take care of filtering, but will keep duplicates if it's in the temp table twice
SELECT * FROM MY_TABLE WHERE ID IN(x,y,z)
union all
SELECT * FROM MY_TABLE WHERE ID IN(y)
To me, IN specify a set of values to search in (and duplication is a concept that conflict with the set one).
You should use other mean to reach your scope.