In PostgreSQL, does this query
SELECT "table".* FROM "table" WHERE "table"."column" IN (1, 5, 3)
always return the results in the 1, 5, 3 order or is it ambiguous?
If it's ambiguous, how do I properly assure the results are in the order 1, 5, 3?
The WHERE clause will not order the results in any way, it will just select matching records, in whatever order the database index finds them in.
You'll have to add an order by clause.
Add something like the following to your select statement
order by CASE WHEN "column"=1 THEN 1
WHEN "column"=2 THEN 2
ELSE 3
END
if you have many more than three values it may be easier to make a lookup table and join to that in you query
False orders before true
SELECT "table".*
FROM "table"
WHERE "table"."column" IN (1, 5, 3)
order by
"column" != 1,
"column" != 5,
"column" != 3
When you use "IN" in a select statement Postgre just return the matching rows against that range of values. It's not ambiguous in any way but if you need to order the results you need to explicitly add ORDER BY "column1,column2..."
A set knows no order per se. A SELECT query needs ORDER BY to return ordered rows.
Other answers have suggested CASE statements or boolean expressions in your ORDER BY, but that's far from elegant and rather inefficient with big tables. I suggest to use an array or a comma-separated string instead of a set for your query.
For a given table:
CREATE TABLE tbl (col int);
Using an array it can work like this:
SELECT col
FROM tbl
JOIN (
SELECT col, row_number() OVER () AS rn
FROM unnest('{1,5,3}'::int[]) AS col
) u USING (col)
ORDER BY rn;
Returns all rows found in the sequence of the input array:
-> SQLfiddle
For more details and future-proof code consider this closely related question:
PostgreSQL unnest() with element number
Or the corresponding question on dba.SE:
How to preserve the original order of elements in an unnested array?
Related
Ok so we know we can check if a column has duplicate values by using using Group By along with Having keyword and an aggregate function like Count().
What if I want to do is to print those duplicate values, not once but as many times as they are present in the table?
So if a column contains: 1,2,2,2,3,4,5,5,6,7,8,8,8.
I want my query to return: 2,2,2,5,5,8,8,8.
Group by/distinct will only show 2, 5, 8. How can I get what I want?
Thank you.
You can compute a window count in a subquery, then use it to filter the resultset:
select val
from (select val, count(*) over(partition by val) cnt from mytable) t
where cnt > 1
I have a table with two columns namely ID and KEY (let key here be an integer) such as
ID KEY
ABC 6
DEF 1
GHI 12
TASK: Get the ID of the MAX key
Solution 1:
Select Top(1) ID
from TABLE
order by KEY desc
Solution 2:
Select ID
from TABLE
where ID = MAX(ID)
EDIT: The query was invalid. This is what I meant:
Select ID
from TABLE
where KEY = (select max(KEY) from TABLE)
Is one of these solutions categorically better than the other? What are the advantages/disvantages of each solution.
EDIT:
Assume there is no index.
Case 1 - large table
Case 2 - small table
Background:
I am doing code review and I have found both solutions multiple times in different context - sometimes with indices, sometimes without, sometimes for large tables, sometimes for small.
The two queries are different (after your edits fixing the second one).
The first necessarily returns a single row.
The second returns all matching rows.
The first returns a row even when key is NULL.
The second does not.
You should use the logic that does what you want.
An aggregate may not appear in the WHERE clause unless it is in a subquery contained in a HAVING clause or a select list..
Solution 1 will be the best. A subquery in a where clause will be less optimal.
There really are lots of design techniques to look at for performance which I am not going to go into with this answer. I found this article yesterday which gave me more perspective https://www.red-gate.com/simple-talk/sql/database-administration/sql-server-storage-internals-101/
In Solution 1, the order by clause will just sort your query result.
Query execution order:
FROM clause ON clause OUTER clause WHERE clause GROUP BY clause HAVING clause SELECT clause DISTINCT clause ORDER BY clause TOP clause
You can use the following query:
Select ID,
RANK() OVER (ORDER BY KEY DESC) AS KeyRank
from table1
HAVING keyRank = 1
Solution 1 will work but Solution 2 will throw exception like bellow
Msg 147, Level 15, State 1, Line 22 An aggregate may not appear in the
WHERE clause unless it is in a subquery contained in a HAVING clause
or a select list, and the column being aggregated is an outer
reference.
You can go with query 1 ,
You cannot use query 2 because you cannot use aggregate function like that if you want to use where clause and aggregate function in your query you have to go with as below :
Select id from table where key in (select max(key) from test);
reference only using aggregate function and having clause
Select ID ,max(key)
from test
group by ID,key
having (key) >= 12
order by 1
My situation is that a SQL statement which is not predictable, is given to the program and I need to do pagination on top of it. The final SQL statement would be similar to the following one:
SELECT * FROM (*Given SQL Statement*) b
OFFSET 0 ROWS FETCH NEXT 50 ROWS ONLY;
The problem here is that the *Given SQL Statement* is unpredictable. It may or may not contain order by clause. I am not able to change the query result of this SQL Statement and I need to do pagination on it.
I searched for solution on the Internet, but all of them suggested to use an arbitrary column, like primary key, in order by clause. But it will change the original order.
The short answer is that it can't be done, or at least can't be done properly.
The problem is that SQL Server (or any RDBMS) does not and can not guarantee the order of the records returned from a query without an order by clause.
This means that you can't use paging on such queries.
Further more, if you use an order by clause on a column that appears multiple times in your resultset, the order of the result set is still not guaranteed inside groups of values in said column - quick example:
;WITH cte (a, b)
AS
(
SELECT 1, 'a'
UNION ALL
SELECT 1, 'b'
UNION ALL
SELECT 2, 'a'
UNION ALL
SELECT 2, 'b'
)
SELECT *
FROM cte
ORDER BY a
Both result sets are valid, and you can't know in advance what will you get:
a b
-----
1 b
1 a
2 b
2 a
a b
-----
1 a
1 b
2 a
2 b
(and of course, you might get other sorts)
The problem here is that the *Given SQL Statement" is unpredictable. It may or may not contain order by clause.
your inner query(unpredictable sql statement) should not contain order by,even if it contains,order is not guaranteed.
To get guaranteed order,you have to order by some column.for the results to be deterministic,the ordered column/columns should be unique
Please note: what I'm about to suggest is probably horribly inefficient and should really only be used to help you go back to the project leader and tell them that pagination of an unordered query should not be done. Having said that...
From your comments you say you are able to change the SQL statement before it is executed.
You could write the results of the original query to a temporary table, adding row count field to be used for subsequent pagination ordering.
Therefore any original ordering is preserved and you can now paginate.
But of course the reason for needing pagination in the first place is to avoid sending large amounts of data to the client application. Although this does prevent that, you will still be copying data to a temp table which, depending on the row size and count, could be very slow.
You also have the problem that the page size is coming from the client as part of the SQL statement. Parsing the statement to pick that out could be tricky.
As other notified using anyway without using a sorted query will not be safe, But as you know about it and search about it, I can suggest using a query like this (But not recommended as a good way)
;with cte as (
select *,
row_number() over (order by (select 0)) rn
from (
-- Your query
) t
)
select *
from cte
where rn between (#pageNumber-1)*#pageSize+1 and #pageNumber*#pageSize
[SQL Fiddle Demo]
I finally found a simple way to do it without any order by on a specific column:
declare #start AS INTEGER = 1, #count AS INTEGER = 5;
select * from (SELECT *,ROW_NUMBER() OVER (ORDER BY (SELECT 1)) AS fakeCounter
FROM (select * from mytable) AS t) AS t2 order by fakeCounter OFFSET #start ROWS
FETCH NEXT #count ROWS ONLY
where select * from mytable can be any query
I have two table functions that return a single column each. One function is guaranteed to return the same number of rows as the other.
I want to insert the values into a new two-column table. One colum will receive the value from the first udf, the second column from the second udf. The order of the inserts will be the order in which the rows are returned by the udfs.
How can I JOIN these two udfs given that they do not share a common key? I've tried using a ROW_NUMBER() but can't quite figure it out:
INSERT INTO dbo.NewTwoColumnTable (Column1, Column2)
SELECT udf1.[value], udf2.[value]
FROM dbo.udf1() udf1
INNER JOIN dbo.udf2() udf2 ON ??? = ???
This will not help you, but SQL does not guarantee row order unless it is asked to explicitly, so the idea that they will be returned in the order you expect may be true for a given set, but as I understand the idea of set based results, is fundamentally not guaranteed to work properly. You probably want to have a key returned from the UDF if it is associated with something that guarantees the order.
Despite this, you can do the following:
declare #val int
set #val=1;
Select Val1,Val2 from
(select Value as Val2, ROW_NUMBER() over (order by #val) r from udf1) a
join
(select Value as Val2, ROW_NUMBER() over (order by #val) r from udf2) b
on a.r=b.r
The variable addresses the issue of needing a column to sort by.
If you have the privlidges to edit the UDF, I think the better practice is to already sort the data coming out of the UDF, and then you can add ident int identity(1,1) to your output table in the udf, which makes this clear.
The reaosn this might matter is if your server decided to split the udf results into two packets. If the two arrive out of the order you expected, SQL could return them in the order received, which ruins the assumption made that he UDF will return rows in order. This may not be an issue, but if the result is needed later for a real system, proper programming here prevents unexpected bugs later.
In SQL, the "order returned by the udfs" is not guaranteed to persist (even between calls).
Try this:
WITH q1 AS
(
SELECT *, ROW_NUMBER() OVER (ORDER BY whatever1) rn
FROM udf1()
),
q2 AS
(
SELECT *, ROW_NUMBER() OVER (ORDER BY whatever2) rn
FROM udf2()
)
INSERT
INTO dbo.NewTwoColumnTable (Column1, Column2)
SELECT q1.value, q2.value
FROM q1
JOIN q2
ON q2.rn = q1.rn
PostgreSQL 9.4+ could append a INT8 column at the end of the udfs result using the WITH ORDINALITY suffix
-- set returning function WITH ORDINALITY
SELECT * FROM pg_ls_dir('.') WITH ORDINALITY AS t(ls,n);
ls | n
-----------------+----
pg_serial | 1
pg_twophase | 2
postmaster.opts | 3
pg_notify | 4
official doc: http://www.postgresql.org/docs/devel/static/functions-srf.html
related blogspot: http://michael.otacoo.com/postgresql-2/postgres-9-4-feature-highlight-with-ordinality/
I have a table:
abc_test with columns n_num, k_str.
This query doesnt work:
select distinct(n_num) from abc_test order by(k_str)
But this one works:
select n_num from abc_test order by(k_str)
How do DISTINCT and ORDER BY keywords work internally that output of both the queries is changed?
As far as i understood from your question .
distinct :- means select a distinct(all selected values should be unique).
order By :- simply means to order the selected rows as per your requirement .
The problem in your first query is
For example :
I have a table
ID name
01 a
02 b
03 c
04 d
04 a
now the query select distinct(ID) from table order by (name) is confused which record it should take for ID - 04 (since two values are there,d and a in Name column). So the problem for the DB engine is here when you say
order by (name).........
You might think about using group by instead:
select n_num
from abc_test
group by n_num
order by min(k_str)
The first query is impossible.
Lets explain this by example. we have this test:
n_num k_str
2 a
2 c
1 b
select distinct (n_num) from abc_test is
2
1
Select n_num from abc_test order by k_str is
2
1
2
What do you want to return
select distinct (n_num) from abc_test order by k_str?
it should return only 1 and 2, but how to order them?
How do extended sort key columns
The logical order of operations in SQL for your first query, is (simplified):
FROM abc_test
SELECT n_num, k_str i.e. add a so called extended sort key column
ORDER BY k_str DESC
SELECT n_num i.e. remove the extended sort key column again from the result.
Thanks to the SQL standard extended sort key column feature, it is possible to order by something that is not in the SELECT clause, because it is being temporarily added to it behind the scenes prior to ordering, and then removed again after ordering.
So, why doesn't this work with DISTINCT?
If we add the DISTINCT operation, it would need to be added between SELECT and ORDER BY:
FROM abc_test
SELECT n_num, k_str i.e. add a so called extended sort key column
DISTINCT
ORDER BY k_str DESC
SELECT n_num i.e. remove the extended sort key column again from the result.
But now, with the extended sort key column k_str, the semantics of the DISTINCT operation has been changed, so the result will no longer be the same. This is not what we want, so both the SQL standard, and all reasonable databases forbid this usage.
Workarounds
PostgreSQL has the DISTINCT ON syntax, which can be used here for precisely this job:
SELECT DISTINCT ON (k_str) n_num
FROM abc_test
ORDER BY k_str DESC
It can be emulated with standard syntax as follows, if you're not using PostgreSQL
SELECT n_num
FROM (
SELECT n_num, MIN(k_str) AS k_str
FROM abc_test
GROUP BY n_num
) t
ORDER BY k_str
Or, just simply (in this case)
SELECT n_num, MIN(k_str) AS k_str
FROM abc_test
GROUP BY n_num
ORDER BY k_str
I have blogged about SQL DISTINCT and ORDER BY more in detail here.
You are selecting the collection distinct(n_num) from the resultset from your query. So there is no actual relation with the column k_str anymore. A n_num can be from two rows each having a different value for k_str. So you can't order the collection distinct(n_num) by k_str.
According to SQL Standards, a SELECT clause may refer either to as clauses ("aliases") in the top level SELECT clause or columns of the resultset by ordinal position, and therefore nether of your queries would be compliant.
It seems Oracle, in common with other SQL implemetations, allows you to refer to columns that existed (logically) immediately prior to being projected away in the SELECT clause. I'm not sure whether such flexibility is such a good thing: IMO it is good practice to expose the sort order to the calling application by including the column/expressions etc in the SELECT clause.
As ever, you need to apply dsicpline to get meaningful results. For your first query, the definition of order is potentially entirely arbitrary.You should be grateful for the error ;)
This approach is available in SQL server 2000, you can select distinct values from a table and order by different column which is not included in Distinct.
But in SQL 2012 this will through you an error
"ORDER BY items must appear in the select list if SELECT DISTINCT is specified."
So, still if you want to use the same feature as of SQL 2000 you can use the column number for ordering(its not recommended in best practice).
select distinct(n_num) from abc_test order by 1
This will order the first column after fetching the result. If you want the ordering should be done based on different column other than distinct then you have to add that column also in select statement and use column number to order by.
select distinct(n_num), k_str from abc_test order by 2
When I got same error, I got it resolved by changing it as
SELECT n_num
FROM(
SELECT DISTINCT(n_num) AS n_num, k_str
FROM abc_test
) as tbl
ORDER BY tbl.k_str
My query doesn't match yours exactly, but it's pretty close.
select distinct a.character_01 , (select top 1 b.sort_order from LookupData b where a.character_01 = b.character_01 )
from LookupData a
where
Dataset_Name = 'Sample' and status = 200
order by 2, 1
did you try this?
SELECT DISTINCT n_num as iResult
FROM abc_test
ORDER BY iResult
you can do
select distinct top 10000 (n_num) --assuming you won't have more than 10,000 rows
from abc_test order by(k_str)