I ordered my results by their id's by:
CREATE TABLE my_table2 AS SELECT * FROM my_table ORDER BY record_group_id;
now when i execute:
SELECT DISTINCT record_group_id FROM my_table2 where rownum <=1000000;
I get gorup id's in random order, though my order by went fine:
Here is few of the records in result set
1599890050
1647717203
1647717120
1647717172
1647716972
1647717196
1647717197
1647717205
1599889999
1599889986
What could be the possible reason?
Shouldn't DISTINCT statement return records in same order as they are in table?
Neither SELECT or DISTINCT defines the order of data.
If you want ordered data explicitly define the Order you need.
SELECT DISTINCT record_group_id
FROM my_table2
WHERE rownum <=1000000
ORDER BY record_group_id;
The ordering only determines the order of the source data that is inserted in the table. If there is no clustered index in the table, that means that the records will be stored in that order physically.
However, how the records are stored doesn't guarantee that they will be selected in that order. The execution planner determines the most efficient way to run the query, which means that the data might not be fetched the way that you think it is, and it can differ from time to time as the data changes, or just the statistics about the data.
For a simple query like in the example, you usually get a predictable result, but there is no guarantee, so you always need to sort the query where you fetch the data to be sure to get a predictable result.
One reason that you don't get the data in the order that they are stored in the table in this case, may be that an index is used for filtering the result, and the records are returned in the order of the index rather than the order of the table.
Use ORDER BY on your SELECT statement:
SELECT DISTINCT record_group_id
FROM my_table2
WHERE rownum <=1000000
ORDER BY record_group_id;
Using DISTINCT has no effect on order, only on uniqueness of values.
If you want to control order too:
SELECT DISTINCT record_group_id
FROM my_table2
WHERE rownum <= 1000000
ORDER BY record_group_id -- Added this line
Your assumption that data in the table is ordered is wrong.
There is no implicit ordering in a database table - it's just a bag of unsorted data.
If you need ordered data, you'll have to use ORDER BY - there's no way around it (neither DISTINCT nor GROUP BY nor ...), see TomKyte Blog on Order By
Related
Are the following queries identical, or might I get different results (in any major DB system, e.g. MSSQL, MySQL, Postgres, SQLite):
Doing both in the same query:
SELECT group, some_agg_func(some_value)
FROM my_table
GROUP BY group
ORDER BY some_other_value
vs. ordering in a subquery:
SELECT group, some_agg_func(some_value)
FROM (
SELECT group, some_value
FROM my_table
ORDER BY some_other_value
) as alias
GROUP BY group
Looking at the first sample:
SELECT group, some_agg_func(some_value)
FROM my_table
GROUP BY group
ORDER BY some_other_value
Let's think about what GROUP BY does by looking at this imaginary sample data:
A B
- -
1 1
1 2
Then think about this query:
SELECT A
FROM SampleData
GROUP BY A
ORDER BY B
The GROUP BY clause puts the two rows into a single group. Then we want to order by B... but the two rows in the group have different values for B. Which should it use?
Obviously in this situation it doesn't really matter: there's only one row in the results, so the order is not relevant. But generally, how does the database know what to do?
The database could guess which one you want, or just take the first value, or the last — whatever those mean in a setting where the data is unordered by definition. And in fact this is what MySql will try to do for you: it will try to guess are your meaning. But this response is really inappropriate. You specified an in-exact query; the only correct thing to do is throw an error, which is what most databases will do.
Now let's look at the second sample:
SELECT group, some_agg_func(some_value)
FROM (
SELECT group, some_value
FROM my_table
ORDER BY some_other_value
) as alias
GROUP BY group
Here it is important to remember databases have their roots in relational set theory, and what we think of as "tables" are more formally described as Unordered Relations. Again: the idea of being "unordered" is baked into the very nature of a table at the deepest level.
In this case the inner query can run and create results in the specified order, and then the outer query can use that with GROUP BY to create a new set... but just like tables, query results are unordered relations. Without an ORDER BY clause the final result is also unordered by definition.
Now you might tend to get results in the order you want, but the reality is all bets are off. In fact, the databases that run this query will tend to give you results in the order in which they first encountered each group, which will not tend to match the ORDER BY because the GROUP BY expression is looking at completely different columns. Other databases (Sql Server is in this group) will not even allow the query to run, though I might prefer a warning here.
So now we come to the final section, where we must re-think the question, like this:
How can I use GROUP BY on the one group column, while also ordering by some_other_column not in the group?
The answer is each group can contain multiple rows, and so you must tell the database which row to look at to get the correct (specific) some_other_column value. The typical way to do this is with another aggregate function, which might look like this:
SELECT group, some_agg_func(some_value)
FROM my_table
GROUP BY group
ORDER BY some_other_agg_func(some_other_column)
That code will run without error on pretty much any database.
Just be careful here. On one hand, when people want to do this it's often for the common case where they know every record for some_other_column in each group will have the same value. For example, you might GROUP BY UserID, but ORDER BY Email, where of course every record with the same UserID should have the same Email address. As humans, we have the ability to make that kind of inference. Computers, however, don't handle that kind of thinking as well, and so we help it out with an extra aggregate function like MIN() or MAX().
On the other hand, if you're not careful sometimes the two different aggregate functions don't match up, and you end up showing the value from one row in the group, while using a completely different row from the group for the ORDER BY expression in a way that is not good.
Tables are unordered sets of data. A query result is a table. So if you select from a subquery that contains an ORDER BY clause, that clause means nothing; the data set is unordered by definition. The DBMS is free to ignore the ORDER BY clause. Some DBMS may even issue a warning or error, but I suppose it's more common that the ORDER BY clause just has no effect - at least not guaranteed.
In this query
SELECT group, some_agg_func(some_value)
FROM my_table
GROUP BY group
ORDER BY some_other_value
you try to order your results by some_other_value. If this is meant to be a column, you can't, because that other column is no part of your results. You'll get a syntax error. If some_other_value is a fixed value, then there is nothing ordered, because you'd have the same sort key for every row. But it can be an expression based on your result data (group key and aggreation results) and you can order your result rows by that.
In this query
SELECT group, some_agg_func(some_value)
FROM (
SELECT group, some_value
FROM my_table
ORDER BY some_other_value
) as alias
GROUP BY group
the ORDER BY clause has no effect. You could just as well just select FROM my_table directly:
SELECT group, some_agg_func(some_value)
FROM my_table as alias
GROUP BY group
This gets the results unordered (or at least the order you see is not guaranteed to be thus every time you run that query), because your query doesn't have an ORDER BY clause.
I want to run a SQL query in Postgres that is exactly the reverse of the one that you'd get by just running the initial query without an order by clause.
So if your query was:
SELECT * FROM users
Then
SELECT * FROM users ORDER BY <something here to make it exactly the reverse of before>
Would it just be this?
ORDER BY Desc
You are building on the incorrect assumption that you would get rows in a deterministic order with:
SELECT * FROM users;
What you get is really arbitrary. Postgres returns rows in any way it sees fit. For simple queries typically in order of their physical storage, which typically is the order in which rows were entered. But there are no guarantees, and the order may change any time between two calls. For instance after any UPDATE (writing a new physical row version), or when any background process reorders rows - like VACUUM. Or a more complex query might return rows according to an index or a join. Long story short: there is no reliable order for table rows in a relational database unless you specify it with ORDER BY.
That said, assuming you get rows from the above simple query in the order of physical storage, this would get you the reverse order:
SELECT * FROM users
ORDER BY ctid DESC;
ctid is the internal tuple ID signifying physical order. Related:
In-order sequence generation
How list all tables with data changes in the last 24 hours?
here is a tsql solution, thid might give you an idea how to do it in postgres
select * from (
SELECT *, row_number() over( order by (select 1)) rowid
FROM users
) x
order by rowid desc
I searched everywhere to find an SQL query to select rows randomly without changing the order. Almost everyone uses something like this:
SELECT * FROM table WHERE type = 1 ORDER BY RAND() LIMIT 25
But above query changes the order. I need a query which selects randomly among the rows but doesn't changes the order, cause every record has a date also.
Select the random rows and then re-order them:
select t.*
from (select *
from table t
where type = 1
order by rand()
limit 25
) t
order by datecol;
In SQL, if you want rows in a particular order, you need to use an explicit order by clause. You should never depend on the ordering of results with no order by. SQL does not guarantee the ordering. MySQL does not guarantee the ordering, unless the query has an order by.
I have SQL SELECT query which returns a lot of rows, and I have to split it into several partitions. Ie, set max results to 10000 and iterate the rows calling the query select time with increasing first result (0, 10000, 20000). All the queries are done in same transaction, and data that my queries are fetching is not changing during the process (other data in those tables can change, though).
Is it ok to use just plain select:
select a from b where...
Or do I have to use order by with the select:
select a from b where ... order by c
In order to be sure that I will get all the rows? In other word, is it guaranteed that query without order by will always return the rows in the same order?
Adding order by to the query drops performance of the query dramatically.
I'm using Oracle, if that matters.
EDIT: Unfortunately I cannot take advantage of scrollable cursor.
Order is definitely not guaranteed without an order by clause, but whether or not your results will be deterministic (aside from the order) would depend on the where clause. For example, if you have a unique ID column and your where clause included a different filter range each time you access it, then you would have non-ordered deterministic results, i.e.:
select a from b where ID between 1 and 100
select a from b where ID between 101 and 200
select a from b where ID between 201 and 300
would all return distinct result sets, but order would not be any way guaranteed.
No, without order by it is not guaranteed that query will ALWAYS return the rows in the same order.
No guarantees unless you have an order by on the outermost query.
Bad SQL Server example, but same rules apply. Not guaranteed order even with inner query
SELECT
*
FROM
(
SELECT
*
FROM
Mytable
ORDER BY SomeCol
) foo
Use Limit
So you would do:
SELECT * FROM table ORDER BY id LIMIT 0,100
SELECT * FROM table ORDER BY id LIMIT 101,100
SELECT * FROM table ORDER BY id LIMIT 201,100
The LIMIT would be from which position you want to start and the second variable would be how many results you want to see.
Its a good pagnation trick.
If the following SQL statements are executed in the order shown:
CREATE TABLE orders
(order_num INTEGER NOT NULL,
Buyer_name VARCHAR(35),
Amount NUMERIC(5,2));
CREATE UNIQUE INDEX idx_orderno ON orders(order_num);
whenever the ORDERS table is queried rows should be displayed in order of increasing ORDER_NUM values?
Relational databases don't work that way. An index is a means to select values doing less work (this is, not having to scan all rows to find a value), it's not a means to order the values in a result set.
In fact, most (all?) relational databases guarantee only one thing related to order, and that is that there is no guaranteed order if no order by clause is present in the query.
So, if you want ordered results you have to use the order by clause, like
select * from orders order by order_num
By default, order by column will order in ascending order. If you want it the other way around you can use order by column desc. order by column asc also exists.
To order by multiple columns you specify them separated by comma
select * from orders order by order_num asc, name desc
You need to specify that in the query, e.g.
SELECT * FROM orders ORDER BY order_num ASC;
There's no portable way to order by default for a certain table.
Unless you specify an ORDER BY clause in an SQL SELECT statement, there is no guarantee in which order the rows are returned. A given database may return them in the order of the first UNIQUE INDEX but this is not certain, and may even change over time for the same database.
To be sure of the order in which your rows are returned, always specify an ORDER BY clause in your SELECT statement. Like so:
SELECT * FROM orders ORDER BY order_num;
No, the rows will be displayed in an undefined order depending on the actual database software--usually the order they are stored in the database.
The unique index does have a sort order, but that is just to control how the index is stored and how efficiently it is used.
The correct way to do this is to add the desired sort to all your queries:
SELECT *
FROM [orders]
ORDER BY [order_num]
In general no, there is no reason for it.