Recently i was using the rownum in the sql query to fetch the data from oracle db in batch of 1000 records. for example 1 to 1000, 1001 to 2000 etc.
Note:
I have about a million records in the table
I am querying the table to get records 1000 at a time
I used the below query
SELECT NAME FROM (
SELECT NAME, ROWNUM RN
FROM employee
) WHERE RN >= ? AND RN <= ?
to fetch the data but stuck in strange issue.
The records in the db are unique but after complete execution I end up getting some duplicate record.
Is there any issue with the query? Is rownum causing issue? Is it possible that the record fetched in first batch of 1000 is also coming in the next subsequent batch?
You can't trust the order of records returned from a query without an explicit order by clause. If the inner query doesn't return values in the same order in each execution, you may get the same row in two separate executions, and thus duplicates in the overall execution.
Is there any issue with the query? Is rownum causing issue?
Yes, Oracle does not guarantee you that a common ordering will be used each time you see the output from a query. So ROWNUM, without an ORDER BY is not useful.Try these options.
In 11g use row_number with an order by, in Oracle 12c use FETCH..FIRST syntax, you should have a primary key / unique key column for this to work.
SELECT NAME
FROM (SELECT NAME,
row_number()
OVER(
ORDER BY primary_key_col ) rn
FROM employee)
WHERE rn >=?
AND rn <=? ;
SELECT NAME
FROM employee
ORDER BY primary_key_col offset ? FETCH first ? rows only;
Related
A Relational database table holds the information of Insurance details, say id and amount. Table consists of millions of records. requirement is to fetch top 5 records with max amount without using order by clause.
A solution I could think of is to use the temp table to maintain the max 5 records and update these entries each time the main table is updated but would like to know if there are better solution to above problem ?
An efficient way is to put an index on amount desc and use order by. Something like:
select t.*
from t
order by t.amount desc
fetch first 5 rows only; -- or however your database does this
This should be quite efficient.
You can try using analytic functions (example below), but you still have to order at some stage
select id,
amount
from (select id,
amount,
row_number() over (order by amount desc nulls last) as rn
from t)
where rn<=5;
I have a table with n number of records
How can i retrieve the nth record and (n-1)th record from my table in SQL without using derived table ?
I have tried using ROWID as
select * from table where rowid in (select max(rowid) from table);
It is giving the nth record but i want the (n-1)th record also .
And is there any other method other than using max,derived table and pseudo columns
Thanks
You cannot depend on rowid to get you to the last row in the table. You need an auto-incrementing id or creation time to have the proper ordering.
You can use, for instance:
select *
from (select t.*, row_number() over (order by <id> desc) as seqnum
from t
) t
where seqnum <= 2
Although allowed in the syntax, the order by clause in a subquery is ignored (for instance http://docs.oracle.com/javadb/10.8.2.2/ref/rrefsqlj13658.html).
Just to be clear, rowids have nothing to do with the ordering of rows in a table. The Oracle documentation is quite clear that they specify a physical access path for the data (http://docs.oracle.com/cd/B28359_01/server.111/b28318/datatype.htm#i6732). It is true that in an empty database, inserting records into a newtable will probably create a monotonically increasing sequence of row ids. But you cannot depend on this. The only guarantees with rowids are that they are unique within a table and are the fastest way to access a particular row.
I have to admit that I cannot find good documentation on Oracle handling or not handling order by's in subqueries in its most recent versions. ANSI SQL does not require compliant databases to support order by in subqueries. Oracle syntax allows it, and it seems to work in some cases, at least. My best guess is that it would probably work on a single processor, single threaded instance of Oracle, or if the data access is through an index. Once parallelism is introduced, the results would probably not be ordered. Since I started using Oracle (in the mid-1990s), I have been under the impression that order bys in subqueries are generally ignored. My advice would be to not depend on the functionality, until Oracle clearly states that it is supported.
select * from (select * from my_table order by rowid) where rownum <= 2
and for rows between N and M:
select * from (
select * from (
select * from my_table order by rowid
) where rownum <= M
) where rownum >= N
Try this
select top 2 * from table order by rowid desc
Assuming rowid as column in your table:
SELECT * FROM table ORDER BY rowid DESC LIMIT 2
Which of the following two SQL statements will return the desired result set (i.e. the ten rows with Status=0 and the highest StartTimes)?
Will both statements always return the same result set (StartTime is unique)?
SELECT *
FROM MyTable
WHERE Status=0
AND ROWNUM <= 10
ORDER BY StartTime DESC
SELECT *
FROM (
SELECT *
FROM MyTable
WHERE Status=0
ORDER BY StartTime DESC
)
WHERE ROWNUM <= 10
Background
My DBAdmin told me that the first statement will first limit the table to 10 rows and than order those random rows by StartTime, which is definitly not what I want.
From what I learned in that answer, the order by clause of the second statement is redundant and could be removed by an optimizer, which is also not what I want.
I asked a similar question concering the limit clause in a query to a SQLite database and am interested in understanding any differences to the above statements (using limit Vs rownum) used with an oracle db.
Your Second Query will work
Because in the first ,the first ten rows with Status 0 are selected and then the order by is done in that case the first ten rows fetched need not be in the highest order
I want to implement paging in a gridview or in an html table which I will fill using ajax. How should I write queries to support paging? For example if pagesize is 20 and when the user clicks page 3, rows between 41 and 60 must be shown on table. At first I can get all records and put them into cache but I think this is the wrong way. Because data can be very huge and data can be change from other sessions. so how can I implement this? Is there any generic way ( for all databases ) ?
As others have suggested, you can use rownum in Oracle. It's a little tricky though and you have to nest your query twice.
For example, to paginate the query
select first_name from some_table order by first_name
you need to nest it like this
select first_name from
(select rownum as rn, first_name from
(select first_name from some_table order by first_name)
) where rn > 100 and rn <= 200
The reason for this is that rownum is determined after the where clause and before the order by clause. To see what I mean, you can query
select rownum,first_name from some_table order by first_name
and you might get
4 Diane
2 Norm
3 Sam
1 Woody
That's because oracle evaluates the where clause (none in this case), then assigns rownums, then sorts the results by first_name. You have to nest the query so it uses the rownum assigned after the rows have been sorted.
The second nesting has to do with how rownum is treated in a where condition. Basically, if you query "where rownum > 100" then you get no results. It's a chicken and egg thing where it can't return any rows until it finds rownum > 100, but since it's not returning any rows it never increments rownum, so it never counts to 100. Ugh. The second level of nesting solves this. Note it must alias the rownum column at this point.
Lastly, your order by clause must make the query deterministic. For example, if you have John Doe and John Smith, and you order by first name only, then the two can switch places from one execution of the query to the next.
There are articles here http://www.oracle.com/technology/oramag/oracle/06-sep/o56asktom.html
and here http://www.oracle.com/technology/oramag/oracle/07-jan/o17asktom.html. Now that I see how long my post is, I probably should have just posted those links...
Unfortunately, the methods for restricting the range of rows returned by a query vary from one DBMS to another: Oracle uses ROWNUM (see ocdecio's answer), but ROWNUM won't work in SQL Server.
Perhaps you can encapsulate these differences with a function that takes a given SQL statement and first and last row numbers and generates the appropriate paginatd SQL for the target DBMS - i.e. something like:
sql = paginated ('select empno, ename from emp where job = ?', 101, 150)
which would return
'select * from (select v.*, ROWNUM rn from ('
+ theSql
+ ') v where rownum < 150) where rn >= 101'
for Oracle and something else for SQL Server.
However, note that the Oracle solution is adding a new column RN to the results that you'll need to deal with.
I believe that both have a ROWNUM analytic Function. Use that and you'll be identical.
In Oracle it is here
ROW_NUMBER
Yep, just verified that ROW_NUMBER is the same function in both.
"Because...data can be change from other sessions."
What do you want to happen for this ?
For example, user gets the 'latest' ten rows at 10:30.
At 10:31, 3 new rows are added (so those ten being view by the user are no longer the latest).
At 10:32, the user requests then 'next' ten entries.
Do you want that new set to include those three that have been bumped from 8/9/10 down to 11/12/13 ?
If not, in Oracle you can select the data as it was at 10:30
SELECT * FROM table_1 as of timestamp (timestamp '2009-01-29 10:30:00');
You still need the row_number logic, eg
select * from
(SELECT a.*, row_number() over (order by hire_date) rn
FROM hr.employees as of timestamp (timestamp '2009-01-29 10:30:00') a)
where rn between 10 and 19
select *
from ( select /*+ FIRST_ROWS(n) */ a.*,
ROWNUM rnum
from ( your_query_goes_here,
with order by ) a
where ROWNUM <=
:MAX_ROW_TO_FETCH )
where rnum >= :MIN_ROW_TO_FETCH;
Step 1: your query with order by
Step 2: select a.*, ROWNUM rnum from ()a where ROWNUM <=:MAX_ROW_TO_FETCH
Step 3: select * from ( ) where rnum >= :MIN_ROW_TO_FETCH;
put 1 in 2 and 2 in 3
If the expected data set is huge, I'd recommend to create a temp table, a view or a snapshot (materialized view) to store the query results + a row number retrieved either using ROWNUM or ROW_NUMBER analytic function. After that you can simply query this temp storage using row number ranges.
Basically, you need to separate the actual data fetch from the paging.
There is no uniform way to ensure paging across various RDBMS products. Oracle gives you rownum which you can use in where clause like:
where rownum < 1000
SQL Server gives you row_id( ) function which can be used similar to Oracle's rownum. However, row_id( ) isn't available before SQL Server 2005.
What is the most efficient way to read the last row with SQL Server?
The table is indexed on a unique key -- the "bottom" key values represent the last row.
If you're using MS SQL, you can try:
SELECT TOP 1 * FROM table_Name ORDER BY unique_column DESC
select whatever,columns,you,want from mytable
where mykey=(select max(mykey) from mytable);
You'll need some sort of uniquely identifying column in your table, like an auto-filling primary key or a datetime column (preferably the primary key). Then you can do this:
SELECT * FROM table_name ORDER BY unique_column DESC LIMIT 1
The ORDER BY column tells it to rearange the results according to that column's data, and the DESC tells it to reverse the results (thus putting the last one first). After that, the LIMIT 1 tells it to only pass back one row.
If some of your id are in order, i am assuming there will be some order in your db
SELECT * FROM TABLE WHERE ID = (SELECT MAX(ID) FROM TABLE)
I think below query will work for SQL Server with maximum performance without any sortable column
SELECT * FROM table
WHERE ID not in (SELECT TOP (SELECT COUNT(1)-1
FROM table)
ID
FROM table)
Hope you have understood it... :)
I tried using last in sql query in SQl server 2008 but it gives this err:
" 'last' is not a recognized built-in function name."
So I ended up using :
select max(WorkflowStateStatusId) from WorkflowStateStatus
to get the Id of the last row.
One could also use
Declare #i int
set #i=1
select WorkflowStateStatusId from Workflow.WorkflowStateStatus
where WorkflowStateStatusId not in (select top (
(select count(*) from Workflow.WorkflowStateStatus) - #i ) WorkflowStateStatusId from .WorkflowStateStatus)
You can use last_value: SELECT LAST_VALUE(column) OVER (PARTITION BY column ORDER BY column)...
I test it at one of my databases and it worked as expected.
You can also check de documentation here: https://msdn.microsoft.com/en-us/library/hh231517.aspx
OFFSET and FETCH NEXT are a feature of SQL Server 2012 to achieve SQL paging while displaying results.
The OFFSET argument is used to decide the starting row to return rows from a result and FETCH argument is used to return a set of number of rows.
SELECT *
FROM table_name
ORDER BY unique_column desc
OFFSET 0 Row
FETCH NEXT 1 ROW ONLY
SELECT TOP 1 id from comission_fees ORDER BY id DESC
In order to retrieve the last row of a table for MS SQL database 2005, You can use the following query:
select top 1 column_name from table_name order by column_name desc;
Note: To get the first row of the table for MS SQL database 2005, You can use the following query:
select top 1 column_name from table_name;
If you don't have any ordered column, you can use the physical id of each lines:
SELECT top 1 sys.fn_PhysLocFormatter(%%physloc%%) AS [File:Page:Slot],
T.*
FROM MyTable As T
order by sys.fn_PhysLocFormatter(%%physloc%%) DESC
SELECT * from Employees where [Employee ID] = ALL (SELECT MAX([Employee ID]) from Employees)
This is how you get the last record and update a field in Access DB.
UPDATE compalints SET tkt = addzone &'-'& customer_code &'-'& sn where sn in (select max(sn) from compalints )
If you have a Replicated table, you can have an Identity=1000 in localDatabase and Identity=2000 in the clientDatabase, so if you catch the last ID you may find always the last from client, not the last from the current connected database.
So the best method which returns the last connected database is:
SELECT IDENT_CURRENT('tablename')
Well I'm not getting the "last value" in a table, I'm getting the Last value per financial instrument. It's not the same but I guess it is relevant for some that are looking to look up on "how it is done now". I also used RowNumber() and CTE's and before that to simply take 1 and order by [column] desc. however we nolonger need to...
I am using SQL server 2017, we are recording all ticks on all exchanges globally, we have ~12 billion ticks a day, we store each Bid, ask, and trade including the volumes and the attributes of a tick (bid, ask, trade) of any of the given exchanges.
We have 253 types of ticks data for any given contract (mostly statistics) in that table, the last traded price is tick type=4 so, when we need to get the "last" of Price we use :
select distinct T.contractId,
LAST_VALUE(t.Price)over(partition by t.ContractId order by created ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING)
from [dbo].[Tick] as T
where T.TickType=4
You can see the execution plan on my dev system it executes quite efficient, executes in 4 sec while the exchange import ETL is pumping data into the table, there will be some locking slowing me down... that's just how live systems work.
It is very simple
select top 10 * from TableName order by 1 desc
SELECT * FROM TABLE WHERE ID = (SELECT MAX(ID) FROM TABLE)
I am pretty sure that it is:
SELECT last(column_name) FROM table
Becaause I use something similar:
SELECT last(id) FROM Status