I have a pipelined function . I have two sql statements as below.
The first one is as select * from table
and the second one is a select count(*) from table.
SELECT *
FROM table (es_feed_api_da.invoice_daily ('10-sep-2014'));
SELECT count(*)
FROM table (es_feed_api_da.invoice_daily ('10-sep-2014'));
i am running the two queries in toad.
I find that the second one (select count(*)) takes relatively more time than first one(select *)
Can someone please explain the reason to me..
Thanks
i am running the two queries in toad
I find that the second one (select count(*)) takes relatively more time than first one(select *) Can someone please explain the reason to me.
It is quite obvious that SELECT * would be faster than SELECT COUNT(*) because you are executing it on TOAD which is a GUI based client tool and gives you only first few rows(like only 50 rows in SQL Developer) when you simply project/select the rows. The time elapsed would keep increasing as and when you fetch more rows by scrolling down the query result.
On the other hand, when you do a SELECT COUNT(*), it must count all the rows in the table as opposed to SELECT * which only returns the first few rows in TOAD.
I don't have TOAD, but I can demonstrate the behaviour in SQL Developer.
Output of SELECT * ONLY first 50 rows:
After scrolling down to 500 rows:
The time taken to fetch further rows will increase as and when you scroll down further.
Related
I have the following query structure
CREATE TABLE <Table Name> AS
(
SELECT .... FROM ...
)
When i run the SELECT statement on its own, this compiles and returns the results within seconds. however when I run that with the CREATE Table Statement it takes hours to the point where I believe it has hung and will never compile.
What is the reason for this? and what could a work around be?
Oracle Database 12c <12.1.0.2.0>
If you ran that SELECT in some GUI, note that most (if not all) of them return only a few hundred rows, not the whole result set. For example: if your query really returns 20 million rows, GUI displays the first 50 (or 500, depending on tool you use) rows which is kind of confusing - just like it confused you.
If you used current query as an inline view, e.g.
select count(*)
from
(select ... from ...) --> this is your current query
it would "force" Oracle to fetch all rows, so you'd see how long it actually takes.
Apart from that, see if SELECT can be optimized, e.g.
see whether columns used in WHERE clause are indexed
collect statistics for all involved tables (used in the FROM clause)
remove ORDER BY clause (if there's any; it is irrelevant in CTAS operation)
check explain plan
Performance Tuning is far more from what I've suggested; those are just a few suggestions you might want to look at.
Have you tried Direct Load insert by first creating the table using CTAS where 1= 2and then doing the insert. This will atleast tell us if anything is wrong in data(corrupt data) or if it is a performance issue.
I had the same problem before since the new data is too large (7 million rows) and it took me 3 hours to execute the code.
My best suggestion is to create a view since it took less space instead of a new table.
So the answer to this one.
CREATE TABLE <Table Name> AS
(
SELECT foo
FROM baa
LEFT JOIN
( SELECT foo FROM baa WHERE DATES BETWEEN SYSDATE AND SYSDATE - 100 )
WHERE DATES_1 BETWEEN SYSDATE - 10 AND SYSDATE - 100
)
The problem was that the BETWEEN statements did not match the same time period and the sub query was looking at more data than the main query (I guess this was causing a full scan over the tables?)
The below query has the matching between statement time period and this returned the results in less than 3 minutes.
CREATE TABLE <Table Name> AS
(
SELECT foo FROM baa
LEFT JOIN ( SELECT foo FROM baa WHERE DATES BETWEEN SYSDATE - 10 AND SYSDATE - 100 )
WHERE DATES_1 BETWEEN SYSDATE - 10 AND SYSDATE - 100
)
I have a query as following:
SELECT Brand,Id,Model FROM product
Which takes time in order of seconds as Product table has more than 1 million records.
But the query executes within no time. (less than even one second))
select count(*) as numberOfRows from (SELECT Brand,Id,Model FROM product) result
Why is that?
When you execute a Query, the Time taken will vary depending on the Number of Columns and Rows and their datatypes.
In a table where you have 10 Columns, The Performance will be different if you select all columns (*) for all records and Just 1 or Two Columns for All records.
Because The Amount of data loaded is less in the Second case, it will execute faster.
Just like that, When you say Count(*) the result is Just a single Cell whereas in your first Select, you are selecting Millions of rows for those 3 columns, So the amount of Data is high.
That's why you are getting the Count(*) result faster. You don't need to give * inside the count, Instead Just use Count(1) for even more better performance.
I have an application for query management. Previously I was using SQL Server database, and to get the number of affected rows by a query I used to do:
SELECT * FROM TABLE (or any other select query)
and then I do SELECT ##ROWCOUNT to get the number of rows affected by the last executed query.
I have read about SQL%ROWCOUNT, but I am not able to make it work in a SELECT statement
Is there any way to do this in a Oracle database?. Thank you!
Edited:
I have solved this by doing SELECT COUNT(*) FROM (QUERY) to get the number of rows affected by the query, I discarted this approach because it was not working in SQL Server if the query contains an ORDER BY clause.
I don't know of any exact equivalent in Oracle that you can use in pure SQL.
An alternative that may work for you, depending on your specific need, is to add a count(*) over () to your select statement to give you the total number of rows. It would at least save you from having to re-execute the query a 2nd time.
select t.*,
count(*) over () as num_rows
from table t
where ...
Or, if you can't change the original query, then you can wrap it like this:
select t.*,
count(*) over () as num_rows
from (query) t
I have a table called "dutyroster". I want to make a random selection from this table's "names" column, but, I want the selection be different than the last 10 records so that the same guy is not given a second duty in 10 days. Is that possible ?
Create a temporary table with only one column called oldnames which will have no records initially. For each select, execute a query like
select names from dutyroster where dutyroster.names not in (select oldnamesfrom temporarytable) limit 10
and when execution is done add the resultset to the temporary table
The other answer already here is addressing the portion of the question on how to avoid duplicating selections.
To accomplish the random part of the selection, leverage newid() directly within your select statement. I've made this sqlfiddle as an example.
SELECT TOP 10
newid() AS [RandomSortColumn],
*
FROM
dutyroster
ORDER BY
[RandomSortColumn] ASC
Keep executing the query, and you'll keep getting different results. Use the technique in the other answer for avoiding doubling a guy up.
The basic idea is to use a subquery to get all but users from the last ten days, then sort the rest randomly:
select dr.*
from dutyroster dr
where dr.name not in (select dr2.name
from dutyroster dr2
where dr2.datetimecol >= date_sub(curdate(), interval 10 day)
)
order by rand()
limit 1;
Different databases may have different syntax for limit, rand(), and for the date/time functions. The above gives the structure of the query, but the functions may differ.
If you have a large amount of data and performance is a concern, there are other (more complicated) ways to take a random sample.
you could use TOP function for SQL Server
and for MYSQL you could use LIMIT function
Maybe this would help...
SELECT TOP number|percent column_name(s)
FROM table_name;
Source: http://www.w3schools.com/sql/sql_top.asp
I need to do a process on all the records in a table. The table could be very big so I rather process the records page by page. I need to remember the records that have already been processed so there are not included in my second SELECT result.
Like this:
For first run,
[SELECT 100 records FROM MyTable]
For second run,
[SELECT another 100 records FROM MyTable]
and so on..
I hope you get the picture. My question is how do I write such select statement?
I'm using oracle btw, but would be nice if I can run on any other db too.
I also don't want to use store procedure.
Thank you very much!
Any solution you come up with to break the table into smaller chunks, will end up taking more time than just processing everything in one go. Unless the table is partitioned and you can process exactly one partition at a time.
If a full table scan takes 1 minute, it will take you 10 minutes to break up the table into 10 pieces. If the table rows are physically ordered by the values of an indexed column that you can use, this will change a bit due to clustering factor. But it will anyway take longer than just processing it in one go.
This all depends on how long it takes to process one row from the table of course. You could chose to reduce the load on the server by processing chunks of data, but from a performance perspective, you cannot beat a full table scan.
You are most likely going to want to take advantage of Oracle's stopkey optimization, so you don't end up with a full tablescan when you don't want one. There are a couple ways to do this. The first way is a little longer to write, but let's Oracle automatically figure out the number of rows involved:
select *
from
(
select rownum rn, v1.*
from (
select *
from table t
where filter_columns = 'where clause'
order by columns_to_order_by
) v1
where rownum <= 200
)
where rn >= 101;
You could also achieve the same thing with the FIRST_ROWS hint:
select /*+ FIRST_ROWS(200) */ *
from (
select rownum rn, t.*
from table t
where filter_columns = 'where clause'
order by columns_to_order_by
) v1
where rn between 101 and 200;
I much prefer the rownum method, so you don't have to keep changing the value in the hint (which would need to represent the end value and not the number of rows actually returned to the page to be accurate). You can set up the start and end values as bind variables that way, so you avoid hard parsing.
For more details, you can check out this post