I'm using Metabase for analytics. Sometimes it counts table rows.
I catch its query
SELECT count(*) AS "count" FROM "dbo"."TableName"
But the problem is that TableName isn't a table, it's a view.
This view is a union of many tables. That query runs very long time. Instead of SELECT COUNT I'd like to use
SELECT row_count FROM sys.dm_db_partition_stats
In that case, I get the result instantly.
Is there any possibility to replace metabase's query with my query?
Thanks
Related
I ran two queries to get count of records for two different dates from a Hive managed table partitioned on process date field.
SELECT COUNT(1) FROM prd_fct.mktng WHERE process_dt='2018-01-01' --returned 2 million
SELECT COUNT(1) FROM prd_fct.mktng WHERE process_dt='2018-01-02' --returned 3 million
But if I ran the below query with a UNION ALL clause, the counts returned are different from that of above mentioned individual queries.
SELECT '2018-01-01', COUNT(1) FROM prd_fct.mktng WHERE process_dt='2018-01-01'
UNION ALL
SELECT '2018-01-02', COUNT(1) FROM prd_fct.mktng WHERE process_dt='2018-01-02'
What can be the root cause for this difference?
One of our teammate helped us to identify the issue.
When we run a single count() query,the query is not physically executed on table rather count will be taken from statistics.
One of the remedy is to collect the stats on table agian,then the count() on single table will reflect actual count
Regards,
Anoop
I too faced a similar issue with count(*) returning incorrect count. I added the below to my code and the counts are consistent now.
For non-partitioned table use:
ANALYZE TABLE your_table_name COMPUTE STATISTICS
For partitioned table, analyze the recently added partition by specifying the partition value:
ANALYZE TABLE your_table_name
PARTITION(your_partition_name=your_partition_value)
COMPUTE STATISTICS;
I have an app that has to query hundreds of BigQuery tables (in a Dataflow job) , some of which may not exist (there are tables named by day for events which occur on each day, and some days some tables may not have been created).
Is there a way to write a BQ SQL query such that it makes a SELECT against some_table if and only if the named table exists, and returns no rows otherwise?
Someone had posted a query which returns if a table exists
#standardSQL
SELECT COUNT(1) AS table_count
FROM `my-project:blah.__TABLES_SUMMARY__`
WHERE table_id = 'some-table-name-2017-04-02'
But we are trying to do a job in Dataflow, and its difficult to make these queries first outside of the dataflow control structure.
Is there a way to combine something like the query above with a SELECT against that table 'some-table-name-2017-04-02', in a single SQL statement such that if the table does not exist, we just get no rows back, rather than an error?
The problem is that the BigQuery SQL parser will not even compile a query if it references a table name that does not exist, even if no query is done to that table.
Here you can check budd
if not exists
(select * from sysobjects where name="my table")
begin
execute "create table mytable(x int)"
end
I have a pipelined function . I have two sql statements as below.
The first one is as select * from table
and the second one is a select count(*) from table.
SELECT *
FROM table (es_feed_api_da.invoice_daily ('10-sep-2014'));
SELECT count(*)
FROM table (es_feed_api_da.invoice_daily ('10-sep-2014'));
i am running the two queries in toad.
I find that the second one (select count(*)) takes relatively more time than first one(select *)
Can someone please explain the reason to me..
Thanks
i am running the two queries in toad
I find that the second one (select count(*)) takes relatively more time than first one(select *) Can someone please explain the reason to me.
It is quite obvious that SELECT * would be faster than SELECT COUNT(*) because you are executing it on TOAD which is a GUI based client tool and gives you only first few rows(like only 50 rows in SQL Developer) when you simply project/select the rows. The time elapsed would keep increasing as and when you fetch more rows by scrolling down the query result.
On the other hand, when you do a SELECT COUNT(*), it must count all the rows in the table as opposed to SELECT * which only returns the first few rows in TOAD.
I don't have TOAD, but I can demonstrate the behaviour in SQL Developer.
Output of SELECT * ONLY first 50 rows:
After scrolling down to 500 rows:
The time taken to fetch further rows will increase as and when you scroll down further.
I have an application for query management. Previously I was using SQL Server database, and to get the number of affected rows by a query I used to do:
SELECT * FROM TABLE (or any other select query)
and then I do SELECT ##ROWCOUNT to get the number of rows affected by the last executed query.
I have read about SQL%ROWCOUNT, but I am not able to make it work in a SELECT statement
Is there any way to do this in a Oracle database?. Thank you!
Edited:
I have solved this by doing SELECT COUNT(*) FROM (QUERY) to get the number of rows affected by the query, I discarted this approach because it was not working in SQL Server if the query contains an ORDER BY clause.
I don't know of any exact equivalent in Oracle that you can use in pure SQL.
An alternative that may work for you, depending on your specific need, is to add a count(*) over () to your select statement to give you the total number of rows. It would at least save you from having to re-execute the query a 2nd time.
select t.*,
count(*) over () as num_rows
from table t
where ...
Or, if you can't change the original query, then you can wrap it like this:
select t.*,
count(*) over () as num_rows
from (query) t
I need to get the total no of records in a table for pagination purpose..it has some around 1 million records...the count is calculated as part of page load..i can do count query but it takes little more time thus increasing the time to page load.So to avoid that i have used
select ROW_NUMS,OWNER from ALL_TABLES where table_name='table1'
But the problem is that this query for some tables gives result as zero and also sometimes the count is not correct(different from count query)
any idea how to update the all_tables data for a table and how all_table
If You want the exact number of records in a table, You need to select count(*). This result will be correct. NUM_ROWS has never been to provide the exact number of rows.
You can gather the table, but this will give you the estimate count.
exec dbms_stats.gather_table_stats('<OWNER>', '<TABLE_NAME>');
To get an accurate row count you can get the database to do the heavy lifting before it sends the data to the client, e.g.:
SELECT t.*
,COUNT(*) OVER () AS row_count
FROM my_table t;
You need to run ANALYZE statement to properly update that column