We are currently doing database performance tuning. Does Oracle database have any DMV Select Queries to find missing indexes directly?
Microsoft SqlServer has this below: https://blog.sqlauthority.com/2011/01/03/sql-server-2008-missing-index-script-download/
-- Missing Index Script
-- Original Author: Pinal Dave
SELECT TOP 25
dm_mid.database_id AS DatabaseID,
dm_migs.avg_user_impact*(dm_migs.user_seeks+dm_migs.user_scans) Avg_Estimated_Impact,
dm_migs.last_user_seek AS Last_User_Seek,
OBJECT_NAME(dm_mid.OBJECT_ID,dm_mid.database_id) AS [TableName],
'CREATE INDEX [IX_' + OBJECT_NAME(dm_mid.OBJECT_ID,dm_mid.database_id) + '_'
+ REPLACE(REPLACE(REPLACE(ISNULL(dm_mid.equality_columns,''),', ','_'),'[',''),']','')
+ CASE
WHEN dm_mid.equality_columns IS NOT NULL
AND dm_mid.inequality_columns IS NOT NULL THEN '_'
ELSE ''
END
+ REPLACE(REPLACE(REPLACE(ISNULL(dm_mid.inequality_columns,''),', ','_'),'[',''),']','')
+ ']'
+ ' ON ' + dm_mid.statement
+ ' (' + ISNULL (dm_mid.equality_columns,'')
+ CASE WHEN dm_mid.equality_columns IS NOT NULL AND dm_mid.inequality_columns
IS NOT NULL THEN ',' ELSE
'' END
+ ISNULL (dm_mid.inequality_columns, '')
+ ')'
+ ISNULL (' INCLUDE (' + dm_mid.included_columns + ')', '') AS Create_Statement
FROM sys.dm_db_missing_index_groups dm_mig
INNER JOIN sys.dm_db_missing_index_group_stats dm_migs
ON dm_migs.group_handle = dm_mig.index_group_handle
INNER JOIN sys.dm_db_missing_index_details dm_mid
ON dm_mig.index_handle = dm_mid.index_handle
WHERE dm_mid.database_ID = DB_ID()
ORDER BY Avg_Estimated_Impact DESC
GO
This is one good resource: https://www.dba-scripts.com/scripts/diagnostic-and-tuning/troubleshooting/find-missing-index/
If anyone has more answers, feel free to post
ACCEPT SCHEMA_NAME PROMPT 'Choose the schema to analyze:'
select * from (
select 'the column ' || c.name || ' of the table ' || us.name || '.' || o.name || ' was used ' || u.equality_preds || ' times in an equality predicate and ' || u.equijoin_preds || ' times in an equijoin predicate and is not indexed' as colum_to_index
from sys.col_usage$ u,
sys.obj$ o,
sys.col$ c,
sys.user$ us
where u.obj# = o.obj#
and u.obj# = c.obj#
and us.user# = o.owner#
and u.intcol# = c.col#
and us.name='&SCHEMA_NAME'
and c.name not in (select column_name from dba_ind_columns where index_owner ='&SCHEMA_NAME')
and (u.equality_preds > 100 OR u.equijoin_preds > 100)
order by u.equality_preds+u.equijoin_preds desc)
WHERE rownum <11;
Related
I have a large Snowflake database with 70+ tables and 3000+ fields. Is there a query I can use across the entire database to find all columns with all NULLs? I have a command I can use to find all the columns
select * from prod_db.information_schema.columns
Is there a way to modify that command to identify which columns are all NULLs? If there is not a way to do it across the entire database. Is there a way to do it across a table? I do not want to type:
select column_name from prod_db.information_schema.table_name
3000+ times. Thanks!
This uses a SQL generator to generate a SQL statement that will locate columns matching two criteria:
The column is in a table with one or more rows
The column has all nulls.
To be highly efficient, rather than checking the each table entirely, it uses a UNION ALL block that looks for a single non-null row in each table. It uses TOP 1 to find a not null row. That way as soon as it finds a not null row, it returns that row and stops scanning that table so it can move to another table scan.
This means that the large UNION ALL section will list tables where it finds a not null row, which is the opposite of what we want. To use this information, a CTE wrapped around the UNION ALL will do an anti-join against the column view in the information schema.
with COLS as
(
select 'select top 1 ''' || C.TABLE_CATALOG || ''' as TABLE_CATALOG, ''' || C.TABLE_SCHEMA ||
''' as TABLE_SCHEMA, ''' || C.TABLE_NAME || ''' as TABLE_NAME, ''' || C.COLUMN_NAME ||
''' as COLUMN_NAME from "' ||
C.TABLE_CATALOG || '"."' || C.TABLE_SCHEMA || '"."' || C.TABLE_NAME || '"' ||
' where "' || C.COLUMN_NAME || '" is not null'
as NULL_CHECK
from INFORMATION_SCHEMA.COLUMNS C
left join INFORMATION_SCHEMA.TABLES T on
C.TABLE_CATALOG = T.TABLE_CATALOG and
C.TABLE_SCHEMA = T.TABLE_SCHEMA and
C.TABLE_NAME = T.TABLE_NAME
where C.IS_NULLABLE = 'YES' and T.TABLE_TYPE = 'BASE TABLE'
and T.ROW_COUNT > 0
), UNIONED as
(
select listagg(NULL_CHECK, '\nunion all\n') as UNIONED from COLS
)
select replace($$
with NON_NULL_COLUMNS as (
!~UNIONED~!
)
select C.TABLE_CATALOG, C.TABLE_SCHEMA, C.TABLE_NAME, C.COLUMN_NAME from INFORMATION_SCHEMA.COLUMNS C
left join NON_NULL_COLUMNS NN
on C.TABLE_CATALOG = NN.TABLE_CATALOG
and C.TABLE_SCHEMA = NN.TABLE_SCHEMA
and C.TABLE_NAME = NN.TABLE_NAME
and C.COLUMN_NAME = NN.COLUMN_NAME
left join INFORMATION_SCHEMA.TABLES T
on C.TABLE_CATALOG = T.TABLE_CATALOG
and C.TABLE_SCHEMA = T.TABLE_SCHEMA
and C.TABLE_NAME = T.TABLE_NAME
where NN.COLUMN_NAME is null and T.ROW_COUNT > 0
;$$, '!~UNIONED~!', UNIONED) as SQL_TO_RUN from UNIONED
;
You can produce a list of SELECT queries for each column as follows
SELECT CONCAT(CONCAT(CONCAT(CONCAT(CONCAT(CONCAT(CONCAT(CONCAT(CONCAT(CONCAT(CONCAT(CONCAT('SELECT ''', TABLE_NAME), ''', '''), COLUMN_NAME), ''', '), 'COUNT(*) FROM '), TABLE_NAME), ' WHERE '), COLUMN_NAME), ' IS NULL OR '), LEN(COLUMN_NAME)), ' = 0'), ' UNION ')
from information_schema.columns
The result of the above query can then be taken and executed to get the result you need (PS: Remove the UNION on the last row produced from Step 1 before executing)
Hope this helps.
I have a redshift database that is being updated with new tables so I can't just manually list the tables I want. I want to get a count of the rows of all the tables from my query. So far I have:
select 'SELECT ''' || table_name || ''' as table_name, count(*) As con ' ||
'FROM ' || table_name ||
CASE WHEN lead(table_name) OVER (order by table_name ) IS NOT NULL
THEN ' UNION ALL ' END
FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_NAME LIKE '%results%'
but when I do this I get the error:
Specified types or functions (one per INFO message) not supported on Redshift tables.
I've searched a lot but I can't seem to find a solution for my problem. Any help would be greatly appreciated. Thanks!
EDIT:
I've changed my approach to this and decided to use a for loop in R to get the row counts of each but I'm running into the issue that 'row_counts' is only saving one number, not the count of each row like I want. Here is the code:
schema <- "x"
table_prefix <- "results"
geos <- ad_districts %>% filter(geo != "geo")
row_count <- list()
i = 1
for (geo in geos){
table_name <- paste0(schema, ".", table_prefix, geo)
row_count[[i]] <- dbGetQuery(con,
paste("SELECT COUNT(*) FROM", table_name))
i = i + 1
}
Your query is doing a select * for all tables, this will take a lot of time and resources. Instead use a system table to get the same info
select name, sum(rows) as rows
from stv_tbl_perm
where name like '%results%'
group by 1
[EDIT] - I think this is the root cause - some sql functions are only supported on the leader node. Try connecting to that node and re-run your SQL.
https://docs.aws.amazon.com/redshift/latest/dg/c_sql-functions-leader-node.html
Hope this helps.
select 'select count(*) as "' || table_schema || '.' || table_name || '" from ' || table_schema || '.' || table_name || ' ;' as sql_text
from information_schema.tables
;
[EDIT - refined this a bit to generate a series of statements that can be run at once]
select rownum, case when rownum > 1 then sql_text else replace(sql_text, 'union all', '') end as sql_text
from
(
select rank() over (order by sql_text DESC) as rownum,
sql_text
from
(
select 'select ''' || table_schema || ' ' || table_name || ''' , count(*) as "' || table_schema || '.' || table_name || '" from ' || table_schema || '.' || table_name || ' union all ' as sql_text
from information_schema.tables
where table_schema = 'public'
order by table_schema, table_name
)X
)Y
order by rownum desc ;
SELECT ' Select count(*) , '''+ tablename + ''' from '+'"' + tablename +'"' +' Union ALL '
FROM pg_table_def
GROUP BY tablename
Above query eliminates any table name with space. Remove UNION ALL at the end of the query and query will be ready to be executed.
I am trying to create a SQL code in SQL Server that dynamically selects all the tables in a specific database and then for each column in each table, counts the number of missing values and non-null values. I also want this result inserted into another table.
Is there any way I can do this without manually changing the column names for each:
Table Name - Column selection
I have a teradata code for the same which I tried to convert to SQL Server code. But I am unable to get the dynamic allocation and insertion parts right.
insert into temp
values (select ''CAMP'',
rtrim(''' || tablename || '''),
rtrim(''' || columnname || '''),
rtrim(''' || columnformat || '''),
count(1),
count(rtrim(upper(case when ' || columnname || '='''' then NULL else '|| columnname ||' end))),
(cast (count(rtrim(upper(case when ' || columnname || '='''' then NULL else ' || columnname || ' end))) as float) / (cast (count(1) as float))) * 100,
count(distinct rtrim(upper(case when ' || columnname || '='''' then NULL else '|| columnname ||' end))),
min(rtrim(upper(case when ' || columnname || '='''' then NULL else '|| columnname ||' end))),
max(rtrim(upper(case when ' || columnname || '='''' then NULL else '|| columnname ||' end))),
min(len(rtrim(upper(case when ' || columnname || '='''' then NULL else '|| columnname ||' end)))),
max(len(rtrim(upper(case when ' || columnname || '='''' then NULL else '|| columnname ||' end))))
from ' || tablename ||')
Any help on this front would be great!
Thanks!
Not sure if you need a UNION or a JOIN, but in either case you can just use a three-part name for the object in the other database if you are using multi-database:
USE database1; // Your database name
GO
CREATE VIEW dbo.MyView
AS
SELECT columns FROM dbo.Table1
UNION ALL
SELECT columns FROM database2.dbo.Table2; //second database
GO
select * from dbo.MyView // Getting all data from view
Hope that helps
Does something like this help you:
SELECT [Table] = t.[name]
, [Column] = c.[name]
FROM sys.tables t
INNER JOIN sys.columns c
ON c.[object_id] = t.[object_id]
SELECT *
FROM XYZ
WHERE column1 IN (X1,Y1,Z1);
I want to use LIKE with this select query; how can I write LIKE statement with IN, similar to the query as below:
SELECT *
FROM XYZ
WHERE column1 LIKE IN ($P{COLUMN});
ANOTHER EXAMPLE:-
select * from ( select REPLACE(REFEV_VEH_TYPE, ' ', '') ||
REPLACE(REFEV_CATEGORY, ' ', '') || REPLACE(REFEV_USAGE, ' ', '') ||
REPLACE(REFEV_BODY_CODE, ' ', '') as UC from TREF_ENF_VEHICLE_TYPE )
aa where aa.UC LIKE ('%06TBISUM%')
UC
------
06TBISUM
select * from ( select REPLACE(REFEV_VEH_TYPE, ' ', '') ||
REPLACE(REFEV_CATEGORY, ' ', '') || REPLACE(REFEV_USAGE, ' ', '') ||
REPLACE(REFEV_BODY_CODE, ' ', '') as UC from TREF_ENF_VEHICLE_TYPE )
aa where aa.UC LIKE ('%B')
UC
----------
06TGISJB
06TGITJB
select * from ( select REPLACE(REFEV_VEH_TYPE, ' ', '') ||
REPLACE(REFEV_CATEGORY, ' ', '') || REPLACE(REFEV_USAGE, ' ', '') ||
REPLACE(REFEV_BODY_CODE, ' ', '') as UC from TREF_ENF_VEHICLE_TYPE )
aa where aa.UC LIKE ('%BAS')
UC
----------
06BCIBAS
05BCABAS
05BCBBAS
The result that i want to know is :
select * from (
select REPLACE(REFEV_VEH_TYPE, ' ', '') || REPLACE(REFEV_CATEGORY, ' ', '') || REPLACE(REFEV_USAGE, ' ', '') || REPLACE(REFEV_BODY_CODE, ' ', '') as UC
from TREF_ENF_VEHICLE_TYPE ) aa
where aa.UC IN LIKE ('%06TBISUM%', '%B','%BAS').
UC
------
06TBISUM
06TGISJB
06TGITJB
06BCIBAS
05BCABAS
05BCBBAS
The parameter that i use is $P{COLUMN}.
i want to search multiple by $P{COLUMN} parameter using IN LIKE($P{COLUMN}).Anyone know about this?
A LIKE operation is functionally equivalent to multiple OR operations.
So..
SELECT *
FROM XYZ
WHERE ( column1 like 'X1' OR column1 like 'X2' OR column1 like 'X3' )
Try
where REGEXP_LIKE(aa.UC,'.+[06TBISUM.*|B|BAS].+')
Replace + with * (if I put * in it translates to italics)
You can't use a single variable as an array in DB2, but you can split it with a recursive CTE.
Here's a version without needing to define a function:
WITH Split AS (SELECT '' AS value, txt, LOCATE(',', txt) AS nxt
FROM (VALUES($P{COLUMN})) t(txt)
UNION ALL
SELECT DECODE(nxt, 0, txt, SUBSTR(txt, 1, nxt - 1)),
DECODE(nxt, 0, '', SUBSTR(txt, nxt + 1)),
DECODE(txt, '', 0, LOCATE(',', txt, nxt + 1))
FROM Split
WHERE LENGTH(txt) > 0),
SearchData AS (SELECT value
FROM Split
WHERE value <> '')
SELECT uc
FROM (SELECT REPLACE(refev_veh_type, ' ', '')
|| REPLACE(refev_category, ' ', '')
|| REPLACE(refev_usage, ' ', '')
|| REPLACE(refev_body_code, ' ', '') AS uc
FROM TRef_ENF_Vehicle_Type ) T
JOIN SearchData
ON T.uc LIKE SearchData.value
(not tested, as I don't have a DB2 instance handy)
$P{COLUMN} is assumed to contain the following string: '%06TBISUM%,%B,%BAS'.
I have a table with mixed types of data (real, integrer, character ...) but i would only recover columns that have real values.
I can construct this:
SELECT 'SELECT ' || array_to_string(ARRAY(
select 'o' || '.' || c.column_name
from information_schema.columns as c
where table_name = 'final_datas'
and c.data_type = 'real'), ',') || ' FROM final_datas as o' As sqlstmt
that gives that:
"SELECT o.random,o.struct2d_pred2_num,o.pfam_num,o.transmb_num [...] FROM final_datas as o"
The i would like to create a table with these columns. Of course, do this, doesn't work:
create table table2 as (
SELECT 'SELECT ' || array_to_string(ARRAY(
select 'o' || '.' || c.column_name
from information_schema.columns as c
where table_name = 'final_datas'
and c.data_type = 'real'), ',') || ' FROM final_datas as o' As sqlstmt
)
Suggestions?
You need to generate the whole CREATE TABLE statement as dynamic SQL:
SELECT 'CREATE TABLE table2 AS SELECT ' || array_to_string(ARRAY(
select 'o' || '.' || c.column_name
from information_schema.columns as c
where table_name = 'final_datas'
and c.data_type = 'real'), ',') || ' FROM final_datas as o' As sqlstmt
The result can be run with EXECUTE sqlstmt;