We are currently doing database performance tuning. Does Oracle database have any DMV Select Queries to find missing indexes directly?
Microsoft SqlServer has this below: https://blog.sqlauthority.com/2011/01/03/sql-server-2008-missing-index-script-download/
-- Missing Index Script
-- Original Author: Pinal Dave
SELECT TOP 25
dm_mid.database_id AS DatabaseID,
dm_migs.avg_user_impact*(dm_migs.user_seeks+dm_migs.user_scans) Avg_Estimated_Impact,
dm_migs.last_user_seek AS Last_User_Seek,
OBJECT_NAME(dm_mid.OBJECT_ID,dm_mid.database_id) AS [TableName],
'CREATE INDEX [IX_' + OBJECT_NAME(dm_mid.OBJECT_ID,dm_mid.database_id) + '_'
+ REPLACE(REPLACE(REPLACE(ISNULL(dm_mid.equality_columns,''),', ','_'),'[',''),']','')
+ CASE
WHEN dm_mid.equality_columns IS NOT NULL
AND dm_mid.inequality_columns IS NOT NULL THEN '_'
ELSE ''
END
+ REPLACE(REPLACE(REPLACE(ISNULL(dm_mid.inequality_columns,''),', ','_'),'[',''),']','')
+ ']'
+ ' ON ' + dm_mid.statement
+ ' (' + ISNULL (dm_mid.equality_columns,'')
+ CASE WHEN dm_mid.equality_columns IS NOT NULL AND dm_mid.inequality_columns
IS NOT NULL THEN ',' ELSE
'' END
+ ISNULL (dm_mid.inequality_columns, '')
+ ')'
+ ISNULL (' INCLUDE (' + dm_mid.included_columns + ')', '') AS Create_Statement
FROM sys.dm_db_missing_index_groups dm_mig
INNER JOIN sys.dm_db_missing_index_group_stats dm_migs
ON dm_migs.group_handle = dm_mig.index_group_handle
INNER JOIN sys.dm_db_missing_index_details dm_mid
ON dm_mig.index_handle = dm_mid.index_handle
WHERE dm_mid.database_ID = DB_ID()
ORDER BY Avg_Estimated_Impact DESC
GO
This is one good resource: https://www.dba-scripts.com/scripts/diagnostic-and-tuning/troubleshooting/find-missing-index/
If anyone has more answers, feel free to post
ACCEPT SCHEMA_NAME PROMPT 'Choose the schema to analyze:'
select * from (
select 'the column ' || c.name || ' of the table ' || us.name || '.' || o.name || ' was used ' || u.equality_preds || ' times in an equality predicate and ' || u.equijoin_preds || ' times in an equijoin predicate and is not indexed' as colum_to_index
from sys.col_usage$ u,
sys.obj$ o,
sys.col$ c,
sys.user$ us
where u.obj# = o.obj#
and u.obj# = c.obj#
and us.user# = o.owner#
and u.intcol# = c.col#
and us.name='&SCHEMA_NAME'
and c.name not in (select column_name from dba_ind_columns where index_owner ='&SCHEMA_NAME')
and (u.equality_preds > 100 OR u.equijoin_preds > 100)
order by u.equality_preds+u.equijoin_preds desc)
WHERE rownum <11;
I'm hoping someone has a quick suggestion to the following, based on the following sample table:
|Column1 |Column2 |Column3 |Column4 |
|--------|--------|--------|--------|
|ABC XYZ |DFG KIL |YUI XYZ |ABC IOH |
|YDT NJK |ABC HJK |NJM XYZ |WEC OPP |
I was thinking to be able to build a query to search multiple words either in the same column or different columns to show those keywords and the respected columns. This, without leaning on cursors.
I.e. I would like the following output if the user search keywords 'ABC', 'YUI', 'OPP':
|Col1 |Col2 |Col3 |Col4 |Found Keyword|Keyword Found in |
|--------|--------|--------|--------|-------------|-----------------|
|ABC XYZ |DFG KIL |YUI XYZ |ABC IOH |ABC, YUI |Col1,Col3,Col4 |
|YDT NJK |ABC HJK |NJM XYZ |WEC OPP |ABC, OPP |Col2,Col4 |
I've been trying to do various things like (sys.columns, UNION ALL, CROSS JOIN), but so far to little avail.
There are more than 100K rows, what is the best way to do it?
Note: I'm using SQL Server 2012 version.
Argg! This is a horrible data format. You should not be storing multiple value in a column. But you can use giant case expressions. For instance, for the "keywords found in" result:
select t.*,
stuff( (case when col1 like '%' + #word1 + '%' or col1 like '%' + #word2 + '%' or col1 like '%' + #word3'
then ',col1' else ''
end) +
(case when col2 like '%' + #word1 + '%' or col2 like '%' + #word2 + '%' or col2 like '%' + #word3'
then ',col2' else ''
end) +
(case when col3 like '%' + #word1 + '%' or col3 like '%' + #word2 + '%' or col3 like '%' + #word3'
then ',col3' else ''
end), 1, 1, '') as keyword_found_in
from t;
You can apply similar logic to the keywords that are found. You can adjust the logic to take into account delimiters, if that is necessary.
It is already been commented that your design should be reworked.
Now, here is a query that should respond to your use case. This works by declaring the words that you search for in a subquery (which makes it quite easy to add more), and then JOIN with the table : basically, it is enough to check if all columns concatenated match with the searched value (assuming the space separator).
Then, the outer query does the aggregation job, using STRING_AGG() (available in SQL Server 2017). The part that generates the list of columns that matched is a bit tricky, since CSV values need to be generated manually for each record.
NB : it would be good if you had a unique column (or combination of column) to avoid mixing records when aggregating. I assumed a primary key column named id.
Query:
SELECT
t.Col1,
t.Col2,
t.Col3,
t.Col4,
STRING_AGG(words.txt, ',') FoundKeywords,
STRING_AGG(
STUFF(CONCAT(
IIF(CONCAT(' ', t.Col1, ' ') LIKE CONCAT(' %', words.txt, '% '), ',Col1', ''),
IIF(CONCAT(' ', t.Col2, ' ') LIKE CONCAT(' %', words.txt, '% '), ',Col2', ''),
IIF(CONCAT(' ', t.Col3, ' ') LIKE CONCAT(' %', words.txt, '% '), ',Col3', ''),
IIF(CONCAT(' ', t.Col4, ' ') LIKE CONCAT(' %', words.txt, '% '), ',Col4', '')
), 1, 1, ''),
','
) KeywordFoundin
FROM
mytable t
INNER JOIN (
SELECT 'ABC' txt UNION ALL SELECT 'YUI' UNION ALL SELECT 'OOP'
) words
ON CONCAT(' ', t.Col1, ' ', t.Col2, ' ', t.Col3, ' ', t.col4, ' ')
LIKE CONCAT(' %', words.txt, '% ')
GROUP BY
t.id,
t.Col1,
t.Col2,
t.Col3,
t.Col4
Demo on DB Fiddle:
Col1 | Col2 | Col3 | Col4 | FoundKeywords | KeywordFoundin
:------ | :------ | :------ | :------ | :------------ | :-------------
ABC XYZ | DFG KIL | YUI XYZ | ABC IOH | ABC,YUI | Col1,Col4,Col3
YDT NJK | ABC HJK | NJM XYZ | WEC OPP | ABC | Col2
In SQL Server < 2017, where STRING_AGG() is not available, I would use a simple SELECT with (very lenghty) expressions to compare and concatenate the required information:
SELECT
Col1,
Col2,
Col3,
Col4,
STUFF(CONCAT(
IIF(CONCAT(' ', Col1, ' ', Col2, ' ', Col3, ' ', Col4, ' ') LIKE CONCAT(' %', 'ABC', '% '), ',ABC', ''),
IIF(CONCAT(' ', Col1, ' ', Col2, ' ', Col3, ' ', Col4, ' ') LIKE CONCAT(' %', 'YUI', '% '), ',YUI', ''),
IIF(CONCAT(' ', Col1, ' ', Col2, ' ', Col3, ' ', Col4, ' ') LIKE CONCAT(' %', 'OOP', '% '), ',OOP', '')
), 1, 1, '') FoundKeywords,
STUFF(CONCAT(
IIF(
CONCAT(' ', Col1, ' ') LIKE CONCAT(' %', 'ABC', '% ')
OR CONCAT(' ', Col1, ' ') LIKE CONCAT(' %', 'YUI', '% ')
OR CONCAT(' ', Col1, ' ') LIKE CONCAT(' %', 'OOP', '% '),
',Col1',
''
),
IIF(
CONCAT(' ', Col2, ' ') LIKE CONCAT(' %', 'ABC', '% ')
OR CONCAT(' ', Col2, ' ') LIKE CONCAT(' %', 'YUI', '% ')
OR CONCAT(' ', Col2, ' ') LIKE CONCAT(' %', 'OOP', '% '),
',Col2',
''
),
IIF(
CONCAT(' ', Col3, ' ') LIKE CONCAT(' %', 'ABC', '% ')
OR CONCAT(' ', Col3, ' ') LIKE CONCAT(' %', 'YUI', '% ')
OR CONCAT(' ', Col3, ' ') LIKE CONCAT(' %', 'OOP', '% '),
',Col3',
''
),
IIF(
CONCAT(' ', Col4, ' ') LIKE CONCAT(' %', 'ABC', '% ')
OR CONCAT(' ', Col4, ' ') LIKE CONCAT(' %', 'YUI', '% ')
OR CONCAT(' ', Col4, ' ') LIKE CONCAT(' %', 'OOP', '% '),
',Col4',
''
)
), 1, 1, '') KeywordFoundin
FROM mytable
WHERE
CONCAT(' ', Col1, ' ', Col2, ' ', Col3, ' ', col4, ' ') LIKE CONCAT(' %', 'ABC', '% ')
OR CONCAT(' ', Col1, ' ', Col2, ' ', Col3, ' ', col4, ' ') LIKE CONCAT(' %', 'YUI', '% ')
OR CONCAT(' ', Col1, ' ', Col2, ' ', Col3, ' ', col4, ' ') LIKE CONCAT(' %', 'OOP', '% ')
Demo on DB Fiddle
I have been trying hard to optimize below query using indexes in place of where clause. can any one help on the same. Data volume is very high so I want to optimize it using indexing or anyother way?
Query goes like this:
SELECT *
FROM
(SELECT
a.*,
rownum as row_num
FROM
(SELECT DISTINCT
lot.lot_no AS lotNo, lot.sel as sel,
lot.C_ARRIVAL_NOTICE_NO AS arrNoticeNo,
lot.C_SHIPMENT_DIRECTION AS shipmentDirection,
lot.C_SENDERS_REFERENCE_NUM AS externalReference,
lot.booking_seq AS bookingNo,lot.shipper AS shipperCode,
nvl(lot.shipper_name ,lot.shipper_addr1) AS shipperName,
NVL2 (lot.commdesc, lot.commdesc || ' ', '') || NVL2 (lot.comdesc1,lot. comdesc1 || ' ', '') || NVL2 (lot.comdesc2, lot.comdesc2 || ' ', '') || NVL2 (lot.comdesc3, lot.comdesc3 || ' ', '') || NVL2 (lot.comdesc4, lot.comdesc4 || ' ', '') || NVL2 (lot.comdesc5, lot.comdesc5 || ' ', '') || NVL2 (lot.comdesc6,lot. comdesc6 || ' ', '') || NVL2 (lot.comdesc7,lot. comdesc7 || ' ', '') || NVL2 (lot.comdesc8, lot.comdesc8 || ' ', '') || NVL2 (lot.comdesc9, lot.comdesc9, '') AS description,
lot.lot_qty AS pieces, lot.lot_wght AS weight,
lot.lot_cube AS cube,
to_char(lot.input_date, 'dd-Mon-YYYY') AS lotDate,
lot.e_m AS e_m, lot.warehouse AS warehouse,
to_char(lot.in_date , 'dd-Mon-YYYY') AS inDate,
lot.in_time AS inTime, lot.hold AS hold,
to_char(lot.sail_date, 'dd-Mon-YYYY') AS sailDate,
to_char(lot.sail_date, 'dd-Mon-YYYY') AS etdDate,
lot.container AS container,
lot.oml_bl_no AS billOfLadingNumber,
lot.reference AS fileNumber,
lot.inland_carrier AS trucker,
lot.pro_number AS proNumber,
lot.comments AS exceptions, lot.vessel AS vessel,
lot.cstatus AS status, lot.voyage AS voyage,
(SELECT count(*) FROM tra_shipment_status
WHERE c_reference = lot.lot_no AND i_status_code = '857940') as transmitcount,
(SELECT c_finalcfs_code FROM imp_bl_top
WHERE inv_no = lot.C_Arrival_Notice_No) as cfsCode,
'STI_COUNTRY_US' AS schemaName
FROM
itemdata lot
WHERE
lot.in_date BETWEEN TO_DATE('27-FEB-2017 00:00', 'DD-MON-YYYY HH24:MI')
AND TO_DATE('29-MAR-2017 23:59', 'DD-MON-YYYY HH24:MI')
AND lot.sel IS NOT NULL
AND (lot.C_ARRIVAL_NOTICE_NO IS NOT NULL OR
lot.C_SHIPMENT_DIRECTION ='IO')
AND lot.I_STATUS > 0 AND Lot.Cstatus = 'D'
ORDER BY
lot.lot_no desc, lot.in_date desc) a
WHERE
rownum <= 60)
WHERE
row_num >= 1
Assume that the DBA_TAB_COLUMNS looks like this:
I'd like to write a SQL or PL/SQL script to generate following text:
select 'NULL' as A1, B1, QUERY, RECORD_KEY from SMHIST.probsummarym1
union all
select 'NULL' as A1, 'NULL' as B1, QUERY, RECORD_KEY from SMHIST_EIT200.probsummarym1
union all
select A1, 'NULL' as B1, QUERY, RECORD_KEY from SMHIST_EIT300.probsummarym1
the requirements are:
If the table under any of the SMHIST% schemas do not have that column, then insert a default NULL alias for that columns.
the column list is in alphabetical order.
so can anybody tell me how to write this script?
EDIT: Added better alias names and en explicit CROSS JOIN. Added XMLAGG version.
NB: LISTAGG exists from Oracle version 11.2 and onwards and returns VARCHAR2. If the output string is larger than 4000K or if on a prior version you can use XMLAGG which is a bit more cumbersome to work with (eg. http://psoug.org/definition/xmlagg.htm).
With LISTAGG (returning VARCHAR2):
SELECT LISTAGG (line,
CHR (13) || CHR (10) || 'union all' || CHR (13) || CHR (10))
WITHIN GROUP (ORDER BY sortorder)
script
FROM (SELECT line, ROWNUM sortorder
FROM ( SELECT 'select '
|| LISTAGG (
CASE
WHEN tc.column_name IS NULL
THEN
'''NULL'' as '
END
|| col_join.column_name,
', ')
WITHIN GROUP (ORDER BY col_join.column_name)
|| ' from '
|| col_join.owner
|| '.'
|| col_join.table_name
line
FROM dba_tab_columns tc
RIGHT OUTER JOIN
(SELECT DISTINCT
owner, table_name, col_list.column_name
FROM dba_tab_columns
CROSS JOIN
(SELECT DISTINCT column_name
FROM dba_tab_columns
WHERE owner LIKE 'SMHIST%') col_list
WHERE owner LIKE 'SMHIST%') col_join
ON tc.owner = col_join.owner
AND tc.table_name = col_join.table_name
AND tc.column_name = col_join.column_name
GROUP BY col_join.owner, col_join.table_name
ORDER BY col_join.owner, col_join.table_name))
With XMLAGG (returning CLOB by adding .getclobval (), note: RTRIM works here because table names cannot include ',' and ' ' (space)):
SELECT REPLACE (SUBSTR (script, 1, LENGTH (script) - 12),
'&' || 'apos;',
'''')
FROM (SELECT XMLAGG (
XMLELEMENT (
e,
line,
CHR (13)
|| CHR (10)
|| 'union all'
|| CHR (13)
|| CHR (10))).EXTRACT ('//text()').getclobval ()
script
FROM (SELECT line, ROWNUM sortorder
FROM ( SELECT 'select '
|| RTRIM (
REPLACE (
XMLAGG (XMLELEMENT (
e,
CASE
WHEN tc.column_name
IS NULL
THEN
'''NULL'' as '
END
|| col_join.column_name,
', ') ORDER BY
col_join.column_name).EXTRACT (
'//text()').getclobval (),
'&' || 'apos;',
''''),
', ')
|| ' from '
|| col_join.owner
|| '.'
|| col_join.table_name
line
FROM dba_tab_columns tc
RIGHT OUTER JOIN
(SELECT DISTINCT
owner,
table_name,
col_list.column_name
FROM dba_tab_columns
CROSS JOIN
(SELECT DISTINCT column_name
FROM dba_tab_columns
WHERE owner LIKE 'SMHIST%') col_list
WHERE owner LIKE 'SMHIST%') col_join
ON tc.owner = col_join.owner
AND tc.table_name = col_join.table_name
AND tc.column_name = col_join.column_name
GROUP BY col_join.owner, col_join.table_name
ORDER BY col_join.owner, col_join.table_name)))