I'm new to a company that makes heavy use of stored procedures (500+). To help learn the system, I was hoping there was an easy way to build a tree type list that shows all stored procedures in the system and which stored procedures they themselves call...thus creating a map of the stored procedures that could be executed. Is there an easy way to do this via a query in SQL Server? Is there a tool/utility that can do this?
For example, I want to see the following type of list without having to painstakingly try and follow the logic in each procedure and manually make a list.
build_house
-->pour_foundation
-->order_cement_truck
-->frame_house
-->hire_workers
-->buy_nails_and_hammers
-->wire_house
-->hire_electricians
-->check_certifications
-->test_wiring
The only thing I've found searching so far is:
http://www.codeproject.com/Articles/10019/Find-Stored-Procedures-called-within-a-procedure
To be clear, I'm looking to pass in / select a stored procedure name and have returned to me all of the stored procedures that it calls/uses.
#JackLock, I downloaded and installed SQL Search, but I don't see how this solves my problem. This tool aids in searching for stored procedures by name, or searching for text in stored procedures, but how does it help me automatically list out all stored procedures that are called from within a particular stored procedure? Maybe I'm missing something? For example, in my example above, I want the ability to run a system query or tool that returns me a list of the stored procedures that are called by whatever stored procedure name I pass it. So in the example, if I give the query or tool "build_house" it returns me the results in the example.
EDIT/UPDATE:
OK, I'd like to try and solve this with a query but need some help. I "think" what I want to do is query the sys.procedures to get the name of all the stored procedures in my system. Once I have them, I want to then pass them into the following query to determine how many stored procedures get called from it:
SELECT referenced_entity_name
FROM sys.dm_sql_referenced_entities (#ProcName, 'OBJECT')
Where #ProcName would get passed in for each row returned by the call to sys.procedures.
What is the most efficient way to do this in t-sql (2008)?
Thanks in advance,
Michael
You can Enter the particular Procedure name in the below code and check, you will get the particular procedure used in others or not
SELECT OBJECT_NAME(id)
FROM syscomments
WHERE [text] LIKE '% procedure_or_function_name %'
GROUP BY OBJECT_NAME(id)
You have not mentioned which version of SQL Server you are working on. But there is a free utility (actually SSMS addin) by RedGate called SQL Search.
I have it working on SSMS 2005,2008,R2 and 2012
It should solve your problem.
I know it's being along time since the question, however I think I found a workaround for this purpose and I want to share it, I've used an Aaron's Bertrand function to find a pattern within a text and this way of sorting,
Function :
CREATE FUNCTION dbo.FindPatternLocation
(
#string NVARCHAR(MAX),
#term NVARCHAR(255)
)
RETURNS TABLE
AS
RETURN
(
SELECT pos = Number - LEN(#term)
FROM
(
SELECT Number, Item = LTRIM(RTRIM(SUBSTRING(#string, Number, CHARINDEX(#term, #string + #term, Number) - Number)))
FROM (
SELECT ROW_NUMBER() OVER (ORDER BY [object_id])
FROM sys.all_columns) AS n(Number)
WHERE Number > 1 AND Number <= CONVERT(INT, LEN(#string)+1)
AND SUBSTRING(#term + #string, Number, LEN(#term)) = #term
)
AS y
);
Final query:
declare #object_name varchar(1000) = 'stored_procedure_name'
;
with cte as (
SELECT o.name AS parent_object_name
, SUBSTRING( m.definition, rs_fn.pos+4, CHARINDEX( ' ' , stuff( m.definition, 1, rs_fn.pos + 4 , '' )) ) as child_object
, cast(row_number()over(partition by o.object_id order by o.name) as varchar(max)) as [path]
, 0 as level
, row_number()over(partition by o.object_id order by o.name) / power(10.0,0) as x
FROM sys.sql_modules m
INNER JOIN sys.objects o ON m.object_id = o.object_id
cross apply (
select *
from dbo.FindPatternLocation( m.definition, 'EXEC ' ) as res
) as rs_fn
WHERE 1=1
and o.name like #object_name
union all
SELECT o.name AS parent_object_name
, SUBSTRING( m.definition, rs_fn.pos+4, CHARINDEX( ' ' , stuff( m.definition, 1, rs_fn.pos + 4 , '' )) ) as child_object
, [path] +'-'+ cast(row_number()over(partition by o.object_id order by o.name) as varchar(max))
, level+1
, c.x + row_number()over(partition by o.object_id order by o.name) / power(10.0,level+1)
FROM sys.sql_modules m
INNER JOIN sys.objects o ON m.object_id = o.object_id
inner join cte c on c.child_object = o.name
cross apply (
select *
from dbo.FindPatternLocation( m.definition, 'EXEC ' ) as res
) as rs_fn
WHERE 1=1
-- and o.name like #object_name
)
select * from cte
order by x
;
Feel free to use it as you wish.
Related
I've made some research on the matter but don't have solution yet. What I want to get is column-level dependencies in a view. So, let's say we have a table like this
create table TEST(
first_name varchar(10),
last_name varchar(10),
street varchar(10),
number int
)
and a view like this:
create view vTEST
as
select
first_name + ' ' + last_name as [name],
street + ' ' + cast(number as varchar(max)) as [address]
from dbo.TEST
What I'd like is to get result like this:
column_name depends_on_column_name depends_on_table_name
----------- --------------------- --------------------
name first_name dbo.TEST
name last_name dbo.TEST
address street dbo.TEST
address number dbo.TEST
I've tried sys.dm_sql_referenced_entities function, but referencing_minor_id is always 0 there for views.
select
referencing_minor_id,
referenced_schema_name + '.' + referenced_entity_name as depends_on_table_name,
referenced_minor_name as depends_on_column_name
from sys.dm_sql_referenced_entities('dbo.vTEST', 'OBJECT')
referencing_minor_id depends_on_table_name depends_on_column_name
-------------------- --------------------- ----------------------
0 dbo.TEST NULL
0 dbo.TEST first_name
0 dbo.TEST last_name
0 dbo.TEST street
0 dbo.TEST number
The same is true for sys.sql_expression_dependencies and for obsolete sys.sql_dependencies.
So do I miss something or is it impossible to do?
There're some related questions (Find the real column name of an alias used in a view?), but as I said - I haven't found a working solution yet.
EDIT 1: I've tried to use DAC to query if this information is stored somewhere in System Base Tables but haven't find it
This solution could answer your question only partially. It won't work for columns that are expressions.
You could use sys.dm_exec_describe_first_result_set to get column information:
#include_browse_information
If set to 1, each query is analyzed as if it has a FOR BROWSE option on the query. Additional key columns and source table information are returned.
CREATE TABLE txu(id INT, first_name VARCHAR(10), last_name VARCHAR(10));
CREATE TABLE txd(id INT, id_fk INT, address VARCHAR(100));
CREATE VIEW v_txu
AS
SELECT t.id AS PK_id,
t.first_name AS name,
d.address,
t.first_name + t.last_name AS name_full
FROM txu t
JOIN txd d
ON t.id = d.id_fk
Main query:
SELECT name, source_database, source_schema,
source_table, source_column
FROM sys.dm_exec_describe_first_result_set(N'SELECT * FROM v_txu', null, 1) ;
Output:
+-----------+--------------------+---------------+--------------+---------------+
| name | source_database | source_schema | source_table | source_column |
+-----------+--------------------+---------------+--------------+---------------+
| PK_id | fiddle_0f9d47226c4 | dbo | txu | id |
| name | fiddle_0f9d47226c4 | dbo | txu | first_name |
| address | fiddle_0f9d47226c4 | dbo | txd | address |
| name_full | null | null | null | null |
+-----------+--------------------+---------------+--------------+---------------+
DBFiddleDemo
It is a solution based on query plan. It has some adventages
almost any select queries can be processed
no SchemaBinding
and disadventages
has not been tested properly
can become broken suddenly if Microsoft change XML query plan.
The core idea is that every column expression inside XML query plan is defined in "DefinedValue" node. First subnode of "DefinedValue" is a reference to output column and second one is a expression. The expression computes from input columns and constant values.
As mentioned above It's based only on empirical observation and needs to be tested properly.
It's a invocation example:
exec dbo.GetColumnDependencies 'select * from dbo.vTEST'
target_column_name | source_column_name | const_value
---------------------------------------------------
address | Expr1007 | NULL
name | Expr1006 | NULL
Expr1006 | NULL | ' '
Expr1006 | [testdb].[dbo].first_name | NULL
Expr1006 | [testdb].[dbo].last_name | NULL
Expr1007 | NULL | ' '
Expr1007 | [testdb].[dbo].number | NULL
Expr1007 | [testdb].[dbo].street | NULL
It's code.
First of all get XML query plan.
declare #select_query as varchar(4000) = 'select * from dbo.vTEST' -- IT'S YOUR QUERY HERE.
declare #select_into_query as varchar(4000) = 'select top (1) * into #foo from (' + #select_query + ') as src'
, #xml_plan as xml = null
, #xml_generation_tries as tinyint = 10
;
while (#xml_plan is null and #xml_generation_tries > 0) -- There is no guaranty that plan will be cached.
begin
execute (#select_into_query);
select #xml_plan = pln.query_plan
from sys.dm_exec_query_stats as qry
cross apply sys.dm_exec_sql_text(qry.sql_handle) as txt
cross apply sys.dm_exec_query_plan(qry.plan_handle) as pln
where txt.text = #select_into_query
;
end
if (#xml_plan is null
) begin
raiserror(N'Can''t extract XML query plan from cache.' ,15 ,0);
return;
end
;
Next is a main query. It's biggest part is recursive common table expression for column extraction.
with xmlnamespaces(default 'http://schemas.microsoft.com/sqlserver/2004/07/showplan'
,'http://schemas.microsoft.com/sqlserver/2004/07/showplan' as shp -- Used in .query() for predictive namespace using.
)
, cte_column_dependencies as
(
The seed of recursion is a query that extracts columns for #foo table that store 1 row of interested select query.
select
(select foo_col.info.query('./ColumnReference') for xml raw('shp:root') ,type) -- Becouse .value() can't extract attribute from root node.
as target_column_info
, (select foo_col.info.query('./ScalarOperator/Identifier/ColumnReference') for xml raw('shp:root') ,type)
as source_column_info
, cast(null as xml) as const_info
, 1 as iteration_no
from #xml_plan.nodes('//Update/SetPredicate/ScalarOperator/ScalarExpressionList/ScalarOperator/MultipleAssign/Assign')
as foo_col(info)
where foo_col.info.exist('./ColumnReference[#Table="[#foo]"]') = 1
The recursive part searches for "DefinedValue" node with depended column and extract all "ColumnReference" and "Const" subnodes that used in column expression. It's over complicated by XML to SQL conversions.
union all
select
(select internal_col.info.query('.') for xml raw('shp:root') ,type)
, source_info.column_info
, source_info.const_info
, prev_dependencies.iteration_no + 1
from #xml_plan.nodes('//DefinedValue/ColumnReference') as internal_col(info)
inner join cte_column_dependencies as prev_dependencies -- Filters by depended columns.
on prev_dependencies.source_column_info.value('(//ColumnReference/#Column)[1]' ,'nvarchar(4000)') = internal_col.info.value('(./#Column)[1]' ,'nvarchar(4000)')
and exists (select prev_dependencies.source_column_info.value('(.//#Schema)[1]' ,'nvarchar(4000)') intersect select internal_col.info.value('(./#Schema)[1]' ,'nvarchar(4000)'))
and exists (select prev_dependencies.source_column_info.value('(.//#Database)[1]' ,'nvarchar(4000)') intersect select internal_col.info.value('(./#Database)[1]' ,'nvarchar(4000)'))
and exists (select prev_dependencies.source_column_info.value('(.//#Server)[1]' ,'nvarchar(4000)') intersect select internal_col.info.value('(./#Server)[1]' ,'nvarchar(4000)'))
cross apply ( -- Becouse only column or only constant can be places in result row.
select (select source_col.info.query('.') for xml raw('shp:root') ,type) as column_info
, null as const_info
from internal_col.info.nodes('..//ColumnReference') as source_col(info)
union all
select null as column_info
, (select const.info.query('.') for xml raw('shp:root') ,type) as const_info
from internal_col.info.nodes('..//Const') as const(info)
) as source_info
where source_info.column_info is null
or (
-- Except same node selected by '..//ColumnReference' from its sources. Sorry, I'm not so well to check it with XQuery simple.
source_info.column_info.value('(//#Column)[1]' ,'nvarchar(4000)') <> internal_col.info.value('(./#Column)[1]' ,'nvarchar(4000)')
and (select source_info.column_info.value('(//#Schema)[1]' ,'nvarchar(4000)') intersect select internal_col.info.value('(./#Schema)[1]' ,'nvarchar(4000)')) is null
and (select source_info.column_info.value('(//#Database)[1]' ,'nvarchar(4000)') intersect select internal_col.info.value('(./#Database)[1]' ,'nvarchar(4000)')) is null
and (select source_info.column_info.value('(//#Server)[1]' ,'nvarchar(4000)') intersect select internal_col.info.value('(./#Server)[1]' ,'nvarchar(4000)')) is null
)
)
Finally, It's select statement that convert XML to appropriate human text.
select
-- col_dep.target_column_info
--, col_dep.source_column_info
--, col_dep.const_info
coalesce(col_dep.target_column_info.value('(.//shp:ColumnReference/#Server)[1]' ,'nvarchar(4000)') + '.' ,'')
+ coalesce(col_dep.target_column_info.value('(.//shp:ColumnReference/#Database)[1]' ,'nvarchar(4000)') + '.' ,'')
+ coalesce(col_dep.target_column_info.value('(.//shp:ColumnReference/#Schema)[1]' ,'nvarchar(4000)') + '.' ,'')
+ col_dep.target_column_info.value('(.//shp:ColumnReference/#Column)[1]' ,'nvarchar(4000)')
as target_column_name
, coalesce(col_dep.source_column_info.value('(.//shp:ColumnReference/#Server)[1]' ,'nvarchar(4000)') + '.' ,'')
+ coalesce(col_dep.source_column_info.value('(.//shp:ColumnReference/#Database)[1]' ,'nvarchar(4000)') + '.' ,'')
+ coalesce(col_dep.source_column_info.value('(.//shp:ColumnReference/#Schema)[1]' ,'nvarchar(4000)') + '.' ,'')
+ col_dep.source_column_info.value('(.//shp:ColumnReference/#Column)[1]' ,'nvarchar(4000)')
as source_column_name
, col_dep.const_info.value('(/shp:root/shp:Const/#ConstValue)[1]' ,'nvarchar(4000)')
as const_value
from cte_column_dependencies as col_dep
order by col_dep.iteration_no ,target_column_name ,source_column_name
option (maxrecursion 512) -- It's an assurance from infinite loop.
All what you need is mentioned into definition of view.
so we can extract this information via following the next steps:-
Assign the view definition into a string variable.
Split it with (,) comma.
Split the alias with (+) plus operator via using CROSS APPLY with XML.
use the system tables for getting the accurate information like original table.
Demo:-
Create PROC psp_GetLevelDependsView (#sViewName varchar(200))
AS
BEGIN
Declare #stringToSplit nvarchar(1000),
#name NVARCHAR(255),
#dependsTableName NVARCHAR(50),
#pos INT
Declare #returnList TABLE ([Name] [nvarchar] (500))
SELECT TOP 1 #dependsTableName= table_schema + '.'+ TABLE_NAME
FROM INFORMATION_SCHEMA.VIEW_COLUMN_USAGE
select #stringToSplit = definition
from sys.objects o
join sys.sql_modules m on m.object_id = o.object_id
where o.object_id = object_id( #sViewName)
and o.type = 'V'
WHILE CHARINDEX(',', #stringToSplit) > 0
BEGIN
SELECT #pos = CHARINDEX(',', #stringToSplit)
SELECT #name = SUBSTRING(#stringToSplit, 1, #pos-1)
INSERT INTO #returnList
SELECT #name
SELECT #stringToSplit = SUBSTRING(#stringToSplit, #pos+1, LEN(#stringToSplit)-#pos)
END
INSERT INTO #returnList
SELECT #stringToSplit
select COLUMN_NAME , b.Name as Expression
Into #Temp
FROM INFORMATION_SCHEMA.COLUMNS a , #returnList b
WHERE TABLE_NAME= #sViewName
And (b.Name) like '%' + ( COLUMN_NAME) + '%'
SELECT A.COLUMN_NAME as column_name,
Split.a.value('.', 'VARCHAR(100)') AS depends_on_column_name , #dependsTableName as depends_on_table_name
Into #temp2
FROM
(
SELECT COLUMN_NAME,
CAST ('<M>' + REPLACE(Expression, '+', '</M><M>') + '</M>' AS XML) AS Data
FROM #Temp
) AS A CROSS APPLY Data.nodes ('/M') AS Split(a);
SELECT b.column_name , a.COLUMN_NAME as depends_on_column_name , b.depends_on_table_name
FROM INFORMATION_SCHEMA.VIEW_COLUMN_USAGE a , #temp2 b
WHERE VIEW_NAME= #sViewName
and b.depends_on_column_name like '%' + a.COLUMN_NAME + '%'
drop table #Temp
drop table #Temp2
END
Test:-
exec psp_GetLevelDependsView 'vTest'
Result:-
column_name depends_on_column_name depends_on_table_name
----------- --------------------- --------------------
name first_name dbo.TEST
name last_name dbo.TEST
address street dbo.TEST
address number dbo.TEST
I was playing around with this but didn't have time to go any further. Maybe this will help:
-- Returns all table columns called in the view and the objects they pull from
SELECT
v.[name] AS ViewName
,d.[referencing_id] AS ViewObjectID
,c.[name] AS ColumnNames
,OBJECT_NAME(d.referenced_id) AS ReferencedTableName
,d.referenced_id AS TableObjectIDsReferenced
FROM
sys.views v
INNER JOIN sys.sql_expression_dependencies d ON d.referencing_id = v.[object_id]
INNER JOIN sys.objects o ON d.referencing_id = o.[object_id]
INNER JOIN sys.columns c ON d.referenced_id = c.[object_id]
WHERE v.[name] = 'vTEST'
-- Returns all output columns in the view
SELECT
OBJECT_NAME([object_id]) AS ViewName
,[object_id] AS ViewObjectID
,[name] AS OutputColumnName
FROM sys.columns
WHERE OBJECT_ID('vTEST') = [object_id]
-- Get the view definition
SELECT
VIEW_DEFINITION
FROM INFORMATION_SCHEMA.VIEWS
WHERE TABLE_NAME = 'vTEST'
Unfortunately, SQL Server does not explicitly store mapping between source table columns and view columns. I suspect the main reason is simply due to the potential complexity of views (expression columns, functions called on those columns, nested queries etc.).
The only way that I can think of to determine the mapping between view columns and source columns would be to either parse the query associated to the view or parse the execution plan of the view.
The approach I have outlined here focuses on the second option and relies on the fact that SQL Server will avoid generating output lists for columns not required by a query.
The first step is to get the list of dependent tables and their associated columns required for the view. This can be achieved via the standard system tables in SQL Server.
Next, we enumerate all of the view’s columns via a cursor.
For each view column, we create a temporary wrapper stored procedure that only selects the single column in question from view. Because only a single column is requested SQL Server will only retrieve the information needed to output that single view column.
The newly created procedure will run the query in format only mode and will therefore not cause any actual I/O operations on the database, but it will generate an estimated execution plan when executed. After the query plan is generate, we query the output lists from the execution plan. Since we know which view column was selected we can now associate the output list to view column in question. We can further refine the association by only associating columns that form part of our original dependency list, this will eliminate expression outputs from the result set.
Note that with this method if the view needs to join different tables together to generate the output then all columns required to generate the output will be returned even if it is not directly used in the column expression since it is still in directly required.
The following stored procedure demonstrates the above implementation method:
CREATE PROCEDURE ViewGetColumnDependencies
(
#viewName NVARCHAR(50)
)
AS
BEGIN
CREATE TABLE #_suppress_output
(
result NVARCHAR(500) NULL
);
DECLARE #viewTableColumnMapping TABLE
(
[ViewName] NVARCHAR(50),
[SourceObject] NVARCHAR(50),
[SourceObjectColumnName] NVARCHAR(50),
[ViewAliasColumn] NVARCHAR(50)
)
-- Get list of dependent tables and their associated columns required for the view.
INSERT INTO #viewTableColumnMapping
(
[ViewName]
,[SourceObject]
,[SourceObjectColumnName]
)
SELECT v.[name] AS [ViewName]
,'[' + OBJECT_NAME(d.referenced_major_id) + ']' AS [SourceObject]
,c.[name] AS [SourceObjectColumnName]
FROM sys.views v
LEFT OUTER JOIN sys.sql_dependencies d ON d.object_id = v.object_id
LEFT OUTER JOIN sys.columns c ON c.object_id = d.referenced_major_id AND c.column_id = d.referenced_minor_id
WHERE v.[name] = #viewName;
DECLARE #aliasColumn NVARCHAR(50);
-- Next, we enumerate all of the views columns via a cursor.
DECLARE ViewColumnNameCursor CURSOR FOR
SELECT aliases.name AS [AliasName]
FROM sys.views v
LEFT OUTER JOIN sys.columns AS aliases on v.object_id = aliases.object_id -- c.column_id=aliases.column_id AND aliases.object_id = object_id('vTEST')
WHERE v.name = #viewName;
OPEN ViewColumnNameCursor
FETCH NEXT FROM ViewColumnNameCursor
INTO #aliasColumn
DECLARE #tql_create_proc NVARCHAR(MAX);
DECLARE #queryPlan XML;
WHILE ##FETCH_STATUS = 0
BEGIN
/*
For each view column, we create a temporary wrapper stored procedure that
only selects the single column in question from view. The stored procedure
will run the query in format only mode and will therefore not cause any
actual I/O operations on the database, but it will generate an estimated
execution plan when executed.
*/
SET #tql_create_proc = 'CREATE PROCEDURE ___WrapView
AS
SET FMTONLY ON;
SELECT CONVERT(NVARCHAR(MAX), [' + #aliasColumn + ']) FROM [' + #viewName + '];
SET FMTONLY OFF;';
EXEC (#tql_create_proc);
-- Execute the procedure to generate a query plan. The insert into the temp table is only done to
-- suppress the empty result set from being displayed as part of the output.
INSERT INTO #_suppress_output
EXEC ___WrapView;
-- Get the query plan for the wrapper procedure that was just executed.
SELECT #queryPlan = [qp].[query_plan]
FROM [sys].[dm_exec_procedure_stats] AS [ps]
JOIN [sys].[dm_exec_query_stats] AS [qs] ON [ps].[plan_handle] = [qs].[plan_handle]
CROSS APPLY [sys].[dm_exec_query_plan]([qs].[plan_handle]) AS [qp]
WHERE [ps].[database_id] = DB_ID() AND OBJECT_NAME([ps].[object_id], [ps].[database_id]) = '___WrapView'
-- Drop the wrapper view
DROP PROCEDURE ___WrapView
/*
After the query plan is generate, we query the output lists from the execution plan.
Since we know which view column was selected we can now associate the output list to
view column in question. We can further refine the association by only associating
columns that form part of our original dependency list, this will eliminate expression
outputs from the result set.
*/
;WITH QueryPlanOutputList AS
(
SELECT T.X.value('local-name(.)', 'NVARCHAR(max)') as Structure,
T.X.value('./#Table[1]', 'NVARCHAR(50)') as [SourceTable],
T.X.value('./#Column[1]', 'NVARCHAR(50)') as [SourceColumnName],
T.X.query('*') as SubNodes
FROM #queryPlan.nodes('*') as T(X)
UNION ALL
SELECT QueryPlanOutputList.structure + N'/' + T.X.value('local-name(.)', 'nvarchar(max)'),
T.X.value('./#Table[1]', 'NVARCHAR(50)') as [SourceTable],
T.X.value('./#Column[1]', 'NVARCHAR(50)') as [SourceColumnName],
T.X.query('*')
FROM QueryPlanOutputList
CROSS APPLY QueryPlanOutputList.SubNodes.nodes('*') as T(X)
)
UPDATE #viewTableColumnMapping
SET ViewAliasColumn = #aliasColumn
FROM #viewTableColumnMapping CM
INNER JOIN
(
SELECT DISTINCT QueryPlanOutputList.Structure
,QueryPlanOutputList.[SourceTable]
,QueryPlanOutputList.[SourceColumnName]
FROM QueryPlanOutputList
WHERE QueryPlanOutputList.Structure like '%/OutputList/ColumnReference'
) SourceColumns ON CM.[SourceObject] = SourceColumns.[SourceTable] AND CM.SourceObjectColumnName = SourceColumns.SourceColumnName
FETCH NEXT FROM ViewColumnNameCursor
INTO #aliasColumn
END
CLOSE ViewColumnNameCursor;
DEALLOCATE ViewColumnNameCursor;
DROP TABLE #_suppress_output
SELECT *
FROM #viewTableColumnMapping
ORDER BY [ViewAliasColumn]
END
The stored procedure can now be executed as follow:
EXEC dbo.ViewGetColumnDependencies #viewName = 'vTEST'
I'm struggling to create a query that will return all information from a table (like SELECT *), but I would like to omit the column(s) that are auto-incremental.
Reason is, I'm displaying all data (using SELECT *, because I don't always know what columns are available) in a grid-view control, then I open the table up to allow for updates to be carried out. However this also opens up the column(s) that are assigned as auto-incremental for edit and prevents the update query from working.
So far I found the 'sys.columns.is_identity' table which seems it would help in some fashion, I'm just not sure how I could use this with a dynamic SELECT.
It should be noted that the columns are not always known, hence I use SELECT * to retrieve the initial required data.
As you mentioned, only way to do this is using sys.columns and dynamic query
DECLARE #col_list VARCHAR(8000)
SET #col_list = (SELECT ',' + Quotename(c.NAME)
FROM sys.columns c
JOIN sys.objects o
ON c.object_id = o.object_id
WHERE o.NAME = 'table_name'
AND is_identity <> 1
ORDER BY column_id
FOR xml path(''))
SET #col_list = Stuff(#col_list, 1, 1, '')
EXEC('select '+#col_list +' from yourtable')
I'm looking for a schema-independent query. That is, if I have a users table or a purchases table, the query should be equally capable of catching duplicate rows in either table without any modification (other than the from clause, of course).
I'm using T-SQL, but I'm guessing there should be a general solution.
I believe that this should work for you. Keep in mind that CHECKSUM() isn't 100% perfect - it's theoretically possible to get a false positive here (I think), but otherwise you can just change the table name and this should work:
;WITH cte AS (
SELECT
*,
CHECKSUM(*) AS chksum,
ROW_NUMBER() OVER(ORDER BY GETDATE()) AS row_num
FROM
My_Table
)
SELECT
*
FROM
CTE T1
INNER JOIN CTE T2 ON
T2.chksum = T1.chksum AND
T2.row_num <> T1.row_num
The ROW_NUMBER() is needed so that you have some way of distinguishing rows. It requires an ORDER BY and that can't be a constant, so GETDATE() was my workaround for that.
Simply change the table name in the CTE and it should work without spelling out the columns.
I'm still confused about what "detecting them might be" but I'll give it a shot.
Excluding them is easy
e.g.
SELECT DISTINCT * FROM USERS
However if you wanted to only include them and a duplicate is all the fields than you have to do
SELECT
[Each and every field]
FROM
USERS
GROUP BY
[Each and every field]
HAVING COUNT(*) > 1
You can't get away with just using (*) because you can't GROUP BY *
so this requirement from your comments is difficult
a schema-independent means I don't want to specify all of the columns
in the query
Unless that is you want to use dynamic SQL and read the columns from sys.columns or information_schema.columns
For example
DECLARE #colunns nvarchar(max)
SET #colunns = ''
SELECT #colunns = #colunns + '[' + COLUMN_NAME +'], '
FROM INFORMATION_SCHEMA.columns
WHERE table_name = 'USERS'
SET #colunns = left(#colunns,len(#colunns ) - 1)
DECLARE #SQL nvarchar(max)
SET #SQL = 'SELECT ' + #colunns
+ 'FROM USERS' + 'GROUP BY '
+ #colunns
+ ' Having Count(*) > 1'
exec sp_executesql #SQL
Please note you should read this The Curse and Blessings of Dynamic SQL if you haven't already
I have done this using CTEs in SQL Server.
Here is a sample on how to delete dupes but you should be able to adapt it easily to find dupes:
WITH CTE (COl1, Col2, DuplicateCount)
AS
(
SELECT COl1,Col2,
ROW_NUMBER() OVER(PARTITION BY COl1,Col2 ORDER BY Col1) AS DuplicateCount
FROM DuplicateRcordTable
)
DELETE
FROM CTE
WHERE DuplicateCount > 1
GO
Here is a link to an article where I got the SQL:
http://blog.sqlauthority.com/2009/06/23/sql-server-2005-2008-delete-duplicate-rows/
I recently was looking into the same issue and noticed this question.
I managed to solve it using a stored procedure with some dynamic SQL. This way you only need to specify the table name. And it will get all the other relevant data from sys tables.
/*
This SP returns all duplicate rows (1 line for each duplicate) for any given table.
to use the SP:
exec [database].[dbo].[sp_duplicates]
#table = '[database].[schema].[table]'
*/
create proc dbo.sp_duplicates #table nvarchar(50) as
declare #query nvarchar(max)
declare #groupby nvarchar(max)
set #groupby = stuff((select ',' + [name]
FROM sys.columns
WHERE object_id = OBJECT_ID(#table)
FOR xml path('')), 1, 1, '')
set #query = 'select *, count(*)
from '+#table+'
group by '+#groupby+'
having count(*) > 1'
exec (#query)
In Visual Studio 2k8, I press Ctrl-F and then type some text I want to search for in my project. I can quickly search many files at once.
I have a SQL Server with some stored procedures. How can I easily search my stored procedures for arbitrary strings? I know that I can list the text of one stored procedure in SQL Server Studio and do a Ctrl-F search from there, but I want to search them all at once.
SQL Search (from Redgate) is your friend. Free download, integrates straight into SSMS :)
This searches all views, procedures and user defined functions:
CREATE PROCEDURE usp_SearchFor
#PatternIn AS varchar(max)
,#ObjectNamePattern AS varchar(max) = NULL
AS
BEGIN
DECLARE #Pattern AS varchar(max)
SET #Pattern = '%' + #PatternIn + '%' ;
WITH ROUTINES
AS (
-- CANNOT use INFORMATION_SCHEMA.ROUTINES because of 4000 character limit
SELECT o.type_desc AS ROUTINE_TYPE
,s.[name] AS [SCHEMA_NAME]
,o.[name] AS ROUTINE_NAME
,m.definition AS ROUTINE_DEFINITION
FROM sys.sql_modules AS m WITH (NOLOCK)
INNER JOIN sys.objects AS o WITH (NOLOCK)
ON m.[object_id] = o.[OBJECT_ID]
INNER JOIN sys.schemas AS s WITH (NOLOCK)
ON s.[schema_id] = o.[schema_id]
WHERE m.definition LIKE #Pattern
),
Results
AS (
SELECT ROUTINE_TYPE
,[SCHEMA_NAME]
,ROUTINE_NAME
,PATINDEX(#Pattern, ROUTINE_DEFINITION) AS StartPos
,SUBSTRING(ROUTINE_DEFINITION,
PATINDEX(#Pattern, ROUTINE_DEFINITION),
255) AS WorkItem
,RIGHT(ROUTINE_DEFINITION,
LEN(ROUTINE_DEFINITION) - PATINDEX(#Pattern, ROUTINE_DEFINITION)) AS Remainder
,1 AS Occurrence
FROM ROUTINES
UNION ALL
SELECT ROUTINE_TYPE
,[SCHEMA_NAME]
,ROUTINE_NAME
,PATINDEX(#Pattern, Remainder) AS StartPos
,SUBSTRING(Remainder, PATINDEX(#Pattern, Remainder),
255) AS WorkItem
,RIGHT(Remainder,
LEN(Remainder) - PATINDEX(#Pattern,
Remainder)) AS Remainder
,Occurrence + 1 AS Occurrence
FROM Results
WHERE PATINDEX(#Pattern, Remainder) > 0
)
SELECT 'Search For:' + #PatternIn AS Problem
,ROUTINE_TYPE
,QUOTENAME([SCHEMA_NAME]) + '.' + QUOTENAME([ROUTINE_NAME]) AS ROUTINE_NAME
,WorkItem
FROM Results
WHERE #ObjectNamePattern IS NULL
OR ROUTINE_NAME LIKE #ObjectNamePattern
ORDER BY ROUTINE_TYPE
,[SCHEMA_NAME]
,ROUTINE_NAME
,Occurrence
END
Stored proc text is stored in sysobjects,syscomments.
Example
Just script all of the procs to a query window. Then ctrl-f to your hearts content.
SELECT DISTINCT OBJECT_NAME(ID) FROM SysComments WHERE Text LIKE '%Search%'
Deprecated apparently, but still works in SQL 2008
You can use a simple search as below
select ROUTINE_name from INFORMATION_SCHEMA.ROUTINES
where CHARINDEX('demo',routine_definition,1)>0
substitute your arbitrary string for 'demo'
This question already has answers here:
How to concatenate text from multiple rows into a single text string in SQL Server
(47 answers)
Closed 5 years ago.
To illustrate, assume that I have two tables as follows:
VehicleID Name
1 Chuck
2 Larry
LocationID VehicleID City
1 1 New York
2 1 Seattle
3 1 Vancouver
4 2 Los Angeles
5 2 Houston
I want to write a query to return the following results:
VehicleID Name Locations
1 Chuck New York, Seattle, Vancouver
2 Larry Los Angeles, Houston
I know that this can be done using server side cursors, ie:
DECLARE #VehicleID int
DECLARE #VehicleName varchar(100)
DECLARE #LocationCity varchar(100)
DECLARE #Locations varchar(4000)
DECLARE #Results TABLE
(
VehicleID int
Name varchar(100)
Locations varchar(4000)
)
DECLARE VehiclesCursor CURSOR FOR
SELECT
[VehicleID]
, [Name]
FROM [Vehicles]
OPEN VehiclesCursor
FETCH NEXT FROM VehiclesCursor INTO
#VehicleID
, #VehicleName
WHILE ##FETCH_STATUS = 0
BEGIN
SET #Locations = ''
DECLARE LocationsCursor CURSOR FOR
SELECT
[City]
FROM [Locations]
WHERE [VehicleID] = #VehicleID
OPEN LocationsCursor
FETCH NEXT FROM LocationsCursor INTO
#LocationCity
WHILE ##FETCH_STATUS = 0
BEGIN
SET #Locations = #Locations + #LocationCity
FETCH NEXT FROM LocationsCursor INTO
#LocationCity
END
CLOSE LocationsCursor
DEALLOCATE LocationsCursor
INSERT INTO #Results (VehicleID, Name, Locations) SELECT #VehicleID, #Name, #Locations
END
CLOSE VehiclesCursor
DEALLOCATE VehiclesCursor
SELECT * FROM #Results
However, as you can see, this requires a great deal of code. What I would like is a generic function that would allow me to do something like this:
SELECT VehicleID
, Name
, JOIN(SELECT City FROM Locations WHERE VehicleID = Vehicles.VehicleID, ', ') AS Locations
FROM Vehicles
Is this possible? Or something similar?
If you're using SQL Server 2005, you could use the FOR XML PATH command.
SELECT [VehicleID]
, [Name]
, (STUFF((SELECT CAST(', ' + [City] AS VARCHAR(MAX))
FROM [Location]
WHERE (VehicleID = Vehicle.VehicleID)
FOR XML PATH ('')), 1, 2, '')) AS Locations
FROM [Vehicle]
It's a lot easier than using a cursor, and seems to work fairly well.
Update
For anyone still using this method with newer versions of SQL Server, there is another way of doing it which is a bit easier and more performant using the
STRING_AGG method that has been available since SQL Server 2017.
SELECT [VehicleID]
,[Name]
,(SELECT STRING_AGG([City], ', ')
FROM [Location]
WHERE VehicleID = V.VehicleID) AS Locations
FROM [Vehicle] V
This also allows a different separator to be specified as the second parameter, providing a little more flexibility over the former method.
Note that Matt's code will result in an extra comma at the end of the string; using COALESCE (or ISNULL for that matter) as shown in the link in Lance's post uses a similar method but doesn't leave you with an extra comma to remove. For the sake of completeness, here's the relevant code from Lance's link on sqlteam.com:
DECLARE #EmployeeList varchar(100)
SELECT #EmployeeList = COALESCE(#EmployeeList + ', ', '') +
CAST(EmpUniqueID AS varchar(5))
FROM SalesCallsEmployees
WHERE SalCal_UniqueID = 1
I don't belive there's a way to do it within one query, but you can play tricks like this with a temporary variable:
declare #s varchar(max)
set #s = ''
select #s = #s + City + ',' from Locations
select #s
It's definitely less code than walking over a cursor, and probably more efficient.
In a single SQL query, without using the FOR XML clause.
A Common Table Expression is used to recursively concatenate the results.
-- rank locations by incrementing lexicographical order
WITH RankedLocations AS (
SELECT
VehicleID,
City,
ROW_NUMBER() OVER (
PARTITION BY VehicleID
ORDER BY City
) Rank
FROM
Locations
),
-- concatenate locations using a recursive query
-- (Common Table Expression)
Concatenations AS (
-- for each vehicle, select the first location
SELECT
VehicleID,
CONVERT(nvarchar(MAX), City) Cities,
Rank
FROM
RankedLocations
WHERE
Rank = 1
-- then incrementally concatenate with the next location
-- this will return intermediate concatenations that will be
-- filtered out later on
UNION ALL
SELECT
c.VehicleID,
(c.Cities + ', ' + l.City) Cities,
l.Rank
FROM
Concatenations c -- this is a recursion!
INNER JOIN RankedLocations l ON
l.VehicleID = c.VehicleID
AND l.Rank = c.Rank + 1
),
-- rank concatenation results by decrementing length
-- (rank 1 will always be for the longest concatenation)
RankedConcatenations AS (
SELECT
VehicleID,
Cities,
ROW_NUMBER() OVER (
PARTITION BY VehicleID
ORDER BY Rank DESC
) Rank
FROM
Concatenations
)
-- main query
SELECT
v.VehicleID,
v.Name,
c.Cities
FROM
Vehicles v
INNER JOIN RankedConcatenations c ON
c.VehicleID = v.VehicleID
AND c.Rank = 1
From what I can see FOR XML (as posted earlier) is the only way to do it if you want to also select other columns (which I'd guess most would) as the OP does.
Using COALESCE(#var... does not allow inclusion of other columns.
Update:
Thanks to programmingsolutions.net there is a way to remove the "trailing" comma to.
By making it into a leading comma and using the STUFF function of MSSQL you can replace the first character (leading comma) with an empty string as below:
stuff(
(select ',' + Column
from Table
inner where inner.Id = outer.Id
for xml path('')
), 1,1,'') as Values
In SQL Server 2005
SELECT Stuff(
(SELECT N', ' + Name FROM Names FOR XML PATH(''),TYPE)
.value('text()[1]','nvarchar(max)'),1,2,N'')
In SQL Server 2016
you can use the FOR JSON syntax
i.e.
SELECT per.ID,
Emails = JSON_VALUE(
REPLACE(
(SELECT _ = em.Email FROM Email em WHERE em.Person = per.ID FOR JSON PATH)
,'"},{"_":"',', '),'$[0]._'
)
FROM Person per
And the result will become
Id Emails
1 abc#gmail.com
2 NULL
3 def#gmail.com, xyz#gmail.com
This will work even your data contains invalid XML characters
the '"},{"":"' is safe because if you data contain '"},{"":"', it will be escaped to "},{\"_\":\"
You can replace ', ' with any string separator
And in SQL Server 2017, Azure SQL Database
You can use the new STRING_AGG function
The below code will work for Sql Server 2000/2005/2008
CREATE FUNCTION fnConcatVehicleCities(#VehicleId SMALLINT)
RETURNS VARCHAR(1000) AS
BEGIN
DECLARE #csvCities VARCHAR(1000)
SELECT #csvCities = COALESCE(#csvCities + ', ', '') + COALESCE(City,'')
FROM Vehicles
WHERE VehicleId = #VehicleId
return #csvCities
END
-- //Once the User defined function is created then run the below sql
SELECT VehicleID
, dbo.fnConcatVehicleCities(VehicleId) AS Locations
FROM Vehicles
GROUP BY VehicleID
I've found a solution by creating the following function:
CREATE FUNCTION [dbo].[JoinTexts]
(
#delimiter VARCHAR(20) ,
#whereClause VARCHAR(1)
)
RETURNS VARCHAR(MAX)
AS
BEGIN
DECLARE #Texts VARCHAR(MAX)
SELECT #Texts = COALESCE(#Texts + #delimiter, '') + T.Texto
FROM SomeTable AS T
WHERE T.SomeOtherColumn = #whereClause
RETURN #Texts
END
GO
Usage:
SELECT dbo.JoinTexts(' , ', 'Y')
Mun's answer didn't work for me so I made some changes to that answer to get it to work. Hope this helps someone.
Using SQL Server 2012:
SELECT [VehicleID]
, [Name]
, STUFF((SELECT DISTINCT ',' + CONVERT(VARCHAR,City)
FROM [Location]
WHERE (VehicleID = Vehicle.VehicleID)
FOR XML PATH ('')), 1, 2, '') AS Locations
FROM [Vehicle]
VERSION NOTE: You must be using SQL Server 2005 or greater with Compatibility Level set to 90 or greater for this solution.
See this MSDN article for the first example of creating a user-defined aggregate function that concatenates a set of string values taken from a column in a table.
My humble recommendation would be to leave out the appended comma so you can use your own ad-hoc delimiter, if any.
Referring to the C# version of Example 1:
change: this.intermediateResult.Append(value.Value).Append(',');
to: this.intermediateResult.Append(value.Value);
And
change: output = this.intermediateResult.ToString(0, this.intermediateResult.Length - 1);
to: output = this.intermediateResult.ToString();
That way when you use your custom aggregate, you can opt to use your own delimiter, or none at all, such as:
SELECT dbo.CONCATENATE(column1 + '|') from table1
NOTE: Be careful about the amount of the data you attempt to process in your aggregate. If you try to concatenate thousands of rows or many very large datatypes you may get a .NET Framework error stating "[t]he buffer is insufficient."
With the other answers, the person reading the answer must be aware of the vehicle table and create the vehicle table and data to test a solution.
Below is an example that uses SQL Server "Information_Schema.Columns" table. By using this solution, no tables need to be created or data added. This example creates a comma separated list of column names for all tables in the database.
SELECT
Table_Name
,STUFF((
SELECT ',' + Column_Name
FROM INFORMATION_SCHEMA.Columns Columns
WHERE Tables.Table_Name = Columns.Table_Name
ORDER BY Column_Name
FOR XML PATH ('')), 1, 1, ''
)Columns
FROM INFORMATION_SCHEMA.Columns Tables
GROUP BY TABLE_NAME
Try this query
SELECT v.VehicleId, v.Name, ll.LocationList
FROM Vehicles v
LEFT JOIN
(SELECT
DISTINCT
VehicleId,
REPLACE(
REPLACE(
REPLACE(
(
SELECT City as c
FROM Locations x
WHERE x.VehicleID = l.VehicleID FOR XML PATH('')
),
'</c><c>',', '
),
'<c>',''
),
'</c>', ''
) AS LocationList
FROM Locations l
) ll ON ll.VehicleId = v.VehicleId
If you're running SQL Server 2005, you can write a custom CLR aggregate function to handle this.
C# version:
using System;
using System.Data;
using System.Data.SqlClient;
using System.Data.SqlTypes;
using System.Text;
using Microsoft.SqlServer.Server;
[Serializable]
[Microsoft.SqlServer.Server.SqlUserDefinedAggregate(Format.UserDefined,MaxByteSize=8000)]
public class CSV:IBinarySerialize
{
private StringBuilder Result;
public void Init() {
this.Result = new StringBuilder();
}
public void Accumulate(SqlString Value) {
if (Value.IsNull) return;
this.Result.Append(Value.Value).Append(",");
}
public void Merge(CSV Group) {
this.Result.Append(Group.Result);
}
public SqlString Terminate() {
return new SqlString(this.Result.ToString());
}
public void Read(System.IO.BinaryReader r) {
this.Result = new StringBuilder(r.ReadString());
}
public void Write(System.IO.BinaryWriter w) {
w.Write(this.Result.ToString());
}
}