I need to understand the following SQL query and would like to ask if anybody could explain it to me a bit more in detail (like the xml path) as well as update it with a replace element?
So I want to find all values with BlaBlaBlaBla and replace them with HaHaHaHa instead. At the moment the query is finding all values of BlaBlaBlaBla only.
DECLARE #searchstring NVARCHAR(255)
SET #searchstring = '%BlaBlaBlaBla%'
DECLARE #sql NVARCHAR(max)
SELECT #sql = STUFF((
SELECT
' UNION ALL SELECT ''' + TABLE_SCHEMA + '.' + TABLE_NAME + ''' AS tbl, ''' + COLUMN_NAME + ''' AS col, [' + COLUMN_NAME + '] AS val' +
' FROM [' + TABLE_SCHEMA + '].[' + TABLE_NAME + '] WHERE [' + COLUMN_NAME + '] LIKE ''' + #searchstring + ''''
FROM INFORMATION_SCHEMA.COLUMNS
WHERE DATA_TYPE in ('nvarchar', 'varchar', 'char', 'ntext') FOR XML PATH('')) ,1, 11, '')
Exec (#sql)
I believe that the XML PATH is a trick to get the strings to all concatenate together.
You could change it to REPLACE with something like this:
SELECT #sql =
SELECT
' UPDATE ' + QUOTENAME(TABLE_SCHEMA) + '.' + QUOTENAME(TABLE_NAME) + '
SET ' + QUOTENAME(COLUMN_NAME) + ' = REPLACE(' + QUOTENAME(COLUMN_NAME) + ', ''' + #search_string + ''', ' + ', ''' + #replace_string + '''
WHERE ' + QUOTENAME(COLUMN_NAME) + ' LIKE ''' + #searchstring + ''''
FROM INFORMATION_SCHEMA.COLUMNS
WHERE
DATA_TYPE in ('nvarchar', 'varchar', 'char', 'ntext') FOR XML PATH('')
EXEC(#sql)
Some caveats:
I haven't tested this. When you're generating code like this it's very easy to make minor errors with all of the start and end quotes, etc. I would print out the SQL and check it, repeating as necessary until you get the output SQL correct.
Also, this is generally not a good idea. If your database is large and/or has a large number of tables then performance is going to be miserable. You should usually do the analysis of where you think this sort of data is going to appear and write code that will correct it as necessary. The fact that data elements are buried in strings throughout your data is concerning.
Finally, be aware that this might easily update additional data that you didn't intend to update. If you try to update "123" with "456" and there's a string out there that is "My ID is 1234" it's going to become "My ID is 4564".
BTW, the QUOTENAME function is a way of enclosing your table and column names in [ and ], but if the quote character is changed in a DB implementation it should still work.
Related
I have spent so long trying to solve my problem to absolutely no avail. I need to create a procedure or loop through whatever you want to call it that scans all tables in all databases in my warehouse and identifies duplicate records. Clearly I want to avoid writing hundreds of queries and manually entering each column name from each table so it needs to be done dynamically. I have 12 or so databases but hundreds of tables and perhaps thousands of columns.
how does one do this?
Something like this might get you started. You can write a cursor to go through the tables in INFORMATION_SCHEMA.TABLES then set the variables from there. Once you run this in a database you'll then need to copy and paste the code that gets generated and run that code to see duplicates.
DECLARE
#column_list NVARCHAR(MAX),
#sql NVARCHAR(MAX),
#table_name SYSNAME,
#table_schema SYSNAME
SELECT #table_schema = 'MySchema', #table_name = 'MyTable'
SELECT #column_list = ''
SELECT #column_list = #column_list + '[' + C.COLUMN_NAME + '], '
FROM INFORMATION_SCHEMA.COLUMNS C
WHERE C.TABLE_NAME = #table_name AND C.TABLE_SCHEMA = #table_schema
SELECT #column_list = SUBSTRING(#column_list, 1, LEN(#column_list) - 1)
SELECT #sql = 'SELECT ''' + #table_schema + ''' AS TABLE_SCHEMA, ''' + #table_name + ''' AS TABLE_NAME, ' + #column_list + ' FROM [' + #table_schema + '].[' + #table_name + '] GROUP BY ' + #column_list + ' HAVING COUNT(*) > 1'
SELECT #sql
I am rather new to this and I am having a hard time into converting the below SQL query into an UPDATE statement for a stored procedure.
SELECT 'select'+
stuff((
SELECT ',' + 'dbo.' + Function_Name + '(' + Parameters_List + ')' FROM
[SPECIFIC_DATABASE]..Specific_table c WHERE c.Table_Name = t.Table_Name FOR
XML PATH('')),1,1,'')
+' from [' + Database_Name +'].[dbo].['+Table_Name+'] '
+ 'Where Audit_ID>' + CAST(#Audit_ID as nvarchar(100))
As 'Specific Queries'
FROM (SELECT Distinct Database_Name, Table_Name FROM [SPECIFIC_DATABASE]..Specific_table) t
The UPDATE query should be something like
UPDATE Table_name
SET Column_name = Function_Name(Parameters_List)
WHERE Audit_id >= #Audit_ID
FROM [SPECIFIC_DATABASE]..Specific_table
Any suggestions and guidelines on this would be much appreciated!
I think this should give you what you want, but I don't see any any reference to a Column_Name, so I'm assuming you will hardcode that.
select 'UPDATE tbl' + stuff((
select ' set Column_Name = ' + 'dbo.' + Function_Name + '(' + Parameters_List + ')'
from [SPECIFIC_DATABASE]..Specific_table c
where c.Table_Name = t.Table_Name
for xml PATH('')
), 1, 1, '')
+ ' from [' + Database_Name + '].[dbo].[' + Table_Name + '] tbl'
+ 'Where Audit_ID>'
+ CAST(#Audit_ID as nvarchar(100)) as 'Specific Queries'
from (
select distinct Database_Name, Table_Name
from [SPECIFIC_DATABASE]..Specific_table
) t
If the answer's not right then it might be helpful if you post what is the current output of your first query and maybe some more details as to what are the contents of the table called "Specific_table".
I have a stored procedure which when run gives a table output. I want to export this procedure to a csv file but want to append double/single quotes for all the columns with a datatype CHAR/VARCHAR.
For Example:
Stored Proc O/P:
ID Name Address SSN
1 abd 9301,LeeHwy, 22031 64279100
Output in CSV File:
1,"abd","9301,LeeHwy, 22031",64279100
Can anyone also help me on how I can use a BAT file to execute the procedure and generate this csv file.
One way to do this, is to loop through the table schema to extract the varchar columns. I have tested this for one of my tables, and it worked:
DECLARE #tableName VARCHAR(Max) = '[Put your table name here]';
DECLARE #currColumns VARCHAR(Max) = NULL;
SELECT #currColumns = COALESCE(#currColumns + ','
+ CASE WHEN t.Name = 'varchar' THEN '''"'' + ' ELSE '' END
+ '[', '[') + c.name + ']'
+ CASE WHEN t.Name = 'varchar' THEN '+ ''"''' ELSE '' END
+ ' as [' + c.name + ']'
FROM
sys.columns c
INNER JOIN
sys.types t ON c.user_type_id = t.user_type_id
WHERE
c.object_id = OBJECT_ID(#tableName)
EXEC('SELECT ' + #currColumns + ' FROM ' + #tableName);
It's a quick and dirty way.
UPDATE (comment):
Inserting into a table is really easy. Just do this:
INSERT INTO [TABLE]
EXEC('SELECT ' + #currColumns + ' FROM ' + #tableName);
I have found a solution for my problem.
Credits also go to #Rogala (The developer who gave initial answer to the question) for triggering the idea of using system tables.
The code is as below:
DECLARE #tableName VARCHAR(Max) = '[Put your table name here]';
DECLARE #currColumns VARCHAR(Max) = NULL;
Declare #Delim CHAR(5)='''"''+'
SELECT #currColumns = COALESCE(#currColumns + ','+ CASE WHEN DATA_TYPE= 'varchar' THEN '''"'' + ' ELSE '' END + '[', '[') + COLUMN_NAME + ']'
+ CASE WHEN DATA_TYPE = 'varchar' THEN '+ ''"''' ELSE '' END + ' as [' + COLUMN_NAME + ']'
FROM INFORMATION_SCHEMA.Columns
WHERE table_name = #tableName
Set #currColumns= #Delim+#currColumns
EXEC('SELECT ' + #currColumns + ' FROM ' + #tableName);
I have been notified that a company website we have has had a problem, and quickly I can see it had a SQL injection attack. I do not manage the website and have no access to the files (and know how I can prevent this in the future) but my current task is to clean the database. It seems there is HTML appended to almost all varchar columns in a Microsoft SQL Server database.
Is there any way in an easy query or function I can run to check all columns for the offending HTML and update the columns in all tables?
For example a column that was:
---------------------------
|Title
---------------------------
|product1
is now
---------------------------
|Title
---------------------------
|product1</title><style>.atpd{position:absolute;clip:rect(400px,auto,auto,400px);}</style><div class=atpd>Apply here <a href=http://abbypaydayloansonline.com >online payday loans</a></div>
Thanks in advance.
Here's one option, assuming the offending text always starts with </title> and that </title> wouldn't naturally appear in the data.
UPDATE dbo.table_name
SET Title = LEFT(Title, CHARINDEX('</title>', Title)-1)
WHERE Title LIKE '%</title>%'
AND Title LIKE '%abbypaydayloansonline.com%';
If you need to do this for multiple columns across multiple tables (assuming, again, that </title> appears first and </title> would never have appeared naturally in the data prior to the incident), you don't need an explicit cursor:
DECLARE #sql NVARCHAR(MAX);
SET #sql = N'';
SELECT #sql = #sql + N'UPDATE ' + QUOTENAME(OBJECT_SCHEMA_NAME([object_id]))
+ '.' + QUOTENAME(OBJECT_NAME([object_id]))
+ ' SET ' + QUOTENAME(name) + ' = LEFT(' + QUOTENAME(name)
+ ', CHARINDEX(''</title>'', ' + QUOTENAME(name) + ')-1)
WHERE ' + QUOTENAME(name) + ' LIKE ''%</title>%''
AND ' + QUOTENAME(name) + ' LIKE ''%abbypaydayloansonline.com%'''
+ ';' + CHAR(13) + CHAR(10)
FROM sys.columns
WHERE OBJECTPROPERTY([object_id], 'IsMsShipped') = 0
AND system_type_id IN (35,99,167,175,231,239,231);
SELECT #sql;
-- EXEC sp_executesql #sql;
Absolutely inspect the output before running it. Note that Management Studio will only show you a small subset of the actual command that will get executed, so you might also want to run this query to see all of the tables and columns that will be checked:
SELECT [table] = QUOTENAME(OBJECT_SCHEMA_NAME([object_id]))
+ '.' + QUOTENAME(OBJECT_NAME([object_id])),
[column] = name
FROM sys.columns
WHERE OBJECTPROPERTY([object_id], 'IsMsShipped') = 0
AND system_type_id IN (35,99,167,175,231,239,231)
ORDER BY [table], [column];
Assuming you don't want to have to type in all your queries manually, I would recommend creating a CURSOR (as much as I hate cursors) and loop through the system tables, something like this:
SELECT t.name AS table_name,
SCHEMA_NAME(schema_id) AS schema_name,
c.name AS column_name
FROM sys.tables AS t
INNER JOIN sys.columns c ON t.OBJECT_ID = c.OBJECT_ID
Then you can run dynamic SQL to do something like #AaronBertrand suggested.
Declare your cursor and fetch your variables into #table_name and #column_name. Then run something like (untested):
DECLARE #updateSQL NVARCHAR(MAX)
SET #updateSQL = 'UPDATE ' + #table_name + ' SET ' + #column_name + ' = LEFT(' + #column_name + ', CHARINDEX(''</title>'', ' + #column_name + ')-1)'
EXECUTE (#updateSQL)
Good luck.
I'm trying to determine the percent of null items for all fields of a table. I have to run this on several tables with a ton of fields, and I was looking for an automated way of doing this.
I know I can query "information_schema.columns" and get a nice clean list of field names ala:
select Column_name
from information_schema.columns
where table_name='TableName'
But I can't seem to come up with something quick and dirty to do the percentage count for each field, I'm guessing I'll need some dynamic sql of some sort? Anyone have a suggestion on a good way to approach this?
Maybe too simplistic, but the basic idea can be expanded in different ways (I normally have variables for #CRLF and #TAB for code generation):
DECLARE #sql AS varchar(MAX)
SELECT #sql = COALESCE(#sql + CHAR(13) + CHAR(10) + 'UNION' + CHAR(13) + CHAR(10), '')
+ 'SELECT ''' + QUOTENAME(TABLE_SCHEMA) + '.' + QUOTENAME(TABLE_NAME) + '.' + QUOTENAME(COLUMN_NAME) + ''' AS COLUMN_NAME' + CHAR(13) + CHAR(10)
+ CHAR(9) + ',COUNT(*) AS TotalRows' + CHAR(13) + CHAR(10)
+ CHAR(9) + ',COUNT(' + COLUMN_NAME + ') AS NonNullCount' + CHAR(13) + CHAR(10)
+ 'FROM ' + QUOTENAME(TABLE_SCHEMA) + '.' + QUOTENAME(TABLE_NAME)
FROM INFORMATION_SCHEMA.COLUMNS WHERE IS_NULLABLE = 'YES'
PRINT #sql
EXEC (#sql)
As far as your percentages, I wasn't sure if that was over the entire table or a particular column only, so I'll leave that as an exercise for the reader.
AFAIK the only way to do it is to use dynamic sql (e.g., sp_executesql). There are index statistics but nulls aren't stored in indexes...