Is there a way to change all occurrences of a certain value within SQL regardless of column?
I have a table with ~200 columns which was imported from a text file. The NULL values came through as the string value 'NULL' and occur in most columns within the table. Is there a way to convert those values to true NULL values? I would like to avoid using UPDATE on each individual column is possible.
A single update may not be too painful:
update t
set col1 = nullif(col1, 'NULL'),
col2 = nullif(col2, 'NULL'),
. . .;
You can generate the code in SQL or a spreadsheet by querying INFORMATION_SCHEMA.COLUMNS(or similar) for string columns.
You can use dynamic sql to build out the update script...
DECLARE #update_sql NVARCHAR(MAX) = N''
SELECT
#update_sql = CONCAT(#update_sql, N',
mt.', c.name, N' = NULLIF(mt.', c.name, N', ''NULL'')')
FROM
sys.columns c
WHERE
c.object_id = OBJECT_ID(N'dbo.MyTable')
AND c.collation_name IS NOT NULL; -- easy way to make sure you're only looking at columns that can hold test data.
SET #update_sql = CONCAT(N'
UPDATE mt SET',
STUFF(#update_sql, 1, 1, ''), N'
FROM
dbo.MyTable mt;')
PRINT(#update_sql);
You'll end up with output formatted like the following...
UPDATE mt SET
mt.column_9 = NULLIF(mt.column_9, 'NULL'),
mt.column_10 = NULLIF(mt.column_10, 'NULL'),
mt.column_11 = NULLIF(mt.column_11, 'NULL'),
mt.column_14 = NULLIF(mt.column_14, 'NULL'),
...
mt.column_165 = NULLIF(mt.column_165, 'NULL'),
mt.column_166 = NULLIF(mt.column_166, 'NULL'),
mt.column_167 = NULLIF(mt.column_167, 'NULL'),
mt.column_168 = NULLIF(mt.column_168, 'NULL')
FROM
dbo.MyTable mt;
Note... The PRINT command is limited to 8000 characters of ASCII and 4000 characters of unicode. So, if you notice that the output script is being truncated, post back, I have a "long print" procedure that get around that limitation.
use the merge statement and set null for all matching rows which is fasters and efficient way to do it.
There is no way to do this without doing an update on each individual column.
There are shortcuts to writing such an update, like right-click>script as... or dynamic sql, but so far that's not what you've asked.
Related
Im trying to update a field by appending data to it.
if it contains the data already i wont update it otherwise I will.
if it already contains data i want it to append a comma and space followed by the word. e.g.
update myTable
set Prefixes = convert(nvarchar(max),Prefixes) + ', abc'
where MyCol='xyz' and Prefixes not like '%abc%'
im trying to get this to work so that if the prefixes column is empty initially it only includes the word 'abc'
and not ', abc'
How can i do this?
Sounds like you need a CASE:
update myTable
set Prefixes =
case
when Prefixes is null or Prefixes = ''
then 'abc'
else convert(nvarchar(max),Prefixes) + ', abc'
end
where MyCol='xyz' and (Prefixes not like '%abc%' or Prefixes is null)
See SQL Fiddle with Demo
You need to check for null values of Prefixes before filtering as NOT LIKE in the WHERE clause as well. Sql-Demo
update myTable
set Prefixes = isnull(nullif(rtrim(Prefixes),'') + ', abc','abc')
where MyCol='xyz' and isnull(Prefixes,'') not like ', abc%'
I need to select 90 columns out of 107 columns from my table.
Is it possible to write select * except( column1,column2,..) from table or any other way to get specific columns only, or I need to write all the 90 columns in select statement?
You could generate the column list:
select name + ', '
from sys.columns
where object_id = object_id('YourTable')
and name not in ('column1', 'column2')
It's possible to do this on the fly with dynamic SQL:
declare #columns varchar(max)
select #columns = case when #columns is null then '' else #columns + ', ' end +
quotename(name)
from sys.columns
where object_id = object_id('YourTable')
and name not in ('column1', 'column2')
declare #query varchar(max)
set #query = 'select ' + #columns + ' from YourTable'
exec (#query)
No, there's no way of doing * EXCEPT some columns. SELECT * itself should rarely, if ever, be used outside of EXISTS tests.
If you're using SSMS, you can drag the "columns" folder (under a table) from the Object Explorer into a query window, and it will insert all of the column names (so you can then go through them and remove the 17 you don't want)
There is no way in SQL to do select everything EXCEPT col1, col2 etc.
The only way to do this is to have your application handle this, and generate the sql query dynamically.
You could potentially do some dynamic sql for this, but it seems like overkill. Also it's generally considered poor practice to use SELECT *... much less SELECT * but not col3, col4, col5 since you won't get consistent results in the case of table changes.
Just use SSMS to script out a select statement and delete the columns you don't need. It should be simple.
No - you need to write all columns you need. You might create an view for that, so your actual statement could use select * (but then you have to list all columns in the view).
Since you should never be using select *, why is this a problem? Just drag the columns over from the Object Explorer and delete the ones you don't want.
I have as a result from a dynamic query a table with only one row, it has 1 column + d + n columns so, the problem here is that the number of 'd' and 'n' is variable so I could have a row with 5,6,7,..d values, and 10,11....n values or more...
like
Input:
x f1 f2 f3 ... fd Other1 Other2 Other3 ... Othern
10 1.0000 139.0000 60.0000 ... 59.0000 846.0000 30.1000 0.3980 ... 0.398
If I need to do some calculus with, lets say, n,f1,Other1 for the first column; n,f1,f2,Other1,Other2 for column 2, n,f1,f3,Other1,Other3 for third column... of another table like:
Column_1 Column_2 Column_3 ..Column_d
x*(f1*f1)/(Other1*Other1) x*(f1*f2)/(Other1*Other2) x*(f1*f3)/(Other1*Other3)..x*(f1*fn)/(Other1*Othern)
x*(f2*f1)/(Other2*Other1) x*(f2*f2)/(Other2*Other2) x*(f2*f3)/(Other2*Other3)..x*(f2*fn)/(Other2*Othern) ...
...
x*(fd*f1)/(Otherd*Other1) x*(fd*f2)/(Otherd*Other2) x*(fd*f3)/(Otherd*Other3)..x*(fd*fn)/(Otherd*Othern)
I was thinking to first save the columns that I need in a nested loop, and updating it until I reach the end of the table. but As I need to do that d times I'm getting a little confused, so my questions are:
Could I use a cursor to get the output?
Could I select all vars first to do that?
Using a pivot should do the trick, How?
Do not know, and the main issue is that the input table has d dynamic columns
I am trying to do a Stored procedure but have no luck, dynamically constructing SQL query in code before executing.
Thanks in advance.
Hope the question is more readeable. thank you
PS.
The x is another input, so it has nothing to do with the 'n' elements in columns Other1...Othern
--------------EDIT-----------------
To generate the input table with one row:
I use a dynamic query to select various fields, As they are dynamic I use a string which will be replaced later so the general code is:
SET #template = 'SELECT SUM(1) AS x,{f}, {other} FROM '+ #table_name
--then in some loops I calculate sums, powers, etc...
--so after I replace the strings with cosen queries I replace them like
SET #template = REPLACE(#template, '{f}' , #dynamicStringForf )
SET #template = REPLACE(#template, '{Other}', #dynamicStringForOther )
--Finally I get large query with the objetive I need
--something like:
'SELECT SUM(1) AS x, sum(a+b) as f1,pow(b,c) as f2....,sum(x+y) as Other1 ,pow(y+z) as Other 2... FROM '+ #table_name
the result is a one row with data like:
x f1 f2 f3 ... fd Other1 Other2 Other3 ... Othern
10 1.0000 139.0000 60.0000 ... 59.0000 846.0000 30.1000 0.3980 ... 0.398
Now I have created a new temp table dynamically
--#d could be any number, but at this stage I know it
Set #TempColumn = ''
Set #TempCol = ''
Set #Comma = ''
Set #ColumnNo = 1
Set #SQL = 'Create Table temp ('
WHILE #ColumnNo <= #d Begin
Set #TempColumn =#TempColumn + #Comma + ' Column_' + Cast(#ColumnNo as nvarchar)
Set #SQL =#SQL + #Comma + ' Column_' + Cast(#ColumnNo as nvarchar) + ' FLOAT'
Set #Comma = ','
Set #ColumnNo = #ColumnNo + 1
END
Set #SQL = #SQL + ' )'
EXEC (#SQL) --create temp table
--the result is a new table like:
Column_1 Column_2 Column_3 ... Column_d
Now I want to populate it, something like:
Column_1 Column_2 Column_3 ..Column_d
x*(f1*f1)/(Other1*Other1) x*(f1*f2)/(Other1*Other2) x*(f1*f3)/(Other1*Other3)..x*(f1*fn)/(Other1*Othern)
x*(f2*f1)/(Other2*Other1) x*(f2*f2)/(Other2*Other2) x*(f2*f3)/(Other2*Other3)..x*(f2*fn)/(Other2*Othern) ...
...
x*(fd*f1)/(Otherd*Other1) x*(fd*f2)/(Otherd*Other2) x*(fd*f3)/(Otherd*Other3)..x*(fd*fn)/(Otherd*Othern)
Any idea how to acomplish this, union, cursor, pivot, what could be the best
SQL is really good at working with sets of data -- as you want to do, if that data is stored as rows.
I would think the best way to solve this problem is to transform the data into a table with two columns (one column storing the f values and the second column storing the other values) with D rows.
Then the solution is fairly simple (a join and a pivot statement).
Even better -- re-write the prior query to give you the data in this format. (Do you have the prior query -- I could show you how to do that.
Well to generate the row I described I have something like:
SET #template = 'SELECT SUM(1) AS N,{f}, {other} FROM '+ #table_name
then I replace in a loop the fields I need so,
SET #template = REPLACE(#template, '{f}' , #f)
SET #template = REPLACE(#template, '{other}', #other)
I don't understand how this works... it looks like you are just selecting variables -- are those variables column names? Please clarify -- I'm sure there is a better way to build this query.
Could you explain me how to do that join and that pivot you describe?
If I use a pivot, how do I change the selected columns, to compute the terms as I described?
I will when we know what your data structure looks like for sure, I need something to test against.
f has d data and Other has n, if I put the values ina 2 column table how fo I handle the n > d, and a lot of nulls in the d column??
Often times when you have a lot of nulls you use group by to "squish" the rows down.
I have been using the statement
insert into target
select * from source
where [set of conditions] for a while.
Recently found this MERGE command that will be more effective to use for my purpose so that I can change the above statement to
MERGE target
USING source ON [my condtion]
WHEN NOT MATCHED BY TARGET
THEN INSERT VALUES (source.col1, source.col2, source.col3)
But the problem for me is lets say if I have 20+ columns in my source table I have to list all of them, I need a way to specify it to insert source.* . Is there a way ? I'm new to SQL. Appreciate your help.
Thanks in advance :)
Me too; I hate typing column names.
I normally build the Merge statement in dynamic SQL.
I have a function that takes a table name as a parameter, and returns a string containing all column names formatted properly with Table Name prefix, [] brackets and comma, as in S.Col1, S.Col2, S.Col3
I could also tell you that I build a temp table with the required columns, and pass the temp table to my function, because some times you don't want a list of all columns. But that would probably be a confusing wooble, obscuring the important bits;
Use dynamic sql
Use a function to create csv list of columns.
Everything that I have read regarding the MERGE statement says that you need to specify the columns for your INSERT statement. If you are looking for a quick way to get the INSERT statment, you can right mouse click the table in SSMS and select Script Table As->INSERT To->Clipboard. You can then paste this into your query and alter just the VALUES part.
Merge statement
There's simply no advantage of using MERGE in this situation. Why overcomplicate? Stick to the KISS principle, for chrissake.
Anyways, here's the script:
declare
#targetTableName varchar(100) = 'target'
,#targetSchemaName varchar(20) = 'dbo'
,#sourceTableName varchar(100) = 'source'
,#sourceSchemaName varchar(20) = 'dbo2'
,#matchCondition varchar(50) = 't.id = s.id'
,#columns varchar(max)
set #columns = (select ','+quotename(c.name)
from sys.tables t
join sys.columns as c on t.object_id = c.object_id
join sys.schemas s on s.schema_id = t.schema_id
where t.name = #targetTableName and s.name = isnull(#targetSchemaName, s.name)
for xml path(''))
--a column name starts with a comma
declare #sql varchar(max) = '
merge #target t
using #source s on #matchCondition
when not matched then
insert (#columns)
values #sourceColumns'
set #sql =
replace(replace(replace(replace(replace(#sql
, '#matchCondition', #matchCondition)
--replace #columns with column list with the first comma removed
, '#columns', stuff(#columns, 1, 1, ''))
--replace #sourceColumns with column list with the 's.' prefix and comma removed
, '#sourceColumns', stuff(replace(#columns, ',', ',s.'),1,1,''))
, '#target', quotename(#targetSchemaName)+'.'+quotename(#targetTableName))
, '#source', quotename(#sourceSchemaName)+'.'+quotename(#sourceTableName))
print #sql
--exec(#sql)
And we'll get something like this:
merge [dbo].[target] t
using [dbo2].[source] s on t.id = s.id
when not matched then
insert ([column1], [column2], [column3], [column4])
values s.[column1], s.[column2], s.[column3], s.[column4]
I have a table (SQL Sever) which references paths (UNC or otherwise), but now the path is going to change.
In the path column, I have many records and I need to change just a portion of the path, but not the entire path. And I need to change the same string to the new one, in every record.
How can I do this with a simple update?
It's this easy:
update my_table
set path = replace(path, 'oldstring', 'newstring')
UPDATE [table]
SET [column] = REPLACE([column], '/foo/', '/bar/')
I tried the above but it did not yield the correct result. The following one does:
update table
set path = replace(path, 'oldstring', 'newstring') where path = 'oldstring'
UPDATE CustomReports_Ta
SET vchFilter = REPLACE(CAST(vchFilter AS nvarchar(max)), '\\Ingl-report\Templates', 'C:\Customer_Templates')
where CAST(vchFilter AS nvarchar(max)) LIKE '%\\Ingl-report\Templates%'
Without the CAST function I got an error
Argument data type ntext is invalid for argument 1 of replace function.
You can use this query
update table_name set column_name = replace (column_name , 'oldstring' ,'newstring') where column_name like 'oldstring%'
all answers are great but I just want to give you a good example
select replace('this value from table', 'table', 'table but updated')
this SQL statement will replace the existence of the word "table"
(second parameter) inside the given statement(first parameter) with the third parameter
the initial value is this value from table but after executing replace function it will be this value from table but updated
and here is a real example
UPDATE publication
SET doi = replace(doi, '10.7440/perifrasis', '10.25025/perifrasis')
WHERE doi like '10.7440/perifrasis%'
for example if we have this value
10.7440/perifrasis.2010.1.issue-1
it will become
10.25025/perifrasis.2010.1.issue-1
hope this gives you better visualization
select replace(ImagePath, '~/', '../') as NewImagePath from tblMyTable
where "ImagePath" is my column Name. "NewImagePath" is temporery
column Name insted of "ImagePath" "~/" is my current string.(old
string) "../" is my requried string.(new string)
"tblMyTable" is my table in database.
you need to replace path with the help of replace function.
update table_name set column_name = replace(column_name, 'oldstring', 'newstring')
here column_name refers to that column which you want to change.
Hope it will work.
If target column type is other than varchar/nvarchar like text, we need to cast the column value as string and then convert it as:
update URL_TABLE
set Parameters = REPLACE ( cast(Parameters as varchar(max)), 'india', 'bharat')
where URL_ID='150721_013359670'
You also can replace large text for email template at run time, here is an simple example for that.
DECLARE #xml NVARCHAR(MAX)
SET #xml = CAST((SELECT [column] AS 'td','',
,[StartDate] AS 'td'
FROM [table]
FOR XML PATH('tr'), ELEMENTS ) AS NVARCHAR(MAX))
select REPLACE((EmailTemplate), '[#xml]', #xml) as Newtemplate
FROM [dbo].[template] where id = 1