I am trying to become more efficient in my SQL programming.
I am trying to run a loop to repeat an update command on field names that only change by a numerical suffix.
For example, instead of writing out x_1, y_1, then x_2, y_2 for each update:
DECLARE #a INT
DECLARE #b VARCHAR
SET #a = 1
WHILE #a < 30
set #b = #a
BEGIN
UPDATE source set h = h + "x_"+#b
where "y_"+#b = 'Sold'
SET #a = #a + 1
END
Let me know if I can clarify. I'm using SQL Server 2005.
Thanks for any guidance.
I'm trying to apply Adams's solution and need to understand what is proper usage of N' in the following:
exec sp_executesql update source_temp set pmt_90_day = pmt_90_day + convert(money,'trans_total_'+#b'')
where convert(datetime,'effective_date_'+#b) <= dateadd(day,90,ORSA_CHARGE_OFF_DATE)
and DRC_FLAG_'+#b = 'C'
This won't actually work, as you can't have the column name in quotes. What you're essentially doing is having SQL compare two strings that will always be different, meaning you'll never perform an update.
If you must do it this way, you'd have to have something like...
DECLARE #a INT
DECLARE #b VARCHAR
SET #a = 1
WHILE #a < 30
BEGIN
set #b = #a
exec sp_executesql N'UPDATE source set h = h + 'x_'+#b + N'
where y_'+#b + N' = ''Sold'''
SET #a = #a + 1
END
In general, however, I'd discourage this practice. I'm not a fan of dynamic SQL being generated inside another SQL statement for any sort of production code. Very useful for doing one-off development tasks, but I don't like it for code that could get executed by a user.
Adam hit a lot around the problem itself, but I'm going to mention the underlying problem of which this is just a symptom. Your data model is almost certainly bad. If you plan to be doing much (any) SQL development you should read some introductory books on data modeling. One of the first rules of normalization is that entities should not contain repeating groups in them. For example, you shouldn't have columns called "phone_1", "phone_2", etc.
Here is a much better way to model this kind of situation:
CREATE TABLE Contacts (
contact_id INT NOT NULL,
contact_name VARCHAR(20) NOT NULL,
contact_description VARCHAR(500) NULL,
CONSTRAINT PK_Contacts PRIMARY KEY CLUSTERED (contact_id)
)
CREATE TABLE Contact_Phones (
contact_id INT NOT NULL,
phone_type VARCHAR(10) NOT NULL,
phone_number VARCHAR(20) NOT NULL,
CONSTRAINT PK_Contact_Phones PRIMARY KEY CLUSTERED (contact_id, phone_type),
CONSTRAINT CK_Contact_Phones_phone_type CHECK (phone_type IN ('HOME', 'FAX', 'MOBILE'))
)
Now, instead of trying to concatenate a string to deal with different columns you can deal with them as a set and get at the phone numbers that you want through business logic. (Sorry that I didn't use your example, but it seemed a bit too general and hard to understand).
while #count < #countOfSession
begin
if #day = 'Saturday' or #day = 'Tuesday'
begin
if #day='Saturday'
begin
select #date
set #day='Tuesday'
set #count=#count+1
set #date=#date+3
end
else if #day='Tuesday'
begin
select #date
set #day='Saturday'
set #count=#count+1
set #date=#date+4
end
end
end
Related
I have problem in optimizing an SQL query to do some data cleansing.
In fact, I have a table which is a sort of referential of a multiple special characters and word. Let's call it ABNORMAL(ID,PATTERN)
I have also another table INDIVIDUALS containing a column (NAME) which I want to clean by removing from it all characters that exist in the table ABNORMAL.
Currently, I have tried to use update statements, but I'm not sure if there is a better way to do this.
Approach one
Use a while loop to build a replace containing all characters from ABNORMALS by a blank '' and do one update using the built-in REPLACE
DECLARE #REPLACE_EXPRESSION nvarchar(max) ='REPLACE(NAME,'''','''')'
DECLARE #i int = 1
DECLARE #nbr int = (SELECT COUNT(*) FROM ABNORMAL)
-- CURRENT_CHARAC
DECLARE #CURRENT_CHARAC nvarchar(max)
-- NEW REPLACE EXPRESSION TO IMBRICATE INTO THE REPLACE EXPRESSION VARIABLE
DECLARE #CURR_REP NVARCHAR(max)
-- STRING TO BUILD AN SQL QUERY CONTAINING THE REPLACE EXPRESSION
DECLARE #UPDATE_QUERY nvarchar(max)
WHILE #i < #nbr
BEGIN
SELECT #CURRENT_CHARAC=PATTERN FROM CLEANSING_STG_PRISM_FRA_REF_UNSIGNIFICANT_VALUES WHERE ID_PATTERN=#i ;
SET #REPLACE_EXPRESSION = REPLACE(#REPLACE_EXPRESSION ,'NAME','REPLACE(NAME,'+''''+#CURRENT_CHARAC+''''+','''')')
set #i=#i+1
END
SET #UPDATE_QUERY = 'UPDATE INDIVIDUAL SET NAME ='+ #REPLACE_EXPRESSION
EXEC sp_executesql #UPDATE_QUERY
Approach two
Use a while loop to select every character in abnormal and do an update using replace containing the characters to remove:
DECLARE #i int = 1
DECLARE #nbr int = (SELECT COUNT(*) FROM ABNORMAL)
-- CURRENT_CHARAC
DECLARE #CURRENT_CHARAC nvarchar(max)
-- STRING TO BUILD AN SQL QUERY CONTAINING THE REPLACE EXPRESSION
DECLARE #UPDATE_QUERY nvarchar(max)
WHILE #i < #nbr
BEGIN
SELECT #CURRENT_CHARAC=PATTERN FROM CLEANSING_STG_PRISM_FRA_REF_UNSIGNIFICANT_VALUES WHERE ID_PATTERN=#i ;
UPDATE INDIVIDUAL
SET NAME = REPLACE(NAME,#CURRENT_CHARAC,'')
SET #i=#i+1
END
I already tested both approaches for 2 millions records, and I found that the first approach is faster than the second. I would know if you have already done something similar and new (better) ideas to try.
If you are using SQL Server 2017 you could use TRANSLATE and avoid dynamic SQL:
SELECT i.*
, REPLACE(TRANSLATE(i.NAME, f, REPLICATE('!', s.l)), '!', '') AS cleansed
FROM INDIVIDUALS i
OUTER APPLY (SELECT STRING_AGG(PATTERN, '') AS f
,LEN(STRING_AGG(PATTERN,'')) AS l
FROM ABNORMAL) AS s
DBFiddle Demo
Anyway 1st approach is better becasue you do one UPDATE, with second approach you remove characters one character at time (so you will have multiple UPDATE).
I would also track transaction log growth with both approaches.
If there's not too many characters that to be cleaned, then this trick might work.
Basically, you build 1 big update statement with a replace for each value in the table with the characters to be removed.
Example code:
Test data (using temp tables)
create table #ABNORMAL_CHARACTERS (id int identity(1,1), chr varchar(30));
insert into #ABNORMAL_CHARACTERS (chr) values ('!'),('&'),('#');
create table #INDIVIDUAL (id int identity(1,1), name varchar(30));
insert into #INDIVIDUAL (name) values ('test 1 &'),('test !'),('test 3');
Code:
declare #FieldName varchar(30) = 'name';
declare #Replaces varchar(max) = #FieldName;
declare #UpdateSQL varchar(max);
select #Replaces = concat('replace('+#Replaces+', ', ''''+chr+''','''')') from #ABNORMAL_CHARACTERS order by id;
set #UpdateSQL = 'update #INDIVIDUAL
set name = '+#Replaces + '
where exists (select 1 from #ABNORMAL_CHARACTERS where charindex(chr,name)>0)';
exec (#UpdateSQL);
select * from #INDIVIDUAL;
A test here on rextester
And if you would have a UDF that can do a regex replace.
For example here
Then the #Replaces variable could be simplified with only 1 RegexReplace function and a pattern.
WHILE #i < #deptcount + 1
BEGIN
--creating dynamic tables
DECLARE #tablenames NVARCHAR(50)
SET #tablenames = 'dept' + Cast(#i AS NVARCHAR)
EXECUTE ('create table '+#tablenames+
' (deptno int, formno int, stdpr int, agg int)')
SET #i = #i + 1
END
Your code seems to work:
DECLARE #i INT = 0, #deptcount INT = 4;
while #i < #deptcount+1
Begin
--creating dynamic tables
declare #tablenames nvarchar(50)
set #tablenames = '##dept'+CAST(#i as nvarchar)
execute('create table '+#tablenames+' (deptno int, formno int, stdpr int, agg int)')
set #i = #i +1
End
SELECT *
FROM ##dept1
UNION ALL
SELECT *
FROM ##dept2
UNION ALL
SELECT *
FROM ##dept3;
LiveDemo
But reconsider your approach:
CREATE TABLE #tbl
The desire here is to create a table of which the name is determined
at run-time.
If we just look at the arguments against using dynamic SQL in stored
procedures, few of them are really applicable here. If a stored
procedure has a static CREATE TABLE in it, the user who runs the
procedure must have permissions to create tables, so dynamic SQL will
not change anything. Plan caching obviously has nothing to do with it.
Etc.
Nevertheless: Why? Why would you want to do this? If you are creating
tables on the fly in your application, you have missed some
fundamentals about database design. In a relational database, the set
of tables and columns are supposed to be constant. They may change
with the installation of new versions, but not during run-time.
Sometimes when people are doing this, it appears that they want to
construct unique names for temporary tables. This is completely
unnecessary, as this is a built-in feature in SQL Server. If you say:
CREATE TABLE #nisse (a int NOT NULL)
then the actual name behind the scenes will be something much longer,
and no other connections will be able to see this instance of #nisse.
Thank you for helping me out through my previous tides.. I am currently working on SQL Server 2008 for one of my project, a part of which needs to use 22 columns for a set of similar operations.
The column names only differ by the number, e.g.
C1_1,C1_2,c1_3
Is there any way to loop through the column names? I tried out the following code, but it throws out an error
DECLARE #i INT
SET #i=2
while (#i<=22)
begin
SELECT [DEF].[CONCATENATE], SUM(DEF.[C1_+#i+_PREV]) as
[C1_#i_prev]
INTO #TMP_C1_#i_CONCATENATE_PREV
FROM DEF
GROUP BY DEF.[CONCATENATE]
SELECT [ABC].[CONCATENATE], SUM(ABC.[C1_#i_CURR]) as
[c1_#i_curr]
INTO #TMP_C1_#i_CONCATENATE_CURR
FROM ABC
GROUP BY ABC.[CONCATENATE]
UPDATE #tmp_var_concatenate_c1_#i
SET [Amount] = #TMP_C1_#i_CONCATENATE_PREV.[C1_#i_PREV]
FROM #tmp_var_concatenate_c1_#i
INNER JOIN
#TMP_C1_#i_CONCATENATE_PREV ON
#tmp_var_concatenate_c1_#i.[CONCATENATE] = #TMP_C1_#i_CONCATENATE_PREV.
[CONCATENATE]
Please forgive my ignorance, if I am doing something idiotic.
Thanks
This is part of the code in which the code is burping.
alter table #tmp_var_concatenate_C1_'+cast(#i as varchar)+'
add [ColA] varchar(255),
[ColB] Varchar(255),
[ColC] Varchar(255),
[ColD] VARCHAR(50),
[ColE] MONEY,
[ColF] MONEY
Is it because of the #tables that I am using ?? but, ideally, it shouldnt be an issue whether am using a Temp table or a reg. one..
You can use dynamic sql:
DECLARE #SQL varchar(max), #i INT
SET #i=2
while (#i<=22)
begin
/* Then cover all calculations with this one: */
SET #SQL='SELECT [DEF].[CONCATENATE], SUM(DEF.[C1_'+cast(#i as varchar)+'_PREV]) as
[C1_'+cast(#i as varchar)+'_prev]
INTO #TMP_C1_'+cast(#i as varchar)+'_CONCATENATE_PREV
FROM DEF
GROUP BY DEF.[CONCATENATE]
/* and all your code with the same trick in #i to the END */
'
--PRINT (#SQL) -- print it before use to see the result script
EXEC (#SQL)
/* Than do your iterations etc. */
set #i+=1
end
And don't forget to substitute all ' inside #SQL with ''.
Also you need to do all manipulations with temp tables inside #SQL, if you want to do final update outside the dynamic sql, just make tables real and then delete them.
[UPDATE]
As far as you faced with problem of altering temp tables, I tried to reproduce this error, but nothing happens, everything works fine. Please use this code as an example.
declare #sql varchar(max),#i int
set #i=2
while #i<=22
begin
set #sql='
select ID,Code into #TMP_C1_'+cast(#i as varchar)+'_CONCATENATE_PREV from (select 0 as ID, ''a'' as Code) t1
alter table #TMP_C1_'+cast(#i as varchar)+'_CONCATENATE_PREV add [Col1] varchar(255), [Col2] Varchar(255), [Col3] Money
select * from #TMP_C1_'+cast(#i as varchar)+'_CONCATENATE_PREV'
--print (#sql)
exec (#sql)
set #i+=1
end
First, I create temp table with dynamic name. Second, add new columns. The last is successful verifying. Did you execute all creations/alters in the same #sql-batch? If no, this won't work, because this tables are available only inside this batch (that's why we used varchar(max) when declared). Please describe your actions in details, maybe there is a mistake somewhere.
I've got a software suite that is based off of multiple libraries where:
1 library = 1 SQL Database.
Different users can have different access to different libraries.
In addition, the databases are named in a specific manner to help identify which are "mine" and which aren't.
I'd like to create a stored procedure that takes a variable called #UserName and returns the databases that have a name starting with MYDB, where #UserName is found in a table USERS.
I'm figuring that I'll start with EXEC sp_databases, but I'm unsure how to continue.
What I need to know is:
How do I iterate the results of sp_databases to pull out just the databases that have a name matching my pattern?
How do I then check for #UserName in the [USER NAME] column of the USERS table of each database returned from #1?
I'm guessing it has something to do with temp tables and cursors, but I'm not really sure where to start.
Any help?
Thanks!
Here is some proof of concept code to show you an approach. sys.databases contains a more accessible list of databases. You'll pretty much have to use dynamic sql at some point though.
CREATE PROCEDURE MyDBs #userName VARCHAR(255)
AS
BEGIN
DECLARE #max INT
DECLARE #i INT
DECLARE #sql VARCHAR(500)
CREATE TABLE #SQL
(
rid int identity primary key clustered,
query varchar(500)
)
INSERT INTO #SQL(query)
SELECT 'SELECT * FROM ['+ name '+].USERS WHERE username = #UserName'
FROM master.sys.databases
WHERE NAME LIKE '%yourpattern%'
SELECT #max = ##rowcount, #i = 1
WHILE #i <= #max
BEGIN
SELECT #sql = query FROM #sql WHERE rid = #i
EXEC #sql
SET #i = #i + 1
END
DROP TABLE #SQL
For 1, just look at the sp_databases code, copy it and modify it to your needs. For Example (see last 2 conditions of where clause. This is the actual code of the sp_databases stored proc. You can look at it on the master db):
declare #UserName varchar(50)='someuser'
select
DATABASE_NAME = db_name(s_mf.database_id),
DATABASE_SIZE = convert(int,
case -- more than 2TB(maxint) worth of pages (by 8K each) can not fit an int...
when convert(bigint, sum(s_mf.size)) >= 268435456
then null
else sum(s_mf.size)*8 -- Convert from 8192 byte pages to Kb
end),
REMARKS = convert(varchar(254),null)
from
sys.master_files s_mf
where
s_mf.state = 0 and -- ONLINE
has_dbaccess(db_name(s_mf.database_id)) = 1 and
--db_name(s_mf.database_id) like '%'+#UserName+'%' and exists -- you may or may not want to leave this condition here. You'll figure out what condition to use
(select 1 from databasename.dbo.Users where [UserName]=#UserName)
group by s_mf.database_id
order by 1
i write this alghorithm a and it's correct:
Tables were :
create table PrimKeyTest (primarykeycolumn varchar(8), nextcolumn int)
GO
insert into PrimKeyTest values ('P09-0001', 1)
GO
and My function is :
set ANSI_NULLS ON
set QUOTED_IDENTIFIER ON
GO
CREATE function [dbo].[GetSpecialPrimaryKey](#yearvalue int)
returns nvarchar(8)
as
begin
declare #maxkey varchar(4)
declare #maxLength int, #maxkeylength int
set #maxLength = 4
select #maxkey = ISNULL(cast(max(cast(substring(primaryKeycolumn, 5, 4) as integer)+1) as varchar),'1')
from PrimKeyTest
where substring(primaryKeycolumn, 2, 2) = substring(convert(varchar, #yearvalue), 3, 2)
set #maxkeylength = len(#maxkey)
while #maxkeylength < #maxLength
begin
set #maxkey = '0' + #maxkey
set #maxkeylength = len(#maxkey)
end
return 'P' + substring(convert(varchar, #yearvalue), 3, 2) + '-' + #maxkey
end
Now when i delete last row of this table ,new last record give correct number eg.
P09-0001 P09-0002 P09-0003 P09-0004 P09-0005
but when i delete 2nd row of this table order of primary column has incorrect
eg. P09-0001 P09-0003 P09-0004 P09-0005
can you help me?
i want this: P09-0001 P09-0002 P09-0003 P09-0004
This is actually not a great approach to dealing with primary keys. Doing this means "realigning" all of your pkeys whenever one is deleted (except when it's the last.) Doing this could be a complex, costly and error-prone process. For example, you have a pkey in this table which will probably be referenced via a foreign key from other tables. If you change the value of the pkey in the first table then you also have to change it in all the other tables that reference it. This means dropping any constraints for the duration of the change etc.
It looks like you're trying to create an identifier that will most likely be presented to the end user. You can go ahead and use your function to do that, BUT do not make it a primary key. Use an auto-incrementing column as the primary key and the 'P09-N' value as a separate field. Then, if you want to modify the values you can do so without affecting the rest of your table design.
Now to update the identifier values for the table whenever one is deleted you'll probably need to use a cursor in a stored procedure. Here's a good overview on cursors. You could also use CTEs (Common Table Expressions) to do the updating.
Here is a cursor example where Col1 is your pkey and Col2 is the identifier you want to change:
begin tran -- it's important to wrap this in a transaction!
declare #counter int
set #counter = 1
declare #val varchar(50)
DECLARE crs CURSOR
FOR SELECT Col1 FROM TblTest ORDER By Col1
OPEN crs
FETCH NEXT FROM crs INTO #val
WHILE ##FETCH_STATUS = 0
BEGIN
UPDATE TblTest
SET Col2 = 'P09-' + cast(#counter as varchar(50))
WHERE Col1 = #val
SET #counter = #counter + 1
FETCH NEXT FROM crs INTO #val
END
CLOSE crs
DEALLOCATE crs
commit tran
I didn't do the leading zero logic but you can Google for that pretty easily.