So I have a situation where I know the procedure will query out a row that is formatted in UTCStampTime fashion. THe problem I am running into is when I want to build a query on the two CTE's I have need to be returned only when the time is equal on each CTE variables. But the problem is that my query only is looking at the last 10 minutes and I only want the two rows to be equal when the minutes of the utcTimeStamp columns are the same. here is my query that i am running.
GO
declare #fqnLine1Downtime varChar(4000)
declare #fqnLine1Throughput varChar(400)
declare #fqnLine2Downtime varChar(4000)
declare #fqnLine2Throughput varChar(400)
declare #parameter1Line1 varChar(4000)
declare #parameter2Line1 varChar(4000)
declare #parameter1Line2 varChar(4000)
declare #parameter2Line2 varChar(4000)
declare #server varChar(200)
declare #timeStart datetime
declare #timeEnd datetime
WITH
CTEThroughputLine1(Time, Amount, ThroughputLine1)
AS
--Define the query
(
SELECT CONVERT(VarChar(20),utcTimeStamp) as Time, shortName as
ThroughputLine1, valueAsInt as Amount
From dbo.History(#fqnLine1Throughput, #parameter2Line1, #timeStart,
#timeEnd, #server)
),
CTEThroughputLine2(ThroughputLine2, Amount, Time)
AS
(
Select shortName as ThroughputLine2, valueAsFloat as Amount,
CONVERT(VarChar(20),utcTimeStamp) as Time
From dbo.History(#fqnLine2Throughput, #parameter2Line2, #timeStart,
#timeEnd, #server)
)
select * from CTEThroughputLine1 as Line1 inner join
CTEThroughputLine2 as Line2 on Line1.Time = Line2.Time
I have purposefully not include the setting of the variables because it includes private info on my Company. I think this should be enough to tackle the problem. Please help or point out hints.
Related
I have problem in optimizing an SQL query to do some data cleansing.
In fact, I have a table which is a sort of referential of a multiple special characters and word. Let's call it ABNORMAL(ID,PATTERN)
I have also another table INDIVIDUALS containing a column (NAME) which I want to clean by removing from it all characters that exist in the table ABNORMAL.
Currently, I have tried to use update statements, but I'm not sure if there is a better way to do this.
Approach one
Use a while loop to build a replace containing all characters from ABNORMALS by a blank '' and do one update using the built-in REPLACE
DECLARE #REPLACE_EXPRESSION nvarchar(max) ='REPLACE(NAME,'''','''')'
DECLARE #i int = 1
DECLARE #nbr int = (SELECT COUNT(*) FROM ABNORMAL)
-- CURRENT_CHARAC
DECLARE #CURRENT_CHARAC nvarchar(max)
-- NEW REPLACE EXPRESSION TO IMBRICATE INTO THE REPLACE EXPRESSION VARIABLE
DECLARE #CURR_REP NVARCHAR(max)
-- STRING TO BUILD AN SQL QUERY CONTAINING THE REPLACE EXPRESSION
DECLARE #UPDATE_QUERY nvarchar(max)
WHILE #i < #nbr
BEGIN
SELECT #CURRENT_CHARAC=PATTERN FROM CLEANSING_STG_PRISM_FRA_REF_UNSIGNIFICANT_VALUES WHERE ID_PATTERN=#i ;
SET #REPLACE_EXPRESSION = REPLACE(#REPLACE_EXPRESSION ,'NAME','REPLACE(NAME,'+''''+#CURRENT_CHARAC+''''+','''')')
set #i=#i+1
END
SET #UPDATE_QUERY = 'UPDATE INDIVIDUAL SET NAME ='+ #REPLACE_EXPRESSION
EXEC sp_executesql #UPDATE_QUERY
Approach two
Use a while loop to select every character in abnormal and do an update using replace containing the characters to remove:
DECLARE #i int = 1
DECLARE #nbr int = (SELECT COUNT(*) FROM ABNORMAL)
-- CURRENT_CHARAC
DECLARE #CURRENT_CHARAC nvarchar(max)
-- STRING TO BUILD AN SQL QUERY CONTAINING THE REPLACE EXPRESSION
DECLARE #UPDATE_QUERY nvarchar(max)
WHILE #i < #nbr
BEGIN
SELECT #CURRENT_CHARAC=PATTERN FROM CLEANSING_STG_PRISM_FRA_REF_UNSIGNIFICANT_VALUES WHERE ID_PATTERN=#i ;
UPDATE INDIVIDUAL
SET NAME = REPLACE(NAME,#CURRENT_CHARAC,'')
SET #i=#i+1
END
I already tested both approaches for 2 millions records, and I found that the first approach is faster than the second. I would know if you have already done something similar and new (better) ideas to try.
If you are using SQL Server 2017 you could use TRANSLATE and avoid dynamic SQL:
SELECT i.*
, REPLACE(TRANSLATE(i.NAME, f, REPLICATE('!', s.l)), '!', '') AS cleansed
FROM INDIVIDUALS i
OUTER APPLY (SELECT STRING_AGG(PATTERN, '') AS f
,LEN(STRING_AGG(PATTERN,'')) AS l
FROM ABNORMAL) AS s
DBFiddle Demo
Anyway 1st approach is better becasue you do one UPDATE, with second approach you remove characters one character at time (so you will have multiple UPDATE).
I would also track transaction log growth with both approaches.
If there's not too many characters that to be cleaned, then this trick might work.
Basically, you build 1 big update statement with a replace for each value in the table with the characters to be removed.
Example code:
Test data (using temp tables)
create table #ABNORMAL_CHARACTERS (id int identity(1,1), chr varchar(30));
insert into #ABNORMAL_CHARACTERS (chr) values ('!'),('&'),('#');
create table #INDIVIDUAL (id int identity(1,1), name varchar(30));
insert into #INDIVIDUAL (name) values ('test 1 &'),('test !'),('test 3');
Code:
declare #FieldName varchar(30) = 'name';
declare #Replaces varchar(max) = #FieldName;
declare #UpdateSQL varchar(max);
select #Replaces = concat('replace('+#Replaces+', ', ''''+chr+''','''')') from #ABNORMAL_CHARACTERS order by id;
set #UpdateSQL = 'update #INDIVIDUAL
set name = '+#Replaces + '
where exists (select 1 from #ABNORMAL_CHARACTERS where charindex(chr,name)>0)';
exec (#UpdateSQL);
select * from #INDIVIDUAL;
A test here on rextester
And if you would have a UDF that can do a regex replace.
For example here
Then the #Replaces variable could be simplified with only 1 RegexReplace function and a pattern.
I am trying to run a simple script in SQL Server Management Studio.
Declare #total varchar (max)
set #total = 1
Declare #total2 varchar (max)
set #total2 = '1,4'
if #total in (#total2)
print 'Success'
For some reason I cannot get this simple script to work. It will work when I do this though "if 1 in ('1,2')" it will print "Success" Any help[ would be greatly appreciated.
You need to use Dynamic SQL because your #total contains varchar not numbers
Declare #total varchar (max)
set #total = 1
Declare #total2 varchar (max)
set #total2 = '1,4'
exec ('if '+#total+' in ('+#total2+') print ''Success'' ' )
IN works against sets, not strings. You could either declare a table variable and perform INSERT statements:
Declare #total varchar (max)
set #total = 1
Declare #total2 TABLE varchar (max)
(
total int
)
INSERT INTO #total2 VALUES (1)
INSERT INTO #total2 VALUES (4)
if #total in (SELECT total FROM #total2)
print 'Success'
or use string functions like CHARINDEX, but then you have to consider partial matches, etc.
You could convert a comma-delimited string to a table, but you'd still need to insert the values one-by one using a loop.
I am getting an error stating that I Must declare the table variable "#tempattend".
Please help me. How can I pass #tempattend table to #tempTableSelect variable?
ALTER PROCEDURE [dbo].[Rl_LM_AHS]
#SupEmpID nvarchar(10),
#SectorName nvarchar(300),
#dateList nvarchar(300),
#Month nvarchar(5),
#Year nvarchar(5),
#SearchType nvarchar(10)
AS
BEGIN
SET NOCOUNT ON;
declare #tempTableSelect nvarchar(2000)
DECLARE #tempattend Table
(
[Emp.ID] nvarchar(10),
[Name] nvarchar(60),
[1] nvarchar(3) null,
[2] nvarchar(3) null,
[3] nvarchar(3) null
upto ..............
[31] nvarchar(3) null
)
IF (#SearchType = 1)
BEGIN
--INSERT TEAM LIST TO #tempattend TABLE
insert into #tempattend([Emp.ID],[Name]) (Select EMP.empID as [Emp.ID],CONCAT(EMP.emp_fname,' ',COALESCE(nullif(EMP.emp_Mname,'') +' ',''),COALESCE(nullif(EMP.emp_Lname,'') +' ','')) as [Name] from EShiftHistory)
set #tempTableSelect = 'select [Emp.ID],Name,' + #dateList +' from #tempattend'
EXEC (#tempTableSelect)
END
END
You should write
set #tempTableSelect = 'select [Emp.ID],Name,' + #dateList +' from #tempattend'
#tempattend is a temporary table variable. It holds a table, not some value like #datelist.
But why do you EXEC instead of just selecting directly from the table?
Come to think of it: It may not be possible to use memory temp tables in EXEC statements. Try turning this
DECLARE #tempattend Table
into
CREATE TABLE #tempattend
and change every occurance of #tempattend to #tempattend.
Following up on Thorsten Dittmar's answer, why not something like this?
select empID,
CONCAT(emp_fname,' ',
COALESCE(emp_Mname,''),
COALESCE(nullif(emp_Lname,''))) as [Name]
from EShiftHistory
I realize you're trying to do some dynamic selection with the #dateList variable, but that's a strong signal that you should either normalize your tables or just let your client code pick out which columns it wants. (Not everything needs to be done in raw SQL.)
Numbers as column names? Definitely problematic.
I need to retrieve change-data-capture rows for several tables, and I'm required (by company IT policy) to access the database via stored procedures. I would rather create a single stored procedure with the table name as a parameter, rather than one stored procedure for each table I'm monitoring. Where I get hung up is that CDC defines a separate table-valued function name for each table monitored, and I'm not sure how best to generalize around that.
Is it possible to modify the following example code so that it invokes cdc.fn_cdc_get_net_changes_dbo_ + #Table instead of cdc.fn_cdc_get_net_changes_dbo_TABLE ?
Is there another approach I should use?
create proc [dbo].GetChangesForTable
#Table varchar(50),
#BeginTime datetime,
#EndTime datetime
as
begin
DECLARE #begin_lsn binary(10), #end_lsn binary(10);
DECLARE #func nvarchar(128)
if #EndTime is null select #EndTime=GETDATE()
SELECT #begin_lsn = sys.fn_cdc_map_time_to_lsn('smallest greater than', #BeginTime);
SELECT #end_lsn = sys.fn_cdc_map_time_to_lsn('largest less than or equal', #EndTime);
-- HOW TO GET THE CORRECT FUNCTION CALLED HERE?
SELECT * FROM cdc.fn_cdc_get_net_changes_dbo_TABLE(#begin_lsn, #end_lsn, 'all')
end
GO
I think that's possible with sp_executesql like that :
DECLARE #sql nvarchar(4000)
SET #sql = N'SELECT * FROM cdc.fn_cdc_get_net_changes_dbo_'+cast(#TABLE as varchar)+'(#begin_lsn, #end_lsn, 'all')'
EXEC sp_executesql #sql, N'#Table varchar(50), #BeginTime datetime, #EndTime datetime',#Table,#BeginTime,#EndTime
I am having a small problem with the IN SQL statement. I was just wondering if anyone could help me?
#Ids = "1,2,3,4,5"
SELECT * FROM Nav WHERE CONVERT(VARCHAR,NavigationID) IN (CONVERT(VARCHAR,#Ids))
This is coming back with the error below, I am sure this is pretty simple!
Conversion failed when converting the varchar value '1,' to data type int.
The SQL IN clause does not accept a single variable to represent a list of values -- no database does, without using dynamic SQL. Otherwise, you could use a Table Valued Function (SQL Server 2000+) to pull the values out of the list & return them as a table that you can join against.
Dynamic SQL example:
EXEC('SELECT *
FROM Nav
WHERE NavigationID IN ('+ #Ids +')')
I recommend reading The curse and blessings of dynamic SQL before using dynamic SQL on SQL Server.
Jason:
First create a function like this
Create FUNCTION [dbo].[ftDelimitedAsTable](#dlm char, #string varchar(8000))
RETURNS
--------------------------------------------------------------------------*/
/*------------------------------------------------------------------------
declare #dlm char, #string varchar(1000)
set #dlm=','; set #string='t1,t2,t3';
-- tHIS FUNCION RETUNRS IN THE ASCENDING ORDER
-- 19TH Apr 06
------------------------------------------------------------------------*/
--declare
#table_var TABLE
(id int identity(1,1),
r varchar(1000)
)
AS
BEGIN
declare #n int,#i int
set #n=dbo.fnCountChars(#dlm,#string)+1
SET #I =1
while #I <= #N
begin
insert #table_var
select dbo.fsDelimitedString(#dlm,#string,#i)
set #I= #I+1
end
if #n =1 insert #TABLE_VAR VALUES(#STRING)
delete from #table_var where r=''
return
END
And then
set quoted_identifier off
declare #ids varchar(max)
select #Ids = "1,2,3,4,5"
declare #nav table ( navigationid int identity(1,1),theother bigint)
insert #nav(theother) select 10 union select 11 union select 15
SELECT * FROM #Nav WHERE CONVERT(VARCHAR,NavigationID) IN (select id from dbo.ftDelimitedAsTable(',',#Ids))
select * from dbo.ftDelimitedAsTable(',',#Ids)
What you're doing is not possible with the SQL IN statement. You cannot pass a string to it and expect that string to be parsed. IN is for specific, hard-coded values.
There are two ways to do what you want to do here.
One is to create a 'dynamic sql' query and execute it, after substituting in your IN list.
DECLARE #query varchar(max);
SET #query = 'SELECT * FROM Nav WHERE CONVERT(VARCHAR,NavigationID) IN (' + #Ids + ')'
exec (#query)
This can have performance impacts and other complications. Generally I'd try to avoid it.
The other method is to use a User Defined Function (UDF) to split the string into its component parts and then query against that.
There's a post detailing how to create that function here
Once the function exists, it's trivial to join onto it
SELECT * FROM Nav
CROSS APPLY dbo.StringSplit(#Ids) a
WHERE a.s = CONVERT(varchar, Nav.NavigationId)
NB- the 'a.s' field reference is based on the linked function, which stores the split value in a column named 's'. This may differ based on the implementation of your string split function
This is nice because it uses a set based approach to the query rather than an IN subquery, but a CROSS JOIN may be a little complex for the moment, so if you want to maintain the IN syntax then the following should work:
SELECT * FROM Nav
WHERE Nav.NavigationId IN
(SELECT CONVERT(int, a.s) AS Value
FROM dbo.StringSplit(#Ids) a