SQL Delete Where Not In - sql

I have a relation mapping table like this:
attributeid bigint
productid bigint
To clean relations that are not used any more, I want to delete all recors where productid = x and attributeid not in (#includedIds), like the following example:
#attributetypeid bigint,
#productid bigint,
#includedids varchar(MAX)
DELETE FROM reltable
WHERE productid = #productid AND
attributetypeid = #attributetypeid AND
attributeid NOT IN (#includedids);
When running the SQL with the includedids param containing more than 1 id - like this: 25,26 - I get a SqlException saying:
Error converting data type varchar to bigint.
And that is of course due to the , in that varchar(max) param...
How should I construct my delete statement to make it work?

SET QUOTED_IDENTIFIER ON
GO
CREATE FUNCTION [dbo].[ListToTable] (
/*
FUNCTION ListToTable
Usage: select entry from listtotable('abc,def,ghi') order by entry desc
PURPOSE: Takes a comma-delimited list as a parameter and returns the values of that list into a table variable.
*/
#mylist varchar(8000)
)
RETURNS #ListTable TABLE (
seqid int not null,
entry varchar(255) not null)
AS
BEGIN
DECLARE
#this varchar(255),
#rest varchar(8000),
#pos int,
#seqid int
SET #this = ' '
SET #seqid = 1
SET #rest = #mylist
SET #pos = PATINDEX('%,%', #rest)
WHILE (#pos > 0)
BEGIN
set #this=substring(#rest,1,#pos-1)
set #rest=substring(#rest,#pos+1,len(#rest)-#pos)
INSERT INTO #ListTable (seqid,entry) VALUES (#seqid,#this)
SET #pos= PATINDEX('%,%', #rest)
SET #seqid=#seqid+1
END
set #this=#rest
INSERT INTO #ListTable (seqid,entry) VALUES (#seqid,#this)
RETURN
END
Run that script in your SQL Server database to create the function ListToTable. Now, you can rewrite your query like so:
#attributetypeid bigint,
#productid bigint,
#includedids varchar(MAX)
DELETE FROM reltable
WHERE productid = #productid AND
attributetypeid = #attributetypeid AND
attributeid NOT IN (SELECT entry FROM ListToTable(#includedids));
Where #includedids is a comma delimited list that you provide. I use this function all the time when working with lists. Keep in mind this function does not necessarily sanitize your inputs, it just looks for character data in a comma delimited list and puts each element into a record. Hope this helps.

Joel Spolsky answered a very similar question here: Parameterize an SQL IN clause
You could try something similar, making sure to cast your attributetypeid as a varchar.

You can't pass a list as an parameter (AFAIK).
Can you rewrite the sql to use a subquery, something like this:
delete from reltable
WHERE productid = #productid AND
attributetypeid = #attributetypeid AND
attributeid NOT IN (select id from ... where ... );
?

That comma delimited list can be sent to a user defined function which will return it as a simple table. That table can then be queried by your NOT IN.
If you need the fn I can provide.. It's been about 5 yrs since I used sql much and I'll have to dust off that section of my brain..

Erland has the definitive guide for dealing with lists to table in SQL 2005, SQL 2008 gives you table based params.
On a side note I would avoid a NOT IN pattern for large lists, cause it does not scale, instead look at using left joins.

Related

SQL Server function returns no value

I might be overlooking something here or it could be the lack of coffee this am... I have a SQL Server function I'm trying to create and have it spit out results.
This is my code:
ALTER FUNCTION [dbo].[fn_top_forms] ()
RETURNS #topforms TABLE
(
-- Add the column definitions for the TABLE variable here
clicks varchar(50),
title varchar(150),
urlpath varchar(max)
)
AS
BEGIN
DECLARE #clicks as varchar(50)
DECLARE #title varchar(150)
DECLARE #urlpath varchar(max)
DECLARE #everyone_relevant as bit
IF #everyone_relevant = 1
BEGIN
INSERT INTO #topforms
SELECT TOP (10) clicks, title, url
FROM dbo.view_all_forms
ORDER BY clicks DESC
END
RETURN
END
My issue is that its not returning any values but I know in the view_all_forms, there is indeed data there. I'm using this in new query to return the results, it has the columns but no data. Any ideas?
Yup, looks like I just didnt have the thing set write... Note to self, DRINK MORE coffee in the am.

SQL Server 2012: Remove text from end of string

I'm new to SQL so please forgive me if I use incorrect terminology and my question sounds confused.
I've been tasked with writing a stored procedure which will be sent 3 variables as strings (varchar I think). I need to take two of the variables and remove text from the end of the variable and only from the end.
The strings/text I need to remove from the end of the variables are
co
corp
corporation
company
lp
llc
ltd
limited
For example this string
Global Widgets LLC
would become
Global Widgets
However it should only apply once so
Global Widgets Corporation LLC
Should become
Global Widgets Corporation
I then need to use the altered variables to do a SQL query.
This is to be used as a backup for an integration piece we have which makes a callout to another system. The other system takes the same variables and uses Regex to remove the strings from the end of variables.
I've tried different combinations of PATINDEX, SUBSTRING, REPLACE, STUFF but cannot seem to come up with something that will do the job.
===============================================================
Edit: I want to thank everyone for the answers provided so far, but I left out some information that I didn't think was important but judging by the answers seems like it would affect the processing.
My proc will start something like
ALTER PROC [dbo].[USP_MyDatabaseTable] #variableToBeAltered nvarchar(50)
AS
I will then need to remove all , and . characters. I've already figured out how to do this. I will then need to do the processing on #variableToBeAltered (technically there will be two variables) to remove the strings I listed previously. I must then remove all spaces from #variableToBeAltered. (Again I figured that part out). Then finally I will use #variableToBeAltered in my SQL query something like
SELECT [field1] AS myField
,[field2] AS myOtherField
FROM [MyData].[dbo].[MyDatabaseTable]
WHERE [field1] = (#variableToBeAltered);
I hope this information is more useful.
I'd keep all of your suffixes in a table to make this a little easier. You can then perform code like this either within a query or against a variable.
DECLARE #company_name VARCHAR(50) = 'Global Widgets Corporation LLC'
DECLARE #Suffixes TABLE (suffix VARCHAR(20))
INSERT INTO #Suffixes (suffix) VALUES ('LLC'), ('CO'), ('CORP'), ('CORPORATION'), ('COMPANY'), ('LP'), ('LTD'), ('LIMITED')
SELECT #company_name = SUBSTRING(#company_name, 1, LEN(#company_name) - LEN(suffix))
FROM #Suffixes
WHERE #company_name LIKE '%' + suffix
SELECT #company_name
The keys here are that you are only matching with strings that end in the suffix and it uses SUBSTRING rather than REPLACE to avoid accidentally removing copies of any of the suffixes from the middle of the string.
The #Suffixes table is a table variable here, but it makes more sense for you to just create it and fill it as a permanent table.
The query will just find the one row (if any) that matches its suffix with the end of your string. If a match is found then the variable will be set to a substring with the length of the suffix removed from the end. There will usually be a trailing space, but for a VARCHAR that will just get dropped off.
There are still a couple of potential issues to be aware of though...
First, if you have a company name like "Watco" then the "co" would be a false positive here. I'm not sure what can be done about that other than maybe making your suffixes include a leading space.
Second, if one suffix ends with one of your other suffixes then the ordering that they get applied could be a problem. You could get around this by only applying the row with the greatest length for suffix, but it gets a little more complicated, so I've left that out for now.
Building on the answer given by Tom H, but applying across the entire table:
set nocount on;
declare #suffixes table(tag nvarchar(20));
insert into #suffixes values('co');
insert into #suffixes values('corp');
insert into #suffixes values('corporation');
insert into #suffixes values('company');
insert into #suffixes values('lp');
insert into #suffixes values('llc');
insert into #suffixes values('ltd');
insert into #suffixes values('limited');
declare #companynames table(entry nvarchar(100),processed bit default 0);
insert into #companynames values('somecompany llc',0);
insert into #companynames values('business2 co',0);
insert into #companynames values('business3',0);
insert into #companynames values('business4 lpx',0);
while exists(select * from #companynames where processed = 0)
begin
declare #currentcompanyname nvarchar(100) = (select top 1 entry from #companynames where processed = 0);
update #companynames set processed = 1 where entry = #currentcompanyname;
update #companynames
set entry = SUBSTRING(entry, 1, LEN(entry) - LEN(tag))
from #suffixes
where entry like '%' + tag
end
select * from #companynames
You can use a query like below:
-- Assuming that you can maintain all patterns in a table or a temp table
CREATE TABLE tbl(pattern varchar(100))
INSERT INTO tbl values
('co'),('llc'),('beta')
--#a stores the string you need to manipulate, #lw & #b are variables to aid
DECLARE #a nvarchar(100), #b nvarchar(100), #lw varchar(100)
SET #a='alpha beta gamma'
SET #b=''
-- #t is a flag
DECLARE #t int
SET #t=0
-- Below is a loop
WHILE(#t=0 OR LEN(#a)=0 )
BEGIN
-- Store the current last word in the #lw variable
SET #lw=reverse(substring(reverse(#a),1, charindex(' ', reverse(#a)) -1))
-- check if the word is in pattern dictionary. If yes, then Voila!
SELECT #t=1 FROM tbl WHERE #lw like pattern
-- remove the last word from #a
SET #a=LEFT(#a,LEN(#a)-LEN(#lw))
IF (#t<>1)
BEGIN
-- all words which were not pattern are joined back onto this stack
SET #b=CONCAT(#lw,#b)
END
END
-- get back the remaining word
SET #a=CONCAT(#a,#b)
SELECT #a
drop table tbl
Do note that this method overcomes Tom's problem of
if you have a company name like "Watco" then the "co" would be a false positive here. I'm not sure what can be done about that other than maybe making your suffixes include a leading space.
use the replace function in SQL 2012,
declare #var1 nvarchar(20) = 'ACME LLC'
declare #var2 nvarchar(20) = 'LLC'
SELECT CASE
WHEN ((PATINDEX('%'+#var2+'%',#var1) <= (LEN(#var1)-LEN(#var2)))
Or (SUBSTRING(#var1,PATINDEX('%'+#var2+'%',#var1)-1,1) <> SPACE(1)))
THEN #var1
ELSE
REPLACE(#var1,#var2,'')
END
Here is another way to overcome the 'Runco Co' situation.
declare #var1 nvarchar(20) = REVERSE('Runco Co')
declare #var2 nvarchar(20) = REVERSE('Co')
Select REVERSE(
CASE WHEN(CHARINDEX(' ',#var1) > LEN(#var2)) THEN
SUBSTRING(#var1,PATINDEX('%'+#var2+'%',#var1)+LEN(#var2),LEN(#var1)-LEN(#var2))
ELSE
#var1
END
)

comparable varchar "arrays" in seperate fields but on same row

I have a table that looks like this:
memberno(int)|member_mouth (varchar)|Inspected_Date (varchar)
-----------------------------------------------------------------------------
12 |'1;2;3;4;5;6;7' |'12-01-01;12-02-02;12-03-03' [7 members]
So by looking at how this table has been structured (poorly yes)
The values in the member_mouth field is a string that is delimited by a ";"
The values in the Inspected_Date field is a string that is delimited by a ";"
So - for each delimited value in member_mouth there is an equal inspected_date value delimited inside the string
This table has about 4Mil records, we have an application written in C# that normalizes the data and stores it in a separate table. The problem now is because of the size of the table it takes a long time for this to process. (the example above is nothing compared to the actual table, it's much larger and has a couple of those string "array" fields)
My question is this: What would be the best and fastest way to normilize this data in MSSQL proc? let MSSQL do the work and not a C# app?
The best way will be SQL itself. The way followed in the below code is something which worked for me well with 2-3 lakhs of data.
I am not sure about the below code when it comes to 4 Million, but may help.
Declare #table table
(memberno int, member_mouth varchar(100),Inspected_Date varchar(400))
Insert into #table Values
(12,'1;2;3;4;5;6;7','12-01-01;12-02-02;12-03-03;12-04-04;12-05-05;12-07-07;12-08-08'),
(14,'1','12-01-01'),
(19,'1;5;8;9;10;11;19','12-01-01;12-02-02;12-03-03;12-04-04;12-07-07;12-10-10;12-12-12')
Declare #tableDest table
(memberno int, member_mouth varchar(100),Inspected_Date varchar(400))
The table will be like.
Select * from #table
See the code from here.
------------------------------------------
Declare #max_len int,
#count int = 1
Set #max_len = (Select max(Len(member_mouth) - len(Replace(member_mouth,';','')) + 1)
From #table)
While #count <= #max_len
begin
Insert into #tableDest
Select memberno,
SUBSTRING(member_mouth,1,charindex(';',member_mouth)-1),
SUBSTRING(Inspected_Date,1,charindex(';',Inspected_Date)-1)
from #table
Where charindex(';',member_mouth) > 0
union
Select memberno,
member_mouth,
Inspected_Date
from #table
Where charindex(';',member_mouth) = 0
Delete from #table
Where charindex(';',member_mouth) = 0
Update #table
Set member_mouth = SUBSTRING(member_mouth,charindex(';',member_mouth)+1,len(member_mouth)),
Inspected_Date = SUBSTRING(Inspected_Date,charindex(';',Inspected_Date)+1,len(Inspected_Date))
Where charindex(';',member_mouth) > 0
Set #count = #count + 1
End
------------------------------------------
Select *
from #tableDest
Order By memberno
------------------------------------------
Result.
You can take a reference here.
Splitting delimited values in a SQL column into multiple rows
Do it on SQl server side, if possible a SSIS package would be great.

How to pass multiple values to single parameter in stored procedure

I'm using SSRS for reporting and executing a stored procedure to generate the data for my reports
DECLARE #return_value int
EXEC #return_value = [dbo].[MYREPORT]
#ComparePeriod = 'Daily',
#OverrideCompareDate = NULL,
#PortfolioId = '5,6',
#OverrideStartDate = NULL,
#NewPositionsOnly = NULL,
#SourceID = 13
SELECT 'Return Value' = #return_value
GO
In the above when I passed #PortfolioId = '5,6' it is giving me wrong inputs
I need all records for portfolio id 5 and 6 also is this correct way to send the multiple values ?
When I execute my reports only giving #PortfolioId = '5' it is giving me 120 records
and when I execute it by giving #PortfolioId = '6' it is giving me 70 records
So when I will give #PortfolioId = '5,6' it should have to give me only 190 records altogether, but it is giving me more no of records I don't understand where I exactly go wrong .
Could anyone help me?
thanks
all code is too huge to paste , i'm pasting relevant code please suggest clue.
CREATE PROCEDURE [dbo].[GENERATE_REPORT]
(
#ComparePeriod VARCHAR(10),
#OverrideCompareDate DATETIME,
#PortfolioId VARCHAR(50) = '2', --this must be multiple
#OverrideStartDate DATETIME = NULL,
#NewPositionsOnly BIT = 0,
#SourceID INT = NULL
) AS
BEGIN
SELECT
Position.Date,
Position.SecurityId,
Position.Level1Industry,
Position.MoodyFacilityRating,
Position.SPFacilityRating,
Position.CompositeFacilityRating,
Position.SecurityType,
Position.FacilityType,
Position.Position
FROM
Fireball_Reporting.dbo.Reporting_DailyNAV_Pricing POSITION WITH (NOLOCK, READUNCOMMITTED)
LEFT JOIN Fireball.dbo.AdditionalSecurityPrice ClosingPrice WITH (NOLOCK, READUNCOMMITTED) ON
ClosingPrice.SecurityID = Position.PricingSecurityID AND
ClosingPrice.Date = Position.Date AND
ClosingPrice.SecurityPriceSourceID = #SourceID AND
ClosingPrice.PortfolioID IN (
SELECT
PARAM
FROM
Fireball_Reporting.dbo.ParseMultiValuedParameter(#PortfolioId, ',') )
This can not be done easily. There's no way to make an NVARCHAR parameter take "more than one value". What I've done before is - as you do already - make the parameter value like a list with comma-separated values. Then, split this string up into its parts in the stored procedure.
Splitting up can be done using string functions. Add every part to a temporary table. Pseudo-code for this could be:
CREATE TABLE #TempTable (ID INT)
WHILE LEN(#PortfolioID) > 0
BEGIN
IF NOT <#PortfolioID contains Comma>
BEGIN
INSERT INTO #TempTable VALUES CAST(#PortfolioID as INT)
SET #PortfolioID = ''
END ELSE
BEGIN
INSERT INTO #Temptable VALUES CAST(<Part until next comma> AS INT)
SET #PortfolioID = <Everything after the next comma>
END
END
Then, change your condition to
WHERE PortfolioId IN (SELECT ID FROM #TempTable)
EDIT
You may be interested in the documentation for multi value parameters in SSRS, which states:
You can define a multivalue parameter for any report parameter that
you create. However, if you want to pass multiple parameter values
back to a data source by using the query, the following requirements
must be satisfied:
The data source must be SQL Server, Oracle, Analysis Services, SAP BI
NetWeaver, or Hyperion Essbase.
The data source cannot be a stored procedure. Reporting Services does
not support passing a multivalue parameter array to a stored
procedure.
The query must use an IN clause to specify the parameter.
This I found here.
I spent time finding a proper way. This may be useful for others.
Create a UDF and refer in the query -
http://www.geekzilla.co.uk/view5C09B52C-4600-4B66-9DD7-DCE840D64CBD.htm
USE THIS
I have had this exact issue for almost 2 weeks, extremely frustrating but I FINALLY found this site and it was a clear walk-through of what to do.
http://blog.summitcloud.com/2010/01/multivalue-parameters-with-stored-procedures-in-ssrs-sql/
I hope this helps people because it was exactly what I was looking for
Either use a User Defined Table
Or you can use CSV by defining your own CSV function as per This Post.
I'd probably recommend the second method, as your stored proc is already written in the correct format and you'll find it handy later on if you need to do this down the road.
Cheers!
I think, below procedure help you to what you are looking for.
CREATE PROCEDURE [dbo].[FindEmployeeRecord]
#EmployeeID nvarchar(Max)
AS
BEGIN
DECLARE #sqLQuery VARCHAR(MAX)
Declare #AnswersTempTable Table
(
EmpId int,
EmployeeName nvarchar (250),
EmployeeAddress nvarchar (250),
PostalCode nvarchar (50),
TelephoneNo nvarchar (50),
Email nvarchar (250),
status nvarchar (50),
Sex nvarchar (50)
)
Set #sqlQuery =
'select e.EmpId,e.EmployeeName,e.Email,e.Sex,ed.EmployeeAddress,ed.PostalCode,ed.TelephoneNo,ed.status
from Employee e
join EmployeeDetail ed on e.Empid = ed.iEmpID
where Convert(nvarchar(Max),e.EmpId) in ('+#EmployeeId+')
order by EmpId'
Insert into #AnswersTempTable
exec (#sqlQuery)
select * from #AnswersTempTable
END

STORED PROCEDURE Calculations & performance improvements

I currently have the following stored procedure;
CREATE PROCEDURE web.insertNewCampaign
(
#tmp_Id BIGINT,
#tmp_Title VARCHAR(100),
#tmp_Content VARCHAR(8000),
#tmp_Pledge DECIMAL(7,2),
--#tmp_Recipients BIGINT,
#tmp_Date DATETIME,
#tmp_Private BIT,
#tmp_Template BIGINT,
#tmp_AddyBook BIGINT
)
AS
declare #recipients BIGINT
declare #tmp_IDENTITY BIGINT
declare #fave BIGINT
declare #allocation VARCHAR(50)
--insert campaign data
BEGIN TRAN
SELECT #recipients = addMaster_NoRecipients FROM tbl_AddressBookMaster
WHERE addMaster_UserId = #tmp_Id AND addMaster_Key = #tmp_AddyBook;
INSERT INTO TBL_CAMPAIGNS ([campaign_MemberId], [campaign_Title], [campaign_Content], [campaign_Pledge], [campaign_Date], [campaign_Private], [campaign_Template], [campaign_AddressBook], [campaign_Recipients])
VALUES (#tmp_Id, #tmp_Title, #tmp_Content, #tmp_Pledge, #tmp_Date, #tmp_Private, #tmp_Template, #tmp_AddyBook, #recipients)
SELECT #tmp_IDENTITY = SCOPE_IDENTITY() --this returns the newly added IDENTITY ID
COMMIT
......
So i have 2 questions:
1) How do i divide #tmp_Pledge by #recipients to give #allocation eg:(#allocation = #tmp_Pledge / #recipients)
2) Is it possible to compound these statements into a more efficient statement(s) with #allocation effectively being inserted as a value into the column [campaign_RecipShare], and reducing the need for these declared variables?
Many Thanks for any help you can offer for either question.
;-)
After the first select, you can do this to set #allocation:
set #allocation = #tmp_pledge / #recepients
As for making it more efficient, it's already fairly efficient--you won't go through any less steps, but you can condense the code a bit:
INSERT INTO TBL_CAMPAIGNS (
[campaign_MemberId], [campaign_Title], [campaign_Content],
[campaign_Pledge], [campaign_Date], [campaign_Private],
[campaign_Template], [campaign_AddressBook], [campaign_Recipients],
[capmain_RecipShare])
SELECT
#tmp_Id, #tmp_Title, #tmp_Content,
#tmp_Pledge, #tmp_Date, #tmp_Private,
#tmp_Template, #tmp_AddyBook, addMaster_NoRecipients,
#tmp_Pledge / addMaster_NoReceipients as Allocation
FROM
tbl_AddressBookMaster
WHERE
addMaster_UserId = #tmp_Id
AND addMaster_Key = #tmp_AddyBook
SELECT #tmp_IDENTITY = SCOPE_IDENTITY() --this returns the newly added IDENTITY ID
This also removes the need for you calculating the #allocation member outside of the insert statement.
1) #tmp_pledge / #recepients - I'll assume allocation is a numeric field of some form in TBL_CAMPAIGNS holding a number in varchar is not a good idea.
2) You just need to build a select that returns all the values from the other table and the parameters matching the columns to insert into.
insert into TBL_CAMPAIGNS ([campaign_MemberId], [campaign_Title], [campaign_Content], [campaign_Pledge], [campaign_Date], [campaign_Private], [campaign_Template], [campaign_AddressBook], [campaign_Recipients], [campaign_allocation)
select #tmp_Id, #tmp_Title, #tmp_Content, #tmp_Pledge, #tmp_Date, #tmp_Private, #tmp_Template, #tmp_AddyBook, addMaster_NoRecipients, #tmp_pledge / addMaster_NoRecipients
FROM FROM tbl_AddressBookMaster
WHERE addMaster_UserId = #tmp_Id AND addMaster_Key = #tmp_AddyBook;
SELECT #tmp_IDENTITY = SCOPE_IDENTITY() --this returns the newly added IDENTITY ID
set #allocation = #tmp_pledge / (#recepients* 1.0)
You want to do that because othewise you will run into integer math and the result will round to an integer.