Code
CREATE TYPE [dbo].[IntListType] AS TABLE([Number] [int] NULL)
DECLARE #ids as IntListType;
DECLARE #TempIds as IntListType;
Situation
I have a long long list of products which need to be filtered in a stored procedure. The filter is given as a User-Defined Table Type and each filter can be empty or full. I've attempted to create a single query for this, but it won't work if one of the filters is empty.
So I decided to create another User-Defined Table Type which only contains the column: Number.
Then, for each Filter type it will check wether it's empty
IF (select filterName from #customtable) is not null
and if so, it will query the product Id's which match that name and add it to the list of numbers.
If the next filter isn't emmpty, it will add the id's to a temp table, which in turn will be combined with the first table.
So far, I found this to be the most effective way to handle whether a filter is null or not. In any way it will get the id's I need!
Now, every filter will eventually perfrom the following code:
IF (select count(*) from #ids) > 0
BEGIN
IF(select count(*) from #TempIds)
BEGIN
delete from #ids
where Number not in
(select C.Number
from #ids C
inner join #TempIds T on C.Number = T.Number)
END
END
ELSE
BEGIN
insert into #ids
select * from #TempIds
END
delete from #TempIds
Problem
I'm looking for a way to implement the last named code into a stored procedure which gives me back a list of IntListType. However, every implementation I have of #ids (declare #ids as IntListType) can't be set. (asinset #ids = #TempIds' > Will complain about 'Must declare the scalar variable #ids' aswell as '#TempIds')
How can I fix this?
Attempts
I've already attempted to skip this all by combining every table with inner join, outer join and other joins.. but in no way will I get the same result when any (or all) of the filters is empty.
As far as I can see, this is the only way to effectively progress every filter type, check if they are null or not, if not; process the results etc etc
In the part of the query you shared, there is no declaration of #ids or #TempIds.
Can you add that part?
Apart from that, I think you should not be looking at a User-Defined Type: Can T-SQL function return user-defined table type?
Have a look at User-Defined Functions:
http://www.sqlteam.com/article/user-defined-functions
http://blog.sqlauthority.com/2007/05/29/sql-server-user-defined-functions-udf-limitations/
tsql returning a table from a function or store procedure
Related
I am having serious performance issues when using a nested loop in a WHERE clause.
When I run the below code as is, it takes several minutes. The trick is I'm using the WHERE clause to pull ALL data if the report_id is NULL, but only certain report_id's if I set them in the parameter string.
The function [fn_Parse_List] turns a VARCHAR string such as '123,456,789' into a table where each row is each number in integer form, which is then used in the IN clause.
When I run the code below with report_id = '456' (the dashed out portion), the code takes seconds, but passing the temporary table and using the SELECT statement in the WHERE clause kills it.
alter procedure dbo.p_revenue
(#report_id varchar(max) = NULL)
as
select cast(value as int) Report_ID
into #report_ID_Temp
from [fn_Parse_List] (#report_id)
SELECT *
FROM BIGTABLE
where #report_id is null
or a.report_id in (select Report_ID from #report_ID_Temp)
--Where #report_id is null or a.report_id in (456)
exec p_revenue #report_id = '456'
Is there a way to optimize this? I tried a JOIN with the table #report_ID_Temp, but it still takes just as long and doesn't work when the report_id is NULL.
You're breaking three different rules.
If you want two query plans, you need two queries: OR does not give you two query plans. IF does.
If you have a temporary table, make sure it has a primary key and any appropriate indexes. In your case, you need an ALTER TABLE statement to add the primary key clustered index. Or you can CREATE TABLE to declare the structure in the first place.
If you think fn_Parse_List is a good idea, you haven't read enough Sommarskog
If I were to write the Stored Procedure for your case, I would use a Table Valued Parameter (TVP) instead of passing multiple values as a comma-seperated string.
Something like the following:
-- Create a type for the TVP
CREATE TYPE REPORT_IDS_PAR AS TABLE(
report_id INT
);
GO
-- Use the TVP type instead of VARCHAR
CREATE PROCEDURE dbo.revenue
#report_ids REPORT_IDS_PAR READONLY
AS
BEGIN
SET NOCOUNT ON;
IF NOT EXISTS(SELECT 1 FROM #report_ids)
SELECT
*
FROM
BIGTABLE;
ELSE
SELECT
*
FROM
#report_ids AS ids
INNER JOIN BIGTABLE AS bt ON
bt.report_id=ids.report_id;
-- OPTION(RECOMPILE) -- see remark below
END
GO
-- Execute the Stored Procedure
DECLARE #ids REPORT_IDS_PAR;
-- Empty table for all rows:
EXEC dbo.revenue #ids;
-- Specific report_id's for specific rows:
INSERT INTO #ids(report_id)VALUES(123),(456),(789);
EXEC dbo.revenue #ids;
GO
If you run this procedure with a TVP with a lot of rows or a wildly varying number of rows, I suggest you add the option OPTION(RECOMPILE) to the query.
I see 2 possible things that could help improve performance. Depends on which part is taking the longest. First off, SELECT INTO is a single threaded operation until SQL Server 2014. If this is taking a long time, create an explicitly defined temp table with CREATE TABLE. Secondly, depending on the number of records inserted into the temp table, you probably need an index on the Report_ID column. That can all be done in the body of the stored procedure. If you do end up using an explicitly defined temp table, I would create the index after the data is loaded.
If that doesn't help, first check that the report_id column on the BIGTABLE is indexed. Then try splitting the select into 2 and combining with a UNION ALL like this:
ALTER PROCEDURE dbo.p_revenue
(
#report_id VARCHAR(MAX) = NULL
)
AS
SELECT CAST(value AS INT) Report_ID
INTO #report_ID_Temp
FROM fn_Parse_List(#report_id);
SELECT *
FROM BIGTABLE
WHERE #report_id IS NULL
UNION ALL
SELECT *
FROM BIGTABLE
WHERE a.report_id IN ( SELECT Report_ID
FROM #report_ID_Temp );
GO
EXEC p_revenue #report_id = '456';
Are you saying I should have two queries, one where it pulls if the report_id doesn't exists and one where there is a list of report_ids?
Yes, yes, yes. The fact, that it somehow works when You enter the numbers directly, distracts You from the core problem. You need table scan when #report_id is null and index seek when it is not and You can not have both in one execution plan. The performance would inevitably have to suffer, one way or another.
I would prefer not to, as the table i'm pulling from is actually a
view with 800 lines with an additional parameter not shown above.
I do not see where is the problem, SELECT * FROM BIGTABLE and SELECT * FROM BIGVIEW seems the same. If You need parameters You can use inline table valued function. If You have more parameters with variable selectivity like #report_id, I guess You would end up with dynamic sql anyway, sooner or later.
UNION ALL as proposed by #db_brad would help, but one of those subquery is executed even when there is no need for it.
As a quick patch You can append OPTION(RECOMPILE) to the SELECT and have table scan one time and index seek the other time, but recompiling every time would induce nontrivial overhead.
I want to create a column based on COUNT(*) on another table, and when a record is deleted from that table it should decrease the value in this new column and vice versa. So, here is the query:
SELECT COUNT (*) FROM dbo.Korisnik1_FakturaStavka GROUP BY dbo.Korisnik1_FakturaStavka.FakturaID
And it returns this:
And when I try to create a computated column like this:
CREATE TABLE test(
NumberOF as (SELECT COUNT (*) FROM dbo.Korisnik1_FakturaStavka GROUP BY dbo.Korisnik1_FakturaStavka.FakturaID) )
I get the following error:
Subqueries are not allowed in this context. Only scalar expressions are allowed.
Here is the main table that I want to compute from:
How can I resolve this ?
You can define a UDF:
create function dbo.NumberOfFakturaID(#id int) returns int as begin
return (select count(1) from Korisnik1_FakturaStavka where id=#id)
end
and then use it as the computed column:
CREATE TABLE test(FakturaID int, NumberOF as dbo.NumberOfFakturaID(FakturaID))
But putting that sort of calc as a computed column should be used with care.
This is too long for a comment.
You can do this by defining a function to calculate the count and using that function in the computed column definition. However, I don't think this is a good idea for frequently used columns, because you will be doing a lot of counting "behind the scenes".
Alternatives:
Set up a view or materialized view with the additional count column.
Do the count explicitly when you need it.
Set up a trigger to store the count in the first table, whenever rows are inserted/updated/deleted from the second table.
So I have a table which has a bunch of information and a bunch of records. But there will be one field in particular I care about, in this case #BegAttField# where only a subset of records have it populated. Many of them have the same value as one another as well.
What I need to do is get a count (minus 1) of all duplicates, then populate the first record in the bunch with that count value in a new field. I have another field I call BegProd that will match #BegAttField# for each "first" record.
I'm just stuck as to how to make this happen. I may have been on the right path, but who knows. The SELECT statement gets me two fields and as many records as their are unique #BegAttField#'s. But once I have them, I haven't been able to work with them.
Here's my whole set of code, trying to use a temporary table and SELECT INTO to try and populate it. (Note: the fields with # around the names are variables for this 3rd party app)
CREATE TABLE #temp (AttCount int, BegProd varchar(255))
SELECT COUNT(d.[#BegAttField#])-1 AS AttCount, d.[#BegAttField#] AS BegProd
INTO [#temp] FROM [Document] d
WHERE d.[#BegAttField#] IS NOT NULL GROUP BY [#BegAttField#]
UPDATE [Document] d SET d.[#NumAttach#] =
SELECT t.[AttCount] FROM [#temp] t INNER JOIN [Document] d1
WHERE t.[BegProd] = d1.[#BegAttField#]
DROP TABLE #temp
Unfortunately I'm running this script through a 3rd party database application that uses SQL as its back-end. So the errors I get are simply: "There is already an object named '#temp' in the database. Incorrect syntax near the keyword 'WHERE'. "
Comment out the CREATE TABLE statement. The SELECT INTO creates that #temp table.
I have a T-SQL routine that copies user information from one table 'Radius' to another 'Tags'. However, as the rows are transfered, I would also like to include a unique randomly generated code in the INSERT (3 chars long). The code is generated by the WHILE loop below. Any way to do this?
INSERT Tags (UserID, JobID, Code)
SELECT UserID, #JobID, ?????
FROM Radius
Unique random code generator:
WHILE EXISTS (SELECT * FROM Tags WHERE Code = #code)
BEGIN
select #code=#code+char(n) from
(
select top 3 number as n from master..spt_values
where type='p' and number between 48 and 57 or number between 65 and 90
order by newid()
)
END
CLARIFICATION: The reason for doing this is that I want to keep the random code generation logic at the level of the SQL stack. Implementing this in the app code would require me to check the db everytime a potential random code is generated to see if it is unique. As the number of code records increases so will the number of calls to the db as probability increases that there will be more duplicate codes generated before a unique one is generated.
Part One, Generate a table with all possible values
DECLARE #i int
CREATE TABLE #AllChars(value CHAR(1))
SET #i=48
WHILE #i<=57
BEGIN
INSERT INTO #Allchars(value) VALUES(CHAR(#i))
SET #i=#i+1
END
SET #i=65
WHILE #i<=90
BEGIN
INSERT INTO #Allchars(value) VALUES(CHAR(#i))
SET #i=#i+1
END
CREATE TABLE AllCodes(value CHAR(3),
CONSTRAINT PK_AllChars PRIMARY KEY CLUSTERED(value))
INSERT INTO AllCodes(value)
SELECT AllChars1.Value+AllChars2.Value+AllChars3.Value
FROM #AllChars AS AllChars1,#AllChars AS AllChars2,#AllChars AS AllChars3
This is a one off operation and takes around 1 second to run on SQL Azure. Now that you have all possible values in a table any future inserts become, something along the lines of
SELECT
RadiusTable.UserID,
RadiusTable.JobID,
IDTable.Value
FROM
(
SELECT ROW_NUMBER() OVER (ORDER BY UserID,JobID) As RadiusRow,
UserID,JobID
FROM Radius
) AS RadiusTable INNER JOIN
(
SELECT ROW_NUMBER() OVER (ORDER BY newID()) As IDRow,
Value
FROM AllCodes
) AS IDTable ON RadiusTable.RadiusRow = IDTable.IDRow
Before going with any of these schemes you had better be certain that you are not going to have more than 46656 rows in your table otherwise you will run out of unique ID Values.
I do not know if this is possible and suitable for your situation, but to me it seems that a scalar-valued function would be a solution.
Well, let me start over then.
This seems kind of ugly but it might work: newid() inside sql server function
The accepted answer that is.
Ah, been there done that too. The problem with this is that I am using T-SQL Stored Procedures that are called by Asp.net Where would I put the CREATE VIEW statement? I can't add it to the function file.
I'm phrasing the question title poorly as I'm not sure what to call what I'm trying to do but it really should be simple.
I've a link / join table with two ID columns. I want to run a check before saving new rows to the table.
The user can save attributes through a webpage but I need to check that the same combination doesn't exist before saving it. With one record it's easy as obviously you just check if that attributeId is already in the table, if it is don't allow them to save it again.
However, if the user chooses a combination of that attribute and another one then they should be allowed to save it.
Here's an image of what I mean:
So if a user now tried to save an attribute with ID of 1 it will stop them, but I need it to also stop them if they tried ID's of 1, 10 so long as both 1 and 10 had the same productAttributeId.
I'm confusing this in my explanation but I'm hoping the image will clarify what I need to do.
This should be simple so I presume I'm missing something.
If I understand the question properly, you want to prevent the combination of AttributeId and ProductAttributeId from being reused. If that's the case, simply make them a combined primary key, which is by nature UNIQUE.
If that's not feasible, create a stored procedure that runs a query against the join for instances of the AttributeId. If the query returns 0 instances, insert the row.
Here's some light code to present the idea (may need to be modified to work with your database):
SELECT COUNT(1) FROM MyJoinTable WHERE AttributeId = #RequestedID
IF ##ROWCOUNT = 0
BEGIN
INSERT INTO MyJoinTable ...
END
You can control your inserts via a stored procedure. My understanding is that
users can select a combination of Attributes, such as
just 1
1 and 10 together
1,4,5,10 (4 attributes)
These need to enter the table as a single "batch" against a (new?) productAttributeId
So if (1,10) was chosen, this needs to be blocked because 1-2 and 10-2 already exist.
What I suggest
The stored procedure should take the attributes as a single list, e.g. '1,2,3' (comma separated, no spaces, just integers)
You can then use a string splitting UDF or an inline XML trick (as shown below) to break it into rows of a derived table.
Test table
create table attrib (attributeid int, productattributeid int)
insert attrib select 1,1
insert attrib select 1,2
insert attrib select 10,2
Here I use a variable, but you can incorporate as a SP input param
declare #t nvarchar(max) set #t = '1,2,10'
select top(1)
t.productattributeid,
count(t.productattributeid) count_attrib,
count(*) over () count_input
from (select convert(xml,'<a>' + replace(#t,',','</a><a>') + '</a>') x) x
cross apply x.x.nodes('a') n(c)
cross apply (select n.c.value('.','int')) a(attributeid)
left join attrib t on t.attributeid = a.attributeid
group by t.productattributeid
order by countrows desc
Output
productattributeid count_attrib count_input
2 2 3
The 1st column gives you the productattributeid that has the most matches
The 2nd column gives you how many attributes were matched using the same productattributeid
The 3rd column is how many attributes exist in the input
If you compare the last 2 columns and the counts
match - you can use the productattributeid to attach to the product which has all these attributes
don't match - then you need to do an insert to create a new combination