I am attempting to convert a SQL stored procedure to LINQ to do some performance testing (trying to figure out if using LINQ in some of our methods will speed things up at all)
I am fairly new to doing anything in LINQ so I am just modifying examples in books / online to suit my needs and am stuck on something.
I have this code so far:
DIM TicketID as INTEGER = 1
DIM s =
FROM User in PersonnelTbl
WHERE !(from t in tblSupportTicketNotifications where t.TicketID = TicketID select t.EmployeeID).Contains(User.EmployeeID)
Select user
Im not sure why but I get an Identifier expected error message on the Where clause line. Can anyone point me in the right direction? Cheers
I've based my code so far on this example:
var query =
from c in dc.Customers
where !(from o in dc.Orders
select o.CustomerID)
.Contains(c.CustomerID)
select c;
This is what I am trying to convert
CREATE PROCEDURE spGetEmployeesToBeNotified
(
#TicketID INT
)
AS
BEGIN
SELECT
ID,
FirstName,
Surname,
FirstName+' '+Surname As FullName,
WorkEmail,
0 AS Checked
FROM
PersonnelTbl
WHERE
ID NOT IN(SELECT EmployeeID FROM tblSupportTicketNotifications WHERE TicketID = #TicketID)
AND
(FirstName IS NOT NULL
AND
FirstName <> ''
AND
Surname IS NOT NULL
AND
Surname <> '')
UNION
SELECT
person.ID,
person.FirstName,
person.Surname,
person.FirstName +' '+person.Surname As FullName,
person.WorkEmail,
1 AS Checked
FROM
PersonnelTbl person
JOIN
tblSupportTicketNotifications notify
ON
person.ID = notify.EmployeeID
WHERE
TicketID = #TicketID
ORDER BY
FirstName ASC,
Surname ASC
END
Assuming you have an association between User and SupportTicketNotifications, you could try the following which should use an Exists clause rather than In. You can then profile the differences in SQL to see which one actually works faster (or if the SQL engine optimizes them to the same things.)
DIM TicketID as INTEGER = 1
DIM s =
FROM User in PersonnelTbl
WHERE Not User.SupportTicketNotifications.Any(Function(t) t.TicketID = TicketID)
' The Select is optional in VB if you are just returning the item you are selecting.
Regarding performance with LINQ to SQL as compared to raw ADO, you may want to check out http://blogs.msdn.com/ricom/archive/2007/06/22/dlinq-linq-to-sql-performance-part-1.aspx. As you are learning LINQ, you should make an effort to profile what you are doing in any regard to help you learn what's happening and where you need to make performance improvements (including using Stored Procs/custom ADO where necessary).
Linq-2-sql will NEVER outperform a stored procedure.
In the end, all what Linq-2-sql does is give you a more object oriented approach but in the end it IS SQL that is being send to the database. So if you put the same SQL in the stored procedure it by definition is at least equally fast.
! is a C# operator. You want "Not" instead for VB.Net.
Related
I am using XrmToolBox with SQL 4 CDS to make some edits to a user in MS Dynamics. I don't quite know where the error lies-- in the SQL expression? In the FetchXML that it gets converted to? Anyway, here's my expression:
INSERT INTO systemuser (
firstname,
lastname,
internalemailaddress,
departmentid,
internalspecialtyid)
VALUES (
'John',
'Smith',
'john.smith#example.com',
(SELECT TOP 1 departmentid FROM department WHERE name = 'Commercial'),
(SELECT TOP 1 internalspecialtyid FROM internal_specialties WHERE name = 'B2B Comms'));
When I run this, I get this message from XrmToolBox:
Unhandled expression type: (SELECT TOP 1 departmentid FROM department WHERE name = 'Commercial')
I really don't understand this. I can select the subquery and execute it to get one response back. I can take its results and make it a static (non-subquery) expression with no problem. But why isn't this working? I'm pretty new to SQL, assuming that's where the problem is, so I could easily misunderstand the subquery syntax.
The SQL 4 CDS tool attempts to provide as much a SQL implementation for CDS/Dataverse as possible but it's not perfect. Reach out to the tool author directly by opening an issue on the the SQL 4 CDS's GitHub page: https://github.com/MarkMpn/Sql4Cds/issues
So I'm creating a query for a report that could have several optional filters. I've only included client and station here to keep it simple. Each of these options could be an include or an exclude and could contain NULL, 1, or multiple values. So I split the varchar into a table before joining it to the query.
This test takes about 15 minutes to execute, which... just won't do :p Is there a better way? We have similar queries written with dynamic sql, and I was trying to avoid that, but maybe there's no way around it for this?
DECLARE
#ClientsInc VARCHAR(10) = 'ABCD, EFGH',
#ClientsExc VARCHAR(10) = NULL,
#StationsInc VARCHAR(10) = NULL,
#StationsExc VARCHAR(10) = 'SomeStation'
SELECT *
INTO #ClientsInc
FROM dbo.StringSplit(#ClientsInc, ',')
SELECT *
INTO #ClientsExc
FROM dbo.StringSplit(#ClientsExc, ',')
SELECT *
INTO #StationsInc
FROM dbo.StringSplit(#StationsInc, ',')
SELECT *
INTO #StationsExc
FROM dbo.StringSplit(#StationsExc, ',')
SELECT [some stuff]
FROM media_order mo
LEFT JOIN #ClientsInc cInc WITH(NOLOCK) ON cInc.Value = mo.client_code
LEFT JOIN #ClientsExc cExc WITH(NOLOCK) ON cExc.Value = mo.client_code
LEFT JOIN #StationsInc sInc WITH(NOLOCK) ON sInc.Value = mo.station_name
LEFT JOIN #StationsExc sExc WITH(NOLOCK) ON sExc.Value = mo.station_name
WHERE ((#ClientsInc IS NOT NULL AND cInc.Value IS NOT NULL)
OR (#ClientsExc IS NOT NULL AND cExc.Value IS NULL)
)
AND ((#StationsInc IS NOT NULL AND sInc.Value IS NOT NULL)
OR (#StationsExc IS NOT NULL AND sExc.Value IS NULL)
)
First of all, I always tend to mention Erland Sommarskog's Dynamic Search Conditions in such cases.
However, you already seem to be aware of the two options: one is dynamic SQL. The other is usually the old trick and (#var is null or #var=respective_column). This trick, however, works only for one value per variable.
Your solution indeed seems to work for multiple values. But in my opinion, you are trying too hard to avoid dynamic sql. Your requirements are complex enough to guarantee it. And remember, usually, dynamic sql is harder for you to code, but easier for the server in complex cases - and this one certainly is. Making a performance guess is always risky, but I would guess an improvement in this case.
I would use exists and not exists:
select ...
from media_order mo
where
(
#ClientsInc is null
or exists (
select 1
from string_split(#ClientsInc, ',')
where value = mo.client_code
)
)
and not exist (
select 1
from string_split(#ClientsExc, ',')
where value = mo.client_code
)
and (
#StationsInc is null
or exists (
select 1
from string_split(#StationsInc, ',')
where value = mo.station_name
)
)
and not exist (
select 1
from string_split(#StationsExc, ',')
where value = mo.station_name
)
Notes:
I used buil-in function string_split() rather than the custom splitter that you seem to be using. It is available in SQL Server 2016 and higher, and returns a single column called value. You can change that back to your customer function if you are running an earlier version
as I understand the logic you want, "include" parameters need to be checked for nullness before using exists, while it is unnecessary for "exclude" variables
I have a Table which has 3 different Select queries.
e.g.
Staff
------------------------------------------------------------
ID Code Name Phone DOB Email Addr1 Addr2 Addr3
Query1
Select ID, Code, Phone From Staff Where Code = 'ABC'
Query2
Select ID, Code, Phone From Staff Where Name = 'ABCXYZ' And Code = 'B'
Query3
Select ID, Code, Phone From Staff Where Phone= '1234' And Email = 'a#b'
These 3 queries are there in 3 different stored procedure.
I want to find the names of Attributes which i have used in all my Where clauses. But want to Automate this as I have more than 100 tables to look for
Something like this
exec fxGetWhereColList ('Staff');
Result:
Code
Name
Phone
Email
I agree with all posts before, it's a complex case.
I post you a small idea to help you in your work.
If you can retrieve each execution plan, you can parse XML result and get the impacted columns.
For example: On the node <Predicate>
Like Lamak says, doing this right is nearly impossible, at least in SQL.
To do it properly you should use a SQL Parser for the SQL variant you are using. A SQL Parser will identify the objects referenced in the statement, and the elements of the where clauses.
But if you know that the queries are simple and look like the one you are listing, you can use some simple queries to pick apart the statements. I have made an example for a single statement:
declare
#foo nvarchar(max)=N'Select ID, Code, Phone From Staff Where Phone= ''1234'' And Email = ''a#b'' '
, #From nvarchar(max)=' From '
, #Table nvarchar(max)
declare #a int
-- find FROM
set #a= PATINDEX('%'+#From+'%',#foo)
-- Find statement to the right of from
set #foo=ltrim(RIGHT(#foo,len(#foo)-(#a)))
-- Find first space
set #a=CHARINDEX(' ',#foo)
-- find first word, we assume it is the table name
set #Table=ltrim(rtrim(LEFT(#foo,#a)))
-- Find WHERE statement
set #foo=ltrim(rtrim(replace(right(#foo,len(#foo)-#a),'Where','')))
-- Now find matching columns in table, I am using SQL Server so I look up column names in information_schema.columns
select #Table,column_name
from INFORMATION_SCHEMA.COLUMNS
--from (values ('ID'),('Code'),('Name'),('Phone'),('DOB'),('Email'),('Addr1'),('Addr2'),('Addr3'))cols(column_name)
where TABLE_NAME=#table
and #foo like '%'+column_name+'%'
This solution will only work for some simple statements, it assumes a lot of stuff.
Writing a stored procedure that will have multiple input parameters. The parameters may not always have values and could be empty. But since the possibility exists that each parameter may contain values I have to include the criterion that utilizing those parameters in the query.
My query looks something like this:
SELECT DISTINCT COUNT(*) AS SRM
FROM table p
WHERE p.gender IN (SELECT * FROM Fn_SplitParms(#gender)) AND
p.ethnicity IN (SELECT * FROM Fn_SplitParms(#race)) AND
p.marital_status IN (SELECT * FROM Fn_SplitParms(#maritalstatus))
So my problem is if #gender is empty(' ') the query will return data where gender field is empty when I really want to just ignore p.gender all together. I don't want to have to accomplish this task using IF/ELSE conditional statements because they would be too numerous.
Is there any way to use CASE with IN for this scenario? OR
Is there other logic that I'm just not comprehending that will solve this?
Having trouble finding something that works well...
Thanks!
Use or:
SELECT DISTINCT COUNT(*) AS SRM
FROM table p
WHERE
(p.gender IN (SELECT * FROM Fn_SplitParms(#gender)) OR #gender = '')
AND (p.ethnicity IN (SELECT * FROM Fn_SplitParms(#race)) OR #race = '')
AND (p.marital_status IN (SELECT * FROM Fn_SplitParms(#maritalstatus)) OR #maritalstatus = '')
You might also want to consider table-valued parameters (if using SQL Server 2008 and up) - these can sometimes make the code simpler, since they are treated as tables (which in your case, may be empty) and you can join - plus no awkward split function required.
I have a table which looks something like
Event ID Date Instructor
1 1/1/2000 Person 1
1 1/1/2000 Person 2
Now what I want to do is return this data so that each event is on one row and the Instructors are all in one column split with a <br> tag like 'Person 1 <br> Person 2'
Currently the way I have done this is to use a function
CREATE FUNCTION fnReturnInstructorNamesAsHTML
(
#EventID INT
)
RETURNS VARCHAR(max)
BEGIN
DECLARE #Result VARCHAR(MAX)
SELECT
#result = coalesce(#result + '<br>', '') + inst.InstructorName
FROM
[OpsInstructorEventsView] inst
WHERE
inst.EventID = #EventID
RETURN #result
END
Then my main stored procedure calls it like
SELECT
ev.[BGcolour],
ev.[Event] AS name,
ev.[eventid] AS ID,
ev.[eventstart],
ev.[CourseType],
ev.[Type],
ev.[OtherType],
ev.[OtherTypeDesc],
ev.[eventend],
ev.[CourseNo],
ev.[Confirmed],
ev.[Cancelled],
ev.[DeviceID] AS resource_id,
ev.Crew,
ev.CompanyName ,
ev.Notes,
dbo.fnReturnInstructorNamesAsHTML(ev.EventID) as Names
FROM
[OpsSimEventsView] ev
JOIN
[OpsInstructorEventsView] inst
ON
ev.EventID = inst.EventID
This is very slow, im looking at 4seconds per call to the DB. Is there a way for me to improve the performance of the function? Its a fairly small function so im not sure what I can do here, and I couldnt see a way to work the COALESCE into the SELECT of the main procedure.
Any help would be really appreciated, thanks.
You could try something like this.
SELECT
ev.[BGcolour],
ev.[Event] AS name,
ev.[eventid] AS ID,
ev.[eventstart],
ev.[CourseType],
ev.[Type],
ev.[OtherType],
ev.[OtherTypeDesc],
ev.[eventend],
ev.[CourseNo],
ev.[Confirmed],
ev.[Cancelled],
ev.[DeviceID] AS resource_id,
ev.Crew,
ev.CompanyName ,
ev.Notes,
STUFF((SELECT '<br>'+inst.InstructorName
FROM [OpsInstructorEventsView] inst
WHERE ev.EventID = inst.EventID
FOR XML PATH(''), TYPE).value('.', 'nvarchar(max)'), 1, 4, '') as Names
FROM
[OpsSimEventsView] ev
Not sure why you have joined OpsInstructorEventsView in the main query. I removed it here but if you needed you can just add it again.
A few things to look at:
1) The overhead of functions makes them expensive to call, especially in the select statement of a query that could potentially be returning thousands of rows. It will have to execute that function for every one of them. Consider merging the behavior of the function into your main stored procedure, where the SQL Server can make better use of its optimizer.
2) Since you are joining on event id in both tables, make sure you have an index on those two columns. I would expect that you do, given that those both appear to be primary key columns, but make sure. An index can make a huge difference.
3) Convert your coalesce call into its equivalent case statements to remove the overhead of calling that function.
Yes make it an INLINE Table-Valued SQL function:
CREATE FUNCTION fnReturnInstructorNamesAsHTML
( #EventID INT )
RETURNS Table
As
Return
SELECT InstructorName + '<br>' result
FROM OpsInstructorEventsView
WHERE EventID = #EventID
Go
Then, in your SQL Statement, use it like this
SELECT ]Other stuff],
(Select result from dbo.fnReturnInstructorNamesAsHTML(ev.EventID)) as Names
FROM OpsSimEventsView ev
JOIN OpsInstructorEventsView inst
ON ev.EventID = inst.EventID
I'm not exactly clear how the query you show in your question is concatenating data from multiple rows in one row of the result, but the problem is that ordinary UDFs are compiled on use, on EVERY use, so for each row in your output result the Query processopr has to recompile the UDF again. THis is NOT True for an "inline table valued" UDF, as it's sql is folded into the outer sql before it is passed to the SQL optimizer, (the subsystem that generates the statement cache plan) and so the UDF is only compiled once.