SSIS Lookup Transformation - SQL query not working - sql

I have an SSIS package for deployment into a SQL Server 2012 SSISDB and it uses a Lookup transformation. I am using a result from a SQL query to perform the lookup comparison.
This does not work and I get all rows as "No matched".
The query is the following:
DECLARE #LastJobDate DATETIME
SELECT #LastJobDate = COALESCE(MIN(S.LastImportDate), DATEADD(DAY, -2, GETDATE()))
FROM Stations S INNER JOIN
Lines L ON S.ID_Line = L.ID_Line
WHERE L.Name LIKE 'lineType%' AND S.ImportData = 1 AND S.Active = 1
SELECT J.ID_Line, J.ID_Job, J.SerialNumber
FROM [Jobs] J INNER JOIN
[Lines] L ON J.ID_Line = L.ID_Line
WHERE L.Name LIKE 'lineType%'AND J.TimeStamp >= DATEADD(HOUR, -1, #LastJobDate)
By accident, I found that if I place a [SET NOCOUNT ON] at the beggining of the query, it will work.
DECLARE #LastJobDate DATETIME
SET NOCOUNT ON
SELECT #LastJobDate = COALESCE(MIN(S.LastImportDate), DATEADD(DAY, -2, GETDATE()))
FROM Stations S INNER JOIN
Lines L ON S.ID_Line = L.ID_Line
WHERE L.Name LIKE 'lineType%' AND S.ImportData = 1 AND S.Active = 1
SELECT J.ID_Line, J.ID_Job, J.SerialNumber
FROM [Jobs] J INNER JOIN
[Lines] L ON J.ID_Line = L.ID_Line
WHERE L.Name LIKE 'lineType%'AND J.TimeStamp >= DATEADD(HOUR, -1, #LastJobDate)
Am I missing something? Why this behavior?

Why this behavior?
An SSIS Lookup Component can only consider the first result returned by a multi-statement query such as yours.
When you don't have SET NOCOUNT ON, the first result returned by your query will be the message "1 row(s) affected" or something like that. The Lookup Component will not be able to look at the result set returned by the second half of your query.
This is why setting NOCOUNT ON fixes the problem. The "row(s) affected" message will not be returned by the first part of the query, and the only thing returned will be the resultset of the second part of the query, which the Lookup Component will then process.

Related

SQL query running slowly - parameter sniffing

I have a simple query where I return a list of orders by date range. This query is used in a report which feeds it parameters(Site, From Date, and To Date).
ALTER PROCEDURE [dbo].[Z_N_ECOM_ORDER_STATUS_DATERANGE]
#Site VARCHAR(5),
#FromDate DATETIME,
#ToDate DATETIME
AS
BEGIN
SET NOCOUNT ON;
SELECT
o.Company_Code,
o.Division_Code,
o.Control_Number,
RTRIM(o.Customer_Purchase_Order_Number) AS Shopify_Num,
CASE
WHEN p.PickTicket_Number IS NULL
THEN i.PickTicket_Number
ELSE p.PickTicket_Number
END PickTicket_Number,
i.Invoice_Number,
o.Date_Entered,
CASE
WHEN ph.packslip IS NULL AND i.invoice_number IS NULL
AND P.pickticket_number IS NULL
THEN 'Cancelled'
WHEN ph.packslip IS NULL AND i.invoice_number IS NULL
AND DATEADD(minute, 90, o.date_entered) > CURRENT_TIMESTAMP
THEN 'Not Entered Yet'
WHEN ph.packslip IS NULL
THEN 'SHIPPED & UPLOADED'
ELSE RTRIM (z.status)
END Accellos_Status,
b.UPS_Tracking_Number Tracking_Number
FROM
[JMNYC-AMTDB].[AMTPLUS].[dbo].Orders o (nolock)
LEFT JOIN
[JMNYC-AMTDB].[AMTPLUS].[dbo].PickTickets p (nolock) ON o.Company_Code = p.Company_Code
AND o.Division_Code = p.Division_Code
AND o.Control_Number = p.Control_Number
LEFT JOIN
[JMNYC-AMTDB].[AMTPLUS].[dbo].Invoices i (nolock) ON o.Company_Code = i.Company_Code
AND o.Division_Code = i.Division_Code
AND o.Control_Number = i.Control_Number
LEFT JOIN
[JMNYC-AMTDB].[AMTPLUS].[dbo].box b (nolock) ON o.Company_Code = b.Company_Code
AND o.Division_Code = b.Division_Code
AND i.PickTicket_Number = b.PickTicket_Number
LEFT JOIN
pickhead ph (nolock) ON p.PickTicket_Number = ph.packslip
LEFT JOIN
Z_Status z (nolock) ON ph.PROCSTEP = z.procstep
WHERE
o.Company_Code = LEFT(#Site, 2)
AND o.Division_Code = RIGHT(#Site, 3)
AND o.Customer_Number = 'ecom2x'
AND o.Date_Entered BETWEEN #FromDate AND DATEADD(dayofyear, 1, #ToDate)
ORDER BY
o.date_entered DESC
END
The problem with this query is that it takes way too long and the problem lines are
WHERE
o.Company_Code = LEFT(#Site, 2)
AND o.Division_Code = RIGHT(#Site, 3)
The format of the variable site is something like '09001' or '03001' where the left side is the company and the right side is the division
Because when I run this query with hard-coded values, it runs pretty much instantaneously. When I use the parameters, it takes minutes.
So I looked it up and I discovered about parameter sniffing. So I added the following line after the begin statement.
DECLARE #LocalSite VARCHAR(5) = CAST(#Site AS VARCHAR(5))
However, it still runs extremely slow.
My new where statement would be
WHERE
o.Customer_Number = 'ecom2x'
AND o.Date_Entered BETWEEN #FromDate AND DATEADD(dayofyear, 1, #ToDate)
AND ((#LocalSite = '00000') OR (O.Company_Code = LEFT(#LocalSite, 2) AND O.Division_Code = RIGHT(#LocalSite, 3)))
order by o.date_entered desc*
I also want the user to have the functionality of selecting all sites which will make the site variable be '00000' and thus it shouldn't run the company/division code check. This current where statement makes the query run very slow.
Does anyone know what I am doing wrong?
Can you try to avoid using LEFT() and RIGHT() by declaring few variables and assigning values to those variables and then using them in the SELECT statement?
a hint OPTIMIZED FOR UNKNOWN to avoid parameter sniffing:
option (OPTIMIZE FOR (#p1 UNKNOWN, #p2 UNKNOWN))
Where p1 and p2 those two variables mentioned above
I also want the user to have the functionality of selecting all sites
which will make the site variable be '00000' and thus it shouldn't run
the company/division code check. This current where statement makes
the query run very slow.
This can be optimized by replacing current SELECT with IF statement that uses two SELECTs. If value is 00000 just avoid to check on company and division, else run the same select but with those extra checks
Another striking thing is querying linked server objects with further join to local tables. Consider to split this into a separate step, for instance by storing data in temporary table (not a table variable!) as intermediate result. Then temp table to be joined with local objects. This can improve accuracy of query plan because of better estimates.
Did you try taking left and right values of#site parameter in two different variables and using those variables in SP.
For eg.
Declare #compcode as varchar(2)
Declare #divcode as varchar(3)
Set #compcode=LEFT(#Site, 2)
Set #divcode=RIGHT(#Site, 3)
Your where condition
WHERE
o.Company_Code = #compcode
AND o.Division_Code = #divcode

Query to update subscriber status to a list in salesforce marketing cloud

I have a data extension that I will be populating my subscriber information into (I have to do this due to my API set up). However, I want this information to be pasted to my publication list in salesforce. If people unsubscribe, a field in my data extension called 'Subscribed' will be updated to 'False' (and vica-versa). I want to set up a query to run everday that scans this data extension and updates all the false records to unsubscibed in my publication list. I built the query below, but I am getting an error that say
"Incorrect syntax near the keyword 'INNER'."
SELECT b.EMAIL_ADDRESS, 'Unsubscribed' AS Status
FROM Welcome b
WHERE b.Subscribed = 'False'
AND Date_Changed >= DATEADD(day, -1, GETDATE())
INNER JOIN _ListSubscribers a
ON a.EmailAddress = b.EMAIL_ADDRESS
WHERE a.ListID = '112'
Thanks!
Where clause always goes after joins
SELECT b.EMAIL_ADDRESS, 'Unsubscribed' AS Status
FROM Welcome b
INNER JOIN _ListSubscribers a ON a.EmailAddress = b.EMAIL_ADDRESS
WHERE b.Subscribed = 'False' and
a.ListID = '112' and
b.Date_Changed >= DATEADD(day, -1, GETDATE());

multi part indentifier could not be bound?

I'm trying to use two tables in a stored procedure but am getting this error on the final line
''multi part identifier T.HireDate could not be bound''
I'm guessing its to do with the joining of the tables but am a little lost. Heres my code:
CREATE PROC spPayIncreaseCheck
AS
SELECT T.ID,FName,LName,HireDate,Payrate
FROM Assignment.dbo.Payments pay
join Assignment.dbo.Teachers T
ON pay.ID = T.ID
UPDATE Assignment.dbo.Payments
SET Payrate = 'High'
where T.HireDate < dateadd(year, -3, GETDATE())
Are you trying to update, or are you trying to select?
That is the wrong form for an update using a join in sql server.
The correct form would be like so:
update pay
set Payrate = 'High'
from Assignment.dbo.Payments pay
inner join Assignment.dbo.Teachers T
on pay.ID = T.ID
where T.HireDate < dateadd(year, -3, getdate());
Also, are you sure on pay.ID = T.ID is the correct join clause? Normally you would expect to see something like on pay.TeacherID = T.ID instead.
You are updating the Assignment.dbo.Payments table and in the WHERE condition you have a t.HireDate but t is not an alias that is used in the UPDATE statement.
In UPDATE you cannot use aliases.
Also, please notice that the SELECT and UPDATE statements are completely non-related inside your stored procedure. It is not clear though if you need them to be related.
You need an UPDATE FROM to do a set-based update:
UPDATE Assignment.dbo.Payments
SET Payrate = 'High'
FROM
SELECT T.ID,FName,LName,HireDate,Payrate
FROM Assignment.dbo.Payments pay
JOIN Assignment.dbo.Teachers T ON pay.ID = T.ID
WHERE T.HireDate < DATEADD(YEAR, -3, GETDATE())
The simplest way to achieve this, is using an exists clause to check if the payment record matches the teacher record:
update Assignment.dbo.Payments
set Payrate = 'High'
where exists ( select 1
from Assignment.dbo.Teachers
where Payments.ID = Teachers.ID
and Teachers.HireDate < dateadd(year, -3, getdate()) )

SQL - SELECT subquery AS BIT value for EXIST check

I have a problem.
I'm trying to get a BIT value to check whether a person has entered the building last night between 10pm to midnight. When I run the subquery code by itself, it gives me the results I need. As I'm working with SSRS2008 I need for all the results to be in the same stored procedure.
So the problem is, it gives me the bit values somewhat right, for the ones that are obviously false, it gives false, for the ones that are obviously true, it gives true. But for the ones in the middle (the day shift, who leave at 23) it gives the results somewhat random..
Does anyone have a clue?
SELECT DISTINCT TOP 200
Events.LoggedTime,
PTUsers.Name,
PTDoors.PTDoorsID,
PTUsers.AccessLevel,
CAST(CASE
WHEN EXISTS (SELECT Events.LoggedTime
FROM Events
INNER JOIN PTUsers AS PTUsers_1 ON Events.GlobalIndex1 = PTUsers.GlobalRecord
INNER JOIN PTDoors AS PTDoors_1 ON Events.RecordIndex2 + 1 = PTDoors.Address
WHERE (DATEPART(day, Events.LoggedTime) = DATEPART(day, GETDATE() - 1))
AND (DATEPART(hour, Events.LoggedTime) IN (22, 23))
AND (PTDoors_1.PTDoorsID = 14)) THEN 1 ELSE 0 END AS BIT) AS Night
FROM
Events
INNER JOIN
PTUsers ON Events.GlobalIndex1 = PTUsers.GlobalRecord
INNER JOIN
PTDoors ON Events.RecordIndex2 + 1 = PTDoors.Address
WHERE
(PTUsers.Panel = 0)
AND (PTDoors.Panel = 0)
AND (PTDoors.PTDoorsID = 14)
AND (DATEPART(day, Events.LoggedTime) = DATEPART(day, GETDATE()) - 1)
AND (PTUsers.AccessLevel IN (3))
ORDER BY
Events.LoggedTime DESC
#lrd i did the corrections you suggested,
thanks for pointing out the table aliases :)
i removed the cast, so now i get the BIT column. but now the problem is that i get all "1"'s as results.
What baffles me is the that subquery works as it should as a query on it's own. it goes back a day, and displays the entries on that door in the given timeframe.
now i'm trying to compare that information for the same person, meaning - i see a person in the list, arriving yesterday at 6am, check if that person also arrived the day before that between 22 & midnight and return a bit value to display that.
SELECT DISTINCT TOP 200 Events.LoggedTime, PTUsers.Name, PTDoors.PTDoorsID, PTUsers.AccessLevel, CASE WHEN EXISTS
(SELECT Events.LoggedTime
FROM Events INNER JOIN
PTUsers AS PTUsers_1 ON Events.GlobalIndex1 = PTUsers_1.GlobalRecord INNER JOIN
PTDoors AS PTDoors_1 ON Events.RecordIndex2 + 1 = PTDoors_1.Address
WHERE (DATEPART(day, Events.LoggedTime) = DATEPART(day, GETDATE() - 1)) AND (DATEPART(hour, Events.LoggedTime) IN (22, 23)) AND
(PTDoors_1.PTDoorsID = 14)) THEN 1 ELSE 0 END AS BIT
FROM Events INNER JOIN
PTUsers ON Events.GlobalIndex1 = PTUsers.GlobalRecord INNER JOIN
PTDoors ON Events.RecordIndex2 + 1 = PTDoors.Address
WHERE (PTUsers.Panel = 0) AND (PTDoors.Panel = 0) AND (PTDoors.PTDoorsID = 14) AND (DATEPART(day, Events.LoggedTime) = DATEPART(day, GETDATE())
- 1) AND (PTUsers.AccessLevel IN (3))
ORDER BY Events.LoggedTime DESC
I don't think you need a CAST because you are explicitly defining Night as a BIT By setting the result to EXISTS(), which is a bit. I removed the query that was incorrect.
I see your problem. You are not using the correct table alias for your join constraint in your subquery.
It should be:
INNER JOIN PTUsers AS PTUsers_1 ON Events.GlobalIndex1 = PTUsers_1.GlobalRecord
INNER JOIN PTDoors AS PTDoors_1 ON Events.RecordIndex2 + 1 = PTDoors_1.Address
Also,check and make sure you are using the correct values in your join. I would change the following for a test.
FROM Events Events_2
INNER JOIN PTUsers AS PTUsers_1 ON Events_2.GlobalIndex1 = PTUsers_1.GlobalRecord
INNER JOIN PTDoors AS PTDoors_1 ON Events_2.RecordIndex2 + 1 = PTDoors_1.Address

Trying to combine Inner Join and two where statements with a Select Max by date and save result into variable

I'm trying to use a inner join to combine 3 tables into one and pull 2 variables based on the most recent date of entry in a medical database.
I'm looking to pull the last location a patient was seen at and the doctor at that visit, which I will pass into another command later in my query. I will save those as #last_location and #doctor_name respectively. I am using #last_date to try to make a select max to get the last visit only for each given patient. However, blank values are passing forward on my variables (both #doctor_name and #last_location). Can anyone help clean this up? Appreciate the help!
My code is:
SELECT #last_location = location_mstr.location_name, #last_date = Substring (CONVERT(CHAR(8), patient_encounter.billable_timestamp, 112), 5, 2) + '/' + Substring (CONVERT(CHAR(8), patient_encounter.billable_timestamp, 112), 7, 2) + '/' + Substring (CONVERT(CHAR(8), patient_encounter.billable_timestamp, 112), 3, 2), #doctor_name = provider_mstr.description
FROM
location_mstr
INNER JOIN patient_encounter ON location_mstr.location_id = patient_encounter.location_id
INNER JOIN provider_mstr ON patient_encounter.rendering_provider_id = provider_mstr.provider_id
Where (patient_encounter.person_id = #person_id) and (#last_date = MAX(#last_date))
I also tried changing the last statement based on some threads here to:
Where #last_date = (SELECT MAX(#last_date) FROM patient_encounter WHERE patient_encounter.person_id = #person_id)
In the where clause of your original query, it looks like you are trying to find the patient_encounter record with the most recent billable_timestamp. The way you are referencing the #last_date variable in the where clause isn't going to do anything.
Instead, you can just get the TOP 1 most recent row and ORDER BY billable_timestamp DESC to get what you're looking for.
I also changed the string expression in your select list that sets #last_date to use the SQL CONVERT function. You can see a list of date formats that CONVERT can do here: http://msdn.microsoft.com/en-us/library/ms187928.aspx
SELECT TOP 1
#last_location = location_mstr.location_name
,#last_date = CONVERT(VARCHAR, patient_encounter.billable_timestamp, 1)
,#doctor_name = provider_mstr.description
FROM location_mstr
INNER JOIN patient_encounter
ON location_mstr.location_id = patient_encounter.location_id
INNER JOIN provider_mstr
ON patient_encounter.rendering_provider_id = provider_mstr.provider_id
WHERE patient_encounter.person_id = #person_id
ORDER BY patient_encounter.billable_timestamp DESC