CREATE PROCEDURE [dbo].[uspGetLogs]
(
#StartDate DATETIME,
#EndDate DATETIME
)
AS
SELECT
sl.ID,
LOG10(sl.Value)
FROM
dbo.SampleList sl
INNER JOIN
(
SELECT
ID,
RANK() OVER(PARTITION BY Codec ORDER BY TimeStampUTC DESC, d.ID DESC) ranked
FROM
dbo.SampleList
WHERE
ListDate BETWEEN #StartDate AND #EndDate
) r
ON
r.ID = sl.ID AND
r.ranked = 1
I tried this stored procedure with this #StartDate = 2014-01-29 #EndDate = 2015-03-14.
And gets this error
An invalid floating point operation occurred
The reason of the error "An Invalid floating point operation occured" is the invalid usage of mathematical function.
SELECT LOG10(-3);
SELECT LOG10(0);
If the above functions are run it will return the error.
I able to get a single value from the whole table set where value is less than one. But the ListDate for that value is 2015-03-14 so it should not be included because it is not coverted by the date range passed in the stored procedure.
So it seems that the stored procedure executes the function in the whole set first before joining and filtering the dataset with date range.
Is this expected?
I think there could be a data issue here as the basic logic of your code doesn't cause an issue, see the below sample:
CREATE TABLE #temp1 ( id INT, val INT )
CREATE TABLE #temp2 ( id INT, val INT )
INSERT INTO #temp1 ( id, val )
VALUES ( 1, 1 ), ( 2, 10 ), ( 3, -1 ) -- Negative value for id=3 exculded in subquery
INSERT INTO #temp2 ( id, val )
VALUES ( 1, 1 ), ( 2, 10 ), ( 3, 20 )
SELECT t1.id ,
LOG10(t1.val) AS Val
FROM #temp1 t1
INNER JOIN ( SELECT * ,
RANK() OVER ( PARTITION BY id ORDER BY val ) ranked
FROM #temp2
WHERE id BETWEEN 1 AND 2 -- excludes id 3
) t2 ON t2.id = t1.id
AND t2.ranked = 1
DROP TABLE #temp1
DROP TABLE #temp2
Produces:
id val
1 0
2 1
If you modify the BETWEEN clause to WHERE id BETWEEN 1 AND 3, you do see the error as the negative value is included.
So I'd triple check the data and if there's still an issue, try to post a small sample that recreates the issue.
I dont think there's a guaranteed point where function gets executed, so my answer is it depends, it depends on query plan, on how "early" the optimizer decides to execute the function - before or after the join. To make sure the function is only executed for valid values, you can change to:
CASE WHEN sl.Value < 1 THEN 0 ELSE LOG10(sl.Value) END
Related
I'm trying to copy data from one table to another, while transposing it and combining it into appropriate rows, with different columns in the second table.
First time posting. Yes this may seem simple to everyone here. I have tried for a couple hours to solve this. I do not have much support internally and have learned a great deal on this forum and managed to get so much accomplished with your other help examples. I appreciate any help with this.
Table 1 has the data in this format.
Type Date Value
--------------------
First 2019 1
First 2020 2
Second 2019 3
Second 2020 4
Table 2 already has the Date rows populated and columns created. It is waiting for the Values from Table 1 to be placed in the appropriate column/row.
Date First Second
------------------
2019 1 3
2020 2 4
For an update, I might use two joins:
update t2
set first = tf.value,
second = ts.value
from table2 t2 left join
table1 tf
on t2.date = tf.date and tf.type = 'First' left join
table1 ts
on t2.date = ts.date and ts.type = 'Second'
where tf.date is not null or ts.date is not null;
use conditional aggregation
select date,max(case when type='First' then value end) as First,
max(case when type='Second' then value end) as Second from t
group by date
You can do conditional aggregation :
select date,
max(case when type = 'first' then value end) as first,
max(case when type = 'Second' then value end) as Second
from table t
group by date;
After that you can use cte :
with cte as (
select date,
max(case when type = 'first' then value end) as first,
max(case when type = 'Second' then value end) as Second
from table t
group by date
)
update t2
set t2.First = t1.First,
t2.Second = t1.Second
from table2 t2 inner join
cte t1
on t1.date = t2.date;
Seems like you're after a PIVOT
DECLARE #Table1 TABLE
(
[Type] NVARCHAR(100)
, [Date] INT
, [Value] INT
);
DECLARE #Table2 TABLE(
[Date] int
,[First] int
,[Second] int
)
INSERT INTO #Table1 (
[Type]
, [Date]
, [Value]
)
VALUES ( 'First', 2019, 1 )
, ( 'First', 2020, 2 )
, ( 'Second', 2019, 3 )
, ( 'Second', 2020, 4 );
INSERT INTO #Table2 (
[Date]
)
VALUES (2019),(2020)
--Show us what's in the tables
SELECT * FROM #Table1
SELECT * FROM #Table2
--How to pivot the data from Table 1
SELECT * FROM #Table1
PIVOT (
MAX([Value]) --Pivot on this Column
FOR [Type] IN ( [First], [Second] ) --Make column where [Value] is in one of this
) AS [pvt] --Table alias
--which gives
--Date First Second
------------- ----------- -----------
--2019 1 3
--2020 2 4
--Using that we can update #Table2
UPDATE [tbl2]
SET [tbl2].[First] = pvt.[First]
,[tbl2].[Second] = pvt.[Second]
FROM #Table1 tbl1
PIVOT (
MAX([Value]) --Pivot on this Column
FOR [Type] IN ( [First], [Second] ) --Make column where [Value] is in one of this
) AS [pvt] --Table alias
INNER JOIN #Table2 tbl2 ON [tbl2].[Date] = [pvt].[Date]
--Results from #Table 2 after updated
SELECT * FROM #Table2
--which gives
--Date First Second
------------- ----------- -----------
--2019 1 3
--2020 2 4
I need to update the current row using the following logic:
if current row is null, then set it as previous row
if current row is not null, then no action
the 1st row is not null, then NULL appears randomly
Those NULLs need to be updated using the logic previously mentioned
e.g.
1. 1
2. null
3. null
4. 2
5. null
6. null
needs to be updated as
1. 1
2. 1
3. 1
4. 2
5. 2
6. 2
How to do it in SQL?
Thanks
r
In case of two Null values in a row, you need to define the least non-null value of the table, so I think Outer Apply will handle your problem:
CREATE TABLE #TB(ID Int Identity(1, 1), Value Int)
INSERT INTO #TB([Value]) VALUES(1),(Null),(Null),(2),(Null),(Null)
UPDATE G SET G.Value = GG.Value
FROM
#TB AS G
OUTER APPLY
(SELECT
TOP 1 *
FROM
#TB AS GG
WHERE
GG.Value IS NOT NULL
AND
GG.ID < G.ID
ORDER BY
GG.ID DESC
) AS GG
WHERE
G.Value IS NULL
SELECT * FROM #TB AS T
but note, that if the first value is Null it will not give you the results, as you have not defined the logic for this scenario.
This might help:
SELECT
t1.col1,
t1.col2 AS previous,
(SELECT
t2.col2
FROM table_1 t2
WHERE t2.col1 = (SELECT
MAX(t3.col1)
FROM table_1 t3
WHERE t3.col1 <= t1.col1
AND col2 IS NOT NULL))
AS new
FROM table_1 t1;
result
Where are you using this SQL code? If you are using Hive SQL for example, there is a function which allows you to directly get last non null value:
LAST_VALUE(col, true) over (PARTITION BY id ORDER BY date)
Oracle 10g has also a function to do this, as adressed in this thread:
Fill null values with last non-null amount - Oracle SQL
Are you familiar with window functions?
while (select count(*) FROM Table_1 where c1_derived = '') > 0
begin
update top(1) Table_1
set c1_derived = (select c1_derived from Table_1 t2 where (t2.id = [Table_1].id-1))
where c1_derived = ''
end
Try the below script. (sql 2008 +)
CREATE TABLE #table(id Int Identity(1, 1), value Int)
INSERT INTO #table([Value]) VALUES(1),(Null),(Null),(2),(Null),(Null)
;WITH cte AS
(
SELECT ID,Value,ROW_NUMBER() OVER(ORDER BY (SELECT 1)) AS row
FROM #table
)
SELECT a.ID,max(b.Value)
FROM cte a
INNER JOIN cte b ON a.row >=b.row
GROUP BY a.ID
drop table #table
Edit2 this also another script using "UNBOUNDED PRECEDING "
CREATE TABLE #table(id Int Identity(1, 1), value Int)
INSERT INTO #table([Value]) VALUES(1),(Null),(Null),(2),(Null),(Null)
select * ,max(t.value) over(order by Id Rows UNBOUNDED PRECEDING) maxValue
from #table t
drop table #table
check this link about "OVER Clause"
https://learn.microsoft.com/en-us/sql/t-sql/queries/select-over-clause-transact-sql
I would like my SQL query to return a row even if there is no row matching in my IN clause.
For exemple this query:
SELECT id, foo
FROM table
WHERE id IN (0, 1, 2, 3)
would return:
id|foo
0|bar
1|bar
2|bar
3|null
But instead I have (because no row with id 3):
id|foo
0|bar
1|bar
2|bar
I have been able to find this trick:
SELECT tmpTable.id, table.bar
FROM (
SELECT 0 as id
UNION SELECT 1
UNION SELECT 2
UNION SELECT 3
) tmpTable
LEFT JOIN
(
SELECT table.foo, table.id
FROM table
WHERE table.id IN (0, 1, 2, 3)
) table
on table.id = tmpTable.id
Is there a better way?
Bonus: How to make it work with myBatis's list variable?
overslacked is right. Most SQL developers use an auxiliary table that stores integers (and one that stores dates). This is outlined in an entire chapter of Joe Celko's "SQL for Smarties".
Example:
CREATE TABLE numeri ( numero INTEGER PRIMARY KEY )
DECLARE #x INTEGER
SET #x = 0
WHILE #x < 1000
BEGIN
INSERT INTO numeri ( numero ) VALUES ( #x )
SET #x = #x + 1
END
SELECT
numero AS id,
foo
FROM
numeri
LEFT OUTER JOIN my_table
ON my_table.id = numero
WHERE
numero BETWEEN 0 AND 3
Main Goal of Programming minimal code high performance no need this things just remove id 3 from in clause
What about just saying:
SELECT id, foo
FROM table
WHERE id >= 0 AND <= 3
I'm using Sql Server 2012.
I need to select rows from a table for processing. The number of rows needs to be variable. I need to update the rows I'm selecting to a "being processed" status - I have a guid to populate for this purpose.
I've encountered several examples of using row_number() and a couple of examples of ways of using CTE's, but I'm not sure on how to combine them (or if that's even the correct strategy). I would appreciate any insight.
Here is what I have so far:
DECLARE #SessionGuid uniqueidentifier, #rowcount bigint
SELECT #rowcount = 1000
SELECT #sessionguid = newid()
DECLARE #myProductChanges table (
ProductChangeId bigint
, ProductTypeId smallint
, SourceSystemId tinyint
, ChangeTypeId tinyint );
WITH NextPage AS
(
SELECT
ProductChangeId, ServiceSessionGuid,
ROW_NUMBER() OVER (ORDER BY ProductChangeId) AS 'RowNum'
FROM dbo.ProductChange
WHERE 'RowNum' < #rowcount
)
UPDATE dbo.ProductChange
SET ServiceSessionGuid = #sessionguid, ProcessingStateId = 2, UpdatedDate = getdate()
OUTPUT
INSERTED.ProductChangeId,
INSERTED.ProductTypeId,
INSERTED.SourceSystemId,
INSERTED.ChangeTypeId
INTO #myProductChanges
FROM dbo.ProductChange as pc join NextPage on pc.ProductChangeId = NextPage.ProductChangeId
From here I will select from my temp table and return the data:
SELECT mpc.ProductChangeId
, pt.ProductName as ProductType
, ss.Name as SourceSystem
, ct.ChangeDescription as ChangeType
FROM #myProductChanges as mpc
join dbo.R_ProductType pt on mpc.ProductTypeId = pt.ProductTypeId
join dbo.R_SourceSystem ss on mpc.SourceSystemId = ss.SourceSystemId
join dbo.R_ChangeType ct on mpc.ChangeTypeId = ct.ChangeTypeId
ORDER BY ProductType asc
So far this doesn't work for me. I get an error when I try to run it:
Msg 8114, Level 16, State 5, Line 20
Error converting data type varchar to bigint.
I'm not clear on what I'm doing wrong - so - any help is appreciated.
Thanks!
BTW, here are some of the questions I've used as reference to try and solve this:
https://stackoverflow.com/questions/9777178
https://stackoverflow.com/questions/3319842
https://stackoverflow.com/questions/6402103
This subquery makes no sense:
SELECT
ProductChangeId, ServiceSessionGuid,
ROW_NUMBER() OVER (ORDER BY ProductChangeId) AS 'RowNum'
FROM dbo.ProductChange
WHERE 'RowNum' < #rowcount
You can't reference the alias RowNum at the same scope (and you are trying to compare a string, not an alias, anyway), because when the WHERE clause is parsed, the SELECT list hasn't been materialized yet. What you need is either another nest:
SELECT ProductChangeId, ServiceSessionGuid, RowNum
FROM (SELECT ProductChangeId, ServiceSessionGuid,
ROW_NUMBER() OVER (ORDER BY ProductChangeId) AS RowNum
FROM dbo.ProductChange
) AS x WHERE RowNum < #rowcount
Or:
SELECT TOP (#rowcount-1) ProductChangeId, ServiceSessionGuid,
ROW_NUMBER() OVER (ORDER BY ProductChangeId) AS RowNum
FROM dbo.ProductChange
ORDER BY ProductChangeId
Also please stop using 'alias' - when you need to delimit aliases (you don't in this case), use [square brackets].
I'm guessing, but I think you want <= rather than < if you want to affect #rowcount rows, not one less.
Another tip is that CTEs can be updated directly*, as shown here:
WITH NextPage AS
(
SELECT TOP(#rowcount) *
FROM dbo.ProductChange
)
UPDATE NextPage
SET ServiceSessionGuid = #sessionguid, ProcessingStateId = 2, UpdatedDate = getdate()
OUTPUT
INSERTED.ProductChangeId,
INSERTED.ProductTypeId,
INSERTED.SourceSystemId,
INSERTED.ChangeTypeId
INTO #myProductChanges
* The updates affect the base table in the CTE, i.e. dbo.ProductChange
I have three address line columns, aline1, aline2, aline3 for a street
address. As staged from inconsistent data, any or all of them can be
blank. I want to move the first non-blank to addrline1, 2nd non-blank
to addrline2, and clear line 3 if there aren't three non blank lines,
else leave it. ("First" means aline1 is first unless it's blank,
aline2 is first if aline1 is blank, aline3 is first if aline1 and 2
are both blank)
The rows in this staging table do not have a key and there could be
duplicate rows. I could add a key.
Not counting a big case statement that enumerates the possible
combination of blank and non blank and moves the fields around, how
can I update the table? (This same problem comes up with a lot more
than 3 lines, so that's why I don't want to use a case statement)
I'm using Microsoft SQL Server 2008
Another alternative. It uses the undocumented %%physloc%% function to work without a key. You would be much better off adding a key to the table.
CREATE TABLE #t
(
aline1 VARCHAR(100),
aline2 VARCHAR(100),
aline3 VARCHAR(100)
)
INSERT INTO #t VALUES(NULL, NULL, 'a1')
INSERT INTO #t VALUES('a2', NULL, 'b2')
;WITH cte
AS (SELECT *,
MAX(CASE WHEN RN=1 THEN value END) OVER (PARTITION BY %%physloc%%) AS new_aline1,
MAX(CASE WHEN RN=2 THEN value END) OVER (PARTITION BY %%physloc%%) AS new_aline2,
MAX(CASE WHEN RN=3 THEN value END) OVER (PARTITION BY %%physloc%%) AS new_aline3
FROM #t
OUTER APPLY (SELECT ROW_NUMBER() OVER (ORDER BY CASE WHEN value IS NULL THEN 1 ELSE 0 END, idx) AS
RN, idx, value
FROM (VALUES(1,aline1),
(2,aline2),
(3,aline3)) t (idx, value)) d)
UPDATE cte
SET aline1 = new_aline1,
aline2 = new_aline2,
aline3 = new_aline3
SELECT *
FROM #t
DROP TABLE #t
Here's an alternative
Sample table for discussion, don't worry about the nonsensical data, they just need to be null or not
create table taddress (id int,a varchar(10),b varchar(10),c varchar(10));
insert taddress
select 1,1,2,3 union all
select 2,1, null, 3 union all
select 3,null, 1, 2 union all
select 4,null,null,2 union all
select 5,1, null, null union all
select 6,null, 4, null
The query, which really just normalizes the data
;with tmp as (
select *, rn=ROW_NUMBER() over (partition by t.id order by sort)
from taddress t
outer apply
(
select 1, t.a where t.a is not null union all
select 2, t.b where t.b is not null union all
select 3, t.c where t.c is not null
--- EXPAND HERE
) u(sort, line)
)
select t0.id, t1.line, t2.line, t3.line
from taddress t0
left join tmp t1 on t1.id = t0.id and t1.rn=1
left join tmp t2 on t2.id = t0.id and t2.rn=2
left join tmp t3 on t3.id = t0.id and t3.rn=3
--- AND HERE
order by t0.id
EDIT - for the update back into table
;with tmp as (
select *, rn=ROW_NUMBER() over (partition by t.id order by sort)
from taddress t
outer apply
(
select 1, t.a where t.a is not null union all
select 2, t.b where t.b is not null union all
select 3, t.c where t.c is not null
--- EXPAND HERE
) u(sort, line)
)
UPDATE taddress
set a = t1.line,
b = t2.line,
c = t3.line
from taddress t0
left join tmp t1 on t1.id = t0.id and t1.rn=1
left join tmp t2 on t2.id = t0.id and t2.rn=2
left join tmp t3 on t3.id = t0.id and t3.rn=3
Update - Changed statement to an Update statement. Removed Case statement solution
With this solution, you will need a unique key in the staging table.
With Inputs As
(
Select PK, 1 As LineNum, aline1 As Value
From StagingTable
Where aline1 Is Not Null
Union All
Select PK, 2, aline2
From StagingTable
Where aline2 Is Not Null
Union All
Select PK, 3, aline3
From StagingTable
Where aline3 Is Not Null
)
, ResequencedInputs As
(
Select PK, Value
, Row_Number() Over( Order By LineNum ) As LineNum
From Inputs
)
, NewValues As
(
Select S.PK
, Min( Case When R.LineNum = 1 Then R.addrline1 End ) As addrline1
, Min( Case When R.LineNum = 2 Then R.addrline1 End ) As addrline2
, Min( Case When R.LineNum = 3 Then R.addrline1 End ) As addrline3
From StagingTable As S
Left Join ResequencedInputs As R
On R.PK = S.PK
Group By S.PK
)
Update OtherTable
Set addrline1 = T2.addrline1
, addrline2 = T2.addrline2
, addrline3 = T2.addrline3
From OtherTable As T
Left Join NewValues As T2
On T2.PK = T.PK
R. A. Cyberkiwi, Thomas, and Martin, thanks very much - these were very generous responses by each of you. All of these answers were the type of spoonfeeding I was looking for. I'd say they all rely on a key-like device and work by dividing addresses into lines, some of which are empty and some of which aren't, excluding the empties. In the case of lines of addresses, in my opinion this is semantically a gimmick to make the problem fit what SQL does well, and it's not a natural way to conceptualize the problem. Address lines are not "really" separate rows in a table that just got denormalized for a report. But that's debatable and whether you agree or not, I (a rank beginner) think each of your alternatives are idiomatic solutions worth elaborating on and studying.
I also get lots of similar cases where there really is normalization to be done - e.g., collatDesc1, collatCode1, collatLastAppraisal1, ... collatLastAppraisal5, with more complex criteria about what in excludeand how to order than with addresses, and I think techniques from your answers will be helpful.
%%phsloc%% is fun - since I'm able to create a key in this case I won't use it (as Martin advises). There was other stuff in Martin's stuff I wasn't familiar with too, and I'm still tossing them all around.
FWIW, here's the trigger I tried out, I don't know that I'll actually use it for the problem at hand. I think this qualifies a "bubble sort", with the swapping expressed in a peculiar way.
create trigger fixit on lines
instead of insert as
declare #maybeblank1 as varchar(max)
declare #maybeblank2 as varchar(max)
declare #maybeblank3 as varchar(max)
set #maybeBlank1 = (select line1 from inserted)
set #maybeBlank2 = (select line2 from inserted)
set #maybeBlank3 = (select line3 from inserted)
declare #counter int
set #counter = 0
while #counter < 3
begin
set #counter = #counter + 1
if #maybeBlank2 = ''
begin
set #maybeBlank2 =#maybeblank3
set #maybeBlank3 = ''
end
if #maybeBlank1 = ''
begin
set #maybeBlank1 = #maybeBlank2
set #maybeBlank2 = ''
end
end
select * into #kludge from inserted
update #kludge
set line1 = #maybeBlank1,
line2 = #maybeBlank2,
line3 = #maybeBlank3
insert into lines
select * from #kludge
You could make an insert and update trigger that check if the fields are empty and then move them.