Guys I have a stored procedure that inserts a new value in the table, only when the last inserted value is different.
CREATE PROCEDURE [dbo].[PutData]
#date datetime,
#value float
AS
IF NOT EXISTS(SELECT * FROM Sensor1 WHERE SensorTime <= #date AND SensorTime = (SELECT MAX(SensorTime) FROM Sensor1) AND SensorValue = #value)
INSERT INTO Sensor1 (SensorTime, SensorValue) VALUES (#date, #value)
RETURN 0
Now, since I'm doing this at a high frequency (say every 10ms), the IF NOT EXISTS (SELECT) statement is often getting old data, and because of this I'm getting duplicate data. Would it be possible to lock the entire table during the stored procedure, to make sure the SELECT statement always receives the latest data?
According to the poster's comments to the question, c# code receives a value from a sensor. The code is supposed to insert the value only if it is different from the previous value.
Rather than solving this in the database, why not have the code store the last value inserted and only invoke the procedure if the new value is different? Then the procedure will not need to check whether the value exists in the database; it can simply insert. This will be much more efficient.
You could write it like this :
CREATE PROCEDURE [dbo].[PutData]
#date datetime,
#value float
AS
BEGIN TRANSACTION
INSERT INTO Sensor1 (SensorTime, SensorValue)
SELECT SensorTime = #date,
SensorValue = #value
WHERE NOT EXISTS(SELECT *
FROM Sensor1 WITH (UPDLOCK, HOLDLOCK)
WHERE SensorValue = #value
AND SensorTime <= #date
AND SensorTime = (SELECT MAX(SensorTime) FROM Sensor1) )
COMMIT TRANSACTION
RETURN 0
Thinking a bit about it, you could probably write it like this too:
CREATE PROCEDURE [dbo].[PutData]
#date datetime,
#value float
AS
BEGIN TRANSACTION
INSERT INTO Sensor1 (SensorTime, SensorValue)
SELECT SensorTime = #date,
SensorValue = #value
FROM (SELECT TOP 1 SensorValue, SensorTime
FROM Sensor1 WITH (UPDLOCK, HOLDLOCK)
ORDER BY SensorTime DESC) last_value
WHERE last_value.SensorTime <= #date
AND last_value.SensorValue <> #value
COMMIT TRANSACTION
RETURN 0
Assuming you have a unique index (PK?) on SensorTime this should be quite fast actually.
Related
I've created a stored procedure that filters and paginates for a DataTable.
Problem: I need to set an OUTPUT variable for #TotalRecords found before an OFFSET occurs, otherwise it sets #TotalRecord to #RecordPerPage.
I've messed around with CTE's and also simply trying this:
SELECT *, #TotalRecord = COUNT(1)
FROM dbo
But that doesn't work either.
Here is my stored procedure, with most of the stuff pulled out:
ALTER PROCEDURE [dbo].[SearchErrorReports]
#FundNumber varchar(50) = null,
#ProfitSelected bit = 0,
#SortColumnName varchar(30) = null,
#SortDirection varchar(10) = null,
#StartIndex int = 0,
#RecordPerPage int = null,
#TotalRecord INT = 0 OUTPUT --NEED TO SET THIS BEFORE OFFSET!
AS
BEGIN
SET NOCOUNT ON;
SELECT *
FROM
(SELECT *
FROM dbo.View
WHERE (#ProfitSelected = 1 AND Profit = 1)) AS ERP
WHERE
((#FundNumber IS NULL OR #FundNumber = '')
OR (ERP.FundNumber LIKE '%' + #FundNumber + '%'))
ORDER BY
CASE
WHEN #SortColumnName = 'FundNumber' AND #SortDirection = 'asc'
THEN ERP.FundNumber
END ASC,
CASE
WHEN #SortColumnName = 'FundNumber' AND #SortDirection = 'desc'
THEN ERP.FundNumber
END DESC
OFFSET #StartIndex ROWS
FETCH NEXT #RecordPerPage ROWS ONLY
Thank you in advance!
You could try something like this:
create a CTE that gets the data you want to return
include a COUNT(*) OVER() in there to get the total count of rows
return just a subset (based on your OFFSET .. FETCH NEXT) from the CTE
So your code would look something along those lines:
-- CTE definition - call it whatever you like
WITH BaseData AS
(
SELECT
-- select all the relevant columns you need
p.ProductID,
p.ProductName,
-- using COUNT(*) OVER() returns the total count over all rows
TotalCount = COUNT(*) OVER()
FROM
dbo.Products p
)
-- now select from the CTE - using OFFSET/FETCH NEXT, get only those rows you
-- want - but the "TotalCount" column still contains the total count - before
-- the OFFSET/FETCH
SELECT *
FROM BaseData
ORDER BY ProductID
OFFSET 20 ROWS FETCH NEXT 15 ROWS ONLY
As a habit, I prefer non-null entries before possible null. I did not reference those in my response below, and limited a working example to just the two inputs you are most concerned with.
I believe there could be some more clean ways to apply your local variables to filter the query results without having to perform an offset. You could return to a temp table or a permanent usage table that cleans itself up and use IDs that aren't returned as a way to set pages. Smoother, with less fuss.
However, I understand that isn't always feasible, and I become frustrated myself with those attempting to solve your use case for you without attempting to answer the question. Quite often there are multiple ways to tackle any issue. Your job is to decide which one is best in your scenario. Our job is to help you figure out the script.
With that said, here's a potential solution using dynamic SQL.
I'm a huge believer in dynamic SQL, and use it extensively for user based table control and ease of ETL mapping control.
use TestCatalog;
set nocount on;
--Builds a temp table, just for test purposes
drop table if exists ##TestOffset;
create table ##TestOffset
(
Id int identity(1,1)
, RandomNumber decimal (10,7)
);
--Inserts 1000 random numbers between 0 and 100
while (select count(*) from ##TestOffset) < 1000
begin
insert into ##TestOffset
(RandomNumber)
values
(RAND()*100)
end;
set nocount off;
go
create procedure dbo.TestOffsetProc
#StartIndex int = null --I'll reference this like a page number below
, #RecordsPerPage int = null
as
begin
declare #MaxRows int = 30; --your front end will probably manage this, but don't trust it. I personally would store this on a table against each display so it can also be returned dynamically with less manual intrusion to this procedure.
declare #FirstRow int;
--Quick entry to ensure your record count returned doesn't excede max allowed.
if #RecordsPerPage is null or #RecordsPerPage > #MaxRows
begin
set #RecordsPerPage = #MaxRows
end;
--Same here, making sure not to return NULL to your dynamic statement. If null is returned from any variable, the entire statement will become null.
if #StartIndex is null
begin
set #StartIndex = 0
end;
set #FirstRow = #StartIndex * #RecordsPerPage
declare #Sql nvarchar(2000) = 'select
tos.*
from ##TestOffset as tos
order by tos.RandomNumber desc
offset ' + convert(nvarchar,#FirstRow) + ' rows
fetch next ' + convert(nvarchar,#RecordsPerPage) + ' rows only'
exec (#Sql);
end
go
exec dbo.TestOffsetProc;
drop table ##TestOffset;
drop procedure dbo.TestOffsetProc;
I'd like to create a table that contains two columns (id int, today datetime) and, using while loops, to insert the current date every 1 second. However, the resulting table shows the same time for all rows. Below is my code. Can anyone help me understand what I'm doing wrong, please? Thank you!
declare #mytable table (id int, today datetime)
declare #id int=1
declare #today datetime=getdate()
while #id<10
begin
waitfor delay '00:00:01'
insert into #mytable values (#id,#today)
set #id=#id+1
end
The reason every row has the same value is because you aren't setting the value of #Today anywhere apart from before your WHILE loop. GETDATE() returns a scalar value, and setting a variable to that value means it will be set the value that GETDATE() returned at the time the SET was run. The value of the variable won't change after time has passed. For example:
DECLARE #d datetime;
SET #d = GETDATE();
SELECT #d, GETDATE(); --Will return very similar values
WAITFOR DELAY '00:00:05';
SELECT #d, GETDATE(); --#d will have the same value as before, as its value is static, but GETDATE()'s value will have changed.
To do what you're after, I don't see any need for the variable for #Today, this would work fine:
DECLARE #mytable table (id int,
today datetime);
DECLARE #id int = 1;
WHILE #id < 10
BEGIN
WAITFOR DELAY '00:00:01';
INSERT INTO #mytable
VALUES (#id, GETDATE());
SET #id = #id + 1;
END;
However a loop is a bad choice anyway, as an RDBMS excels at set based operations, not iterative. You would be far better to achieve what you're after by doing:
DECLARE #mytable table (id int,
today datetime);
DECLARE #id int = 1;
WITH N AS (
SELECT N
FROM (VALUES(NULL),(NULL),(NULL),(NULL),(NULL),(NULL),(NULL),(NULL),(NULL),(NULL)) N(N)),
Tally AS(
SELECT ROW_NUMBER() OVER (ORDER BY (SELECT NULL)) -1 AS I
FROM N N1
CROSS JOIN N N2 --Not actually eneded here, but shows how to increase row count
)
INSERT INTO #mytable (id,
today)
SELECT TOP 10
T.I + #ID,
DATEADD(SECOND, T.I, GETDATE())
FROM Tally T
ORDER BY T.I;
This builds an inline tally table, and then inserts a value for a row for 10 ID, and adds 1 second to each incremented ID.
This is not a duplicate.
I do understand what the issue means but I don't understand why because the variable contains data. I'm basically trying to make a char(4) column increase alone (just like identity with integers). If the table doesn't contain anything, the first value would be 'C001' otherwise, It simply increase based on the last record.
CREATE PROCEDURE ADD_CL(#nom VARCHAR(20),
#dn DATE)
AS
BEGIN
DECLARE #B CHAR(4)
DECLARE #B_to_int INT
DECLARE #B_new_value CHAR(4)
IF EXISTS(SELECT TOP 1 *
FROM CLIENT)
SET #B_new_value = 'C001'
ELSE
BEGIN
SELECT TOP 1 #B = code_client
FROM client
ORDER BY code_client DESC
SET #B_to_int = CAST(SUBSTRING(#B, 2, 3) AS INTEGER)
SET #B_to_int = #B_to_int + 1;
SET #B_new_value = LEFT(#B, 1) + RIGHT('00' + CAST(#B_to_int AS INT), 3)
END
INSERT INTO CLIENT
VALUES (#B_new_value,
#nom,
#dn)
END
Cannot insert the value NULL into column 'code_client', table 'dbo.CLIENT'; column does not allow nulls. INSERT fails.
#B_new_value represent code_client
Your If Exists should be If Not Exists.
So change
if exists(select TOP 1 * from CLIENT)
to
if not exists(select TOP 1 * from CLIENT)
Also you are adding 00 to your final #B_to_int which is cast as int. so it will show C2,C3 and so on.
If you want to retain the same format, cast it to varchar
SET #B_new_value = LEFT(#B,1) + '00' + CAST(#B_to_int as varchar)
Above line will work only till the count is 9. and then it will continue replicating itself with 1 because 10 will be 0010 and final output will be C0010. To eliminate this issue, use replicate and replicate 0 until 3 characters.
SET #B_new_value = LEFT(#B,1) + REPLICATE('0',3-LEN(#B_to_int)) + #B_to_int
Good Luck.
The other answers already tell you that you should be using NOT EXISTS.
This numbering scheme is quite possibly something you'll regret but you could simplify this a lot as well as making it safer in conditions of concurrency and when you run out of numbers by just doing
CREATE PROCEDURE ADD_CL(#nom VARCHAR(20),
#dn DATE)
AS
BEGIN
DECLARE #B VARCHAR(5);
SET XACT_ABORT ON;
BEGIN TRAN
SELECT #B = FORMAT(1 + RIGHT(ISNULL(MAX(code_client), 'C000'), 3), '\C000')
FROM CLIENT WITH(ROWLOCK, UPDLOCK, HOLDLOCK);
IF ( LEN(#B) > 4 )
THROW 50000, 'Exceeded range',1;
INSERT INTO CLIENT
VALUES (#B,
#nom,
#dn);
COMMIT
END
I believe the following should be 'NOT EXISTS'
if EXISTS(select TOP 1 * from CLIENT)
I am using MS SQL Server 2005 at work to build a database. I have been told that most tables will hold 1,000,000 to 500,000,000 rows of data in the near future after it is built... I have not worked with datasets this large. Most of the time I don't even know what I should be considering to figure out what the best answer might be for ways to set up schema, queries, stuff.
So... I need to know the start and end dates for something and a value that is associated with in ID during that time frame. SO... we can the table up two different ways:
create table xxx_test2 (id int identity(1,1), groupid int, dt datetime, i int)
create table xxx_test2 (id int identity(1,1), groupid int, start_dt datetime, end_dt datetime, i int)
Which is better? How do I define better? I filled the first table with about 100,000 rows of data and it takes about 10-12 seconds to set up in the format of the second table depending on the query...
select y.groupid,
y.dt as [start],
z.dt as [end],
(case when z.dt is null then 1 else 0 end) as latest,
y.i
from #x as y
outer apply (select top 1 *
from #x as x
where x.groupid = y.groupid and
x.dt > y.dt
order by x.dt asc) as z
or
http://consultingblogs.emc.com/jamiethomson/archive/2005/01/10/t-sql-deriving-start-and-end-date-from-a-single-effective-date.aspx
Buuuuut... with the second table.... to insert a new row, I have to go look and see if there is a previous row and then if so update its end date. So... is it a question of performance when retrieving data vs insert/update things? It seems silly to store that end date twice but maybe...... not? What things should I be looking at?
this is what i used to generate my fake data... if you want to play with it for some reason (if you change the maximum of the random number to something higher it will generate the fake stuff a lot faster):
declare #dt datetime
declare #i int
declare #id int
set #id = 1
declare #rowcount int
set #rowcount = 0
declare #numrows int
while (#rowcount<100000)
begin
set #i = 1
set #dt = getdate()
set #numrows = Cast(((5 + 1) - 1) *
Rand() + 1 As tinyint)
while #i<=#numrows
begin
insert into #x values (#id, dateadd(d,#i,#dt), #i)
set #i = #i + 1
end
set #rowcount = #rowcount + #numrows
set #id = #id + 1
print #rowcount
end
For your purposes, I think option 2 is the way to go for table design. This gives you flexibility, and will save you tons of work.
Having the effective date and end date will allow you to have a query that will only return currently effective data by having this in your where clause:
where sysdate between effectivedate and enddate
You can also then use it to join with other tables in a time-sensitive way.
Provided you set up the key properly and provide the right indexes, performance (on this table at least) should not be a problem.
for anyone who can use LEAD Analytic function of SQL Server 2012 (or Oracle, DB2, ...), retrieving data from the 1st table (that uses only 1 date column) would be much much quicker than without this feature:
select
groupid,
dt "start",
lead(dt) over (partition by groupid order by dt) "end",
case when lead(dt) over (partition by groupid order by dt) is null
then 1 else 0 end "latest",
i
from x
I have a table that receives data on an hourly basis. Part of this import process writes the timestamp of the import to the table. My question is, how can I build a query to produce a result set of the periods of time when the import did not write to the table?
My first thought is to have a table of static int and just do an outer join and look for nulls on the right side, but this seems kind of sloppy. Is there a more dynamic way to produce a result set for the times the import failed based on the timestamp?
This is a MS SQL 2000 box.
Update: I think I've got it. The two answers already provided are great, but instead what I'm working on is a function that returns a table of the values I am looking for for a given time frame. Once I get it finished I'll post the solution here.
Here's a slightly modified solution from this article in my blog:
Flattening timespans: SQL Server
DECLARE #t TABLE
(
q_start DATETIME NOT NULL,
q_end DATETIME NOT NULL
)
DECLARE #qs DATETIME
DECLARE #qe DATETIME
DECLARE #ms DATETIME
DECLARE #me DATETIME
DECLARE cr_span CURSOR FAST_FORWARD
FOR
SELECT s_timestamp AS q_start,
DATEADD(minute, 1, s_timestamp) AS q_end
FROM [20090611_timespans].t_span
ORDER BY
q_start
OPEN cr_span
FETCH NEXT
FROM cr_span
INTO #qs, #qe
SET #ms = #qs
SET #me = #qe
WHILE ##FETCH_STATUS = 0
BEGIN
FETCH NEXT
FROM cr_span
INTO #qs, #qe
IF #qs > #me
BEGIN
INSERT
INTO #t
VALUES (#ms, #me)
SET #ms = #qs
END
SET #me = CASE WHEN #qe > #me THEN #qe ELSE #me END
END
IF #ms IS NOT NULL
BEGIN
INSERT
INTO #t
VALUES (#ms, #me)
END
CLOSE cr_span
This will return you the consecutive ranges when updates did happen (with a minute resolution).
If you have an index on your timestamp field, you may issue the following query:
SELECT *
FROM records ro
WHERE NOT EXISTS
(
SELECT NULL
FROM records ri
WHERE ri.timestamp >= DATEADD(minute, -1, ro.timestamp)
AND ri.timestamp < ro.timestamp
)
I was thinking something like this:
select 'Start' MissingStatus, o1.LastUpdate MissingStart
from Orders o1
left join Orders o2
on o1.LastUpdate between
dateadd(ss,1,o2.LastUpdate) and dateadd(hh,1,o2.LastUpdate)
where o2.LastUpdate is null
union all
select 'End', o1.LastUpdate MissingEnd
from Orders o1
left join Orders o2
on o1.LastUpdate between
dateadd(hh,-1,o2.LastUpdate) and dateadd(ss,-1,o2.LastUpdate)
where o2.LastUpdate is null
order by 2