I have a SQL Server table with just 3 columns, one of which is of type varbinary. The data in this column is actually a Json document which among other properties contains information about when the data was last modified. Unfortunately the SQL table itself does not contain information about when its rows were modified.
Now when doing sorting and filtering of the data I of course don't want fetch all rows in order to find e.g. the latest 100 entries.
So my question is: does SQL Server somehow remember when a row was added/modified? I have tried adding a timestamp and this is applied to all existing rows but this is applied randomly I think, because the sorting doesn't work. I don't need a datetime or anything, I just want to be able sort the records based on when they were last modified.
Thanks
For those looking to insert a tamestamp column of type DateTime into an existing DB table, you can do this like so:
ALTER TABLE TestTable
ADD DateInserted DATETIME NOT NULL DEFAULT (GETDATE());
The existing records will automatically get the value equal to the date/time of the moment when column is added.
New records will get up-to-date value upon insertion.
SQL Server will not track historically when a row was inserted or modified so you need to rely on the JSON data to figure that out yourself. You are going to need a new column to make this efficient to query. Once you have your new column you have some options:
Loop through all your records populating the new column with the relevant value from the JSON data.
If your version of SQL Server is recent enough, you can query the JSON data directly. Populate this column using a query like this:
UPDATE MyTable
SET MyNewColumn = JSON_VALUE(JsonDataColumn, '$.Customer.DateCreated')
The downside of this method is that you need to maintain this
Make SQL Server compute the value from the JSON automatically, for example:
ALTER TABLE MyTable
ADD MyNewColumn AS JSON_VALUE(JsonDataColumn, '$.Customer.DateCreated')
And, create an index to make it efficient:
CREATE INDEX IX_MyTable_MyNewColumn
ON MyTable(MyNewColumn)
Use a new column CreatedDate and store datetime every time you make an Insert.
You could use GetDate() for inserting date in the column.
A UpdatedDate column can be used for updates.
in order to find e.g. the latest 100 entries.
Timestamp is indeed what you need.
It's ever-increasing value, it's updated automatically, so you are always able to find all last modified/inserted rows.
Here is an example:
create table dbo.test1 (id int);
insert into dbo.test1 values(1), (2), (3);
alter table dbo.test1 add ts timestamp;
update dbo.test1
set id = 10
where id = 2
select top 1 *
from dbo.test1
order by ts desc;
--id ts
--10 0x000000001FCFABD2
insert into dbo.test1 (id)
values (100);
select top 1 *
from dbo.test1
order by ts desc;
--id ts
--100 0x000000001FCFABD3
As you see, you always get the last modified/inserted row.
For your purpose just use
select top 100 *
...
order by ts desc;
Thanks. Apparently I didn't look hard enough before I posted this question. The question has been asked a couple of times before and the answer is: Nope! There is no easy solution to this.
SQL Server does not keep track of when a record was created or modified, which was somehow what I was looking for. So I will go for the next best solution, which is probably to create a datetime column, retrieve the modified date from the Json document and then update the record. Or rather, the 1,4 million records:-(
Related
I am trying to manually add records to a SQL table
However in this table we have a column called Trackseq this column determines how the data is viewed on our CMS system.
The highest TrackSeq will appear first.
I want to manually add a new row but i want the code to check based on CLNTID what the current trackSeq is and add one to it. So if the trackseq of the last record was 10 I want the new record to go in with a trackseq of 11.
Here is my code.
INSERT INTO tbl_CommTracking ( CLNTID,
TRACKSEQ,
COMMDATE,
COMMTIME,
PRODCODE,
COMMTYPE,
EMPLOYEEID,
COMMDETAILS,
COMMSTATUS)
VALUES ('0000005566','999',GETDATE(),GETDATE(),'BS','Note','0000000786','Testing a manual import','A')
Thanks
Assuming TRACKSEQ is a numeric format, you can read the actual highest value and store it in a variable to use in your insert statement.
Please note that, in case of high concurrency, you may want to synchronize this execution in order to avoid duplicate #ts values.
As stated by #Tim Biegeleisen above, an IDENTITY column is best suited for this kind of tasks, but keep in mind that it may leave holes in number sequence in case of DELETE and failed INSERTS.
Eg:
DECLARE #ts int; --or whatever numeric datatype it is
SELECT #ts = MAX(TRACKSEQ) + 1 FROM tbl_CommTracking
INSERT INTO tbl_CommTracking (
--...
TRACKSEQ
--..
)
VALUES (
--...
#ts
--...
)
You should probably make TRACKSEQ an auto increment column. First drop your current TRACKSEQ column, then add it back:
ALTER TABLE tbl_CommTrackingDROP COLUMN TRACKSEQ;
ALTER TABLE tbl_CommTracking ADD TRACKSEQ INT IDENTITY;
Note that typically you would also make TRACKSEQ the primary key of the table. If you don't want to, that's OK, but then you'll have to make sure you can generate your own unique values for the CLNTID column.
It isn't entirely clear why you need this; if you just want the latest records, use the date/timestamp column available, which would have been set during insertion.
I want to delete new record if the same record created before.
My columns are date, time and MsgLog. If date and time are same, I want to delete new one.
I need help .
You can check in the table whether that value exists or not in the column using a query. If it exists, you can show message that a record already exists.
To prevent such kind of erroneous additions you can add restriction to your table to ensure unique #Date #Time pairs; if you don't want to change data structure (e.g. you want to add records with such restrictions once or twice) you can exploit insert select counstruction.
-- MS SQL version, check your DBMS
insert into MyTable(
Date,
Time,
MsgLog)
select #Date,
#Time,
#MsgLog
where not exists(
select 1
from MyTable
where (#Date = Date) and
(#Time = Time)
)
P.S. want to delete new one equals to do not insert new one
You should create a unique constraint in the DB level to avoid invalid data no matter who writes to your DB.
It's always important to have your schema well defined. That way you're safe that no matter how many apps are using your DB or even in case someone just writes some inserts manually.
I don't know which DB are you using but in MySQL can use to following DDL
alter table MY_TABLE add unique index(date, time);
And in Oracle you can :
alter table MY_TABLE ADD CONSTRAINT constaint_name UNIQUE (date, time);
That said, you can also (not instead of) do some checks before inserting new values to avoid dealing with exceptions or to improve performance by avoiding making unnecessary access to your DB (length \ nulls for example could easily be dealt with in the application level).
You can avoid deleting by checking for duplicate while inserting.
Just modify your insert procedure like this, so no duplicates will entered.
declare #intCount as int;
select #intCount =count(MsgLog) where (date=#date) and (time =#time )
if #intCount=0
begin
'insert procedure
end
> Edited
since what you wanted is you need to delete the duplicate entries after your bulk insert. Think about this logic,
create a temporary table
Insert LogId,date,time from your table to the temp table order by date,time
now declare four variables, #preTime,#PreDate,#CurrTime,#CurrDate
Loop for each items in temp table, like this
while
begin
#pkLogID= ' Get LogID for the current row
select #currTime=time,#currDate=date from tblTemp where pkLogId=#pkLogID 'Assign Current values
'Delete condition check
if (#currDate=#preDate) and (#currTime=#preTime)
begin
delete from MAINTABLE where pkLogId=#pkLogID
end
select #preDate=#currDate,#preTime=#currTime 'Assign current values as preValues for next entries
end
The above strategy is we sorted all entries according to date and time, so duplicates will come closer, and we started to compare each entry with its previous, when match found we deleting the duplicate entry.
I'm trying to get the last id inserted into a table.
I was using
SELECT IDENT_CURRENT('TABLE')
But the problem is that it doesn't return the last inserted id, it returns the max inserted id.
For example, if i do:
INSERT INTO 'TABLA' (ID) VALUES (100)
SELECT IDENT_CURRENT('TABLE') returns 100
but then if i do
INSERT INTO 'TABLA' (ID) VALUES (50)
SELECT IDENT_CURRENT('TABLE') returns 100
and I want to get 50
I need the ID of a specific table, and I generate the id dinamically, so it's not an identity
How can i do it?
From your code, it looks like ID is not an identity (auto-increment) column, so IDENT_CURRENT isn't going to do what you are expecting.
If you want to find the last row inserted, you will need a datetime column that represents the insert time, and then you can do something like:
SELECT TOP 1 [ID] FROM TABLEA ORDER BY [InsertedDate] DESC
Edited: a few additional notes:
Your InsertedDate column should have a default set to GetDate() unless your application, stored procs or whatever you use to perform inserts will be responsible for setting the value
The reason I said your ID is not an identity/auto-increment is because you are inserting a value into it. This is only possible if you turn identity insert off.
SQL Server does not keep track of the last value inserted into an IDENTITY column, particularly when you use SET IDENTITY_INSERT ON;. But if you are manually specifying the value you are inserting, you don't need SQL Server to tell you what it is. You already know what it is, because you just specified it explicitly in the INSERT statement.
If you can't get your code to keep track of the value it just inserted, and can't change the table to have a DateInserted column with a default of CURRENT_TIMESTAMP (which would allow you to see which row was inserted last), perhaps you could add a trigger to the table that logs all inserts.
SELECT SCOPE_IDENTITY()
will return the last value inserted in current session.
Edit
Then what you are doing is the best way to go just make sure that the ID Column is an IDENTITY Column, IDENT_CURRENT('Table_name'), ##IDENTITY and SCOPE_IDENTITY() returns last value generated by the Identity column.
If the ID column is not an Identity Column, all of these functions will return NULL.
I am using MS SQL Server 2008 and I have an sql table with some data that is inserted daily at 6 am by an sql job. The problem I have is that some data has been inserted separately into the job and I need to know when this data was added.
Is there a query I can run that will show me this?
I think the short answer is NO, there's no magic, ad hoc SQL query that will let you go back after the fact and find out when a row was inserted.
If you want to know when a row is inserted, the easiest thing would be to simply add a date or timestamp field with a default value (like getDate()) that automatically fills in the date/time when the row is inserted.
There are, of course, SQL logs available that will let you track when rows are inserted, updated, deleted, etc., but those require set up and maintenance.
Third option would be to have the program that's inserting the data perform some logging.
Add a date field to the table. You can give it a default value of GETDATE()
Then ORDER BY that field.
SELECT Column1, Column2, NewDateColumn
FROM YourTable
ORDER BY NewDateColumn
what i would do is :
/* add new column to keep inserted row date */
ALTER TABLE [schemaName].[tableName] ADD [RecTime] DATETIME;
/* update the existing rows with the current date since there is no way to guess their insertion date */
UPDATE [schemaName].[tableName] SET [RecTime] = GETDATE();
/* and set a constraint to the RecTime column to set current date on every new row added */
ALTER TABLE [schemaName].[tableName] ADD CONSTRAINT [DF_tableName_RecTime] DEFAULT (GETDATE()) FOR [RecTime]
then you can get those rows like :
SELECT *
FROM [schemaName].[tableName]
WHERE NOT(DATEPART(hh, RecTime) = 6 AND DATEPART(mi, RecTime) <= 20)
you can 'play' with '20' if you know how long sql job run
you probably need to look at SQL CREATE TRIGGER to add the logic to know when the data is being added and log that info in another table for further actions. Without further details I am not sure we can say more than that.
As you're referring to data which has already been inserted, the answer is No, unless you already have a datetime column which has a default value of GETDATE(). The best you can manage after the event has occurred is to look at the sequence of rows and determine that it was between two known times.
I have a table in a database that represents dates textually (i.e. "2008-11-09") and I would like to replace them with the UNIX timestamp. However, I don't think that MySQL is capable of doing the conversion on its own, so I'd like to write a little script to do the conversion. The way I can think to do it involves getting all the records in the table, iterating through them, and updating the database records. However, with no primary key, I can't easily get the exact record I need to update.
Is there a way to get MySQL to assign temporary IDs to records during a SELECT so that I refer back to them when doing UPDATEs?
Does this not do it?
UPDATE
MyTable
SET
MyTimeStamp = UNIX_TIMESTAMP(MyDateTime);
If for some reason you do have to iterate (the other answers cover the situation where you don't), I can think of two ways to do it (these aren't MySQL-specific):
Add a column to the table that's an auto-assigned number. Use that as the PK for your updates, then drop the column afterwards (or just keep it around for future use).
In a table with no defined PK, as long as there are no exact duplicate rows, you can use the entire row as a composite PK; just use every column in the row as your distinguishing characteristic. i.e., if the table has 3 columns, "name", "address", and "updated", do the following:
UPDATE mytable SET updated = [timestamp value] WHERE name = [name] AND address = [address] AND timestamp = [old timestamp]
Many data access frameworks use this exact strategy to implement optimistic concurrency.
No, you should be able to do this with a single update statement. If all of the dates are yyyy-mm-dd and they are just stored in some sort of text column instead of DATETIME, you can just move the data over. SQL would be like:
ALTER TABLE t ADD COLUMN dates DATETIME;
UPDATE t set t.dates=t.olddate;
This shouldn't be dependent on a PK because MySQL can scan through each row in the table. The only time PK's become an issue is if you need to update a single row, but the row may not be unique.
You can generate values during a SELECT using the MySQL user variables feature, but these values do not refer to the row; they're temporary parts of the result set only. You can't use them in UPDATE statements.
SET #v := 0;
SELECT #v:=#v+1, * FROM mytable;
Here's how I'd solve the problem. You're going to have to create another column for your UNIX timestamps anyway, so you can add it first. Then convert the values in the old datetime column to the UNIX timestamp and place it in the new column. Then drop the old textual datetime column.
ALTER TABLE mytable ADD COLUMN unix_timestamp INT UNSIGNED NOT NULL DEFAULT 0;
UPDATE mytable
SET unix_timestamp = UNIX_TIMESTAMP( STR_TO_DATE( text_timestamp, '%Y-%m-%d' ) );
ALTER TABLE mytable DROP COLUMN text_timestamp;
Of course you should confirm that the conversion has been done correctly before you drop the old column!
See UNIX_TIMESTAMP() and STR_TO_DATE()