Due to some activity in my project I want to run this query in some frequency and check "where my query can fetch table timestamp is <= 1 min from current timestamp"
SQL Query to check the updated data in the table.
Even though your question is incomplete, you haven't provided your existing table structure or any queries. I'm just giving you some generic solution here which should work as long as you can convert that based on your specific need.
so, you are trying to get the difference of two-time values in minutes
Time that record was saved
Current Time
If you have a table : LogRecords with below fields:
LogId
LogMessage
LogTimestamp
then you would write your query to pull last-minute logs as :
select * from LogRecords
where DATEDIFF(MINUTE, LogTimestamp , GETDATE()) <= 1
I haven't tested this code but it should be 99% similar if it won't work. Please try and let me know.
Related
I just started using the SSIS tool and I need quick help to load data quarterly
Here`s my scenario:
I came up with a query ( source Database: DB2 ) that will extract data from 2010-01-01 to 2021-12-31,(11 years of data) however the data volume is too high ( around 300 M), so I would like to split the data source query to load data into quarter wise.
I tried year wise and still, I am getting more volume of data which my SSIS server is not able to handle.
I have created a year loop to loop it through, in that created a script task into it followed by a data flow task.
For example,
select * from tab1 where start_date >= '2010-01-01' and end_Date <= '2010-12-31'
This I would like to loop it as, ( 4 times load, 1 for each quarter)
select from tab1 where start_date >= '2010-01-01' and end_Date <= '2010-03-31'
select from tab1 where start_date >= '2010-04-01' and end_Date <= '2010-06-30'
select from tab1 where start_date >= '2010-07-01' and end_Date <= '2010-09-30'
select from tab1 where start_date >= '2010-10-01' and end_Date <= '2010-12-31'
Year-wise perfectly works fine, however, I am not getting any idea how do I need to load the data into quarter-wise.
I want to pass each quarter parameters to the source query as parameters, so overall I need to loop to 48 times ( 2010 to 2021 = 11 yrs * 4 quarters)
Any help is greatly appreciated.
I can send screenshots of what I have created for the year loop which is working perfectly fine.
I think the solution is to use the OFFSET FETCH clause to iterate over data. Why looping over data quarterly while using a number of rows is more precise (each iteration will handle the same amount of data). A step-by-step guide is provided in the following article:
SQL OFFSET FETCH Feature: Loading Large Volumes of Data Using Limited Resources with SSIS
One thing worth mentioning is that the article handles an SQL Server source, while you are using DB2. Then you should take into consideration any syntax difference while using the OFFSET FETCH clause:
Getting top n to n rows from db2
Example: Using the OFFSET clause with a cursor
Similar questions:
Reading Huge volume of data from Sqlite to SQL Server fails at pre-execute
Loop 10 records at a time and assign it to variable
SSIS failing to save packages and reboots Visual Studio
If I have two tables:
items
Id VARCHAR(26)
CreateAt bigint(20)
Type VARCHAR(26)
expiry
Id VARCHAR(26)
Expiry bigint(20)
The items table contains when the item was created, and what type it is. Then another table, expiry, is a lookup table to say how long certain types should last for. A query is run every day to make sure that items that have expired are removed.
At the moment this query is written in our app, as programming code:
for item in items {
expiry = expiry.get(item.Type)
if (currentDate() - expiry.Expiry > item.CreateAt) {
item.delete()
}
}
This was fine when we only had a few thousand items, but now we have tens of millions it takes a significant amount of time to run. Is there a way to put this into just an SQL statement?
Assuming all date values are actually UNIX timestamps, you could write a query such as:
SELECT * -- DELETE
FROM items
WHERE EXISTS (
SELECT 1
FROM expiry
WHERE expiry.id = items.type
AND items.CreateAt + expiry.Expiry < UNIX_TIMESTAMP()
)
Replace SELECT with DELETE once you're sure that the query selects the correct rows.
If the dates stored are in seconds since the UNIX epoch, you could use this PostgreSQL query:
DELETE FROM items
USING expiry
WHERE items.type = expiry.id
AND items.createat < EXTRACT(epoch FROM current_timestamp) - expiry.expiry;
A standard SQL solution that should work anywhere would be
DELETE FROM items
WHERE items.createat < EXTRACT(epoch FROM current_timestamp)
- (SELECT expiry.expiry FROM expiry
WHERE expiry.id = items.type);
That can be less efficient in PostgreSQL.
Your code is getting slow because you do the join between the tables outside the database.
Second slowing aspect is that you delete the items 1 by 1.
So using the compact delete statements which were provided is the correct solution.
It seems that you are using something like python-sqlalchemy. There the code would be something like:
items.delete().\
where(items.c.type==\
select([expiry.c.id]).\
where(currentDate() - expiry.Expiry > item.c.CreateAt ))
I have a column (ROW_UPDATE_TIME) in a table where it stores the timestamp when an update happens in this table.
I'd like to know how to check rows that the timestamp is today.
This is what I'm using now, but it's not a pretty solution I think:
SELECT
*
FROM
TABLE
WHERE
ROW_UPDATE_TIME BETWEEN (CURRENT TIMESTAMP - 1 DAY) AND (CURRENT TIMESTAMP + 1 DAY);
Is there a better solution, example: ROW_UPDATE_TIME = CURRENT DATE, or something like that?
Found it:
SELECT
*
FROM
TABLE
WHERE
DATE(ROW_UPDATE_TIME) = CURRENT DATE;
The first version you have provided will not return you the results you expect, because you will get in the result timestamps from today or tomorrow, depends on the hour you run it.
Use the query below to get the results from today:
SELECT
*
FROM
table
WHERE
row_update_time
BETWEEN TIMESTAMP(CURRENT_DATE,'00:00:00')
AND TIMESTAMP(CURRENT_DATE,'23:59:59')
Avoid applying a function to a column you compare in the where clause(DATE(row_update_time) = CURRENT_DATE) . That will cause the optimizer to run the function against each row, just to allocate the data you need. It could slow down the query dramatically. Try to run explain against the two versions and you will see what I mean.
I want to query order information from 4 days ago. I cannot quite get the query to use the full 24hours of the day though:
<cfquery name="rsByDate" datasource="#request.dsn#">
SELECT tbl_customers.cst_FirstName
, tbl_customers.cst_LastName
, tbl_customers.cst_Email
, tbl_orders.order_ID
, tbl_orders.order_Date
FROM tbl_customers INNER JOIN tbl_orders
ON tbl_customers.cst_ID = tbl_orders.order_CustomerID
WHERE tbl_orders.order_Date >= #CreateODBCDateTime(DateAdd("d",-5,Now()))#
AND tbl_orders.order_Date <= #CreateODBCDateTime(DateAdd("d",-4,Now()))#
AND order_Status = 3
ORDER BY tbl_orders.order_Date DESC
</cfquery>
What I'm hoping to achieve is to perform this query and then use cfmail to ask the customers to review us four days after their order is marked as status 3 'shipped'.
I think I have the date settings not quite right. Where am I going wrong?
It's because you're basing your filter dates on now(), which has a time component. So now()-4 for me is 5/11/2012, but at 12:10pm. You only want to use the date part for the filter.
You should never hard-code your dynamic values in your SQL string anyhow, so firstly use <cfqueryparam> to parameterise your dynamic values, and then use a CF_SQL_DATE as the type, which should only pass the date through, not the date/time. I am not in a position to test this for you at the moment, but try that... if it still passes the time part of the date/time, then create a date object with just the date part, using createDate().
HI all,
i have one sql table and field for that table is
id
name
expireydate
Now i want only those record which one is expired within 45 days or 30 days.
how can i do with sql query .?
I have not much more exp with sql .
Thanks in advance,
If you are using mysql then try DATEDIFF.
for 45 days
select * from `table` where DATEDIFF(now(),expireydate)<=45;
for 30 days
select * from `table` where DATEDIFF(now(),expireydate)<=30;
In oracle - will do the trick instead of datediff and SYSDATE instead of now().[not sure]
In sql server DateDiff is quite different you have to provide unit in which difference to be taken out from 2 dates.
DATEDIFF(datepart,startdate,enddate)
to get current date try one of this: CURRENT_TIMESTAMP or GETDATE() or {fn NOW()}
You can use a simple SELECT * FROM yourtable WHERE expireydate < "some formula calculating today+30 or 45 days".
Simple comparison will work there, the tricky part is to write this last bit concerning the date you want to compare to. It'll depend of your environment and how you stored the "expireydate" in the database.
Try Below:-
SELECT * FROM MYTABLE WHERE (expireydate in days) < ((CURRENTDATE in days)+ 45)
Do not execute directly! Depending of your database, way of obtaining a date in days will be different. Go look at your database manual or please precise what is your database.