I have a PostgreSQL database with posts and I am trying to implement pagination for it.
The table looks like this:
postid | title | author | created
where created has the type timestamp without timezone.
My query looks like
SELECT * from posts
WHERE extract(EPOCH FROM created) < :limit
ORDER BY created DESC
LIMIT 3
Here :limit is a long in java which I pass as a parameter.
However, I always retrieve the same three newest posts, even if I have a limit smaller than the timestamp of the three posts. So I guess that the extract(EPOCH FROM created) part is wrong but I do not know how to fix it.
Your code looks fine, and should do what you want, provided that :limit is really what you think it is.
I would, however, suggest moving the conversion to the right operand rather than converting the stored value to epoch. This is much more efficient, and may take advantage of an index on the timestamp column:
SELECT *
from posts
WHERE created < date'1970-01-01' + :limit * interval '1 second'
ORDER BY created DESC
LIMIT 3
Or:
WHERE created < to_timestamp(:limit::bigint)
A possible problem is that :limit is given in milliseconds rather than seconds. If so:
WHERE created < to_timestamp((:limit/1000)::bigint)
Related
I'm doing pagination over a feed of data with a date (timestamptz) used as the cursor. The feed consists of multiple different types of posts.
One of the queries (simplified) look like this:
SELECT
a.id, a.title,
EXTRACT (EPOCH FROM a.created) * 1000 AS stamp
FROM
table a
ORDER BY
a.created DESC
LIMIT 2
Which returns:
722 Hello text 1 1460040343523.98
721 Hello text 2 1460027168427.71
Now, (for whatever reason) I only want to get the posts earlier than id 721, I do:
SELECT
a.id, a.title,
EXTRACT (EPOCH FROM a.created) * 1000 AS stamp
FROM
table a
WHERE EXTRACT(EPOCH FROM a.created) * 1000 > 1460027168427.71
ORDER BY
a.created DESC
LIMIT 2
Note the 1460027168427.71 > 1460027168427.71
Result: (no difference)
722 Hello text 1 1460040343523.98
721 Hello text 2 1460027168427.71
The comparison of the timestamps does not work, behaving like a >= rather like a >.
Note, I've tried extracting the EPOCH only, comparing the dates without any type conversion - same result. I did however note that when I got date strings of the type:
2016-04-07T11:06:08.427Z
it end up comparing
2016-04-07T11:06:08.427Z > 2016-04-07 11:06:08.427713+00
which makes me think it's a precision issue, since 427000 actually is less than 427713.
I'm grateful for any help with this (seemingly basic) issue. And before everyone telling me to use the id's to traverse the feed - I can't, because the feed is built out of multiple different sources - with the timestamp as the common point.
I know its late but Ive just had the same problem with my feed query today:
# ...
WHERE post."updatedAt" > '2019-07-29 13:57:47'
was returning like >= too
the solution was increment one second with the date/time operator found on the docs: https://www.postgresql.org/docs/9.4/functions-datetime.html
The final where looks like:
# ...
WHERE post."updatedAt" > TIMESTAMP '2019-07-29 13:57:47' + INTERVAL '1 second'
Trying to use the statement:
SELECT *
FROM data.example
WHERE TIMESTAMP(timeCollected) < DATE_ADD(USEC_TO_TIMESTAMP(NOW()), 60, 'MINUTE')
to get data from my bigquery data. It seems to return same set of result even when time is not within the range. timeCollected is of the format 2015-10-29 16:05:06.
I'm trying to build a query that is meant to return is data that is not older than an hour. So data collected within the last hour should be returned, the rest should be ignored.
Using Standard SQL:
SELECT * FROM data
WHERE timestamp > TIMESTAMP_ADD(CURRENT_TIMESTAMP(), INTERVAL -60 MINUTE)
The query you made means "return to me anything that has a collection time smaller than an hour in the future" which will literally mean your whole table. You want the following (from what I got through your comment, at least) :
SELECT *
FROM data.example
WHERE TIMESTAMP(timeCollected) > DATE_ADD(USEC_TO_TIMESTAMP(NOW()), -60, 'MINUTE')
This means that any timeCollected that is NOT greater than an hour ago will not be returned. I believe this is what you want.
Also, unless you need it, Select * is not ideal in BigQuery. Since the data is saved by column, you can save money by selecting only what you need down the line. I don't know your use case, so * may be warranted though
To get table data collected within the last hour:
SELECT * FROM [data.example#-3600000--1]
https://cloud.google.com/bigquery/table-decorators
Using Standard SQL:
SELECT * FROM data WHERE timestamp > **TIMESTAMP_SUB**(CURRENT_TIMESTAMP(), INTERVAL 60 MINUTE)
I am trying to write an SQLite statement to get the closest datetime from an user input (from a WPF datepicker). I have a table IRquote(rateId, quoteDateAndTime, quoteValue).
For example, if the user enter 10/01/2000 and the database have only fixing stored for 08/01/2000, 07/01/2000 and 14/01/2000, it would return 08/01/2000, being the closest date from 10/01/2000.
Of course, I'd like it to work not only with dates but also with time.
I tried with this query, but it returns the row with the furthest date, and not the closest one:
SELECT quoteValue FROM IRquote
WHERE rateId = '" + pRefIndexTicker + "'
ORDER BY abs(datetime(quoteDateAndTime) - datetime('" + DateTimeSQLite(pFixingDate) + "')) ASC
LIMIT 1;
Note that I have a function DateTimeSQLite to transform user input to the right format.
I don't get why this does not work.
How could I do it? Thanks for your help
To get the closest date, you will need to use the strftime('%s', datetime) SQLite function.
With this example/demo, you will get the most closest date to your given date.
Note that the date 2015-06-25 10:00:00 is the input datetime that the user selected.
select t.ID, t.Price, t.PriceDate,
abs(strftime('%s','2015-06-25 10:00:00') - strftime('%s', t.PriceDate)) as 'ClosestDate'
from Test t
order by abs(strftime('%s','2015-06-25 10:00:00') - strftime('%s', PriceDate))
limit 1;
SQL explanation:
We use the strftime('%s') - strftime('%s') to calculate the difference, in seconds, between the two dates (Note: it has to be '%s', not '%S'). Since this can be either positive or negative, we also need to use the abs function to make it all positive to ensure that our order by and subsequent limit 1 sections work correct.
If the table is big, and there is an index on the datetime column, this will use the index to get the 2 closest rows (above and below the supplied value) and will be more efficient:
select *
from
( select *
from
( select t.ID, t.Price, t.PriceDate
from Test t
where t.PriceDate <= datetime('2015-06-23 10:00:00')
order by t.PriceDate desc
limit 1
) d
union all
select * from
( select t.ID, t.Price, t.PriceDate
from Test t
where t.PriceDate > datetime('2015-06-23 10:00:00')
order by t.PriceDate asc
limit 1
) a
) x
order by abs(julianday('2015-06-23 10:00:00') - julianday(PriceDate))
limit 1 ;
Tested in SQLfiddle.
Another useful solution is using BETWEEN operator, if you can determine upper and lower bounds for your time/date query. I encountered this solution just recently here in this link. This is what i've used for my application on a time column named t (changing code for date column and date function is not difficult):
select *
from myTable
where t BETWEEN '09:35:00' and '09:45:00'
order by ABS(strftime('%s',t) - strftime('%s','09:40:00')) asc
limit 1
Also, i must correct my comment on above post. I tried a simple examination of speed of these 3 approaches proposed by #BerndLinde, #ypercubeᵀᴹ and me . I have around 500 tables with 150 rows in each and medium hardware in my PC. The result is:
Solution 1 (using strftime) takes around 12 seconds.
Adding index of column t to solution 1 improves speed by around 30% and takes around 8 seconds. I didn't face any improvement for using index of time(t).
Solution 2 also has around 30% of speed improvement over Solution 1 and takes around 8 seconds
Finally, Solution 3 has around 50% improvement and takes around 5.5 seconds. Adding index of column t gives a little more improvement and takes around 4.8 seconds. Index of time(t) has no effect in this solution.
Note: I'm a simple programmer and this is a simple test in .NET code. A real performance test must consider more professional aspects, which i'm not aware of them. There was also some computations in my code, after querying and reading from database. Also, as #ypercubeᵀᴹ states, this result my not work for large amount of data.
I have a table which has job listings, which when displayed are normally ordered by the created_at field descending. I am in the process of adding a "featured" boolean flag which would add the ability for customers to get more visibility to their job listing. I'd like to have the featured listings pinned to the top of the search results if the job is less than X days old. How would I modify by existing query to support this?
Jobs.where("expiration_date >= ? and published = ?", Date.today, true).order("created_at DESC")
Current query pulls back all current, published jobs, ordered by created_at.
Unlike some other databases (like Oracle) PostgreSQL has a fully functional boolean type. You can use it directly in an ORDER BY clause without applying a CASE statement - those are great for more complex situations.
Sort order for boolean values is:
FALSE -> TRUE -> NULL
If you ORDER BY bool_expressionDESC, you invert the order to:
NULL -> TRUE -> FALSE
If you want TRUE first and NULL last, use the NULLS LAST clause of ORDER BY:
ORDER BY (featured AND created_at > now() - interval '11 days') DESC NULLS LAST
, created_at DESC
Of course, NULLS LAST is only relevant if featured or created_at can be NULL. If the columns are defined NOT NULL, then don't bother.
Also, FALSE would be sorted before NULL. If you don't want to distinguish between these two, you are either back to a CASE statement, or you can throw in NULLIF() or COALESCE().
ORDER BY NULLIF(featured AND created_at > now() - interval '11 days'), FALSE)
DESC NULLS LAST
, created_at DESC
Performance
Note, how I used:
created_at > now() - interval '11 days'
and not:
now() - created_at < interval '11 days'
In the first example, the expression to the right is a constant that is calculated once. Then an index can be utilized to look up matching rows. Very efficient.
The latter cannot usually be used with an index. A value has to be computed for every single row, before it can be checked against the constant expression to the right. Don't do this if you can avoid it. Ever!
Not sure what you want to achieve here.
I guess you'll be paginating the results. If so, and you want to display featured jobs always on top, regardless of the page, then you should pull them from the DB separately. If you just want to display them on the first page, order by published like this :
Jobs.where("expiration_date >= ?", Date.today).order("published DESC, created_at DESC")
If you want to pull them separately :
#featured_jobs = Jobs.where("expiration_date >= ? and published = ?", Date.today, true).order("created_at DESC")
#regular_jobs = Jobs.where("expiration_date >= ? AND published = ?", Date.today, false).order("created_at DESC") #Paginate them too ... depends on the gem you're using
What's the best way to select only those rows from the table that have been created in last 7 days?
There are dozens of time and date functions in MySQL and I'm a little bit confused about what's the easiest way to do this.
For the sake of this question, assume that you have a table called "my_table" and it contains a row "created_at" which is a DATETIME.
SELECT * FROM my_table WHERE ...
What would you fill in the WHERE clause?
WHERE DATEDIFF(NOW(), created_at) <= 7;
I like it because it reads: "Where the Difference in Date between Now and when it was created is at most 7 (days)." in my own head
...WHERE created_at >= Date_Add(now(), INTERVAL -7 DAY)
This is my preferred way because it's so...clear. But ADDDATE is fine too (and you can use the INTERVAL form with that for clarity as well; its default is days so you see people leaving it off). You don't want to do a calculation on created_at and compare it to now() because that requires the computation on created_at on each row (assuming MySQL doesn't optimise it out), whereas modifying now() and comparing to an unmodified created_at means MySQL does that bit once and uses the result when comparing against rows, not to mention indexes.
...... WHERE created_at >= DATE_SUB(CURRENT_DATE, INTERVAL 7 DAY)
hopefully that will help
WHERE ADDDATE(datefield, 7) > NOW();
SELECT * FROM my_table
WHERE DATE(created_at) >= SUBDATE(DATE(NOW()), 7)
SELECT * FROM my_table WHERE my_table.datefield > sysdate - 7