To extract the week of a given year we can use:
SELECT EXTRACT(WEEK FROM timestamp '2014-02-16 20:38:40');
However, I am trying to group weeks together in a bit of an odd format. My start of a week would begin on Mondays at 4am and would conclude the following Monday at 3:59:59am.
Ideally, I would like to create a query that provides a start and end date, then groups the total sales for that period by the weeks laid out above.
Example:
SELECT
(some custom week date),
SUM(sales)
FROM salesTable
WHERE
startDate BETWEEN 'DATE 1' AND 'DATE 2'
I am not looking to change the EXTRACT() function, rather create a query that would pull from the following sample table and output the sample results.
If 'DATE 1' in query was '2014-07-01' AND 'DATE 2' was '2014-08-18':
Sample Table:
itemID | timeSold | price
------------------------------------
1 | 2014-08-13 09:13:00 | 12.45
2 | 2014-08-15 12:33:00 | 20.00
3 | 2014-08-05 18:33:00 | 10.00
4 | 2014-07-31 04:00:00 | 30.00
Desired result:
weekBegin | priceTotal
----------------------------------
2014-07-28 04:00:00 | 30.00
2014-08-04 04:00:00 | 10.00
2014-08-11 04:00:00 | 32.45
Produces your desired output:
SELECT date_trunc('week', time_sold - interval '4h')
+ interval '4h' AS week_begin
, sum(price) AS price_total
FROM tbl
WHERE time_sold >= '2014-07-01 0:0'::timestamp
AND time_sold < '2014-08-19 0:0'::timestamp -- start of next day
GROUP BY 1
ORDER BY 1;
db<>fiddle here (extended with a row that actually shows the difference)
Old sqlfiddle
Explanation
date_trunc() is the superior tool here. You are not interested in week numbers, but in actual timestamps.
The "trick" is to subtract 4 hours from selected timestamps before extracting the week - thereby shifting the time frame towards the earlier bound of the ISO week. To produce the desired display, add the same 4 hours back to the truncated timestamps.
But apply the WHERE condition on unmodified timestamps. Also, never use BETWEEN with timestamps, which have fractional digits. Use the WHERE conditions like presented above. See:
Unexpected results from SQL query with BETWEEN timestamps
Operating with data type timestamp, i.e. with (shifted) "weeks" according to the current time zone. You might want to work with timestamptz instead. See:
Ignoring time zones altogether in Rails and PostgreSQL
Related
I have a bunch of data sitting in a Postgres database for a website I am building. Problem is, I don't really know what I should do with the date information. For example, I have a table called events that has a date column that stores date information as a string until I can figure out what to do with it.
The reason they are in string format is unfortunately for the topic of my website, their is not a good API, so I had to scrape data. Here's what some of the data looks like inside the column:
| Date |
|-------------------------------------|
| Friday 07.30.2021 at 08:30 AM ET |
| Wednesday 04.07.2021 at 10:00 PM ET |
| Saturday 03.27.2010 |
| Monday 01.11.2010 |
| Saturday 02.09.2019 at 05:00 PM ET |
| Wednesday 03.31.2010 |
It would have been nice to have every row have the time with it, but a lot of entries don't. I have no problem doing some string manipulation on the data to get them into a certain format where they can be turned into a date, I am just somewhat stumped on what I should do next.
What would you do in this situation if you were restricted to the data seen in the events table? Would you store it as UTC? How would you handle the dates without a time? Would you give up and just display everything as EST dates regardless of where the user lives (lol)?
It would be nice to use these dates to display correctly for anyone anywhere in the world, but it looks like I might be pigeonholed because of the dates that don't have a time associated with them.
Converting your mishmash free form looking date to a standard timestamp is not all that daunting as it seems. Your samples indicate you have a string with 5 separate pieces of information: day name, date (mm.dd.yyyy), literal (at),time of day, day part (AM,PM) and some code for timezone each separated by spaces. But for them to be useful the first step is splitting this into the individual parts. For that us a regular expression to ensure single spaces then use string_to_array to create an array of up to 6 elements. This gives:
+--------------------------+--------+----------------------------------+
| Field | Array | Action / |
| | index | Usage |
+--------------------------+--------+----------------------------------+
| day name | 1 | ignore |
| date | 2 | cast to date |
| literal 'at' | 3 | ignore |
| time of day | 4 | interval as hours since midnight |
| AM/PM | 5 | adjustment for time of day |
| some code for timezone | 6 | ??? |
+--------------------------+--------+----------------------------------+
Putting it all together we arrive at:
with test_data ( stg ) as
( values ('Friday 07.30.2021 at 08:30 AM ET' )
, ('Wednesday 04.07.2021 at 10:00 PM ET')
, ('Saturday 03.27.2010' )
, ('Monday 01.11.2010' )
, ('Saturday 02.09.2019 at 05:00 PM ET' )
, ('Wednesday 03.31.2010')
)
-- <<< Your query begins here >>>
, stg_array( strings) as
( select string_to_array(regexp_replace(stg, '( ){1,}',' ','g'), ' ' )
from test_data --<<< your actual table >>>
) -- select * from stg_array
, as_columns( dt, tod_interval, adj_interval, tz) as
( select strings[2]::date
, case when array_length(strings,1) >= 4
then strings[4]::interval
else '00:00':: interval
end
, case when array_length(strings,1) >= 5 then
case when strings[5]='PM'
then interval '12 hours'
else interval '0 hours'
end
else interval '0 hours'
end
, case when array_length(strings,1) >= 6
then strings[6]
else current_setting('TIMEZONE')
end
from stg_array
)
select dt + tod_interval + adj_interval dtts, tz
from as_columns;
This gives the corresponding timestamp for date, time, and AM/PM indicator (in the current timezone) for items without a timezone specifies. For those containing a timezone code, you will have to convert to a proper timezone name. Note ET is not a valid timezone name nor a valid abbreviation. Perhaps a lookup table. See example here; it also contains a regexp based solution. Also the example in run on db<>fiddle. Their server is in the UK, thus the timezone.
I have a table with timestamp and someother metrics like riskscore and current of a machine.Here plant shift starts # 08:00 am and ends # next day 08:00am.
i want to group the data by day(shift: 08:00am to nextday 08:00am) of timetamp and label it as shift start date.i have a 6months of data.)
expected output:
machine | date | avg_riskscore | avg_current
2 | 2020-12-02 | 25.5 | 10
here this record is group of data between '2020-12-02 08:00:00' and '2020-12-03 08:00:00' and should insert with date '2020-12-02'
here i need to aggregate the 6 months of data like this.
DB Fiddle
You can just offset the timestamp by 8 hours, then truncate to date and aggregate. Based on your fiddle, that would be:
select
equipment_id,
(telemetry_time - interval '8 hour')::date as date,
avg(riskscore) as avg_riskscore,
avg(i_rms) as avg_i_rms
from telemetry_test
group by equipment_id, date
I am trying to group my sum results by custom day in Postgresql.
As regular day starts at 00:00 , I would like mine to start at 04:00am , so if there is entry with time 2019-01-03 02:23 it would count into '2019-01-02' instead.
Right now my code looks like this:
Bottom part works perfectly on day type 00:00 - 23.59 , however I would like to group it by my upper range created above. I just don't know how to connect those two parts.
with dateRange as(
SELECT
generate_series(
MIN(to_date(payments2.paymenttime,'DD Mon YYYY')) + interval '4 hour',
max(to_date(payments2.paymenttime,'DD Mon YYYY')),
'24 hour') as theday
from payments2
)
select
sum(cast(payments2.servicecharge as money)) as total,
to_date(payments2.paymenttime,'DD Mon YYYY') as date
from payments2
group by date
Result like this
+------------+------------+
| total | date |
+------------+------------+
| 20 | 2019-01-01 |
+------------+------------+
| 60 | 2019-01-02 |
+------------+------------+
| 35 | 2019-01-03 |
+------------+------------+
| 21 | 2019-01-04 |
+------------+------------+
Many thanks for your help.
If I didn't misunderstand your question, you just need to subtract 4 hours from the timestamp before casting to date, you don't even need the CTE.
Something like
select
sum(cast(payments2.servicecharge as money)) as total,
(to_timestamp(payments2.paymenttime,'DD Mon YYYY HH24:MI:SS') - interval '4 hours')::date as date
from payments2
group by date
Yu may need to use a different format in the to_timestamp function depending on the format of the payments2.paymenttime string
I have a table:
ID | Name | TDate
1 | John | 1 May 2013, 8:67AM
2 | Jack | 2 May 2013, 6:43AM
3 | Adam | 3 May 2013, 9:53AM
4 | Max | 4 May 2013, 2:13AM
5 | Leny | 5 May 2013, 5:33AM
I need a query that will return all the items where TDate is a weekend. How would I write such a
query?
WHAT I HAVE SO FAR
select
table.*,
EXTRACT (DAY FROM table.tdate )
from table
I did a select using EXTRACT to just see if I can get the right values. However, EXTRACT with the parameter DAY returns the day of the month. If I instead use WEEKDAY, as per the documentation here, then I get error:
ERROR: timestamp units "weekday" not recognized
SQL state: 22023
limit 1250
EDIT
TDate has a data type of datetime (timestamp). I just wrote it like that for easy reading. But regardless of the type, I could easily cast between types if need be.
I know dates 4May and 5May are weekends (as they fall on a Saturday and a Sunday). Does firebird allow for a way to write a query that will return dates if they fall on weekends.
try this:
SELECT ID, Name, TDate
FROM your_table
WHERE EXTRACT(WEEKDAY FROM TDate) IN (6,0)
UPDATE
condition must be (0,6) not (0,1).
I want to group the result by week no and get the start date and end date for the week. I had not idea on how to do this sqlite. Could somebody help me on this.
I also really confiused the way sqlite works. Because if run the following query i get the week no as 00
SELECT strftime('%W','2012-01-01');
and week no as 01 instead of 00 for the following query
SELECT strftime('%W','2012-01-02');
Could somebody explain why the sqlite behaves like this.
Try this out:
select * from t1;
+------------+
| ADate |
+------------+
| 2012-01-04 |
| 2012-01-10 |
| 2012-01-19 |
| 2012-01-22 |
| 2012-01-01 |
| 2012-01-01 |
+------------+
select
strftime('%W', aDate) WeekNumber,
max(date(aDate, 'weekday 0', '-7 day')) WeekStart,
max(date(aDate, 'weekday 0', '-1 day')) WeekEnd,
count(*) as GroupedValues
from t1
group by WeekNumber;
+------------+------------+------------+------------+
| WeekNumber | WeekStart | WeekEnd | WeekNumber |
+------------+------------+------------+------------+
| 00 | 2011-12-25 | 2011-12-31 | 2 |
| 01 | 2012-01-01 | 2012-01-07 | 1 |
| 02 | 2012-01-08 | 2012-01-14 | 1 |
| 03 | 2012-01-15 | 2012-01-21 | 2 |
+------------+------------+------------+------------+
To be honest... I don't know why 2012-01-01 is week 0 and 2012-01-02 is week 1. Sounds very weird, particularly if the week starts on sundays! :s
try something like this:
select
strftime('%W', aDate, 'weekday 1') WeekNumber,
max(date(aDate, 'weekday 1')) WeekStart,
max(date(aDate, 'weekday 1', '+6 day')) WeekEnd,
count(*) as GroupedValues
from t1
group by WeekNumber;
I know very much late to the party, but I can explain why. The Strftime reference used, comes from C. The way C calculates things is frustrating.
Days run from 0 to 6, with 0 being sunday, 6 being saturday.
However Weeks run from Monday to Sunday.
From the C documentations:
%w Weekday as a decimal number with Sunday as 0 (0-6)
%W Week number with the first Monday as the first day of week one (00-53)
So Week 00 would be a sunday, before the first monday of the year. If the year started on a thursday, all the days till the Monday would be week 00. Not sure what the ISO standard is, I think it's first Thursday of the year is week 1.
It took me a while to resolve this as I had to account for weeks running from Thursday to Wednesday.
In short, sqlite defines week of year (%W) on Monday, according to https://sqlite.org/lang_datefunc.html and its link on strftime function (http://opengroup.org/onlinepubs/007908799/xsh/strftime.html).
Thus, if anyone still needs to group weeks starting on Sunday, just advance a day as below:
SELECT strftime('%W','2012-01-01', '+1 day')
The accepted answer can be simplified a little and allow more flexibility. Note that this_sunday includes the day itself if that day is Sunday.
SELECT
DATE(created_at, 'weekday 0') AS this_sunday,
COUNT(*) AS rows_this_week
FROM my_table
GROUP BY this_sunday;
This technique is simpler than strftime('%W', created_at) and it's more powerful. For example if you want to aggregate weeks ending on Monday:
SELECT
DATE(created_at, 'weekday 1') AS this_monday,
COUNT(*) AS rows_this_week
FROM my_table
GROUP BY this_monday;
This works thanks to the modifier 'weekday N' which moves the date to the day of week specified if it's in the future or keeps it if it's the same day.
Finally, to verify which dates are included in your week range, I recommend this method:
SELECT
DATE(created_at, 'weekday 1') AS this_monday,
MIN(DATE(created_at)) AS earliest,
MAX(DATE(created_at)) AS latest,
COUNT(*) AS rows_this_week
FROM my_table
GROUP BY this_monday;
according to SQL Lite documentation Sunday==0 and 2012-01-01 is a sunday so its spitting out 00;
take a look at the date time functions for sqllite here
http://www.sqlite.org/lang_datefunc.html
If the given date is the first of the week, then DATE('2014-07-20', 'weekday 0', '-7 days') will return '2014-07-13', which is 7 days earlier than required.
This is the best I've come up with so far:
CASE DATE(myDate, 'weekday 0')
WHEN DATE(myDate) THEN DATE(myDate)
ELSE DATE(myDate, 'weekday 0', '-7 days')
END