In my Rails app's PostgreSQL DB are records containing hourly prices for the last 10 years:
10(24 x 365) of these: "12/31/2012 01:00:00", "11.99"
The following query, groups prices by day, averages the prices in those daily groupings to create daily price averages, and returns "day", "daily average" pairs for each day:
HourlyPrice.average(:price, :group => "DATE_TRUNC('day', date)")
The problem is, the hourly prices in my source data actually reflect the price for the previous hour. So, in my data source .CSV, the day starts at the time 01:00:00 and ends at the time 24:00:00.
This conflicts with how PostgreSQL likes to save records in its DateTime column. Upon importing the CSV data, PostgreSQL converts my records containing the time 24:00:00 to 00:00:00 of the next day.
This throws off the accuracy of my Averaging Query above. To fix the query, I still want to group by day, but offset 1 hour. So, that the range averaged starts at 01:00:00 and ends with the 00:00:00 value of the next day.
Is it possible to adjust the above query to reflect this?
You could subtract one hour from date before applying the DATE_TRUNC function to it, like this:
HourlyPrice.average(:price, :group => "DATE_TRUNC('day', date - INTERVAL '1 hour')")
Related
I need to find the result of a calculation that is nothing more than the average time in days from creation to completion of a task.
In this case, using a Redshift database (looker).
I have two dates (2022/10/01 to 2022/10/21) and I need to find the average day of execution of the creation of an object from start to finish.
Previously, I was able to calculate the totals of objects created per day, but I can't bring up the average:
SELECT created::date, count(n1pk_package_id)
FROM dbt_dw.base_package
WHERE fk_company_id = 245821 and created >= '2022-10-01' and created < '2022-10-22'
GROUP BY created::date
ORDER BY created DESC
I'm not able to do the opposite way of the count to bring the average of the range of days.
Assumption:
There is a created column in your table
You want to know the 'average' of the created column
You could extract the number of days that each date is different from a base date, and then use that to determine the 'average date'. It would be something like this:
select
date '2022-10-01' + interval '1 day' * int(avg(created - date '2022-10-01'))
from table
It subtracts a date (any date will do) from created, finds the average of that value against all desired rows, converts it to days and adds it back to that same date.
Need help to get the data of particular format
We have a table which have a data which of production now we need to select the data of each day with particular time period which is differentiate between three shift A,B,C.
In our table we have a datetime column which capture's each seconds data now that data we need in shiftwise like 6am to 2pm is of A shift production count and 2pm to 10pm of shift B and 10pm to 6 am of shift C.
here i am getting the data for single day where i have written the below query which is working good.
select distinct(count(PRD_SERIAL_NUMBER)),(select convert(date,getdate())) as date,'B' as shift_name
from table_name
where status=02
and LAST_UPDATED_DATE
between (SELECT FORMAT(GETDATE(),'yyyy-MM-dd 14:01:00.000')) and
(SELECT FORMAT(GETDATE()-26,'yyyy-MM-dd 22:01:00.000'))
refer below output image 1
Here i am getting the count for single day and for upcoming days i have solution but now the question arise is i have a past 4 Month data which i need to get in datewise and shiftwise count and for the column prd_serial_number have duplicate entries so it should be in distinct.
please refer below image 2 for required output format
I have a PostgreSQL database in which one table rapidly grows very large (several million rows every month or so) so I'd like to periodically archive the contents of that table into a separate table.
I'm intending to use a cron job to execute a .sql file nightly to archive all rows that are older than one month into the other table.
I have the query working fine, but I need to know how to dynamically create a timestamp of one month prior.
The time column is stored in the format 2013-10-27 06:53:12 and I need to know what to use in an SQL query to build a timestamp of exactly one month prior. For example, if today is October 27, 2013, I want the query to match all rows where time < 2013-09-27 00:00:00
Question was answered by a friend in IRC:
'now'::timestamp - '1 month'::interval
Having the timestamp return 00:00:00 wasn't terrible important, so this works for my intentions.
select date_trunc('day', NOW() - interval '1 month')
This query will return date one month ago from now and round time to 00:00:00.
When you need to query for the data of previous month, then you need to query for the respective date column having month values as (current_month-1).
SELECT *
FROM {table_name}
WHERE {column_name} >= date_trunc('month', current_date-interval '1' month)
AND {column_name} < date_trunc('month', current_date)
The first condition of where clause will search the date greater than the first day (00:00:00 Day 1 of Previous Month)of previous month and second clause will search for the date less than the first day of current month(00:00:00 Day 1 of Current Month).
This will includes all the results where date lying in previous month.
In my Rails app, I have two custom Rake Tasks running every 30 minutes. Task A scrapes hourly prices from the internet and saves them to a database as HourlyPrice. Task B goes into the db, takes hourly prices from each day for the last seven days, and averages them to create a new DailyAveragePrice record in a separate DB Table.
However, when running Task B, the last day's (of the seven) average price is incorrect.
After fiddling with the hourly prices of that day in an Excel spreadsheet, I see that the average price Task B is generating is the result of taking only the last three hours and averaging them.
Task B is mostly done with this single query:
averages = HourlyPrice.where('date >= ?', 7.days.ago).average(:price, :group => "DATE_TRUNC('day', date - INTERVAL '1 hour')")
I can't figure out why this is happening?
Clues
HourlyPrice has two attributes (datetime,price). Each HourlyPrice actually represents a price for the previous hour. So, source data lists a 24:00:00 price for each day which PostgreSQL does not want to import as is into a datetime column. Instead, it converts all 24:00:00 prices to 00:00:00 of the next day. To make up for this, I've tried to subtract an hour interval, as you can see in the query. Is this causing the problem?
My ActiveRecord's time zone is currently set to 'Mountain Time (US & Canada)'. That is where the price exchange is located. I have not adjusted my PostgreSQL DB's timezone, and I believe it defaults to UTC. When running Task B, I noticed that it was 9:20PM UTC, leaving three hours left in the UTC day, which might explain the averaging of only three HourlyPrices of the last of the seven days. I'll try running Task B again in the next hour to see if it will average only two hours. Update to come... Is this timezone conflict causing a problem, or is what I am doing insulated from timezones since I have my own date columns?
UPDATE - Problem identified, but how to fix?
Clue #2 is correct. It is a timezone issue. I just ran Task B again (an hour later, with 2 hours left until UTC day change), and it only averages two HourlyPrices now for the last of the seven days.
How can I fix my query above to average ONLY if there are 24 HourlyPrice records available?
I have hourly price data for 10 years. Meaning, 24 prices for each day.
The problem is, the price is from the previous hour of trading. So, the source of my data has listed a 24th hour for each day, and there is no 0 hour.
Example (for further clarity):
The records for a day start at: 07/20/2010 01:00:00
The records for a day end at: 07/20/2010 24:00:00
This conflicts with the way my Rails Apps PostgreSQL DB wants to save DateTime value. When I imported this data from CSV into my DB and saved the dates into a DateTime column, it changed all of the 24:00:00 into 00:00:00 of the following day. This throws off the accuracy of my various end-uses.
Is there anyway I can modify my Postgres DB's behavior to not do this? Any other suggestions?
You could always subtract an hour after you perform the import.
I don't know your database schema so to do this in a general fashion you'd have to execute this SQL on each column that has a date.
UPDATE table SET date_field = date_field - INTERVAL '1 hour'