I have a table on BigQuery with one of the columns being "timestamp". This column is of datatype INT64. I want to add a new column based on that column with the exact dates.
The data in the timestamp column is as follows:
-600 represents 19:00 EDT on Sunday May 1, 2011
-It is in microseconds, for e.g In one record there is 2506199602819 as the timestamp, this should be around 29 days after.
What would be the right way to proceed with this? I'm having this table on BigQuery but any SQL would be helpful.
You can do:
select timestamp_add(timestamp('2011-05-01T19:00:00', 'America/New_York'), interval 2506199602819 - 600 microsecond)
Related
Need help to get the data of particular format
We have a table which have a data which of production now we need to select the data of each day with particular time period which is differentiate between three shift A,B,C.
In our table we have a datetime column which capture's each seconds data now that data we need in shiftwise like 6am to 2pm is of A shift production count and 2pm to 10pm of shift B and 10pm to 6 am of shift C.
here i am getting the data for single day where i have written the below query which is working good.
select distinct(count(PRD_SERIAL_NUMBER)),(select convert(date,getdate())) as date,'B' as shift_name
from table_name
where status=02
and LAST_UPDATED_DATE
between (SELECT FORMAT(GETDATE(),'yyyy-MM-dd 14:01:00.000')) and
(SELECT FORMAT(GETDATE()-26,'yyyy-MM-dd 22:01:00.000'))
refer below output image 1
Here i am getting the count for single day and for upcoming days i have solution but now the question arise is i have a past 4 Month data which i need to get in datewise and shiftwise count and for the column prd_serial_number have duplicate entries so it should be in distinct.
please refer below image 2 for required output format
I have a large database with a column containing timestamps (date/time). Some records were inserted with the time being exactly midnight (no milliseconds). I want a query that will return all records from the table regardless of date, that have a time of midnight.
I know how to create a record with timestamp of midnight, but don't know how to specifically search for them.
select TRUNC(SYSDATE) FROM DUAL;
You can compare the column value with the truncated value:
where datetimecol = trunc(datetimecol)
I have the following problem to solve. I have a hive table, that store events, and each event timestamp is stored as unix timestamp (e.g. 1484336244).
Every day I want to run a query that fetches yesterdays events.
How could I form this query in Hive?
So for example, today is the 9th February, I want to get only the events that occurred on the 8th February.
Subtract one day from current_date and compare it with the column converted to yyyy-MM-dd format.
date_add(current_date,-1) = from_unixtime(colName,'yyyy-MM-dd')
I'm putting together a query that will get refreshed daily that needs to pull records from the last ten dates.
The tables I'm accessing have a 'xxdatetime' column with the unix time stamp and an 'eventdate' column with the date in a yyyy-mm-dd.
In Impala, the answer was easy:
where eventdate > to_date(days_sub(now(), 10))
I used a variation of it in Hive that failed because I guess it was scanning the whole table and the tables are MASSIVE:
where datediff(cast(current_timestamp() as string), eventdate)=10
Is there a light weight way in Hive SQL to filter the xxdatetime or eventdate columns by 'today - 10 days'?
Eventdate is my indexed column, date_sub is (string, days to subtract) and current_date is the current Hive yyyy mm dd.
eventdate > date_sub(current_date, 5)
I'm not sure if this is super light weight but it works!
I have hourly price data for 10 years. Meaning, 24 prices for each day.
The problem is, the price is from the previous hour of trading. So, the source of my data has listed a 24th hour for each day, and there is no 0 hour.
Example (for further clarity):
The records for a day start at: 07/20/2010 01:00:00
The records for a day end at: 07/20/2010 24:00:00
This conflicts with the way my Rails Apps PostgreSQL DB wants to save DateTime value. When I imported this data from CSV into my DB and saved the dates into a DateTime column, it changed all of the 24:00:00 into 00:00:00 of the following day. This throws off the accuracy of my various end-uses.
Is there anyway I can modify my Postgres DB's behavior to not do this? Any other suggestions?
You could always subtract an hour after you perform the import.
I don't know your database schema so to do this in a general fashion you'd have to execute this SQL on each column that has a date.
UPDATE table SET date_field = date_field - INTERVAL '1 hour'