I am new to hive and sql.
Is there any way if we run a query today with count fields then it should fetch last 7 days data ( example- if i run a query with count fields on monday then I should get the total count from last week monday to sunday) And date in my table is in the format 20150910. (yyyyMMdd).
Kindly please help me on this.
You can use date_sub() in this case. Something like this should work...
select * from table
where date_field >= date_sub(current_date, 7)
assuming that the current day's data is not loaded yet. If you want to exclude the current day's data too, you will have to include that too in the filter condition
and date_field <= date_sub(current_date, 1)
current_date would work if your hive version > 0.12
else, you can explicitly pull the date from unix using to_date(from_unixtime(unix_timestamp()))
Related
I have a below query that I run to extract material movements from the last 7 days.
Purpose is to get the data for the last calender week for certain reports.
select
*
From
redshift
where
posting_date between CURRENT_DATE - 7 and CURRENT_DATE - 1
That means I need to run the query on every Monday to get the data for the former week.
Sometimes I am too busy on Monday or its vacation/bank holiday. In that case I would need to change the query or pull the data via SAP.
Question:
Is there a function for redshift that pulls out the data for the last calender week regardless when I run the query?
I already found following solution
SELECT id FROM table1
WHERE YEARWEEK(date) = YEARWEEK(NOW() - INTERVAL 1 WEEK)
But this doesnt seem to be working for redshift sql
Thanks a lot for your help.
Redshift offers a DATE_TRUNC('week', datestamp) function. Given any datestamp value, either a date or datetime, it gives back the date of the preceding Sunday.
So this might work for you. It filters rows from the Sunday before last, up until but not including, the last Sunday, and so gets a full week.
SELECT id
FROM table1
WHERE date >= DATE_TRUNC('week', NOW()) - INTERVAL 1 WEEK
AND date < DATE_TRUNC('week', NOW())
Pro tip: Every minute you spend learning your DBMS's date/time functions will save you an hour in programming.
I have daily data and the code im using creates a report between some dates. I want to replace the date interval with month to date. So every time it will generate a month to date report based on which month we are in. Is there a simple way to do it? thanks :)
An example using BigQuery Standard SQL
SELECT
*
FROM
your_table
WHERE
your_date_field BETWEEN
DATE_TRUNC(CURRENT_DATE(), month) --to get start date of current month
AND
CURRENT_DATE()
You should be able to use that in a WHERE clause and
I'm putting together a query that will get refreshed daily that needs to pull records from the last ten dates.
The tables I'm accessing have a 'xxdatetime' column with the unix time stamp and an 'eventdate' column with the date in a yyyy-mm-dd.
In Impala, the answer was easy:
where eventdate > to_date(days_sub(now(), 10))
I used a variation of it in Hive that failed because I guess it was scanning the whole table and the tables are MASSIVE:
where datediff(cast(current_timestamp() as string), eventdate)=10
Is there a light weight way in Hive SQL to filter the xxdatetime or eventdate columns by 'today - 10 days'?
Eventdate is my indexed column, date_sub is (string, days to subtract) and current_date is the current Hive yyyy mm dd.
eventdate > date_sub(current_date, 5)
I'm not sure if this is super light weight but it works!
I need to run a weekly report based on the arrival date. How can I set the arrival date in the where clause so that I can get the result only for each week. The hard part is I DO NOT want to modify the dates each week. I need the permanent where clause for the date. I have to provide a list of customers who arrived every week and I just want to run the same script without changing the week dates.
Please assist.
Thanks.
SELECT * FROM TABLE WHERE
(ARRIVAL_DATE>DATEFROMPARTS(YEAR(GETDATE()-7), MONTH(GETDATE()-7), DAY(GETDATE()-7)))//7 days before starting at midnight
AND
(ARRIVAL_DATE<DATEFROMPARTS(YEAR(GETDATE()), MONTH(GETDATE()), DAY(GETDATE()))) //NOW in the YYYY, MM,DD format
This will get everything that happened in the current calendar week (Mon-Sun)
SELECT * FROM Table1
WHERE ArrivalDate BETWEEN
CAST(DATEADD(dd,(((DATEPART(dw,getdate())+##DATEFIRST) % 7)+5) % 7 ,getdate()) as date) AND
CAST(DATEADD(dd,6+((((DATEPART(dw,getdate())+##DATEFIRST) % 7)+5) % 7 ),getdate()) as date)
Edit - Removed extra E in BETWEEN
I have a PostgreSQL database in which one table rapidly grows very large (several million rows every month or so) so I'd like to periodically archive the contents of that table into a separate table.
I'm intending to use a cron job to execute a .sql file nightly to archive all rows that are older than one month into the other table.
I have the query working fine, but I need to know how to dynamically create a timestamp of one month prior.
The time column is stored in the format 2013-10-27 06:53:12 and I need to know what to use in an SQL query to build a timestamp of exactly one month prior. For example, if today is October 27, 2013, I want the query to match all rows where time < 2013-09-27 00:00:00
Question was answered by a friend in IRC:
'now'::timestamp - '1 month'::interval
Having the timestamp return 00:00:00 wasn't terrible important, so this works for my intentions.
select date_trunc('day', NOW() - interval '1 month')
This query will return date one month ago from now and round time to 00:00:00.
When you need to query for the data of previous month, then you need to query for the respective date column having month values as (current_month-1).
SELECT *
FROM {table_name}
WHERE {column_name} >= date_trunc('month', current_date-interval '1' month)
AND {column_name} < date_trunc('month', current_date)
The first condition of where clause will search the date greater than the first day (00:00:00 Day 1 of Previous Month)of previous month and second clause will search for the date less than the first day of current month(00:00:00 Day 1 of Current Month).
This will includes all the results where date lying in previous month.