Selecting YYYYMM of the previous month in HIVE - sql

I am using Hive, so the SQL syntax might be slightly different. How do I get the data from the previous month? For example, if today is 2015-04-30, I need the data from March in this format 201503? Thanks!
select
employee_id, hours,
previous_month_date--YYYYMM,
from
employees
where
previous_month_date = cast(FROM_UNIXTIME(UNIX_TIMESTAMP(),'yyyy-MM-dd') as int)

From experience, it's safer to use DATE_ADD(Today, -1-Day(Today)) to compute last-day-of-previous-month without having to worry about edge cases. From there you can do what you want e.g.
select
from_unixtime(unix_timestamp(), 'yyyy-MM-dd') as TODAY,
date_add(from_unixtime(unix_timestamp(), 'yyyy-MM-dd'), -1-cast(from_unixtime(unix_timestamp(), 'd') as int)) as LAST_DAY_PREV_MONTH,
substr(date_add(from_unixtime(unix_timestamp(), 'yyyy-MM-dd'), -1-cast(from_unixtime(unix_timestamp(), 'd') as int)), 1,7) as PREV_MONTH,
cast(substr(regexp_replace(date_add(from_unixtime(unix_timestamp(), 'yyyy-MM-dd'), -1-cast(from_unixtime(unix_timestamp(), 'd') as int)), '-',''), 1,6) as int) as PREV_MONTH_NUM
from WHATEVER limit 1
-- today last_day_prev_month prev_month prev_month_num
-- 2015-08-13 2015-07-30 2015-07 201507
See Hive documentation about date functions, string functions etc.

below works across year boundaries w/o complex calcs:
date_format(add_months(current_date, -1), 'yyyyMM') --previous month's yyyyMM
in general,
date_format(add_months(current_date, -n), 'yyyyMM') --previous n-th month's yyyyMM
use proper sign for needed direction (back/ahead)

You could do (year('2015-04-30')*100+month('2015-04-30'))-1 for the above mentioned date, it will return 201503 or something like (year(from_unixtime(unix_timestamp()))*100+month(from_unixtime(unix_timestamp())))-1 for today's previous month. Assuming your date column is in 'yyyy-mm-dd' format you can use the first example and substitute the date string with your table column name; for any other format the second example will do, add the column name in the unix_timestamp() operator.

Angelo's reply is a good start but it returns 201500 if the original date was 2015-01-XX. Building on his answer, I suggest using the following:
IF(month(${DATE}) = 1,
(year(${DATE})-1)*100 + 12,
year(${DATE})*100 + month(${DATE})-1
) as month_key

provided you get rid of those hyphens in your input string , previous date's month id in YYYYMM format you can get by:-
select if( ((${hiveconf:MonthId}-1)%100)=0 ,${hiveconf:MonthId}-89,${hiveconf:MonthId}-1 ) as PreviousMonthId;

Related

Is there a way to convert a yyyy/mm varchar data to date format in snowflake? [duplicate]

This question already has an answer here:
Split quarters to individual months
(1 answer)
Closed 3 months ago.
I need to convert a yyyy/dd varchar type data( ex: 2021/03) to a monthly sort of date. (Ex: 2021/01, 2021/02, 2021/03). So, i need to convert the quarterly format to a monthly format in snowflake. Can we do this?
I tried many things but didn't get the expected results
select TO_DATE(date_column, 'YYYY/MM')
It may help to have a calendar table in your environment to deal with more complicated date conversions like this. An example of what this table may look like is in this CTE below. The SELECT statement following converts a string date of format YYYY/QQ of '2021/03' to YYYY/MM
WITH calendar AS
(
SELECT
dateadd('DAY', seq4(), '2000-01-01'::DATE) as calendar_date,
MONTH(calendar_date) as month_of_year,
QUARTER(calendar_date) as quarter_of_year,
YEAR(calendar_date) as year_of_calendar,
DAY(calendar_date) as day_of_month,
WEEK(calendar_date) as week_of_year
FROM table(generator(rowcount => 365*50))
)
SELECT DISTINCT '2021/03' as YYYYQQ_date,
year_of_calendar || '/' || LPAD(month_of_year, 2, '0') as YYYYMM
FROM calendar
WHERE year_of_calendar = strtok(YYYYQQ_date, '/', 1)
AND quarter_of_year = strtok(YYYYQQ_date, '/', 2);
2021/07
2021/08
2021/09
That's merely an example for turning yyyy/qq into multiple yyyy/mm outputs for that quarter, but this same logic can be applied to any date-part conversion and the calendar table can be customized to hold even organization-specific date things like oddball fiscal periods, company holiday flags, or a business day flag (as an example).

How to convert an int to DateTime in BigQuery

I have an INT64 column called "Date" which contains many different numbers like: "20210209" or "20200305". I want to turn those numbers into a date with this format: MM-YYYY (so in these cases, 02-2021 and 03-2020). Ultimately I want to sum all the data in each month together. The problem is that BigQuery can't convert INT64 to date, only to strings. I'm not sure if I should convert to a string and then to a date or if there is a better way.
Although converting to a string then a date both works and is very concise, over large enough numbers of rows (which may be the case in Big Query) you may be better off using integer maths and using DATE(year, month, day)...
https://cloud.google.com/bigquery/docs/reference/standard-sql/date_functions#date
SELECT
DATE(
DIV( 20210209 , 10000), -- Which gives 2021
DIV(MOD(20210209, 10000), 100), -- Which gives 02
MOD(20210209, 100) -- Which gives 09
)
You can convert the value to a string and use parse_date():
select parse_date('%Y%m%d', cast(20210209 as string))
Another option
select date,
regexp_replace('' || date, r'(\d{4})(\d{2})(\d{2})', r'\2-\1') as MM_YYYY
from your_table
if applied to sample data in your question - output is
Yet another option
select date,
format_date('%m-%Y', parse_date('%Y%m%d', '' || date)) as MM_YYYY
from your_table
with same output

Get first day of first month of previous year in yyyy-mm-dd format

How do I get the first day of the first month of previous year in yyyy-mm-dd format? ie. 2019-01-01.
This is the code I have tried:
SELECT DATEADD(yy,-1,DATEADD(yy,DATEDIFF(yy,0,GETDATE()),0))
You can use DATEFROMPARTS() function in SQL Server for creating date from given year, month and date in integer as shown below. To get the previous year you can use Year() function and subtract 1 from that. First date and month is always 1 so it has been hard-coded here.
declare #IntYear int = Year(Getdate()) - 1 --Previous Year
Select datefromparts(#Intyear, 1, 1)
The output in SSMS is as shown below.
To get the output in the different format you can follow this link.
You seem to be looking to generate a string, not a date. Consider using date functions and string concatenation: you just need to substract 1 year from the current date, and then append '-01-01'
concat_ws('-', year(getdate()) - 1, '01', '01')
Demo on DB Fiddle
You basically have it. You just need the FORMAT function.
SELECT FORMAT (DATEADD(yy,-1,DATEADD(yy,DATEDIFF(yy,0,GETDATE()),0)), 'yyyy-MM-dd') as date

How to convert an YYYY-MM-DD date to YYYY-MM date

In SQL. How to convert a column A from (YYYY-MM-DD) to (YYYYMM)? I want to show the dates in YYYYMM format instead of YYYY-MM-DD.
Data type is TIMESTAMP. Using Teradata Studio 15.10.10.
For Teradata either use
to_char(tscol, 'YYYYMM') -- varchar result
or
extract(year from tscol) * 100 + extract(month from tscol) -- integer result
In Teradata you can format dates pretty much at will. To get YYYYMM, you would use
select <your date> (format 'yyyymm') (char(6))
Your date column needs to be actual date for this, not a string.
There are 3 functions you'll need.
MONTH() function. Returns the MONTH for the date within a range of 1 to 12 ( January to December). It Returns 0 when MONTH part for the date is 0.
YEAR() function. Returns a 4 digit YEAR.
CONCAT() function is used to concatenate two or more strings together.
So here's an example of combining the 3 functions.
SELECT CONCAT(YEAR('1969-02-18'),MONTH('1969-02-18'))
or you can do it in one with
select DATE_FORMAT('1969-02-18','%Y%m')
So to answer your question if it is referring to column A, you can use
SELECT DATE_FORMAT(A,'%Y%m')
SQL Fiddle:
http://www.sqlfiddle.com/#!9/a6c585/48362
You can use DATEPART to get the year and month parts of the date, cast to a varchar, pad and the concaternate.
SELECT DATEPART(YEAR,GETDATE())
SELECT DATEPART(MONTH,GETDATE())
SELECT CAST(DATEPART(YEAR,GETDATE()) AS VARCHAR(4)) + RIGHT('00' + CAST(DATEPART(MONTH,GETDATE()) AS VARCHAR(2)),2)

Date arithmetic in SQL on DB2/ODBC

I'm building a query against a DB2 database, connecting through the IBM Client Access ODBC driver. I want to pull fields that are less than 6 days old, based on the field 'a.ofbkddt'... the problem is that this field is not a date field, but rather a DECIMAL field, formatted as YYYYMMDD.
I was able to break down the decimal field by wrapping it in a call to char(), then using substr() to pull the year, month and day fields. I then formatted this as a date, and called the days() function, which gives a number that I can perform arithmetic on.
Here's an example of the query:
select
days( current date) -
days( substr(char(a.ofbkddt),1,4) concat '-' -- YYYY-
concat substr(char(a.ofbkddt),5,2) concat '-' -- MM-
concat substr(char(a.ofbkddt),7,2) ) as difference, -- DD
a.ofbkddt as mydate
from QS36F.ASDF a
This yields the following:
difference mydate
2402 20050402
2025 20060306
...
4 20110917
3 20110918
2 20110919
1 20110920
This is what I expect to see... however when I use the same logic in the where clause of my query:
select
days( current date) -
days( substr(char(a.ofbkddt),1,4) concat '-' -- YYYY-
concat substr(char(a.ofbkddt),5,2) concat '-' -- MM-
concat substr(char(a.ofbkddt),7,2) ) as difference, -- DD
a.ofbkddt as mydate
from QS36F.ASDF a
where
(
days( current date) -
days( substr(char(a.ofbkddt),1,4) concat '-' -- YYYY-
concat substr(char(a.ofbkddt),5,2) concat '-' -- MM
concat substr(char(a.ofbkddt),7,2) ) -- DD
) < 6
I don't get any results back from my query, even though it's clear that I am getting date differences of as little as 1 day (obviously less than the 6 days that I'm requesting in the where clause).
My first thought was that the return type of days() might not be an integer, causing the comparison to fail... according to the documentation for days() found at http://publib.boulder.ibm.com/iseries/v5r2/ic2924/index.htm?info/db2/rbafzmst02.htm, it returns a bigint. I cast the difference to integer, just to be safe, but this had no effect.
You're going about this backwards. Rather than using a function on every single value in the table (so you can compare it to the date), you should pre-compute the difference in the date. It's costing you resources to run the function on every row - you'd save a lot if you could just do it against CURRENT_DATE (it'd maybe save you even more if you could do it in your application code, but I realize this might not be possible). Your dates are in a sortable format, after all.
The query looks like so:
SELECT ofbkddt as myDate
FROM QS36F.ASDF
WHERE myDate > ((int(substr(char(current_date - 6 days, ISO), 1, 4)) * 10000) +
(int(substr(char(current_date - 6 days, ISO), 6, 2)) * 100) +
(int(substr(char(current_date - 6 days, ISO), 9, 2))))
Which, when run against your sample datatable, yields the following:
myDate
=============
20110917
20110918
20110919
20110920
You might also want to look into creating a calendar table, and add these dates as one of the columns.
What if you try a common table expression?
WITH A AS
(
select
days( current date) -
days( substr(char(a.ofbkddt),1,4) concat '-' -- YYYY-
concat substr(char(a.ofbkddt),5,2) concat '-' -- MM-
concat substr(char(a.ofbkddt),7,2) ) as difference, -- DD
a.ofbkddt as mydate
from QS36F.ASDF a
)
SELECT
*
FROM
a
WHERE
difference < 6
Does your data have some nulls in a.ofbkddt? Maybe this is causing some funny behaviour in how db2 is evaluating the less than operation.