REGEX Date Match Format - sql

Question

I currently have a dataset with varying date entries (and a mixture of string entries) for which I need to parse. There are a few: 'M/DD/YY', 'M/D/YY', 'MM/DD/YY', 'MM/D/YY', 'MM/DD/YYYY'...). I could use some support with improving my regex to handle the varying formats and possible text entered in the date field.
My current Postgres query breaks out other entries into another column and reformats the date. Although, I've increased the year to 4 digits rather than 2, I believe the issue may live somewhere in the 'YYYY-MM-DD' formatting or that my query does not properly accommodate additional formatting within.
CASE WHEN date ~ '^\\\\d{1,2}/\\\\d{1,2}/\\\\d{4}$' THEN TO_DATE(date::date, 'YYYY-MM-DD')
ELSE NULL END AS x_date,
CASE WHEN NOT date ~ '^\\\\d{1,2}/\\\\d{1,2}/\\\\d{4}$' AND date <> '' THEN date
ELSE NULL END AS x_date_text
For the various date formats, they should be reformatted accordingly and for other non-date values, they should be moved over to the other column.

Answer 1

Based on your list of formats, I believe that just two regexes should be enough to check the values:
'^[0-9]{1,2}/[0-9]{1,2}/[0-9]{4}/$' would map to date format 'MM/DD/YYYY'
'^[0-9]{1,2}/[0-9]{1,2}/[0-9]{2}/$' would map to 'MM/DD/YY'
You can use a CASE construct to check the value against the regex and apply the proper mask when using TO_DATE().
However, since you need to split the data over two columns, you would need to tediously repeat the CASE expression twice, one for each column.
One way to simplify the solution (and to make it easier to maintain afterwards) would be to use a CTE to list the regexes and the associated date format. You can LEFT JOIN the CTE with the table.
Consider the following query:
WITH vars AS (
SELECT '^[0-9]{1,2}/[0-9]{1,2}/[0-9]{4}/$' reg, 'MM/DD/YYYY' format
UNION ALL '^[0-9]{1,2}/[0-9]{1,2}/[0-9]{2}/$', 'MM/DD/YY'
)
SELECT
CASE WHEN vars.reg IS NOT NULL THEN TO_DATE(t.date, vars.format) END x_date,
CASE WHEN vars.reg IS NULL THEN t.date END x_date_text
FROM
mytable t
LEFT JOIN vars ON t.date ~ vars.reg
If more regex/format pairs are needed, you just have to expand the CTE. Just pay attention to the fact that regexes should be exclusives (ie two different regexes should not possibly match on a single value), else you will get duplicated records in the result.

Answer 2

While the regex by #GMB insures format validity it passes many invalid dates, and with liberal to_date conversion by Postgres could introduce errors and or confusion. Run the following to see the liberal conversion:
set datestyle = 'ISO';
select dd,'01/' || dd || '/2019' mmddyyyy, to_date ( '01/' || dd || '/2019', 'mm/dd/yyyy')
from ( select generate_series( 0,40)::text dd) d;
select mm , mm ||'/01/2019' mmddyyyy, to_date ( mm ||'01/2019', 'mm/dd/yyyy')
from ( select generate_series( 0,40)::text mm) d;
If that liberal date conversion is acceptable - Great. But if not we can tighten it down considerable (although still not 100% valid results). Lets break the format down:
for date formats mm/dd/yyyy or mm/dd/yy
breakdown MM valid 1 - 12
valid character 0 followed by 1-9
1 followed by 0-2
regex (0?[1-9]|1[0-2)
DD valid 0 - 31 (sort of)
day 31 valid for April, June, Sep, Nov also evaluate valid but become
day 1 of May, July, Oct, Dec respectivally
days 29-31 of Feb also eveluate valid but become day
1-3 of march and 1-2 in lead yearsin non-leap years
valid character optional 0 followed by 1-9
1-2 followed by 0-9
3 followed by 0-1
regex (0?[1-9]|[1-2][0-9]|3[0-2])
YEAR valid 1900 - 2999 (no ancient history)
valid character 1-2 followed by 0-9,0-9,0-9
0-9,0-9
Now putting that together we get.
-- setup
drop table if exists my_dates;
create table my_dates(test_date text, status text);
insert into my_dates (test_date, status)
values ('01/15/2019', 'valid')
, ('12/25/0001', 'invalid year < 1900')
, ('12/01/2020', 'valid')
, ('oops', 'yea a date NOT')
, ('6/3/19', 'valid')
, ('2/29/2019', 'valid sort of, Postgres liberal evaluation of to_date')
, ('2/30/2019', 'valid sort of, Postgres liberal evaluation of to_date')
, ('2/31/2019', 'valid sort of, Postgres liberal evaluation of to_date')
, ('2/29/2020', 'valid')
, ('14/29/2020', 'invalid month 14')
, ('01/32/2019', 'invalid day 32')
, ('04/31/2019', 'valid sort of, Postgres liberal evaluation of to_date')
;
-- as query
set datestyle = 'ISO';
with patterns (pat, fmt) as (values ('^(0?[1-9]|1[0-2])/(0?[1-9]|[1-2][0-9]|3[0-1])/[12][0-9]{3}$'::text, 'mm/dd/yyyy')
, ('^(0?[1-9]|1[0-2])/(0?[1-9]|[1-2][0-9]|3[0-1])/[0-9]{2}$'::text, 'mm/dd/yy')
)
select to_date(test_date, fmt),status, test_date, pat, fmt
from my_dates
left join patterns on test_date ~ pat;
------------------------------------------------------------------
-- function accessable from SQL
create or replace function parse_date(check_date_in text)
returns date
language sql
as $$
with patterns (pat, fmt) as (values ('^(0?[1-9]|1[0-2])/(0?[1-9]|[1-2][0-9]|3[0-1])/[12][0-9]{3}$'::text, 'mm/dd/yyyy')
, ('^(0?[1-9]|1[0-2])/(0?[1-9]|[1-2][0-9]|3[0-1])/[0-9]{2}$'::text, 'mm/dd/yy')
)
select to_date(check_date_in, fmt)
from patterns
where check_date_in ~ pat;
$$;
--- test function
select test_date, parse_date(test_date), status from my_dates;
-- use demo
select * from my_dates
where parse_date(test_date) >= date '2020-01-02';