Pandas Error Handling - "day is out of range for month" - pandas

Question

I wonder how do I handle and spot the data which has wrong format in pandas.
I tried to convert string into pd data form however,
somewhere in the middle the format is not in line with the format that was expected (I assume), an error message popped.
I want to print what the data is
and I want to fix it into the right format that I could keep moving on
how do I handle this issue?
My code is as below.
def date_format(df):
target = pd.to_datetime(df['Issue Date'])
return target
The error message is
--------------------------------------------------------------------------- TypeError Traceback (most recent call
last)
~/anaconda3/lib/python3.8/site-packages/pandas/core/arrays/datetimes.py
in objects_to_datetime64ns(data, dayfirst, yearfirst, utc, errors,
require_iso8601, allow_object) 2053 try:
-> 2054 values, tz_parsed = conversion.datetime_to_datetime64(data) 2055 # If
tzaware, these values represent unix timestamps, so we
pandas/_libs/tslibs/conversion.pyx in
pandas._libs.tslibs.conversion.datetime_to_datetime64()
TypeError: Unrecognized value type: <class 'str'>
During handling of the above exception, another exception occurred:
ValueError Traceback (most recent call
last) in
----> 1 seg0=generateDb(csv,2017,1000000,0)
in generateDb(csv, year, segment,
index)
2
3 targetSeg0=segment_df(csv,segment,index)
----> 4 targetSeg0['Issue Date']=date_format(targetSeg0)
5 targetSeg0=remove_etc(targetSeg0)
6 filter_date(targetSeg0,year)
in date_format(df)
1 def date_format(df):
----> 2 target = pd.to_datetime(df['Issue Date'])
3 return target
~/anaconda3/lib/python3.8/site-packages/pandas/core/tools/datetimes.py
in to_datetime(arg, errors, dayfirst, yearfirst, utc, format, exact,
unit, infer_datetime_format, origin, cache)
797 result = result.tz_localize(tz)
798 elif isinstance(arg, ABCSeries):
--> 799 cache_array = _maybe_cache(arg, format, cache, convert_listlike)
800 if not cache_array.empty:
801 result = arg.map(cache_array)
~/anaconda3/lib/python3.8/site-packages/pandas/core/tools/datetimes.py
in _maybe_cache(arg, format, cache, convert_listlike)
168 unique_dates = unique(arg)
169 if len(unique_dates) < len(arg):
--> 170 cache_dates = convert_listlike(unique_dates, format)
171 cache_array = Series(cache_dates, index=unique_dates)
172 return cache_array
~/anaconda3/lib/python3.8/site-packages/pandas/core/tools/datetimes.py
in _convert_listlike_datetimes(arg, format, name, tz, unit, errors,
infer_datetime_format, dayfirst, yearfirst, exact)
457 assert format is None or infer_datetime_format
458 utc = tz == "utc"
--> 459 result, tz_parsed = objects_to_datetime64ns(
460 arg,
461 dayfirst=dayfirst,
~/anaconda3/lib/python3.8/site-packages/pandas/core/arrays/datetimes.py
in objects_to_datetime64ns(data, dayfirst, yearfirst, utc, errors,
require_iso8601, allow_object) 2057 return
values.view("i8"), tz_parsed 2058 except (ValueError,
TypeError):
-> 2059 raise e 2060 2061 if tz_parsed is not None:
~/anaconda3/lib/python3.8/site-packages/pandas/core/arrays/datetimes.py
in objects_to_datetime64ns(data, dayfirst, yearfirst, utc, errors,
require_iso8601, allow_object) 2042 2043 try:
-> 2044 result, tz_parsed = tslib.array_to_datetime( 2045 data, 2046 errors=errors,
pandas/_libs/tslib.pyx in pandas._libs.tslib.array_to_datetime()
pandas/_libs/tslib.pyx in pandas._libs.tslib.array_to_datetime()
pandas/_libs/tslib.pyx in
pandas._libs.tslib.array_to_datetime_object()
pandas/_libs/tslib.pyx in
pandas._libs.tslib.array_to_datetime_object()
pandas/_libs/tslibs/parsing.pyx in
pandas._libs.tslibs.parsing.parse_datetime_string()
pandas/_libs/tslibs/parsing.pyx in
pandas._libs.tslibs.parsing._parse_delimited_date()
datetime.pxd in cpython.datetime.datetime_new()
ValueError: day is out of range for month
I'm eventually gonna filter the data as below.
def filter_date(df,year):
booldf = df['Issue Date'] >= pd.to_datetime(f"{year}-01-01")
booldf2 = df['Issue Date'] <= pd.to_datetime(f"{year}-12-31")
return df[
(df['Issue Date'] >= pd.to_datetime(f"{year}-01-01")) & (df['Issue Date'] <= pd.to_datetime(f"{year}-12-31"))
]
Sample data form is as below.
The Error occurs because of wrong data like below. (51th month doesn't exist).. I guess this is where the error came from.

Answer 1

Based on your comments I assume this will help:
Using as example (99/99/9999 is the incorrect data):
df = pd.DataFrame(["09/26/2016", "06/14/2017", "09/05/2018", "06/16/2017", "05/09/2018", "99/99/9999"], columns=['Issue Date']).
You mean something like this:
pd.to_datetime(df["Issue Date"], errors="coerce")
output:
see https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.to_datetime.html
if you want to you drop the rows with NaTs next with:
df = df.dropna(axis=0, subset=['Issue Date'])