I'm trying to get today's date in a few different formats and I keep getting errors:
pd.to_datetime('Today',format='%m/%d/%Y') + MonthEnd(-1)
ValueError: time data 'Today' does not match format '%m/%d/%Y' (match)
What is the correct syntax to get todays date in yyyy-mm-dd and yyyymm formats?
For YYYY-MM-DD format, you can do this:
import datetime as dt
print(dt.datetime.today().date())
2017-05-23
For YYYY-MM format, you can do this:
print(dt.datetime.today().date().strftime('%Y-%m'))
2017-05
If you need to do this on just a few columns you can use:
import pandas as pd
dataframe_name['Date_Column_name'].apply(pd.tslib.normalize_date)
This method doesn't use any other module except pandas. If you need a "custom" date format you can always do:
from datetime import datetime as dt
dataframe_name['Date_Column_name'].dt.strftime('%d/%m/%Y')
Here is a list of strftime options.
Related
I have data from an S3 bucket and want to convert the Date column from string to date. The current Date column is in the format 7/1/2022 12:0:15 AM.
Current code I am using in AWS Glue Studio to attempt the custom transformation:
MyTransform (glueContext, dfc) -> DynamicFrameCollection:
from pyspark.sql.functions import col, to_timestamp
df = dfc.select(list(dfc.keys())[0]).toDF()
df = df.withColumn('Date',to_timestamp(col("Date"), 'MM/dd/yyyy HH:MM:SS'))
df_res = DynamicFrame.fromDF(df, glueContext, "df")
return(DynamicFrameCollection({"CustomTransform0": df_res}, glueContext))
With MM/dd/yyyy HH:MM:SS date formatting, it runs but returns null for the Date column. When I try any other date format besides this, it errors out. I suspect the date formatting may be the issue, but I am not certain.
After converting string to timestamp you need to cast it to date type, like this:
df = df.withColumn(df_col, df[df_col].cast("date"))
We ended up removing the HH:MM:SS portion of the date format and this worked for our needs. I would still be interested if anyone can figure out how to get the hours, minutes, seconds, and AM/PM to work, but we can do without for now.
I have dates in this format: 2015-02-02 14:19:00.
I use this code:
dateparse = lambda dates: pd.datetime.strptime(dates, '%Y/%m/%d %H:%M:%S')
df = pd.read_csv('3df_uniti.csv', parse_dates=True, index_col='date', date_parser=dateparse)
df.head()
but it doesn't work because it gives me the follow error:
time data does not match format
Can you help me to set the right format?
Your format uses / instead of -. Try changing it to %Y-%m-%d %H:%M:%S.
I would like to subset a data frame based on a date column, which originally has this format:
3/22/13
After I transform it to a date:
df['date']=pd.to_datetime(df['date'], format='%m/%d/%y')
I get this:
2013-03-22 00:00:00
Now I would like to subset it with something like this:
df.loc[(df['date']>'2014-06-22')]
But that either gives me an empty data frame or full data frame, that is no filtering.
Any suggestions how I can get this to work?
remark: I am well aware that similar questions have been asked in other forums but I could not figure out a solution since my date column looks different.
First you have to convert your starting date and final date into a datetime format. Then you can apply multiple conditions inside df.loc. Do not forget to reassign your modifications to your df :
import pandas as pd
from datetime import datetime
df['date']=pd.to_datetime(df['date'], format='%m/%d/%y')
date1 = datetime.strptime('2013-03-23', '%Y-%m-%d')
date2 = datetime.strptime('2013-03-25', '%Y-%m-%d')
df = df.loc[(df['date']>date1) & (df['date']<date2)]
I'm using python and pandas to transform csv's,
I imported the csv as a dataframe.
the dataframe has a string-column containing dates in format "January 1, 2016, Hour 0"
I'd like to convert that string into a date or even string in format dd/mm/yyyy hh:mm
note that in this case the minutes will always be 0.
thanks !
I've done it using datetime and strptime as following
df['field'].apply(lambda x : datetime.strptime(x, '%B %d, %Y, Hour %H').strftime('%d/%m/%Y %H:00'))
or generally:
df['field'].apply(lambda x : datetime.strptime(x, old_formatted_date).strftime(new_formatted_date))
for more on datetime default lib check:
datetime
and datetime format codes:
datetime format codes
Good Day! I have this problem I have a code like this
tree_model = self.env['hr.reception'].search([('create_date','=',date_from)])
the date_from is only a date only parameter while the column create_date is datetime column I only want to format the create_date as date only format
is this possible just like in postgres
(create_date::date)
Thanks for the Help
You can achieve it using date() method which will return the date format instead of datetime.
For example:
1. datetime.now() will result in datetime format.
2. datetime.datetime.now().date() will result in date format.
Try : create_date.strftime('%Y-%m-%d')