A column in dataframe keeps date like:
2019-06-19 23:04:36
2018-06-29 20:06:56
2019-03-04 11:12:35
2019-07-12 21:16:44
I tried the below code but it gives not correct results:
df['timestamps'] = pd.to_datetime(df['datetimes']).astype('int64') / 10**9
The results are like these:
1.465506e+09
1.465516e+09
1.465503e+09
If I convert them again to date, I get incorrect date time:
df['new'] = df['timestamps'].apply(lambda x: pd.Timestamp(x))
1970-01-01 00:00:01.465506396
1970-01-01 00:00:01.465506397
1970-01-01 00:00:01.465506397
Something is not correct...
What is the way to convert date time that is as string like "2019-06-19 23:04:36" to timestamp?
Thank you.
Related
I have a dataframe in pandas with some columns with dates in the following format
dates
202001
202002
I want to convert them to the following format
dates
2020-01-01
2020-02-01
Could anyone assist with converting the date format? Thanks
If need datetimes use to_datetime with format='%Y%m':
df['dates'] = pd.to_datetime(df['dates'], format='%Y%m')
You may use to_datetime here:
df["dates"] = pd.to_datetime(df["dates"] + '01', format='%Y%m%d', errors='ignore')
Note that your current text dates are year month only, so I concatenate 01 to the end of each one to form the first of the month, for each date.
Try this:
df['dates'] = df['dates'].astype(str)
df['dates'] = pd.to_datetime(df['dates'].str[:4] + ' ' + df['dates'].str[4:])
print(df)
Output:
dates
0 2020-01-01
1 2020-02-01
I need to parse out '%Y%m%d' from the column in BigQuery. My data looks like this:
datetime_published
2000-09-25 13:28:15 UTC
2018-12-22 16:03:00 UTC
2018-05-04 03:05:00 UTC
I have tried the following:
SELECT PARSE_DATE('%Y%m%d', datetime_published) as date
The error message: No matching signature for function PARSE_DATE for argument types: STRING, TIMESTAMP. Supported signature: PARSE_DATE(STRING, STRING)
Desired output:
2000-09-25
Why not just convert to a date?
select date(datetime)
Note: This works for both datetime and timestamp values. These are different in BigQuery. You have a timestamp column which you have called datetime -- a bit of a misnomer.
this is my code I did not put hour and seconds in my data but the result I got includes time, do you know if it is possible to remove the time?
input
fitness['Date']=pd.to_datetime(fitness['Date'])
result
1970-01-01 00:00:00.000000000 1979-09-09
1970-01-01 00:00:00.000000001 1979-09-09
1970-01-01 00:00:00.000000002 1979-09-09
1970-01-01 00:00:00.000000003 1979-09-09
1970-01-01 00:00:00.000000004 1979-09-09
As discussed here, the dt.strftime function can be utilised to format the output.
fitness['Date_formatted'] = pd.to_datetime(fitness['Date']).dt.strftime('%Y-%m-%d')
This will format the output of to_datetime() to follow %Y-%m-%d. However, the dtype of the column will be converted to string. You can find documentation for the function here.
I'm using Pandas to read a .csv file that a 'Timestamp' date column in the format:
31/12/2016 00:00
I use the following line to convert it to a datetime64 dtype:
time = pd.to_datetime(df['Timestamp'])
The column has an entry corresponding to every 15mins for almost a year, and I've run into a problem when I want to plot more than 1 months worth.
Pandas seems to change the format from ISO to US upon reading (so YYYY:MM:DD to YYYY:DD:MM), so my plots have 30 day gaps whenever the datetime represents a new day. A plot of the first 5 days looks like:
This is the raw data in the file either side of the jump:
01/01/2017 23:45
02/01/2017 00:00
If I print the values being plotted (after reading) around the 1st jump, I get:
2017-01-01 23:45:00
2017-02-01 00:00:00
So is there a way to get pandas to read the dates properly?
Thanks!
You can specify a format parameter in pd.to_datetime to tell pandas how to parse the date exactly, which I suppose is what you need:
time = pd.to_datetime(df['Timestamp'], format='%d/%m/%Y %H:%M')
pd.to_datetime('02/01/2017 00:00')
#Timestamp('2017-02-01 00:00:00')
pd.to_datetime('02/01/2017 00:00', format='%d/%m/%Y %H:%M')
#Timestamp('2017-01-02 00:00:00')
After loading data from a csv file, I set the index to the "Date" column and then convert the index to datetime.
df1=pd.read_csv('Data.csv')
df1=df1.set_index('Date')
df1.index=pd.to_datetime(df1.index)
However after conversion the datetime format shows it has been misinterpreted:
original date was e.g. 01-10-2014 00:00:00
but Pandas converts it to 2014-01-10 00:00:00
How can I get Pandas to respect or recognize the original date format?
Thank you
Your datestrings were being interpreted as month first, you need to specify the correct format:
df1.index=pd.to_datetime(df1.index, format='%d-%m-%Y %H:%M:%S')
so that it doesn't interpret the first part as the month
In [128]:
pd.to_datetime('01-10-2014 00:00:00', format='%d-%m-%Y %H:%M:%S')
Out[128]:
Timestamp('2014-10-01 00:00:00')