Parsing a transposed CSV with datetime values on multiple rows in pandas - pandas

Question

I am attempting to import a csv file which includes a number of time-series.
The challenges I am facing are:
a) the csv file is transposed so dates cannot be parsed from columns. Transposing the file using a read_csv().T command would generally work, but it is not appropriate given the datetime information.
b) since the datetime index is on a header row, repeated data points are added a numeral (i.e. Jan becomes Jan, Jan.1, Jan.2 etc.), so stripping datetime values becomes difficult.
c) the first column headers (which do not include datetime information) are placed on the last row of datetime data (third row), which further complicates parsing headers.
Is there an easy way to go from the csv to a 'standard' dataframe structure, with a datetime index parsed from the csv and values in columns?
An example of the csv data structure is here provided:
empty | empty | Jan | Jan | Jan | ... | Dec |
empty | empty | 1 | 1 | 1 | ... | 31 |
head1 | head2 | 00:00 | 01:00 | 02:00 | ... | 23:00 |
---
value1 | value2 | 0.35 | 0.38 | 0.44 | ... | 0.20 |
...

Answer 1

Try:
# read csv with no header
df = pd.read_csv('untitled.txt', header=None).T
# create an index by joining all the columns
df['idx'] = [ ' '.join((a,b,c)) for a,b,c in
zip(df[0].fillna(''),
df[1].fillna(''),
df[2].fillna('')) ]
# drop the unnecessary columns
df.drop([0,1,2], axis=1, inplace=True)
# output
df.set_index('idx').T.reset_index(drop=True)
Output:
+----+-----------+-----------+---------------+---------------+---------------+----------------+
| | head1 | head2 | Jan 1 00:00 | Jan 1 01:00 | Jan 1 02:00 | Dec 31 23:00 |
|----+-----------+-----------+---------------+---------------+---------------+----------------|
| 0 | value1 | value2 | 0.35 | 0.38 | 0.44 | 0.2 |
+----+-----------+-----------+---------------+---------------+---------------+----------------+
As above, the columns are still text (str type). You need to convert them back into timestamps if need be.