I have some indices that are missing in my dataframe, I would like to fill those indices that are missing with null values, that is, add a row that corresponds to the missing index.
If the row with the index 60 is missing.
In the "index" column, you can see that the number "60" is missing. I need to be able to add to that dataframe that index, or add a row with null values in the falt index
index.
I have tried to add this line of code.
remained_df= remained_df.reindex(range(remained_df.index[0], remained_df.index[-1] + 1), fill_value=0)
But it fills the end of the dataframe with null values. I need it to be added but in the row corresponding to the index.
Related
I have to format a dataframe so some code can receive it. It has to be in a specific format. The raw dataframe produced gives me a multi-index format. When I pass to the code it gives an IndexError because it was expecting 1 index.
Raw dataframe
I make a copy of the dataframe and remove the index.
ticker_data_2 = ticker_data.copy().reset_index()
Removed index
I need the timestamp column to be the index, so I reset the index to be timestamp. But now I have 2 columns named timestamp. Set index is supposed to remove the timestamp column and place it as the index, not make a copy.
ticker_data_2.set_index(ticker_data_2['timestamp'], inplace=True)
Duplicate timestamp columns
How do I fix to make it so only timestamp shows as the index, and not have a second timestamp column.
I want to drop rows in housing_plot_end that does not contain one of 5 values specified in plot_test_data['housing price median'] dataframe object.
'Dł. geograficzna' is the name of column and it translates to 'latitiude' in english but I left it as it was because maybe the space between these two words is causing a problem?
But I am receiving empty df with:
values_to_save = [plot_test_data['Dł. geograficzna']]
housing_plot_end=housing_plot_end[~housing_plot_end['Dł. geograficzna'].isin(values_to_save)
== False]
enter code here
The column in plot_test_data contains of 5 numerical values thus 5 rows:
-121.46
-117.23
-119.04
-117.13
-118.7
Meanwhile housing_plot_end has tens of thousands of rows and I need to drop every row which does not contain one of these specific values in the column of housing_plot_end['Dł. geograficzna']
But I am receiving empty dataframe object when I've runned this code:
values_to_save = [plot_test_data['Dł. geograficzna']]
housing_plot_end=housing_plot_end[~housing_plot_end['Dł. geograficzna'].isin(values_to_save)
== False]
I don't know what to do.
I am selecting row by row as follows:
for i in range(num_rows):
row = df.iloc[i]
as a result I am getting a Series object where row.index.values contains names of df columns.
But I wanted instead dataframe with only one row having dataframe columns in place.
When I do row.to_frame() instead of 1x85 dataframe (1 row, 85 cols) I get 85x1 dataframe where index contains names of columns and row.columns
outputs
Int64Index([0], dtype='int64').
But all I want is just original data-frame columns with only one row. How do I do it?
Or how do I convert row.index values to row.column values and change 85x1 dimension to 1x85
You just need to adding T
row.to_frame().T
Also change your for loop with adding []
for i in range(num_rows):
row = df.iloc[[i]]
I am trying to reposition the index column in the output CSV from pandas DataFrame.to_csv()
I can order the non index columns using columns but it is unclear how to move the index column.
If i have 2 columns Name and Age and index i want the columns to come out in the following order in resulting CSV Name, Age,index
Anyone know how to do this?
index cannot be moved, it is always first column in DataFrame or Series or Panel. But you can copy data from index to another column.
But if need last column created from index:
df['new_last'] = df.index
If need custom position of new column:
df.insert(2, 'new', df.index)
And last for prevent write index to csv, thanks #Vivek Kalyanarangan:
df.to_csv(file, index=False)
I have a pandas dataframe with 10 columns. I would like to add a column which will uniquely identify every row. I do have to come up with the unique value(could be as simple as a running sequence). How can I do this? I tried adding index as a column itself but for some reason I get a KeyError when I do this.
add a column from range of len of you index
df['new'] = range(1, len(df.index)+1)