Wondering if anyone could clarify this for me.
Basically, I have a dataframe that looks like this:
Data_Value
Month_Day
01-01 1.1
01-02 3.9
01-03 3.9
01-04 4.4
I can generate a line plot based on this dataframe using this code:
ax.plot(df.values)
I have had some problems generating a scatter plot from the same data frame and I am wondering if it's possible given that there is a "-" in the index column of the dataframe. However, I am also thinking that since it's possible to generate a line plot it should also be possible to do a scatter plot?
Any insights would be most welcome.
When I try this code:
df = df.reset_index()
df['Month_Day'] = pd.to_datetime(df['Month_Day'], format='%m-%d')
df.plot(type='scatter',x='Month_Day',y='Data_Value')
I get this error msg:
AttributeError: Unknown property type
My Pandas version: 0.19.2
Not sure if I understood your issue totally, but if its just to create scatter plots, you can try to reset the index to convert 'Month_Date' to a regular column and also convert it to datetime. I tried the following:
df.reset_index(inplace=True)
df['Month_Day'] = pd.to_datetime(df['Month_Day'], format='%m-%d')
# you can replace the year with any value, using 2020 as an example
df['Month_Day'] = [val.replace(year=2020) for val in df['Month_Day']]
print(df)
Output:
Month_Day Data_Value
0 2020-01-01 1.1
1 2020-01-02 3.9
2 2020-01-03 3.9
3 2020-01-04 4.4
Then generate a scatter plot:
import matplotlib.pyplot as plt
# generate the plot
plt.scatter(df['Month_Day'], df['Data_Value'])
plt.show()
You can do it, but I believe you have to have 'Month_Day' in the columns so you reset the index.
df = df.reset_index()
df.plot(kind='scatter',x='Month_Day',y='Data_Value')
Result:
Related
I have a excel (.xslx) file with 4 columns:
pmid (int)
gene (string)
disease (string)
label (string)
I attempt to load this directly into python with pandas.read_excel
df = pd.read_excel(path, parse_dates=False)
capture from excel
capture from pandas using my ide debugger
As shown above, pandas tries to be smart, automatically converting some of gene fields such as 3.Oct, 4.Oct to a datetime type. The issue is that 3.Oct or 4.Oct is a abbreviation of Gene type and totally different meaning. so I don't want pandas to do so. How can I prevent pandas from converting types automatically?
Update:
In fact, there is no conversion. The value appears as 2020-10-03 00:00:00 in Pandas because it is the real value stored in the cell. Excel show this value in another format
Update 2:
To keep the same format as Excel, you can use pd.to_datetime and a custom function to reformat the date.
# Sample
>>> df
gene
0 PDGFRA
1 2021-10-03 00:00:00 # Want: 3.Oct
2 2021-10-04 00:00:00 # Want: 4.Oct
>>> df['gene'] = (pd.to_datetime(df['gene'], errors='coerce')
.apply(lambda dt: f"{dt.day}.{calendar.month_abbr[dt.month]}"
if dt is not pd.NaT else np.NaN)
.fillna(df['gene']))
>>> df
gene
0 PDGFRA
1 3.Oct
2 4.Oct
Old answer
Force dtype=str to prevent Pandas try to transform your dataframe
df = pd.read_excel(path, dtype=str)
Or use converters={'colX': str, ...} to map the dtype for each columns.
pd.read_excel has a dtype argument you can use to specify data types explicitly.
Using the following data frame (utilities):
Security_Name Rating Duracion Spread
0 COLBUN 3.95 10/11/27 BBB 6.135749 132
1 ENELGX 4 1/4 04/15/24 BBB+ 3.197206 124
2 PROMIG 3 3/4 10/16/29 BBB- 7.628048 243
3 IENOVA 4 3/4 01/15/51 BBB 15.911632 364
4 KALLPA 4 7/8 05/24/26 BBB- 4.792474 241
5 TGPERU 4 1/4 04/30/28 BBB+ 4.935607 130
dataframe
I am trying to create a sns relplot which should annotate the scatter plot points in respective facetgrid. However the out put i get looks something like this(without the annotations)
relplot
I can't see any annotation in any plot
I have tried the following code:
sns.relplot(x="Duracion", y="Spread", col="Rating", data=utilities)
I really don't know where to start to bring the annotations for this replot using facetrgid. The annotation should be the values of the column Security_Name
please advise the modifications. thanks in advance.
Using FacetGrid and a custom annotation function, you can get the desired result. Note that there is a good chance the annotation will overlap given the example dataframe provided:
def annotate_points(x,y,t, **kwargs):
ax = plt.gca()
data = kwargs.pop('data')
for i,row in data.iterrows():
ax.annotate(row[t], xy=(row[x],row[y]))
g = sns.FacetGrid(col="Rating", data=df)
g.map(sns.scatterplot, "Duracion", "Spread")
g.map_dataframe(annotate_points, "Duracion", "Spread", 'Security_Name')
I have the table with 5 columns with 8000 rows:
Market DeliveryWindowID #Orders #UniqueShoppersAvailable #UniqueShoppersFulfilled
NY 296 2 2 5
MA 365 3 4 8
How do I plot a graph in pandas or seaborn that will show the #Order, #UniqueShoppersAvailable, #UniqueShoppersFulfilled v/s the market and delivery window?
Using Seaborn, reshape your dataframe with melt first:
df_chart = df.melt(['Market','DeliveryWindowID'])
sns.barplot('Market', 'value',hue='variable', data=df_chart)
Output:
One way is to set Market as index forcing it onto the x axis and do a bar graph if you wanted a quick visualization. This can be stacked or not.
Not Stacked
import matplotlib .pyplot as plt
df.drop(columns=['DeliveryWindowID']).set_index(df.Market).plot(kind='bar')
Stacked
df.drop(columns=['DeliveryWindowID']).set_index(df.Market).plot(kind='bar', stacked=True)
I'm trying to generate 4 plots from a DataFrame using Seaborn
Date A B C D
2019-04-05 330.665 161.975 168.69 0
2019-04-06 322.782 150.243 172.539 0
2019-04-07 322.782 150.243 172.539 0
2019-04-08 295.918 127.801 168.117 0
2019-04-09 282.674 126.894 155.78 0
2019-04-10 293.818 133.413 160.405 0
I have casted dates using pd.to_DateTime and numbers using pd.to_numeric. Here is the df.info():
<class 'pandas.core.frame.DataFrame'>
Int64Index: 6 entries, 460 to 465
Data columns (total 5 columns):
Date 6 non-null datetime64[ns]
A 6 non-null float64
B 6 non-null float64
C 6 non-null float64
D 6 non-null float64
dtypes: datetime64[ns](1), float64(4)
memory usage: 288.0 bytes
I can do a wide column plot by just calling .plot() on df.
However,
The legend of the plot is covering the plot itself
I would instead like to have 4 separate plots in 1 diagram and have tried using lmplot to achieve this.
I would like to add labels to the plot like so:
Plot with image
I first melted the data:
df=pd.melt(df,id_vars='Date', var_name='Var', value_name='Unit')
And then tried lmplot
sns.lmplot(x = df['Date'], y='Unit', col='Var', data=df)
However, I get the traceback:
TypeError: Invalid comparison between dtype=datetime64[ns] and str
I have also tried setting df.set_index['Date'] and replotting that using x=df.index and that gave me the same error.
The data can be plotted using Google Sheets but I am trying to automate a workflow where the chart can be generated and sent via Slack to selected recipients.
I hope I have expressed myself clearly enough as I am rather new to Python and Seaborn and hope to get some help from the experts here.
Regarding the legend you can just use .legend(loc="upper left", bbox_to_anchor=(1,1)) as in this example
%matplotlib inline
import pandas as pd
import numpy as np
data = np.random.rand(10,4)
df = pd.DataFrame(data, columns=["A", "B", "C", "D"])
df.plot()\
.legend(loc="upper left", bbox_to_anchor=(1,1));
While for the second IIUC you can play from
df.plot(subplots=True, layout=(2,2));
I am using Seaborn to make boxplots from pandas dataframes. Seaborn boxplots seem to essentially read the dataframes the same way as the pandas boxplot functionality (so I hope the solution is the same for both -- but I can just use the dataframe.boxplot function as well). My dataframe has 12 columns and the following code generates a single plot with one boxplot for each column (just like the dataframe.boxplot() function would).
fig, ax = plt.subplots()
sns.set_style("darkgrid", {"axes.facecolor":"darkgrey"})
pal = sns.color_palette("husl",12)
sns.boxplot(dataframe, color = pal)
Can anyone suggest a simple way of overlaying all the values (by columns) while making a boxplot from dataframes?
I will appreciate any help with this.
This hasn't been added to the seaborn.boxplot function yet, but there's something similar in the seaborn.violinplot function, which has other advantages:
x = np.random.randn(30, 6)
sns.violinplot(x, inner="points")
sns.despine(trim=True)
A general solution for the boxplot for the entire dataframe, which should work for both seaborn and pandas as their are all matplotlib based under the hood, I will use pandas plot as the example, assuming import matplotlib.pyplot as plt already in place. As you have already have the ax, it would make better sense to just use ax.text(...) instead of plt.text(...).
In [35]:
print df
V1 V2 V3 V4 V5
0 0.895739 0.850580 0.307908 0.917853 0.047017
1 0.931968 0.284934 0.335696 0.153758 0.898149
2 0.405657 0.472525 0.958116 0.859716 0.067340
3 0.843003 0.224331 0.301219 0.000170 0.229840
4 0.634489 0.905062 0.857495 0.246697 0.983037
5 0.573692 0.951600 0.023633 0.292816 0.243963
[6 rows x 5 columns]
In [34]:
df.boxplot()
for x, y, s in zip(np.repeat(np.arange(df.shape[1])+1, df.shape[0]),
df.values.ravel(), df.values.astype('|S5').ravel()):
plt.text(x,y,s,ha='center',va='center')
For a single series in the dataframe, a few small changes is necessary:
In [35]:
sub_df=df.V1
pd.DataFrame(sub_df).boxplot()
for x, y, s in zip(np.repeat(1, df.shape[0]),
sub_df.ravel(), sub_df.values.astype('|S5').ravel()):
plt.text(x,y,s,ha='center',va='center')
Making scatter plots is also similar:
#for the whole thing
df.boxplot()
plt.scatter(np.repeat(np.arange(df.shape[1])+1, df.shape[0]), df.values.ravel(), marker='+', alpha=0.5)
#for just one column
sub_df=df.V1
pd.DataFrame(sub_df).boxplot()
plt.scatter(np.repeat(1, df.shape[0]), sub_df.ravel(), marker='+', alpha=0.5)
To overlay stuff on boxplot, we need to first guess where each boxes are plotted at among xaxis. They appears to be at 1,2,3,4,..... Therefore, for the values in the first column, we want them to be plot at x=1; the 2nd column at x=2 and so on.
Any efficient way of doing it is to use np.repeat, repeat 1,2,3,4..., each for n times, where n is the number of observations. Then we can make a plot, using those numbers as x coordinates. Since it is one-dimensional, for the y coordinates, we will need a flatten view of the data, provided by df.ravel()
For overlaying the text strings, we need a anther step (a loop). As we can only plot one x value, one y value and one text string at a time.
I have the following trick:
data = np.random.randn(6,5)
df = pd.DataFrame(data,columns = list('ABCDE'))
Now assign a dummy column to df:
df['Group'] = 'A'
print df
A B C D E Group
0 0.590600 0.226287 1.552091 -1.722084 0.459262 A
1 0.369391 -0.037151 0.136172 -0.772484 1.143328 A
2 1.147314 -0.883715 -0.444182 -1.294227 1.503786 A
3 -0.721351 0.358747 0.323395 0.165267 -1.412939 A
4 -1.757362 -0.271141 0.881554 1.229962 2.526487 A
5 -0.006882 1.503691 0.587047 0.142334 0.516781 A
Use the df.groupby.boxplot(), you get it done.
df.groupby('Group').boxplot()