journal quality kde plots with seaborn/pandas - pandas
I'm trying to do some comparative analysis for a publication. I came across seaborn and pandas and really like the ease with which I can create the analysis that I want. However, I find the manuals a bit scanty on the things that I'm trying to understand about the example plots and how to modify the plots to my needs. I'm hoping for some advice here on to get the plots I'm want. Perhaps pandas/seaborn is not what I need.
So, I would like to create subplots, (3,1) or (2,3), of the following figure:
Questions:
I would like the attached plot to have a title on the colorbar. Not sure if this is possible or exactly what is shown, i.e., is it relative frequency or occurrence or a percentage, etc? How can I put a explanatory tile on the colorbar (oriented vertically).
The text is a nice addition. The pearsonr is the correlation, but I'm not sure what is p. My guess is that it is showing the lag, or? If so, how can I remove the p in the text?
I would like to make the same kind of figure for different variables and put it all in a subplot.
Here's the code I pieced together from the seaborn manual/examples and from other users here on SO (thanks guys).
import netCDF4 as nc
import pandas as pd
import xarray as xr
import numpy as np
import seaborn as sns
import pdb
import matplotlib.pyplot as plt
from scipy import stats, integrate
import matplotlib as mpl
import matplotlib.ticker as tkr
import matplotlib.gridspec as gridspec
sns.set(style="white")
sns.set(color_codes=True)
octp = [622.0, 640.0, 616.0, 731.0, 668.0, 631.0, 641.0, 589.0, 801.0,
828.0, 598.0, 742.0,665.0, 611.0, 773.0, 608.0, 734.0, 725.0, 716.0,
699.0, 686.0, 671.0, 700.0, 656.0,686.0, 675.0, 678.0, 653.0, 659.0,
682.0, 674.0, 684.0, 679.0, 704.0, 624.0, 727.0,739.0, 662.0, 801.0,
633.0, 896.0, 729.0, 659.0, 741.0, 510.0, 836.0, 720.0, 685.0,430.0,
833.0, 710.0, 799.0, 534.0, 532.0, 605.0, 519.0, 850.0, 357.0, 858.0,
497.0,404.0, 456.0, 448.0, 836.0, 462.0, 381.0, 499.0, 673.0, 642.0,
641.0, 458.0, 809.0,562.0, 742.0, 732.0, 710.0, 658.0, 533.0, 811.0,
853.0, 856.0, 785.0, 659.0, 697.0,654.0, 673.0, 707.0, 711.0, 423.0,
751.0, 761.0, 638.0, 576.0, 538.0, 596.0, 718.0,843.0, 640.0, 647.0,
692.0, 599.0, 607.0, 537.0, 679.0, 712.0, 612.0, 641.0, 665.0,658.0,
722.0, 656.0, 656.0, 742.0, 505.0, 688.0, 805.0]
cctp = [482.0, 462.0, 425.0, 506.0, 500.0, 464.0, 486.0, 473.0, 577.0,
735.0, 390.0, 590.0,464.0, 417.0, 722.0, 410.0, 679.0, 680.0, 711.0,
658.0, 687.0, 621.0, 643.0, 690.0,630.0, 661.0, 608.0, 658.0, 624.0,
646.0, 651.0, 634.0, 612.0, 636.0, 607.0, 539.0,706.0, 614.0, 706.0,
401.0, 720.0, 746.0, 511.0, 700.0, 453.0, 677.0, 637.0, 605.0,454.0,
733.0, 535.0, 725.0, 668.0, 513.0, 470.0, 589.0, 765.0, 596.0, 749.0,
462.0,469.0, 514.0, 511.0, 789.0, 647.0, 324.0, 555.0, 670.0, 656.0,
786.0, 374.0, 757.0,645.0, 744.0, 708.0, 497.0, 654.0, 288.0, 705.0,
703.0, 446.0, 675.0, 440.0, 652.0,589.0, 542.0, 661.0, 631.0, 343.0,
585.0, 632.0, 591.0, 602.0, 365.0, 535.0, 663.0,561.0, 448.0, 582.0,
591.0, 535.0, 475.0, 422.0, 599.0, 594.0, 569.0, 576.0, 622.0,483.0,
539.0, 515.0, 621.0, 443.0, 435.0, 502.0, 443.0]
cctp = pd.Series(cctp, name='CTP [hPa]')
octp = pd.Series(octp, name='CTP [hPa]')
formatter = tkr.ScalarFormatter(useMathText=True)
formatter.set_scientific(True)
formatter.set_powerlimits((-2, 2))
g = sns.jointplot(cctp,octp, kind="kde",size=8,space=0.2,cbar=True,
n_levels=50,cbar_kws={"format": formatter})
# add a line x=y
x0, x1 = g.ax_joint.get_xlim()
y0, y1 = g.ax_joint.get_ylim()
lims = [max(x0, y0), min(x1, y1)]
g.ax_joint.plot(lims, lims, ':k')
plt.show()
plt.savefig('test_fig.png')
I know I'm asking a lot here. So I put the questions in order of priority.
1: To set the colorbar label, you can add the label key to the cbar_kws dict:
cbar_kws={"format": formatter, "label": 'My colorbar'}
2: To change the stats label, you need to first slightly modify the stats.pearsonr function to only return the first value, instead of the (pearsonr, p) tuple:
pr = lambda a, b: stats.pearsonr(a, b)[0]
Then, you can change that function using jointplot's stat_func kwarg:
stat_func=pr
and finally, you need to change the annotation to get the label right:
annot_kws={'stat':'pearsonr'})
Putting that all together:
pr = lambda a, b: stats.pearsonr(a, b)[0]
g = sns.jointplot(cctp,octp, kind="kde",size=8,space=0.2,cbar=True,
n_levels=50,cbar_kws={"format": formatter, "label": 'My colorbar'},
stat_func=pr, annot_kws={'stat':'pearsonr'})
3: I don't think its possible to put everything in a subplot with jointplot. Happy to be proven wrong there though.
Related
How to solve a set of nonlinear equations multiple times when one of the coefficient of the variable changes
I want to solve for the variables s,L for different values of t. t is a part of my second equation and its values changes, I tried to solve for s,L for different t values then append the values to an empty list so that i could have different values of s, L for diiferent t values. But what i was getting is just an empty list.PLease help me with this from scipy.optimize import fsolve import numpy as np import math as m q0=0.0011 thetas,thetai,thetar=0.43,0.1,0.05 ks=0.0022#m/hr psib=-0.15# m lamda=1 eta=2+3*lamda ki=8.676*10**(-8) si=0.13157 t=np.array([3,18,24]) S=0.02/24 delta=-0.1001 b=[] n=[] for i in range(3): def equations(p): s, L = p f1=(ks*s**(3+(2/lamda))-(psib/(1-eta))*(((ki*si**(-1/lamda))-(ks*s**(3+(1/lamda))))/L)-q0) f2=(L*(s*(thetas-thetar))+S*t[i]*0.5*(m.exp(-delta*psib*(-1+s**(-1/lamda))))-(q0-ki)*t[i]) return(f1,f2) s,L=fsolve(equations,([0.19,0.001])) b.append(s) n.append(L) print(b) print(n)
There are several ways to evaluate this system with an adjustable parameter. You could plug each value in before solving, which would make it compatible with additional solvers if fsolve didn't give you the desired results, or you could utilize the args parameter within fsolve. If I set up a dummy system such that I try to find x,y,z for some initial guess, and step through a parameter, I can append a preallocated solution array with the results import numpy as np from scipy.optimize import fsolve a = np.linspace(0,10,21) def equations(variables, a): x,y,z = variables eq1 = x+y+z*a eq2 = x-y-z eq3 = x*y*x*a return tuple([eq1, eq2, eq3]) solutions = np.zeros((21,3)) for idx, i in enumerate(a): solutions[idx] = fsolve(equations, [-1,0,1], args=(i)) print(solutions) which gives [[ 5.00000000e-01 -5.00000000e-01 1.00000000e+00] [ 9.86864911e-17 -2.96059473e-16 3.94745964e-16] [ 1.62191889e-39 -1.28197512e-16 1.28197512e-16] [-2.15414908e-17 -1.07707454e-16 8.61659633e-17] [ 2.19853562e-28 6.59560686e-28 -4.39707124e-28] [-1.20530409e-28 -2.81237621e-28 1.60707212e-28] [-3.34744837e-17 -6.69489674e-17 3.34744837e-17] [ 6.53567253e-17 1.17642106e-16 -5.22853803e-17] [-3.14018492e-17 -5.23364153e-17 2.09345661e-17] [-5.99115518e-17 -9.41467242e-17 3.42351724e-17] [ 5.18199815e-29 7.77299722e-29 -2.59099907e-29] [-2.70691440e-17 -3.90998747e-17 1.20307307e-17] [-2.57288510e-17 -3.60203914e-17 1.02915404e-17] [-2.44785120e-17 -3.33797891e-17 8.90127708e-18] [-1.27252940e-28 -1.69670587e-28 4.24176466e-29] [ 2.24744956e-56 2.93897250e-56 -6.91522941e-57] [-2.12580678e-17 -2.73318015e-17 6.07373366e-18] [-2.03436865e-17 -2.57686696e-17 5.42498307e-18] [-3.89960988e-17 -4.87451235e-17 9.74902470e-18] [-1.87148635e-17 -2.31183608e-17 4.40349730e-18] [-7.19531738e-17 -8.79427680e-17 1.59895942e-17]]
How to make a graded colourmap in Matplotlib with specified intervals
I am trying to make a custom colourmap in matplotlib but can't quite get it right. What I want is a specific colour for specific values, but for there to be a smooth gradation between them. So something along the lines of this: colourA = { 0.01: "#51158d" 0.25: "#5c2489" 0.5: "#693584" 1: "#9d9933" 3: "#e3dc56" } I've seen the method LinearSegmentedColormap.from_list() which I can pass all the hex codes into, but how can I assign specific colours to specific values so that the steps are not equally distributed?
You don't have to use the from_list classmethod, that just exists for convenience, assuming a commonly used equal spacing between them. You can convert your colors to the cdict required for the LinearSegmentedColormap. Starting with what you already have: import matplotlib as mpl colors = { 0.01: "#51158d", 0.25: "#5c2489", 0.5: "#693584", 1: "#9d9933", 3: "#e3dc56", } my_cmap = mpl.colors.LinearSegmentedColormap.from_list("my_cmap", list(colors.values())) You basically need to do two things; normalize your range between 0 and 1, and convert the hex format to rgb. For example: from collections import defaultdict cdict = defaultdict(list) vmin = min(colors.keys()) vmax = max(colors.keys()) for k,v in colors.items(): k_norm = (k - vmin) / (vmax - vmin) r,g,b = mpl.colors.to_rgb(v) cdict["red"].append((k_norm, r, r)) cdict["green"].append((k_norm, g, g)) cdict["blue"].append((k_norm, b, b)) my_cmap = mpl.colors.LinearSegmentedColormap("my_cmap", cdict) Note that the cdict requires the x-values to increase monotonically, which is the case with this example. Otherwise first sort the colors in the dictionary. When using this to plot, you can/should use a norm to scale it between whatever values you want, for example: norm = mpl.colors.Normalize(vmin=0.01, vmax=3) Details about the format of the cdict can be found in the docstring: https://matplotlib.org/stable/api/_as_gen/matplotlib.colors.LinearSegmentedColormap.html
Getting rid of a specific line in plots
x = [0.0000000,0.0082707,0.0132000, 0.0255597, 0.0503554, 0.0751941, 0.1000570, 0.1498328, 0.1996558, 0.2495240, 0.2994312, 0.3993490, 0.4993711, 0.5994664, 0.6996058, 0.7997553, 0.8998927, 0.9499514, 1.0000000, 0.0000000, 0.006114, 0.0062188, 0.0087532, 0.0138088, 0.0264052, 0.0515127, 0.0765762, 0.1016176, 0.1516652, 0.2016828, 0.2516733, 0.3016387, 0.4015163, 0.5013438, 0.6011363, 0.7008976, 0.8006328, 0.9003380, 0.9501740, 1.0000000] y = [0.0000000, 0.0233088, 0.0298517, 0.0425630, 0.0603942, 0.0739301, 0.0850687, 0.1023515, 0.1149395, 0.1230325, 0.1272298, 0.1253360, 0.1130538, 0.0934796, 0.0695104, 0.0445423, 0.0207728, 0.0098870, 0.0000000, 0.0000000, -.0208973, -.0210669, -.0247377, -.0307807, -.0416431, -.0548774, -.0637165, -.0703581, -.0801452, -.0869356, -.0910290, -.0926252, -.0905235, -.0834273, -.0728351, -.0591463, -.0428603, -.0235778, -.0122883, 0.0000000] plt.plot(x,y) This is a data point I have web scraped from a website. If I plot this there is a line that crosses from (0,0) to (1,0) which I don't want. I've tried manually removing points and seeing what points I need to remove but I can't seem to get it to work. There are two pairs of the same points in this data point. How can you remove the line that crosses from (0,0) to (1,0) and can you automate it by using if statement?
If you look carefully at your data, there appears to be a discontinuity. Your x values start going up, then goes back to 0, and goes up again. This discontinuity is creating the horizontal line that's bothering you. You can loop through x and y, detect when the values of x drop instead of increase, find this index and separate the two lists. However, using np.diff can help with this too. In this specific case, there's only one drop, so you have only one negative diff value. Here's my code and the graph I got. It might not generalize to any number of discontinuities, but it fixes your case. import matplotlib.pyplot as plt import pandas as pd import numpy as np x = [0.0000000,0.0082707,0.0132000, 0.0255597, 0.0503554, 0.0751941, 0.1000570, 0.1498328, 0.1996558, 0.2495240, 0.2994312, 0.3993490, 0.4993711, 0.5994664, 0.6996058, 0.7997553, 0.8998927, 0.9499514, 1.0000000, 0.0000000, 0.006114, 0.0062188, 0.0087532, 0.0138088, 0.0264052, 0.0515127, 0.0765762, 0.1016176, 0.1516652, 0.2016828, 0.2516733, 0.3016387, 0.4015163, 0.5013438, 0.6011363, 0.7008976, 0.8006328, 0.9003380, 0.9501740, 1.0000000] y = [0.0000000, 0.0233088, 0.0298517, 0.0425630, 0.0603942, 0.0739301, 0.0850687, 0.1023515, 0.1149395, 0.1230325, 0.1272298, 0.1253360, 0.1130538, 0.0934796, 0.0695104, 0.0445423, 0.0207728, 0.0098870, 0.0000000, 0.0000000, -.0208973, -.0210669, -.0247377, -.0307807, -.0416431, -.0548774, -.0637165, -.0703581, -.0801452, -.0869356, -.0910290, -.0926252, -.0905235, -.0834273, -.0728351, -.0591463, -.0428603, -.0235778, -.0122883, 0.0000000] discont = np.diff(x).argmin() x1 = x[:discont+1] y1 = y[:discont+1] x2 = x[discont+1:] y2 = y[discont+1:] plt.plot(x1, y1) plt.plot(x2, y2) plt.show()
Plotting Line and Stacked Bar plots on the same graph in Time Series
So, my goal is to display two line plots and one stacked bar plot on the same graph using pandas matplotlib in Time Series. I have followed the steps in the following link Plot Pandas DataFrame as Bar and Line on the same one chart, but it did not work as expected. What I get is only the stacked bar plot displayed: Here is my code: import pandas as pd import matplotlib.pyplot as plt df_max_scaled = pd.read_csv('dataset.csv',index_col='sample_date') start, end = '2021-06-01', '2021-07-08' fig, ax = plt.subplots(figsize=(20, 10)) ax.plot(df_max_scaled.loc[start:end,'cumPeopleVaccinatedFirstDoseByVaccinationDate'],marker='.', linestyle='-', linewidth=0.5, label='FirstVaccine') ax.plot(df_max_scaled.loc[start:end,'cumPeopleVaccinatedSecondDoseByVaccinationDate'],marker='.', linestyle='-', linewidth=0.5, label='SecondVaccine') df_max_scaled.loc[start:end].plot(kind='bar', stacked=True, y=['B.1.1.7','B.1.617.2'],ax=ax) Here is the dataset.csv: sample_date,B.1.1.7,B.1.617.2,cumPeopleVaccinatedFirstDoseByVaccinationDate,cumPeopleVaccinatedSecondDoseByVaccinationDate 2020-06-14,0.00031938677738741617,0.0,0.0,0.0 2020-06-22,0.0,0.0,0.0,0.0 2020-08-30,0.0,0.0,0.0,0.0 2020-09-03,0.0,0.0,0.0,0.0 2020-09-04,0.0,0.0,0.0,0.0 2020-09-20,0.00031938677738741617,0.0,0.0,0.0 2020-09-21,0.00031938677738741617,0.0,0.0,0.0 2020-09-23,0.00031938677738741617,0.0,0.0,0.0 2020-09-30,0.0006387735547748323,0.0,0.0,0.0 2020-10-01,0.00031938677738741617,0.0,0.0,0.0 2020-10-08,0.00031938677738741617,0.0,0.0,0.0 2020-10-09,0.0,0.0,0.0,0.0 2020-10-11,0.0009581603321622484,0.0,0.0,0.0 2020-10-12,0.0,0.0,0.0,0.0 2020-10-15,0.0009581603321622484,0.0,0.0,0.0 2020-10-17,0.00031938677738741617,0.0,0.0,0.0 2020-10-19,0.00031938677738741617,0.0,0.0,0.0 2020-10-20,0.0006387735547748323,0.0,0.0,0.0 2020-10-21,0.0047908016608112424,0.0,0.0,0.0 2020-10-22,0.00415202810603641,0.0,0.0,0.0 2020-10-23,0.0019163206643244969,0.0,0.0,0.0 2020-10-24,0.0015969338869370809,0.0,0.0,0.0 2020-10-25,0.00031938677738741617,0.0,0.0,0.0 2020-10-26,0.0006387735547748323,0.0,0.0,0.0 2020-10-27,0.00031938677738741617,0.0,0.0,0.0 2020-10-29,0.005748961992973491,0.0,0.0,0.0 2020-10-30,0.0028744809964867456,0.0,0.0,0.0 2020-10-31,0.0015969338869370809,0.0,0.0,0.0 2020-11-01,0.012456084318109231,0.0,0.0,0.0 2020-11-02,0.019801980198019802,0.0,0.0,0.0 2020-11-03,0.0025550942190993293,0.0,0.0,0.0 2020-11-04,0.0025550942190993293,0.0,0.0,0.0 2020-11-05,0.03257745129351645,0.0,0.0,0.0 2020-11-06,0.01341424465027148,0.0,0.0,0.0 2020-11-07,0.0006387735547748323,0.0,0.0,0.0 2020-11-08,0.031938677738741615,0.0,0.0,0.0 2020-11-09,0.013733631427658896,0.0,0.0,0.0 2020-11-10,0.005748961992973491,0.0,0.0,0.0 2020-11-11,0.013094857872884063,0.0,0.0,0.0 2020-11-12,0.048866176940274675,0.0,0.0,0.0 2020-11-13,0.05908655381667199,0.0,0.0,0.0 2020-11-14,0.041200894282976686,0.0,0.0,0.0 2020-11-15,0.028744809964867453,0.0,0.0,0.0 2020-11-16,0.0047908016608112424,0.0,0.0,0.0 2020-11-17,0.032896838070903864,0.0,0.0,0.0 2020-11-18,0.0316192909613542,0.0,0.0,0.0 2020-11-19,0.021079527307569467,0.0,0.0,0.0 2020-11-20,0.011817310763334398,0.0,0.0,0.0 2020-11-21,0.03928457361865219,0.0,0.0,0.0 2020-11-22,0.08463749600766528,0.0,0.0,0.0 2020-11-23,0.012456084318109231,0.0,0.0,0.0 2020-11-24,0.011817310763334398,0.0,0.0,0.0 2020-11-25,0.0063877355477483235,0.0,0.0,0.0 2020-11-26,0.03385499840306611,0.0,0.0,0.0 2020-11-27,0.029383583519642285,0.0,0.0,0.0 2020-11-28,0.024912168636218462,0.0,0.0,0.0 2020-11-29,0.01820504631108272,0.0,0.0,0.0 2020-11-30,0.012136697540721815,0.0,0.0,0.0 2020-12-01,0.0316192909613542,0.0,0.0,0.0 2020-12-02,0.023634621526668797,0.0,0.0,0.0 2020-12-03,0.018524433088470137,0.0,0.0,0.0 2020-12-04,0.053018205046311086,0.0,0.0,0.0 2020-12-05,0.03481315873522836,0.0,0.0,0.0 2020-12-06,0.02331523474928138,0.0,0.0,0.0 2020-12-07,0.09773235388054935,0.0,0.0,0.0 2020-12-08,0.07697221335036729,0.0,0.0001401194667996399,0.0 2020-12-09,0.06259980836793357,0.0,0.0003936790610460604,0.0 2020-12-10,0.3136378153944427,0.0,0.0007062084648938405,0.0 2020-12-11,0.12903225806451613,0.0,0.001036365289385474,0.0 2020-12-12,0.09070584477802619,0.0,0.0013162336766048529,0.0 2020-12-13,0.18300862344298946,0.0,0.001478638861395671,0.0 2020-12-14,0.2111146598530821,0.0,0.0018011465499022088,0.0 2020-12-15,0.23890130948578728,0.0,0.0033831149160930807,0.0 2020-12-16,0.25167678058128395,0.0,0.00595748589045405,0.0 2020-12-17,0.31523474928137973,0.0,0.008517961375568739,0.0 2020-12-18,0.32864899393165126,0.0,0.01056933260229826,0.0 2020-12-19,0.17470456723091665,0.0,0.012964306193590104,0.0 2020-12-20,0.15713829447460875,0.0,0.015065754116803365,0.0 2020-12-21,0.21047588629830724,0.0,0.01676306827753765,0.0 2020-12-22,0.14723730437559884,0.0,0.01906437303751028,0.0 2020-12-23,0.14564037048866177,0.0,0.021198931926527536,0.0 2020-12-24,0.15170871925902268,0.0,0.02192679088704701,0.0 2020-12-25,0.09070584477802619,0.0,0.021933593059878045,0.0 2020-12-26,0.2823379112104759,0.0,0.02197991135736565,0.0 2020-12-27,0.19801980198019803,0.0,0.02210968199312663,0.0 2020-12-28,0.1782178217821782,0.0,0.022348710875761225,0.0 2020-12-29,0.4231874800383264,0.0,0.023582259774451067,8.051846123113752e-05 2020-12-30,0.264771638454168,0.0,0.026482023405562088,0.00023117729180303356 2020-12-31,0.30150111785372086,0.0,0.02885891962703448,0.0004006508263802765 2021-01-01,0.2034493771957841,0.0,0.02951309279315082,0.00043944512225621693 2021-01-02,0.5397636537847333,0.0,0.030306909009292643,0.0006053533939731596 2021-01-03,0.41807729160012774,0.0,0.03073999832460101,0.0007397743936961635 2021-01-04,0.4573618652187799,0.0,0.03147405070318454,0.0010677079775792268 2021-01-05,0.39220696263174704,0.0,0.03301920263977241,0.002730497156138075 2021-01-06,0.25902267646119453,0.0,0.03538426784468655,0.005584401826727731 2021-01-07,0.3813478122005749,0.0,0.040748244304933734,0.008378297122441045 2021-01-08,0.49313318428617053,0.0,0.047347569460573134,0.01048031962546243 2021-01-09,0.5889492175023954,0.0,0.05298479740839862,0.012722975863488133 2021-01-10,0.4072181411689556,0.0,0.05586873341556816,0.014303340400672711 2021-01-11,0.5959757266049186,0.0,0.05884138881312539,0.01486601653921911 2021-01-12,0.7294793995528586,0.0,0.06347779745643731,0.01523750986933594 2021-01-13,0.46343021398914086,0.0,0.0701575311765152,0.015462545025122218 2021-01-14,0.5752155860747366,0.0,0.07776344557315519,0.015593859657204565 2021-01-15,0.6528265729798787,0.0,0.0865841433411029,0.015673283829835016 2021-01-16,0.4768444586394123,0.0,0.0942485511305701,0.015717161272759305 2021-01-17,0.48067709996806135,0.0,0.09819584937766074,0.015743847794763136 2021-01-18,0.6384541679974449,0.0,0.10217414118267024,0.015776394055725482 2021-01-19,0.5835196422868093,0.0,0.11000275395363061,0.01581991850232697 2021-01-20,0.5263494091344618,0.0,0.11841203936949415,0.01587629201507845 2021-01-21,0.5052698818268924,0.0,0.12788680443316394,0.015928253073794904 2021-01-22,0.6684765250718621,0.0,0.14033306032005413,0.015950385943234592 2021-01-23,0.510380070265091,0.0,0.15252191881090607,0.01596965954245958 2021-01-24,0.4065793676141808,0.0,0.157528741496126,0.015986356268528117 2021-01-25,0.9773235388054935,0.0,0.16253823740880094,0.01600291179606753 2021-01-26,0.8543596295113383,0.0,0.1685279343667078,0.01601685515081821 2021-01-27,0.8361545832002555,0.0,0.1747353802580035,0.016035634555191275 2021-01-28,0.7869690194825935,0.0,0.18360956904264975,0.01607344046136337 2021-01-29,1.0,0.0,0.1961699532145494,0.01615095845385069 2021-01-30,0.7502395400830406,0.0,0.20836087617808935,0.016230382626481143 2021-01-31,0.442031299904184,0.0,0.2157251149322383,0.016313301462707333 2021-02-01,0.5059086553816672,0.0,0.22264443335452064,0.01635809669607091 2021-02-02,0.45959757266049184,0.0,0.23046865250411935,0.016400244457013467 2021-02-03,0.4410731395720217,0.0,0.24005317869903547,0.016479527431114797 2021-02-04,0.5534972852123922,0.0,0.2503569883525362,0.016573106756289608 2021-02-05,0.4682210156499521,0.0,0.26091893649483505,0.016632233640358946 2021-02-06,0.42606196103481314,0.0,0.27339626534243977,0.016651613138480775 2021-02-07,0.44746087511977006,0.0,0.2793116148159794,0.016663544414191482 2021-02-08,0.6256786969019482,0.0,0.2855313152054104,0.016695843577727867 2021-02-09,0.5570105397636538,0.0,0.2935841085362108,0.01677900951138002 2021-02-10,0.6202491216863621,0.0,0.3023860937119735,0.01693062143202348 2021-02-11,0.5752155860747366,0.0,0.3126834717390231,0.017082868746047985 2021-02-12,0.6327052060044714,0.0,0.3240296548267835,0.017226714747589803 2021-02-13,0.47971893963589907,0.0,0.3355582261362507,0.01730670371433674 2021-02-14,0.5729798786330246,0.00024857071836937607,0.34041264838893714,0.01733219004884304 2021-02-15,0.8738422229319707,0.0,0.34572345144366856,0.01738662208181911 2021-02-16,0.7093580325774513,0.0,0.35385675920930165,0.01750099289040696 2021-02-17,0.5397636537847333,0.0,0.36488453708633295,0.0176663716676397 2021-02-18,0.5509421909932929,0.0,0.3753096212764555,0.017836339397068867 2021-02-19,0.5723411050782498,0.0,0.3836967797799193,0.017989116205577575 2021-02-20,0.5688278505269881,0.0,0.3918650247756573,0.018087143284419693 2021-02-21,0.6675183647396997,0.0,0.39479408803837907,0.018168332438664152 2021-02-22,0.6387735547748323,0.0,0.39863657459515495,0.018356479478717623 2021-02-23,0.560204407537528,0.0,0.40608336578922605,0.018701709882417983 2021-02-24,0.5519003513254551,0.0,0.4166513750115961,0.01922732140707017 2021-02-25,0.8153944426700734,0.0,0.4282178479222075,0.019838428641105004 2021-02-26,0.5697860108591505,0.0,0.44006715359107745,0.020426132218938062 2021-02-27,0.39252634940913445,0.0,0.45002084985076907,0.02096392211672692 2021-02-28,0.422868093260939,0.0,0.4541660463323253,0.021288855231866164 2021-03-01,0.6499520919833919,0.0,0.4580335977049416,0.02170327291483572 2021-03-02,0.6084318109230278,0.0,0.4625840925233137,0.02271393668665014 2021-03-03,0.613222612583839,0.0,0.46862818039627024,0.024328153571191433 2021-03-04,0.4525710635579687,0.0,0.47711837726094497,0.025956702106438488 2021-03-05,0.5560523794314916,0.0,0.48735432399172673,0.027281815002604938 2021-03-06,0.3538805493452571,0.0,0.4976691706338289,0.02805353556351468 2021-03-07,0.25806451612903225,0.0,0.5013857243566355,0.028346381312911225 2021-03-08,0.527307569466624,0.0,0.50581486523559,0.02915160122485488 2021-03-09,0.5697860108591505,0.0,0.5104909223415514,0.030859679831629225 2021-03-10,0.7920792079207921,0.0,0.5158690854346291,0.03337760260217593 2021-03-11,0.7604599169594379,0.0,0.5194781542396659,0.035289042390516515 2021-03-12,0.6180134142446503,0.0,0.5277388240895907,0.037709538175969864 2021-03-13,0.5413605876716704,0.0,0.5402115475750452,0.03908922430364151 2021-03-14,0.5138933248163526,0.0,0.5461634488022022,0.03955870941296817 2021-03-15,0.8907697221335037,0.0,0.5546333450187781,0.04064170213132473 2021-03-16,0.7141488342382626,0.0,0.5644990894484404,0.04301588479922646 2021-03-17,0.4573618652187799,0.0,0.5748059693433746,0.046385270000003885 2021-03-18,0.4816352603002236,0.0004971414367387521,0.586929849879751,0.0502180334737235 2021-03-19,0.40242733950814435,0.0,0.6012620809699422,0.05381379521631385 2021-03-20,0.5822420951772597,0.0,0.6196960752124455,0.056342307876544616 2021-03-21,0.5927818588310444,0.0,0.6274884009057742,0.057369986071118095 2021-03-22,0.9220696263174705,0.0,0.6345120546015971,0.05964614166017771 2021-03-23,0.6949856275950176,0.00024857071836937607,0.641433887445743,0.06460175113709823 2021-03-24,0.6129032258064516,0.0,0.6485488278890145,0.07229904775358964 2021-03-25,0.6595336953050144,0.0004971414367387521,0.6560138023232993,0.08041898686737191 2021-03-26,0.5528585116576173,0.00024857071836937607,0.6644618892386582,0.08855516381200305 2021-03-27,0.5091025231555414,0.00024857071836937607,0.6753999154889578,0.09695817067704024 2021-03-28,0.46151389332481635,0.00024857071836937607,0.6809472859396574,0.10025240296069311 2021-03-29,0.6368572341105079,0.0,0.6855452900974524,0.10494916023410293 2021-03-30,0.4899393165122964,0.0009942828734775043,0.6900052921963329,0.11347458622388784 2021-03-31,0.47013733631427657,0.0007457121551081282,0.694733305198276,0.12577580208089215 2021-04-01,0.4270201213669754,0.0004971414367387521,0.6978404953993359,0.14059727048417364 2021-04-02,0.3647396997764293,0.0004971414367387521,0.6996097220253694,0.1483431393946826 2021-04-03,0.3695305014372405,0.0014914243102162564,0.7015638512949736,0.15385955412963867 2021-04-04,0.31555413605876714,0.0007457121551081282,0.7022449155412313,0.1551709707684804 2021-04-05,0.3618652187799425,0.0012428535918468805,0.7026847011589764,0.1568343953404203 2021-04-06,0.38486106675183646,0.004225702212279393,0.70398418084569,0.1623714603602603 2021-04-07,0.3474928137975088,0.003231419338801889,0.7054079259076633,0.17582054965975213 2021-04-08,0.26700734589587993,0.0017399950285856326,0.7067551266745884,0.1901418222736232 2021-04-09,0.272436921111466,0.0014914243102162564,0.7083082718259427,0.20461763667761695 2021-04-10,0.20568508463749602,0.0012428535918468805,0.7102509141577699,0.22047444329567675 2021-04-11,0.23890130948578728,0.002982848620432513,0.7111627493959102,0.22601694445888787 2021-04-12,0.3024592781858831,0.002982848620432513,0.7117576086736438,0.2319028404449208 2021-04-13,0.312360268284893,0.002734277902063137,0.7126862243692739,0.24010905185999587 2021-04-14,0.3043755988502076,0.00571712652249565,0.7147450065236014,0.2503608416660763 2021-04-15,0.28393484509741296,0.007208550832711907,0.7172177154518766,0.2632542441542132 2021-04-16,0.25135739380389654,0.008451404424558787,0.71991113768458,0.2789361057946923 2021-04-17,0.21909932928776749,0.011185682326621925,0.7229351931585493,0.2957060433005763 2021-04-18,0.18428617055253912,0.004225702212279393,0.7247768880694521,0.3024863260696927 2021-04-19,0.3557968700095816,0.017399950285856326,0.7269435786231264,0.3095352390404614 2021-04-20,0.25934206323858194,0.013174248073576932,0.7291318138020322,0.3190416771109454 2021-04-21,0.3359948898115618,0.015659955257270694,0.7315792991088756,0.3309818130293555 2021-04-22,0.3417438518045353,0.019388516032811335,0.734294715915986,0.34329736053706555 2021-04-23,0.23826253593101246,0.017399950285856326,0.7371483729903979,0.35715112482101763 2021-04-24,0.15937400191632067,0.014168530947054437,0.7404653189100802,0.3744793613110453 2021-04-25,0.1526668795911849,0.02212279393487447,0.7420892648875941,0.38236198099700774 2021-04-26,0.33567550303417437,0.025105642555306985,0.7437305206745691,0.39070024833644307 2021-04-27,0.2551900351325455,0.025105642555306985,0.745941465053043,0.40154842542989744 2021-04-28,0.25998083679335676,0.03231419338801889,0.7486244325843135,0.415239987903522 2021-04-29,0.2618971574576813,0.02609992542878449,0.7515495257072506,0.4292295851724883 2021-04-30,0.21271159374001916,0.03529704200845141,0.7545810715116135,0.4418150689682451 2021-05-01,0.14595975726604918,0.03156848123291076,0.7578259196928064,0.4535969919347753 2021-05-02,0.10507824976045992,0.03181705195128014,0.7596885769417722,0.4583241774912107 2021-05-03,0.17087192590226766,0.036539895600298286,0.7612201510002966,0.46162003355794845 2021-05-04,0.2510380070265091,0.05070842654735272,0.7637852053799629,0.4688939816845034 2021-05-05,0.22452890450335355,0.06587124036788466,0.7666046398494312,0.48124641730794626 2021-05-06,0.2235707441711913,0.06388267462092966,0.7692614309257214,0.49615786447390997 2021-05-07,0.20121366975407218,0.06984837186179468,0.7721223930573372,0.5108081296606325 2021-05-08,0.13765570105397637,0.055182699478001494,0.7758037184064547,0.5250891606937041 2021-05-09,0.12871287128712872,0.061894108873974646,0.7777605473711124,0.5305886316051657 2021-05-10,0.264771638454168,0.11981108625403927,0.779709541926595,0.53693402290459 2021-05-11,0.19482593420632385,0.09942828734775043,0.7823364775517048,0.5477584080458875 2021-05-12,0.17725966145001598,0.10937111608252548,0.7863224184926993,0.5620122936617357 2021-05-13,0.13765570105397637,0.0686055182699478,0.7910200202238921,0.575307370665589 2021-05-14,0.14116895560523796,0.08376833209047974,0.7960093213588624,0.5873053626813064 2021-05-15,0.10284254231874801,0.0842654735272185,0.8016227814031771,0.5998483457197978 2021-05-16,0.07952730756946662,0.07680835197613721,0.8044619077659771,0.605310857916262 2021-05-17,0.18811881188118812,0.18568232662192394,0.8066140146728427,0.6124156855045625 2021-05-18,0.14500159693388695,0.20059656972408652,0.8107374018531819,0.62181396550174 2021-05-19,0.17885659533695306,0.20730797912005966,0.8170590289646842,0.6326657784573192 2021-05-20,0.15298626636857235,0.19786229182202336,0.8233315851484482,0.6456308038993527 2021-05-21,0.09517725966145002,0.16281382053194135,0.8284054502608316,0.6586966450911784 2021-05-22,0.08272117534334078,0.17176236639323889,0.8332911836826196,0.6771013791672231 2021-05-23,0.06802938358351965,0.16803380561769823,0.8360274834249972,0.6848652155905627 2021-05-24,0.10092622165442351,0.29778772060651254,0.8383513650465453,0.6939854050846367 2021-05-25,0.09453848610667519,0.3161819537658464,0.8425278991648016,0.7053534398626816 2021-05-26,0.10986905142127117,0.32861048968431517,0.8480174908478353,0.718261138701892 2021-05-27,0.1047588629830725,0.2923191648023863,0.8543014280989819,0.7308443633211829 2021-05-28,0.10795273075694667,0.3755903554561273,0.858992015916555,0.7439463160368313 2021-05-29,0.06802938358351965,0.4021874223216505,0.8631968703652754,0.7573141455828141 2021-05-30,0.05908655381667199,0.3731046482724335,0.8657854809076639,0.7634324190481475 2021-05-31,0.06930693069306931,0.4335073328361919,0.8676171445987109,0.7686419034804273 2021-06-01,0.08559565633982753,0.5824011931394482,0.8695091279468858,0.7781746750765922 2021-06-02,0.08623442989460237,0.6211782252050708,0.8729536212240496,0.7881017435653271 2021-06-03,0.060044714148834236,0.6132239622172508,0.877128037934421,0.7989947864914096 2021-06-04,0.09421909932928776,0.6619438230176485,0.8810232744151355,0.8103056358651606 2021-06-05,0.06739061002874482,0.6273924931643052,0.8857536165657524,0.8252986251816916 2021-06-06,0.05557329926541041,0.6189410887397464,0.888354190462692,0.8329703648644121 2021-06-07,0.07569466624081762,0.8883917474521501,0.8905588884879048,0.841233867582678 2021-06-08,0.05365697860108592,0.7874720357941835,0.8933270287513553,0.8499272845224844 2021-06-09,0.044714148834238264,0.6967437235893612,0.8972709656134282,0.8588417830592617 2021-06-10,0.0297029702970297,0.5575441213025105,0.9018691285768148,0.8676180482360297 2021-06-11,0.01916320664324497,0.4337559035545613,0.9065201208669356,0.8763553073191281 2021-06-12,0.016927499201533056,0.3348247576435496,0.9125850916751594,0.8861791243834434 2021-06-13,0.008942829766847652,0.29132488192890876,0.9158080511523397,0.8915240888044145 2021-06-14,0.009262216544235069,0.3338304747700721,0.9186771123691161,0.8972807881363019 2021-06-15,0.022037687639731716,0.6952522992791449,0.9226334360673172,0.9033125422029166 2021-06-16,0.015011178537208559,0.623912503107134,0.9271279783823242,0.9095643541771661 2021-06-17,0.009581603321622485,0.49540144171016653,0.9327092008915875,0.9156017208853133 2021-06-18,0.01341424465027148,0.6850608998260005,0.9378672859026027,0.9213436296712754 2021-06-19,0.0063877355477483235,0.3867760377827492,0.9445483430026088,0.9283495476899267 2021-06-20,0.0063877355477483235,0.46109868257519265,0.9481971661406893,0.9311337708863953 2021-06-21,0.009262216544235069,0.596818294804872,0.9513817475999512,0.9343933036315167 2021-06-22,0.009900990099009901,0.7991548595575442,0.9555178656951999,0.9386170811320041 2021-06-23,0.009900990099009901,0.686055182699478,0.960054809103107,0.942789144671201 2021-06-24,0.00830405621207282,0.7119065374098931,0.9651041916868184,0.9474591801229747 2021-06-25,0.00670712232513574,0.4742729306487696,0.9696391764924318,0.9520469674315356 2021-06-26,0.009581603321622485,0.7979120059656972,0.9751310179213434,0.9582717398874584 2021-06-27,0.006068348770360907,0.5846383296047726,0.9781309084778232,0.9617049115237661 2021-06-28,0.005110188438198659,0.540641312453393,0.9806936072412173,0.9652578901120329 2021-06-29,0.005748961992973491,1.0,0.983542420745092,0.9691595937675958 2021-06-30,0.0047908016608112424,0.4926671638081034,0.9864800859773472,0.9738878383129995 2021-07-01,0.0019163206643244969,0.30002485707183696,0.9894941366990591,0.9789364976206106 2021-07-02,0.0,0.002734277902063137,0.9920933607484719,0.9837332587522702 2021-07-03,0.0,0.0012428535918468805,0.995212990720857,0.9896178133522765 2021-07-04,0.002235707441711913,0.002734277902063137,0.9967851925431775,0.9926236476402002 2021-07-05,0.0,0.00024857071836937607,0.9982705012394115,0.9962877847705173 2021-07-06,0.0,0.00024857071836937607,1.0,1.0 Also, when I delete the stacked bar line of code, I get my two line plots displayed: That's weird. Why can't I get all of the 3 plots displayed at once?
I saved your CSV data to file (thanks for adding that to your question) and find that your code appears to run as expected without modification: It may be that you're encountering some kind of environment issue. Please update your question with your OS, pandas, and matplotlib versions. If you're running your code within an IDE (like Spyder or PyCharm), please list that as well.
Break text files into multiple datasets at each \newline with Pandas
I have a text file, textfile.qdp: Line to skip 1 Line to skip 2 Line to skip 3 1.25 0.649999976 2.24733017E-2 2.07460159E-3 3.01663446 1.89463757E-2 1.48296626E-2 2.98285842 2.0999999 0.199999988 7.33829737E-2 6.63989689E-3 3.48941302 3.8440533E-2 6.34965161E-3 3.44462299 2.5 0.200000048 0.118000358 8.37391801E-3 2.64556909 3.93543094E-2 6.16234308E-3 2.60005236 2.9000001 0.199999928 0.145619139 9.26280301E-3 2.56852388 4.85827066E-2 6.0398886E-3 2.51390147 3.29999995 0.200000048 0.167878062 9.94068757E-3 2.46484375 5.69529012E-2 6.81256084E-3 2.40107822 3.70000005 0.200000048 0.175842062 1.01562217E-2 2.28405786 6.24930188E-2 8.10874719E-3 2.21345592 4.10000038 0.200000048 0.181325018 1.03028165E-2 2.02467489 6.38177395E-2 1.2183371E-2 1.94867384 4.5 0.199999809 0.157546207 9.59824398E-3 1.76375055 6.11177757E-2 6.072836E-2 1.64190447 4.94999981 0.25 0.156071633 8.54758453E-3 1.51925421 5.52904457E-2 0.149736568 1.3142271 5.5 0.300000191 0.125403479 6.9860979E-3 1.52551162 4.61589135E-2 0.511757791 0.967594922 6.10000038 0.299999952 9.54503566E-2 6.10219687E-3 3.56054449 3.59460302E-2 2.85172343 0.672874987 6.86499977 0.464999914 5.7642214E-2 3.80936684E-3 4.10104704 2.42055673E-2 3.67026114 0.406580269 8.28999996 0.960000038 2.10143197E-2 1.60136714E-3 0.142320022 8.9181494E-3 6.96837786E-4 0.132705033 9.48999977 0.239999771 5.72929019E-3 1.6677354E-3 3.82030606E-2 2.56266794E-3 4.94769251E-4 3.51456255E-2 4.13999987 1.99999809E-2 2.47749758 4.67826687E-2 30.4350224 0.973279834 0.754008532 28.7077332 4.17999983 1.99999809E-2 2.44065595 4.64052781E-2 30.5456734 0.99132967 0.677066088 28.8772774 4.21999979 1.99999809E-2 2.4736743 4.67251018E-2 30.8877811 1.01084304 0.807663918 29.0692749 4.26000023 2.00002193E-2 2.48481822 4.68727946E-2 30.9508438 1.02374947 0.834929705 29.092165 4.30000019 1.99999809E-2 2.54010344 4.73690033E-2 31.119503 1.03903878 0.93061626 29.1498489 4.34000015 1.99999809E-2 2.49571872 4.69326451E-2 31.1599998 1.05370748 0.892735004 29.2135563 4.38000011 1.99999809E-2 2.58409572 4.77907397E-2 31.367794 1.06788957 1.05168498 29.2482204 4.42000008 1.99999809E-2 2.6437602 4.83172201E-2 31.5764256 1.08456028 1.1402396 29.3516254 4.46000004 1.99999809E-2 2.65394902 4.84031737E-2 31.5579567 1.09554553 1.1519351 29.3104763 4.5 1.99999809E-2 2.62269425 4.81106751E-2 31.644083 1.11161876 1.12954116 29.4029236 Each column is a different parameter value, and the text file includes several datasets that, in the end, I want to plot with, e.g., different colors. A new line in the text file marks a different set. import pandas as pd import matplotlib as plt names = ['e', 'de', 'y', 'y_err', 'total', 'model1', 'model2', 'model3'] for j in range(8): names.append('model%i' %j) df = pd.read_table('textfile.qdp', skiprows=3, names=names, delimiter=' ', skip_blank_lines=True) fig, ax = plt.subplots(figsize=(10, 6)) ax.errorbar(df.e, df.y, xerr=df.de, yerr=df.y_err, fmt='o', label='data') # Here I want to plot different dfs How do I do that with Pandas? I think it is related to this question where: dfs = { k: pd.read_csv(pd.io.common.StringIO('\n'.join(dat)), delim_whitespace=True) for k, *dat in map(str.splitlines, open('my.csv').read().split('\n\n')) } but I am not sure how this translates with read_table (also *dat returns Invalid syntax).
You could do it this way: First the basics (note that you should call matplotlib.pyplot and not matplotlib to access the .subplots function at the end): import pandas as pd import matplotlib.pyplot as plt import io names = ['e', 'de', 'y', 'y_err', 'total'] for j in range(8): names.append('model%i' %j) Keep your arguments of the read_table stored: kwargs = { "names":names, "delimiter":' ', "skip_blank_lines":True } Read and store the content of your files, skipping the rows not needed: skiprows=3 with open('textfile.qdp') as f: collect_lines_as_list = f.readlines() selected_lines = collect_lines_as_list[skiprows:] content = "".join(selected_lines) # Join all lines to get one string only Then split the content on blank lines (that is, two successive line return) and store it in a temp StringIO to recreate an object that Pandas will manage. Using dict comprehension (quite the same as the other answer you linked), you can collect all data in one shot: dfs = { k: pd.read_table(io.StringIO(data), **kwargs) for k, data in enumerate(content.split('\n\n')) } The plot your dataframes the way you want (by iterating through your dict values): fig, axs = plt.subplots(len(dfs), figsize=(10, 6)) for ax, df in zip(list(axs), dfs.values()): ax.errorbar(df.e, df.y, xerr=df.de, yerr=df.y_err, fmt='o', label='data') plt.show() You will get something like this: If you want to plot all data in the same subplot, proceed like this: fig, ax = plt.subplots(figsize=(10, 6)) for df in dfs.values(): ax.errorbar(df.e, df.y, xerr=df.de, yerr=df.y_err, fmt='o', label='data') plt.show()]