Numpy.polyfit Not Returning Polynomial - numpy

I am trying to create a python program in which the user inputs a set of data and the program spits out an output in which it creates a graph with a line/polynomial which best fits the data.
This is the code:
from matplotlib import pyplot as plt
import numpy as np
x = []
y = []
x_num = 0
while True:
sequence = int(input("Input 1 number in the sequence, type 9040321 to stop"))
if sequence == 9040321:
poly = np.polyfit(x, y, deg=2, rcond=None, full=False, w=None, cov=False)
plt.plot(poly)
plt.scatter(x, y, c="blue", label="data")
plt.legend()
plt.show()
break
else:
y.append(sequence)
x.append(x_num)
x_num += 1
I used the polynomial where I inputed 1, 2, 4, 8 each in separate inputs. MatPlotLib graphed it properly, however, for the degree of 2, the output was the following image:
This is clearly not correct, however I am unsure what the problem is. I think it has something to do with the degree, however when I change the degree to 3, it still does not fit. I am looking for a graph like y=sqrt(x) to go over each of the points and when that is not possible, create the line that fits the best.
Edit: I added a print(poly) feature and for the selected input above, it gives [0.75 0.05 1.05]. I do not know what to make of this.

Approximation by a second degree polynomial
np.polyfit gives the coefficients of a polynomial close to the given points. To plot the polynomial as a smooth curve with matplotlib, you need to calculate a lot of x,y pairs. Using np.linspace(start, stop, numsteps) for the xs, numpy's vectorization allows calculating all the corresponding ys in one go. E.g. ys = a * x**2 + b * x + c.
from matplotlib import pyplot as plt
import numpy as np
x = [0, 1, 2, 3, 4, 5, 6]
y = [1, 2, 4, 8, 16, 32, 64]
plt.scatter(x, y, color='crimson', label='given points')
poly = np.polyfit(x, y, deg=2, rcond=None, full=False, w=None, cov=False)
xs = np.linspace(min(x), max(x), 100)
ys = poly[0] * xs ** 2 + poly[1] * xs + poly[2]
plt.plot(xs, ys, color='dodgerblue', label=f'$({poly[0]:.2f})x^2+({poly[1]:.2f})x + ({poly[2]:.2f})$')
plt.legend()
plt.show()
Higher degree approximating polynomials
Given N points, an N-1 degree polynomial can pass exactly through each of them. Here is an example with 7 points and polynomials of up to degree 6,
from matplotlib import pyplot as plt
import numpy as np
x = [0, 1, 2, 3, 4, 5, 6]
y = [1, 2, 4, 8, 16, 32, 64]
plt.scatter(x, y, color='black', zorder=3, label='given points')
for degree in range(0, len(x)):
poly = np.polyfit(x, y, deg=degree, rcond=None, full=False, w=None, cov=False)
xs = np.linspace(min(x) - 0.5, max(x) + 0.5, 100)
ys = sum(poly_i * xs**i for i, poly_i in enumerate(poly[::-1]))
plt.plot(xs, ys, label=f'degree {degree}')
plt.legend()
plt.show()
Another example
x = [0, 1, 2, 3, 4]
y = [1, 1, 6, 5, 5]

import numpy as np
import matplotlib.pyplot as plt
x = [1, 2, 3, 4]
y = [1, 2, 4, 8]
coeffs = np.polyfit(x, y, 2)
print(coeffs)
poly = np.poly1d(coeffs)
print(poly)
x_cont = np.linspace(0, 4, 81)
y_cont = poly(x_cont)
plt.scatter(x, y)
plt.plot(x_cont, y_cont)
plt.grid(1)
plt.show()
Executing the code, you have the graph above and this is printed in the terminal:
[ 0.75 -1.45 1.75]
2
0.75 x - 1.45 x + 1.75
It seems to me that you had false expectations about the output of polyfit.

Related

Cublic spline interpolation produces straight lines

I would like to obtain a smooth curve going through specific points with integer coordinates. Instead of that I get straight line segments between the points. I tried interp1d(x,y,kind='cubic') and also CubicSpline, nothing works. Here is my code:
import matplotlib.pyplot as plt
import numpy as np
from scipy.interpolate import interp1d,CubicSpline
x = np.arange(34)
y = [8,3,0,1,6,2,1,7,6,2,0,2,6,0,1,6,2,2,0,2,7,0,2,8,6,3,6,2,0,1,6,2,7,2]
f = CubicSpline(x, y)
plt.figure(figsize=(10,3))
plt.plot(x, y, 'o', x, f(x))
plt.show()
and here is the result:
Can you tell me how to get smooth curves instead?
Now you are using the original x-values to draw the curve. You need a new array with much more intermediate x-values. Numpy's np.linspace() creates such an array between a given minimum and maximum.
import matplotlib.pyplot as plt
import numpy as np
from scipy.interpolate import interp1d, CubicSpline
y = [8, 3, 0, 1, 6, 2, 1, 7, 6, 2, 0, 2, 6, 0, 1, 6, 2, 2, 0, 2, 7, 0, 2, 8, 6, 3, 6, 2, 0, 1, 6, 2, 7, 2]
x = np.arange(len(y))
f = CubicSpline(x, y)
plt.figure(figsize=(10, 3))
xs = np.linspace(x.min(), x.max(), 500)
plt.plot(x, y, 'o', xs, f(xs))
plt.tight_layout()
plt.show()

Apply 2 different masks to a seaborn heatmap or manually change the color of a cell

I have a dataframe and I want to plot the seaborn heatmap:
import seaborn as sns
res = sns.heatmap(df, cmap='flare',xticklabels=1, yticklabels=1,linecolor='white',linewidths=0.5,
cbar=True,mask=df.isnull(),cbar_kws={'shrink': 0.6},vmin=vmin, vmax=vmax)
I have applied a mask for NaN cells. Now, I want to change the color of few cells by a customized color which is not in the colormap, for example blue, to show that those cells belong to another category.
My question is:
Is it possible to apply 2 masks or more with different colors to a seaborn heatmap or manually change the color of a cell to totally another color?
It is unclear how the blue squares are represented. The following solution supposes they are represented as ones in a second matrix. The first heatmap is drawn as before. Then the second heatmap uses a special colormap (in this case with one color, but also a full range is possible), masking out all places where nothing should be drawn.
Note that masks can be combined via the logical or (symbol: |).
from matplotlib import pyplot as plt
from matplotlib.colors import ListedColormap
import seaborn as sns
import pandas as pd
import numpy as np
N = 10
data = np.random.uniform(0, 45, size=(N, N))
for x, y in np.random.randint(0, N, 50).reshape(-1, 2):
data[x, y] = np.nan # fill in some nans at random places
df = pd.DataFrame(data)
up_triang = np.triu(np.ones_like(data)).astype(bool)
ax = sns.heatmap(df, cmap='flare', xticklabels=True, yticklabels=True, square=True,
linecolor='white', linewidths=0.5,
cbar=True, mask=df.isnull() | up_triang, cbar_kws={'shrink': 0.6, 'pad': 0}, vmin=0, vmax=45)
data_special = np.random.randint(0, 5, size=(N, N)) // 4
sns.heatmap(data_special, cmap=ListedColormap(['cornflowerblue']), linecolor='white', linewidths=0.5,
square=True, cbar=False, mask=(data_special != 1) | up_triang, ax=ax)
ax.plot([0, N, 0, 0], [0, N, N, 0], clip_on=False, color='black', lw=2)
ax.tick_params(left=False, bottom=False)
plt.show()
An alternative approach, when there is only one color for the special cells, is to use an "under" color for the colormap, and give these cells negative values. An additional benefit is that the color can be shown in the colorbar. Here is some sample code:
N = 10
data = np.random.uniform(0, 45, size=(N, N))
for x, y in np.random.randint(0, N, 50).reshape(-1, 2):
data[x, y] = np.nan
data_special = np.random.randint(0, 5, size=(N, N)) // 4
data[data_special == 1] = -1
df = pd.DataFrame(data)
up_triang = np.triu(np.ones_like(data)).astype(bool)
cmap = sns.color_palette('mako', as_cmap=True).copy()
cmap.set_under('crimson ')
ax = sns.heatmap(df, cmap=cmap, xticklabels=True, yticklabels=True, square=True,
linecolor='white', linewidths=0.5, cbar=True, mask=df.isnull() | up_triang,
cbar_kws={'shrink': 0.6, 'pad': 0, 'extend': 'min', 'extendrect': True}, vmin=0, vmax=45)
ax.plot([0, N, 0, 0], [0, N, N, 0], clip_on=False, color='black', lw=2)
ax.tick_params(left=False, bottom=False)
plt.show()

Matplotlib `fill_between`: Remove thin boundary

Consider the following code:
import matplotlib.pyplot as plt
import numpy as np
from pylab import *
graph_data = [[0, 1, 2, 3], [5, 8, 7, 9]]
x = range(len(graph_data[0]))
y = graph_data[1]
fig, ax = plt.subplots()
alpha = 0.5
plt.plot(x, y, '-o',markersize=3, color=[1., alpha, alpha], markeredgewidth=0.0)
ax.fill_between(x, 0, y, facecolor=[1., alpha, alpha], interpolate=False)
plt.show()
filename = 'test1.pdf'
fig.savefig(filename, bbox_inches='tight')
It works fine. However, when zoomed in the generated PDF, I can see two thin gray/black boundaries that separate the line:
I can see this when viewing in both Edge and Chrome. My question is, how can I get rid of the boundaries?
UPDATE I forgot to mention, I was using Sage to generate the graph. Now it seems a problem specific to Sage (and not to Python in general). This time I used native Python, and got correct result.
I could not reproduce it but maybe you can try to not plot the line.
import matplotlib.pyplot as plt
import numpy as np
from pylab import *
graph_data = [[0, 1, 2, 3], [5, 8, 7, 9]]
x = range(len(graph_data[0]))
y = graph_data[1]
fig, ax = plt.subplots()
alpha = 0.5
plt.plot(x, y, 'o',markersize=3, color=[1., alpha, alpha])
ax.fill_between(x, 0, y, facecolor=[1., alpha, alpha], interpolate=False)
plt.show()
filename = 'test1.pdf'
fig.savefig(filename, bbox_inches='tight')

Matplotlib using image for points on plot

I have the following matplotlib script. I want to replace the points on the plot with images. Let's say 'red.png' for the red points and 'blue.png' for the blue points. How can I adjust the following to plot these images instead of the default points?
from scipy import linalg
import numpy as np
import pylab as pl
import matplotlib as mpl
import matplotlib.image as image
from sklearn.qda import QDA
###############################################################################
# load sample dataset
from sklearn.datasets import load_iris
iris = load_iris()
X = iris.data[:, 0:2] # Take only 2 dimensions
y = iris.target
X = X[y > 0]
y = y[y > 0]
y -= 1
target_names = iris.target_names[1:]
###############################################################################
# QDA
qda = QDA()
y_pred = qda.fit(X, y, store_covariances=True).predict(X)
###############################################################################
# Plot results
# constants
dpi = 72; imageSize = (32,32)
# read in our png file
im_red = image.imread('red.png')
im_blue = image.imread('blue.png')
def plot_ellipse(splot, mean, cov, color):
v, w = linalg.eigh(cov)
u = w[0] / linalg.norm(w[0])
angle = np.arctan(u[1] / u[0])
angle = 180 * angle / np.pi # convert to degrees
# filled gaussian at 2 standard deviation
ell = mpl.patches.Ellipse(mean, 2 * v[0] ** 0.5, 2 * v[1] ** 0.5,
180 + angle, color=color)
ell.set_clip_box(splot.bbox)
ell.set_alpha(0.5)
splot.add_artist(ell)
xx, yy = np.meshgrid(np.linspace(4, 8.5, 200), np.linspace(1.5, 4.5, 200))
X_grid = np.c_[xx.ravel(), yy.ravel()]
zz_qda = qda.predict_proba(X_grid)[:, 1].reshape(xx.shape)
pl.figure()
splot = pl.subplot(1, 1, 1)
pl.contourf(xx, yy, zz_qda > 0.5, alpha=0.5)
pl.scatter(X[y == 0, 0], X[y == 0, 1], c='b', label=target_names[0])
pl.scatter(X[y == 1, 0], X[y == 1, 1], c='r', label=target_names[1])
pl.contour(xx, yy, zz_qda, [0.5], linewidths=2., colors='k')
print(xx)
pl.axis('tight')
pl.show()
You can plot images instead of markers in a figure using BboxImage as in this tutorial.
from matplotlib import pyplot as plt
from matplotlib.image import BboxImage
from matplotlib.transforms import Bbox, TransformedBbox
# Load images.
redMarker = plt.imread('red.jpg')
blueMarker = plt.imread('blue.jpg')
# Data
blueX = [1, 2, 3, 4]
blueY = [1, 3, 5, 2]
redX = [1, 2, 3, 4]
redY = [3, 2, 3, 4]
# Create figure
fig = plt.figure()
ax = fig.add_subplot(111)
# Plots an image at each x and y location.
def plotImage(xData, yData, im):
for x, y in zip(xData, yData):
bb = Bbox.from_bounds(x,y,1,1)
bb2 = TransformedBbox(bb,ax.transData)
bbox_image = BboxImage(bb2,
norm = None,
origin=None,
clip_on=False)
bbox_image.set_data(im)
ax.add_artist(bbox_image)
plotImage(blueX, blueY, blueMarker)
plotImage(redX, redY, redMarker)
# Set the x and y limits
ax.set_ylim(0,6)
ax.set_xlim(0,6)
plt.show()

How to flip y axis in a bar3d() plot?

I use bar3d() to plot a 3D barchart, and I'd like to flip the y axis. I've tried to use invert_yaxis(), but it seems effectless. I've also tried manually reverse the values in the list with [::-1], but it didn't help either. It keeps displaying the 3D barchart in the very same way.
Any idea how can I flip the y axis?
Here's an example how it's not working for me (not even with 3D line plots):
from matplotlib.pyplot import *
from mpl_toolkits.mplot3d.axes3d import Axes3D
fig1 = figure(1)
ax11 = subplot(2, 2, 1, projection='3d')
ax11.plot([1, 2, 3, 4], [1, 2, 3, 4])
ax12 = subplot(2, 2, 2, projection='3d')
ax12.invert_xaxis()
ax12.plot([1, 2, 3, 4], [1, 2, 3, 4])
ax21 = subplot(2, 2, 3)
ax21.plot([1, 2, 3, 4])
ax22 = subplot(2, 2, 4)
ax22.invert_xaxis()
ax22.plot([1, 2, 3, 4])
show()
And the plot looks like this: http://we.tl/cqSsecVy6P
Thanks,
Daniel
If I understand the question correctly I think the problem is that matplotlib rotates the 3D plot. To remedy this just set the initial viewing angle using ax.view_init(elev, azim). Taking the matplotlib hist3d demo then we just have
from mpl_toolkits.mplot3d import Axes3D
import matplotlib.pyplot as plt
import numpy as np
fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')
x, y = np.random.rand(2, 100) * 4
hist, xedges, yedges = np.histogram2d(x, y, bins=4)
elements = (len(xedges) - 1) * (len(yedges) - 1)
xpos, ypos = np.meshgrid(xedges[:-1]+0.25, yedges[:-1]+0.25)
xpos = xpos.flatten()
ypos = ypos.flatten()
zpos = np.zeros(elements)
dx = 0.5 * np.ones_like(zpos)
dy = dx.copy()
dz = hist.flatten()
ypos_inv = ypos
ax.bar3d(xpos, ypos, zpos, dx, dy, dz, color='b', zsort='average')
ax.view_init(ax.elev, ax.azim+90)
plt.show()
Here I have rotated the axis by 90 degrees which flips one of the axis but not the other.