I am currently trying to generate visualizations in zeppelin (0.8.1) notebooks using the pyspark interpreter with python 3.7.3.
Generating the following simple plot with seaborn (0.9.0) takes around 5 minutes (with very high CPU usage throughout the duration):
%pyspark
import seaborn as sns
import numpy as np
import pandas as pd
data = pd.DataFrame(np.random.rand(100,3))
sns.pairplot(data)
This behavior is rather inconsistent as the following (much more data intensive) plot is rendered instantly
%pyspark
import seaborn as sns
import numpy as np
import pandas as pd
df = pd.DataFrame(data = np.random.rand(10000,2))
sns.lineplot(x = 0, y = 1, data = df)
I noticed that using matplotlib (3.1.0) is generally much faster for and almost as snappy as I am used to from jupyter notebook environments.
I have already read about issue ZEPPELIN-1894 but I can render the mentioned scatterplot instantly as well.
Ok, after posting here the solution is to use the %spark.ipyspark interpreter, this might require installing additional packages:
pip install protobuf grpcio
Related
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from matplotlib import style
style.use("fivethirtyeight")
df = pd.DataFrame({'day':[1,2,3,4,5],'visitors':[200,302,480,590,680],'Bounce_rate':[20,30,40,50,60]})
df.set_index('day',inplace=True)
df.plot()
plt.show()
output is <Figure size 640x480 with 1 Axes>, the desired output is a graph (in VS Code).
I can see my code is correct when using cloud jupyter by achieving the desired output but it's not possible in VS Code jupyter view...
Am I doing something wrong? or is it something else?
I am using python 3.8 on Windows 10; trying to make a plot with about 700M points in it, sound wave analysis. Here: Interactive large plot with ~20 million sample points and gigabytes of data
Vaex was highly recommended. I am trying to use examples from the Vaex tutorial but the graph does not appear. I could not find a good example on Internet.
import vaex
import numpy as np
df = vaex.example()
df.plot1d(df.x, limits='99.7%');
The Vaex documents don't mention that pyplot.show() should be used to display. Plot1d plots a histogram. How to plot just connected points?
I am pretty sure that the vaex documentation explains that the (now deprecated) method .plot1d(...) is a wrapper around matplotlib plotting routines.
If you would like to create custom plots using the binned data, you can take this approach (I also found it in their docs)
import vaex
import numpy as np
import pylab as plt
# Load example data
df = vaex.example()
# Do the binning yourself
counts = df.count(binby=df.x, shape=64, limits='99.7%')
# Take care of the x-axis
limits = df.limits_percentage(df.x, percentage=99.7)
xvals = np.linspace(limits[0], limits[1], num=64)
# Create your custom plot via matplotlib, plotly or your favorite tool
p.plot(xvals, counts, marker='o', ms=5);
I was running seaborn ver. 0.10.1 on my jupyter notebook. This morning I upgraded to the latest version 0.11.0. Now, my kde jointplot doesn't give the color mapping that it used to. The code is the same. Only the versions are different.
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
%matplotlib notebook
np.random.seed(1234)
v1 = pd.Series(np.random.normal(0,10,1000), name='v1')
v2 = pd.Series(np.random.normal(60,15,1000), name='v2')
v3 = pd.Series(2*v1 + v2, name='v3')
# set the seaborn style for all the following plots
sns.set_style('white')
sns.jointplot(v1, v3, kind='kde', space=0);
The function kdeplot (which is used internally by jointplot()to draw the bivariate density plot) has been extensively changed in v.0.11. See What's new and the documentation.
You now have to pass fill=True to get a filled KDE, and you need to specify thresh=0 if you want to fill the available space with color.
sns.jointplot(x=v1, y=v3, kind='kde', space=0, fill=True, thresh=0, cmap='Blues');
I am new to JupyterLab trying to learn.
When I try to plot a graph, it works fine on jupyter notebook, but does not show the result on jupyterlab. Can anyone help me with this?
Here are the codes below:
import pandas as pd
import pandas_datareader.data as web
import time
# import matplotlib.pyplot as plt
import datetime as dt
import plotly.graph_objects as go
import numpy as np
from matplotlib import style
# from matplotlib.widgets import EllipseSelector
from alpha_vantage.timeseries import TimeSeries
Here is the code for plotting below:
def candlestick(df):
fig = go.Figure(data = [go.Candlestick(x = df["Date"], open = df["Open"], high = df["High"], low = df["Low"], close = df["Close"])])
fig.show()
JupyterLab Result:
Link to the image (JupyterLab)
JupyterNotebook Result:
Link to the image (Jupyter Notebook)
I have updated both JupyterLab and Notebook to the latest version. I do not know what is causing JupyterLab to stop showing the figure.
Thank you for reading my post. Help would be greatly appreciated.
Note*
I did not include the parts for data reading (Stock OHLC values). It contains the API keys. I am sorry for inconvenience.
Also, this is my second post on stack overflow. If this is not a well-written post, I am sorry. I will try to put more effort if it is possible. Thank you again for help.
TL;DR:
run the following and then restart your jupyter lab
jupyter labextension install #jupyterlab/plotly-extension
Start the lab with:
jupyter lab
Test with the following code:
import plotly.graph_objects as go
from alpha_vantage.timeseries import TimeSeries
def candlestick(df):
fig = go.Figure(data = [go.Candlestick(x = df.index, open = df["1. open"], high = df["2. high"], low = df["3. low"], close = df["4. close"])])
fig.show()
# preferable to save your key as an environment variable....
key = # key here
ts = TimeSeries(key = key, output_format = "pandas")
data_av_hist, meta_data_av_hist = ts.get_daily('AAPL')
candlestick(data_av_hist)
Note: Depending on system and installation of JupyterLab versus bare Jupyter, jlab may work instead of jupyter
Longer explanation:
Since this issue is with plotly and not matplotlib, you do NOT have to use the "inline magic" of:
%matplotlib inline
Each extension has to be installed to the jupyter lab, you can see the list with:
jupyter labextension list
For a more verbose explanation on another extension, please see related issue:
jupyterlab interactive plot
Patrick Collins already gave the correct answer.
However, the current JupyterLab might not be supported by the extension, and for various reasons one might not be able to update the JupyterLab:
ValueError: The extension "#jupyterlab/plotly-extension" does not yet support the current version of JupyterLab.
In this condition a quick workaround would be to save the image and show it again:
from IPython.display import Image
fig.write_image("image.png")
Image(filename='image.png')
To get the write_image() method of Plotly to work, kaleido must be installed:
pip install -U kaleido
This is a full example (originally from Plotly) to test this workaround:
import os
import pandas as pd
import plotly.express as px
from IPython.display import Image
df = pd.DataFrame([
dict(Task="Job A", Start='2009-01-01', Finish='2009-02-28', Resource="Alex"),
dict(Task="Job B", Start='2009-03-05', Finish='2009-04-15', Resource="Alex"),
dict(Task="Job C", Start='2009-02-20', Finish='2009-05-30', Resource="Max")
])
fig = px.timeline(df, x_start="Start", x_end="Finish", y="Resource", color="Resource")
if not os.path.exists("images"):
os.mkdir("images")
fig.write_image("images/fig1.png")
Image(filename='images/fig1.png')
I am changing the font-sizes in my python pandas dataframe plot. The only part that I could not change is the scaling of y-axis values (see the figure below).
Could you please help me with that?
Added:
Here is the simplest code to reproduce my problem:
import pandas as pd
start = 10**12
finish = 1.1*10**12
y = np.linspace(start , finish)
pd.DataFrame(y).plot()
plt.tick_params(axis='x', labelsize=17)
plt.tick_params(axis='y', labelsize=17)
You will see that this result in the graph similar to above. No change in the scaling of the y-axis.
Ma
There are just so many features that you can control with the plotting capabilities of pandas, which leverages matplotlib. I found that seaborn is a lot easier to produce pretty charts, and you have a lot more control over the parameters of your plots.
This is not the most elegant solution, but it works; however, it has a seborn dependency:
%pylab inline
import pandas as pd
import seaborn as sns
import numpy as np
sns.set(style="darkgrid")
sns.set(font_scale=1.5)
start = 10**12
finish = 1.1*10**12
y = np.linspace(start , finish)
pd.DataFrame(y).plot()
plt.tick_params(axis='x', labelsize=17)
plt.tick_params(axis='y', labelsize=17)
I use Jupyter Notebook an that's why I use %pylab inline. The key element here is the use of
font_scale=1.5
Which you can set to whatver you want that produces your desired result. This is what I get: