How do I achieve consistent appearance in different backends in HoloViews? - matplotlib

I'm mystified by how to use HoloViews styles to customize plots and achieve a consistent appearance across backends. HoloViews is billed as a package that provides an abstraction layer to several backends, notably Bokeh and Matplotlib, but I'm completely failing in my attempts to get plots generated using these backends to look the same. Settings in one backend are ignored by another, and each backend has many (most) formatting options missing, so that it is necessary to break through the abstraction to lower level calls directly the the backends.
I suspect I'm just missing something or have failed to discover the appropriate documentation.
The code below for example (using settings that don't attempt to produce the same appearance, but expose some of the issues) results in Matplotlib figures (right) that
ignore the attempt to achieve a uniform appearance for scatter plot point color,
ignore the attempt to override the color of histogram bars,
have marginal histograms with axis labels that are explicitly removed in the Bokeh versions (left),
have marginal histograms that are not framed and lack the vertical axis present in the Bokeh version,
have no control or customizations over the styling of axes, and
have additional subplot labels not present on the Bokeh plot.
In addition, there are many further customizations to both backend's plots (gridlines, frame color, for example) that I can't find settings for.
How do I set styles in HoloViews to achieve full and consistent control over plots produced by Bokeh and Matplotlib?
import numpy as np
import pandas as pd
import holoviews as hv
hv.extension('bokeh', 'matplotlib')
ds = hv.Dataset({'x': np.random.randn(100), 'y1': np.random.randn(100), 'y2': np.random.randn(100), 'y3': np.random.randn(100)},
['x'],['y1', 'y2', 'y3'])
def mpl_style_hook(plot, element):
# Settings required here are neither complete, nor do they correspond directly to the backend's naming
# Where is the correspondence between handles and the backend's names documented?
pass
def bok_style_hook(plot, element):
# Such a small set of abstractions is provided, it is almost always necessary to resort to hooks
plot.state.title.align = "center"
plot.handles['xaxis'].axis_label_text_color = 'red'
plot.handles['yaxis'].axis_label_text_color = 'green'
plot.handles['xaxis'].axis_label_text_font_style = "normal"
plot.handles['yaxis'].axis_label_text_font_style = "normal"
# Attempt to set options that apply to both backends; but ignored by Matplotlib
hv.opts.defaults(hv.opts.Scatter(color='green'), hv.opts.Histogram(fill_color='yellow'))
# Explictily set backend to avoid warnings (`backend=` isn't sufficient)
hv.Store.current_backend = 'bokeh'
hv.opts.defaults(
hv.opts.Scatter(line_color='orange', size=6, fill_alpha=1.0, hooks=[bok_style_hook]),
hv.opts.Histogram(fill_color='cyan', fill_alpha=0.9, line_width=1, line_color='gray', hooks=[bok_style_hook]),
backend='bokeh')
hv.Store.current_backend = 'matplotlib'
hv.opts.defaults(
hv.opts.Scatter(hooks=[mpl_style_hook]),
# Histogram color ignored
hv.opts.Histogram(color='orange', hooks=[mpl_style_hook]),
backend='matplotlib')
hv.Store.current_backend = 'bokeh'
s1 = hv.Scatter(ds, 'x', 'y1').opts(hv.opts.Scatter(labelled=[None, 'y'])).hist(num_bins=51, dimension=['x','y1'])
s2 = hv.Scatter(ds, 'x', 'y2').opts(hv.opts.Scatter(labelled=[None, 'y'])).hist(num_bins=51, dimension='y2')
s3 = hv.Scatter(ds, 'x', 'y3').hist(num_bins=51, dimension='y3')
p = (s1 + s2 + s3).opts(hv.opts.Histogram(labelled=[None, None]), hv.opts.Layout(shared_axes=True)).cols(1)
hv.save(p, '_testHV.html', backend='bokeh')
hv.save(p, '_testHV.png', backend='matplotlib')
p

I don't think you're missing anything in terms of actual software support; what you're missing is that HoloViews in no way promises to make it simple to make plots from different backends to look the same. The plots are meant to show the same data in roughly the same way, but the backends each work in different ways, and some of those differences are in fact reasons to choose that particular backend over another.
There are certainly ways that HoloViews could map from an abstract notion of styling into the details of how that's done in different backends, but that's surprisingly tricky. And very few users ask for that; most pick their favorite backend and just use it, and would rather we spend our limited development time working on other features.
That said, if the backends can produce similar plots, you should be able to work out settings for use with HoloViews that will generate them in matching form. To do this, you'd work out the settings one backend at a time, then apply them per backend. E.g. .opts(line_width=3, backend='bokeh').opts(linewidth=4.5, backend='matplotlib'), with the appropriate option being used when that object is displayed by each backend. Here the two options differ only by one character in their names, but they work very differently for such a seemingly simple concept of line width: matplotlib accepts a width in "points" (which depends on dpi and knowing the absolute size in inches), while bokeh accepts pixels in screen space. They are both widths, but there's not necessarily any direct way to compare the two values, as it depends on separate settings you may have done for dpi and fig_size. You should be able to get it to look similar with enough effort, but trying to achieve that across all plots for all time is a massive task that would need some separate funding and developers to achieve! Still, it's already much easier to do that in HoloViews than it would be to completely rewrite a plot between Matplotlib and Bokeh natively, so HoloViews is still helping a good bit, just not solving everything for you...

Related

GeoViews: Applying matplotlib styling parameters to Polygons elements

Installed packages
holoviews 1.14.4, geoviews 1.9.1., matplotlib 3.4.2.
What I'm trying to do
I am trying to apply simple per-feature styles using GeoViews and the matplolib backend. I cannot figure out how to apply different edgecolor= parameters to different gv.Polygons elements in the same overlay. For some reason, they're always lightblue...
Similarly, facecolor= seems to have no effect.
Reproducible code sample
This uses a very small sample of the full dataset.
import pandas as pd
import geopandas as gpd
import geoviews as gv
from geoviews import opts
# loading both extensions as the full script calls for user input
# to choose between an interactive or static output
gv.extension('bokeh', 'matplotlib')
d1 = {'use': {0: 'Residential', 1: 'Residential'},
'geometry': {0: 'POLYGON ((13.80961103741604 51.04076975651729, 13.80965521888065 51.04079016168103, 13.80963851766593 51.04080454197601, 13.80959433642561 51.04078412781548, 13.80961103741604 51.04076975651729))',
1: 'POLYGON ((13.80977831740752 51.04313480566009, 13.80987122363639 51.04306085051974, 13.8099989591537 51.04312462457182, 13.80995486494384 51.04315973323087, 13.8099651184249 51.04316486464228, 13.80991634926543 51.04320371166482, 13.80977831740752 51.04313480566009))'}}
gdf1 = gpd.GeoDataFrame(pd.DataFrame(d1), geometry=gpd.GeoSeries.from_wkt(pd.DataFrame(d1)['geometry']), crs="EPSG:4326")
d2 = {'geometry': {1: 'POLYGON ((13.80894179055831 51.04544128170094, 13.80952887156242 51.0450399782091, 13.80954152432486 51.04504668985658, 13.80896834397535 51.04545611172818, 13.80894179055831 51.04544128170094))'}}
gdf2 = gpd.GeoDataFrame(pd.DataFrame(d2), geometry=gpd.GeoSeries.from_wkt(pd.DataFrame(d2)['geometry']), crs="EPSG:4326")
layout = gv.Polygons(gdf1, group="group1") * gv.Polygons(gdf2, group="group2")
layout.opts(
opts.Polygons('group1', cmap=['red'], edgecolor='black', linewidth=0.5, xaxis=None, yaxis=None, backend="matplotlib"),
opts.Polygons('group2', cmap=['lightblue'], edgecolor='blue', linewidth=0.5, backend="matplotlib"),
opts.Overlay(fig_size=500, backend='matplotlib')
)
gv.output(layout, backend='matplotlib')
gv.save(layout, "test.svg", dpi=600, backend='matplotlib')
Screenshot of the observed behaviour
This is a screen from the full dataset.
Expected behaviour
The red fill polygons belong to gdf1 and should have a black edgecolor but it's light blue instead. The blue fill polygon belongs to gdf2 and should have a lightblue fill and blue edgecolor, though the same color seems to be applied to both fill and edge.
What I've tried
Instead of using the group= parameter to specify styling for each of the Polygon elements (which I accidentally stumbled upon through the datashader documentation), I tried making multiple opts calls 'in-line' as suggested in the documentation for HoloViews here. This also has no effect.
Also, cmap=['color'] is the only method I've found to work to have GeoViews not use the automatically detected 'use' column in gdf1 as a vdim for color mapping. Is this the canonical approach and/or expected behaviour? color= or facecolor= seems to have no effect even though they are listed when calling gv.help(gv.opts.Polygons).
In short, I don't understand how to apply these particular styling parameters for the matplotlib backend and would very much appreciate any pointers.
2-Aug-21 Edit
Another strange behaviour seems to be that the figure in the, in my case VSCode-Python, interpreter, where the symbology seems to be faithfully represented, looks different from the .svg output generated by gv.save(layout, "test.svg", dpi=600, backend='matlplotlib'). The below images are outputs from the same run of the script.
Interpreter output:
gv.save() output:

Where is the list of available built-in colormap names?

Question
Where in the matplotlib documentations lists the name of available built-in colormap names to set as the name argument in matplotlib.cm.get_cmap(name)?
Choosing Colormaps in Matplotlib says:
Matplotlib has a number of built-in colormaps accessible via matplotlib.cm.get_cmap.
matplotlib.cm.get_cmap says:
matplotlib.cm.get_cmap(name=None, lut=None)
Get a colormap instance, defaulting to rc values if name is None.
name: matplotlib.colors.Colormap or str or None, default: None
https://www.kite.com/python/docs/matplotlib.pyplot.colormaps shows multiple names.
autumn sequential linearly-increasing shades of red-orange-yellow
bone sequential increasing black-white color map with a tinge of blue, to emulate X-ray film
cool linearly-decreasing shades of cyan-magenta
copper sequential increasing shades of black-copper
flag repetitive red-white-blue-black pattern (not cyclic at endpoints)
gray sequential linearly-increasing black-to-white grayscale
hot sequential black-red-yellow-white, to emulate blackbody radiation from an object at increasing temperatures
hsv cyclic red-yellow-green-cyan-blue-magenta-red, formed by changing the hue component in the HSV color space
inferno perceptually uniform shades of black-red-yellow
jet a spectral map with dark endpoints, blue-cyan-yellow-red; based on a fluid-jet simulation by NCSA [1]
magma perceptually uniform shades of black-red-white
pink sequential increasing pastel black-pink-white, meant for sepia tone colorization of photographs
plasma perceptually uniform shades of blue-red-yellow
prism repetitive red-yellow-green-blue-purple-...-green pattern (not cyclic at endpoints)
spring linearly-increasing shades of magenta-yellow
summer sequential linearly-increasing shades of green-yellow
viridis perceptually uniform shades of blue-green-yellow
winter linearly-increasing shades of blue-green
However, simply google 'matplotlib colormap names' seems not hitting the right documentation. I suppose there is a page listing the names as a enumeration or constant strings. Please help find it out.
There is some example code in the documentation (thanks to #Patrick Fitzgerald for posting the link in the comments, because it's not half as easy to find as it should be) which demonstrates how to generate a plot with an overview of the installed colormaps.
However, this uses an explicit list of maps, so it's limited to the specific version of matplotlib for which the documentation was written, as maps are added and removed between versions. To see what exactly your environment has, you can use this (somewhat crudely) adapted version of the code:
import numpy as np
import matplotlib.pyplot as plt
gradient = np.linspace(0, 1, 256)
gradient = np.vstack((gradient, gradient))
def plot_color_gradients(cmap_category, cmap_list):
# Create figure and adjust figure height to number of colormaps
nrows = len(cmap_list)
figh = 0.35 + 0.15 + (nrows + (nrows - 1) * 0.1) * 0.22
fig, axs = plt.subplots(nrows=nrows + 1, figsize=(6.4, figh))
fig.subplots_adjust(top=1 - 0.35 / figh, bottom=0.15 / figh,
left=0.2, right=0.99)
axs[0].set_title(cmap_category + ' colormaps', fontsize=14)
for ax, name in zip(axs, cmap_list):
ax.imshow(gradient, aspect='auto', cmap=plt.get_cmap(name))
ax.text(-0.01, 0.5, name, va='center', ha='right', fontsize=10,
transform=ax.transAxes)
# Turn off *all* ticks & spines, not just the ones with colormaps.
for ax in axs:
ax.set_axis_off()
cmaps = [name for name in plt.colormaps() if not name.endswith('_r')]
plot_color_gradients('all', cmaps)
plt.show()
This plots just all of them, without regarding the categories.
Since plt.colormaps() produces a list of all the map names, this version only removes all the names ending in '_r', (because those are the inverted versions of the other ones), and plots them all.
That's still a fairly long list, but you can have a look and then manually update/remove items from cmaps narrow it down to the ones you would consider for a given task.
You can also automatically reduce the list to monochrome/non-monochrome maps, because they provide that properties as an attribute:
cmaps_mono = [name for name in cmaps if plt.get_cmap(name).is_gray()]
cmaps_color = [name for name in cmaps if not plt.get_cmap(name).is_gray()]
That should at least give you a decent starting point.
It'd be nice if there was some way within matplotlib to select just certain types of maps (categorical, perceptually uniform, suitable for colourblind viewers ...), but I haven't found a way to do that automatically.
You can use my CMasher to make simple colormap overviews of a list of colormaps.
In your case, if you want to see what every colormap in MPL looks like, you can use the following:
import cmasher as cmr
import matplotlib.pyplot as plt
cmr.create_cmap_overview(plt.colormaps(), savefig='MPL_cmaps.png')
This will give you an overview with all colormaps that are registered in MPL, which will be all built-in colormaps and all colormaps my CMasher package adds, like shown below:

save pyplot figure "as figure" (not as image)

How can I save a figure using PyPlot in Julia, so that the figure can be reloaded as a figure later in Julia? (not as an image)
You can use serialize to store any Julia object. This beautifully works for plots as well.
Let us start by generating a plot:
using Plots
pyplot()
p = plot(rand(10));
using Serialization
Serialization.serialize("myfile.jld", p);
Note that you need a semicolon after plot command so it does not appear on the screen.
Let us now read the plot (to have a full test I ended the previous Julia session and started a new one):
using Plots
pyplot();
using Serialization
p2 = Serialization.deserialize("myfile.jld");
In order to display it now it is enough to type in REPL:
julia> p2
You might want also want to use plain PyPlot (I strongly recommend Plots for flexibility). In that case your best bet is to follow rules described in object-oriented API of Matplotlib:
using PyPlot
ioff()
fig = subplot()
fig.plot(rand(10))
fig.set_title("Hello world")
using Serialization
serialize("pp.jld", fig)
In order to plot de-serialize back the object:
using PyPlot
ioff()
using Serialization
fig = deserialize("pp.jld")
show()
Finally, note that the serialization is good only for short term storage. If anything changes (e.g. you update Julia packages) you might not be able to de-serialize the plot.
Hence another good alternative for processable plots are saving them to LaTeX or SVG format - both is possible in Julia.

Python: Setting Seaborn lineplot error band edge color

I am using Seaborn to make lineplots with a band indicating standard deviations. Something just like the second/third plot in the doc below:
https://seaborn.pydata.org/generated/seaborn.lineplot.html?highlight=lineplot#seaborn.lineplot
I am wondering is that possible to set the edgecolor for the error band separately? I can change linestyle of the band through err_kws. But, if I pass "edgecolor" through err_kws, it seems that nothing happens. Is there someway to allow me to get control with the edges?
Thanks!
As djakubosky notes, the color of the line and the error band are coupled together internally in seaborn's lineplot. I suggest that it is cleaner to modify the properties of the artists after the plot has been generated. This is a cleaner alternative than editing the library source code directly (maintenance headaches, etc).
For the example data shown on the sns.lineplot docs, we can update the error band properties as follows:
import seaborn as sns
fmri = sns.load_dataset("fmri")
ax = sns.lineplot(x="timepoint", y="signal", data=fmri)
# by inspection we see that the PolyCollection is the first artist
for child in ax.get_children():
print(type(child))
# and so we can update its properties
ax.get_children()[0].set_color('k')
ax.get_children()[0].set_hatch('//')
It may be more robust to select by property of the artist rather than selecting the first artist (especially if you have already rendered something on the same axes), e.g. along these lines:
from matplotlib.collections import PolyCollection
for child in ax.findobj(PolyCollection):
child.set_color('k')
child.set_hatch('//')
It appears that it isn't really possible to change this color under the current seaborn implementation. This is because they pass the color of the main line explicitly to the error band as ax.fillbetweenx(... color=original_color). After playing around in the past, I found that this color arg seems to supersede the other color arguments such as facecolor and edgecolor, thus it doesn't matter what you put in there in the err_kws. However you could fix it by editing line 810 in site-packages/seaborn/relational.py from:
ax.fill_between(x, low, high, color=line_color, **err_kws)
to
ax.fill_between(x, low, high, **err_kws)
and passing the colors explicitly through err_kws.

Accessing backend specific functionality with Julia Plots

Plots is simple and powerful but sometimes I would like to have a little bit more control over individual elements of the plot to fine-tune its appearance.
Is it possible to update the plot object of the backend directly?
E.g., for the default pyplot backend, I tried
using Plots
p = plot(sin)
p.o[:axes][1][:xaxis][:set_ticks_position]("top")
but the plot does not change. Calling p.o[:show]() afterwards does not help, either.
In other words: Is there a way to use the PyPlot interface for a plot that was initially created with Plots?
Edit:
The changes to the PyPlot object become visible (also in the gui) when saving the figure:
using Plots
using PyPlot
p = Plots.plot(sin, top_margin=1cm)
gui() # not needed when using the REPL
gca()[:xaxis][:set_ticks_position]("top")
PyPlot.savefig("test.png")
Here, I used p.o[:axes][1] == gca(). One has to set top_margin=1cm because the plot area is not adjusted automatically (for my actual fine-tuning, this doesn't matter).
This also works for subsequent updates as long as only the PyPlot interface is used. E.g., after the following commands, the plot will have a red right border in addition to labels at the top:
gca()[:spines]["right"][:set_color]("red")
PyPlot.savefig("test.png")
However, when a Plots command like plot!(xlabel="foo") is used, all previous changes made with PyPlot are overwritten (which is not suprising).
The remaining question is how to update the gui interactively without having to call PyPlot.savefig explicitly.
No - the plot is a Plots object, not a PyPlot object. In your specific example you can do plot(sin, xmirror = true).
I'm trying to do the same but didn't find a solution to update an existing plot. But here is a partial answer: you can query information from the PyPlot axes object
julia> Plots.plot(sin, 1:4)
julia> Plots.PyPlot.plt[:xlim]()
(1.0,4.0)
julia> Plots.plot(sin, 20:24)
julia> ax = Plots.PyPlot.plt[:xlim]()
(20.0,24.0)
and it gets updated.