Dynamically updating plot with watchdog - matplotlib

Whenever a new txt file is added to a directory, I would like to plot and show the data from the file. If another file appears, I want the plot to update and show the new data. Creating a plot outside the main caused thread errors, so I made a (not very good) fix using global variables.
The problem is that a white figure appears and the plots do not show. After stopping the program, the white figure disappears and the plot appears. The correct image is saved to file, but I would like the image to be shown in real time. If I comment out the plt.show(), no plots appear.
I tried the "Dynamically updating plot in matplotlib" answer (Dynamically updating plot in matplotlib) but found that because it never called show(), no window appeared. If I tried calling show(), it blocked updates.
Inserting plt.pause() did not work (Real time matplotlib plot is not working while still in a loop)
The example code did not work (How to update a plot in matplotlib?)
Here is my code:
import time
from watchdog.observers import Observer
from watchdog.events import PatternMatchingEventHandler
import matplotlib.pyplot as plt
import numpy as np
import config
class MyHandler(PatternMatchingEventHandler):
patterns=["*.txt", "*.TXT"]
def on_created(self, event):
self.process(event)
def process(self, event):
filename = event.src_path
if '_AIt' in filename:
config.isnew=True
config.fname=filename
if __name__ == '__main__':
observer = Observer()
observer.schedule(MyHandler(), path='.', recursive=True)
observer.start()
dat=[0,1]
fig = plt.figure()
ax = fig.add_subplot(111)
Ln, = ax.plot(dat)
ax.set_xlim([0,10])
ax.set_ylim([-1,1])
plt.ion()
plt.show()
try:
while True:
time.sleep(1)
if config.isnew:
config.isnew=False
dataarray = np.array(np.transpose(np.loadtxt(config.fname)))
dat = dataarray[15] #AI0
Ln.set_ydata(dat)
Ln.set_xdata(range(len(dat)))
plt.savefig(config.fname[:-4] + '.png', bbox_inches='tight')
plt.draw()
except KeyboardInterrupt:
observer.stop()
observer.join()
config.py (creates default values of the configuration setting)
isnew=False
fname=""
I am confused because the following example code works well (from pylab.ion() in python 2, matplotlib 1.1.1 and updating of the plot while the program runs)
import pylab
import time
import matplotlib.pyplot as plt
import numpy as np
dat=[0,1]
fig = plt.figure()
ax = fig.add_subplot(111)
Ln, = ax.plot(dat)
ax.set_xlim([0,20])
ax.set_ylim([0,40])
plt.ion()
plt.show()
for i in range (18):
dat=np.array(range(20))+i
Ln.set_ydata(dat)
Ln.set_xdata(range(len(dat)))
plt.pause(1)
print 'done with loop'

The following should do what you want:
import time
from watchdog.observers import Observer
from watchdog.events import PatternMatchingEventHandler
import matplotlib.pyplot as plt
plt.ion() # enter interactive mode
ax = fig.add_subplot(111)
Ln, = ax.plot(dat)
ax.set_xlim([0,10])
ax.set_ylim([-1,1])
plt.draw() # non-blocking drawing
plt.pause(.001) # This line is essential, without it the plot won't be shown
try:
while True:
time.sleep(1)
if config.isnew:
...
plt.draw()
plt.pause(.001)
except KeyboardInterrupt:
observer.stop()
observer.join()
The essential thin is to call plt.pause after the plt.draw call, else it won't be drawn, as the other python code block matplotlib. The value .001 is simply a try. I don't really know how it works under the hood, but this seems to work.

Related

Undo plt.gcf().subplots_adjust when plotting and saving more than one chart

When plotting successive charts in a single Python script, plt.clf() does not reverse the effect of plt.gcf().subplots_adjust(bottom=0.5). The first chart is adjusted to allow x-axis labels more display space, but this change persists into the second chart. How can one plot the second chart in dist() to look normal? Even calling sns.set() or re-importing plt in the function dist() or after box_adj() is called didn't seem to fix the issue.
import seaborn as sns
import matplotlib.pyplot as plt
import numpy as np
def box_adj():
tips = sns.load_dataset("tips")
ax = sns.boxplot(x="day", y="total_bill", hue="smoker",
data=tips, palette="Set3")
plt.gcf().subplots_adjust(bottom=0.5)
plt.savefig('box.png')
plt.clf()
def dist():
ax = sns.distplot(np.random.randn(1000), kde=False)
plt.savefig('dist.png')
plt.clf()
if __name__ == "__main__":
box_adj()
dist()
plt.clf() only clears the contents of the plot figure, leaving the figure intact. Its whitespace settings and its size don't change.
You could replace plt.clf() with plt.close() to fully close the figure (matplotlib will automatically create a new one when needed). Alternatively, you can explicitly call fig = plt.figure(...) or fig, ax = plt.subplots(...) to create a new figure with default settings.

How to draw graphics dynamically on jupyterlab notebook

I found an example that can run normally on my laptop, but there is a problem. When the drawing is finished, a repeated result graph will be drawn again. I want to know how to not display the last repeated image.
import numpy as np
import matplotlib
%matplotlib inline
import matplotlib.pyplot as plt
# set up matplotlib
is_ipython = 'inline' in matplotlib.get_backend()
if is_ipython:
from IPython import display
plt.ion()
def plot_durations(y):
plt.figure(2)
plt.clf()
plt.subplot(211)
plt.plot(y[:,0])
plt.subplot(212)
plt.plot(y[:,1])
if is_ipython:
display.clear_output(wait=True)
display.display(plt.gcf())
x = np.linspace(-10,10,10)
y = []
for i in range(len(x)):
y1 = np.cos(i/(3*3.14))
y2 = np.sin(i/(3*3.14))
y.append(np.array([y1,y2]))
plot_durations(np.array(y))
plt.ioff()
plt.show()
Replacing plt.show() with plt.close() at the end of your code will prevent jupyter notebook from displaying the final plot twice. An explanation is included here.

updated graphs through iteration, matplotlib

I'm trying to graph features of a data-set one by one by, via iteration.
So I want the graph to continuously update as I proceed through the loop.
I refered to this thread,real-time plotting in while loop with matplotlib but the answers are all over the place, and despite incorporating some of their suggestions as shown below, I still can't seem to get the code working. I'm using Jupyter Notebook.
import matplotlib.pyplot as plt
%matplotlib inline
import numpy as np
colors = ["darkblue", "darkgreen"]
f, (ax1, ax2) = plt.subplots(1, 2, sharey=True, sharex = True)
for i in range(X.shape[-1]-1):
idx = np.where(y == 1)[0]
ax1.scatter(X[idx, i], X[idx, i+1], color=colors[0], label=1)
idx = np.where(y == 0)[0]
ax2.scatter(X[idx, i], X[idx, i+1], color=colors[1], label=0)
plt.draw()
plt.pause(0.0001)
Any suggestions?
Thank you.
This is an example for real-time plotting in a Jupyter Notebook
%matplotlib inline
%load_ext autoreload #Reload all modules every time before executing the Python code typed.
%autoreload 2
%matplotlib notebook
import matplotlib.pyplot as plt
import numpy as np
import time
colors = ["darkblue", "darkgreen"]
# initialise the graph and settings
fig = plt.figure()
ax1 = fig.add_subplot(111)
ax2 = fig.add_subplot(211)
plt.ion() # interactive mode
fig.show()
fig.canvas.draw() # matplotlib canvas drawing
# plotting loop
for i in range(X.shape[-1]-1):
ax1.clear()
ax2.clear()
idx = np.where(y == 1)[0]
ax1.scatter(X[idx, i], X[idx, i+1], color=colors[0], label=1)
idx = np.where(y == 0)[0]
ax2.scatter(X[idx, i], X[idx, i+1], color=colors[1], label=0)
fig.canvas.draw() # draw
time.sleep(0.5) # sleep
For an animation you need an interactive backend. %matplotlib inline is no interactive backend (it essentially shows a printed version of the figure).
You may decide not to run you code in jupyter but as a script. In this case you would need to put plt.ion() to put interactive mode on.
Another option would be to use a FuncAnimation, as e.g in this example. To run such a FuncAnimation in Jupyter you will still need some interactive backend, either %matplotlib tk or %matplotlib notebook.
From matplotlib 2.1 on, we can also create an animation using JavaScript.
from IPython.display import HTML
HTML(ani.to_jshtml())
Some complete example:
import matplotlib.pyplot as plt
import matplotlib.animation
import numpy as np
t = np.linspace(0,2*np.pi)
x = np.sin(t)
fig, ax = plt.subplots()
ax.axis([0,2*np.pi,-1,1])
l, = ax.plot([],[])
def animate(i):
l.set_data(t[:i], x[:i])
ani = matplotlib.animation.FuncAnimation(fig, animate, frames=len(t))
from IPython.display import HTML
HTML(ani.to_jshtml())

Matplotlib animation not working in IPython Notebook (blank plot)

I've tried multiple animation sample codes and cannot get any of them working. Here's a basic one I've tried from the Matplotlib documentation:
"""
A simple example of an animated plot
"""
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.animation as animation
fig, ax = plt.subplots()
x = np.arange(0, 2*np.pi, 0.01) # x-array
line, = ax.plot(x, np.sin(x))
def animate(i):
line.set_ydata(np.sin(x+i/10.0)) # update the data
return line,
#Init only required for blitting to give a clean slate.
def init():
line.set_ydata(np.ma.array(x, mask=True))
return line,
ani = animation.FuncAnimation(fig, animate, np.arange(1, 200), init_func=init,
interval=25, blit=True)
plt.show()
When I execute the above in an IPython Notebook, I just see a blank plot generated. I've tried running this from multiple servers (including Wakari), on multiple machines, using multiple browsers (Chrome, FF, IE).
I can save the animation to an mp4 file just fine and it looks good when played.
Any help is appreciated!
To summarize the options you have:
Using display in a loop Use IPython.display.display(fig) to display a figure in the output. Using a loop you would want to clear the output before a new figure is shown. Note that this technique gives in general not so smooth resluts. I would hence advice to use any of the below.
import matplotlib.pyplot as plt
import matplotlib.animation
import numpy as np
from IPython.display import display, clear_output
t = np.linspace(0,2*np.pi)
x = np.sin(t)
fig, ax = plt.subplots()
l, = ax.plot([0,2*np.pi],[-1,1])
animate = lambda i: l.set_data(t[:i], x[:i])
for i in range(len(x)):
animate(i)
clear_output(wait=True)
display(fig)
plt.show()
%matplotlib notebook Use IPython magic %matplotlib notebook to set the backend to the notebook backend. This will keep the figure alive instead of displaying a static png file and can hence also show animations.
Complete example:
%matplotlib notebook
import matplotlib.pyplot as plt
import matplotlib.animation
import numpy as np
t = np.linspace(0,2*np.pi)
x = np.sin(t)
fig, ax = plt.subplots()
l, = ax.plot([0,2*np.pi],[-1,1])
animate = lambda i: l.set_data(t[:i], x[:i])
ani = matplotlib.animation.FuncAnimation(fig, animate, frames=len(t))
plt.show()
%matplotlib tk Use IPython magic %matplotlib tk to set the backend to the tk backend. This will open the figure in a new plotting window, which is interactive and can thus also show animations.
Complete example:
%matplotlib tk
import matplotlib.pyplot as plt
import matplotlib.animation
import numpy as np
t = np.linspace(0,2*np.pi)
x = np.sin(t)
fig, ax = plt.subplots()
l, = ax.plot([0,2*np.pi],[-1,1])
animate = lambda i: l.set_data(t[:i], x[:i])
ani = matplotlib.animation.FuncAnimation(fig, animate, frames=len(t))
plt.show()
Convert animation to mp4 video:
from IPython.display import HTML
HTML(ani.to_html5_video())
or use plt.rcParams["animation.html"] = "html5" at the beginning of the notebook.
This will require to have ffmpeg video codecs available to convert to HTML5 video. The video is then shown inline. This is therefore compatible with %matplotlib inline backend. Complete example:
%matplotlib inline
import matplotlib.pyplot as plt
plt.rcParams["animation.html"] = "html5"
import matplotlib.animation
import numpy as np
t = np.linspace(0,2*np.pi)
x = np.sin(t)
fig, ax = plt.subplots()
l, = ax.plot([0,2*np.pi],[-1,1])
animate = lambda i: l.set_data(t[:i], x[:i])
ani = matplotlib.animation.FuncAnimation(fig, animate, frames=len(t))
ani
%matplotlib inline
import matplotlib.pyplot as plt
import matplotlib.animation
import numpy as np
t = np.linspace(0,2*np.pi)
x = np.sin(t)
fig, ax = plt.subplots()
l, = ax.plot([0,2*np.pi],[-1,1])
animate = lambda i: l.set_data(t[:i], x[:i])
ani = matplotlib.animation.FuncAnimation(fig, animate, frames=len(t))
from IPython.display import HTML
HTML(ani.to_html5_video())
Convert animation to JavaScript:
from IPython.display import HTML
HTML(ani.to_jshtml())
or use plt.rcParams["animation.html"] = "jshtml" at the beginning of the notebook.
This will display the animation as HTML with JavaScript. This highly compatible with most new browsers and also with the %matplotlib inline backend. It is available in matplotlib 2.1 or higher.
Complete example:
%matplotlib inline
import matplotlib.pyplot as plt
plt.rcParams["animation.html"] = "jshtml"
import matplotlib.animation
import numpy as np
t = np.linspace(0,2*np.pi)
x = np.sin(t)
fig, ax = plt.subplots()
l, = ax.plot([0,2*np.pi],[-1,1])
animate = lambda i: l.set_data(t[:i], x[:i])
ani = matplotlib.animation.FuncAnimation(fig, animate, frames=len(t))
ani
%matplotlib inline
import matplotlib.pyplot as plt
import matplotlib.animation
import numpy as np
t = np.linspace(0,2*np.pi)
x = np.sin(t)
fig, ax = plt.subplots()
l, = ax.plot([0,2*np.pi],[-1,1])
animate = lambda i: l.set_data(t[:i], x[:i])
ani = matplotlib.animation.FuncAnimation(fig, animate, frames=len(t))
from IPython.display import HTML
HTML(ani.to_jshtml())
According to this answer, you can get animation (and full interactivity support) working in an IPython notebook enabling the nbagg backend with %matplotlib nbagg.
I was having the exact same problem as you until a moment ago. I am a complete novice, so tcaswell's answer was a bit cryptic to me. Perhaps you figured out what he meant or found your own solution. In case you have not, I will put this here.
I googled "matplotlib inline figures" and found this site, which mentions that you have to enable matplotlib mode. Unfortunately, just using %maplotlib didn't help at all.
Then I typed %matplotlib qt into the IPython console on a lark and it works just fine now, although the plot appears in a separate window.
I ran into this issue as well and found I needed to understand the concept of matplotlib backends, how to enable a specific backend, and which backends work with FuncAnimation. I put together an ipython notebook that explains the details and summarizes which backends work with FuncAnimation on Mac, Windows, and wakari.io. The notebook also summarizes which backends work with the ipython interact() widget, and where plots appear (inline or secondary window) for basic matplotlib plotting. Code and instructions are included so you can reproduce any of the results.
The bottom line is that you can't get an animation created with FuncAnimation to display inline in an ipython notebook. However, you can get it to display in a separate window. It turns out that I needed this to create visualizations for an undergraduate class I am teaching this semester, and while I would much prefer the animations to be inline, at least I was able to create some useful visualizations to show during class.
No inline video in Jupyter at the end of an animation also happens when
HTML(ani.to_html5_video())
is not at the very end of a notebook cell, as the output is then suppressed.
You may use it then as follows
out = HTML(ani.to_html5_video())
and just type out` in a new cell to get the video online.

Matplotlib plot in Tkinter - every update adds new NavigationToolbar?

I am working on a Tkinter-GUI to interactively generate Matplotlib-plots, depending on user-input. For this end, it needs to re-plot after the user changes the input.
I have gotten it to work in principle, but would like to include the NavigationToolbar. However, I cannot seem to get the updating of the NavigationToolbar to work correctly.
Here is a basic working version of the code (without the user input Entries):
# Import modules
from Tkinter import *
import numpy as np
import matplotlib.pyplot as plt
from matplotlib.backends.backend_tkagg import FigureCanvasTkAgg, NavigationToolbar2TkAgg
# global variable: do we already have a plot displayed?
show_plot = False
# plotting function
def plot(x, y):
fig = plt.figure()
ax1 = fig.add_subplot(1,1,1)
ax1.plot(x,y)
return fig
def main():
x = np.arange(0.0,3.0,0.01)
y = np.sin(2*np.pi*x)
fig = plot(x, y)
canvas = FigureCanvasTkAgg(fig, master=root)
toolbar = NavigationToolbar2TkAgg(canvas, toolbar_frame)
global show_plot
if show_plot:
#remove existing plot and toolbar widgets
canvas.get_tk_widget().grid_forget()
toolbar_frame.grid_forget()
toolbar_frame.grid(row=1,column=1)
canvas.get_tk_widget().grid(row=0,column=1)
show_plot=True
# GUI
root = Tk()
draw_button = Button(root, text="Plot!", command = main)
draw_button.grid(row=0, column=0)
fig = plt.figure()
canvas = FigureCanvasTkAgg(fig, master=root)
canvas.get_tk_widget().grid(row=0,column=1)
toolbar_frame = Frame(root)
toolbar_frame.grid(row=1,column=1)
root.mainloop()
Pressing "Plot!" once generates the plot and the NavigationToolbar.
Pressing it a second time replots, but generates a second NavigationToolbar (and another every time "Plot!" is pressed). Which sounds like grid_forget() is not working.
However, when I change
if show_plot:
#remove existing plot and toolbar widgets
canvas.get_tk_widget().grid_forget()
toolbar_frame.grid_forget()
toolbar_frame.grid(row=1,column=1)
canvas.get_tk_widget().grid(row=0,column=1)
show_plot=True
to
if show_plot:
#remove existing plot and toolbar widgets
canvas.get_tk_widget().grid_forget()
toolbar_frame.grid_forget()
else:
toolbar_frame.grid(row=1,column=1)
canvas.get_tk_widget().grid(row=0,column=1)
show_plot=True
then the NavigationToolbar does vanish when "Plot!" is pressed a second time (but then there is, as expected, no new NavigationToolbar to replace the old). So grid_forget() is working, just not as expected.
What am I doing wrong? Is there a better way to update the NavigationToolbar?
Any help greatly appreciated!
Lastalda
Edit:
I found that this will work if you destroy the NavigationToolbar instead of forgetting it. But you have to completely re-create the widget again afterwards, of course:
canvas = FigureCanvasTkAgg(fig, master=root)
toolbar_frame = Frame(root)
global show_plot
if show_plot: # if plot already present, remove plot and destroy NavigationToolbar
canvas.get_tk_widget().grid_forget()
toolbar_frame.destroy()
toolbar_frame = Frame(root)
toolbar = NavigationToolbar2TkAgg(canvas, toolbar_frame)
toolbar_frame.grid(row=21,column=4,columnspan=3)
canvas.get_tk_widget().grid(row=1,column=4,columnspan=3,rowspan=20)
show_plot = True
However, the updating approach showed by Hans below is much nicer since you don't have to destroy and recreate anything. I just wanted to highlight that the issue with my approach (apart from the inelegance and performance) was probably that I didn't use destroy().
A slightly different approach might be to reuse the figure for subsequent plots by clearing & redrawing it. That way, you don't have to destroy & regenerate neither the figure nor the toolbar:
from Tkinter import Tk, Button
import numpy as np
import matplotlib.pyplot as plt
from matplotlib.backends.backend_tkagg import FigureCanvasTkAgg, NavigationToolbar2TkAgg
# plotting function: clear current, plot & redraw
def plot(x, y):
plt.clf()
plt.plot(x,y)
# just plt.draw() won't do it here, strangely
plt.gcf().canvas.draw()
# just to see the plot change
plotShift = 0
def main():
global plotShift
x = np.arange(0.0,3.0,0.01)
y = np.sin(2*np.pi*x + plotShift)
plot(x, y)
plotShift += 1
# GUI
root = Tk()
draw_button = Button(root, text="Plot!", command = main)
draw_button.grid(row=0, column=0)
# init figure
fig = plt.figure()
canvas = FigureCanvasTkAgg(fig, master=root)
toolbar = NavigationToolbar2TkAgg(canvas, root)
canvas.get_tk_widget().grid(row=0,column=1)
toolbar.grid(row=1,column=1)
root.mainloop()
When you press the "Plot!" button it calls main. main creates the navigation toolbar. So, each time you press the button you get a toolbar. I don't know much about matplot, but it's pretty obvious why you get multiple toolbars.
By the way, grid_forget doesn't destroy the widget, it simply removes it from view. Even if you don't see it, it's still in memory.
Typically in a GUI, you create all the widgets exactly once rather than recreating the same widgets over and over.