I am trying to get an fft plot on realtime audio using a USB microphone plugged into my raspi. I want to be able to activate an LED when a certain frequency is detected through the fft plot. I have so far tried to get just a live sound wave to be plotted but I am having trouble. I have followed this video: https://www.youtube.com/watch?v=AShHJdSIxkY&lc=z22efhti3uaff52pv04t1aokgg3rlotuia3kw5mpcsnubk0h00410.1510779722591217
I have tried changing the chunk size to a greater value and a lower value but have had no success.For some reason I get the -9981 error but it takes a long time to print the error. No plot is displayed. I have even tried overclocking my Raspberry Pi to see if that would work but it still doesn't work.
I was wondering if anyone else had tried something like this on their Pi and if it was possible or if I had to do it using a different package other than pyaudio.
Here is my python code:
import pyaudio
import struct
import numpy as np
import matplotlib.pyplot as plt
CHUNK = 100000
FORMAT = pyaudio.paInt16
CHANNELS = 1
RATE = 44100
p = pyaudio.PyAudio()
stream = p.open(
format = FORMAT,
channels = CHANNELS,
rate = RATE,
input = True,
output = True,
frames_per_buffer = CHUNK,
start = True
)
fig, ax = plt.subplots()
x = np.arange(0, 2 * CHUNK, 2)
line, = ax.plot(x, np.random.rand(CHUNK))
ax.set_ylim(0, 255)
ax.set_xlim(0, CHUNK)
while True:
data = stream.read(CHUNK)
data_int = np.array(struct.unpack(str(CHUNK*2) + 'B', data), dtype='b')[::2] + 127
line.set_ydata(data_int)
fig.canvas.draw()
fig.canvas.flush_events()
To display add:
plt.show(block=False)
after
ax.set_xlim(0, CHUNK)
But with rpi you have to configure your usb sound card as default card
Related
I have this kind of an animation and I want to integrate it to my GUI.
here is the plot
But, the background color is set to black right now. Here is the code. I am using Windows 10 and for GUI I am mostly using PyQt6 but for the matplotlib I used mlp.use("TkAgg") because it didn't create output if I dont use TkAgg.
I want to make it transparent. I only want the curves. I searched on the internet but everything is about save() function. Isn't there another solution for this? I don't want to save it, I am using animation, therefore it should be transparent everytime, not in a image.
import queue
import sys
from matplotlib.animation import FuncAnimation
import PyQt6.QtCore
import matplotlib as mlp
from matplotlib.backends.backend_qt5agg import FigureCanvasQTAgg
as FigureCanvas
mlp.use("TkAgg")
import matplotlib.pyplot as plt
import numpy as np
import sounddevice as sd
plt.rcParams['toolbar'] = 'None'
plt.rcParams.update({
"figure.facecolor": "black", # red with alpha = 30%
})
# Lets define audio variables
# We will use the default PC or Laptop mic to input the sound
device = 0 # id of the audio device by default
window = 1000 # window for the data
downsample = 1 # how much samples to drop
channels = [1] # a list of audio channels
interval = 40 # this is update interval in miliseconds for plot
# lets make a queue
q = queue.Queue()
# Please note that this sd.query_devices has an s in the end.
device_info = sd.query_devices(device, 'input')
samplerate = device_info['default_samplerate']
length = int(window*samplerate/(1000*downsample))
plotdata = np.zeros((length,len(channels)))
# next is to make fig and axis of matplotlib plt
fig,ax = plt.subplots(figsize=(2,1))
fig.subplots_adjust(0,0,1,1)
ax.axis("off")
fig.canvas.manager.window.overrideredirect(1)
# lets set the title
ax.set_title("On Action")
# Make a matplotlib.lines.Line2D plot item of color green
# R,G,B = 0,1,0.29
lines = ax.plot(plotdata,color = "purple")
# We will use an audio call back function to put the data in
queue
def audio_callback(indata,frames,time,status):
q.put(indata[::downsample,[0]])
# now we will use an another function
# It will take frame of audio samples from the queue and update
# to the lines
def update_plot(frame):
global plotdata
while True:
try:
data = q.get_nowait()
except queue.Empty:
break
shift = len(data)
plotdata = np.roll(plotdata, -shift,axis = 0)
# Elements that roll beyond the last position are
# re-introduced
plotdata[-shift:,:] = data
for column, line in enumerate(lines):
line.set_ydata(plotdata[:,column])
return lines
# Lets add the grid
ax.set_yticks([0])
# ax.yaxis.grid(True)
""" INPUT FROM MIC """
stream = sd.InputStream(device = device, channels = max(channels),
samplerate = samplerate, callback = audio_callback)
""" OUTPUT """
ani = FuncAnimation(fig,update_plot,interval=interval,blit=True, )
plt.get_current_fig_manager().window.wm_geometry("200x100+850+450")
with stream:
plt.show()
For a ML project I'm currently on, I need to verify if the trained data are good or not.
Let's say that I'm "splitting" the sky into several altitude grids (let's take 3 values for the moment) and for a given region (let's say, Europe).
One grid could be a signal reception strength (RSSI), another one the signal quality (RSRQ)
Each cell of the grid is therefor a rectangle and it has a mean value of each measurement (i.e. RSSI or RSRQ) performed in that area.
I have hundreds of millions of data
In the code below, I know how to draw a coloured mesh with xarray for each altitude: I just use xr.plot.pcolormesh(lat,lon, the_data_set); that's fine
But this will only give me a "flat" figure like this:
RSSI value at 3 different altitudes
I need to draw all the pcolormesh() of a dataset for each altitude in such way that:
1: I can have the map at the bottom
2: Each pcolormesh() is stacked and "displayed" at its altitude
3: I need to add a 3d scatter plot for testing my trained data
4: Need to be interactive as I have to zoom in areas
For 2 and 3 above, I managed to do something using plt and cartopy :
enter image description here
But plt/cartopy combination is not as interactive as plotly.
But plotly doesn't have the pcolormesh functionality
And still ... I don't know in anycase, how to "stack" the pcolormesh results that I did get above.
I've been digging Internet for few days but I didn't find something that could satisfy all my criteria.
What I did to get my pcolormesh:
import numpy as np
import xarray as xr
import cartopy.crs as ccrs
import matplotlib.pyplot as plt
class super_data():
def __init__(self, lon_bound,lat_bound,alt_bound,x_points,y_points,z_points):
self.lon_bound = lon_bound
self.lat_bound = lat_bound
self.alt_bound = alt_bound
self.x_points = x_points
self.y_points = y_points
self.z_points = z_points
self.lon, self.lat, self.alt = np.meshgrid(np.linspace(self.lon_bound[0], self.lon_bound[1], self.x_points),
np.linspace(self.lat_bound[0], self.lat_bound[1], self.y_points),
np.linspace(self.alt_bound[0], self.alt_bound[1], self.z_points))
self.this_xr = xr.Dataset(
coords={'lat': (('latitude', 'longitude','altitude'), self.lat),
'lon': (('latitude', 'longitude','altitude'), self.lon),
'alt': (('latitude', 'longitude','altitude'), self.alt)})
def add_data_array(self,ds_name,ds_min,ds_max):
def create_temp_data(ds_min,ds_max):
data = np.random.randint(ds_min,ds_max,size=self.y_points * self.x_points)
return data
temp_data = []
# Create "z_points" number of layers in the z axis
for i in range(self.z_points):
temp_data.append(create_temp_data(ds_min,ds_max))
data = np.concatenate(temp_data)
data = data.reshape(self.z_points,self.x_points, self.y_points)
self.this_xr[ds_name] = (("altitude","longitude","latitude"),data)
def plot(self,dataset, extent=None, plot_center=False):
# I want t
if np.sqrt(self.z_points) == np.floor(np.sqrt(self.z_points)):
side_size = int(np.sqrt(self.z_points))
else:
side_size = int(np.floor(np.sqrt(self.z_points) + 1))
fig = plt.figure()
i_ax=1
for i in range(side_size):
for j in range(side_size):
if i_ax < self.z_points+1:
this_dataset = self.this_xr[dataset].sel(altitude=i_ax-1)
# Initialize figure with subplots
ax = fig.add_subplot(side_size, side_size, i_ax, projection=ccrs.PlateCarree())
i_ax += 1
ax.coastlines()
this_dataset.plot.pcolormesh('lon', 'lat', ax=ax, infer_intervals=True, alpha=0.5)
else:
break
plt.tight_layout()
plt.show()
if __name__ == "__main__":
# Wanted coverage :
lons = [-15, 30]
lats = [35, 65]
alts = [1000, 5000]
xarr = super_data(lons,lats,alts,10,8,3)
# Add some fake data
xarr.add_data_array("RSSI",-120,-60)
xarr.add_data_array("pressure",700,1013)
xarr.plot("RSSI",0)
Thanks for you help
I have initialized one pipeline for two cameras and I am getting color and depth images from the same.
The problem is that I cannot find camera serial numbers for corresponding frames to determine which camera captured the frames.
Below is my code:
import pyrealsense2 as rs
import numpy as np
import cv2
import logging
import time
# Configure depth and color streams...
pipeline_1 = rs.pipeline()
config_1 = rs.config()
config_1.enable_device('938422072752')
config_1.enable_device('902512070386')
config_1.enable_stream(rs.stream.depth, 640, 480, rs.format.z16, 30)
config_1.enable_stream(rs.stream.color, 640, 480, rs.format.bgr8, 30)
# Start streaming from both cameras
pipeline_1.start(config_1)
try:
while True:
# Camera 1
# Wait for a coherent pair of frames: depth and color
frames_1 = pipeline_1.wait_for_frames()
depth_frame_1 = frames_1.get_depth_frame()
color_frame_1 = frames_1.get_color_frame()
if not depth_frame_1 or not color_frame_1:
continue
# Convert images to numpy arrays
depth_image_1 = np.asanyarray(depth_frame_1.get_data())
color_image_1 = np.asanyarray(color_frame_1.get_data())
# Apply colormap on depth image (image must be converted to 8-bit per pixel first)
depth_colormap_1 = cv2.applyColorMap(cv2.convertScaleAbs(depth_image_1, alpha=0.5), cv2.COLORMAP_JET)
# Camera 2
# Wait for a coherent pair of frames: depth and color
frames_2 = pipeline_1.wait_for_frames()
depth_frame_2 = frames_2.get_depth_frame()
color_frame_2 = frames_2.get_color_frame()
if not depth_frame_2 or not color_frame_2:
continue
# Convert images to numpy arrays
depth_image_2 = np.asanyarray(depth_frame_2.get_data())
color_image_2 = np.asanyarray(color_frame_2.get_data())
# Apply colormap on depth image (image must be converted to 8-bit per pixel first)
depth_colormap_2 = cv2.applyColorMap(cv2.convertScaleAbs(depth_image_2, alpha=0.5), cv2.COLORMAP_JET)
# Stack all images horizontally
images = np.hstack((color_image_1, depth_colormap_1,color_image_2, depth_colormap_2))
# Show images from both cameras
cv2.namedWindow('RealSense', cv2.WINDOW_NORMAL)
cv2.imshow('RealSense', images)
cv2.waitKey(20)
finally:
pipeline_1.stop()
How can I find camera serial numbers after wait_for_frames() to determine which camera captured depth and color image.
I adopted your code, combined it with the C++ example posted by nayab to compose the following code that grabs the color image (only) of multiple RealSense cameras and stacks them horizontally:
import pyrealsense2 as rs
import numpy as np
import cv2
import logging
import time
realsense_ctx = rs.context() # The context encapsulates all of the devices and sensors, and provides some additional functionalities.
connected_devices = []
# get serial numbers of connected devices:
for i in range(len(realsense_ctx.devices)):
detected_camera = realsense_ctx.devices[i].get_info(
rs.camera_info.serial_number)
connected_devices.append(detected_camera)
pipelines = []
configs = []
for i in range(len(realsense_ctx.devices)):
pipelines.append(rs.pipeline()) # one pipeline for each device
configs.append(rs.config()) # one config for each device
configs[i].enable_device(connected_devices[i])
configs[i].enable_stream(rs.stream.color, 1920, 1080, rs.format.bgr8, 30)
pipelines[i].start(configs[i])
try:
while True:
images = []
for i in range(len(pipelines)):
print("waiting for frame at cam", i)
frames = pipelines[i].wait_for_frames()
color_frame = frames.get_color_frame()
images.append(np.asanyarray(color_frame.get_data()))
# Stack all images horizontally
image_composite = images[0]
for i in range(1, len(images)):
images_composite = np.hstack((image_composite, images[i]))
# Show images from both cameras
cv2.namedWindow('RealSense', cv2.WINDOW_NORMAL)
cv2.imshow('RealSense', images_composite)
cv2.waitKey(20)
finally:
for i in range(len(pipelines)):
pipelines[i].stop()
This will look for the connected devices and find the serial numbers.
They are saved in a list and you can use them to start the available cameras.
# Configure depth and color streams...
realsense_ctx = rs.context()
connected_devices = []
for i in range(len(realsense_ctx.devices)):
detected_camera = ealsense_ctx.devices[i].get_info(rs.camera_info.serial_number)
connected_devices.append(detected_camera)
Using Python matplotlib I would like to plot sensor data over a period of several hours. The signal arrives via an audio card and gets sampled over short chunks of data. In the example below amplitude and RMS is plotted.
In order to plot RMS and other properties over much larger time periods than shown here, perhaps down sampling is needed. I am not sure how to accomplish that and would appreciate any further advice. The intention is to run the code on a Raspberry Pi.
Update 1. A very minimal example is shown for getting a longer time view of RMS.
Noticable is a considerable delay in response to audio signals in particular when adding more plots to the figure.
I also tried using Funcanimation without blitting because I would like to show a real-time axis and this is equally slow. Using PyQT should give better results.
import pyaudio
import struct
import matplotlib.pyplot as plt
import numpy as np
mic = pyaudio.PyAudio()
FORMAT = pyaudio.paInt16
CHANNELS = 1
RATE = 44100
CHUNK = int(RATE/20)
stream = mic.open(format=FORMAT, channels=CHANNELS, rate=RATE, input=True,
output=True,
frames_per_buffer=CHUNK)
fig = plt.figure()
ax1 = fig.add_subplot(2, 1, 1)
ax2 = fig.add_subplot(2, 1, 2)
ax1.set_xlabel("Samples = 2*Chunk length ")
ax1.set_ylabel("Amplitude")
ax1.set_title('Audio example')
fig.tight_layout(pad=3.0)
x = np.arange(0, 2 * CHUNK, 2)
ax1.set_ylim(-10e3, 10e3)
ax1.set_xlim(0, CHUNK)
line1, = ax1.plot(x, np.random.rand(CHUNK))
line2, = ax2.plot(x, np.random.rand(CHUNK))
ts = []
rs = []
while True:
data = stream.read(CHUNK)
data = np.frombuffer(data, np.int16)
d = np.frombuffer(data, np.int16).astype(np.float)
rms2 = np.sqrt( np.mean(d**2) )
#print(rms2)
# Add x and y to lists
ts.append(dt.datetime.now())
rs.append(rms2)
#Draw x and y lists
ax2.clear()
ax2.plot(ts,rs,color= 'black')
# Format plot
ax2.set_xlabel("Time in UTC")
ax2.set_ylabel("RMS values")
ax2.set_title('RMS')
line1.set_ydata(data)
line2.set_ydata(rms2)
plt.setp(ax2.get_xticklabels(), ha="right", rotation=45)
fig.gca().relim()
fig.gca().autoscale_view()
#fig.canvas.draw()
#fig.canvas.flush_events()
plt.pause(0.01)
For years, I've been struggling to get efficient live plotting in matplotlib, and to this day I remain unsatisfied.
I want a redraw_figure function that updates the figure "live" (as the code runs), and will display the latest plots if I stop at a breakpoint.
Here is some demo code:
import time
from matplotlib import pyplot as plt
import numpy as np
def live_update_demo():
plt.subplot(2, 1, 1)
h1 = plt.imshow(np.random.randn(30, 30))
redraw_figure()
plt.subplot(2, 1, 2)
h2, = plt.plot(np.random.randn(50))
redraw_figure()
t_start = time.time()
for i in xrange(1000):
h1.set_data(np.random.randn(30, 30))
redraw_figure()
h2.set_ydata(np.random.randn(50))
redraw_figure()
print 'Mean Frame Rate: %.3gFPS' % ((i+1) / (time.time() - t_start))
def redraw_figure():
plt.draw()
plt.pause(0.00001)
live_update_demo()
Plots should update live when the code is run, and we should see the latest data when stopping at any breakpoint after redraw_figure(). The question is how to best implement redraw_figure()
In the implementation above (plt.draw(); plt.pause(0.00001)), it works, but is very slow (~3.7FPS)
I can implement it as:
def redraw_figure():
plt.gcf().canvas.flush_events()
plt.show(block=False)
And it runs faster (~11FPS), but plots are not up-to date when you stop at breakpoints (eg if I put a breakpoint on the t_start = ... line, the second plot does not appear).
Strangely enough, what does actually work is calling the show twice:
def redraw_figure():
plt.gcf().canvas.flush_events()
plt.show(block=False)
plt.show(block=False)
Which gives ~11FPS and does keep plots up-to-data if your break on any line.
Now I've heard it said that the "block" keyword is deprecated. And calling the same function twice seems like a weird, probably-non-portable hack anyway.
So what can I put in this function that will plot at a reasonable frame rate, isn't a giant kludge, and preferably will work across backends and systems?
Some notes:
I'm on OSX, and using TkAgg backend, but solutions on any backend/system are welcome
Interactive mode "On" will not work, because it does not update live. It just updates when in the Python console when the interpreter waits for user input.
A blog suggested the implementation:
def redraw_figure():
fig = plt.gcf()
fig.canvas.draw()
fig.canvas.flush_events()
But at least on my system, that does not redraw the plots at all.
So, if anybody has an answer, you would directly make me and thousands of others very happy. Their happiness would probably trickle through to their friends and relatives, and their friends and relatives, and so on, so that you could potentially improve the lives of billions.
Conclusions
ImportanceOfBeingErnest shows how you can use blit for faster plotting, but it's not as simple as putting something different in the redraw_figure function (you need to keep track of what things to redraw).
First of all, the code that is posted in the question runs with 7 fps on my machine, with QT4Agg as backend.
Now, as has been suggested in many posts, like here or here, using blit might be an option. Although this article mentions that blit causes strong memory leakage, I could not observe that.
I have modified your code a bit and compared the frame rate with and without the use of blit. The code below gives
28 fps when run without blit
175 fps with blit
Code:
import time
from matplotlib import pyplot as plt
import numpy as np
def live_update_demo(blit = False):
x = np.linspace(0,50., num=100)
X,Y = np.meshgrid(x,x)
fig = plt.figure()
ax1 = fig.add_subplot(2, 1, 1)
ax2 = fig.add_subplot(2, 1, 2)
img = ax1.imshow(X, vmin=-1, vmax=1, interpolation="None", cmap="RdBu")
line, = ax2.plot([], lw=3)
text = ax2.text(0.8,0.5, "")
ax2.set_xlim(x.min(), x.max())
ax2.set_ylim([-1.1, 1.1])
fig.canvas.draw() # note that the first draw comes before setting data
if blit:
# cache the background
axbackground = fig.canvas.copy_from_bbox(ax1.bbox)
ax2background = fig.canvas.copy_from_bbox(ax2.bbox)
plt.show(block=False)
t_start = time.time()
k=0.
for i in np.arange(1000):
img.set_data(np.sin(X/3.+k)*np.cos(Y/3.+k))
line.set_data(x, np.sin(x/3.+k))
tx = 'Mean Frame Rate:\n {fps:.3f}FPS'.format(fps= ((i+1) / (time.time() - t_start)) )
text.set_text(tx)
#print tx
k+=0.11
if blit:
# restore background
fig.canvas.restore_region(axbackground)
fig.canvas.restore_region(ax2background)
# redraw just the points
ax1.draw_artist(img)
ax2.draw_artist(line)
ax2.draw_artist(text)
# fill in the axes rectangle
fig.canvas.blit(ax1.bbox)
fig.canvas.blit(ax2.bbox)
# in this post http://bastibe.de/2013-05-30-speeding-up-matplotlib.html
# it is mentionned that blit causes strong memory leakage.
# however, I did not observe that.
else:
# redraw everything
fig.canvas.draw()
fig.canvas.flush_events()
#alternatively you could use
#plt.pause(0.000000000001)
# however plt.pause calls canvas.draw(), as can be read here:
#http://bastibe.de/2013-05-30-speeding-up-matplotlib.html
live_update_demo(True) # 175 fps
#live_update_demo(False) # 28 fps
Update:
For faster plotting, one may consider using pyqtgraph.
As the pyqtgraph documentation puts it: "For plotting, pyqtgraph is not nearly as complete/mature as matplotlib, but runs much faster."
I ported the above example to pyqtgraph. And although it looks kind of ugly, it runs with 250 fps on my machine.
Summing that up,
matplotlib (without blitting): 28 fps
matplotlib (with blitting): 175 fps
pyqtgraph : 250 fps
pyqtgraph code:
import sys
import time
from pyqtgraph.Qt import QtCore, QtGui
import numpy as np
import pyqtgraph as pg
class App(QtGui.QMainWindow):
def __init__(self, parent=None):
super(App, self).__init__(parent)
#### Create Gui Elements ###########
self.mainbox = QtGui.QWidget()
self.setCentralWidget(self.mainbox)
self.mainbox.setLayout(QtGui.QVBoxLayout())
self.canvas = pg.GraphicsLayoutWidget()
self.mainbox.layout().addWidget(self.canvas)
self.label = QtGui.QLabel()
self.mainbox.layout().addWidget(self.label)
self.view = self.canvas.addViewBox()
self.view.setAspectLocked(True)
self.view.setRange(QtCore.QRectF(0,0, 100, 100))
# image plot
self.img = pg.ImageItem(border='w')
self.view.addItem(self.img)
self.canvas.nextRow()
# line plot
self.otherplot = self.canvas.addPlot()
self.h2 = self.otherplot.plot(pen='y')
#### Set Data #####################
self.x = np.linspace(0,50., num=100)
self.X,self.Y = np.meshgrid(self.x,self.x)
self.counter = 0
self.fps = 0.
self.lastupdate = time.time()
#### Start #####################
self._update()
def _update(self):
self.data = np.sin(self.X/3.+self.counter/9.)*np.cos(self.Y/3.+self.counter/9.)
self.ydata = np.sin(self.x/3.+ self.counter/9.)
self.img.setImage(self.data)
self.h2.setData(self.ydata)
now = time.time()
dt = (now-self.lastupdate)
if dt <= 0:
dt = 0.000000000001
fps2 = 1.0 / dt
self.lastupdate = now
self.fps = self.fps * 0.9 + fps2 * 0.1
tx = 'Mean Frame Rate: {fps:.3f} FPS'.format(fps=self.fps )
self.label.setText(tx)
QtCore.QTimer.singleShot(1, self._update)
self.counter += 1
if __name__ == '__main__':
app = QtGui.QApplication(sys.argv)
thisapp = App()
thisapp.show()
sys.exit(app.exec_())
Here's one way to do live plotting: get the plot as an image array then draw the image to a multithreaded screen.
Example using a pyformulas screen (~30 FPS):
import pyformulas as pf
import matplotlib.pyplot as plt
import numpy as np
import time
fig = plt.figure()
screen = pf.screen(title='Plot')
start = time.time()
for i in range(10000):
t = time.time() - start
x = np.linspace(t-3, t, 100)
y = np.sin(2*np.pi*x) + np.sin(3*np.pi*x)
plt.xlim(t-3,t)
plt.ylim(-3,3)
plt.plot(x, y, c='black')
# If we haven't already shown or saved the plot, then we need to draw the figure first...
fig.canvas.draw()
image = np.fromstring(fig.canvas.tostring_rgb(), dtype=np.uint8, sep='')
image = image.reshape(fig.canvas.get_width_height()[::-1] + (3,))
screen.update(image)
#screen.close()
Disclaimer: I'm the maintainer of pyformulas