I have a program which outputs to the terminal a number, one line at a time.
My goal is to have something else read these numbers and graph them in a line plot in real time. matplotlib and wxpython have been suggested, but I'm not sure how to go about implementing these.
See the following links:
What is the best real time plotting widget for wxPython?
Minimalistic Real-Time Plotting in Python
http://eli.thegreenplace.net/2008/08/01/matplotlib-with-wxpython-guis/
http://wxpython-users.1045709.n5.nabble.com/real-time-data-plots-td2344816.html
As some of those point out, you might be able to use wx's PyPlot for something really simple or use Chaco.
I really like this library for HTML5 graphing. Here is demo of real time updates: http://dygraphs.com/gallery/#g/dynamic-update
Are you simply asking for recommendations on plotting libs?
Related
As a physics major I am by no means good at coding but it suffices for modeling stuff. Plotting always ends up a beeing super annoying. It´s easy enough to look up stuff in the the Julia documentation if needed.
With plotting it´s a totally different story. There is just no good ressources to learn handling the available plotting packages.
I´m lost. There is proper documentation for PyPlot for example but only for Python and the code won´t work in Julia. Then there a few examples that obivously do not answer all my questions. Am I missing something? I feel stupid but also know that everyone around me spends to much time on plotting, too.
Any advice on were to look up stuff?
thanks in advance
Plotting in Julia is provided by packages. So you will not find plotting docs in the main documentation for julia.
There are a number of plotting packages to choose between - it's just a question of picking the one you prefer. Here is the documentation for most of the more used packages:
Plots: https://docs.juliaplots.org/latest/
Makie: https://makie.juliaplots.org/dev/
PlotlyJS: http://spencerlyon.com/PlotlyJS.jl/
VegaLite: https://www.queryverse.org/VegaLite.jl/stable/
PGFPlotsX: https://kristofferc.github.io/PGFPlotsX.jl/dev/
Gadfly: http://gadflyjl.org/stable/
GR: https://gr-framework.org/julia.html
UnicodePlots (for plotting in terminal): https://github.com/Evizero/UnicodePlots.jl
For PyPlot the matplotlib syntax should work. It's not clear from your question why it doesn't for you. So there should be ample resources.
I have some LVIS Lidar data in hdf5 format.
The data has Lat and Long co-ordinates, so I have been able to visualise them on a map using Basemap:
f = h5py.File('ILVIS1B_GA2016_0304_R1701_043591.h5','r')
LONG = f['/LON0/']
LAT = f['/LAT0/']
X = LONG[...]
Y = LAT[...]
m = Basemap(projection='merc',llcrnrlat=-0.5,urcrnrlat=0.5,\
llcrnrlon=9,urcrnrlon=10,lat_ts=0.25,resolution='i')
m.drawcoastlines()
m.drawcountries()
parallels = np.arange(-9.,10.,0.5)
m.drawparallels(parallels,labels=[False,True,True,False])
meridians = np.arange(-1.,1.,0.5)
m.drawmeridians(meridians,labels=[True,False,False,True])
m.drawmapboundary(fill_color='white')
x,y = m(X, Y)
scatter = plt.scatter(x,y)
m.scatter(x,y)
plt.show()
This gets me this, where the orange bands are very dense points:
The hdf5 file also has the full waveform data for each mapped point (each datapoint is a reflection detected at the sensor, as a function of time) so that each of the orange points has data associated with it like:
Ultimately, I would like to be able to click on any of the orange points and for the subsequent waveform to be displayed. I have looked into interactive plots for this and have come across a number of libraries (mpl3d, plotly etc).
I'm having some trouble getting my head around some of these and how I can get my data into the examples - my python isn't up to this level. Does anyone have any advice on where to start? Which libraries would be best suited to this? A little help to understand the basics would be appreciated.
Apologies there is no direct question here, I'm just after some info from the knowledgable community.
The question seems to be: How do I tackle a task I have no clue how to solve?
Step 1: Search for a possible solution. It may happen that someone else has already solved your problem. This will mostly not be the case, but you may be lucky.
Step 2: Abstract the task. What would be the general problem that a lot of people might have and for which there might be a solution? Does it need to be hdf5 files? No. Is georeferencing important? Maybe, but one could neglect for the moment. Which requirements are really important, which not?
Step 3: Search again. You will have more success now for finding similar or related problems.
Step 4: Look at the tools in use. Make a list of possible tools and check against your requirements. Interactivity, Application or web-based, accuracy etc.
Step 5: Decide for one tool and go for it. Start with a general case study. Can I plot a map on the left and a graph on the right side using this tool? If not, find out why - maybe there is a general problem with this, maybe there is just an implementation detail missing. At this point you may ask a question about the case study problem, specifying the tool in use and providing the code that gives the problem. Do not think about your actual problem until this is solved.
Step 6: Proceed and try to add interactivity. Can I get something to happen when clicking? Again treat this independent of the actual problem. Search for solutions and if there none, ask a question about it.
Step 7: Proceed further up to the point where you're truely stuck. Now is the time to finally ask a question here, but with all the details that have brought you down to step 7 inside the question.
I have recently begun learning to use Julia, converting over from Matlab/Octave. I decided that the best way to get some experience was to convert some code I was already working on i Octave - a Newton solver for a complicated multidimensional problem. I have been able to convert the code over successfully (and with noticeable speedup relative to Octave, without devectorisation or other performance-based changes), with only one issue arising.
I have chosen to use PyPlot for plotting, due to its similarity to Matlab/Octave's plotting functionality. However, there is some behaviour from PyPlot that is undesired. I use the plotting function to display the current state of the vector I am trying to get to zero (using the Newton solver part of the code), so that I can see what it is doing, and adjust the code to try to improve this behaviour. I input the number of Newton steps to take before the code stops, and then I can make adjustments or re-issue the command to continue attempting to converge.
I have the code set up to plot the current state every few steps, so that I can, for instance, have the code take 200 steps, but show me the status after every 10 steps. In Octave, this works perfectly, providing me with up-to-date information - should the behaviour of the code not be desirable, I can quickly cancel the code with Ctrl-C (this part works in Julia, too).
However, Julia does not produce or update the plots when the plot() command is used; instead, it produces the plot, or updates it if the plot window is already open, only when the code finishes. This entirely defeats the purpose of the intermittent plotting within the code. Once the code has completed, the plot is correctly generated, so I know that the plot() command itself is being used correctly.
I have tried adding either draw() or show() immediately after the plot command. I have also tried display(gcf()). None of these have modified the result. I have confirmed that isinteractive() outputs "true". I have also tried turning interactivity off (ioff()) and switching whether to use the python or julia backend (pygui(true) and pygui(false)), with no effect on this behaviour.
Have I missed something? Is there another package or option that needs to be set in order to force PyPlot to generate the current plot immediately, rather than waiting until Julia finishes its current code run to generate the plot?
Or is it perhaps possible that scope is causing a problem, here, as the intermittent plotting happens inside a while loop?
I am using xubuntu 12.10 with Julia 0.2.1.
PyPlot defaults to this behavior in the REPL. To make it show the plots as they are plotted type ion(). To turn it off again type ioff().
ion() is only effective for the current season so if you want it to stay on across sessions just add it to your .juliarc file.
If you're using iPython ion() will plot to a new window but ioff() will plot inline.
I have a seemingly simple problem, but an easy solution is alluding me. I have a very large series (tens or hundreds of thousands of points), and I just need to visualize it at different zoom levels, but generally zoomed well out. Basically, I want to plot it in a tool like Matlab or Pyplot, but knowing that each pixel can't represent the potentially many hundreds of points that map to it, I'd like to see both the min and the max of all the array entries that map to a pixel, so that I can generally understand what's going on. Is there a simple way of doing this?
Try hexbin. By setting the reduce_C_function I think you can get what you want. Ex:
import matplotlib.pyplot as plt
import numpy as np
plt.hexbin(x,y,C=C, reduce_C_function=np.max) # C = f(x,y)
would give you a hexagonal heatmap where the color in the pixel is the maximum value in the bin.
If you only want to bin in one direction, see this this method.
First option you may want to try is Gephi- https://gephi.org/
Here is another option, though I'm not quite sure it will work. It's hard to say without seeing the data.
Try going to this link- http://bl.ocks.org/3887118. Do you see toward the bottom of the page data.tsv with all of the values? IF you can save your data to resemble this then the HTML code above should be able to build your data in the scatter plot example shown in that link.
Otherwise, try visiting this link to fashion your data to a more appropriate web page.
There are a set of research tools called TimeSearcher 1--3 that provide some examples of how to deal with large time-series datasets. Below are some example images from TimeSearcher 2 and 3.
I realized that simple plot() in MATLAB actually gives me more or less what I want. When zoomed out, it renders all of the datapoints that map to a pixel column as vertical line segments from the minimum to the maximum within the set, so as not to obscure the function's actual behavior. I used area() to increase the contrast.
In case somebody doesn't know: A cartogram is a type of map where some country/region-dependent numeric property scales the respective regions so that that property's density is (close to) constant. An example is
from worldmapper.org. In this example, countries are scaled according to their population, resulting in near-constant population density.
Needless to say, this is really cool. Does anyone know of a Matplotlib-based library for drawing such maps? The method used at worldmapper.org is described in (1), so it would surprise me if no one has implemented this yet...
I'm also interested in hearing about other cartogram libraries, even if they're not made for Matplotlib.
(1) Michael T. Gastner and M. E. J. Newman,
Diffusion-based method for producing density-equalizing maps,
Proc. Nat. Acad. Sci. USA, 101, 7499-7504 (2004). Available at arXiv.
There's this, though it's based and a different algorithm (and though it's on the ESRI site, it doesn't require ArcGIS). Of course, once you have the cartogram you can plot it in matplotlib.
Here is a Javascript plugin to make cartograms using D3. It is a good, simple solution if you are not too concerned about the regions being sized accurately. If accuracy is important, there are other options available that give you more freedom to play with the algorithm's parameters to get to a more accurate result.
Here are two great standalone programs I know of:
Scapetoad
Carto3F
Scapetoad is very easy to use. Just give it a shapefile, tell it which attribute to use for the scaling, and set a few accuracy parameters. If there is any doubt, this post describes the process.
Carto3F is more complex and allows for greater accuracy, though it is a bit trickier to figure out - lots of parameter settings without much documentation explaining them.
There is also a QGIS cartogram plugin, written in Python. Though I have not been able to get it to work, so cannot comment on that one.
In short, no. But Newman has an excellent little implementation of his and Gastner's method on his website. Installing it is easy and it works from the command line. Here's an example of a workflow using this software that worked for me.
Compute a grid of density estimates over some region, e.g. in Python. Store it as a matrix of numbers.
Run the cart program with your density matrix as input from the command line or from as subprocess in Python.
The program returns a list of new coordinates for each grid point.
Pipe your shapefile points through the interp program and into a new shapefile to get the transformed map.
There are nice instructions on the main page.
The geoplot.cartogram function in
Geoplot: geospatial data visualization — geoplot 0.2.0
says it is a high-level Python geospatial plotting library, and an extension to cartopy and matplotlib.
Try this library if you are using geopandas, it is quick and doesnt require much customization. https://github.com/mthh/cartogram_geopandas