I'm writing a program that has, as one facet, a wave filtration/resolution routine. The more data I collect, the bigger the files stored to the device get. I'm collecting data at discrete time steps, and in the interest of accuracy I'm doing this pretty frequently. However, I noticed that the overall wave form tends to be wide enough that I could be collecting data at about half the rate I am and still be able to draw an accurate-enough-for-my-purposes waveform over the data.
So the question: is there a way to, from this data, create a continuous mathematic description of the curve? I haven't been able to find anything. My data is float inside of NSNumbers contained by an NSArray.
The two things I would like to be able to do are get intersections points for a threshold and find local maximums. The ability to do either one of these would be sufficient.
-EDIT-
If anyone knows a good objective-c FFT method for 1-dimensional real arrays I would love to hear it.
Apple includes an FFT in the Accelerate framework.
Using Fourier Transforms
Example: FFT Sample
Also: Using the Apple FFT and Accelerate Framework
Related
Background
I'm working on a project where a user gets scanned by a Kinect (v2). The result will be a generated 3D model which is suitable for use in games.
The scanning aspect is going quite well, and I've generated some good user models.
Example:
Note: This is just an early test model. It still needs to be cleaned up, and the stance needs to change to properly read skeletal data.
Problem
The problem I'm currently facing is that I'm unsure how to place skeletal data inside the generated 3D model. I can't seem to find a program that will let me insert the skeleton in the 3D model programmatically. I'd like to do this either via a program that I can control programmatically, or adjust the 3D model file in such a way that skeletal data gets included within the file.
What have I tried
I've been looking around for similar questions on Google and StackOverflow, but they usually refer to either motion capture or skeletal animation. I know Maya has the option to insert skeletons in 3D models, but as far as I could find that is always done by hand. Maybe there is a more technical term for the problem I'm trying to solve, but I don't know it.
I do have a train of thought on how to achieve the skeleton insertion. I imagine it to go like this:
Scan the user and generate a 3D model with Kinect;
1.2. Clean user model, getting rid of any deformations or unnecessary information. Close holes that are left in the clean up process.
Scan user skeletal data using the Kinect.
2.2. Extract the skeleton data.
2.3. Get joint locations and store as xyz-coordinates for 3D space. Store bone length and directions.
Read 3D skeleton data in a program that can create skeletons.
Save the new model with inserted skeleton.
Question
Can anyone recommend (I know, this is perhaps "opinion based") a program to read the skeletal data and insert it in to a 3D model? Is it possible to utilize Maya for this purpose?
Thanks in advance.
Note: I opted to post the question here and not on Graphics Design Stack Exchange (or other Stack Exchange sites) because I feel it's more coding related, and perhaps more useful for people who will search here in the future. Apologies if it's posted on the wrong site.
A tricky part of your question is what you mean by "inserting the skeleton". Typically bone data is very separate from your geometry, and stored in different places in your scene graph (with the bone data being hierarchical in nature).
There are file formats you can export to where you might establish some association between your geometry and skeleton, but that's very format-specific as to how you associate the two together (ex: FBX vs. Collada).
Probably the closest thing to "inserting" or, more appropriately, "attaching" a skeleton to a mesh is skinning. There you compute weight assignments, basically determining how much each bone influences a given vertex in your mesh.
This is a tough part to get right (both programmatically and artistically), and depending on your quality needs, is often a semi-automatic solution at best for the highest quality needs (commercial games, films, etc.) with artists laboring over tweaking the resulting weight assignments and/or skeleton.
There are algorithms that get pretty sophisticated in determining these weight assignments ranging from simple heuristics like just assigning weights based on nearest line distance (very crude, and will often fall apart near tricky areas like the pelvis or shoulder) or ones that actually consider the mesh as a solid volume (using voxels or tetrahedral representations) to try to assign weights. Example: http://blog.wolfire.com/2009/11/volumetric-heat-diffusion-skinning/
However, you might be able to get decent results using an algorithm like delta mush which allows you to get a bit sloppy with weight assignments but still get reasonably smooth deformations.
Now if you want to do this externally, pretty much any 3D animation software will do, including free ones like Blender. However, skinning and character animation in general is something that tends to take quite a bit of artistic skill and a lot of patience, so it's worth noting that it's not quite as easy as it might seem to make characters leap and dance and crouch and run and still look good even when you have a skeleton in advance. That weight association from skeleton to geometry is the toughest part. It's often the result of many hours of artists laboring over the deformations to get them to look right in a wide range of poses.
What file formats and software could I use to represent vector images over time as an animation, without compromising the advantages of the vector format?
Say I generate data that is best represented as a single point in the plane, moving over time. I would like to make an animation showing the motion of this point. One way to do this is to make a sequence of 2D bitmap images and string these together into an AVI file. But this produces either huge files (orders of magnitude larger than the underlying dataset) or very low quality animations. A stack of raster images is a very inefficient representation of the data.
A much better representation would be a sequence of 2D vector images. Vector images combine very high fidelity with small file size. But is it possible to string such images into an animation? What kind of software could be used to do so, starting from the underlying dataset?
I imagine a tool such as Adobe Flash could be used here, but this seems akin to making scatterplots from scratch in Illustrator: sure, it can be done and will look nice, but this is not how you make scatterplots. You use R, Excel or MATLAB, and then perhaps retouch the plot in a graphics program. I'm looking for a similarly efficient solution, but for making dynamic visualizations rather than plots.
I have a sequence of gps values each containing: timestamp, latitude, longitude, n_sats, gps_speed, gps_direction, ... (some subset of NMEA data). I'm not sure of what quality the direction and speed values are. Further, I cannot expect the sequence to be evenly spaced w.r.t. the timestamp. I want to get a smooth trajectory at an even time step.
I've read the Kalman Filter is the tool of choice for such tasks. Is this indeed the case?
I've found some implementations of the Kalman Filter for Python:
http://www.scipy.org/Cookbook/KalmanFiltering
http://ascratchpad.blogspot.de/2010/03/kalman-filter-in-python.html
These however appear to assume regularly spaced data, i.e. iterations.
What would it take to integrate support of irregularly spaced observations?
One thing I could imagine is to repeat/adapt the prediction step to a time-based model. Can you recommend such a model for this application? Would it need to take into account the NMEA speed values?
Having looked all over for an understandable resource on Kalman filters, I'd highly recommend this one: https://github.com/rlabbe/Kalman-and-Bayesian-Filters-in-Python
To your particular question regarding irregularly spaced observations: Look at Chapter 8 in the reference above, and under the heading "Nonstationary Processes". To summarize, you'll need to use a different state transition function and process noise covariance for each iteration. Those are the only things you'll need to change at each iteration, since they're the only components dependent on delta t.
You could also try kinematic interpolation to see if the results fit to what you expect.
Here's a Python implementation of one of these algorithms: https://gist.github.com/talespaiva/128980e3608f9bc5083b
I have been reading up on FFT and Pitch Detection for a while now, but I'm having trouble piecing it all together.
I have worked out that the Accelerate framework is probably the best way to go with this, and I have read the example code from apple to see how to use it for FFTs. What is the input data for the FFT if I wanted to be running the pitch detection in real time? Do I just pass in the audio stream from the microphone? How would I do this?
Also, after I get the FFT output, how can I get the frequency from that? I have been reading everywhere, and can't find any examples or explanations of this?
Thanks for any help.
Frequency and pitch are not the same thing - frequency is a physical quantity, pitch is a psychological percept - they are similar, but there are important differences, which may or may not matter to you, depending on the type of instrument for which you are trying to measure pitch.
You need to read up a little on the various pitch detection algorithms (and on the meaning of pitch itself), decide what algorithm you want to use and only then set about implementing it. See this Wikipedia page for a good overview of pitch and pitch detection (note that you can use FFT for the autocorrelation-based and frequency domain methods).
As for using the FFT to identify peaks in a spectrum and their associated frequencies, there are many questions and answers related to this on SO already, see for example: How do I obtain the frequencies of each value in an FFT?
I have an example implementation of an Autocorrelation function available online for ios 5.1. Look at this post for a link to the implementation AND functions on how to find the nearest note and how to create a string representing the pitch (A, A#, B, B#, etc...)
While the FFT is very useful in many applications, it might not be the most accurate if you're trying to do simple pitch detection. (It can be as accurate, but you have to deal with complex numbers to do a lot of phase calculations)
I want to get the pitch of a song at any point. I plan on storing the pitches later. How can I read say... an mp3 file or wav file to get the pitch played at a certain point?
Here is a visual example:
Say I wanted to get the pitch that is here at ^this point of the song.
Thanks if you can!
The matter is a tad more complicated than you may be anticipating.
While time-domain approaches exist (that is, approaches which work with the PCM data directly), frequency-domain pitch detection is going to be more accurate. You can read a very simplified overview here.
What you probably want is a Fourier Transform, which can be used to transform blocks of your signal from time-domain to frequency-domain (that is, a distribution of frequency content over a given span of the signal). From there, you would need to analyze the frequency spectrum within that block. The problem becomes even harder still, because there is no best way to deduce pitch from a sampled frequency spectrum in the general case. The aforementioned Wikipedia article should give you a foundation for looking into those algorithms.
Finally, it's worth noting that this is really a language-agnostic question, unless your primary interest is in reading a WAV file specifically using VB.NET.