I have a sequence of gps values each containing: timestamp, latitude, longitude, n_sats, gps_speed, gps_direction, ... (some subset of NMEA data). I'm not sure of what quality the direction and speed values are. Further, I cannot expect the sequence to be evenly spaced w.r.t. the timestamp. I want to get a smooth trajectory at an even time step.
I've read the Kalman Filter is the tool of choice for such tasks. Is this indeed the case?
I've found some implementations of the Kalman Filter for Python:
http://www.scipy.org/Cookbook/KalmanFiltering
http://ascratchpad.blogspot.de/2010/03/kalman-filter-in-python.html
These however appear to assume regularly spaced data, i.e. iterations.
What would it take to integrate support of irregularly spaced observations?
One thing I could imagine is to repeat/adapt the prediction step to a time-based model. Can you recommend such a model for this application? Would it need to take into account the NMEA speed values?
Having looked all over for an understandable resource on Kalman filters, I'd highly recommend this one: https://github.com/rlabbe/Kalman-and-Bayesian-Filters-in-Python
To your particular question regarding irregularly spaced observations: Look at Chapter 8 in the reference above, and under the heading "Nonstationary Processes". To summarize, you'll need to use a different state transition function and process noise covariance for each iteration. Those are the only things you'll need to change at each iteration, since they're the only components dependent on delta t.
You could also try kinematic interpolation to see if the results fit to what you expect.
Here's a Python implementation of one of these algorithms: https://gist.github.com/talespaiva/128980e3608f9bc5083b
Related
I would like to increase the density of my AIS or GPS data in order to carry out more precise analyses afterwards. During my research I came across different approaches like interpolation, filtering or imputation. With the first two approaches, there is no doubt that these can be used to approximate the points between two collected data points.
In the case of imputation (e.g. MICE), however, I have not yet found an approach in the literature for determining position data.
That's why I wanted to ask if anyone knew a paper dealing with this subject and whether it makes sense at all to determine further position data approximately by imputation.
The problem you are describing there is trajectory reconstruction for AIS/GPS data. There's a number of papers for general trajectory reconstruction (see this for example), but AIS data are quite specific.
The irregularity of AIS data is a well know problem with no standard approach to deal with, as far as I know.
However, there is a handful of publications which try to deal with this issue. The problem of reconstruction is connected to the trajectory prediction problem, since both of these two shares some of the methods (the latter is more popular in the scientific community, I think).
Traditionally, AIS trajectory reconstruction is done using some physical models, which take into account the curvature of the earth and other factors, such as data noise (see examples here, here, and here).
More recent approach tries to use LSTM neural networks.
I don't know much about GPS data, but I think the methods are very similar to the ones mentioned above (especially taking into account the fact that you probably want to deal with maritime data).
I want to identify when data values ordered as a sine waveform.
For example in the picture
Until now I have worked with features like STD, RMS etc, on the data to identify waveforms. Now I am looking for a feature which will have such functionality although I am opened for any ideas to achieve such identification.
You could, for example, compute the Fourier transform and see if it has a noticeable peak at one of the frequencies.
I'm trying to figure out how to correct drift errors introduced by a SLAM method using GPS measurements, I have two point sets in euclidian 3d space taken at fixed moments in time:
The red dataset is introduced by GPS and contains no drift errors, while blue dataset is based on SLAM algorithm, it drifts over time.
The idea is that SLAM is accurate on short distances but eventually drifts, while GPS is accurate on long distances and inaccurate on short ones. So I would like to figure out how to fuse SLAM data with GPS in such way that will take best accuracy of both measurements. At least how to approach this problem?
Since your GPS looks like it is very locally biased, I'm assuming it is low-cost and doesn't use any correction techniques, e.g. that it is not differential. As you probably are aware, GPS errors are not Gaussian. The guys in this paper show that a good way to model GPS noise is as v+eps where v is a locally constant "bias" vector (it is usually constant for a few metters, and then changes more or less smoothly or abruptly) and eps is Gaussian noise.
Given this information, one option would be to use Kalman-based fusion, e.g. you add the GPS noise and bias to the state vector, and define your transition equations appropriately and proceed as you would with an ordinary EKF. Note that if we ignore the prediction step of the Kalman, this is roughly equivalent to minimizing an error function of the form
measurement_constraints + some_weight * GPS_constraints
and that gives you a more straigh-forward, second option. For example, if your SLAM is visual, you can just use the sum of squared reprojection errors (i.e. the bundle adjustment error) as the measurment constraints, and define your GPS constraints as ||x- x_{gps}|| where the x are 2d or 3d GPS positions (you might want to ignore the altitude with low-cost GPS).
If your SLAM is visual and feature-point based (you didn't really say what type of SLAM you were using so I assume the most widespread type), then fusion with any of the methods above can lead to "inlier loss". You make a sudden, violent correction, and augment the reprojection errors. This means that you lose inliers in SLAM's tracking. So you have to re-triangulate points, and so on. Plus, note that even though the paper I linked to above presents a model of the GPS errors, it is not a very accurate model, and assuming that the distribution of GPS errors is unimodal (necessary for the EKF) seems a bit adventurous to me.
So, I think a good option is to use barrier-term optimization. Basically, the idea is this: since you don't really know how to model GPS errors, assume that you have more confidance in SLAM locally, and minimize a function S(x) that captures the quality of your SLAM reconstruction. Note x_opt the minimizer of S. Then, fuse with GPS data as long as it does not deteriorate S(x_opt) more than a given threshold. Mathematically, you'd want to minimize
some_coef/(thresh - S(X)) + ||x-x_{gps}||
and you'd initialize the minimization with x_opt. A good choice for S is the bundle adjustment error, since by not degrading it, you prevent inlier loss. There are other choices of S in the litterature, but they are usually meant to reduce computational time and add little in terms of accuracy.
This, unlike the EKF, does not have a nice probabilistic interpretation, but produces very nice results in practice (I have used it for fusion with other things than GPS too, and it works well). You can for example see this excellent paper that explains how to implement this thoroughly, how to set the threshold, etc.
Hope this helps. Please don't hesitate to tell me if you find inaccuracies/errors in my answer.
I have historical aircraft trajectory data with points varying from 1 second - 1 minute separation. Often these points present sharp turns. I'm looking for suggestions of best methods of resampling the data to generate smooth paths (e.g. point every n seconds) that more realistically represent the path followed. It would be useful to be able to parameterize the function with certain performance characteristics (e.g. rate of change of direction).
I'm aware of algorithms like the Kalman filter, Bezier curve fitting, splines etc. for data smoothing. But what algorithms would you suggest exploring as a starting point for generating smooth turns?
Schneider's Algorithm is an algorithm that approximately fits curves through a series of points.
The resulting curves have a drastically reduced point-count and it's error-tolerance is configurable, so you can adjust it as much as you need to.
In general:
Lower error-tolerance: More points, more accurate, less execution
Higher error-tolerance: Less points, less accurate, faster execution
Some useful links:
A live Javascript example, and it's implementation here.
Python Example
C++ implementation
If the resulting curve must pass exactly through your points, you need an interpolation algorithm instead of an approximation algorithm, but keep in mind that those do not reduce point-count.
A really good type of interpolating spline is the Centripetal Catmull-Rom Spline.
I'm writing a program that has, as one facet, a wave filtration/resolution routine. The more data I collect, the bigger the files stored to the device get. I'm collecting data at discrete time steps, and in the interest of accuracy I'm doing this pretty frequently. However, I noticed that the overall wave form tends to be wide enough that I could be collecting data at about half the rate I am and still be able to draw an accurate-enough-for-my-purposes waveform over the data.
So the question: is there a way to, from this data, create a continuous mathematic description of the curve? I haven't been able to find anything. My data is float inside of NSNumbers contained by an NSArray.
The two things I would like to be able to do are get intersections points for a threshold and find local maximums. The ability to do either one of these would be sufficient.
-EDIT-
If anyone knows a good objective-c FFT method for 1-dimensional real arrays I would love to hear it.
Apple includes an FFT in the Accelerate framework.
Using Fourier Transforms
Example: FFT Sample
Also: Using the Apple FFT and Accelerate Framework