Looks like Fabric is recalculating or correcting the count for one of my events (iOS app) 6-12h later. The updated count is always lower, 20-30% down.
Mike from Fabric here. Yes, this can happen either for a count upward or downward around UTC midnight each day.
We have two layers of processing - speed and batch - that allow us to collect data in real-time, while at the same time correcting any errors that happen during the collection of the data. What you're seeing happen is the correction of the speed data layer by the batch layer.
As a result of having less time to calculate speed layer information we use probabilistic algorithms which let us calculate this information in real-time, but a cost of reduced accuracy. For most apps, the correction is extremely small, but there are outliers.
Related
Is there currently a way to incorporate traffic patterns into OptaPlanner with the package and delivery VRP problem?
Eg. Let's say I need to optimize 500 pickup and deliveries today and tomorrow amongst 30 vehicles where each pickup has a 1-4hr time window. I want to avoid busy areas of the city during rush hours when possible.
New pickups can also be added (or cancelled in the meantime).
I'm sure this is a common problem. Does a decent solution exist for this in OptaPlanner?
Thanks!
Users often do this, but there is no out-of-the-box example of it.
There are several ways to do it, but one way is to add a 3th dimension to the distanceMatrix, indicating the departureTime. Typically that use a granularity of 15 minutes, 30 minutes or 1 hour.
There are 2 scaling concerns here:
memory. 15 minutes means 24 * 4 = 96 per day. Given that with 2 dimensions, a 10k locations distanceMatrix uses almost 2 GB RAM, clearly memory can become a concern.
pre-calculation time. Calculation the distance matrix can be time consuming. "bulk algorithms" can help here. For example, graphhopper community doesn't support bulk distance calculations, but their enterprise version - as well OSRM (which is free) does. Getting a 3 dimensional matrix from the remote Google Maps API, or the remote enterprise Graphhopper API, can result in bandwidth concerns (see above, the distance matrix can become several GB in size, especially in non-binary formats such as JSON or CSV).
In any case, one that 3 dimensional matrix is there, it's just a matter of adjust OptaPlanner examples's ArrivalTimeUpdateListener to use getDistance(from, to, departureTime).
How to stabilize the RSSI (Received Signal Strength Indicator) of low energy Bluetooth beacons (BLE) for more accurate distance calculation?
We are trying to develop an indoor navigation system and came across this problem where the RSSI is fluctuating so much that, the distance estimation is nowhere near the correct value. We tried using an advance average calculator but to no use,
The device is constantly getting RSSI values, how to filter them, how to get the mean value, I am completely lost, please help.
Can anyone suggest any npm library or point in the right direction, I have been searching for many days but have not gotten anywhere.
FRONT END: ReactNative BACKEND: NODEJS
In addition to the answer of #davidgyoung, we would like to point out that any filtering method is a compromise between quality of noise level reduction and the time-lag introduced by this filtration (depending on the characteristic filtering time you use in your method). As was pointed by #davidgyoung, if you take characteristic filtering period T you will get an average time-lag of about T/2.
Thus, I think the best approach to solve your problem is not to try to find the best filtering method but to make changes on the transmitter’s end itself.
First you can increase the number of signals, transmitter per second (most of the modern beacon allow to do so by using manufacturer applications and API).
Secondly, you can increase beacon's power (which is also usually one of the beacon’s settings), which usually reduces signal-to-noise ratio.
Finally, you can compare beacons from different vendors. At Navigine company we experimented and tested lots of different beacons from multiple manufacturers, and it appears that signal-to-noise ratio can significantly vary among existing manufacturers. From our side, we recommend taking a look at kontakt.io beacons (https://kontakt.io/) as an one of the recognized leaders with 5+ years experience in the area.
It is unlikely that you will find a pre-built package that will do what you want as your needs are pretty specific. You will most likely have to wtite your own filtering code.
A key challenge is to decide the parameters of your filtering, as an indoor nav use case often is impacted by time lag. If you average RSSI over 30 seconds, for example, the output of your filter will effectively give you the RSSI of where a moving object was on average 15 seconds ago. This may be inappropriate for your use case if dealing with moving objects. Reducing the averaging interval to 5 seconds might help, but still introduces time lag while reducing smoothing of noise. A filter called an Auto-Regressive Moving Average Filter might be a good choice, but I only have an implementation in Java so you would need to translate to JavaScript.
Finally, do not expect a filter to solve all your problems. Even if you smooth out the noise on the RSSI you may find that the distance estimates are not accurate enough for your use case. Make sure you understand the limits of what is possible with this technology. I wrote a deep dive on this topic here.
I've been using oxyplot for a month now and I'm pretty happy with what it delivers. I'm getting data from an oscilloscope and, after a fast processing, I'm plotting it in real time to a graph.
However, if I compare my application CPU usage to the one provided by the oscilloscope manufacturer, I'm loading a lot more the CPU. Maybe they're using some gpu-based plotter, but I think I can reduce my CPU usage with some modifications.
I'm capturing 10.000 samples per second and adding it to a LineSeries. I'm not plotting all that data, I'm decimating it to a constant number of points, let's say 80 points for a 20 secs measure, so I have 4 points/sec while totally zoomed out and a bit more detail if I zoom in to a specific range.
With the aid of ReSharper, I've noticed that the application is calling a lot of times (I've 6 different plots) the IsValidPoint method (something like 400.000.000 times), which is taking a lot of time.
I think the problem is that, when I add new points to the series, it checks for every point if it is a valid point, instead of the added values only.
Also, it spends a lot of time in the MeasureText/DrawText method.
My question is: is there a way to override those methods and to adapt it to my needs? I'm adding 10.000 new values each second, but the first ones remain the same, so there's no need for re-validate them. Also, the text shown doesn't change.
Thank you in advance for any advice you can give me. Have a good day!
The question is simple and direct.Please help.
I am working on bus ETA projects where all the datum i get are:
1.Real time GPS locations from bus
2.Distance between each stop
Do we need more data than these?
My Big question is:
How to make use of these data to calculate the ETA for customer?
My thoughts:
ETA is all about distance/speed,
so at first: I tried to simply get distance from 2 GPS coordinates to calculate
speed and use stop's distance to calculate ETA.
i.e.
while(true){
ETA = stopDis/2ptSpeed;
stopDis = stopDis - 2ptDis;
}
and upDATE next stop when stopDis<0
However, the problem is that the GPS datum will jump quite wildly and hence the
speed calculated is really messy.
Broken downed Question: How to smooth the GPS datum? Kalman Filter? Averaging?
Have heard of those but does not really know what to do ESPECIALLY Kalman filter!
Because of traffic and not-straight lines, distances and speed alone are not reliable indicators.
I think that your best bet is to use the history of the line to compare the current distance with the average distance at that time in the line, and to deduce a deviation from the average ETA.
I don't think it's that simple. What if there are lots of traffic lights between 2 stops or different speed limits.
And you have the gps coordinates so you can calculate distance, but that isn't the distance over the road. So you need to have someway to get the real distance the bus needs to travel (Perhaps the google maps api can help).
So to answer the "Do we need more data than these?" question: yes, i think you do.
Just an idea: The bus schedule already contains info about how much time is needed between stops on average. If you can just read from the GPS position if the bus is on a stop or between two stops you should be able to make a fairly accurate prediction.
I'm guessing you are talking about busses in a city, where GPS signals are weak and bus stops are not far apart.
Edit: This could be further improved with data about current traffic.
Collect travel time data from stop to stop on a continuous basis. Derive an expected travel time between stops from this historical data to use as a baseline. this can be continuously updated. Initial ETA at a stop is time of last stop plus most recent travel time average. If you have continuous GPS data, you can adjust in real time from that.
I've build an embedded GPS tracker which logs coordinates to a database, and I'd like to analyse this data and extract travelled routes and idle moments from it.
The way I think it should work:
Order the coordinates by UTC timestamp, find coordinates close together, and mark them as a 'route start'. Proximity could be found by simply comparing decimals, as I think GPS can be considered linear at a scale of ~3m. This brings up the trouble of selecting the 'center' of the idling: due to GPS inaccuracy, these decimals may drift around a little, potentially over the threshold to be grouped with the first.
10:15 to 15:12 - Idle
15:12 to 16:38 - [coordinates]
16:38 to 17:43 - Idle
etc.
How can I group coordinates in this fashion, and what complexities am I overlooking?
The first thing I would do is work with some experimental data. Are you sure there isn't some distance D, where if someone moved more than D from their previous location it must be they are on the move?
Now, assuming that there isn't. Say that GPS precision is usually within a reasonable accuracy, say 10m, but sometimes you just get a point 100m away. What I would do is: as long as the user didn't move more than, say, 20m, calculate a centre for the "idle location" (just the average of the x and y values of all the coordinates). When the user moves beyond distance 20m from your centre, don't immediately assume they started moving. Instead, check the next (one or more) values; only if you see them several times in distance of more than 20m from the centre, infer they started moving.
I imagine this simple solution should be good enough for most applications, though it depends on what exactly the user is doing (how stable they are during idle time, how fast moving when they move), and the quality of the GPS data. I would start with something like that and see how it works. If it fails, I would analyse in what cases it fails and proceed from there.