I am stuck trying to create an attribute rule/field calculation using Arcade where upstream areas are added to a downstream area called "Cumulative Area"
NodeFrom
NodeTo
Area
CumulativeArea
1
2
10
10
2
3
5
15
4
3
6
6
4
3
6
6
3
5
8
20
Essentially I am thinking of the SQL equivalent of a SUM WHERE NodeTo = NodeFrom. Or an excel equivalent of [Area + SUMIF(CumulativeArea, NodeFrom, NodeTo)]
This is for QA'ing sewer calculations so please let me know if there are any tools out there that may already have this kind of functionality.
Thanks!
I think you're just tired and need to re-evalute the tools you're already using. Other tools won't be node specific. In faster settings the area is more difficult to process at the speed you require. Try disassembly of your current tool for full copyright. You're on the right track.
Related
I want to simulate a loop in Hive. The problem is that of link traversal.
Raw Data:
phone,name
1234,Sam
2345,John
3456,Max
7899,Sam
7899,Tim
8899,Tim
8898,Dan
8899,John
I am only interested in getting phone numbers.
If I take 1234,Sam and go to 7899,Sam that would be 1-hop because Sam used 7899 number too. Then if I go from 7899,Sam to 8899,Tim that would be 1-hop because 7899 was used by Tim and Tim also used 8899. Now where we started from, 1234, to where we are now, 8899, we are 2-hops away. I can do this with a shell script with loops but I am interested to find if this is possible to do purely in Hive. I want the loop to stop if any one criteria is met:
10 hops reached OR no new numbers detected
(If the conditions above are hard to implement, I can just have one condition - 10hops reached)
Visual representation:
1 hop:
1234 -> Sam
Sam linked to 7899
2 hops:
7899 -> Tim
Tim linked to 8899
so 1234 and 8899 are 2 hops away.
Any tips are welcome!
My friends,
In the past couple of years I read a lot about AI with JS and some libraries like TensorFlow. I have great interest in the subject but never used it on a serious project. However, after struggling a lot with linear regression to solve an optimization problem I have, I think that finally I will get much better results, with greater performance, using AI. I work for 12 years with web development and lots of server side, but never worked with any AI library, so please, have a little patience with me if I say something stupid!
My problem is this: every user that visits our platform (website) we save the Hour, Day of the week, if the device requesting the page was a smartphone or computer... and such of the FIRST access the user made. If the user keeps visiting other pages, we dont care, we only save the data of the FIRST visit. And if the user anytime does something that we consider a conversion, we assign that conversion to the record of the first access that user made. So we have almost 3 millions of lines like this:
SESSION HOUR DAY_WEEK DEVICE CONVERSION
9847 7 MONDAY SMARTPHONE NO
2233 13 TUESDAY COMPUTER YES
5543 19 SUNDAY COMPUTER YES
3721 8 FRIDAY SMARTPHONE NO
1849 12 SUNDAY COMPUTER NO
6382 0 MONDAY SMARTPHONE YES
What I would like to do is this: next time a user visits our platform, we wanna know the probability of that user making a conversion. If a user access now, our website, depending on their device, day of week, hour... we wanna know the probability of that user making a future conversion. With that, we can show very specific messages to the user while he is using our platform and a different price model according to that probability.
CURRENTLY we are using a liner regression, and it predicts if the user will make a conversion with an accuracy of around 30%. It's pretty low but so far, it's the best we got it, and this linear regression generates almost 18% increase in conversions when we use it to show specific messages/prices to that specific user compaired to when we dont use it. SO, with a 30% accuracy our linear regression already provides 18% better conversions (and with that, higher revenues and so on).
In case you are curious, our linear regression model works like this: we generate a linear equation to every first user access on our system with variables that our system tries to find in order to minimize error sqr(expected value - value). Using the data above, our model would generate these equations below (SUNDAY = 0, MONDAY = 1...COMPUTER = 0, SMARTPHONE = 1... CONVERSION YES = 1 and NO = 0)
A*7 + B*1 + C*1 = 0
A*13 + B*2 + C*0 = 1
A*19 + B*0 + C*0 = 1
A*8 + B*6 + C*1 = 0
A*12 + B*0 + C*1 = 0
A*0 + B*1 + C*0 = 1
So, our system find the best A, B and C that generates the minimizes error. How can we do that with AI? If possible, it would be nice if we could use TensorFlow or anything with JS! I know there are several AI models, and I have no idea which one would best fit what we need!
The project I am working on involves static offline GTFS data in a mobile app. All the GTFS data is available inside realm-objects (or SQLite if needed).
Now, I would like to establish all train- or bus-connections from A to B (starting after a certain departure-time).
How do I query the GTFS-data in order to get a connection from A to B ???
I reealized to get all trips leaving from A.
I realized to get all station-names along that trip including times.
But I find it very hard to get the connection information between two locations A and B. What SQL queries do i have to set up in order to get that information ?
Any help appreciated !
If you just want to dynamically calculate shortest travel routes between static hubs, in an offline application, determined like the following image, you can use the following formula:
(Source)
Here's the pseudo code:
1 function Dijkstra(Graph, source):
2
3 create vertex set Q
4
5 for each vertex v in Graph: // Initialization
6 dist[v] ← INFINITY // Unknown distance from source to v
7 prev[v] ← UNDEFINED // Previous node in optimal path from source
8 add v to Q // All nodes initially in Q (unvisited nodes)
9
10 dist[source] ← 0 // Distance from source to source
11
12 while Q is not empty:
13 u ← vertex in Q with min dist[u] // Node with the least distance
14 // will be selected first
15 remove u from Q
16
17 for each neighbor v of u: // where v is still in Q.
18 alt ← dist[u] + length(u, v)
19 if alt < dist[v]: // A shorter path to v has been found
20 dist[v] ← alt
21 prev[v] ← u
22
23 return dist[], prev[]
(Source)
Alright, I'll admit so far my answer was a tad facetious...
I suspect you don't realize just how complicated it is to do what you're asking, especially in an offline environment. Companies like Google and Microsoft have spent millions on research with huge teams of data scientists.
If this is something you are serious about, I'd encourage you to start with a 10×10 grid and work on the logic of getting from "Point A → Point B" when you start adding barriers in random places (this simulating roads beginning & ending). Recently I was surprised how complicated a seemingly-simple, somewhat-related Stack Overflow "pipe sizes conversion" question that I answered had become.
If you didn't have the "offline" condition, I would've suggested looking into getting a [free] API Key for Google Web Services Directions API. Basically you tell it a where Points A & B are, and it gives you detailed route information, via transit or other methods.
Another of Google's many API's that could be helpful is the Snap to Roads API, which turns partial or error-ridden paths into driveable ones.
The study Route Logic and Shortest Path Logic is actually really fascinating stuff, and there are some amazing resources to learn about the related theories (see below), including a video from Google explaining how they went about it.
...but unfortunately this isn't something that's going to be accomplished with a simple SQL Query.
Actual Resources:
YouTube : Google Tech Talk: Fast Route Planning
Wikipedia : Shortest path problem
Wikipedia : Dijkstra's algorithm
...and some slightly Lighter Reading:
Wikipedia : Travelling Salesman Problem
Why UPS drivers don’t turn left and you probably shouldn’t either
Google Web Services : Directions API Developer's Guide
Stack Overflow : Shortest Route of converters between two different pipe sizes?
Wikipedia : Seven Bridges of Königsberg
My team have settled on MSpec for our BDD testing framework, which from their usage so far looks really good - but I'm struggling with the documentation/google for finding any implementation similar to SpecFlow's 'Scenario Outline'. I've shown an example of this below, but basically it allows you to write one 'test' and run it multiple times from a table (example) of inputs/expected outputs. I'll be embarrassed if answer turns out to be a LMGTFY but I've not been able to find anything myself. I don't want to say to the team it's not possible if I've just not found how to do it in MSpec (or understood MSpec properly). I wonder if this is why in some of the pro's/con's for MSpec I see references to the number of classes you can end up with listed as a negative.
Example of SpecFlow Scenario Outline
Scenario Outline: Successfully Convert Seconds to Minutes Table
When I navigate to Seconds to Minutes Page
And type seconds for <seconds>
Then assert that <minutes> minutes are displayed as answer
Examples:
| seconds | minutes |
| 1 day, 1 hour, 1 second | 1500 |
| 5 days, 3 minutes | 7203 |
| 4 hours | 240 |
| 180 seconds | 3 |
From: https://gist.github.com/angelovstanton/615da65a8f821d7a43c92ef9e2fd0b01#file-energyandpowerconvertcalculator-feature
Short answer, this is current not supported by by mspec. We planned this several years back, but the contribution never made it back into master.
If you want scenario outlines either use a different framework or create parameterized static methods in a helper class and call these from your context classes. Which will leave you with 1 class per scenario.
Is it possible to handle relations between jobs (in this case services) in jsprit?
For instance one job has to start at the same time with one or more other jobs.
Or one job must not start before the end of another job (normal sequence).
If not, do you know other java libraries that can handle such kind of restrictions?
Thank you!
Yes you can handle such relations with jsprit.
It is a bit of coding, but - I think - easy to implement. To illustrate it, I prepared you five examples that are based on the Vehicle Routing Problems (VRP) developed by Christofides-Mingozzi-Toth's* (first benchmarking instance). Follow the links behind the individual headlines and you get to the code. Note that the first activity in a route is marked with a triangle. The labels denote jobIds.
noConstraints
capacityConstraints
job 13 and 21 in same route
job 13 and 21 in same route AND 21 before 13
job 13 and 21 in same route AND 13 right after 21
Visit http://jsprit.github.io/ and you get to a number of code examples and docs.
*N. Christofides, A. Mingozzi, and P. Toth. The vehicle routing problem. In N. Christofides, A. Mingozzi, P. Toth, and C. Sandi, editors, Combinatorial Optimization. John Wiley, Chichester, 1979.