Querying a feature layer with a large amount of points - arcgis-js-api

Working in ArcGIS JS API 4.22:
I am trying to query a feature layer (polygons) with a large amount of points (~1000) in order to find which polygons the points intersect with (by name only, no need for geometries. Doing this with a loop for each point individually is extremely slow and errors out. I've tried condensing the points to a multipoint geometry in hopes of speeding the process to no avail. Is there a solution to this that I am just overlooking?
Added code:
const layer = new FeatureLayer({
url: "https://arcgis-server.lsa.umich.edu/arcgis/rest/services/IFR/glahf_classification_poly/MapServer/0",
outFields: ["AEU_Code"],
});
//gets a server error at about 250 points
var multipoint = new Multipoint({
points: [
[-86, 45],
[-85, 47],
[-84, 49]
]
});
const query = new Query();
query.geometry = multipoint;
query.outSpatialReference = { wkid: 102100 };
query.returnGeometry = false;
query.outFields = ["AEU_Code"];
layer.queryFeatures(query).then(function (results) {
console.log(results.features); // prints the array of features to the console
});
Thanks!

So you have two separate layers in this situation right? A polygon layer and a points layer? Use the pairwise intersect tool, it should result in an feature class where the attribute table has all of your points along with the fields from the polygon layer they intersect with. 1000 points really isn't that many, this should not be very computationally intense.
If this is not what you're doing can you be more descriptive with the data you have and what you are trying to accomplish?
The other way I interpret the question is that you already have the feature layer I described, but are trying to query it in some way. If this is the case can you elaborate on what you are trying to glean from the data? I don't understand why you would need to query each point individually.

I check the polygon service and it has 77 features. If you can load all the polygons in the view, then one possible option is to do the intersection in the client instead of the server. In this way at least you avoid timeout issues, you will have to time it to see how long it takes.
In order to to the intersection in the client use the layer view instead of the layer, ArcGIS JS API - FeatureLayerView queryFeatures.

Related

AsterixDB error when joining geometry: Cannot invoke \"org.apache.hyracks.control.nc.io.FileHandle.close()\" because \"fHandle\" is null"

I am attempting to use AsterixDB (which uses SQL++) to join two sets together via an sql query. In one dataset, I have a series of points in the form of latitude and longitude. The other dataset is geometries for zip codes. I am trying to append the relevant zip code to the first dataset based on whether the point exists in the zip code or not.
The query is below as well as the schema for each dataset
use csv;
select sett.lat, sett.long, zip.g
from csv_set as sett
left join csv_zipset as zip
on st_contains(zip.g, st_make_point(sett.lat, sett.long));
create type csv_type as {
id:uuid,
...
lat: double,
long: double
};
create type csv_ziptype as {
id: uuid,
g:geometry
};
This is the error I am facing:
ERROR: Code: 1 "java.lang.NullPointerException: Cannot invoke \"org.apache.hyracks.control.nc.io.FileHandle.close()\" because \"fHandle\" is null"
I have tried adding null checks for both the point and geometry with no luck.
I have also validated that st_make_point is working properly, and st_contains works when I pass it a fixed geometry which leads me to believe that this is an issue with the geometry.
Any help is much appreciated
After exhausting more options I came to the realization that there were multiple types in my dataset: polygon, multipolygon, linestring, and geometryCollection. It seems that AsterixDB doesn't yet have the ability to compute st_contains with geometry collections. Once I removed those entries from the dataset the query was able to complete successfully.

Create subgraph query in Gremlin around single node with outgoing and incoming edges

I have a large Janusgraph database and I'd to create a subgraph centered around one node type and including incoming and outgoing nodes of specific types.
In Cypher, the query would look like this:
MATCH (a:Journal)N-[:PublishedIn]-(b:Paper{paperTitle:'My Paper Title'})<-[:AuthorOf]-(c:Author)
RETURN a,b,c
This is what I tried in Gremlin:
sg = g.V().outE('PublishedIn').subgraph('j_p_a').has('Paper','paperTitle', 'My Paper Title')
.inE('AuthorOf').subgraph('j_p_a')
.cap('j_p_a').next()
But I get a syntax error. 'AuthorOf' and 'PublishedIn' are not the only edge types ending at 'Paper' nodes.
Can someone show me how to correctly execute this query in Gremlin?
As written in your query, the outE step yields edges and the has step will check properties on those edges, following that the query processor will expect an inV not another inE. Without your data model it is hard to know exactly what you need, however, looking at the Cypher I think this is what you want.
sg = g.V().outE('PublishedIn').
subgraph('j_p_a').
inV().
has('Paper','paperTitle', 'My Paper Title').
inE('AuthorOf').
subgraph('j_p_a')
cap('j_p_a').
next()
Edited to add:
As I do not have your data I used my air-routes graph. I modeled this query on yours and used some select steps to limit the data size processed. This seems to work in my testing. Hopefully you can see the changes I made and try those in your query.
sg = g.V().outE('route').as('a').
inV().
has('code','AUS').as('b').
select('a').
subgraph('sg').
select('b').
inE('contains').
subgraph('sg').
cap('sg').
next()

How do you do pagination in GUN?

How do you do something like gun.get({startkey, endkey}) ?
Previously: https://github.com/amark/gun/issues/479
#qwe123wsx #sebastianmacias apologies for the delay! Originally posted at: https://github.com/amark/gun/issues/479
The wire spec has a protocol for this but it isn't implemented yet. It looks something like this:
gun.on('out', {get: {'#': {'>': 'a', '<': 'b'}}});
However this doesn't work yet. I would recommend instead:
(1) Pagination behavior is very different from one app to another and will be hard for us to create a "one-size-fits-all" solution, so it would be highly helpful if you could implement your own* pagination and make it available as a user-module, then we can learn from your experience (what worked, what didn't) and make the best solution part of core.
(2) Your app will probably work fine without pagination in the meanwhile, while it can be built (it is targeted for after 1.0), and then as your app becomes more popular, it should be fairly easy to add in without much refactor, once you need it and it is available.
... * How to build your own?
Lots of good articles on this, best one I've seen yet is from Neo4j on how to do it in a graph database (which applies to gun as well) https://graphaware.com/neo4j/2014/08/20/graphaware-neo4j-timetree.html .
Another rough idea is you model your data based on pagination or time. So rather than having ALL tweets go into user's tweet table, instead, the user's tweet table is a table of DAYS (or weeks), and then you put the tweet inside the week table. Now when you load the data, you can scan/skip based off of week very easily while it being super bandwidth efficient.
Rough PSEUDO code:
function onTweetSend(tweet){
gun.get('user').get('alice').get('tweets').get(Date.uniqueYear() + Date.uniqueWeek()).set(tweet)
}
function paginateUserTweet(howMany, cb){
var range = convertToArrayOfUniqueWeekNamesFromToday(howMany);
var all = [];
range.forEach(function(week){
gun.get('user').get('alice').get('tweets').get(week).load(function(tweets){
all.push(tweets);
if(all.length < range.length){ return }
all = flattenArray(all);
cb(all);
});
});
}
Now we can use https://gun.eco/docs/RAD#lex
gun.get(...).get({'.': {'>': startkey, '<': endkey}, '%': 50000}).map().once(...)

Service Area Polygon Geometry

I need to get a service area polygon (graphics) for inserting it into a query as geometry.
This is a piece of code (a serviceAreaTask)
serviceAreaTask.solve(params,function(solveResult){
var polygonSymbol = new SimpleFillSymbol("solid",
new SimpleLineSymbol("solid", new Color([232,104,80]), 2),
new Color([232,104,80,0.25])
);
arrayUtils.forEach(solveResult.serviceAreaPolygons,function(serviceArea){
serviceArea.setSymbol(polygonSymbol);
map.graphics.add(serviceArea);
});
According to API ServiceAreaSolveResult https://developers.arcgis.com/javascript/3/jsapi/serviceareasolveresult-amd.html
ServiceAreaPolygon is already a graphic, and I can use its geometry in my query, but I donĀ“t know how I can get this geometry.
Thanks a lot!
you are correct! As mentined in document serviceAreaPolygons are already in esri graphic format.
Well, geometry is a property of the graphic. below is the way to access this.
In your case-
arrayUtils.forEach(solveResult.serviceAreaPolygons,function(serviceArea){
serviceArea.setSymbol(polygonSymbol);
var serviceAreaGeometry = serviceArea.geometry; // this is the geometry. you can use this geometry in your further query.
map.graphics.add(serviceArea);
});
Hoping this will help you.
Feel free to shoot your further queries.

D3 Graph Example Using In Memory Object

This seems like it should be simple, but I have spent literally hours without any success.
Take the D3 graph example at http://bl.ocks.org/mbostock/950642. The example uses a local file called graph.json. I have set up a Rails app to serve a similar graph, however I don't want to write a file of the JSON. Rather, I generate the nodes and links into an object such as:
{"nodes":[{"node_type":"Person","name":"Damien","id":"damien_person"}, {"node_type":"Person","name":"Grant","id":"grant_person"}}],
"links":[{"source":"damien_person","target":"grant_person","label":"Friends"}}
Now when I render the D3, I need to update the call d3.json("graph.json", function(json) {...}); to reference my in-memory object rather than the local file (or url). However, everything I've tried breaks my html/javascript. For example I tried setting the var dataset = <%= raw(#myInMemoryObject) %>;, and that works for assignment (I did an alert on the dataset), however I can't get the D3 code to use it.
How can I replace the d3.json call in order to use my in-memory object?
Thank you,
Damien
Your idea of using, for example, var dataset = <%= raw(#myInMemoryObject) %>; is the right way to go but you need to prep your object to be in the right format.
The nodes specified in the links need to either be numeric references to nodes in the nodes array eg. 0 for first, 1 for second
var json ={
"nodes":[{"name":"Damien","id":"a"}, {"name":"Bob","id":"b"}],
"links":[{"source":0, "target":1,"value":1}]
}
or links to the actual objects which make the nodes themselves:
var a = {"name":"Damien","id":"a"};
var b = {"name":"Bob","id":"b"}
var json ={
"nodes":[a,b],
"links":[{"source":a,"target":b,"value":1}]
};
Relevant discussion is here: https://groups.google.com/forum/?fromgroups=#!topic/d3-js/LWuhBeEipz4
Example here: http://jsfiddle.net/5A9eV/1/