I would like to pass some overpass request into ox.graph_from_place, but I don't really understand how it works with the doc.
I would like to create a graph with 2 types of roads (where the buses can pass and where the psv can pass too)
Am'i obliged to join my 2 graphs ? Or Is there a more direct method ?
G1 = ox.graph_from_place('Marseille, France', retain_all=True, custom_filter='["bus"="yes"]')
G2 = ox.graph_from_place('Marseille, France', retain_all=True, custom_filter='["psv"="yes"]')
Gtot = nx.disjoint_union(G1,G2)
Does someone know the answer?
Have a good day
As OSMnx does not include a customizable union operator for multiple keys in its querying apparatus, your best bet is indeed to make two queries then combine them. But you should use the compose function to do so:
import networkx as nx
import osmnx as ox
ox.config(use_cache=True, log_console=True)
place = 'Marseille, France'
G1 = ox.graph_from_place(place, network_type='drive', retain_all=True, custom_filter='["bus"="yes"]')
G2 = ox.graph_from_place(place, network_type='drive', retain_all=True, custom_filter='["psv"="yes"]')
G = nx.compose(G1, G2)
print(len(G1), len(G2), len(G)) #784 141 855
See also https://stackoverflow.com/a/62239377/7321942 and https://stackoverflow.com/a/62883614/7321942
Related
I have a Dask Dataframe with the following content:
X_trn y_trn
0 java repeat task every random seconds p m alre... LQ_CLOSE
1 are java optionals immutable p d like to under... HQ
2 text overlay image with darkened opacity react... HQ
3 ternary operator in swift is so picky p questi... HQ
4 hide show fab with scale animation p m using c... HQ
I am trying to use CountVectorizer from dask.ml's library. When I do pass my X_trn to fit_transform, I get the Value Error "Cannot infer dataframe metadata with a dask.delayed argument'".
vectorizer = CountVectorizer()
countMatrix = vectorizer.fit_transform(training['X_trn'])
This answer will probably come too late for the original author but may still help others. The answer is actually in the documentation I also overlooked it at first:
The Dask-ML implementation currently requires that raw_documents is a
dask.bag.Bag of documents (lists of strings).
This apparently innocense sentence is your problem. You are passing a dask.dataframe and not a dask.bag.Bag of documents
import dask.bag as db
corpus = db.from_sequence(training['X_trn'], npartitions=2)
And then, you can pass it to the vectorizer as you were doing:
X = vectorizer.fit_transform(corpus)
I'm trying to retrieve a network of the major roads in Norway. However, the "int_ref" and "ref" labels are inconsistent and are resulting in gaps in the road. When looking at the road in OpenStreetMap I can see that the 'relation' tag under 'Part of' is exactly what I need. Is there any way to retrieve this using OSMnx? Is there any other way to retrieve a full road? I'm using the following line of code when filtering one specific road based on int_ref:
G1 = ox.graph_from_place(query = "Norway", retain_all = True, custom_filter = '["int_ref"~"E 39"]')
No, OSMnx filters on way tags, not on relations. If you want to get only the major roads in a country, see this answer: https://stackoverflow.com/a/52412274/7321942
Something like this may do what you are looking for:
import osmnx as ox
ox.config(use_cache=True, log_console=True)
# get the geometry of the norwegian mainland
gdf = ox.geocode_to_gdf('Norway')
geom = max(gdf['geometry'].iloc[0], key=lambda x: x.area)
# get all motorway/trunk roads
cf = '["highway"~"motorway|motorway_link|trunk|trunk_link"]'
G = ox.graph_from_polygon(geom, network_type='drive', custom_filter=cf)
# plot it
fig, ax = ox.plot_graph(G)
It takes ~370 Overpass API requests to download all the area of the Norwegian mainland, so it takes a while to make all those requests. You can watch its progress in the log in the console.
I have a ndarray representing an image with different channels like this:
image = (8,100,100) where 8=channels, 100x100 the actual image per channel
I am interested in extracting the RGB components of that image:
imageRGB = np.take(image, [4,2,1], axis = 0)
in this way I have an array of (3,100,100) with the RGB components.
However, I need to visualize it so I need an array of (100,100,3), I think it's quite straightforward to do it but I all the methods I try do not work.
numpy einsum is a good tool to be used.
Official document: https://docs.scipy.org/doc/numpy/reference/generated/numpy.einsum.html
import numpy as np
imageRGB = np.random.randint(0,5,size=(3,100,101))
# set the last dim to 101 just to make stuff more clear
imageRGB.shape
# (3,100,101)
imageRGB_reshape = np.einsum('kij->ijk',imageRGB)
imageRGB_reshape.shape
# (100,101,3)
In my opinion it's the most clear way to write and read.
Wow thank you! I have never thought to use Einstein summation, actually it works very well.
Just for curiosity is it possible to build it manually?
For example:
R = image[4,:,:]
G = image[2,:,:]
B = image[1,:,:]
imageRGB = ???
I am creating a graph from an osm file using graph_from_file() (it contains both the roads and buildings) using osmnx and then plotting it. While doing this, it is also plotting buildings along with the roads. Is there a way to remove buildings from this graph/just ignore buildings while creating a graph from that osm file?
Set the parameter retain_all to False. This includes the buildings which are usually disconnected (in my experience). You can observe the differences between the following 2 figures.
G = ox.graph_from_file('try.xml', retain_all=True)
fig, ax = ox.plot_graph(G)
[![Figure_1][1]][1]
G = ox.graph_from_file('try.xml', retain_all=False)
fig, ax = ox.plot_graph(G)
[![Figure_2][1]][1]

Note that I only queried highway = pedestrian and building = university using Overpass Turbo (overpass_code) which produced my .xml file.
I am using NREL's DAKOTA_driver openmdao plugin for parallelized Monte Carlo sampling of a model. In 0.X, I was able to nest assemblies, allowing an outer optimization driver to direct the DAKOTA_driver sampling evaluations. Is it possible for me to nest this setup within an outer optimizer? I would like the outer optimizer's workflow to call the DAKOTA_driver "assembly" then the get_dakota_output component.
import pandas as pd
import subprocess
from subprocess import call
import os
import numpy as np
from dakota_driver.driver import pydakdriver
from openmdao.api import IndepVarComp, Component, Problem, Group
from mpi4py import MPI
import sys
from itertools import takewhile
sigm = .005
n_samps = 20
X_bar=[0.065 , sigm] #2.505463e+03*.05]
dacout = 'dak.sout'
class get_dak_output(Component):
mean_coe = 0
def execute(self):
comm = MPI.COMM_WORLD
rank = comm.Get_rank()
nam ='ape.net_aep'
csize = 10000
with open(dacout) as f:
for i,l in enumerate(f):
pass
numlines = i
dakchunks = pd.read_csv(dacout, skiprows=0, chunksize = csize, sep='there_are_no_seperators')
linespassed = 0
vals = []
for dchunk in dakchunks:
for line in dchunk.values:
linespassed += 1
if linespassed < 49 or linespassed > numlines - 50: continue
else:
split_line = ''.join(str(s) for s in line).split()
if len(split_line)==2:
if (len(split_line) != 2 or
split_line[0] in ('nan', '-nan') or
split_line[1] != nam):
continue
else:vals.append(float(split_line[0]))
self.coe_vals = sorted(vals)
self.mean_coe = np.mean(self.coe_vals)
class ape(Component):
def __init__(self):
super(ape, self).__init__()
self.add_param('x', val=0.0)
self.add_output('net_aep', val=0.0)
def solve_nonlinear(self, params, unknowns, resids):
print 'hello'
x = params['x']
comm = MPI.COMM_WORLD
rank = comm.Get_rank()
outp = subprocess.check_output("python test/exampleCall.py %f"%(float(x)),
shell=True)
unknowns['net_aep'] = float(outp.split()[-1])
top = Problem()
root = top.root = Group()
root.add('ape', ape())
root.add('p1', IndepVarComp('x', 13.0))
root.connect('p1.x', 'ape.x')
drives = pydakdriver(name = 'top.driver')
drives.UQ('sampling', use_seed=False)
#drives.UQ()
top.driver = drives
#top.driver = ScipyOptimizer()
#top.driver.options['optimizer'] = 'SLSQP'
top.driver.add_special_distribution('p1.x','normal', mean=0.065, std_dev=0.01, lower_bounds=-50, upper_bounds=50)
top.driver.samples = n_samps
top.driver.stdout = dacout
#top.driver.add_desvar('p2.y', lower=-50, upper=50)
#top.driver.add_objective('ape.f_xy')
top.driver.add_objective('ape.net_aep')
top.setup()
top.run()
bak = get_dak_output()
bak.execute()
print('\n')
print('E(aep) is %f'%bak.mean_coe)
There are two different options for this situation. Both will work in parallel, and both can be currently supported. But only one of them will work when you want to use analytic derivatives:
1) Nested Problems: You create one problem class that has a DOE driver in it. You pass the list of cases you want run into that driver, and it runs them in parallel. Then you put that problem into a parent problem as a component.
The parent problem doesn't know that it has a sub-problem. It just thinks it has a single component that uses multiple processors.
This is the most similar way to how you would have done it in 0.x. However I don't recommend going this route because it won't work if you want to use ever want to use analytic derivatives.
If you use this way, the dakota driver can stay pretty much as is. But you'll have to use a special sub-problem class. This isn't an officially supported feature yet, but its very doable.
2) Using a multi-point approach, you would create a Group class that represent your model. You would then create one instance of that group for each monte-carlo run you want to do. You put all of these instances into a parallel group inside your overall problem.
This approach avoids the sub-problem messiness. Its also much more efficient for actual execution. It will have a somewhat greater setup-cost than the first method. But in my opinion its well worth the one time setup cost to get the advantage of analytic derivatives. The only issue is that it would probably require some changes to the way the dakota_driver works. You would want to get a list of evaluations from the driver, then hand them them out to the individual children groups.