acutality of building tags fetched by osmnx - osmnx
Can one say something to the actuality of buildings and its tags?
E.g. I fetch https://www.openstreetmap.org/way/557584181 via OSMNX and it says building=construction but on OSM it says building=office. I see it was edited about 1 month ago.
Unfortunately I am using OMSNX v. 0.9 due to I am working with other packages that limit me to pandas=0.23.
Any ideas?
EDIT:
I guess it was cached somewhere.
import osmnx as ox
from shapely import geometry
type(given_geo_area)
shapely.geometry.polygon.Polygon
print(given_geo_area)
POLYGON ((9.444757321553571 47.6640820372137, 9.44469147049122 47.6642370363376, 9.442746638836271 47.6675223477341, 9.44273323863089 47.6675486765942, 9.44272335456083 47.6675846924473, 9.44272337322715 47.6675856076278, 9.44272365679015 47.667586502615, 9.442724194246059 47.6675873426787, 9.442724964738851 47.6675880952201, 9.442725938369239 47.6675887310364, 9.442771524985689 47.6676085195463, 9.442809078141879 47.6676199425426, 9.443599054706221 47.6678292167847, 9.44364298889086 47.6678391758214, 9.44369319196211 47.6678454078942, 9.443694609235431 47.6678454056071, 9.44369600193242 47.6678452280573, 9.443697323077419 47.6678448812336, 9.443698528108349 47.6678443768344, 9.443744598701199 47.6678160313241, 9.443795360184319 47.6677784822092, 9.443991191770809 47.6676599114601, 9.44421793991903 47.667570002951, 9.44446657647225 47.667512336406, 9.44472720190474 47.6674892078198, 9.444989439424219 47.6675015380556, 9.445052589744799 47.6675088406224, 9.445117064489731 47.6675174206873, 9.445183601261711 47.6675274468952, 9.44523747329856 47.6675300193568, 9.445238809309229 47.6675299032106, 9.4452400852424 47.6675296113077, 9.445241251079061 47.667529155091, 9.445242261116221 47.6675285524453, 9.445243075758549 47.6675278269953, 9.44524366307046 47.6675270071801, 9.445257172339209 47.6674916444378, 9.44526138400154 47.6674643786881, 9.44526635876986 47.6674027181877, 9.44527842375639 47.6673216144847, 9.44528702520314 47.6672820624034, 9.445343690252869 47.6671237851091, 9.44544200701166 47.6669751340207, 9.445578690330031 47.6668410756375, 9.445606929899901 47.666818179473, 9.445637520256669 47.6667867600046, 9.445694232828229 47.6667392820943, 9.445718033000571 47.6667110812337, 9.445816550854619 47.6665782840333, 9.445885789521761 47.6664961432372, 9.44592142419992 47.6664586450851, 9.44609642847751 47.6663115101204, 9.446313359526091 47.6661926345661, 9.44656247099808 47.6661073591621, 9.4468325709589 47.6660595150674, 9.447111524661491 47.6660512517601, 9.44717730910928 47.6660540322315, 9.447316189895529 47.6660649297759, 9.44753610831893 47.6660902456686, 9.44758112940486 47.6660938580278, 9.44767071891042 47.6660918313842, 9.447684852061499 47.6660900296329, 9.44774691292624 47.6660795923001, 9.4479881800759 47.6660547725018, 9.448232085468209 47.6660604809193, 9.448470258112661 47.6660965216365, 9.448560914386359 47.6661163562617, 9.44857404167368 47.6661182063231, 9.448691583060739 47.6661430580836, 9.44873711713312 47.6661476818212, 9.44936429835351 47.6661892570172, 9.44940607834077 47.66619064719, 9.44946122599818 47.6661870380657, 9.449463401923399 47.6661858503929, 9.449498075077059 47.6661578721339, 9.449520006942709 47.6661337837734, 9.45042894326825 47.6650175089795, 9.45044714891934 47.6649921250378, 9.450463499380801 47.6649559172176, 9.450463614247271 47.6649550512483, 9.450463463599689 47.664954187754, 9.450463053866351 47.6649533635807, 9.45046240253088 47.6649526138965, 9.450461537386341 47.6649519706911, 9.450460495349169 47.6649514614106, 9.45040935260457 47.6649360626301, 9.45036907619812 47.664928709451, 9.447377834868041 47.6644933952575, 9.447343269935271 47.6644880343118, 9.44730847614896 47.6644823034133, 9.44727413690042 47.6644763151236, 9.444928887305879 47.6640444576946, 9.444886220360861 47.6640381906818, 9.444829205115131 47.6640359557828, 9.4448279422305 47.6640360849357, 9.44482674602855 47.6640363876611, 9.44482566804831 47.6640368509158, 9.444757321553571 47.6640820372137))
buildings = ox.footprints_from_polygon(given_geo_area)
buildings.loc[buildings.index == 557584181]['building']
557584181 construction
Name: building, dtype: object
Edit2:
Now I tried to provide a reproducible code like:
import osmnx as ox
from shapely import geometry
area = [
geometry.Point(9.444757321553571, 47.6640820372137),
geometry.Point(9.44469147049122, 47.6642370363376),
geometry.Point(9.442746638836271, 47.6675223477341),
geometry.Point(9.44273323863089, 47.6675486765942),
geometry.Point(9.44272335456083, 47.6675846924473),
geometry.Point(9.44272337322715, 47.6675856076278),
geometry.Point(9.44272365679015, 47.667586502615),
geometry.Point(9.442724194246059, 47.6675873426787),
geometry.Point(9.442724964738851, 47.6675880952201),
geometry.Point(9.442725938369239, 47.6675887310364),
geometry.Point(9.442771524985689, 47.6676085195463),
geometry.Point(9.442809078141879, 47.6676199425426),
geometry.Point(9.443599054706221, 47.6678292167847),
geometry.Point(9.44364298889086, 47.6678391758214),
geometry.Point(9.44369319196211, 47.6678454078942),
geometry.Point(9.443694609235431, 47.6678454056071),
geometry.Point(9.44369600193242, 47.6678452280573),
geometry.Point(9.443697323077419, 47.6678448812336),
geometry.Point(9.443698528108349, 47.6678443768344),
geometry.Point(9.443744598701199, 47.6678160313241),
geometry.Point(9.443795360184319, 47.6677784822092),
geometry.Point(9.443991191770809, 47.6676599114601),
geometry.Point(9.44421793991903, 47.667570002951),
geometry.Point(9.44446657647225, 47.667512336406),
geometry.Point(9.44472720190474, 47.6674892078198),
geometry.Point(9.444989439424219, 47.6675015380556),
geometry.Point(9.445052589744799, 47.6675088406224),
geometry.Point(9.445117064489731, 47.6675174206873),
geometry.Point(9.445183601261711, 47.6675274468952),
geometry.Point(9.44523747329856, 47.6675300193568),
geometry.Point(9.445238809309229, 47.6675299032106),
geometry.Point(9.4452400852424, 47.6675296113077),
geometry.Point(9.445241251079061, 47.667529155091),
geometry.Point(9.445242261116221, 47.6675285524453),
geometry.Point(9.445243075758549, 47.6675278269953),
geometry.Point(9.44524366307046, 47.6675270071801),
geometry.Point(9.445257172339209, 47.6674916444378),
geometry.Point(9.44526138400154, 47.6674643786881),
geometry.Point(9.44526635876986, 47.6674027181877),
geometry.Point(9.44527842375639, 47.6673216144847),
geometry.Point(9.44528702520314, 47.6672820624034),
geometry.Point(9.445343690252869, 47.6671237851091),
geometry.Point(9.44544200701166, 47.6669751340207),
geometry.Point(9.445578690330031, 47.6668410756375),
geometry.Point(9.445606929899901, 47.666818179473),
geometry.Point(9.445637520256669, 47.6667867600046),
geometry.Point(9.445694232828229, 47.6667392820943),
geometry.Point(9.445718033000571, 47.6667110812337),
geometry.Point(9.445816550854619, 47.6665782840333),
geometry.Point(9.445885789521761, 47.6664961432372),
geometry.Point(9.44592142419992, 47.6664586450851),
geometry.Point(9.44609642847751, 47.6663115101204),
geometry.Point(9.446313359526091, 47.6661926345661),
geometry.Point(9.44656247099808, 47.6661073591621),
geometry.Point(9.4468325709589, 47.6660595150674),
geometry.Point(9.447111524661491, 47.6660512517601),
geometry.Point(9.44717730910928, 47.6660540322315),
geometry.Point(9.447316189895529, 47.6660649297759),
geometry.Point(9.44753610831893, 47.6660902456686),
geometry.Point(9.44758112940486, 47.6660938580278),
geometry.Point(9.44767071891042, 47.6660918313842),
geometry.Point(9.447684852061499, 47.6660900296329),
geometry.Point(9.44774691292624, 47.6660795923001),
geometry.Point(9.4479881800759, 47.6660547725018),
geometry.Point(9.448232085468209, 47.6660604809193),
geometry.Point(9.448470258112661, 47.6660965216365),
geometry.Point(9.448560914386359, 47.6661163562617),
geometry.Point(9.44857404167368, 47.6661182063231),
geometry.Point(9.448691583060739, 47.6661430580836),
geometry.Point(9.44873711713312, 47.6661476818212),
geometry.Point(9.44936429835351, 47.6661892570172),
geometry.Point(9.44940607834077, 47.66619064719),
geometry.Point(9.44946122599818, 47.6661870380657),
geometry.Point(9.449463401923399, 47.6661858503929),
geometry.Point(9.449498075077059, 47.6661578721339),
geometry.Point(9.449520006942709, 47.6661337837734),
geometry.Point(9.45042894326825, 47.6650175089795),
geometry.Point(9.45044714891934, 47.6649921250378),
geometry.Point(9.450463499380801, 47.6649559172176),
geometry.Point(9.450463614247271, 47.6649550512483),
geometry.Point(9.450463463599689, 47.664954187754),
geometry.Point(9.450463053866351, 47.6649533635807),
geometry.Point(9.45046240253088, 47.6649526138965),
geometry.Point(9.450461537386341, 47.6649519706911),
geometry.Point(9.450460495349169, 47.6649514614106),
geometry.Point(9.45040935260457, 47.6649360626301),
geometry.Point(9.45036907619812, 47.664928709451),
geometry.Point(9.447377834868041, 47.6644933952575),
geometry.Point(9.447343269935271, 47.6644880343118),
geometry.Point(9.44730847614896, 47.6644823034133),
geometry.Point(9.44727413690042, 47.6644763151236),
geometry.Point(9.444928887305879, 47.6640444576946),
geometry.Point(9.444886220360861, 47.6640381906818),
geometry.Point(9.444829205115131, 47.6640359557828),
geometry.Point(9.4448279422305, 47.6640360849357),
geometry.Point(9.44482674602855, 47.6640363876611),
geometry.Point(9.44482566804831, 47.6640368509158),
geometry.Point(9.444757321553571, 47.6640820372137)
]
area_p = geometry.Polygon(area)
buildings = ox.footprints_from_polygon(area_p)
buildings.loc[buildings.index == 557584181]['building']
557584181 office
Name: building, dtype: object
print(area_p)
POLYGON ((9.444757321553571 47.6640820372137, 9.44469147049122 47.6642370363376, 9.442746638836271 47.6675223477341, 9.44273323863089 47.6675486765942, 9.44272335456083 47.6675846924473, 9.44272337322715 47.6675856076278, 9.44272365679015 47.667586502615, 9.442724194246059 47.6675873426787, 9.442724964738851 47.6675880952201, 9.442725938369239 47.6675887310364, 9.442771524985689 47.6676085195463, 9.442809078141879 47.6676199425426, 9.443599054706221 47.6678292167847, 9.44364298889086 47.6678391758214, 9.44369319196211 47.6678454078942, 9.443694609235431 47.6678454056071, 9.44369600193242 47.6678452280573, 9.443697323077419 47.6678448812336, 9.443698528108349 47.6678443768344, 9.443744598701199 47.6678160313241, 9.443795360184319 47.6677784822092, 9.443991191770809 47.6676599114601, 9.44421793991903 47.667570002951, 9.44446657647225 47.667512336406, 9.44472720190474 47.6674892078198, 9.444989439424219 47.6675015380556, 9.445052589744799 47.6675088406224, 9.445117064489731 47.6675174206873, 9.445183601261711 47.6675274468952, 9.44523747329856 47.6675300193568, 9.445238809309229 47.6675299032106, 9.4452400852424 47.6675296113077, 9.445241251079061 47.667529155091, 9.445242261116221 47.6675285524453, 9.445243075758549 47.6675278269953, 9.44524366307046 47.6675270071801, 9.445257172339209 47.6674916444378, 9.44526138400154 47.6674643786881, 9.44526635876986 47.6674027181877, 9.44527842375639 47.6673216144847, 9.44528702520314 47.6672820624034, 9.445343690252869 47.6671237851091, 9.44544200701166 47.6669751340207, 9.445578690330031 47.6668410756375, 9.445606929899901 47.666818179473, 9.445637520256669 47.6667867600046, 9.445694232828229 47.6667392820943, 9.445718033000571 47.6667110812337, 9.445816550854619 47.6665782840333, 9.445885789521761 47.6664961432372, 9.44592142419992 47.6664586450851, 9.44609642847751 47.6663115101204, 9.446313359526091 47.6661926345661, 9.44656247099808 47.6661073591621, 9.4468325709589 47.6660595150674, 9.447111524661491 47.6660512517601, 9.44717730910928 47.6660540322315, 9.447316189895529 47.6660649297759, 9.44753610831893 47.6660902456686, 9.44758112940486 47.6660938580278, 9.44767071891042 47.6660918313842, 9.447684852061499 47.6660900296329, 9.44774691292624 47.6660795923001, 9.4479881800759 47.6660547725018, 9.448232085468209 47.6660604809193, 9.448470258112661 47.6660965216365, 9.448560914386359 47.6661163562617, 9.44857404167368 47.6661182063231, 9.448691583060739 47.6661430580836, 9.44873711713312 47.6661476818212, 9.44936429835351 47.6661892570172, 9.44940607834077 47.66619064719, 9.44946122599818 47.6661870380657, 9.449463401923399 47.6661858503929, 9.449498075077059 47.6661578721339, 9.449520006942709 47.6661337837734, 9.45042894326825 47.6650175089795, 9.45044714891934 47.6649921250378, 9.450463499380801 47.6649559172176, 9.450463614247271 47.6649550512483, 9.450463463599689 47.664954187754, 9.450463053866351 47.6649533635807, 9.45046240253088 47.6649526138965, 9.450461537386341 47.6649519706911, 9.450460495349169 47.6649514614106, 9.45040935260457 47.6649360626301, 9.45036907619812 47.664928709451, 9.447377834868041 47.6644933952575, 9.447343269935271 47.6644880343118, 9.44730847614896 47.6644823034133, 9.44727413690042 47.6644763151236, 9.444928887305879 47.6640444576946, 9.444886220360861 47.6640381906818, 9.444829205115131 47.6640359557828, 9.4448279422305 47.6640360849357, 9.44482674602855 47.6640363876611, 9.44482566804831 47.6640368509158, 9.444757321553571 47.6640820372137))
If you are using cached response data, they will reflect whatever was on OSM at the time that you cached them. You can turn caching off with the config function. Here's a minimal reproducible code example.
import osmnx as ox
ox.config(use_cache=False)
point = (47.6654265, 9.4464618)
gdf = ox.geometries_from_point(point, tags={'building':True}, dist=500)
gdf.shape #(137, 38)
gdf.loc[gdf['osmid']==557584181, 'building'] #office
Everything works fine. Note that v0.9 is several years old at this point and this code example uses v1.0.
Related
get the index of string element in np1 while it has a substring in np2
my code is as below:
import numpy as np
keywordlist = ['cpp-4.8.5', 'CUnit-2.1.3', 'CUnit-devel', 'doxygen-1.8.5', 'e2fsprogs-1.42.9', 'e2fsprogs-libs', 'epel-release', 'fuse3-devel', 'fuse3-libs', 'gcc-4.8.5', 'gcc-c++', 'gcc-gfortran', 'ghc-array', 'ghc-base', 'ghc-bytestring', 'ghc-containers', 'ghc-deepseq', 'ghc-directory', 'ghc-filepath', 'ghc-json', 'ghc-mtl', 'ghc-old', 'ghc-parsec', 'ghc-pretty', 'ghc-regex', 'ghc-regex', 'ghc-ShellCheck', 'ghc-syb', 'ghc-text', 'ghc-time', 'ghc-transformers', 'ghc-unix', 'git-1.8.3.1', 'graphviz-2.30.1', 'help2man-1.41.1', 'ibacm-22.4', 'keyutils-libs', 'krb5-devel', 'krb5-libs', 'krb5-workstation', 'lcov-1.13', 'libaio-devel', 'libblkid-2.23.2', 'libcom_err-1.42.9', 'libcom_err-devel', 'libgcc-4.8.5', 'libgfortran-4.8.5', 'libgomp-4.8.5', 'libibumad-22.4', 'libibverbs-22.4', 'libiscsi-devel', 'libkadm5-1.15.1', 'libmount-2.23.2', 'libpmem-1.5.1', 'libpmemblk-1.5.1', 'libpmemblk-devel', 'libpmem-devel', 'libquadmath-4.8.5', 'libquadmath-devel', 'librdmacm-22.4', 'libselinux-2.5', 'libselinux-devel', 'libselinux-python', 'libselinux-utils', 'libsepol-devel', 'libsmartcols-2.23.2', 'libss-1.42.9', 'libstdc++-4.8.5', 'libstdc++-devel', 'libunwind-1.2', 'libunwind-devel', 'libuuid-2.23.2', 'libuuid-devel', 'libverto-devel', 'libXaw-1.0.13', 'libXScrnSaver-1.2.2', 'make-3.82', 'nasm-2.10.07', 'numactl-devel', 'numactl-libs', 'openssl-1.0.2k', 'openssl-devel', 'openssl-libs', 'pcre-devel', 'perl-Digest', 'perl-Digest', 'perl-GD', 'perl-Git', 'python-2.7.5', 'python2-pycodestyle', 'python-libs', 'rdma-core', 'rdma-core', 'sg3_utils-1.37', 'sg3_utils-libs', 'ShellCheck-0.3.8', 'util-linux', 'zlib-devel']
np1 = np.array(keywordlist)
# ['cpp-4.8.5' 'CUnit-2.1.3' 'CUnit-devel' 'doxygen-1.8.5' ... 'ShellCheck-0.3.8' 'util-linux' 'zlib-devel']
result = ['epel-release-7-12.noarch', 'rdma-core-22.4-5.el7.x86_64', 'cpp-4.8.5-44.el7.x86_64', 'doxygen-1.8.5-4.el7.x86_64', 'ghc-base-4.6.0.1-26.4.el7.x86_64', 'libuuid-2.23.2-65.el7.x86_64', 'python-libs-2.7.5-89.el7.x86_64', 'libkadm5-1.15.1-50.el7.x86_64', 'libmount-2.23.2-65.el7.x86_64', 'libquadmath-4.8.5-44.el7.x86_64', 'util-linux-2.23.2-65.el7.x86_64', 'libss-1.42.9-19.el7.x86_64', 'keyutils-libs-1.5.8-3.el7.x86_64', 'e2fsprogs-libs-1.42.9-19.el7.x86_64', 'ghc-pretty-1.1.1.0-26.4.el7.x86_64', 'libXaw-1.0.13-4.el7.x86_64', 'libselinux-2.5-15.el7.x86_64', 'libibverbs-22.4-5.el7.x86_64', 'libselinux-utils-2.5-15.el7.x86_64', 'libgomp-4.8.5-44.el7.x86_64', 'libblkid-2.23.2-65.el7.x86_64', 'gcc-c++-4.8.5-44.el7.x86_64', 'e2fsprogs-1.42.9-19.el7.x86_64', 'CUnit-devel-2.1.3-8.el7.x86_64', 'make-3.82-24.el7.x86_64', 'numactl-libs-2.0.12-5.el7.x86_64', 'perl-Git-1.8.3.1-23.el7_8.noarch', 'openssl-libs-1.0.2k-19.el7.x86_64', 'gcc-4.8.5-44.el7.x86_64', 'CUnit-2.1.3-8.el7.x86_64', 'ghc-syb-0.4.0-35.el7.x86_64', 'gcc-gfortran-4.8.5-44.el7.x86_64', 'libselinux-python-2.5-15.el7.x86_64', 'sg3_utils-libs-1.37-19.el7.x86_64', 'fuse3-libs-3.6.1-4.el7.x86_64', 'libquadmath-devel-4.8.5-44.el7.x86_64', 'libgfortran-4.8.5-44.el7.x86_64', 'krb5-workstation-1.15.1-50.el7.x86_64', 'librdmacm-22.4-5.el7.x86_64', 'sg3_utils-1.37-19.el7.x86_64', 'libsmartcols-2.23.2-65.el7.x86_64', 'fuse3-devel-3.6.1-4.el7.x86_64', 'python-2.7.5-89.el7.x86_64', 'openssl-1.0.2k-19.el7.x86_64', 'libgcc-4.8.5-44.el7.x86_64', 'libaio-devel-0.3.109-13.el7.x86_64', 'ghc-old-locale-1.0.0.5-26.4.el7.x86_64', 'libcom_err-1.42.9-19.el7.x86_64', 'git-1.8.3.1-23.el7_8.x86_64', 'krb5-libs-1.15.1-50.el7.x86_64']
np2 = np.array(result)
# ['epel-release-7-12.noarch' 'rdma-core-22.4-5.el7.x86_64' ... 'krb5-libs-1.15.1-50.el7.x86_64']
expectation = ['cpp-4.8.5-39.el7.x86_64', 'CUnit-2.1.3-8.el7.x86_64', 'CUnit-devel-2.1.3-8.el7.x86_64', 'doxygen-1.8.5-4.el7.x86_64', 'e2fsprogs-1.42.9-17.el7.x86_64', 'e2fsprogs-libs-1.42.9-17.el7.x86_64', 'epel-release-latest-7.noarch', 'fuse3-devel-3.6.1-4.el7.x86_64', 'fuse3-libs-3.6.1-4.el7.x86_64', 'gcc-4.8.5-39.el7.x86_64', 'gcc-c++-4.8.5-39.el7.x86_64', 'gcc-gfortran-4.8.5-39.el7.x86_64', 'ghc-array-0.4.0.1-26.4.el7.x86_64', 'ghc-base-4.6.0.1-26.4.el7.x86_64', 'ghc-bytestring-0.10.0.2-26.4.el7.x86_64', 'ghc-containers-0.5.0.0-26.4.el7.x86_64', 'ghc-deepseq-1.3.0.1-26.4.el7.x86_64', 'ghc-directory-1.2.0.1-26.4.el7.x86_64', 'ghc-filepath-1.3.0.1-26.4.el7.x86_64', 'ghc-json-0.7-4.el7.x86_64', 'ghc-mtl-2.1.2-27.el7.x86_64', 'ghc-old-locale-1.0.0.5-26.4.el7.x86_64', 'ghc-parsec-3.1.3-31.el7.x86_64', 'ghc-pretty-1.1.1.0-26.4.el7.x86_64', 'ghc-regex-base-0.93.2-29.el7.x86_64', 'ghc-regex-tdfa-1.1.8-11.el7.x86_64', 'ghc-ShellCheck-0.3.8-1.el7.x86_64', 'ghc-syb-0.4.0-35.el7.x86_64', 'ghc-text-0.11.3.1-2.el7.x86_64', 'ghc-time-1.4.0.1-26.4.el7.x86_64', 'ghc-transformers-0.3.0.0-34.el7.x86_64', 'ghc-unix-2.6.0.1-26.4.el7.x86_64', 'git-1.8.3.1-23.el7_8.x86_64', 'graphviz-2.30.1-21.el7.x86_64', 'help2man-1.41.1-3.el7.noarch', 'ibacm-22.4-2.el7_8.x86_64', 'keyutils-libs-devel-1.5.8-3.el7.x86_64', 'krb5-devel-1.15.1-46.el7.x86_64', 'krb5-libs-1.15.1-46.el7.x86_64', 'krb5-workstation-1.15.1-46.el7.x86_64', 'lcov-1.13-1.el7.noarch', 'libaio-devel-0.3.109-13.el7.x86_64', 'libblkid-2.23.2-63.el7.x86_64', 'libcom_err-1.42.9-17.el7.x86_64', 'libcom_err-devel-1.42.9-17.el7.x86_64', 'libgcc-4.8.5-39.el7.x86_64', 'libgfortran-4.8.5-39.el7.x86_64', 'libgomp-4.8.5-39.el7.x86_64', 'libibumad-22.4-2.el7_8.x86_64', 'libibverbs-22.4-2.el7_8.x86_64', 'libiscsi-devel-1.9.0-7.el7.x86_64', 'libkadm5-1.15.1-46.el7.x86_64', 'libmount-2.23.2-63.el7.x86_64', 'libpmem-1.5.1-2.1.el7.x86_64', 'libpmemblk-1.5.1-2.1.el7.x86_64', 'libpmemblk-devel-1.5.1-2.1.el7.x86_64', 'libpmem-devel-1.5.1-2.1.el7.x86_64', 'libquadmath-4.8.5-39.el7.x86_64', 'libquadmath-devel-4.8.5-39.el7.x86_64', 'librdmacm-22.4-2.el7_8.x86_64', 'libselinux-2.5-15.el7.x86_64', 'libselinux-devel-2.5-15.el7.x86_64', 'libselinux-python-2.5-15.el7.x86_64', 'libselinux-utils-2.5-15.el7.x86_64', 'libsepol-devel-2.5-10.el7.x86_64', 'libsmartcols-2.23.2-63.el7.x86_64', 'libss-1.42.9-17.el7.x86_64', 'libstdc++-4.8.5-39.el7.x86_64', 'libstdc++-devel-4.8.5-39.el7.x86_64', 'libunwind-1.2-2.el7.x86_64', 'libunwind-devel-1.2-2.el7.x86_64', 'libuuid-2.23.2-63.el7.x86_64', 'libuuid-devel-2.23.2-63.el7.x86_64', 'libverto-devel-0.2.5-4.el7.x86_64', 'libXaw-1.0.13-4.el7.x86_64', 'libXScrnSaver-1.2.2-6.1.el7.x86_64', 'make-3.82-24.el7.x86_64', 'nasm-2.10.07-7.el7.x86_64', 'numactl-devel-2.0.12-5.el7.x86_64', 'numactl-libs-2.0.12-5.el7.x86_64', 'openssl-1.0.2k-19.el7.x86_64', 'openssl-devel-1.0.2k-19.el7.x86_64', 'openssl-libs-1.0.2k-19.el7.x86_64', 'pcre-devel-8.32-17.el7.x86_64', 'perl-Digest-1.17-245.el7.noarch', 'perl-Digest-MD5-2.52-3.el7.x86_64', 'perl-GD-2.49-3.el7.x86_64', 'perl-Git-1.8.3.1-23.el7_8.noarch', 'python-2.7.5-88.el7.x86_64', 'python2-pycodestyle-2.5.0-1.el7.noarch', 'python-libs-2.7.5-88.el7.x86_64', 'rdma-core-22.4-2.el7_8.x86_64', 'rdma-core-devel-22.4-2.el7_8.x86_64', 'sg3_utils-1.37-19.el7.x86_64', 'sg3_utils-libs-1.37-19.el7.x86_64', 'ShellCheck-0.3.8-1.el7.x86_64', 'util-linux-2.23.2-63.el7.x86_64', 'zlib-devel-1.2.7-18.el7.x86_64']
np3 = np.array(expectation)
# ['cpp-4.8.5-39.el7.x86_64' 'CUnit-2.1.3-8.el7.x86_64' ... 'util-linux-2.23.2-63.el7.x86_64' 'zlib-devel-1.2.7-18.el7.x86_64']
ready = []
for i in keywordlist:
for j in result:
x = np.char.startswith(j, i)
if x:
ready.append(np3[np.where(np.char.startswith(np3, i))])
np4 = np.array(ready)
# [array(['cpp-4.8.5-39.el7.x86_64'], dtype='<U39') array(['CUnit-2.1.3-8.el7.x86_64'], dtype='<U39') ... array(['util-linux-2.23.2-63.el7.x86_64'], dtype='<U39')]
notready = [i for i in np3 if i not in np4]
print(f"not ready: {notready}")
The purpose is to use string format keyword in keyword list to examine its existence in all np2 elements.
If any element in np2 starts with any keyword, or keyword is the substring of any element in np2, get the index of element in expectation which also start with that keyword and form into np4.
Finally， get not ready which is made up of elements that are in np3 but not in np4.
To make my explanation more vividly, I have a bunch of rpm files to be installed, the list of expectation.
The keyword list catches the former two keywords of each rpm file name.
Result is the standard output of already installed rpm files.
Taking cpp-4.8.5 as an example, I can see cpp-4.8.5-44.el7.x86_64 in result, which means currently cpp-4.8.5-44.el7.x86_64 has been installed. So, cpp-4.8.5-39.el7.x86_64 in expectation can be removed, since cpp-4.8.5-*.rpm has been successfully installed. Next step, deal with the other left items in expectation.
My question is: there any easier or more efficient way to get the result equivalent to notready? maybe with any other numpy built-in methods, but not with for loop.
Convert CONLL file to a list of Doc objects
Is there a way to convert CONLL file into list of Doc objects without having to parse the sentence using the nlp object. I have a list of annotations that I have to pass to the automatic component that uses Doc objects as input. I have found a way to create the doc: doc = Doc(nlp.vocab, words=[...]) And that I can use the from_array function to recreate the other linguistic features. This array can be recreated by using index value from StringStore object, I have successfully created Doc object with LEMMA and TAG information but cannot recreate HEAD data. My question is how to pass HEAD data to Doc object using from_array method. The confusing thing about the HEAD is that for sentence that has this structure: Ona 2 je 2 otišla 2 u 4 školu 2 . 2 The output of this code snippet: from spacy.attrs import TAG, HEAD, DEP doc.to_array([TAG, HEAD, DEP]) is: array([[10468770234730083819, 2, 429], [ 5333907774816518795, 1, 405], [11670076340363994323, 0, 8206900633647566924], [ 6471273018469892813, 1, 8110129090154140942], [ 7055653905424136462, 18446744073709551614, 435], [ 7173976090571422945, 18446744073709551613, 445]], dtype=uint64) I cannot correlate the center column of the from_array output to dependency tree structure given above. Thanks in advance for the help, Daniel
Ok, so I finally cracked it it appears that head - index if the index is lower than head and 18446744073709551616 - index otherwise. This is the function that I used if anyone else needed it: import numpy as np from spacy.tokens import Doc docs = [] for sent in sents: generated_doc = Doc(doc.vocab, words=[word["word"] for word in sent]) heads = [] for idx, word in enumerate(sent): if word["pos"] not in doc.vocab.strings: doc.vocab.strings.add(word["pos"]) if word["dep"] not in doc.vocab.strings: doc.vocab.strings.add(word["dep"]) if word["head"] >= idx: heads.append(word["head"] - idx) else: heads.append(18446744073709551616-idx) np_array = np.array([np.array([doc.vocab.strings[word["pos"]], heads[idx], doc.vocab.strings[word["dep"]]], dtype=np.uint64) for idx, word in enumerate(sent)], dtype=np.uint64) generated_doc.from_array([TAG, HEAD, DEP], np_array) docs.append(generated_doc)
Julia using package located in .julia/dev
I am beginner to Julia though I have experience with Python and some other languages. I get that this is probably a very simple/beginner issue, but I fail to understand how it should work in Julia.
I want to create a Julia module. I saw recommendations to create it with PkgTemplates, so that is exactly what I have done. My directory structure is thus:
It is located at the default path proposed by PkgTemplates: /home/username/.julia/dev/Keras2Flux.
I want to develop it with Revise package due to the slow start-up time of the Julia REPL. However, I fail to import my module to the Julia REPL in the terminal.
So, I cd to the directory mentioned above, use julia command and try using Keras2Flux. I get the error:
ERROR: ArgumentError: Package Keras2Flux not found in current path:
I tried both using Keras2Flux and using Keras2Flux.jl, and I also tried to call it from one level above in my directory structure (i.e. /home/username/.julia/dev). All has the same problem.
What is wrong (more importantly, why?) and how to fix it?
Current contents of the module (not really relevant to the question but still):
module Keras2Flux
import JSON
using Flux
export convert
function create_dense(config)
in = config["input_dim"]
out = config["output_dim"]
dense = Dense(in, outо)
return dense
end
function create_dropout(config)
p = config["p"]
dropout = Dropout(p)
return dropout
end
function create_model(model_config)
layers = []
for layer_config in model_config
if layer_config["class_name"] == "Dense"
layer = create_dense(layer_config["config"])
elseif layer_config["class_name"] == "Dropout"
layer = create_dropout(layer_config["config"])
else
println(layer_config["class_name"])
throw("unimplemented")
end
push!(layers, layer)
end
model = Chain(layers)
end
function convert(filename)
jsontxt = ""
open(filename, "r") do f
jsontxt = read(f, String)
end
model_params = JSON.parse(jsontxt)
if model_params["keras_version"] == "1.1.0"
create_model(model_params["config"])
else
throw("unimplemented")
end
end
end
Here is a full recipe to get you going:
cd("/home/username/.julia/dev")
using Pkg
pkg"generate Keras2Flux"
cd("Keras2Flux")
pkg"activate ."
pkg"add JSON Flux"
# now copy-paste whatever you need to Keras2Flux\src\Keras2Flux.jl
using Revise
using Keras2Flux
# happy development!
Splinter - Element is not clickable because another element <p> obscures it
I am trying to get some thumbnail pictures from a website, from src, as well as click on a link, so I can later get the big picture.
For that I'm using Splinter with BeautifulSoup.
This is the htmlfor the first element I need to get:
In order to do that I have the following code:
executable_path = {"executable_path": "/path/to/geckodriver"}
browser = Browser("firefox", **executable_path, headless=False
def get_player_images():
url = f'https://www.premierleague.com/players'
# Initiate a splinter instance of the URL
browser.visit(url)
browser.find_by_tag('div[class="table playerIndex"]')
soup = BeautifulSoup(browser.html, 'html.parser')
for el in soup:
td = el.findAll('td')
for each_td in td:
link = each_td.find('a', href=True)
if link:
print (link['href'])
image = each_td.find('img')
if image:
print(image['src'])
# run
get_player_images()
But I'm running into 2 issues, after browser opens:
I'm accessing only first two actual src for players. After that, photos are missing, which I don't get why.
/players/19970/Max-Aarons/overview
https://resources.premierleague.com/premierleague/photos/players/40x40/p232980.png
/players/13279/Abdul-Rahman-Baba/overview
https://resources.premierleague.com/premierleague/photos/players/40x40/p118335.png
/players/13286/Tammy-Abraham/overview
//platform-static-files.s3.amazonaws.com/premierleague/photos/players/40x40/Photo-Missing.png
/players/3512/Adam-Smith/overview
//platform-static-files.s3.amazonaws.com/premierleague/photos/players/40x40/Photo-Missing.png
/players/10905/Che-Adams/overview
....
Also, if I try to click on href link, with:
if link:
browser.click_link_by_partial_href(link['href'])
I get the error:
selenium.common.exceptions.ElementClickInterceptedException: Message: Element <a class="playerName" href="/players/19970/Max-Aarons/overview"> is not clickable at point (244,600) because another element <p> obscures it
What am I doing wrong? I'm running into a lot of troubles with selenium.
The player data is loaded dynamically via Javascript. You can use requests module to obtain the info.
For example:
import re
import json
import requests
from bs4 import BeautifulSoup
url = 'https://footballapi.pulselive.com/football/players?pageSize=30&compSeasons=274&altIds=true&page={page}&type=player&id=-1&compSeasonId=274'
img_url = 'https://resources.premierleague.com/premierleague/photos/players/250x250/{player_id}.png'
headers = {'Origin': 'https://www.premierleague.com'}
for page in range(1, 10): # <--- increase this to desired number of pages
data = requests.get(url.format(page=page), headers=headers).json()
# uncoment this to print all data:
# print(json.dumps(data, indent=4))
for player in data['content']:
print('{:<50} {}'.format(player['name']['display'], img_url.format(player_id=player['altIds']['opta'])))
Prints:
Ethan Ampadu https://resources.premierleague.com/premierleague/photos/players/250x250/p199598.png
Joseph Anang https://resources.premierleague.com/premierleague/photos/players/250x250/p447879.png
Florin Andone https://resources.premierleague.com/premierleague/photos/players/250x250/p93284.png
André Gomes https://resources.premierleague.com/premierleague/photos/players/250x250/p120250.png
Andreas Pereira https://resources.premierleague.com/premierleague/photos/players/250x250/p156689.png
Angeliño https://resources.premierleague.com/premierleague/photos/players/250x250/p145235.png
Faustino Anjorin https://resources.premierleague.com/premierleague/photos/players/250x250/p223332.png
Michail Antonio https://resources.premierleague.com/premierleague/photos/players/250x250/p57531.png
Cameron Archer https://resources.premierleague.com/premierleague/photos/players/250x250/p433979.png
Archie Davies https://resources.premierleague.com/premierleague/photos/players/250x250/p215061.png
Stuart Armstrong https://resources.premierleague.com/premierleague/photos/players/250x250/p91047.png
Marko Arnautovic https://resources.premierleague.com/premierleague/photos/players/250x250/p41464.png
Kepa Arrizabalaga https://resources.premierleague.com/premierleague/photos/players/250x250/p109745.png
Harry Arter https://resources.premierleague.com/premierleague/photos/players/250x250/p48615.png
Daniel Arzani https://resources.premierleague.com/premierleague/photos/players/250x250/p200797.png
... and so on.
Note: to get smaller thumbnails, change 250x250 in the image URLs to 40x40
OSMnx Limit on using "street network from bounding box": TypeError: graph_from_bbox() takes at most 15 arguments (77 given)
I am trying to get the drivable street network within some lat-long bounding box using the example 2a sample code from - https://geoffboeing.com/2016/11/osmnx-python-street-networks/ - but I am getting this error; TypeError: graph_from_bbox() takes at most 15 arguments (77 given). Below is what I have already tried out: G = ox.graph_from_bbox(29.94510876, 29.93205121, 29.93678994, 29.94840128, 29.94297549, 29.96456162, 29.96164721, 29.96828055, 29.91873862, 29.94221035, 29.95584061, 30.04064237, 29.93609316, 30.032814, 29.96624232, 30.00497466, 30.00427683, 29.94665333, 29.957519, 29.943813, 29.93076, 29.927549, 29.967799, 29.969906, 29.951438, 29.975021, 29.95932, 30.00816, 29.95056, 30.007622, 29.951881, 30.016095, 30.031229, 30.05131, 30.044959, 29.9382, 29.919781, 30.030601, -90.04488594, -90.07180566, -90.0849317, -90.12952617, -90.02696213, -90.03235984, -90.06925941, -90.09060393, -90.08716583, -90.11185615, -90.12122927, -89.91899769, -90.0844343, -89.97297866, -90.01417363, -90.10830816, -90.03988187, -90.07825592, -90.076855, -90.083341, -90.03282, -90.101536, -90.066648, -90.030283, -90.121145, -90.08682, -90.15624, -90.0648, -90.20634, -90.106042, -90.102726, -90.019069, -89.978768, -89.955024, -89.903415, -90.0685, -89.99601, -90.066334, network_type='drive') G_projected = ox.project_graph(G) ox.plot_graph(G_projected) G = ox.graph_from_bbox(29.94510876, 29.93205121, 29.93678994, 29.94840128, 29.94297549, 29.96456162, 29.96164721, 29.96828055, 29.91873862, 29.94221035, 29.95584061, 30.04064237, 29.93609316, 30.032814, 29.96624232, 30.00497466, 30.00427683, 29.94665333, 29.957519, 29.943813, 29.93076, 29.927549, 29.967799, 29.969906, 29.951438, 29.975021, 29.95932, 30.00816, 29.95056, 30.007622, 29.951881, 30.016095, 30.031229, 30.05131, 30.044959, 29.9382, 29.919781, 30.030601, -90.04488594, -90.07180566, -90.0849317, -90.12952617, -90.02696213, -90.03235984, -90.06925941, -90.09060393, -90.08716583, -90.11185615, -90.12122927, -89.91899769, -90.0844343, -89.97297866, -90.01417363, -90.10830816, -90.03988187, -90.07825592, -90.076855, -90.083341, -90.03282, -90.101536, -90.066648, -90.030283, -90.121145, -90.08682, -90.15624, -90.0648, -90.20634, -90.106042, -90.102726, -90.019069, -89.978768, -89.955024, -89.903415, -90.0685, -89.99601, -90.066334, network_type='drive') G_projected = ox.project_graph(G) ox.plot_graph(G_projected) I am expecting to get the drivable street network within those listed lat-long bounding box.
So the function is limited to taking only 4 arguments which define the coordinates (bounding box) and should be in the following order as mentioned in the docs (OSMNX). osmnx.core.graph_from_bbox(north, south, east, west) This means that one can only specify a rectangle as opposed to a generalised polygon, which I think you are trying to specify here. Hence the above error was thrown. import osmnx as ox G = ox.graph_from_bbox( -37.7860, -37.8359, 144.9903, 144.9269, network_type='drive', simplify=True, retain_all=False)