I'm trying to simulate the segregation process in a city for a school project. I've managed to plot the city when initialized and after segregation, but I don't manage to create the animation showing the city's inhabitants moving to show the evolution.
I have two methods in my Ville class (I'm coding in French) that should make the animation together.
def afficher(self, inclure_satisfaction=False, inclure_carte_categories=False, size=5):
carte = self.carte_categories(inclure_satisfaction=inclure_satisfaction)
if inclure_carte_categories:
print("Voici la carte des catégories (à titre de vérification)")
print(carte)
mat_rs = masked_array(carte, carte!=1.5)
mat_ri = masked_array(carte, carte!=1)
mat_bs = masked_array(carte, carte!=2.5)
mat_bi = masked_array(carte, carte!=2)
plt.figure(figsize=(size, size))
affichage_rs = plt.imshow(mat_rs, cmap=cmap_rs)
affichage_ri = plt.imshow(mat_ri, cmap=cmap_ri)
affichage_bs = plt.imshow(mat_bs, cmap=cmap_bs)
affichage_bi = plt.imshow(mat_bi, cmap=cmap_bi)
return plt.figure()
(this function plot the map by first getting an array from the method carte_categories in function of the category of each inhabitant and then getting an array for each value to plot)
def resoudre2(self):
fig = plt.figure(figsize=(5,5))
list_of_artists = []
while self.habitants_insatisfaits != []:
self.demenagement_insatisfait_aleatoire()
list_of_artists.append([self.afficher(inclure_satisfaction=True)])
ani = ArtistAnimation(fig, list_of_artists, interval=200, blit=True)
return ani
(habitants_insatisfaits is a list that contains the "insatisfied inhabitants": there are two few people of their category around them, so they want to move somewhere else; so resoudre means solve, and this function loops until all the inhabitants are satisfied where they are (and this way the society is mechanically segregated)
The initialized city looks like this initialized city (dark colors for insatisfied inhabitants), and the segregated city looks like that segregated city.
But when I enter
a = ville1.resoudre2(compter=True)
I don't get an animation but only this error message:
/usr/local/lib/python3.7/dist-packages/ipykernel_launcher.py:211: RuntimeWarning: More than 20 figures have been opened. Figures created through the pyplot interface (`matplotlib.pyplot.figure`) are retained until explicitly closed and may consume too much memory. (To control this warning, see the rcParam `figure.max_open_warning`).
/usr/local/lib/python3.7/dist-packages/ipykernel_launcher.py:206: RuntimeWarning: More than 20 figures have been opened. Figures created through the pyplot interface (`matplotlib.pyplot.figure`) are retained until explicitly closed and may consume too much memory. (To control this warning, see the rcParam `figure.max_open_warning`).
Traceback (most recent call last):
File "/usr/local/lib/python3.7/dist-packages/matplotlib/cbook/__init__.py", line 196, in process
func(*args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/matplotlib/animation.py", line 951, in _start
self._init_draw()
File "/usr/local/lib/python3.7/dist-packages/matplotlib/animation.py", line 1533, in _init_draw
fig.canvas.draw_idle()
AttributeError: 'NoneType' object has no attribute 'canvas'
/usr/local/lib/python3.7/dist-packages/matplotlib/image.py:452: UserWarning: Warning: converting a masked element to nan.
dv = np.float64(self.norm.vmax) - np.float64(self.norm.vmin)
/usr/local/lib/python3.7/dist-packages/matplotlib/image.py:459: UserWarning: Warning: converting a masked element to nan.
a_min = np.float64(newmin)
/usr/local/lib/python3.7/dist-packages/matplotlib/image.py:464: UserWarning: Warning: converting a masked element to nan.
a_max = np.float64(newmax)
<string>:6: UserWarning: Warning: converting a masked element to nan.
/usr/local/lib/python3.7/dist-packages/matplotlib/colors.py:993: UserWarning: Warning: converting a masked element to nan.
data = np.asarray(value)
(first problem) and then every map (corresponding to each step of the segregating city) is plotted (second problem; see here). And when I try to type
print(a)
from IPython.display import HTML
HTML(a.to_html5_video())
to plot the animation, I only get
<matplotlib.animation.ArtistAnimation object at 0x7f4cd376bfd0>
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-20-d7ca1fcdadb6> in <module>()
1 print(a)
2 from IPython.display import HTML
----> 3 HTML(a.to_html5_video())
2 frames
/usr/local/lib/python3.7/dist-packages/matplotlib/animation.py in _init_draw(self)
1531 # Flush the needed figures
1532 for fig in figs:
-> 1533 fig.canvas.draw_idle()
1534
1535 def _pre_draw(self, framedata, blit):
AttributeError: 'NoneType' object has no attribute 'canvas'
So I don't understand why I get this error and not just my animation...
Thank you for your help, it's the first time I ask questions here so don't hesitate if you need more details about my code! :)
Nathan
Had the same issue, downgrading Matplotlib fixed the issue for me.
pip install matplotlib==3.5.1
Related
I got the following code to elaborate on my problem. I'm using python 3.6 with pandas==0.25.3.
import pandas as pd
from enum import Enum, IntEnum
class BookType(Enum):
DRAMA = 5
ROMAN = 3
class AuthorType(IntEnum):
UNKNOWN = 0
GROUP = 1
MAN = 2
def print_num_type(df: pd.DataFrame, col_name: str, enum_type: Enum) -> int:
counts = df[col_name].value_counts()
val = counts[enum_type]
print('value counts:', counts)
print(f'Found "{val}" of type {enum_type}')
d = {'title': ['Charly Morry', 'James', 'Watson', 'Marry L.'], 'isbn': [21412412, 334764712, 12471021, 124141111], 'book_type': [BookType.DRAMA, BookType.ROMAN, BookType.ROMAN, BookType.ROMAN], 'author_type': [AuthorType.UNKNOWN, AuthorType.UNKNOWN, AuthorType.MAN, AuthorType.UNKNOWN]}
df = pd.DataFrame(data=d)
df.set_index(['title', 'isbn'], inplace=True)
df['book_type'] = df['book_type'].astype('category')
df['author_type'] = df['author_type'].astype('category')
print(df)
print(df.dtypes)
print_num_type(df, 'book_type', BookType.DRAMA)
print_num_type(df, 'author_type', AuthorType.UNKNOWN)
My pandas.DataFrame consists of two columns (book_type and author_type) of type categorical.
Furthermore, book_type is a class inheriting from type Enum and author_type from IntEnum. When calling print_num_type(df, 'book_type', BookType.DRAMA) everything works out as expected and the number of books of this type are printed, whereas print_num_type(df, 'author_type', AuthorType.UNKNOWN) raises the error:
Traceback (most recent call last):
File "C:\Users\User\AppData\Local\Programs\Python\Python36-32\lib\abc.py", line 182, in __instancecheck__
if subclass in cls._abc_cache:
File "C:\Users\User\AppData\Local\Programs\Python\Python36-32\lib\_weakrefset.py", line 72, in __contains__
wr = ref(item)
RecursionError: maximum recursion depth exceeded while calling a Python object
Exception ignored in: 'pandas._libs.lib.c_is_list_like'
Traceback (most recent call last):
File "C:\Users\User\AppData\Local\Programs\Python\Python36-32\lib\abc.py", line 182, in __instancecheck__
if subclass in cls._abc_cache:
File "C:\Users\User\AppData\Local\Programs\Python\Python36-32\lib\_weakrefset.py", line 72, in __contains__
wr = ref(item)
RecursionError: maximum recursion depth exceeded while calling a Python object
What am I doing wrong here?
Is there a workaround to get this error fixed? since I can't change the IntEnum type of AuthorType since it's provided from another library.
Thanks in advance!
See answer here
The main idea is that since x.value_counts() or counts in your function is itself a pandas Series, it's best to use .iat or .iloc when calling it, e.g, see iat docs
I think the easiest solution is to just use (x==0).sum(), or in your syntax:
val = (df[col_name]==enum_type).sum()
I put a minimal working example in the comments under your question so you can reproduce the problem/fix easily with the "x" notation.
What version of Pandas are you using? I realized after reproducing the error that upgrading Pandas (now on pandas-1.4.2) fixes the error, and the value_counts()[0] worked as expected.
run pip install --upgrade pandas
I have 2 issues nested as one:
n_rows, n_cols = np.shape(Z)
ZT = Z.transpose()
ZTZ = np.dot(ZT,Z) # does return a value
ZTZ1 = np.matmul(ZT,Z) # error
print("Close?")
print(np.allclose(ZTZ,ZTZ1))
print("----")
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-211-f26bdaebc910> in <module>()
26
27 print
---> 28 coV = getCovariance(df)
29 #print(coV)
30 print
<ipython-input-211-f26bdaebc910> in getCovariance(df)
13 ZT = Z.transpose()
14 ZTZ = np.dot(ZT,Z)
---> 15 ZTZ1 = np.matmul(ZT,Z)
16 print("Close?")
17 print(np.allclose(ZTZ,ZTZ1))
AttributeError: 'module' object has no attribute 'matmul'
Okay ... so obviously matmul doesn't exist on my machine. Got it. Now how do I confirm that the dot is doing the same thing? Because I have a matrix that was once a pandas.DataFrame object and I converted it to a matrix through it's .as_matrix() method and I am getting rounding errors and need to check where things went wrong ... I also tried the standard * operator, but that doesn't work either on np.ndarray matrix objects.
SIDE NOTE: if there are any pro tips on rounding that could be transferred from someone with experience with pandas, that is also much appreciated because I can't seem to find out how pandas has given me a different matrix than a build in function from the numpy class (I have been asked to reimplement the function).
I am trying to make a Choropleth map using matplotlib and cartopy for which I obviously need to plot a shapefile first. However, I did not manage to do so, even though a similar question has been asked here and here. I suspect either the projection or the bounds to be misspecified.
My shapefile has the projection
PROJCS["WGS_1984_UTM_Zone_32Nz",
GEOGCS["GCS_WGS_1984",
DATUM["WGS_1984",
SPHEROID["WGS_84",6378137,298.257223563]],
PRIMEM["Greenwich",0],
UNIT["Degree",0.017453292519943295]],
PROJECTION["Transverse_Mercator"],
PARAMETER["False_Easting",32500000],
PARAMETER["False_Northing",0],
PARAMETER["Central_Meridian",9],
PARAMETER["Scale_Factor",0.9996],
PARAMETER["Latitude_Of_Origin",0],
UNIT["Meter",1]]
and can be downloaded here where I am talking about vg250_2010-01-01.utm32w.shape.ebenen/vg250_ebenen-historisch/de1001/vg250_gem.shp
My code is
#!/usr/local/bin/python
# -*- coding: utf8 -*-
import cartopy.crs as ccrs
import cartopy.io.shapereader as shpreader
import matplotlib.pyplot as plt
fname = 'path/vg250_gem.shp'
proj = ccrs.TransverseMercator(central_longitude=0.0,central_latitude=0.0,
false_easting=32500000.0,false_northing=0.0,
scale_factor=0.9996)
municipalities = list(shpreader.Reader(fname).geometries())
ax = plt.axes(projection=proj)
plt.title('Deutschland')
ax.add_geometries(municipalities,proj,edgecolor='black',facecolor='gray',alpha=0.5)
ax.set_extent([32458044.649189778*0.9, 5556418.748046352*1.1, 32465287.307457082*0.9, 5564153.5456742775*1.1],proj)
plt.show()
where I obtained the bounds using the corresponding method from fiona. Python throws an error
Traceback (most recent call last):
File "***/src/analysis/test.py", line 16, in <module>
ax.set_extent([32458044.649189778, 5556418.748046352, 32465287.307457082, 5564153.5456742775],proj)
File "/usr/local/lib/python2.7/site-packages/cartopy/mpl/geoaxes.py", line 652, in set_extent
ylim=self.projection.y_limits))
ValueError: Failed to determine the required bounds in projection coordinates. Check that the values provided are within the valid range (x_limits=[-20000000.0, 20000000.0], y_limits=[-10000000.0, 10000000.0]).
[Finished in 53.9s with exit code 1]
This doesn't make sense to me. Also, experimenting with ccrs.UTM() gives me a plot showing a white area. I'd appreciate it if anyone can tell me how to fix this. Thank you!
I have found two issues. One is an incorrect specification of limits in your call to set_extent, the documentation specifies [x0,x1,y0,y1] should be the input, you seem to have given [x0,y0,x1,y1].
The other issue seems to be a limitation in cartopy, as best I can tell. It looks like projections outside the limits listed in the error message will always fail, and those limits are hardcoded. You can just edit the source (this line in their latest release), changing -2e7 to -4e7, likewise for the upper bound. After these fixes, your plot is generated without issue:
The new set_extent line:
ax.set_extent([32458044.649189778*0.975, 32465287.307457082*1.025,5556418.748046352*0.9, 556415,3.5456742775*1.1],proj)
You may also want to set central_longitude=9.0 in your TransverseMercator, that seems to be what's specified in your shapefile.
I would recommend contacting the developers about this, they might have a good reason for setting those bounds, or they might have a better workaround, or maybe they'll widen the bounds in a later release!
Update
Your bounds also seem to have been set based on only the first of the municipalities:
In [34]: municipalities[0].bounds
Out[34]: (32458044.649189778, 5556418.748046352, 32465287.307457082, 5564153.5456742775)
But the other elements have different bounds. You can get limits flushed to the actual drawing based on min/max values of the bounds of all municipalities.
bnd = np.array([i.bounds for i in municipalities])
x0,x1 = np.min(bnd[:,0]),np.max(bnd[:,2])
y0,y1 = np.min(bnd[:,1]),np.max(bnd[:,3])
ax.set_extent([x0,x1,y0,y1],proj)
I am trying to make some plots of sea ice data. The data is delivered in the EASE-North grid, an example file (HDF4) can be downloaded at:
ftp://n4ftl01u.ecs.nasa.gov/SAN/OTHR/NISE.004/2013.09.30/
I created a custom projection class for the EASE-Grid, it seems to be working (the coastlines align well with the data).
When i try to add a Natural Earth feature, it returns an empty Matplotlib figure.
import gdal
import cartopy
# projection class
class EASE_North(cartopy.crs.Projection):
def __init__(self):
# see: http://www.spatialreference.org/ref/epsg/3408/
proj4_params = {'proj': 'laea',
'lat_0': 90.,
'lon_0': 0,
'x_0': 0,
'y_0': 0,
'a': 6371228,
'b': 6371228,
'units': 'm',
'no_defs': ''}
super(EASE_North, self).__init__(proj4_params)
#property
def boundary(self):
coords = ((self.x_limits[0], self.y_limits[0]),(self.x_limits[1], self.y_limits[0]),
(self.x_limits[1], self.y_limits[1]),(self.x_limits[0], self.y_limits[1]),
(self.x_limits[0], self.y_limits[0]))
return cartopy.crs.sgeom.Polygon(coords).exterior
#property
def threshold(self):
return 1e5
#property
def x_limits(self):
return (-9000000, 9000000)
#property
def y_limits(self):
return (-9000000, 9000000)
# read the data
ds = gdal.Open('D:/NISE_SSMISF17_20130930.HDFEOS')
# this loads the layers for both hemispheres
data = np.array([gdal.Open(name, gdal.GA_ReadOnly).ReadAsArray()
for name, descr in ds.GetSubDatasets() if 'Extent' in name])
ds = None
# mask anything other then sea ice
sea_ice_concentration = np.ma.masked_where((data < 1) | (data > 100), data, 0)
# plot
lim = 3000000
fig, ax = plt.subplots(figsize=(8,8),subplot_kw={'projection': EASE_North(), 'xlim': [-lim,lim], 'ylim': [-lim,lim]})
land = cartopy.feature.NaturalEarthFeature(
category='physical',
name='land',
scale='50m',
facecolor='#dddddd',
edgecolor='none')
#ax.add_feature(land)
ax.coastlines()
# from the metadata in the HDF
extent = [-9036842.762500, 9036842.762500, -9036842.762500, 9036842.762500]
ax.imshow(sea_ice_concentration[0,:,:], cmap=plt.cm.Blues, vmin=1,vmax=100,
interpolation='none', origin='upper', extent=extent, transform=EASE_North())
The script above works fine and produces this result:
But when i uncomment the ax.add_feature(land) it fails without any error, only returning the empty figure. Am i missing something obvious?
Here is the IPython Notebook:
http://nbviewer.ipython.org/6779935
My Cartopy build is version 0.9 from Christoph Gohlke's website (thanks!).
edit:
Trying to save the figure does throw an exception:
fig.savefig(r'D:\test.png')
C:\Python27\Lib\site-packages\shapely\speedups\_speedups.pyd in shapely.speedups._speedups.geos_linearring_from_py (shapely/speedups/_speedups.c:2270)()
ValueError: A LinearRing must have at least 3 coordinate tuples
Examining the 'land' cartopy.feature reveals no issues, all polygons pass the .isvalid() and all rings (ext en int) are of 4 or more tuples. So the input shape doesnt seem to be the problem (and works fine in PlateCaree()).
Maybe some rings (like on the southern hemisphere) get 'corrupt' after transforming to EASE_North?
edit2:
When i remove the build-in NE features and load the same shapefile (but with anything below 40N clipped) it works. So it seems like some sort of reprojection issue.
for state in shpreader.Reader(r'D:\ne_50m_land_clipped.shp').geometries():
ax.add_geometries([state], cartopy.crs.PlateCarree(),facecolor='#cccccc', edgecolor='#cccccc')
I'd have said that this was a bug. I'm guessing add_feature updates the matplotlib viewLim and the result is that the picture zooms in to a tiny area (which appears white unless you zoom out a lot).
From the top of my head, I think the underlying behaviour has been improved in matplotlib, but cartopy is not yet making use of the new viewLim calculation. In the meantime I'd suggest setting the extents of your map manually with:
ax.set_extent(extent, transform=EASE_North())
HTH
I'm trying to cluster some data with python and scipy but the following code does not work for reason I do not understand:
from scipy.sparse import *
matrix = dok_matrix((en,en), int)
for pub in pubs:
authors = pub.split(";")
for auth1 in authors:
for auth2 in authors:
if auth1 == auth2: continue
id1 = e2id[auth1]
id2 = e2id[auth2]
matrix[id1, id2] += 1
from scipy.cluster.vq import vq, kmeans2, whiten
result = kmeans2(matrix, 30)
print result
It says:
Traceback (most recent call last):
File "cluster.py", line 40, in <module>
result = kmeans2(matrix, 30)
File "/usr/lib/python2.7/dist-packages/scipy/cluster/vq.py", line 683, in kmeans2
clusters = init(data, k)
File "/usr/lib/python2.7/dist-packages/scipy/cluster/vq.py", line 576, in _krandinit
return init_rankn(data)
File "/usr/lib/python2.7/dist-packages/scipy/cluster/vq.py", line 563, in init_rankn
mu = np.mean(data, 0)
File "/usr/lib/python2.7/dist-packages/numpy/core/fromnumeric.py", line 2374, in mean
return mean(axis, dtype, out)
TypeError: mean() takes at most 2 arguments (4 given)
When I'm using kmenas instead of kmenas2 I have the following error:
Traceback (most recent call last):
File "cluster.py", line 40, in <module>
result = kmeans(matrix, 30)
File "/usr/lib/python2.7/dist-packages/scipy/cluster/vq.py", line 507, in kmeans
guess = take(obs, randint(0, No, k), 0)
File "/usr/lib/python2.7/dist-packages/numpy/core/fromnumeric.py", line 103, in take
return take(indices, axis, out, mode)
TypeError: take() takes at most 3 arguments (5 given)
I think I have the problems because I'm using sparse matrices but my matrices are too big to fit the memory otherwise. Is there a way to use standard clustering algorithms from scipy with sparse matrices? Or I have to re-implement them myself?
I created a new version of my code to work with vector space
el = len(experts)
pl = len(pubs)
print el, pl
from scipy.sparse import *
P = dok_matrix((pl, el), int)
p_id = 0
for pub in pubs:
authors = pub.split(";")
for auth1 in authors:
if len(auth1) < 2: continue
id1 = e2id[auth1]
P[p_id, id1] = 1
from scipy.cluster.vq import kmeans, kmeans2, whiten
result = kmeans2(P, 30)
print result
But I'm still getting the error:
TypeError: mean() takes at most 2 arguments (4 given)
What am I doing wrong?
K-means cannot be run on distance matrixes.
It needs a vector space to compute means in, that is why it is called k-means. If you want to use a distance matrix, you need to look into purely distance based algorithms such as DBSCAN and OPTICS (both on Wikipedia).
May I suggest, "Affinity Propagation" from scikit-learn? On the work I've been doing with it, I find that it has generally been able to find the 'naturally' occurring clusters within my data set. The inputs into the algorithm are an affinity matrix, or similarity matrix, of any arbitrary similarity measure.
I don't have a good handle on the kind of data you have on hand, so I can't speak to the exact suitability of this method to your data set, but it may be worth a try, perhaps?
Alternatively, if you're looking to cluster graphs, I'd take a look at NetworkX. That might be a useful tool for you. The reason I suggest this is because it looks like the data you're looking to work with networks of authors. Hence, with NetworkX, you can put in an adjacency matrix and find out which authors are clustered together.
For a further elaboration on this, you can see a question that I had asked earlier for inspiration here.