Why would Pycharm put the second half of my dataset on a new line? (see Image) How do I turn this off?
I would like to display my dataset as wide as possible, with no wrapping.
The console attempts to auto-detect the width of the display area, but when that fails it defaults to 80 characters. This behavior can be overridden with:
import pandas as pd
pd.set_option('display.width', 400)
pd.set_option('display.max_columns', 10)
please take reference from this stackoverflow
Getting wider output in PyCharm's built-in console
Related
I am working on the Jupyter notebook and have been facing issues in increasing the length of the output of the Jupyter Notebook. I can see the output as follows:
I tried increasing the default length of the columns in pandas with no success. Can you please help me with it?
If you were using the typical way to view a dataframe in Jupyter (see my puzzelment about your screenshot in my comments to your original post) it would be things like this:
adapted from answer to 'Pretty-print an entire Pandas Series / DataFrame'
with pd.option_context('display.max_rows', None, 'display.max_columns', None):
display(df)
(Note that will work with the text-based viewing, too. Note it uses print(df) in the answer to 'Pretty-print an entire Pandas Series / DataFrame'.
Adjust the 'display.max_colwidth' if you want the entire column text to show:
with pd.option_context('display.max_rows', None, 'display.max_columns', None,'display.max_colwidth', -1):
display(df)
(If you prefer text like you posted, replace display() with print()
Generally with the solutions above the view window in Jupyter will get scrollbars so you can navigate to view all still.
You can also set the number of rows to show to be lower to save space, see example here.
You may also be interested in Pandas dataframe hide index functionality? or Using python / Jupyter Notebook, how to prevent row numbers from printing?.
As pointed out here, setting some some global options is covered in the Pandas Documentation for top-level options.
For display() to work these days you don't need to do anything extra. But if your are using old Jupyter or it doesn't work then try adding towards the top of your notebook file and running the following as a cell first:
from IPython.display import display
I am working with Matplotlib color maps, and I also happen to be working with the Spyder IDE, which has an ipython console.
As you can see from the screen shot, the ipython console showed me a graphical representation of the color map object. This was unexpected and very helpful.
Normally I expect to see a string representation of an object, as you might see from the print() function call. Function calls to print() and repr() are shown, and they produce text, as is more typical.
I would like my own code to output this graphical representation when it is generating output. I have been poking through the matplotlib.colors.Colormap internals, and so far I haven't been able to figure out how. What is ipython doing? How can I do the same?
Thanks!
This rather seems like a ipython/jupyter feature. ipython detects the object and produces automatically a plot to preview the colormap.
Here using jupyter:
IPython looks if an object has a _repr_html_; if so, it calls it and displays the output as HTML. Here's an example (I ran this in Jupyter but it works the same as long as you're running IPython):
class MyCoolObject:
def _repr_html_(self):
return ("<h1>hello!</h1> <p>this is some html </p>"
"I can even put images:"
"<img src='https://upload.wikimedia.org/wikipedia"
"/commons/thumb/3/38/Jupyter_logo.svg"
"/44px-Jupyter_logo.svg.png'></img>")
MyCoolObject()
To add on to Eduardo's answer, from everything I've read adding a _repr_html_ method should make iPython display the object when you type it into the console. I also use spyder though, and could not get it to work the way I expected. This simple wrapper class should allow you to display any html:
class displayedHTML:
def __init__(self, html):
self.html = html
def _repr_html_(self):
return self.html
But as you can see it does not work for me, instead showing the (implicitly defined) __repr__ of the class.
In [2]: obj = displayedHTML("<h1>" + "hello world" + "</h1>")
In [3]: obj
Out[3]: <__main__.displayedHTML at 0x1e8cda8f0d0>
I was not able to find the reason why this does not work, but I found a workaround if you just want to display a matplotlib colormap in the console from code (like I did).
Since know the matplotlib object works correctly, we can just give it to the ipython display function:
from IPython.display import display #Included without import since IPython 5.4 and 6.1
viridis = matplotlib.cm.get_cmap('viridis')
display(viridis)
And for me this works...
not_allowed_to_insert_pictures_yet.jpg
Hope this helps!
I am trying to represent cubic spline interpolation information for function f(x) as a dataframe.
When trying to print into a spyder, I find that the columns are being cut off. When trying to reproduce the output in Jupiter Lab, I got the same thing.
When I ran in ipython via terminal I got the desired full dataframe output.
I searched the integnet and tried the pandas commands setting options pd.set_options(), but nothing came of it.
I attach a screenshot with the output in ipython.
In Juputer can use:
from IPython.display import display, HTML
and instead of
print(dataframe)
use of in anyway place
display(HTML(dataframe.to_html()))
This will create a nice table.
Unfortunately, this will not work in the spyder. So you can try to adjust the width of the ipython were suggested. But in most cases this will make the output poorly or unreadable.
After trying the dataframe methods, I found what appears to be a cropping setting.
In Spyder I used:
pd.set_option('expand_frame_repr', False)
print(dataframe)
This method explains why increasing max_column didn't help me previously.
You can specify a maximum number for rows or columns using pd.set_options(display.max_columns=1000)
But you don't have to set an arbitrary value, but rather use None instead to make sure every size will be covered.
For rows, use:
pd.set_option('display.max_rows', None)
And for columns, use:
pd.set_option('display.max_columns', None)
It is a result of the display width. You can use the following set_options():
pd.set_options(display.width=1000) #make huge
You may also have to raise max columns but it should be smart enough to adjust automatically after you make width bigger:
pd.set_options(display.max_columns=None)
I am trying to make a Choropleth map using matplotlib and cartopy for which I obviously need to plot a shapefile first. However, I did not manage to do so, even though a similar question has been asked here and here. I suspect either the projection or the bounds to be misspecified.
My shapefile has the projection
PROJCS["WGS_1984_UTM_Zone_32Nz",
GEOGCS["GCS_WGS_1984",
DATUM["WGS_1984",
SPHEROID["WGS_84",6378137,298.257223563]],
PRIMEM["Greenwich",0],
UNIT["Degree",0.017453292519943295]],
PROJECTION["Transverse_Mercator"],
PARAMETER["False_Easting",32500000],
PARAMETER["False_Northing",0],
PARAMETER["Central_Meridian",9],
PARAMETER["Scale_Factor",0.9996],
PARAMETER["Latitude_Of_Origin",0],
UNIT["Meter",1]]
and can be downloaded here where I am talking about vg250_2010-01-01.utm32w.shape.ebenen/vg250_ebenen-historisch/de1001/vg250_gem.shp
My code is
#!/usr/local/bin/python
# -*- coding: utf8 -*-
import cartopy.crs as ccrs
import cartopy.io.shapereader as shpreader
import matplotlib.pyplot as plt
fname = 'path/vg250_gem.shp'
proj = ccrs.TransverseMercator(central_longitude=0.0,central_latitude=0.0,
false_easting=32500000.0,false_northing=0.0,
scale_factor=0.9996)
municipalities = list(shpreader.Reader(fname).geometries())
ax = plt.axes(projection=proj)
plt.title('Deutschland')
ax.add_geometries(municipalities,proj,edgecolor='black',facecolor='gray',alpha=0.5)
ax.set_extent([32458044.649189778*0.9, 5556418.748046352*1.1, 32465287.307457082*0.9, 5564153.5456742775*1.1],proj)
plt.show()
where I obtained the bounds using the corresponding method from fiona. Python throws an error
Traceback (most recent call last):
File "***/src/analysis/test.py", line 16, in <module>
ax.set_extent([32458044.649189778, 5556418.748046352, 32465287.307457082, 5564153.5456742775],proj)
File "/usr/local/lib/python2.7/site-packages/cartopy/mpl/geoaxes.py", line 652, in set_extent
ylim=self.projection.y_limits))
ValueError: Failed to determine the required bounds in projection coordinates. Check that the values provided are within the valid range (x_limits=[-20000000.0, 20000000.0], y_limits=[-10000000.0, 10000000.0]).
[Finished in 53.9s with exit code 1]
This doesn't make sense to me. Also, experimenting with ccrs.UTM() gives me a plot showing a white area. I'd appreciate it if anyone can tell me how to fix this. Thank you!
I have found two issues. One is an incorrect specification of limits in your call to set_extent, the documentation specifies [x0,x1,y0,y1] should be the input, you seem to have given [x0,y0,x1,y1].
The other issue seems to be a limitation in cartopy, as best I can tell. It looks like projections outside the limits listed in the error message will always fail, and those limits are hardcoded. You can just edit the source (this line in their latest release), changing -2e7 to -4e7, likewise for the upper bound. After these fixes, your plot is generated without issue:
The new set_extent line:
ax.set_extent([32458044.649189778*0.975, 32465287.307457082*1.025,5556418.748046352*0.9, 556415,3.5456742775*1.1],proj)
You may also want to set central_longitude=9.0 in your TransverseMercator, that seems to be what's specified in your shapefile.
I would recommend contacting the developers about this, they might have a good reason for setting those bounds, or they might have a better workaround, or maybe they'll widen the bounds in a later release!
Update
Your bounds also seem to have been set based on only the first of the municipalities:
In [34]: municipalities[0].bounds
Out[34]: (32458044.649189778, 5556418.748046352, 32465287.307457082, 5564153.5456742775)
But the other elements have different bounds. You can get limits flushed to the actual drawing based on min/max values of the bounds of all municipalities.
bnd = np.array([i.bounds for i in municipalities])
x0,x1 = np.min(bnd[:,0]),np.max(bnd[:,2])
y0,y1 = np.min(bnd[:,1]),np.max(bnd[:,3])
ax.set_extent([x0,x1,y0,y1],proj)
I wrote a script that calls functions from QIIME to build a bunch of plots among other things. Everything runs fine to completion, but matplotlib always throws the following feedback for every plot it creates (super annoying):
/usr/local/lib/python2.7/dist-packages/matplotlib/pyplot.py:412: RuntimeWarning: More than 20 figures have been opened. Figures created through the pyplot interface (matplotlib.pyplot.figure) are retained until explicitly closed and may consume too much memory. (To control this warning, see the rcParam figure.max_num_figures).
max_open_warning, RuntimeWarning)
I found this page which seems to explain how to fix this problem , but after I follow directions, nothing changes:
import matplotlib as mpl
mpl.rcParams[figure.max_open_warning'] = 0
I went into the file after calling matplotlib directly from python to see which rcparams file I should be investigating and manually changed the 20 to 0. Still no change. In case the documentation was incorrect, I also changed it to 1000, and still am getting the same warning messages.
I understand that this could be a problem for people running on computers with limited power, but that isn't a problem in my case. How can I make this feedback go away permanently?
Try setting it this way:
import matplotlib as plt
plt.rcParams.update({'figure.max_open_warning': 0})
Not sure exactly why this works, but it mirrors the way I have changed the font size in the past and seems to fix the warnings for me.
Another way I just tried and it worked:
import matplotlib as mpl
mpl.rc('figure', max_open_warning = 0)
When using Seaborn you can do it like this
import seaborn as sns
sns.set_theme(rc={'figure.max_open_warning': 0})
Check out this article which basically says to plt.close(fig1) after you're done with fig1. This way you don't have too many figs floating around in memory.
In Matplotlib, figure.max_open_warning is a configuration parameter that determines the maximum number of figures that can be opened before a warning is issued. By default, the value of this parameter is 20. This means that if you open more than 20 figures in a single Matplotlib session, you will see a warning message. You can change the value of this parameter by using the matplotlib.rcParams function. For example:
import matplotlib.pyplot as plt
plt.rcParams['figure.max_open_warning'] = 50
This will set the value of figure.max_open_warning to 50, so that you will see a warning message if you open more than 50 figures in a single Matplotlib session.