Here is my code snippet:
>>> import numpy as np
>>> import pandas as pd
>>> pd.__version__
'0.12.0'
>>> np.__version__
'1.6.2'
>>> pd.DataFrame(None,columns=['x'])['x'].values + np.min([0.5])
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: unsupported operand type(s) for +: 'numpy.ndarray' and 'numpy.float64'
I'm making a sum between an empty array and a scalar. Notice that if I construct the array with np.array or with pd.Series it works fine (the result is an empty array). The same is true if I replace np.min([0.5]) (which is a np.float64) with a float: 0.5.
I find this very strange! Of course I can easily patch my program, but I think this is a bug of pandas, isn't it?
addendum is it correct to initialize pd.DataFrame with None to get an empty DataFrame with given columns?
Related
I am trying to execute the following code:
sapmle2000submission.astype('int32').dtypes
which raises an error:
AttributeError Traceback (most recent call
last) in ()
----> 1 sapmle2000submission.astype('int32').dtypes
AttributeError: 'list' object has no attribute 'astype'
Can someone please help me to figure out why?
Looks like that the obejct sapmle2000submission is a list, not a pandas Series.
You can convert it as Series and specify its dtype:
import pandas as pd
sapmle2000submission_series = pd.Series(sapmle2000submission, dtype='int32')
I imported pandas as pd.
>>>import pandas as pd
Assigned Series variables 'city_names' and 'population'
>>> city_names=pd.Series(['San Fransisco','San Jose','Sacromento'])
>>> population=pd.Series([8964,91598,034892])
Created DataFrame
>>> pd.DataFrame=({'City Name': city_names, 'Population':population})
While assigning DataFrame to the variable 'cities',
>>> cities=pd.DataFrame({'City Name ' : city_names, 'Population' : population})
I am getting the following error :
Traceback (most recent call last):
File "<pyshell#17>", line 1, in <module>
cities=pd.DataFrame({'City Name ' : city_names, 'Population' : population})
TypeError: 'dict' object is not callable
Also aware that 'dict' is a dictionary with (key,value) pairs! please let me know why this error occurs. I looked into other questions as well but could'nt find help.
Your code pd.DataFrame=({'City Name': city_names, 'Population':population}) replaced pd.DataFrame function with a dictionary.
#Ankur please modify this edit to your liking.
PiR edit:
pd.DataFrame is a class defined by pandas. A class is "callable" meaning that you can call it with parentheses like so pd.DataFrame(). However, in Python the = is an assignment operator and you did pd.DataFrame = {} which assigned some dictionary to the same spot that the DataFrame class constructor used to be. You should not use that line specified by #Ankur. It will break your stuff every time.
I would like to create a string of one million keys with 200 different values:
N = 1000000
uniques_keys = [pd.core.common.rands(3) for i in range(200)]
keys = [random.choice(uniques_keys) for i in range(N)]
However, I get the following error
In [250]:import pandas as pd
In [251]:pd.core.common.rands(3)
Traceback (most recent call last):
File "<ipython-input-251-31d12e0a07e7>", line 1, in <module>
pd.core.common.rands(3)
AttributeError: module 'pandas.core.common' has no attribute 'rands'
I use pandas version 0.18.0.
You can use:
In [14]: pd.util.testing.rands_array?
Signature: pd.util.testing.rands_array(nchars, size, dtype='O')
Docstring: Generate an array of byte strings.
Demo:
In [15]: N = 1000000
In [16]: s_arr = pd.util.testing.rands_array(10, N)
In [17]: s_arr
Out[17]: array(['L6d2GwhHdT', '5oki5T8VYm', 'XKUblAUFyL', ..., 'BE5AdCa62a', 'X3zDFKj6iy', 'iwASB9xZV3'], dtype=object)
In [18]: len(s_arr)
Out[18]: 1000000
UPDATE: from 2020-04-21
In newer Pandas versions you might see the following deprecation warning:
FutureWarning: pandas.util.testing is deprecated. Use the functions in
the public API at pandas.testing instead.
in this case import this function as follows:
from pandas._testing import rands_array
There are several solutions:
First solution:
The function rands appears to be in pandas.util.testing now:
pd.util.testing.rands(3)
Second solution:
Go straight for the underlying numpy implementation (as found in the pandas source code):
import string
RANDS_CHARS = np.array(list(string.ascii_letters + string.digits),
dtype=(np.str_, 1))
nchars = 3
''.join(np.random.choice(RANDS_CHARS, nchars))
Third solution:
Call numpy.random.bytes (check that it fulfils your requirements).
Fourth solution:
See this question for other suggestions.
I have produced some software that is processing data for analysis and plotting. For each type of data the data frames are produced in a module dedicated for the type.
Depending on the structure of the data the data frame columns could be normal or multindex.
I will pass the data frames to a procedure function that will produce plots of columns that are numeric.
I would like to be able to "attach" a string to each of the "printable" column with a string that will be used as plot labels. This string will not be the same as the name of the column.
I don't seem to be able to figure out a good way to do this purely with pandas DataFrame, so far I don't have any other solution either.
I have seen posts about metadata but I don't completely understand if this functionality is supported or not? At least I don't get this to work, especially it seems like using frames with MultiIndex columns complicates things.
If it is not supported is it still on the todo list?
From my reading I get the impression it have worked differently in different versions of pandas and even depend on if python 2 or 3 is used.
I would like to know if there is a convenient way to accomplish what I require with Pandas data frames? Is using _metadata for this advisable? If so how?
I have looked around quite a bit but especially the MultiIndex concern seems to not be addressed anywhere.
This one seem to indicate that metadata should be supported but is it for data frames? I need Series in a DataFrame.
Adding meta-information/metadata to pandas DataFrame
This one seem to be a similar question but I have tried the solution and it did not help, I tried the solution but it seems not to help me.
Propagate pandas series metadata through joins
Here is some experimentation I have done based on my understanding of the use of _metadata functionality. It seems to indicate that the _metadata did not make any difference and that the attribute did not persist a copy. Also it shows that using MultiIndex is an even more "unsupported" case.
Python 2.7.9 (default, Dec 10 2014, 12:24:55) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import pandas as pd
>>> from numpy.random import randn # To get values for the test frames
>>> import platform # To print python version
>>> # A function to set labels of the columns
>>> def labelSetter(aDF) :
... DFtmp = aDF.copy() # Just to ensure it is a different dataframe
... for column in DFtmp.columns :
... DFtmp[column].myLab='This is '+column.__str__()
... DFtmp[column].notMyLab='This should not persist'
... return DFtmp
...
>>>
>>> print 'Pandas version: {}'.format(pd.version.version)
Pandas version: 0.15.2
>>>
>>> pd.Series._metadata.append('myLab');print pd.Series._metadata # now _metadata contains 'myLab'
['name', 'myLab']
>>>
>>> # Make dataframes normal columns and MultiIndex
>>> dfS=pd.DataFrame(randn(2, 6),columns=['a1','a2','a3','b1','b2','c1']);print dfS
a1 a2 a3 b1 b2 c1
0 -0.934869 -0.310979 0.362635 -0.994605 -0.880114 -1.663265
1 0.205341 -1.642080 -0.732969 -0.080109 -0.082483 -0.208360
>>>
>>> dfMI=pd.DataFrame(randn(2, 6),columns=[['a','a','a','b','b','c'],['a1','a2','a3','b1','b2','c1']]);print dfMI
a b c
a1 a2 a3 b1 b2 c1
0 -0.578399 0.478925 1.047342 -0.087225 1.905074 0.146105
1 0.640575 0.153328 -1.117847 1.043026 0.671220 -0.218550
>>>
>>> # Run the labelSetter function on the data frames
>>> dfSWlab=labelSetter(dfS)
>>> dfMIWlab=labelSetter(dfMI)
>>>
>>> print dfSWlab['a2'].myLab
This is a2
>>> # This worked
>>>
>>> print dfSWlab['a2'].notMyLab
This should not persist
>>> # 'notMyLab' has not been appended to _metadata but the label still persists.
>>>
>>> dfSWlabCopy=dfSWlab.copy() # make a copy to see if myLab persists.
>>>
>>> dfSWlabCopy['a2'].myLab
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Python27\lib\site-packages\pandas\core\generic.py", line 1942, in __getattr__
return object.__getattribute__(self, name)
AttributeError: 'Series' object has no attribute 'myLab'
>>> # 'myLab' was appended to _metadata but still did not persist the copy
>>>
>>> print dfMIWlab['a']['a2'].myLab
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Python27\lib\site-packages\pandas\core\generic.py", line 1942, in __getattr__
return object.__getattribute__(self, name)
AttributeError: 'Series' object has no attribute 'myLab'
>>> # For the MultiIndex data frame the 'myLab' is not accessible
Hi I am getting the following error
'numpy.ndarray' object is not callable
when performing the calculation in the following manner
rolling_means = pd.rolling_mean(prices,20,min_periods=20)`
rolling_std = pd.rolling_std(prices, 20)`
#print rolling_means.head(20)
upper_band = rolling_means + (rolling_std)* 2
lower_band = rolling_means - (rolling_std)* 2
I am not sure how to resolve, can someone point me in right direction....
The error TypeError: 'numpy.ndarray' object is not callable means that you tried to call a numpy array as a function. We can reproduce the error like so in the repl:
In [16]: import numpy as np
In [17]: np.array([1,2,3])()
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
/home/user/<ipython-input-17-1abf8f3c8162> in <module>()
----> 1 np.array([1,2,3])()
TypeError: 'numpy.ndarray' object is not callable
If we are to assume that the error is indeed coming from the snippet of code that you posted (something that you should check,) then you must have reassigned either pd.rolling_mean or pd.rolling_std to a numpy array earlier in your code.
What I mean is something like this:
In [1]: import numpy as np
In [2]: import pandas as pd
In [3]: pd.rolling_mean(np.array([1,2,3]), 20, min_periods=5) # Works
Out[3]: array([ nan, nan, nan])
In [4]: pd.rolling_mean = np.array([1,2,3])
In [5]: pd.rolling_mean(np.array([1,2,3]), 20, min_periods=5) # Doesn't work anymore...
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
/home/user/<ipython-input-5-f528129299b9> in <module>()
----> 1 pd.rolling_mean(np.array([1,2,3]), 20, min_periods=5) # Doesn't work anymore...
TypeError: 'numpy.ndarray' object is not callable
So, basically you need to search the rest of your codebase for pd.rolling_mean = ... and/or pd.rolling_std = ... to see where you may have overwritten them.
Also, if you'd like, you can put in reload(pd) just before your snippet, which should make it run by restoring the value of pd to what you originally imported it as, but I still highly recommend that you try to find where you may have reassigned the given functions.
For everyone with this problem in 2021, sometimes you can have this problem when you create
a numpy variable with the same name as one of your function, what happens is that instead of calling the function python tries to call the numpy array as a function and you get the error, just change the name of the numpy variable
I met the same question and the solved.
The point is that my function parameters and variables have the same name.
Try to give them different name.