I am doing a project and I need to use both tensorflow and pandas together.
The primary issue i am having is with the numpy version. If I have the latest version of numpy then tensorflow throws an error:
"Cannot convert a symbolic Tensor (lstm/strided_slice:0) to a numpy array. This error may indicate that you're trying to pass a Tensor to a NumPy call, which is not supported"
If i downgrade numpy to 1.19.5 like the internet advises then importing pandas throws this error:
"numpy.ndarray size changed, may indicate binary incompatibility. expected 88 from c header, got 80 from pyobject"
Can someone please help?
Related
I first sub in Modin for Pandas for the benefit of distributed work over multiple cores:
import modin.pandas as pd
from modin.config import Engine
Engine.put("dask")
After initializing my dataframe, I attempt to use:
df['bins'] = pd.cut(df[column],300)
I get this error:
TypeError: ('Could not serialize object of type function.', '<function PandasDataframe._build_mapreduce_func.<locals>._map_reduce_func at 0x7fbe78580680>')
Would be glad to get help.
I can't seem to get Modin to perform the way that I want out of the box, the way I expected.
I use numpy broadcasting to get the differences matrix from a pandas dataframe. I find when dealing with large dataframe, it reports "'bool' object has no attribute 'sum'" error. While dealing with small dataframe, it runs fine.
I post the two csv files in the following links:
large file
small file
import numpy as np
import pandas as pd
df_small = pd.read_csv(r'test_small.csv',index_col='Key')
df_small.fillna(0,inplace=True)
a_small = df_small.to_numpy()
matrix = pd.DataFrame((a_small != a_small[:, None]).sum(2), index=df_small.index, columns=df_small.index)
print(matirx)
when running this, I could get the difference matrix.
when switch to large file, It reports the following error. Does anybody know why this happens?
EDIT:The numpy version is 1.19.5
np.__version__
'1.19.5'
I am a beginner trying to learn simple pandas&numpy and I am using Pycharm. Why doesn't pycharm show all of the objects in numpy library as I am typing code? For instance I can't find int64, int32, float32 and such number types. After importing numpy as np, when I start typing np.int, Pycharm acts as if int is available but as if int32 and int64 didn't exist at all.
Here is a screenshot what the code suggestion box looks like: 1
I have the newest numpy (1.18.0) installed both through commandline to the python interpreter itself and in Pycharm to this particular project. I can also confirm that for example np.int64 is valid code - it executes in Pycharm without error:
import numpy as np
print(np.int64(123)) # Executes correctly even though Pycharm shows as if there's no int64 in numpy library
I am using python 3.6 and numpy 1.13.3. On numpy import, I am getting an error Cannot import name 'dtype'
reticulate::import("numpy") throws
Error in py_run_string_impl(paste0("import sys; sys.path.append('", system.file("python", :
ImportError: cannot import name 'dtype'
Should I need to use some old version of numpy to fix this issue
The title says it all. I want to convert a PyTorch autograd.Variable to its equivalent numpy array. In their official documentation they advocated using a.numpy() to get the equivalent numpy array (for PyTorch tensor). But this gives me the following error:
Traceback (most recent call last): File "stdin", line 1, in module
File
"/home/bishwajit/anaconda3/lib/python3.6/site-packages/torch/autograd/variable.py",
line 63, in getattr raise AttributeError(name) AttributeError:
numpy
Is there any way I can circumvent this?
Two possible case
Using GPU: If you try to convert a cuda float-tensor directly to numpy like shown below,it will throw an error.
x.data.numpy()
RuntimeError: numpy conversion for FloatTensor is not supported
So, you cant covert a cuda float-tensor directly to numpy, instead you have to convert it into a cpu float-tensor first, and try converting into numpy, like shown below.
x.data.cpu().numpy()
Using CPU: Converting a CPU tensor is straight forward.
x.data.numpy()
I have found the way. Actually, I can first extract the Tensor data from the autograd.Variable by using a.data. Then the rest part is really simple. I just use a.data.numpy() to get the equivalent numpy array. Here's the steps:
a = a.data # a is now torch.Tensor
a = a.numpy() # a is now numpy array