Hey dear I am trying to launch spyder on my anaconda environment but I face this errors.
Application Spyder launch may have produced errors Mac M1 Chip
Traceback (most recent call last):
File "/Users/nawrozmohammadi/miniforge3/envs/tf24/bin/spyder", line 7, in
from spyder.app.start import main
File "/Users/nawrozmohammadi/miniforge3/envs/tf24/lib/python3.8/site-packages/spyder/app/start.py", line 35, in
from spyder.utils.external import lockfile
File "/Users/nawrozmohammadi/miniforge3/envs/tf24/lib/python3.8/site-packages/spyder/utils/external/lockfile.py", line 31, in
from spyder.utils.programs import is_spyder_process
File "/Users/nawrozmohammadi/miniforge3/envs/tf24/lib/python3.8/site-packages/spyder/utils/programs.py", line 30, in
import psutil
File "/Users/nawrozmohammadi/miniforge3/envs/tf24/lib/python3.8/site-packages/psutil/__init__.py", line 159, in
from . import _psosx as _psplatform
File "/Users/nawrozmohammadi/miniforge3/envs/tf24/lib/python3.8/site-packages/psutil/_psosx.py", line 15, in
from . import _psutil_osx as cext
ImportError: dlopen(/Users/nawrozmohammadi/miniforge3/envs/tf24/lib/python3.8/site-packages/psutil/_psutil_osx.cpython-38-darwin.so, 2): no suitable image found. Did find:
/Users/nawrozmohammadi/miniforge3/envs/tf24/lib/python3.8/site-packages/psutil/_psutil_osx.cpython-38-darwin.so: mach-o, but wrong architecture
/Users/nawrozmohammadi/miniforge3/envs/tf24/lib/python3.8/site-packages/psutil/_psutil_osx.cpython-38-darwin.so: mach-o, but wrong architecture
**Befor I launch spyder I installed Tensorflow through these two ways
first and second and Tensorflow working successfully **
>>>import tensorflow as tf
>>>tf.__version__
'2.4.0-rc0'
I have tried installing pyqt and did
conda update --all
.But nothing works
System info
conda version : 4.9.2
python version: 3.8.6
platform : maOS BigSur 64
Anaconda runs on x86 (intel) hence even i have faced many issues with the m1 macbook air. But miniconda is specifically tailored for arm64 chips.
hence try to install miniconda and use jupyternotebooks.
Follow the below guide to install tensorflow and Pycharm successfully:
Installation of TensorFlow & Pytorch on M1 Macbooks
I'm a newbie when it comes to AWS and Tensorflow and I've been learning about CNNs over the last week via Udacity's Machine Learning course.
Now I've a need to use an AWS instance of a GPU. I launched a p2.xlarge instance of Deep Learning AMI with Source Code (CUDA 8, Ubuntu) (that's what they recommended)
But now, it seems that tensorflow is not using the GPU at all. It's still training using the CPU. I did some searching and I found some answers to this problem and none of them seemed to work.
When I run the Jupyter notebook, it still uses the CPU
What do I do to get it to run on the GPU and not the CPU?
The problem of tensorflow not detecting GPU can possibly be due to one of the following reasons.
Only the tensorflow CPU version is installed in the system.
Both tensorflow CPU and GPU versions are installed in the system, but the Python environment is preferring CPU version over GPU version.
Before proceeding to solve the issue, we assume that the installed environment is an AWS Deep Learning AMI having CUDA 8.0 and tensorflow version 1.4.1 installed. This assumption is derived from the discussion in comments.
To solve the problem, we proceed as follows:
Check the installed version of tensorflow by executing the following command from the OS terminal.
pip freeze | grep tensorflow
If only the CPU version is installed, then remove it and install the GPU version by executing the following commands.
pip uninstall tensorflow
pip install tensorflow-gpu==1.4.1
If both CPU and GPU versions are installed, then remove both of them, and install the GPU version only.
pip uninstall tensorflow
pip uninstall tensorflow-gpu
pip install tensorflow-gpu==1.4.1
At this point, if all the dependencies of tensorflow are installed correctly, tensorflow GPU version should work fine. A common error at this stage (as encountered by OP) is the missing cuDNN library which can result in following error while importing tensorflow into a python module
ImportError: libcudnn.so.6: cannot open shared object file: No such
file or directory
It can be fixed by installing the correct version of NVIDIA's cuDNN library. Tensorflow version 1.4.1 depends upon cuDNN version 6.0 and CUDA 8, so we download the corresponding version from cuDNN archive page (Download Link). We have to login to the NVIDIA developer account to be able to download the file, therefore it is not possible to download it using command line tools such as wget or curl. A possible solution is to download the file on host system and use scp to copy it onto AWS.
Once copied to AWS, extract the file using the following command:
tar -xzvf cudnn-8.0-linux-x64-v6.0.tgz
The extracted directory should have structure similar to the CUDA toolkit installation directory. Assuming that CUDA toolkit is installed in the directory /usr/local/cuda, we can install cuDNN by copying the files from the downloaded archive into corresponding folders of CUDA Toolkit installation directory followed by linker update command ldconfig as follows:
cp cuda/include/* /usr/local/cuda/include
cp cuda/lib64/* /usr/local/cuda/lib64
ldconfig
After this, we should be able to import tensorflow GPU version into our python modules.
A few considerations:
If we are using Python3, pip should be replaced with pip3.
Depending upon user privileges, the commands pip, cp and ldconfig may require to be run as sudo.
On Windows, TensorFlow reports either or both of the following errors after executing an import tensorflow statement:
No module named "_pywrap_tensorflow"
DLL load failed.
The problem was the cuDNN Library for me - for whatever reason cudnn-8.0-windows10-x64-v6.0 was NOT working - I used cudnn-8.0-windows10-x64-v5.1 - ALL GOOD!
My setup working with Win10 64 and the Nvidia GTX780M:
Be sure you have the lib MSVCP140.DLL by checking your system/path - if not get it here
Run the windows installer for python 3.5.3-amd64 from here - DO NOT try newer versions as they probably won't work
Get the cuDNN v5.1 for CUDA 8.0 from here - put it under your users folder or in another known location (you will need this in your path)
Get CUDA 8.0 x86_64 from here
Set PATH vars as expected to point at the cuDNN libs and python (the python path should be added during the python install)
Make sure that ".DLL" is included in your PATHEXT variable
If you are using tensorflow 1.3 then you want to use cudnn64_6.dll github.com/tensorflow/tensorflow/issues/7705
If you run Windows 32 be sure to get the 32 bit versions of the files mentioned above.
In my case the "cudnn64_6.dll" file in the /bin folder had to be renamed to "cudnn64_5.dll" for the error to go away. I easily spent two hours to figure this out, and I followed the official install guide to the letter. This is true for installation via pip (officially supported) and conda (community supported).
Either error indicates that your system has not installed MSVCP140.DLL,
which TensorFlow requires.
To fix this error:
Determine whether MSVCP140.DLL is in your %PATH% variable.
If MSVCP140.DLL is not in your %PATH%, install the
Visual C++ 2015 redistributable (x64 version), which contains this DLL.
I have Win7 Pro 64-bit on AMD cpu, no gpu. I was following the instructions under "Installing with native pip" at https://www.tensorflow.org/install/install_windows. The installation step went ok but the attempt to import tensorflow produced the infamous:
ImportError: No module named '_pywrap_tensorflow_internal'
This seems to be one of those situations where a lot of unrelated things can go wrong, depending on configuration, which all cascade through to the same error.
In my case, installing MSVCP140.DLL was the answer.
You have MSVCP140.DLL already if
you have a file C:\Windows\System32\MSVCP140.DLL, AND
if you have a 64 bit system, then you additionally have C:\Windows\SysWOW64\MSVCP140.DLL.
I installed it manually, which was unnecessary (the redistributable is not the whole Visual C++ development mess and isn't large). Use the link posted earlier in this thread to install it: Visual C++ 2015 redistributable.
Also, I recommend that you override the default install directory for Python and put it anywhere not under C:\Program Files, because Windows tries to write-protect files there, which causes problems later.
For tensorflow with CPU only:
I had installed tensorflow using command:
pip3 install --upgrade tensorflow
This installed tensorflow 1.7
But could not import the tensorflow from withing python 3.6.5 amd64 using:
import tensorflow as tf
So, i downgraded the tensorflow version from 1.7 to 1.5 using following command:
pip3 install tensorflow==1.5
This uninstalled the previous version and installed 1.5. Now it works.
Seems that, my CPU does not support AVX instruction set that is needed in tensorflow 1.7
I had MSVCP140.DLL in the system folders and .DLL in the PATHEXT variable in Environment Variable.
TensorFlow requires MSVCP140.DLL, which may not be installed on your system.
To solve it open the terminal en type or paste this link:
C:\> pip install --upgrade https://storage.googleapis.com/tensorflow/windows/cpu/tensorflow-1.0.0-cp35-cp35m-win_amd64.whl
Note this is to install the CPU-only version of TensorFlow.
cuDNN causes my problem. PATH variable doesn't work for me. I have to copy the files in my cuDNN folders into respectful CUDA 8.0 folder structure.
For Those Running on Older Hardware:
You may get this same error due to having an older CPU using tensorflow-gpu 1.6.
If your cpu was made before 2011, then your max tensorflow-gpu version is 1.5.
Tensorflow 1.6 requires AVX instructions on your cpu. Verified here: Tensorflow Github docs
AVX enabled CPUs: Wiki AVX CPUs
What I did in my conda environment for tensorflow:
pip install --ignore-installed --upgrade tensorflow-gpu==1.5
I posted a general approach for troubleshooting the "DLL load failed" problem in this post on Windows systems. For reference:
Use the DLL dependency analyzer Dependencies to analyze <Your Python Dir>\Lib\site-packages\tensorflow\python\_pywrap_tensorflow_internal.pyd and determine the exact missing DLL (indicated by a ? beside the
DLL). The path of the .pyd file is based on the TensorFlow 1.9 GPU
version that I installed. I am not sure if the name and path is the
same in other TensorFlow versions.
Look for information of the missing DLL and install the appropriate package to resolve the problem.
The problem for me was the cuDNN library which didn't match the requirements of the graphics card. I downloaded the 6.0 version but for my GTX980ti but the recommended compute capability on the nvidia website was 5.1 ( http://developer.nvidia.com/cuda-gpus ) so I downloaded 5.1 and replaced the 6.0 version and as soon as I've done that it started working.
After much trial and error, and making sure VC++ 2015 Redistributable, cuDNN DLL and all other dependencies are accessible from PATH, it looks like Tensorflow GPU works only with Python 3.5.2 (as of this writing)
So if you're using Anaconda
conda create -n tensorflow-gpu python=3.5.2
activate tensorflow-gpu
pip install tensorflow-gpu
Then open the python interpreter and verify
>>> import tensorflow as tf
>>> sess = tf.Session(config=tf.ConfigProto(log_device_placement=True))
Found device 0 with properties:
name: GeForce 940M
major: 5 minor: 0
memoryClockRate (GHz) 1.176
pciBusID 0000:06:00.0
Total memory: 2.00GiB
Free memory: 1.66GiB
Credits: this neat guide
For each Tensorflow's version, it requires different version of CuDnn. On www.tensorflow.org, they did not mentioned about that in installation guide!
My case use tensorflow version 1.3 which uses cuDNN 6. https://github.com/tensorflow/tensorflow/releases.
Please check your tensorfow version and cuDNN version if they are match together.
And please set path environment for cuDNN, if it still does not work, please check the answer from #Chris Han.
One may be tempted to keep the Powershell/cmd open on Windows. I've spent reasonable time till I decided to close and reopen my Powershell only to realize that I've done everything right.
In case you are trying to install tensorflow GPU in Windows, you can find this easy interesting tutorial.
Note: If you are using PyCharm for example, you have to change the interpreter to the created conda environment.
For the people finding this post in 2019, this error could also occur because the Python version 3.7 does not have support for TensorFlow (see https://www.tensorflow.org/install/pip). So, check the Python version:
python --version
In case it is larger than 3.6, it should be downgraded to 3.6. For Anaconda:
conda install python=3.6
Then, install TensorFlow.
pip install tensorflow
Btw, I did not have the GPU version, so there were no CUDA related issues in my case.
Dll not found. Install Visual C++ 2015 redistributable to fix.
The problem was the cuDNN Library for me. I was able to run the test code after adding the directory (possibly bin folder) of the cuDNN DLL (not LIB file) in the Windows PATH.
For the reference, I installed TensorFlow from the source using PIP and my OS: Windows 7 and IDE: Visual Studio 2015.
With TensorFlow release 1.3.0, you need to use Cudnn 6.0 instead of Cudnn 5.0 as Cudnn 5.0 is giving this error. Don't forget to add path variable to Cudnn 6.0 .With cudnn64_6.dll your Tensorflow will work fine. Read the link below.
https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md#release-130
My two cents:
I had a ton of problems trying to get my CUDA 8.0 installed properly on Windows 7. I had a previous version installed and I wanted to upgrade so I uninstalled it and tried to install CUDA 8.0 (for tensorflow 1.3). The installation failed every single time, I tried to downgrade to CUDA 7.5 and was able to install it but had a ton of problems with tensorflow (similar to the PATH problem described here). Long story short: what worked for me was:
1) Uninstall EVERY NVIDIA component (except the display graphics driver)
2) Download CUDA toolkit 8.0 (and the patch) https://developer.nvidia.com/cuda-downloads
3) Check the CheckSum MD5 (I used MS https://www.microsoft.com/en-ca/download/confirmation.aspx?id=11533 but any would do) to make sure they were OK (it happened several times that the installer was not dowloaded properly because my WiFi router apparently).
4) Run the CUDA toolkit installer as root
5) download the cudnn 8.0 v6 and add its location to the PATH variable https://developer.nvidia.com/rdp/cudnn-download
Hope that helps and saves some headaches...
NOTE: This script helped me a lot to debug the problem! (Thank you mrry)
https://gist.github.com/mrry/ee5dbcfdd045fa48a27d56664411d41c
I will try to give the solution that worked for me. It seems that different set of problems can lead to this situation.
32 bit software works in 64 bit OS. I installed anaconda-3 (32 bit) in my 64 bit OS. It was working perfectly fine. I decided to install tensorflow in my machine and it wouldn't install at first. I was using conda environment to install tensorflow and got this error.
Solution is if you are running 64 bit OS, install 64 bit anaconda and if 32 bit OS then 32 bit anaconda. Then follow the standard procedure mentioned in tensorflow website for windows (anaconda installation). This made it possible to install tensorflow without any problem.
my answer is for windows 10 users only as I have tried the following on windows 10.
Extending some of the answers above I suggest this :
If you are using anaconda then you can avoid everything and simply install anaconda-navigator using the command
conda install -c anaconda anaconda-navigator
Then you can launch the navigator from command prompt using the command
anaconda-navigator
On running this command you get a simple gui where you can create an virtual environment, create the environment with python=3.5.2 and install module tensorflow-gpu or tensorflow by searching the module in the search box using gui, it will also take care of installing correct cuda files for you. Using anaconda navigator is the simplest solution.
If you are not using anaconda then take care about the following
tensorflow-gpu 1.3 requires python 3.5.2, cuda development kit 8.0 and cudaDNN 6.0, hence when installing make sure you run the command
pip install tensorflow-gpu==1.3
tensorflow-gpu 1.2.1 or less requires python 3.5.2, cuda development kit 8.0 and cudaDNN 5.1 hence when installing make sure you run the command
pip install tensorflow-gpu==1.2.1
Below are the steps you need to follow for both of the above processes
Setting up you path variables
You must have the following system variables
CUDA_HOME = "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0"
CUDA_PATH = "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0"
CUDA_PATH_V8.0 = "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0"
You PATHTEXT must include ".DLL" along with other extensions
".COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC;.PY;.DLL"
Also Add the following to you path
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0\lib\x64
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0\extras\CUPTI\libx64;
C:\Windows\SysWOW64;
C:\Windows\System32
If you are getting errors you can download the run the below code by mrry, this code will check your setup and tell you if something is wrong
https://gist.github.com/mrry/ee5dbcfdd045fa48a27d56664411d41c
References :
http://blog.nitishmutha.com/tensorflow/2017/01/22/TensorFlow-with-gpu-for-windows.html
The above reference is very useful.
Please comment for improvements to this answer.
Hope this helps, Thanks.
tensorflow 1.3 does not support cuda 9.0 yet.
I degrade to cuda 8.0, then it works.
Ran into the same problem (in 20190909) while investigating [SO]: Error while training using the estimator API in tensorflow.
Setup:
Win 10 pc064
Python 3.7.3 (pc064)
TensorFlow-GPU 1.13.1 ([TensorFlow]: Install TensorFlow with pip)
Error:
[cfati#CFATI-5510-0:e:\Work\Dev\StackOverflow\q057588589]> "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\Scripts\python.exe" -c "import tensorflow as tf"
Traceback (most recent call last):
File "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\tensorflow\python\pywrap_tensorflow.py", line 58, in <module>
from tensorflow.python.pywrap_tensorflow_internal import *
File "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\tensorflow\python\pywrap_tensorflow_internal.py", line 28, in <module>
_pywrap_tensorflow_internal = swig_import_helper()
File "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\tensorflow\python\pywrap_tensorflow_internal.py", line 24, in swig_import_helper
_mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname, description)
File "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\imp.py", line 242, in load_module
return load_dynamic(name, filename, file)
File "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\imp.py", line 342, in load_dynamic
return _load(spec)
ImportError: DLL load failed: The specified module could not be found.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\tensorflow\__init__.py", line 24, in <module>
from tensorflow.python import pywrap_tensorflow # pylint: disable=unused-import
File "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\tensorflow\python\__init__.py", line 49, in <module>
from tensorflow.python import pywrap_tensorflow
File "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\tensorflow\python\pywrap_tensorflow.py", line 74, in <module>
raise ImportError(msg)
ImportError: Traceback (most recent call last):
File "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\tensorflow\python\pywrap_tensorflow.py", line 58, in <module>
from tensorflow.python.pywrap_tensorflow_internal import *
File "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\tensorflow\python\pywrap_tensorflow_internal.py", line 28, in <module>
_pywrap_tensorflow_internal = swig_import_helper()
File "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\tensorflow\python\pywrap_tensorflow_internal.py", line 24, in swig_import_helper
_mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname, description)
File "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\imp.py", line 242, in load_module
return load_dynamic(name, filename, file)
File "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\imp.py", line 342, in load_dynamic
return _load(spec)
ImportError: DLL load failed: The specified module could not be found.
Failed to load the native TensorFlow runtime.
See https://www.tensorflow.org/install/errors
for some common reasons and solutions. Include the entire stack trace
above this error message when asking for help.
Looking at the "faulty" module (thanks to Dependency Walker), it turns out that it's not itself that's missing, but some of its dependencies (the cu*_100.dll files).
Check [SO]: Python Ctypes - loading dll throws OSError: [WinError 193] %1 is not a valid Win32 application (#CristiFati's answer) (the Conclusions section at the end, and (referenced) [SO]: Discover missing module using command-line ("DLL load failed" error) (#CristiFati's answer)) for more details on this kind of errors.
I had an older CUDA Toolkit version (8), and as a consequence, the cu*_80.dll files.
Upgrading to TensorFlow-GPU 1.14.0 ("e:\Work\Dev\VEnvs\py_064_03.07.03_test0\Scripts\python.exe" -m pip install --upgrade tensorflow-gpu), made the error a bit clearer (and also shorter):
[cfati#CFATI-5510-0:e:\Work\Dev\StackOverflow\q057588589]> "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\Scripts\python.exe" -c "import tensorflow as tf"
Traceback (most recent call last):
File "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\tensorflow\python\platform\self_check.py", line 75, in preload_check
ctypes.WinDLL(build_info.cudart_dll_name)
File "c:\install\x64\python\python\03.07.03\Lib\ctypes\__init__.py", line 356, in __init__
self._handle = _dlopen(self._name, mode)
OSError: [WinError 126] The specified module could not be found
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\tensorflow\__init__.py", line 28, in <module>
from tensorflow.python import pywrap_tensorflow # pylint: disable=unused-import
File "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\tensorflow\python\__init__.py", line 49, in <module>
from tensorflow.python import pywrap_tensorflow
File "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\tensorflow\python\pywrap_tensorflow.py", line 30, in <module>
self_check.preload_check()
File "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\lib\site-packages\tensorflow\python\platform\self_check.py", line 82, in preload_check
% (build_info.cudart_dll_name, build_info.cuda_version_number))
ImportError: Could not find 'cudart64_100.dll'. TensorFlow requires that this DLL be installed in a directory that is named in your %PATH% environment variable. Download and install CUDA 10.0 from this URL: https://developer.nvidia.com/cuda-90-download-archive
Steps:
Uninstall any CUDA Toolkit version (optional)
Install [nVidia.Developer]: CUDA Toolkit 10.0 Archive
Make sure to install v10.0 (that this TensorFlow-GPU version was built against - check [TensorFlow]: Build from source on Windows - GPU). I installed v10.1 (which was the latest, and also the recommended version at the answer time), and the .dll names didn't match (cu*_101.dll). Since I didn't want to install v10.0, I created some symlinks (with the "correct" names) to the existing files, and it worked. But bear in mind that this is unsupported!!! You may experience funny behavior (including crashes). This is a (lame) workaround (gainarie)
Additionally, a compatible (meaning that it's for a specific CUDA Toolkit version) cuDNN version ([nVidia.Developer]: cuDNN Archive) is required. In order to access the download URL, nVidia membership is required
After the above steps, and also setting the correct paths, it worked:
[cfati#CFATI-5510-0:e:\Work\Dev\StackOverflow\q057588589]> set PATH=%PATH%;%CUDA_PATH%\bin;f:\Install\x64\NVidia\GPU Computing Toolkit\cuDNN\7.6\bin
[cfati#CFATI-5510-0:e:\Work\Dev\StackOverflow\q057588589]> "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\Scripts\python.exe" -c "import tensorflow;print(\"Success!!!\")"
Success!!!