I build a conda env with jupyter notebook, tensorflow-gpu and other stuff in it from conda-forge, on ubuntu 21.10. I also did:
python -m ipykernel install --user --name=condaenvname
However when I select the env/kernel I get a conflict. After importing tensorflow as tf I get:
# what version of tensorflow is running in this env
print(tf.version.VERSION)
2.8.0
while when I do !conda list I get this:
tensorflow 2.6.2 cuda112py39h9333c2f_1 conda-forge
tensorflow-base 2.6.2 cuda112py39he9472f8_1 conda-forge
tensorflow-estimator 2.6.2 cuda112py39h9333c2f_1 conda-forge
tensorflow-gpu 2.6.2 cuda112py39h0bbbad9_1 conda-forge
How can this be? How can I get jupyter to use the correct tf version and all other packages in the env? I really don't want to rant because I know a lot of clever people are working hard on improving this, but I've read and worked on this for days, and can still not get a clear answer on how to get jupyter notebooks to call in the correct kernel and execute the correct package versions. Thx. J
I did finally find a solution to the problem. Jupyter/tensorflow/python actually go and find all packages on the system associated with that version of python, so it was finding packages in my own directories at
~/.local/lib/python3.9/site-packages
so erasing the site-packages directory and all of its contents, restarting the jupyter server solved all the problems. This is a very poor software setup, the whole basis of a conda env is to create a new environment, unpolluted and uninfluenced by other software packages on the base system. But apparently jupyter/python goes looking for all packages on the system, thus defeating the whole purpose of creating an environment in the first place!
Related
I recently installed python with the version 3.8.3 and upgraded pip to 20.1.1. According to enter link description here, conda install -c conda-forge tensorflow should work. However, I get this result
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Solving environment: -
Found conflicts! Looking for incompatible packages.
This can take several minutes. Press CTRL-C to abort.
failed
UnsatisfiableError: The following specifications were found
to be incompatible with the existing python installation in your environment:
Specifications:
- tensorflow -> python[version='3.5.*|3.6.*|>=3.5,<3.6.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0|3.7.*']
Your python: python=3.8
If python is on the left-most side of the chain, that's the version you've asked for.
When python appears to the right, that indicates that the thing on the left is somehow
not available for the python version you are constrained to. Note that conda will not
change your python version to a different minor version unless you explicitly specify
that.
since I use
(base) C:\Users\ivan>python --version
Python 3.8.3
(base) C:\Users\ivan>pip --version
pip 20.1.1 from C:\Users\ivan\anaconda3\lib\site-packages\pip (python 3.8)
I wonder if it is possible to solve this issue without downgrading. For users of anaconda 2020.07, python 3.8 is used by default. Downgrading it will break anaconda.
People have reported problems using tensorflow with python 3.8, it is best to use 3.7. You are incorrect about breaking Anaconda. Here is what to do.
In Anaconda home page click on environments. At the bottom left of the page click on create. A window will appear. Give the new environment a name (say python3.7). In the drop down menu select 3.7. Now a new environment is created using python 3.7. Now in the conda terminal type conda activate python3.7. Then use conda to install tensorflow. It will install version 2.1.1, the cuda toolkit version 10.1.243 and cudnn version 7.6.5. Note conda can only install tensorflow up to version 2.1.1. If you want tensorflow 2.2 install it with pip using pip install tensorflow ==2.2.0. after you have installed 2.1. The cuda toolkit and cudnn work with version 2.2. Now use pip or conda to install any other packages you need in your python3.7 environment and you should be good to go!
previously, I installed the tensorflow 1.13 in my machine.
There are some projects depending on different version of tensorflow and I do not want to mixed up different version of tensowflow.
So I just tried create a env called tf2.0 and used pip to install tensorflow 2.0.0b1 in that specific virtual environment.
However, after I ran 'pip install tensorflow-gpu==2.0.0b1` in that "tf2.0" conda environment, I found that it takes effect globally, which mean I have to use tensorflow-gpu 2.0.0b1 even when that virtual env "tf2.0" disactivated.
I wish I could use tensorflow 1.13 when virtual env is deactivated.
It's hard to troubleshoot the described conditions without more details (exact commands run, showing PATH before and after and post activation, etc.). Nevertheless, you can try switching to following the most recent recommendations for mixing Conda and Pip. Namely, avoid installing things ad hoc, which is prone to using the wrong pip and clobbering packages, but instead define a YAML file and always create the whole env in one go.
As a minimal example:
my_env.yaml
name: my_env
channels:
- defaults
dependencies:
- python
- pip
- pip:
- tensorflow-gpu==2.0.0b1
which can be created with conda env create -f my_env.yaml. Typically, it is best to include everything possible in the "non-pip" section of dependencies.
It is mostly that you used a wrong pip. To make sure you are using correct pip, it is usually a good practice to do
python -m pip install —user PACKAGE_NAME
Given that you have conda, pip should be the last resort.
Conda channel conda-forge most likely has the latest package version you are looking for.
conda install -c conda-forge PACKAGE_NAME
If you have to use pip, make sure you are in an environment and that environment has its own pip.
conda create -n test python=3.7
conda activate test
python -m pip install PACKAGE_NAME
From your described problem, I can guess that your environment is not activated in which you are trying to install the tensorflow2.0
Please make sure to activate the environment after making it.
so after creating the environment do this-
conda activate tf2.0
make sure you see this
(tf2.0) C:\Users\XYZ>
And then you install your tensorflow.
I installed Tensorflow for GPU using: pip install tensorflow-gpu
But when I tried the same for Keras pip install keras-gpu, it pulled me an error: could not find the version that satisfies the requirements.
Adding to the answer below which is the correct answer in terms of recommending to use Anaconda package manager, but out of date in that there is now a keras-gpu package on Anaconda Cloud.
So once you have Anaconda installed, you simply need to create a new environment where you want to install keras-gpu and execute the command:
conda install -c anaconda keras-gpu
This will install Keras along with both tensorflow and tensorflow-gpu libraries as the backend. (There is also no need to install separately the CUDA runtime and cudnn libraries as they are also included in the package - tested on Windows 10 and working).
There is not any keras-gpu package [UPDATE: now there is, see other answer above]; Keras is a wrapper around some backends, including Tensorflow, and these backends may come in different versions, such as tensorflow and tensorflow-gpu. But this does not hold for Keras itself, which should be installed simply with
pip install keras
independently of whatever backend is used (see the PyPi docs).
Additionally, and since you have tagged the question as anaconda, too, be informed that it is generally not advisable to mix your package managers (i.e pip with conda), and you may be better off installing Keras from the Anaconda cloud with
conda install -c conda-forge keras
Finally, you may be also interested to know that recent versions of Tensorflow include Keras as a subpackage, so you can use it without any additional installation; see https://www.tensorflow.org/guide/keras
For installing tensorflow-gpu from Anaconda cloud, you should use
conda install -c anaconda tensorflow-gpu
before installing Keras. Be sure you do it in a different virtual environment, or after having uninstalled other versions (i.e. pip-installed ones), as there have been reported problems otherwise.
Adding to the above two answers, ensure your TensorFlow/Keras environment is using Python 3.6. Keras/TensorFlow doesn't work very well with Python 3.7, as of May 10, 2019.
I tried to use Keras/TensorFlow with Python 3.7 and I ended up having to reinstall Anaconda, since it sort of broke my Anaconda Prompt.
To install tensorflow-gpu with particular cuda version 9.0, use:
conda install tensorflow-gpu cudatoolkit==9.0 -c anaconda
Similarly for keras-gpu
I created a conda env and installed tensorflow 1.4
pip install tensorflow==1.4
This installed, and from within the environment, when i type
import tensorflow as tf
tf.__version__
it prints '1.4.0'
Now, i open jupyter notebook from the same environment, and when i type the same in the notebook, it gives me 1.10.0
How is this possible? Jupyter notebook should take the same libraries within which it is running. How does it have a different version of tensorflow
And btw I'm concerned with this because i wanna use 1.4.0 in my notebook
These are the questions I ask myself in situations like this, and they usually resolve my issue:
Was the kernel of my notebook running while I pipped? Shut down and restart.
Are pip and conda sharing PATH, et cetera? I don't take it for granted that my pip is modifying my conda env. You can check with these commands.
Your conda env is created, but is it active? You list created envs with these, and you can activate/deactivate with these.
Besides checking all of these, where am I launching jupyter from? I like to install jupyter with a simply pip install jupyter and that could add jupyter to the command line globally. Anaconda navigator/command line reliably launches the jupyter that lives in your Anaconda distribution.
To add to Charles Landau:
You likely installed tensorflow in parallel with pip and with conda.
Within your env, just check with pip list tensorflow and conda list tensorflow. Most likely you will see two different versions.
To be consistent do:
pip unistall tensorflow to keep only one.
I have been trying to update tensorflow from 1.2.1 to 1.3.
I did the following on my terminal:
pip3 install tensorflow --upgrade
After this, I tried checking the version
python3 -c 'import tensorflow as tf; print(tf.__version__)'
This outputs 1.2.1 but not 1.3.0
All this had been done in an anaconda environment.
Operating system: macOS Sierra
I would like to know how one can perform this update.
Thank you
The solution that I found was to install pip inside the conda environment and use that instance of pip to install the packages. Please refer here
The reason I got the above error was that the python3 and pip3 paths were different.
This became evident when I ran the below commands inside the conda environment.
which python3
/Users/SMBP/anaconda/envs/tensorflow/bin/python3
which pip3
/usr/local/bin/pip3
Also, I think it is better to use virtualenv and virtualenvwrapper if you wish to work with tensorflow as it is the recommended way.