Now that Google Colab is not supporting tensorflow 1.x is there a way to install it through e.g. pip and set up your environment in a similar fashion to what the old %tensorflow_version 1.x did? I really really dont want to rewamp all my code to TF 2
It seems that only tf2 is supported by Colab, but that's not true, you still can use pip to uninstall tf2 and install a specific version of tf1. !yes|pip uninstall tensorflow, !pip install tensorflow==1.15.5 Maybe you should install other dependencies. So use !pip install -r requirements.txt Attention! You must restart the runtime in order to use newly installed versions.
Related
I am working on a jupyter notebook script which I used last year to train a neural network.
When I try to import the keras tokenizer:
from keras.preprocessing.text import Tokenizer
I receive this error
I have seen other posts which suggest that I need to update tensorflow. My anaconda environment tells me I have 1.13.1 installed. But when I try to update tensorflow-base to 1.15 in the anaconda navigator, I receive this error:
I can update tensorflow from my command line using:
conda install tensorflow=1.15.0
But this doesn't update tensorflow in my anaconda environment and the error persists in my notebook.
Any help would be much appreciated! As you can probably tell, I am a novice python user.
The error says some packages needed to update Tensorflow/Andaconda requires Python 3.11 or newer. Since not all of the error log can be seen however, I would upgrade to python 3.7 to be safe. You can download this from the official page: https://www.python.org/downloads/
If the problem persists, try using pip to update the packages(In bash) :
pip install tensorflow
pip install conda
If you get an error while using pip, try:
pip3 install tensorflow
pip3 install conda
This same method can be used to update keras:
pip install keras
or if that does not work:
pip3 install keras
If pip is not recognized at a command, Python 3.7 is not added to path. I do not have experience with macOS, but this article should go into enough depth.
https://realpython.com/add-python-to-path/#how-to-add-python-to-path-on-linux-and-macos
Last night, I made a virtual environment and the only installs I made were as follows:
conda install jupyter
conda install notebook
pip install transformers
And I was able to run huggingface transformers perfectly, but today I reactivate my conda virtual environment and I'm met with this puzzling error:
And this is the runtime error
I even tried making a new virtual environment with the same depencies and today I got these errors upon performing pip install transformers
And this in jupyter notebook
HuggingFace Transformers need to be installed in conjunction to either Tensorflow or Pytorch. From the HuggingFace installation instructions:
First you need to install one of, or both, TensorFlow 2.0 and PyTorch.
So you should choose which machine learning platform you like more and install that too in your virtual environment.
For Tensorflow either pip install tensorflow or pip install tensorflow-gpu and for Pytorch pip install torch torchvision
For example, in a fresh environment, the following instructions should make your errors disappear (I'm assuming you have a GPU):
conda install jupyter
conda install notebook
pip install tensorflow-gpu
pip install transformers
I installed Tensorflow for GPU using: pip install tensorflow-gpu
But when I tried the same for Keras pip install keras-gpu, it pulled me an error: could not find the version that satisfies the requirements.
Adding to the answer below which is the correct answer in terms of recommending to use Anaconda package manager, but out of date in that there is now a keras-gpu package on Anaconda Cloud.
So once you have Anaconda installed, you simply need to create a new environment where you want to install keras-gpu and execute the command:
conda install -c anaconda keras-gpu
This will install Keras along with both tensorflow and tensorflow-gpu libraries as the backend. (There is also no need to install separately the CUDA runtime and cudnn libraries as they are also included in the package - tested on Windows 10 and working).
There is not any keras-gpu package [UPDATE: now there is, see other answer above]; Keras is a wrapper around some backends, including Tensorflow, and these backends may come in different versions, such as tensorflow and tensorflow-gpu. But this does not hold for Keras itself, which should be installed simply with
pip install keras
independently of whatever backend is used (see the PyPi docs).
Additionally, and since you have tagged the question as anaconda, too, be informed that it is generally not advisable to mix your package managers (i.e pip with conda), and you may be better off installing Keras from the Anaconda cloud with
conda install -c conda-forge keras
Finally, you may be also interested to know that recent versions of Tensorflow include Keras as a subpackage, so you can use it without any additional installation; see https://www.tensorflow.org/guide/keras
For installing tensorflow-gpu from Anaconda cloud, you should use
conda install -c anaconda tensorflow-gpu
before installing Keras. Be sure you do it in a different virtual environment, or after having uninstalled other versions (i.e. pip-installed ones), as there have been reported problems otherwise.
Adding to the above two answers, ensure your TensorFlow/Keras environment is using Python 3.6. Keras/TensorFlow doesn't work very well with Python 3.7, as of May 10, 2019.
I tried to use Keras/TensorFlow with Python 3.7 and I ended up having to reinstall Anaconda, since it sort of broke my Anaconda Prompt.
To install tensorflow-gpu with particular cuda version 9.0, use:
conda install tensorflow-gpu cudatoolkit==9.0 -c anaconda
Similarly for keras-gpu
I am using pip3 install tensorflow==1.8.0, but it doesn't have GPU support.
So I am using pip3 install tensorflow-gpu==1.8.0, but it still raises an exception
libcudart.so.VERSION No such file.
Should I use colab to install tensorflow from source?
After pip3 list:
tensorboard 1.10.0
tensorflow 1.10.0
tensorflow-hub 0.1.1
Google recommends you not to do pip installs!!!!
use this instead: %tensorflow_version 1.x
Restart the Runtime and check if its changed:
import tensorflow
print(tensorflow.__version__)
Here is a link to the main article:
https://colab.research.google.com/notebooks/tensorflow_version.ipynb#scrollTo=8UvRkm1JGUrk
You can downgrade Tensorflow to a previous version without GPU support on Google Colab. I ran:
!pip install tensorflow==1.14.0
import tensorflow as tf
print(tf.__version__)
which initially returned
2.0.0-dev20190130
but when I returned to it after a few hours, I got the version I requested:
1.14.0
Trying to downgrade to a version with GPU support:
!pip install tensorflow-gpu==1.14.0
requires restarting the runtime and fails, as importing import tensorflow as tf returns:
ImportError: libcublas.so.9.0: cannot open shared object file: No such file or directory
Update
When the import fails you can always downgrade CUDA to version 9.0 using following commands
!wget https://developer.nvidia.com/compute/cuda/9.0/Prod/local_installers/cuda-repo-ubuntu1604-9-0-local_9.0.176-1_amd64-deb
!dpkg -i cuda-repo-ubuntu1604-9-0-local_9.0.176-1_amd64-deb
!apt-key add /var/cuda-repo-9-0-local/7fa2af80.pub
!apt-get update
!apt-get install cuda=9.0.176-1
You can check the version of CUDA by running:
!nvcc --version
Second update
This code now seems to fail, see the follow-up question at How to downgrade to tensorflow-gpu version 1.12 in google colab
Google gives quite a simple solution to downgrade to the previously used Colab tf v.1.15.2. Just run the following magic line in Colab:
%tensorflow_version 1.x
Ther recommend "against using pip install to specify a particular TensorFlow version for both GPU and TPU backends. Colab builds TensorFlow from the source to ensure compatibility with our fleet of accelerators. Versions of TensorFlow fetched from PyPI by pip may suffer from performance problems or may not work at all". This means if you need GPU support, use one of the two given TF versions. The other versions will not necessary work I guess even for CPU.
The build process for GPU-enabled tensorflow is involved. In particular, old versions of TensorFlow use (or require) older versions of CUDA, which itself depends on system libraries and configuration beyond the scope of a pip install.
I suspect that downgrading TensorFlow on a VM configured for a newer version is going to be an involved process, perhaps involving downgrades / reinstalls of system libraries.
If it's practical, it might be simpler to update your code to use the latest version of TensorFlow, at least until Colab supports persistent backend enivronments.
It seems that only tensorflow 2 is supported by Colab, but that's not true, you still can use pip to uninstall tensorflow 2 and install a specific version of tf1. !yes|pip uninstall tensorflow, !pip install tensorflow==1.15.5 Maybe you should install other dependencies. So use !pip install -r requirements.txt Attention! You must restart the runtime in order to use newly installed versions.
%tensorflow_version 1.x no longer works.
%tensorflow_version 1.x
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-2-8d2919c1d33c> in <module>
----> 1 get_ipython().run_line_magic('tensorflow_version', '1.x')
1 frames
/usr/local/lib/python3.8/dist-packages/google/colab/_tensorflow_magics.py in _tensorflow_version(line)
33
34 if line.startswith("1"):
---> 35 raise ValueError(
36 # pylint: disable=line-too-long
37 textwrap.dedent("""\
ValueError: Tensorflow 1 is unsupported in Colab.
Your notebook should be updated to use Tensorflow 2.
See the guide at https://www.tensorflow.org/guide/migrate#migrate-from-tensorflow-1x-to-tensorflow-2.
I have been trying to update tensorflow from 1.2.1 to 1.3.
I did the following on my terminal:
pip3 install tensorflow --upgrade
After this, I tried checking the version
python3 -c 'import tensorflow as tf; print(tf.__version__)'
This outputs 1.2.1 but not 1.3.0
All this had been done in an anaconda environment.
Operating system: macOS Sierra
I would like to know how one can perform this update.
Thank you
The solution that I found was to install pip inside the conda environment and use that instance of pip to install the packages. Please refer here
The reason I got the above error was that the python3 and pip3 paths were different.
This became evident when I ran the below commands inside the conda environment.
which python3
/Users/SMBP/anaconda/envs/tensorflow/bin/python3
which pip3
/usr/local/bin/pip3
Also, I think it is better to use virtualenv and virtualenvwrapper if you wish to work with tensorflow as it is the recommended way.