On my Mac M1 (Monterey) I have created a new virtual environment with
virtualenv --python=/opt/homebrew/bin/python3.8 ~/.virtualenvs/datascience_env
to link it with specific Python 3.8 version. Now I'm trying to install tensorflow. The following command (issued in the virtual environment):
(datascience_env)% pip install tensorflow
causes the following error:
ERROR: Could not find a version that satisfies the requirement tensorflow (from versions: none)
ERROR: No matching distribution found for tensorflow
Just to be sure, I also did:
(datascience_env)% python -c "import sys; print(sys.version)" or python -c "import struct; print(struct.calcsize('P')*8)"
and the output confirms the 3.8 version:
3.8.14 (default, Sep 6 2022, 23:17:06)
[Clang 13.1.6 (clang-1316.0.21.2.5)]
I've finally opted for a simpler solution. I followed here the procedure to install Tensorflow on Mac M1 with miniforge. Works like a charm. In summary the steps are install miniforge, then:
conda config --set auto_activate_base false
conda create --name mytfenv python=3.8
conda activate mytfenv
conda install -c apple tensorflow-deps
pip install tensorflow-macos
pip install tensorflow-metal
conda install -c conda-forge -y pandas jupyter
I am facing 4 problems when I tried to install TensorFlow on Apple M1:
Conda has supported M1 since 2022.05.06 but most of articles I googled talk about using Miniforge, e.g. So I feel they are all kind of outdated.
How To Install TensorFlow on M1 Mac (The Easy Way)
AI - Apple Silicon Mac M1 natively supports TensorFlow 2.8 GPU acceleration
How to Setup TensorFlow on Apple M1 Pro and M1 Max (works for M1 too)
How To Install TensorFlow 2.7 on MacBook Pro M1 Pro With Ease
I used the latest conda 4.13 to setup my python environment(3.8, 3.9 and 3.10) successfully but when I tried to install tensorflow I got the error "No matching distribution found for tensorflow" (all failed).
ERROR: Could not find a version that satisfies the requirement tensorflow (from versions: none)
ERROR: No matching distribution found for tensorflow
The answers in Could not find a version that satisfies the requirement tensorflow didn't help. I can't find useful information on https://www.tensorflow.org/ too, actually https://www.tensorflow.org/install just said pip install tensorflow.
I tried to run pip install tensorflow-macos and it succeeded.
I read from the above "works for M1 too" article mentioned "Apple's fork of TensorFlow is called tensorflow-macos" although I can't find much information about that. For example, https://www.tensorflow.org/ does not mention that. I also found from https://developer.apple.com/forums/thread/686926 that someone hit that "ERROR: No matching distribution found for tensorflow-macos" (but I didn't).
All the articles I googled, including above 4 articles and this Tensorflow on macOS Apple M1, all say I also need to run the following 2 commands
conda install -c apple tensorflow-deps
pip install tensorflow-metal
But do I really need to that? I can't find this information from https://www.tensorflow.org/.
What are these 2 packages tensorflow-deps and tensorflow-metal ?
Distilling the official directions from Apple (as of 13 July 2022), one would create an environment using the following YAML:
tf-metal-arm64.yaml
name: tf-metal
channels:
- apple
- conda-forge
dependencies:
- python=3.9 ## specify desired version
- pip
- tensorflow-deps
## uncomment for use with Jupyter
## - ipykernel
## PyPI packages
- pip:
- tensorflow-macos
- tensorflow-metal ## optional, but recommended
Edit to include additional packages.
Creating environment
Before creating the environment we need to know what the base architecture is. Check this with conda config --show subdir.
Native (osx-arm64) base
If you have installed a native osx-arm64 Miniforge variant (I recommend Mambaforge), then you can create with:
mamba env create -n my_tf_env -f tf-metal-arm64.yaml
Note: If you don't have Mamba, then substitute conda for mamba; or install it for much faster solving: conda install -n base mamba.
Emulated (osx-64) base
If you do not have a native base, then you will need to override the subdir setting:
## create env
CONDA_SUBDIR=osx-arm64 mamba env create -n my_tf_env -f tf-metal-arm64.yaml
## activate
mamba activate my_tf_env
## permanently set the subdir
conda config --env --set subdir osx-arm64
Be sure to always activate the environment before installing or updating packages.
I battled with this for hours. The current instructions at https://developer.apple.com/metal/tensorflow-plugin/ specify using Miniconda and can be summarized as:
conda create -y --name cv python
conda activate cv
conda install -y -c apple tensorflow-deps
python -m pip install tensorflow-macos tensorflow-metal
As of Jan 2023, these instructions are riddled with issues:
Symptom: you ran conda install -c apple tensorflow-deps expecting to get the current version (2.10.0) , but conda list tensorflow-deps shows tensorflow-deps 2.9.0
Reason: Apple's tensorflow-deps package v2.10.0 depends on numpy >=1.23.2,<1.23.3. There is no such version of numpy in Anaconda (only conda-forge). Anaconda's dependency resolution silently falls back to an older version of tensorflow-deps. This will cause more problems as you continue with the instructions.
Symptom: you ran conda install -c apple tensorflow-deps==2.10.0 and got UnsatisfiableError: The following specifications were found to be incompatible with each other
Reason: Same as above, but at least Anaconda has told you about it. It doesn't mention what the incompatibility is, because that would be helpful.
Symptom: "RuntimeError: module compiled against API version 0x10 but this version of numpy is 0xf"
Reason: If you install tensorflow-deps 2.9.0 (or Anaconda installs it for you due to the above) you will get numpy 1.22.3. When you pip install tensorflow-macos tensorflow-metal, you will get tensorflow-macos 2.11.0 and tensorflow-metal 0.7.0, one or both of which is binary-incompatible with numpy 1.22.3.
Symptom: 2023-01-22 15:16:23.209301: W tensorflow/core/framework/op_kernel.cc:1830] OP_REQUIRES failed at xla_ops.cc:418 : NOT_FOUND: could not find registered platform with id
Problem: TensorFlow 2.11 has introduced an incompatibility between optimizers and pluggable architectures. You can have TF 2.11, but you'll need to use a legacy optimizer.
Solutions
I have found 3 options for a working GPU-accelerated TF install on Apple Silicon using Anaconda.
It will help your debugging to get a minimal test script like the one from https://developer.apple.com/metal/tensorflow-plugin/ which is:
import tensorflow as tf
cifar = tf.keras.datasets.cifar100
(x_train, y_train), (x_test, y_test) = cifar.load_data()
model = tf.keras.applications.ResNet50(
include_top=True,
weights=None,
input_shape=(32, 32, 3),
classes=100,)
loss_fn = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
model.compile(optimizer="adam", loss=loss_fn, metrics=["accuracy"])
model.fit(x_train, y_train, epochs=5, batch_size=64)
This way you can check GPU usage by running TF_MLC_LOGGING=1 python tf_arch_test.py and watching Activity Monitor. While feeling which side of the laptop is burning your legs can be a helpful indicator of GPU usage, it's not always reliable.
Fix #1: Add conda-forge as a channel, use the legacy optimizer in your code.
conda config --add channels conda-forge
conda config --set channel_priority strict
conda create -y --name cv
conda activate cv
conda install -y -c apple tensorflow-deps
python -m pip install tensorflow-macos
python -m pip install tensorflow-metal
python --version
conda list|grep -E '(tensorflow|numpy)'
TF_MLC_LOGGING=1 python tf_arch_test.py
You will get:
Python 3.10.8
numpy 1.23.2 py310h127c7cf_0 conda-forge
tensorflow-deps 2.10.0 0 apple
tensorflow-estimator 2.11.0 pypi_0 pypi
tensorflow-macos 2.11.0 pypi_0 pypi
tensorflow-metal 0.7.0 pypi_0 pypi
You will need to modify your code to use one of the legacy optimizers. e.g.:
model.compile(
optimizer=tf.keras.optimizers.legacy.Adam(learning_rate=1e-3),
loss=loss_fn,
metrics=["accuracy"],
)
Fix #2: Add conda-forge as a channel, pin the version numbers to avoid the pluggable architecture issue.
conda config --add channels conda-forge
conda config --set channel_priority strict
conda create -y --name cv
conda activate cv
conda install -y -c apple tensorflow-deps==2.10.0
python -m pip install tensorflow-macos==2.10.0
python -m pip install tensorflow-metal==0.6.0
python --version
conda list|grep -E '(tensorflow|numpy)'
You will get:
Python 3.10.8
numpy 1.23.2 py310h127c7cf_0 conda-forge
tensorflow-deps 2.10.0 0 apple
tensorflow-estimator 2.10.0 pypi_0 pypi
tensorflow-macos 2.10.0 pypi_0 pypi
tensorflow-metal 0.6.0 pypi_0 pypi
Fix #3: Don't add conda-forge, pin the version numbers way back to the last ones that actually worked:
conda create -y --name cv python
conda activate cv
conda install -y -c apple tensorflow-deps==2.9.0
python -m pip install tensorflow-macos==2.9.2
python -m pip install tensorflow-metal==0.5.1
python --version
conda list|grep -E '(tensorflow|numpy)'
You will get:
Python 3.10.9
numpy 1.22.3 py310hdb36b11_0
numpy-base 1.22.3 py310h5e3e9f0_0
tensorflow-deps 2.9.0 0 apple
tensorflow-estimator 2.9.0 pypi_0 pypi
tensorflow-macos 2.9.2 pypi_0 pypi
tensorflow-metal 0.5.1 pypi_0 pypi
Download and install Conda env:
https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-MacOSX-arm64.sh
chmod +x ~/Downloads/Miniforge3-MacOSX-arm64.sh
sh ~/Downloads/Miniforge3-MacOSX-arm64.sh
source ~/miniforge3/bin/activate
Install the TensorFlow dependencies:
conda install -c apple tensorflow-deps
Install base TensorFlow:
python -m pip install tensorflow-macos
Install base TensorFlow-metal:
python -m pip install tensorflow-metal
Create Conda Environment:
conda create -n tensorflow-env tensorflow
conda activate tensorflow-env
First of all, TensorFlow does not support officially the Mac M1. They don't distribute packages precompiled for the Mac M1 (and its specific arm64 arch), hence the tensorflow-macos package, which is maintained by Apple. TensorFlow distributes, as far as I know, official wheels only for x86 (Linux, Windows, Mac), and the Raspberry PI (arm64).
Apple is using a specific plugin in Tensorflow to make the framework compatible with Metal, the graphic stack of MacOS. To put it in a other way, they are leveraging the PluggableDevice API of Tensorflow to write code that translates the TensorFlow operations to code that the GPU of the M1 understands.
Those two packages contain respectively:
tensorflow-deps the dependencies to run Tensorflow on arm64, i.e python, numpy, grpcio and h5py. This is more of a convenience package, I believe.
tensorflow-metal: a plugin to make tensorflow able to run on metal, the shader API of MacOS (comparable to the low level APIs of Vulkan or DirectX12 on other platforms). You can think of it as a replacement of CUDA, if you are used to run TensorFlow on Nvidia GPUs.
Without the tensorflow-metal package, TensorFlow won't be able to leverage the GPU of the M1, but will still be able to run code on the CPU.
The two answers I got have helped better understand how to install TensorFlow on m1. But I would like share my experience too.
About tensorflow-deps. I do need it, without it pip failed to installed grpcio, and thus actually failed to install tensorflow-macos. When I first asked the question I didn't pay enough attention to output of pip install tensorflow-macos.
About tensorflow-macos package, actually https://blog.tensorflow.org/2020/11/accelerating-tensorflow-performance-on-mac.html has the full information. BTW that article, published 2020-11-18, said "In the near future, we’ll be making updates like this even easier for users to get these performance numbers by integrating the forked version into the TensorFlow master branch." But according to Lescurel's answer it seems they have not.
I didn't know the concept of PluggableDevice (as in Lescurel's), so even when I visited https://github.com/apple/tensorflow_macos I was still confused. Take a look at that article if you do not know that either, basically it will let TensorFlow support new devices.
For 4 articles I listed, "works for M1 too" is the most helpful. It actually explained why I need tensorflow-deps & tensorflow-metal. But part of reasons I did not pay enough attention beforehand were: a) I want to use conda, not miniforge, all these package manager tools scare me a little bit (come from nodejs background, npm, yarn, yarn2, pnmp). The answer from merv also suggested another one mamba, but I think I will pass. b) I don't use homebrew, basically all the articles talking about installing ts on m1 mentioned installing homebrew first. But I use macport, for the reason I mentioned here (again I am bit scared of these these package manager tools)
Using environment.yaml like the one in merv's answer is a reliable way to install tensorflow!
BTW, once I have figured out the whole process of installing tensorflow, install pytorch is a lot easier as pytorch also supports M1 now, check here https://pytorch.org/blog/introducing-accelerated-pytorch-training-on-mac/
Official instructions from Apple are available here.
At the time of writing:
conda create python=3.10.6 --name <NAME>
conda activate <NAME>
conda install -c apple tensorflow-deps
python -m pip install tensorflow-macos
python -m pip install tensorflow-metal
I installed a working version of tensorflow 2.9.2 on my M1 Mac with pip. First, install pyenv and python 3.10.6. Next, pip install tensorflow-metal and finally pip install tensorflow-macos. That's it, no need for tensorflow-deps.
If your model complains about the unavailability of cuDNN and runs slowly, try adjusting your script to enable cuDNN as per tensorflow docs
Worked for me using Python 3.10.8 via Homebrew and following the instructions from Apple but using the instructions for "x86: AMD" instead.
Check Python version:
% which python3.10
/opt/homebrew/bin/python3.10
Create venv, activate it (prompt will change), and update pip:
% python3.10 -m venv ~/py310-tf-metal
% source ~/py310-tf-metal/bin/activate
(py310-tf-metal) % which python
~/py310-tf-metal/bin/python
(py310-tf-metal) % python -m pip install -U pip
...
Successfully installed pip-22.3.1
Install tensorflow-macos:
(py310-tf-metal) % python -m pip install tensorflow-macos
...
Successfully installed MarkupSafe-2.1.1 absl-py-1.3.0 astunparse-1.6.3 cachetools-5.2.0 certifi-2022.9.24 charset-normalizer-2.1.1 flatbuffers-22.11.23 gast-0.4.0 google-auth-2.14.1 google-auth-oauthlib-0.4.6 google-pasta-0.2.0 grpcio-1.50.0 h5py-3.7.0 idna-3.4 keras-2.10.0 keras-preprocessing-1.1.2 libclang-14.0.6 markdown-3.4.1 numpy-1.23.5 oauthlib-3.2.2 opt-einsum-3.3.0 packaging-21.3 protobuf-3.19.6 pyasn1-0.4.8 pyasn1-modules-0.2.8 pyparsing-3.0.9 requests-2.28.1 requests-oauthlib-1.3.1 rsa-4.9 six-1.16.0 tensorboard-2.10.1 tensorboard-data-server-0.6.1 tensorboard-plugin-wit-1.8.1 tensorflow-estimator-2.10.0 tensorflow-macos-2.10.0 termcolor-2.1.1 typing-extensions-4.4.0 urllib3-1.26.13 werkzeug-2.2.2 wheel-0.38.4 wrapt-1.14.1
Install tensorflow-metal:
(py310-tf-metal) % python -m pip install tensorflow-metal
Collecting tensorflow-metal
Downloading tensorflow_metal-0.6.0-cp310-cp310-macosx_12_0_arm64.whl (1.4 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.4/1.4 MB 25.5 MB/s eta 0:00:00
Requirement already satisfied: six>=1.15.0 in ./Venvs/py310-tf-metal/lib/python3.10/site-packages (from tensorflow-metal) (1.16.0)
Requirement already satisfied: wheel~=0.35 in ./Venvs/py310-tf-metal/lib/python3.10/site-packages (from tensorflow-metal) (0.38.4)
Installing collected packages: tensorflow-metal
Successfully installed tensorflow-metal-0.6.0
Test using the CIFAR training script at the Apple page:
import tensorflow as tf
cifar = tf.keras.datasets.cifar100
(x_train, y_train), (x_test, y_test) = cifar.load_data()
model = tf.keras.applications.ResNet50(
include_top=True,
weights=None,
input_shape=(32, 32, 3),
classes=100,)
loss_fn = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
model.compile(optimizer="adam", loss=loss_fn, metrics=["accuracy"])
model.fit(x_train, y_train, epochs=5, batch_size=64)
Save above as testcifar.py and run it:
(py310-tf-metal) % python testcifar.py
Downloading data from https://www.cs.toronto.edu/~kriz/cifar-100-python.tar.gz
169001437/169001437 [==============================] - 3s 0us/step
Metal device set to: Apple M1
systemMemory: 16.00 GB
maxCacheSize: 5.33 GB
2022-11-28 07:58:10.715660: I tensorflow/core/common_runtime/pluggable_device/pluggable_device_factory.cc:306] Could not identify NUMA node of platform GPU ID 0, defaulting to 0. Your kernel may not have been built with NUMA support.
2022-11-28 07:58:10.715837: I tensorflow/core/common_runtime/pluggable_device/pluggable_device_factory.cc:272] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 0 MB memory) -> physical PluggableDevice (device: 0, name: METAL, pci bus id: <undefined>)
2022-11-28 07:58:14.736843: W tensorflow/core/platform/profile_utils/cpu_utils.cc:128] Failed to get CPU frequency: 0 Hz
Epoch 1/5
...
2022-11-28 07:58:21.975675: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:114] Plugin optimizer for device_type GPU is enabled.
...
Epoch 5/5
782/782 [==============================] - 206s 264ms/step - loss: 4.0877 - accuracy: 0.1292
I had to downgrade tensorflow to get it to work on Macbook Pro M2:
pip install tensorflow-macos==2.9
pip install tensorflow-metal==0.5.0
I have been trying to install TensorFlow on my Macbook Air with a M1 chip.
Using Python 3.9.7.
Originally was on MacOS 11, but subsequently upgraded to 12.01
At first, I tried these instructions [https://towardsdatascience.com/installing-tensorflow-on-the-m1-mac-410bb36b776] but got stuck when trying to execute
pip3 install --upgrade --force --no-dependencies https://github.com/apple/tensorflow_macos/releases/download/v0.1alpha3/tensorflow_addons_macos-0.1a3-cp38-cp38-macosx_11_0_arm64.whl https://github.com/apple/tensorflow_macos/releases/download/v0.1alpha3/tensorflow_macos-0.1a3-cp38-cp38-macosx_11_0_arm64.whl
ERROR: tensorflow_addons_macos-0.1a3-cp38-cp38-macosx_11_0_arm64.whl is not a supported wheel on this platform.
So I tried to follow these instructions [https://www.tensorflow.org/install/source#macos_1] to compile TensorFlow, but when I try
bazel build //tensorflow/tools/pip_package:build_pip_package
I get these errors:
ERROR: /Users/scottbrown/tensorflow/tensorflow/lite/python/BUILD:62:10: Target '//tensorflow/lite/python:tflite_convert' depends on toolchain '#local_config_cc//:cc-compiler-darwin', which cannot be found: error loading package '#local_config_cc//': cannot load '#local_config_cc_toolchains//:osx_archs.bzl': no such file'
ERROR: Analysis of target '//tensorflow/tools/pip_package:build_pip_package' failed; build aborted: Analysis failed
When I try
pip3 install tensorflow-macos
I get this error:
Building wheel for h5py (pyproject.toml) ... error
ERROR: Command errored out with exit status 1:
command: /opt/homebrew/opt/python#3.9/bin/python3.9 /opt/homebrew/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py build_wheel /var/folders/gz/28jpdfcd3b3g4pm7zl0wmrkh0000gn/T/tmpz_m057zj
cwd: /private/var/folders/gz/28jpdfcd3b3g4pm7zl0wmrkh0000gn/T/pip-install-kz29fkw2/h5py_0747e63c821445b6944ecb4fc6b2d1e1
I'm basing my answer on the article from Prabhat Kumar Sahu:
How to install Tensorflow on M1 Mac the easy way
Set up environment
Make sure you have homebrew, xcode, and miniforge installed.
create a virtual environment
conda create --name mlp python=3.8
activate environment
conda activate mlp
Install tensorflow for mac-os
(sets up the wheel files etc.)
conda install -c apple tensorflow-deps
pip install tensorflow-macos
pip install tensorflow-metal
That's it. You should have the environment all ready to go. Look at Prabhat's article for a sample Jupyter Notebook test for an example of how to benchmark/test your environment.
Hey guys I had the same issue but I fixed it with the following instructions :
NOTE: If using conda environment built against pre-macOS 11 SDK use:
SYSTEM_VERSION_COMPAT=0 python -m pip install tensorflow-macos
otherwise, you will get errors like: “not a supported wheel on this platform”
STEPS :
OS Requirements macOS 12.0+ (latest beta)
Currently Not Supported
Multi-GPU support
Acceleration for Intel GPUs
V1 TensorFlow Networks
Installation Instructions
Step 1: Environment setup
CPU TYPE x86: AMD
Create virtual environment (recommended):
python3 -m venv ~/tensorflow-metal
source ~/tensorflow-metal/bin/activate
python -m pip install -U pip
NOTE: python version 3.8 required
CPU TYPE : arm64 : Apple Silicon
Download and install Conda env:
chmod +x ~/Downloads/Miniforge3-MacOSX-arm64.sh
sh ~/Downloads/Miniforge3-MacOSX-arm64.sh
source ~/miniforge3/bin/activate
OR
conda env create --file=environment.yml --name tf_m1
and then activate tf_m1
Install the TensorFlow dependencies:
conda install -c apple tensorflow-deps
When upgrading to new base TensorFlow version, we recommend:
uninstall existing tensorflow-macos and tensorflow-metal
python -m pip uninstall tensorflow-macos
python -m pip uninstall tensorflow-metal
Upgrade tensorflow-deps
conda install -c apple tensorflow-deps --force-reinstall
or point to specific conda environment
conda install -c apple tensorflow-deps --force-reinstall -n my_env
tensorflow-deps versions are following base TensorFlow versions so:
For v2.5:
conda install -c apple tensorflow-deps==2.5.0
For v2.6:
conda install -c apple tensorflow-deps==2.6.0
NOTE: Python versions 3.8 and 3.9 supported
Step 2: Install base TensorFlow
python -m pip install tensorflow-macos
NOTE: If using conda environment built against pre-macOS 11 SDK use:
SYSTEM_VERSION_COMPAT=0 python -m pip install tensorflow-macos
otherwise you will get errors like: “not a supported wheel on this
platform
Step 3: Install tensorflow-metal plugin
python -m pip install tensorflow-metal
The current default version of Python running on Google Colab is 3.7, but I need 3.9 for my notebooks to work.
How can I update Google Colab's Python version to 3.9 (or greater)?
In Google Colab you have a Debian-based Linux, and you can do whatever you can on a Debian Linux. Upgrading Python is as easy as upgrading it on your own Linux system.
Detect the current python version in Colab:
!python --version
#Python 3.8.16
Install new python version
Let's first install and upgrade to Python 3.9:
#install python 3.9
!sudo apt-get update -y
!sudo apt-get install python3.9
#change alternatives
!sudo update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.8 1
!sudo update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.9 2
#check python version
!python --version
#3.9.16
Port Colab kernel to the new installed python
As mentioned in the comments, the above commands just add a new python version to your google colab and update the default python for commandline usage. But your runtime packages such as sys are still running on the previous python version. The following commands need to be executed as well, to update the sys version.
# install pip for new python
!sudo apt-get install python3.9-distutils
!wget https://bootstrap.pypa.io/get-pip.py
!python get-pip.py
# credit of these last two commands blongs to #Erik
# install colab's dependencies
!python -m pip install ipython ipython_genutils ipykernel jupyter_console prompt_toolkit httplib2 astor
# link to the old google package
!ln -s /usr/local/lib/python3.8/dist-packages/google \
/usr/local/lib/python3.9/dist-packages/google
Now you can restart runtime and check the sys version. Note that in the new python version you have to install every packages, such as pandas, tensorflow, etc. from scratch.
Also, note that you can see a list of installed Python versions and switch between them at any time with this command:
(If nothing changed after installation, use this command to select python version manually)
!sudo update-alternatives --config python3
#after running, enter the row number of the python version you want.
It's also possible to update the kernel without going through ngrok or conda with some creative package installation.
Raha's answer suggesting making a link between the default google package and the newly installed Python version is the trick that makes this work because, at least with Python 3.9, the version of pandas (0.24.0) that the google package requires fails to build.
Here's the code I used to install and switch my Colab kernel to Python 3.9:
#install python 3.9 and dev utils
#you may not need all the dev libraries, but I haven't tested which aren't necessary.
!sudo apt-get update -y
!sudo apt-get install python3.9 python3.9-dev python3.9-distutils libpython3.9-dev
#change alternatives
!sudo update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.8 1
!sudo update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.9 2
#Check that it points at the right location
!python3 --version
# install pip
!curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
!python3 get-pip.py --force-reinstall
#install colab's dependencies
!python3 -m pip install ipython ipython_genutils ipykernel jupyter_console prompt_toolkit httplib2 astor
# link to the old google package
!ln -s /usr/local/lib/python3.8/dist-packages/google \
/usr/local/lib/python3.9/dist-packages/google
# There has got to be a better way to do this...but there's a bad import in some of the colab files
# IPython no longer exposes traitlets like this, it's a separate package now
!sed -i "s/from IPython.utils import traitlets as _traitlets/import traitlets as _traitlets/" /usr/local/lib/python3.9/dist-packages/google/colab/*.py
!sed -i "s/from IPython.utils import traitlets/import traitlets/" /usr/local/lib/python3.9/dist-packages/google/colab/*.py
If Google updates from Python 3.8, you'll have to change the path to the default package.
Then go the Runtime menu and select Restart runtime. It should reconnect and choose the updated version of Python as the default kernel. You can check that it worked with:
#check python version
import sys
print(sys.version)
!python3 --version
!python --version
To use another python version in google colab, you need to:
1- Installing Anaconda.
2- Adding (fake) google colab library.
3- Starting Jupyterlab.
4- Accessing it with ngrok.
# install Anaconda3
!wget -qO ac.sh https://repo.anaconda.com/archive/Anaconda3-2020.07-Linux-x86_64.sh
!bash ./ac.sh -b
# a fake google.colab library
!ln -s /usr/local/lib/python3.6/dist-packages/google \
/root/anaconda3/lib/python3.8/site-packages/google
# start jupyterlab, which now has Python3 = 3.8
!nohup /root/anaconda3/bin/jupyter-lab --ip=0.0.0.0&
# access through ngrok, click the link
!pip install pyngrok -q
from pyngrok import ngrok
print(ngrok.connect(8888))
you can also use:
# Install the python version
!apt-get install python3.9
# Select the version
!python3.9 setup.py
another way is to use a virtual environment with your desired python version:
virtualenv env --python=python3.9
Update 24.12.2022 - Unfortunately, the method does not work anymore.
This worked for me (copied from GitHub), I successfully installed Python 3.10.
#The code below installs 3.10 (assuming you now have 3.8) and restarts environment, so you can run your cells.
import sys #for version checker
import os #for restart routine
if '3.10' in sys.version:
print('You already have 3.10')
else:
#install python 3.10 and dev utils
#you may not need all the dev libraries, but I haven't tested which aren't necessary.
!sudo apt-get update -y
!sudo apt-get install python3.10 python3.10-dev python3.10-distutils libpython3.10-dev
!sudo apt-get install python3.10-venv binfmt-support #recommended in install logs of the command above
#change alternatives
!sudo update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.8 1
!sudo update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.10 2
# install pip
!curl -sS https://bootstrap.pypa.io/get-pip.py | python3.10
!python3 get-pip.py --force-reinstall
#install colab's dependencies
!python3 -m pip install setuptools ipython ipython_genutils ipykernel jupyter_console prompt_toolkit httplib2 astor
#minor cleanup
!sudo apt autoremove
#link to the old google package
!ln -s /usr/local/lib/python3.8/dist-packages/google /usr/local/lib/python3.10/dist-packages/google
#this is just to verify if 3.10 folder was indeed created
!ls /usr/local/lib/python3.10/
#restart environment so you don't have to do it manually
os.kill(os.getpid(), 9)
In addition to Kaveh's answer, I added the following code. (This colab python version is python 3.8 and I tried to downgrade to python 3.7)
!pip install google-colab==1.0.0
# install colab's dependencies
!python -m pip install ipython==7.9.0 ipython_genutils==0.2.0 ipykernel==5.3.4 jupyter_console==6.1.0 prompt_toolkit==2.0.10 httplib2==0.17.4 astor==0.8.1 traitlets==5.7.1 google==2.0.3
This way, I solved the crashing runtime error.
Simple as that: -
!wget -O mini.sh https://repo.anaconda.com/miniconda/Miniconda3-py39_4.9.2-Linux-x86_64.sh
!chmod +x mini.sh
!bash ./mini.sh -b -f -p /usr/local
!conda install -q -y jupyter
!conda install -q -y google-colab -c conda-forge
!python -m ipykernel install --name "py39" --user
Source: https://colab.research.google.com/drive/1m47aWKayWTwqJG--x94zJMXolCEcfyPS?usp=sharing#scrollTo=r3sLiMIs8If3
I installed the latest Tensorflow 0.5.0 from source via git clone.
and want to update to Tensorflow 0.6.0
git pull
./configure
bazel build -c opt --config=cuda //tensorflow/cc:tutorials_example_trainer
but the Tensorflow library in the directory /usr/lib/python2.7/site-packages still has the version 0.5.0
the version in the result of "pip show tensorflow" also is 0.5.0
To install the TensorFlow library from source, you need to build a PIP package and install it. The steps are as follows:
$ git pull
$ ./configure
$ bazel build -c opt //tensorflow/tools/pip_package:build_pip_package
# ...or, with GPU support
$ bazel build -c opt --config=cuda //tensorflow/tools/pip_package:build_pip_package
$ bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg
# The name of the .whl file will depend on your platform.
$ pip install /tmp/tensorflow_pkg/tensorflow-0.6.0-cp27-none-linux_x86_64.whl
git pull doesn't work for me since some local files are modified by the last build so with a slight modification I update like this:
git fetch --all
git reset --hard origin/master
./configure
bazel build -c opt --config=cuda //tensorflow/tools/pip_package:build_pip_package
bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg
sudo pip install /tmp/tensorflow_pkg/tensorflow-0.8.0-py2-none-any.whl
Tested to work as of today. The Installation from Source instructions in tensorflow docs are misleading in the sense they only include the real pip wheel installation commands for Mac and the example-trainer build command exists instead in Linux instructions.
To show the version:
python -c "import tensorflow; print(tensorflow.__version__);"
And if it is not the latest, you have uninstall it via pip uninstall:
sudo pip uninstall tensorflow
and subsequently install it:
export TF_BINARY_URL=https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow-0.9.0-cp27-none-linux_x86_64.whl
sudo pip install $TF_BINARY_URL
Before trying to update tensorflow try updating pip
pip install --upgrade pip
If you are upgrading from a previous installation of TensorFlow < 0.7.1, you should uninstall the previous TensorFlow and protobuf using,
pip uninstall
first to make sure you get a clean installation of the updated protobuf dependency.
Uninstall the TensorFlow on your system, and check out Download and Setup to reinstall again.
If you are using pip install, go check the available version over https://storage.googleapis.com/tensorflow, search keywords with linux/cpu/tensorflow to see the availabilities.
Then, set the path for download and execute in sudo.
$ export TF_BINARY_URL=https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-1.0.0-py2-none-any.whl
$ sudo pip install --upgrade $TF_BINARY_URL
For more detail, follow this link in here
If you get the error saying not a supported wheel on this platform. You might be updating tensorflow for python3. For that you will need pip3
Try installing pip3
sudo apt-get -y install python3-pip
Then, set the path for download if you haven't already set the path
$ export TF_BINARY_URL=https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-1.1.0rc0-cp35-cp35m-linux_x86_64.whl
$ pip3 install --ignore-installed --upgrade $TF_BINARY_URL
updating tensorflow install with sudo pip install --upgrade https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.7.1-cp27-none-linux_x86_64.whl.
I find it in the below issue,mohamed-ali's comment. https://github.com/tensorflow/tensorflow/issues/1105