I want to check if some packages are installed in the Colab. What is specific folder for storing the installed packages (e.g., keras)?
You can use the pip tool to list installed Python packages and their locations on the system:
!pip list -v | grep [Kk]eras
# Keras 2.2.5 /usr/local/lib/python3.6/dist-packages pip
# Keras-Applications 1.0.8 /usr/local/lib/python3.6/dist-packages pip
# Keras-Preprocessing 1.1.0 /usr/local/lib/python3.6/dist-packages pip
# keras-vis 0.4.1 /usr/local/lib/python3.6/dist-packages pip
Note that in Colab and other Jupyter notebook frontends, the ! character is used to execute a shell command.
If you are not able to find the package, you might have probably downloaded the package instead of installing it.
Incorrect command:
!pip download transformers
Correct command:
!pip install transformers
Next, in order to find the package run the following command:
!pip show transformers
Related
I have successfully downloaded a version of Python 3.10 to Colab, and Colab says I have the most up to date version of pandas. However, whenever I import pandas, the version is 1.3.5, while I need it to be up to date.
I tried restarting the kernel, but this did not fix things. The code and its output are below, excepting the long list I get when I download the newest version of Python.
!wget https://github.com/korakot/kora/releases/download/v0.10/py310.sh
!bash ./py310.sh -b -f -p /usr/local
!python -m ipykernel install --name "py310" --user
!python --version
Python 3.10.6
!pip3 install pandas --upgrade --no-deps
Requirement already satisfied: pandas in /usr/local/lib/python3.10/site-packages (1.5.2)
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
import pandas as pd
print(pd.__version__)
1.3.5
Thank you so much for your time.
I am a beginner in machine learning. I am using colab as the primary development platform.
I would like to use the latest version of sklearn in my coding projects. However, colab's sklearn version is 0.22.
Can I update the scikit learn version in colab?
Thank you.
You mentioned that you have a 0.22 version, hopefully, you checked and received the following results (by default) in your colab notebook.
!pip list | grep scikit-learn
scikit-learn 0.22.2.post1
I presume that you already tried with the following command, which ideally installs the latest version of sckit-learn.
!pip install scikit-learn
Requirement already satisfied: scikit-learn in /usr/local/lib/python3.7/dist-packages (0.22.2.post1)
Requirement already satisfied: numpy>=1.11.0 in /usr/local/lib/python3.7/dist-packages (from scikit-learn) (1.19.5)
Requirement already satisfied: joblib>=0.11 in /usr/local/lib/python3.7/dist-packages (from scikit-learn) (1.0.1)
Requirement already satisfied: scipy>=0.17.0 in /usr/local/lib/python3.7/dist-packages (from scikit-learn) (1.4.1)
As per the latest scikit-learn document, 0.24.2 is the latest version, which you can install/upgrade by overriding the version number as shown in the following script. It should help you if you want to upgrade to 0.24.2.
!pip install scikit-learn==0.24.2
Then, verify the scikit-learn version by following script.
!pip list | grep scikit-learn
scikit-learn 0.24.2
Also, alternative way to upgrade the package in a colab environment.
!pip install scikit-learn --upgrade
Ideally, you'll need to install it each time, however, some workaround such as you can save the current configuration into your google drive. Refer to the following script which was suggested in another post.
from google.colab import drive
drive.mount('/content/gdrive')
pip freeze --local > /content/gdrive/My\ Drive/colab_installed.txt
Refer to the following script to restore the environment from the file.
from google.colab import drive
drive.mount('/content/gdrive')
pip install --upgrade --force-reinstall `cat /content/gdrive/My\ Drive/colab_installed.txt`
I just updated the latest MacOS and it wiped everything out.
I follow the steps from this website.
And I start reinstalling the tensorflow with pip3 and it's not installing anything.
I got this error from typing pip3 install --user --upgrade tensorflow .
ERROR: Could not find a version that satisfies the requirement tensorflow (from versions: none)
ERROR: No matching distribution found for tensorflow
FYI, my MacBook is 64-bit.
and
~ pip3 -V
WARNING: pip is being invoked by an old script wrapper. This will fail in a future version of pip.
Please see https://github.com/pypa/pip/issues/5599 for advice on fixing the underlying issue.
To avoid this problem you can invoke Python with '-m pip' instead of running pip directly.
pip 20.2.4 from /Users/zack/Library/Python/3.8/lib/python/site-packages/pip (python 3.8)
~ python3 -V
Python 3.8.2
I don't know what happened does anyone have any ideas.
According to this documentation https://www.tensorflow.org/install/pip#package-location
and your version of python you should run
python3 -m pip install --upgrade https://storage.googleapis.com/tensorflow/mac/cpu/tensorflow-2.3.0-cp38-cp38-macosx_10_14_x86_64.whl
This is the 3rd step, so check if you got installed correctly the packages from steps 1 and 2
if pip is not up-to-date upgrade pip:
pip install --upgrade pip
then install tensorflow:
pip install tensorflow
for python 3.8 you need to install tensorflow 2.2 or later
system requirements of tensorflow:
Python 3.5–3.8
Python 3.8 support requires TensorFlow 2.2 or later.
pip 19.0 or later (requires manylinux2010 support)
Ubuntu 16.04 or later (64-bit)
macOS 10.12.6 (Sierra) or later (64-bit) (no GPU support)
I'm going over these example from google-cloud Coursera courses, and although they worked till a few weeks ago, I can't install tf.transform or apache_beam on Datalab anymore.
https://github.com/GoogleCloudPlatform/training-data-analyst/blob/master/courses/machine_learning/feateng/tftransform.ipynb
https://github.com/GoogleCloudPlatform/training-data-analyst/blob/master/courses/machine_learning/deepdive/06_structured/4_preproc_tft.ipynb
When installing tensorflow_transform I get the following errors:
%bash
pip install --upgrade --force tensorflow_transform==0.6.0
twisted 18.7.0 requires PyHamcrest>=1.9.0, which is not installed.
datalab 1.1.3 has requirement six==1.10.0, but you'll have six 1.11.0 which is incompatible.
gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
apache-airflow 1.9.0 has requirement bleach==2.1.2, but you'll have bleach 1.5.0 which is incompatible.
apache-airflow 1.9.0 has requirement funcsigs==1.0.0, but you'll have funcsigs 1.0.2 which is incompatible.
google-cloud-monitoring 0.28.0 has requirement google-cloud-core<0.29dev,>=0.28.0, but you'll have google-cloud-core 0.25.0 which is incompatible.
proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible.
pandas-gbq 0.3.0 has requirement google-cloud-bigquery>=0.28.0, but you'll have google-cloud-bigquery 0.25.0 which is incompatible.
googledatastore 7.0.1 has requirement httplib2<0.10,>=0.9.1, but you'll have httplib2 0.11.3 which is incompatible.
googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible.
Cannot uninstall 'dill'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.
The tensorflow version on my Datalab instance was 1.4.
I had to add this one line of code to update tensorflow to 1.10.1
%bash
pip install --upgrade --force-reinstall pip==10.0.1
pip install tensorflow==1.10.1
pip install tensorflow_transform
my environment:
apache-airflow==1.9.0
apache-beam==2.6.0
tensorflow==1.10.1
tensorflow-metadata==0.9.0
tensorflow-tensorboard==0.4.0rc3
tensorflow-transform==0.8.0
The current version of Datalab uses TensorFlow 1.8, so please change the notebook cell in question to:
%bash
pip uninstall -y google-cloud-dataflow
pip install --upgrade --force tensorflow_transform==0.8.0 apache-beam[gcp]
I've updated and checked in the two notebooks linked above.
Another problem might be that you are using Python 2. Datalab by default now uses Python 3 and your pip install (above) happens in Python 3 even if the kernel is Python 2 because %%bash opens up a new shell in which the conda activate of Python 2 has not happened.
To make sure the pip install happens in Python 2, change your pip install of apache-beam[gcp] as follows:
%%bash
source activate py2env
conda install -y dill pytz # do this for all the distutils complaints
pip uninstall -y google-cloud-dataflow
pip install --upgrade --force tensorflow_transform==0.8.0 apache-beam[gcp]
I installed pip via get-pip.py script
$ python get-pip.py --proxy="proxy.intranet.com:8080"
Collecting pip
Downloading pip-8.1.2-py2.py3-none-any.whl (1.2MB)
100% |████████████████████████████████| 1.2MB 559kB/s
Collecting wheel
Downloading wheel-0.29.0-py2.py3-none-any.whl (66kB)
100% |████████████████████████████████| 71kB 2.9MB/s
Installing collected packages: pip, wheel
Successfully installed pip-8.1.2 wheel-0.29.0
it worked fine. On upgrade, it seemed to be the latest version.
$ python -m pip install -U pip
Requirement already up-to-date: pip in /usr/lib/python2.7/site-packages
now, when I try to install tensorflow on windows using the below command, it doesn't work.
$ pip install --upgrade https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.5.0-cp27-none-linux_x86_64.whl
**tensorflow-0.5.0-cp27-none-linux_x86_64.whl is not a supported wheel on this platform**.
I tried to search a wheel file for windows, but couldn't find it. Anyone knows the locations to the whl file? Thanks in advance!
TensorFlow is only compatible with 64bit. Ensure that the Python installation is not 32bit.
TensorFlow is now available on Windows, from version 0.12 onwards. You can install the PIP package from PyPI, using the following command (for the CPU-only build):
C:\> pip install tensorflow
...or the following command if you have a CUDA 8.0-compatible GPU:
C:\> pip install tensorflow-gpu