I am trying to use my local GPU to train an EfficientDetD0 model. I already have a good pipeline (that works on Google Colab for example), I modified it a bit to use it locally, but one problem happens every time I launch the training.
I use conda to install tensorflow-gpu with cuda and cudnn but it makes TensorFlow v2.4.1 environments and when I launch the training the Object Detection API automatically install TensorFlow V2.5.0. So my env is not using the gpu for the training because cuda and cudnn are waiting for TensorFlow to be v2.4.1 and not v2.5.0.
Is there a way to get the Object Detection API in v2.4.1 and not v2.5.0 ?
I tried many things but it doesn't work (training is failing or going for CPU training).
Here is the code that install dependencies and overwrite TensorFlow version to TensorFlow v2.5.0:
os.system("cp object_detection/packages/tf2/setup.py .")
os.system("python -m pip install .")
SYSTEM:
gpu : Nvidia RTX 3070
os : Ubuntu 20.04 LTS
tensorflow: 2.4.1
P.S.: I go with conda install -c conda-forge tensorflow-gpu for installing TensorFlow, cuda and cudnn in my training env because manually there was a dependency problem, so I took the easy way.
EDIT : solution found explained in comments.
Follow these steps to install specific version of tensorflow gpu
1. Set Up Anaconda Environments
conda create -n tf_gpu cudatoolkit=11.0
2. Activate the new Environment
source activate tf_gpu
3. Install tensorflow-gpu 2.4.1
pip install tensorflow==2.4.1
Try to run object_detection without "installing" it. Dont run setup.py. Just setup the neccesery paths and packages manually.
Or edit the setup.py to skip installing the specific verison of TF. I quess that this version is a requirement of some of the packages installed in setup.py.
I use the object_detection without running the setup.py or doing any "installation" without any problems.
I have installed visual studio 2019, and Cuda 10.1 and TensorFlow 2.1.0 and I still can't run face recognition with GPU, can someone give me a complete guide on the steps to use GPU instead of CPU.
note: I'm using windows 10, my GPU is gtx1050 and I am using anaconda spider.
Firstly, you should install tensorflow-gpu package instead of tensorflow.
If your tf is installed correctly, you can run face recognition in gpu within deepface. You can test it with allocate memory function.
#!pip install deepface
from deepface import DeepFace
DeepFace.allocateMemory()
If everything is OK, then it returns "DeepFace will run on GPU" message.
All face recogntion models except Dlib will run on tensorflow-gpu. You can run face recognition with verification function.
from deepface import DeepFace
models = ["VGG-Face", "Facenet", "OpenFace", "DeepFace", "DeepID", "Dlib"]
obj = DeepFace.verify("img1.jpg", "img2.jpg", model_name = models[0])
print(obj)
i solved this by these steps using an anaconda enviroment:
cuda 10.2 installed
python (3.7.7)
conda install pip
conda install tensorflow (latest 2.1.0)
conda install tensorflow-gpu
pip install imutils
pip install opencv-python
pip install opencv-contrib-python
pip install dlib
pip install face_recognition
As Adrian Rosebrock mentioned at https://www.pyimagesearch.com/2018/06/18/face-recognition-with-opencv-python-and-deep-learning/, it is better to build Dlib from source, so the library is aware of the presence of the exact GPU in your system. My guess is - during the building process, it probes the capabilities of the GPU through the CUDA driver and builds and installs accordingly.
git clone https://github.com/davisking/dlib.git
cd dlib
mkdir build
cd build
cmake .. -DDLIB_USE_CUDA=1 -DUSE_AVX_INSTRUCTIONS=1
cmake --build .
cd ..
python setup.py install --yes USE_AVX_INSTRUCTIONS --yes DLIB_USE_CUDA
I am currently using tensorflow 1.2.1 and I am trying to update to version 1.7.0 using conda, but it is downgraded to 1.1.0. Why is this happening?
The default tensorflow version under conda package manager is 1.1.0.
Try creating a new environment within Anaconda with conda virtual environment manager (refer to this doc for more information):
$ conda create -n tensorflow
So the subsequent tensorflow installation won't mess up with your default Anaconda environment (I personally experienced this).
After successfully creating the virtual environment, activate it by:
$ source activate tensorflow
Your prompt should then change to:
(tensorflow) $
To install tensorflow version 1.7.0 in the new prompt, use pip instead of conda:
(tensorflow) $ pip install --ignore-installed --upgrade TF_PYTHON_URL
where TF_PYTHON_URL is the url of the tensorflow package with the latest version 1.7.0 (choose according to your python version).
Note that packages (e.g., spyder) you want to use together with tensorflow which are not already in the new environment should be installed:
(tensorflow) $ conda install spyder
This step can be merged with Step 1 by issuing a single command in the default prompt:
$ conda create -n tensorflow spyder
Every time you work with tensorflow, activate the (tensorflow) environment using Step 2, and after you finish, deactivate the environment to revert back to the default prompt:
(tensorflow) $ source deactivate
Hope these can help :-)
I have been trying to update tensorflow from 1.2.1 to 1.3.
I did the following on my terminal:
pip3 install tensorflow --upgrade
After this, I tried checking the version
python3 -c 'import tensorflow as tf; print(tf.__version__)'
This outputs 1.2.1 but not 1.3.0
All this had been done in an anaconda environment.
Operating system: macOS Sierra
I would like to know how one can perform this update.
Thank you
The solution that I found was to install pip inside the conda environment and use that instance of pip to install the packages. Please refer here
The reason I got the above error was that the python3 and pip3 paths were different.
This became evident when I ran the below commands inside the conda environment.
which python3
/Users/SMBP/anaconda/envs/tensorflow/bin/python3
which pip3
/usr/local/bin/pip3
Also, I think it is better to use virtualenv and virtualenvwrapper if you wish to work with tensorflow as it is the recommended way.
How do I install TensorFlow's tensorboard?
The steps to install Tensorflow are here: https://www.tensorflow.org/install/
For example, on Linux for CPU-only (no GPU), you would type this command:
pip install -U pip
pip install tensorflow
Since TensorFlow depends on TensorBoard, running the following command should not be necessary:
pip install tensorboard
Try typing which tensorboard in your terminal. It should exist if you installed with pip as mentioned in the tensorboard README (although the documentation doesn't tell you that you can now launch tensorboard without doing anything else).
You need to give it a log directory. If you are in the directory where you saved your graph, you can launch it from your terminal with something like:
tensorboard --logdir .
or more generally:
tensorboard --logdir /path/to/log/directory
for any log directory.
Then open your favorite web browser and type in localhost:6006 to connect.
That should get you started. As for logging anything useful in your training process, you need to use the TensorFlow Summary API. You can also use the TensorBoard callback in Keras.
If your Tensorflow install is located here:
/usr/local/lib/python2.7/dist-packages/tensorflow
then the python command to launch Tensorboard is:
$ python /usr/local/lib/python2.7/dist-packages/tensorflow/tensorboard/tensorboard.py --logdir=/home/user/Documents/.../logdir
The installation from pip allows you to use:
$ tensorboard --logdir=/home/user/Documents/.../logdir
It may be helpful to make an alias for it.
Install and find your tensorboard location:
pip install tensorboard
pip show tensorboard
Add the following alias in .bashrc:
alias tensorboard='python pathShownByPip/tensorboard/main.py'
Open another terminal or run exec bash.
For Windows users, cd into pathShownByPip\tensorboard and run python main.py from there.
For Python 3.x, use pip3 instead of pip, and don't forget to use python3 in the alias.
TensorBoard isn't a separate component. TensorBoard comes packaged with TensorFlow.
Adding this just for the sake of completeness of this question (some questions may get closed as duplicate of this one).
I usually use user mode for pip ie. pip install --user even if instructions assume root mode. That way, my tensorboard installation was in ~/.local/bin/tensorboard, and it was not in my path (which shouldn't be ideal either). So I was not able to access it.
In this case, running
sudo ln -s ~/.local/bin/tensorboard /usr/bin
should fix it.
pip install tensorflow.tensorboard # install tensorboard
pip show tensorflow.tensorboard
# Location: c:\users\<name>\appdata\roaming\python\python35\site-packages
# now just run tensorboard as:
python c:\users\<name>\appdata\roaming\python\python35\site-packages\tensorboard\main.py --logdir=<logidr>
If you're using the anaconda distribution of Python, then simply do:
$❯ conda install -c conda-forge tensorboard
or
$❯ conda install -c anaconda tensorboard
Also, you can have a look at various builds by search the packages repo by:
$❯ anaconda search -t conda tensorboard
which would list the channels and the corresponding builds, the supported OS, Python versions etc.,
The pip package you are looking for is tensorflow-tensorboard developed by Google.
If you installed TensorFlow using pip, then the location of TensorBoard can be retrieved by issuing the command which tensorboard on the terminal. You can then edit the TensorBoard file, if necessary.
It is better not to mix up the virtual environments or perform installation on the root directory. Steps I took for hassle free installation are as below. I used conda for installing all my dependencies instead of pip. I'm answering with extra details, because when I tried to install tensor board and tensor flow on my root env, it messed up.
Create a virtual env
conda create --name my_env python=3.6
Activate virtual environment
source activate my_env
Install basic required modules
conda install pandas
conda install tensorflow
Install tensor board
conda install -c condo-forge tensor board
Hope that helps
I have a local install of tensorflow 1.15.0 (with tensorboard obviously included) on MacOS.
For me, the path to the relevant file within my user directory is Library/Python/3.7/lib/python/site-packages/tensorboard/main.py. So, which does not work for me, but you have to look for the file named main.py, which is weird since it apparently is named something else for other users.