How come we need to install tensorflow as a separate environment?
If we do it this way, many common libraries are not available when tensorflow is activated.
Most of the common libraries such as matplotlib, panda, etc. are not within tensorflow environment. So we have to install again to use them.
So why not just install under root so we don't have to re-install all those libraries under the new environment?
Thanks.
Related
I can't seem to find any documentation that describes what parts of TF and TFS need to be installesd/built to create a servable, can anyone shed light on the subject?
I'm not sure if this documentation exists. The approach I would take is to create a new blank environment, on conda or whatever you prefer. Then install Tensorflow and Tensorflow serving into the environment, which will prompt you to install the dependencies into the environment as well.
Then just to pip list or conda list (or equivalent) and see what all libraries got installed. That should give you a list of the base libraries needed to use TF and TF Serving.
I work with Tensorflow and PyTorch and manage each project that I work on inside of a Conda environment. This allows me to share projects and update my stack without breaking old projects. I am trying to get Conda to manage Cuda within an environment as well. Is this possible? I want old projects to be maintained whenever I update to newer versions of Cuda and CUDnn. Is there a way for Anaconda do this?
Upon trying to install Tensorflow for conda environment, I encountered with the following error message, without any progress:
tensorflow-1.1.0-cp35-cp35mwin_amd64.whl is not a supported wheel on this platform
Have you tried uninstalling and re-installing TensorFlow using pip within your Conda environment? I.e.:
pip uninstall tensorflow
Followed by:
pip install tensorflow
If it doesn't work, the issue may be with your Python installation. TensorFlow only supports 64-bit Python 3.5+ on Windows (see more info here).
Perhaps you have Python's default installation, which comes in a 32-bit version. If that's the case, you can download the 64-bit Python 3.5 or later from here to run in your Conda environment and then you should be able to install/run TensorFlow without any issues.
Make sure that the Python version installed in the Environment is 3.5 not 3.6. Since 3.6 was released Conda automatically sets that version as default for python 3. However, it is still not supported by Tensorflow.
You can work using tensorflow library along with other essential libraries using the Dockerfile. Using Docker for environment are a good way to run experiments in reproducible manner as in this blog
You can also try using datmo in order setup environment and track machine learning projects for making it reproducible using datmo CLI tool.
I am looking into using Tensorflow for my research soon, and looked at the online documentation for installing with Conda https://www.tensorflow.org/versions/r0.11/get_started/os_setup.html#anaconda-installation.
It suggested creating a new environment, and installing Tensorflow in it, and the installing other python packages afterwards.
But I already have an existing environment with lots of packages I need, and I'm wondering if its safe to add Tensorflow into that environment?
Also, I have a question about how this installation with conda works. I know that Conda will create a distinct set of folders containing the libraries needed for each environment, but if I install Tensorflow, what happens to all the base low level C++ and CUDA libraries that get compiled? Do they reside in my Conda environment's folder, or are they in some system wide libraries closer to my root?
PS: I'm using Ubuntu 16.04, and have a GPU that I want to run Tensorflow on.
Thank you.
But I already have an existing environment with lots of packages I need, and I'm wondering if its safe to add Tensorflow into that environment?
conda has this awesome feature called "revisions". You can show your current environment with
conda list --revisions
which allows you to revert changes to your conda environment. This allows you to install new packages with confidence that if something breaks you can always revert it later. See this page for more info: https://www.continuum.io/blog/developer/advanced-features-conda-part-2. tl;dr: conda install --revisions <revision_number>
what happens to all the base low level C++ and CUDA libraries that get compiled
Are you talking about the libraries that get compiled when you are trying to run your code? Or the C++/CUDA libraries? If you're talking about the C++/CUDA libs then conda is not compiling them, but merely installing a pre-compiled binary into a specific location that gets picked up. If you're talking about your code, then where exactly those files live would seem to depend on where you put them.
So the documentation mentions that it is better to install from source, then build a pip package. Why is this recommended over doing a direct pip install using the wheel file provided on the downloads page? here
I tried the direct pip install and then runnning some scripts in the inception folder.This results in errors with bazel not finding some of the dependencies. I am guessing this is related to not building tensorflow from source, but I can't figure out why this is the case. Any pointers? Thanks!
Installing from pip is supported, can you provide more details on your os and the specific errors you saw?
The main reason to build from source is simply performance.
Building and installing from source
The default TensorFlow binaries target the broadest range of hardware to make TensorFlow accessible to everyone. If using CPUs for training or inference, it is recommended to compile TensorFlow with all of the optimizations available for the CPU in use. Speedups for training and inference on CPU are documented below in Comparing compiler optimizations.
To install the most optimized version of TensorFlow, build and install from source. If there is a need to build TensorFlow on a platform that has different hardware than the target, then cross-compile with the highest optimizations for the target platform. The following command is an example of using bazel to compile for a specific platform
ref: https://www.tensorflow.org/performance/performance_guide