I was wondering, if I install Ipopt with conda install ipopt, will I still need to install Blas and Lapack, an AMPL solver librairy and a solver for systems of linear equations ?
I am using a remote server and the leader of this doesn't want to install more things if it's not completely necessary.
Thanks for the help!
Related
i followed the instructions given by the official tf documentation, but i just cannot resolve the various problems encountered.
Did anyone have the experience installing tff on m1 mac and can show me your overall process?
conda create -n federated python=3.8
conda activate federated
pip install --upgrade tensorflow_federated
everything seems to be fine according to the terminal output, however,
after
import tensorflow_federated as tff
i got a RunTimeError:
RuntimeError: This version of jaxlib was built using AVX instructions, which your CPU and/or operating system do not support. You may be able work around this issue by building jaxlib from source.
how to resolve this?
I don't have sudo access to the remote pc where cuda is already installed. Now, I have to install tensorflow-gpu on that system. Please give me the step by step guide to install it without sudo.
Operating System : Ubuntu 18.04
I had to do this before. Basically, I installed miniconda (you can also use anaconda, same thing and installation works without sudo), and installed everything using conda.
Create my environment and activate it:
conda create --name myenv python=3.6.8
conda actiavate myenv
Install the CUDA things and Tensorflow
conda install cudatoolkit=9.0 cudnn=7.1.2 tensorflow-gpu
Depending on your system, you may need to change version numbers.
Not sure how familiar you are with conda - it is basically a package-manager/repository and environment manager like pip/venv with the addition that it can handle non-python things as well (such as cudnn for example). As a note - if a package is not availabe through conda, you can still use pip as a fallback.
Untested with pip
I previously tried to do it without conda and using pip (I ended up failing due to some version conflicts, got frustrated with the process and moved to conda). It gets a little more complicated since you need to manually install it. So first, download cudnn from nvidia and unpack it anywhere you want. Then, you need to add it to the LD_LIBRARY_PATH:
export LD_LIBRARY_PATH=/path/to/cuda/lib64:/path/to/cudnn/lib64/:${LD_ LIBRARY_PATH}
Just to say it upfront, I'm aware of all the answers that require bazel and they didn't work for me. I'm using virtualenv as the tensorflow website recommends to.
(tensorflow27)name#computersname:~$ bazel build --linkopt='-lrt' -c opt --copt=-mavx --copt=-msse4.2 --copt=-msse4.1 --copt=-msse3-k //tensorflow/tools/pip_package:build_pip_package
will output
ERROR: The 'build' command is only supported from within a workspace.
Basically I followed all steps from here
But when I run this validation I get
2017-09-02 11:46:52.613368: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
2017-09-02 11:46:52.613396: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
2017-09-02 11:46:52.613416: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
I DO NOT want to surpress the warnings, I actually want to use SSE 4.2 and AVX (my processor supports both) However I wasn't able to find any instructions anywhere how to compile tensorflow inside a virutal environment such that the support for SSE and AVX is enabled from scratch. It's not even listed in their common installation problems section.
Btw. the system I use for this is Ubuntu 14.04 and I don't have an nvidia graphics card (so no cuda for now), but I plan to get one in the future.
I'm a little bit disappointed that tensorflow doesn't detect CPU capabilities before compilation.
Edit: Untrue, later I figured out it actually does
PS: I have setup two virtual environments, one for python 2.7 and the other one for python 3.0. Ideally I would hope that the solution works for both, as I didn't decide yet which python version I will end up using.
OK, so it turned out my problems were pretty much independent of which virtual environment I choose. The bazel build failed simply because I was in the wrong directory. I needed to pull tensorflow from git and then cd into it. Then I could build from there, using the default build option -march=native, which already detects my CPU capabilities.
Setting up two different virtual environments for the two different python versions in advance was useful. I used those environments to compile directly inside of them. This results in automatic python version detection. So building in the 2.7 environment will result in a build for python 2.7 etc.
To be somewhat more detailed:
First I installed bazel. Then I installed the dependencies mentioned on the tensorflow page like this:
sudo apt-get install python3-numpy python3-dev python3-pip python3-wheel
sudo apt-get install python-numpy python-dev python-pip python-wheel
In order to use OpenCL in the build later, I had to download (and complile if I remember correctly) ComputeCpp-CE-0.3.1-Ubuntu.14.04-64bit.tar.gz (For ubuntu 16 it would be ComputeCpp-CE-0.3.1-Ubuntu.16.04-64bit.tar.gz). After compilation I had to move the build result to /usr/local/computecpp:
sudo mkdir /usr/local/computecpp
sudo cp -r ./Downloads/ComputeCpp*/* /usr/local/computecpp
If I remember correctly at this point I then activated my virtual environment with the python version I want to compile against, so the desired python version would be recognized by ./configure.
Then I pulled tensorflow from git and configured it:
git clone https://github.com/tensorflow/tensorflow
cd tensorflow
./configure
During the configuration routine I only answered yes to jemalloc and OpenCL as I don't have a CUDA card right now. When saying yes to OpenCL I was prompted for the ComputeCpp path which was at that location I created under /usr/local/computecpp
Then, while staying in the tensorflow directory I did
bazel build --config=opt --config=mkl //tensorflow/tools/pip_package:build_pip_package
People who activated cuda during ./configure, should also add a "--config=cuda" to this bazel build command. If you have gcc > 5 you will also need to add '--cxxopt="-D_GLIBCXX_USE_CXX11_ABI=0'.
The '--config=mkl' part activates some additional libraries provided by intel specifically to speed up some calculations on their processors, so tensorflow can make use of them. If you don't have an Intel processor, it's probably wise to remove that option.
Btw, at first I also compiled mkl by hand, but that's not necessary it turns out. The bazel build will automatically pull mkl from an online source if it's missing.
Then I created the final package at /tmp/tensorflow_pkg:
bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg
Depending on the python version in your environment you will see a file name reflecting it:
/tmp/tensorflow_pkg/tensorflow-1.3.0-cp27-cp27mu-linux_x86_64.whl
/tmp/tensorflow_pkg/tensorflow-1.3.0-cp36-cp36m-linux_x86_64.whl
Now I could go to the appropriate environment and install it using pip. If you are using conda, the tensorflow website still recommends to use "pip install" and not "conda install". So if I were in a virtual environment with python 2.7 (doesn't matter if conda or not) I would type:
pip install --ignore-installed --upgrade /tmp/tensorflow_pkg/tensorflow-1.3.0-cp27-cp27mu-linux_x86_64.whl
And if I were under python 3.6 I would type:
pip install --ignore-installed --upgrade /tmp/tensorflow_pkg/tensorflow-1.3.0-cp36-cp36m-linux_x86_64.whl
There was one last hoop I needed to jump over: If you stay in the tensorflow directory (which you pulled from git) and then launch python, you won't be able to import tensorflow. I think it's some kind of a bug. So it's important to escape the tensorflow directory before starting python.
cd ..
Now you can start your python inside your virtual environment and import tensorflow. Hopefully without further errors or any warnings about not using the SSE or AVX capabilities of your processor. At least in my case it worked.
Can someone advise me how to build a Debian package for Tensorflow? So, I can install TF with Linux command like
dpkg -i tensorflow.deb
Thanks;
It's a wee bit complicated by the fact that some packages have (Build-)Depends: you need packaged first.
This Debian bug report is the so-called 'Intent To Package' but it itself depends on bazel being packaged first. So in order to get tf, you need to package bazel. Not a small task.
For the first time I'v installed tensorflow with conda installation. Then I actually work with a seq2seq model. After that I have again installed the tensorflow with the pip installation. But now the libraries are very different. All the old scripts are misplaced etc. Why is that ? Why I didn't face this when I was working with coda instillation
It has been claimed that Tensorflow installed with Conda performs a lot faster than a Pip installation, for example:
https://towardsdatascience.com/stop-installing-tensorflow-using-pip-for-performance-sake-5854f9d9eb0c
Conda also installs all of the package dependencies automatically, which Pip does not, as far as I'm aware.
https://www.anaconda.com/blog/developer-blog/tensorflow-in-anaconda/
Pip and conda install to two different locations. You should try to stick to one or the other. I would recommend uninstalling the conda version and sticking to pip but it's up to you how to proceed.
Update 01-02-2019: It seems that conda is now the faster and preferred way to install tensorflow. Note this may change again in the future.