Error when installing Tensorflow with Conda on Windows - tensorflow

Operating on Windows 10.
I am trying to install Tensorflow within a conda environment. The Anaconda 3 version I am using is the one that can be installed from within Visual Studio, conda version number is 4.6.14.
I created a new environment with conda create -n test python=3.6 and afterwards tried to install Tensorflow:
> conda activate test
> (test) conda install tensorflow-gpu
after which I'm getting the following error:
Downloading and Extracting Packages
tensorflow-base-1.13 | 217.6 MB | ############################################################################ | 100%
[Errno 2] No such file or directory: 'C:\\Program Files (x86)\\Microsoft Visual Studio\\Shared\\Anaconda3_64\\pkgs\\tensorflow-base-1.13.1-gpu_py36h871c8ca_0\\Lib\\site-packages\\tensorflow\\include\\tensorflow\\include\\external\\eigen_archive\\unsupported\\Eigen\\src\\SpecialFunctions\\SpecialFunctionsPacketMath.h'
Any idea on what could be the error here?
Edit: conda info returned:
active environment : base
active env location : C:\Program Files (x86)\Microsoft Visual Studio\Shared\Anaconda3_64
shell level : 1
user config file : C:\Users\Me\.condarc
populated config files :
conda version : 4.6.14
conda-build version : 3.10.5
python version : 3.6.5.final.0
base environment : C:\Program Files (x86)\Microsoft Visual Studio\Shared\Anaconda3_64 (writable)
channel URLs : https://repo.anaconda.com/pkgs/main/win-64
https://repo.anaconda.com/pkgs/main/noarch
https://repo.anaconda.com/pkgs/free/win-64
https://repo.anaconda.com/pkgs/free/noarch
https://repo.anaconda.com/pkgs/r/win-64
https://repo.anaconda.com/pkgs/r/noarch
https://repo.anaconda.com/pkgs/msys2/win-64
https://repo.anaconda.com/pkgs/msys2/noarch
package cache : C:\Program Files (x86)\Microsoft Visual Studio\Shared\Anaconda3_64\pkgs
C:\Users\Me\.conda\pkgs
C:\Users\Me\AppData\Local\conda\conda\pkgs
envs directories : C:\Program Files (x86)\Microsoft Visual Studio\Shared\Anaconda3_64\envs
C:\Users\Me\.conda\envs
C:\Users\Me\AppData\Local\conda\conda\envs platform : win-64
user-agent : conda/4.6.14 requests/2.18.4 CPython/3.6.5 Windows/10 Windows/10.0.17134
administrator : True netrc file : None
offline mode : False

I have the same issue. Like you, I also did the convenient "mistake" of installing Anaconda with the Visual Studio installer. As James pointed out, the problem is indeed that the file path is too long (> 260). Anyway, it would seem we have to swallow the bitter pill, uninstall Anaconda and reinstall it at root level (I'll go for Mini-Conda this time).
Before I do that, I'll try to find a way of moving the Anaconda3_64\pkgs\ to a different location. Should that work, I'll post it here.
Edit: Got the solution, no reinstall necessary!
The problem comes from windows itself, which by default does not support long file paths. But this can be changed using a registry flag:
https://knowledge.autodesk.com/support/autocad/learn-explore/caas/sfdcarticles/sfdcarticles/The-Windows-10-default-path-length-limitation-MAX-PATH-is-256-characters.html
Open Regedit as Admin
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem
Change LongPathsEnabled to 1 (DWORD 32 bit)
Warning: it could break compatibility of older programs. So far, for me everything is still working fine. If you should run into problems, simply revert the setting (and remember you have changed this setting!)

Related

Error installing MoveIt 2 from source on Ubuntu 20.04 VM

I'm trying to get involved in open source and also am quite interested in robotics, so I thought about contributing to MoveIt 2. As a first step, I'm installing MoveIt 2 from source on my Ubuntu 20.04 VM in VirtualBox.
I've run through all the instructions here: https://moveit.ros.org/install-moveit2/source/#:~:text=colcon%20build%20%2D%2Devent%2Dhandlers%20desktop_notification%2D%20status%2D%20%2D%2Dcmake%2Dargs%20%2DDCMAKE_BUILD_TYPE%3DRelease
However, the build is failing on the highlighted part. The error message is as follows:
Starting >>> geometric_shapes
--- stderr: geometric_shapes
CMake Error at CMakeLists.txt:47 (find_package):
By not providing "Findeigen3_cmake_module.cmake" in CMAKE_MODULE_PATH this
project has asked CMake to find a package configuration file provided by
"eigen3_cmake_module", but CMake did not find one.
Could not find a package configuration file provided by
"eigen3_cmake_module" with any of the following names:
eigen3_cmake_moduleConfig.cmake
eigen3_cmake_module-config.cmake
Add the installation prefix of "eigen3_cmake_module" to CMAKE_PREFIX_PATH
or set "eigen3_cmake_module_DIR" to a directory containing one of the above
files. If "eigen3_cmake_module" provides a separate development package or
SDK, be sure it has been installed.
---
Failed <<< geometric_shapes [1.20s, exited with code 1]
Summary: 0 packages finished [1.46s]
1 package failed: geometric_shapes
1 package had stderr output: geometric_shapes
33 packages not processed
Any help is really appreciated. Thanks!
You have to install ROS2 before building Move It 2. After installing the ROS2 binaries, the error doesn't occur.

Installing Tensorflow GPU/CUDA dependencies on a machine with no internet access

I have 2 machines -
dccten1a with no internet access where I need to install Tensorflow with GPU support
dccten1b with internet access so that I can download packages and transfer to dccten1a
In the final step of installing Tensorflow, when running the bazel build command to produce a whl file, I get an error which says that it can't find a file in a folder it is looking in, and also cannot download, obviously, as 1a doesn't have internet access.
bazel build --config=opt --config=cuda /home/tensorflow/Documents/tf_dependencies/tensorflow-master/tensorflow/tools/pip_package:build_pip_package --cxxopt="-D_GLIBCXX_USE_CXX11_ABI=0"
ERROR: error loading package '': Encountered error while reading extension file 'closure/defs.bzl': no such package '#io_bazel_rules_closure//closure': Error downloading [http://bazel-mirror.storage.googleapis.com/github.com/bazelbuild/rules_closure/archive/5ca1dab6df9ad02050f7ba4e816407f88690cf7d.tar.gz, https://github.com/bazelbuild/rules_closure/archive/5ca1dab6df9ad02050f7ba4e816407f88690cf7d.tar.gz] to /home/xyzuser/.cache/bazel/_bazel_xyzuser/cb1e63cb5e61cab49a9fd2f5ba92d003/external/io_bazel_rules_closure/5ca1dab6df9ad02050f7ba4e816407f88690cf7d.tar.gz: All mirrors are down: [Unknown host: github.com, Unknown host: mirror.bazel.build]
I checked in the system, and there is no such directory as shown in the error message (i.e., /home/xyzuser/.cache/bazel/_bazel_xyzuser/cb1e63cb5e61cab49a9fd2f5ba92d003/external/io_bazel_rules_closure/). So, I created it, searched and found the requisite (?) file online, downloaded the file in the machine with internet, transferred it to the target machine, moved the file to the just created directory, and tried running the command again:
(tensorflow#dccten1a):
mkdir -p /home/tensorflow/.cache/bazel/_bazel_tensorflow/cb1e63cb5e61cab49a9fd2f5ba92d003/external/io_bazel_rules_closure
(tensorflow#dccten1b):
http://bazel-mirror.storage.googleapis.com/github.com/bazelbuild/rules_closure/archive/5ca1dab6df9ad02050f7ba4e816407f88690cf7d.tar.gz
sudo scp -r /home/tensorflow/Downloads/5ca1dab6df9ad02050f7ba4e816407f88690cf7d.tar.gz tensorflow#160.88.114.17:/home/tensorflow/Documents/tf_dependencies
(tensorflow#dccten1a):
mv /home/tensorflow/Documents/tf_dependencies/5ca1dab6df9ad02050f7ba4e816407f88690cf7d.tar.gz /home/tensorflow/.cache/bazel/_bazel_tensorflow/cb1e63cb5e61cab49a9fd2f5ba92d003/external/io_bazel_rules_closure
Then I run the bazel build command again, but the same error persists.
Use --experimental_repository_cache to download the dependencies on the machine with internet access, transfer the cache to the machine without internet access, and use --experimental_repository_cache to refer to the same cache.
e.g.
1) On the machine with internet access, run
tensorflow#dccten1b $ bazel build --experimental_repository_cache=/path/to/some/folder --config=opt --config=cuda /home/tensorflow/Documents/tf_dependencies/tensorflow-master/tensorflow/tools/pip_package:build_pip_package --cxxopt="-D_GLIBCXX_USE_CXX11_ABI=0""
2) Copy the cache at /path/to/some/folder to the machine without internet access using a SD card or flash drive.
3) On the machine without internet access, run the same command again and setting the flag to the cache's location.
tensorflow#dccten1a $ bazel build --experimental_repository_cache=/path/to/some/folder --config=opt --config=cuda /home/tensorflow/Documents/tf_dependencies/tensorflow-master/tensorflow/tools/pip_package:build_pip_package --cxxopt="-D_GLIBCXX_USE_CXX11_ABI=0""

How to set up and build Apache 2.4 on WINDOWS 7 Pro?

I have a project, and need to set up and build an Apache24 server on windows, but there is almost non of information on that, can you give some advice?
I had to build and install Expat to get Apache running on windows 10 x64 with the instructions above. I build on Windows Visual Studio 2017 Community Edition using the x64 Native Command Prompt. (I installed all the VC and C++ modules - not sure which ones were specifically required.) Also i used the latest version of all of the software listed above so the install commands had to be adjusted accordingly. Trial by fire! Good luck.
I found the answer, after a lot of searching, I found an acceptable way to do it.
Software Requirements:
Visual Studio 2013 (I use the Community Edition)
Make a folder on C, call it BuildTools and install all the following programs there:
ActivePerl for Windows (64-bit, currently using 5.20.1.2000)
CMake for Windows (currently using 3.1.3)
GNU Awk for Windows (currently using 3.1.6-1)
GnuWin32 (any version from 2014+)
Netwide Assembler (NASM) (currently using 2.11.06)
Source Code Packages (I don't use ZLIB for Apache or OpenSSL, or LUA/LIBXML2/EXPAT, therefore these are not included in the process):
httpd-2.4.12.tar.gz
apr-1.5.1.tar.gz
apr-util.1.5.4.tar.gz
openssl-1.0.2a.tar.gz (yes it works with 1.0.2a!)
pcre-8.36.tar.gz
Here are the steps:
Extract all packages into their separate folders in your preferred source tree (e.g. C:\Development\Apache24\src)
Create custom build folders for Apache, PCRE, APR and APR-Util in your preferred build folder (e.g. C:\Development\Apache24\build
Your folder structure should resemble the below:
Make the following file changes so that ApacheMonitor gets built (without the Manifest error):
C:\Development\Apache24\src\httpd-2.4.12\CMakeLists.txt
Uncomment the section to build the ApacheMonitor utility (lines 769-775)
Find the following lines below, they will be in comments
# getting duplicate manifest error with ApacheMonitor
ADD_EXECUTABLE(ApacheMonitor support/win32/ApacheMonitor.c support/win32/ApacheMonitor.rc)
SET(install_targets ${install_targets} ApacheMonitor)
SET(install_bin_pdb ${install_bin_pdb} ${PROJECT_BINARY_DIR}/ApacheMonitor.pdb)
SET_TARGET_PROPERTIES(ApacheMonitor PROPERTIES WIN32_EXECUTABLE TRUE)
SET_TARGET_PROPERTIES(ApacheMonitor PROPERTIES COMPILE_FLAGS "-DAPP_FILE -DLONG_NAME=ApacheMonitor -DBIN_NAME=ApacheMonitor.exe / ${EXTRA_COMPILE_FLAGS}")
TARGET_LINK_LIBRARIES(ApacheMonitor ${EXTRA_LIBS} ${HTTPD_SYSTEM_LIBS} comctl32 wtsapi32)
Also
C:\Development\Apache24\src\httpd-2.4.12\support\win32\ApacheMonitor.rc
Comment out the line that includes ApacheMonitor.manifest (line 29)
//CREATEPROCESS_MANIFEST_RESOURCE_ID RT_MANIFEST "ApacheMonitor.manifest"
4.1 Create a set_path.bat file with the following code:
SET VC_HOME=c:\Program Files (x86)\Microsoft Visual Studio 13.0\VC
call VC_HOME\vcvarsall amd64
SET BUILD_ROOT=C:\BuildTools
SET PATH=%PATH%;%BUILD_ROOT%\cmake\bin
SET PATH=%PATH%;%BUILD_ROOT%\gawk\bin
SET PATH=%PATH%;%BUILD_ROOT%\nasm
SET PATH=%PATH%;%BUILD_ROOT%\perl\bin
You will need to run that file later in the process.
From the Start menu, launch the VS2013 x64 Native Tools Command Prompt found under Visual Studio 2013->Visual Studio Tools. Alternatively, but not recommended, from a Windows Command Prompt.
5.1 Go to GnuWin32, find his lib and the include folder, and copy their content to the BuildTools/perl/lib folder, this should resolve some errors that came up in the process while I was doing this
5.2 This step, you can skip now, BUT, if there is an error that includes the expat.h and the expath_external.h files, Find an older version of Apache, and copy its expat_external.h to the apache24/include folder
Find expat.h file on this site
https://github.com/apache/apr-util/blob/0.9.x/xml/expat/lib/expat.h
copy its content over the file, you find in apache24/include folder Find an older version of Apache (don't know exactly witch version), and copy its expat_external.h to the apache24/include folder
Find expat.h file on this site
https://github.com/apache/apr-util/blob/0.9.x/xml/expat/lib/expat.h
copy its content over the file, you find in apache24/include folder
5.3. Find the set_path.bat folder from the console and run it.
this you need to do, to be able to run the next steps
Navigate into the build sub-folder for PCRE, run CMAKE to generate a suitable Makefile, then compile and install
cd /D C:\Development\Apache24\build\pcre
cmake -G "NMake Makefiles" -DCMAKE_INSTALL_PREFIX=C:\Apache24 -DCMAKE_BUILD_TYPE=RelWithDebInfo -DBUILD_SHARED_LIBS=ON -DPCRE_BUILD_TESTS=OFF -DPCRE_BUILD_PCRECPP=OFF -DPCRE_BUILD_PCREGREP=OFF -DPCRE_SUPPORT_PCREGREP_JIT=OFF -DPCRE_SUPPORT_UTF=ON -DPCRE_SUPPORT_UNICODE_PROPERTIES=ON -DPCRE_NEWLINE=CRLF -DINSTALL_MSVC_PDB=OFF ..\..\src\pcre-8.36
nmake
nmake install
Navigate into the source sub-folder for OpenSSL, configure the build environment for compiling with NASM, then compile and install
cd /D C:\Development\Apache24\src\openssl-1.0.2a
perl Configure VC-WIN64A --prefix=C:\Apache24 --openssldir=C:\Apache24\conf enable-camellia no-idea no-mdc2 no-ssl2 no-ssl3 no-zlib
ms\do_win64a.bat
nmake /f ms\ntdll.mak
nmake /f ms\ntdll.mak install
Navigate into the build sub-folder for APR, run CMAKE to generate a suitable Makefile, then compile and install
cd /D C:\Development\Apache24\build\apr
cmake -G "NMake Makefiles" -DCMAKE_INSTALL_PREFIX=C:\Apache24 -DCMAKE_BUILD_TYPE=RelWithDebInfo -DMIN_WINDOWS_VER=0x0600 -DAPR_HAVE_IPV6=ON -DAPR_INSTALL_PRIVATE_H=ON -DAPR_BUILD_TESTAPR=OFF -DINSTALL_PDB=OFF ..\..\src\apr-1.5.1
nmake
nmake install
Navigate into the build sub-folder for APR-Util, run CMAKE to generate a suitable Makefile, then compile and install
cd /D C:\Development\Apache24\build\apr-util
cmake -G "NMake Makefiles" -DCMAKE_INSTALL_PREFIX=C:\Apache24 -DOPENSSL_ROOT_DIR=C:\Apache24 -DCMAKE_BUILD_TYPE=RelWithDebInfo -DAPU_HAVE_CRYPTO=ON -DAPR_BUILD_TESTAPR=OFF -DINSTALL_PDB=OFF ..\..\src\apr-util-1.5.4
nmake
nmake install
Navigate into the build sub-folder for Apache, run CMAKE to generate a suitable Makefile, then compile and install
cd /D C:\Development\Apache24\build\httpd
cmake -G "NMake Makefiles" -DCMAKE_INSTALL_PREFIX=C:\Apache24 -DCMAKE_BUILD_TYPE=RelWithDebInfo -DENABLE_MODULES=i -DINSTALL_PDB=OFF ..\..\src\httpd-2.4.12
nmake
nmake install
Finally, confirm everything is working
cd /D C:\Apache24\bin
openssl version
httpd -V
These are the steps and the problems I encountered, and I followed the steps from this site:
https://www.apachelounge.com/viewtopic.php?t=6462
Also I added a few more steps and solutions that I needed to do to make it work.
I hope this will help someone else in the future.

Trafodion installation ERROR: Error while running traf_start

I am trying to install trafodion on Hortonworks 2.2 virtual machine.The following are the machine configurations.
Hotonwork 2.2 virual machine
HBase version 0.98.
Centos version.
I have tried following steps to install trafodion.
1) I have downloaded
log4c++ RPM
Trafodion Installer
Trafodion Server
2)mkdir $HOME/trafodion
mkdir $HOME/trafodion/downloads
cd $HOME/trafodion/downloads
3) yum install to install the log4c++ RPM
4)cd $HOME/trafodion/downloads
tar -zxf apache-trafodion-installer-1.3.0-incubating-bin.tar.gz -C $HOME/trafodion
5)cd $HOME/trafodion/installer
cp trafodion_config_default my_config and Edit Configuration File.
6)cd $HOME/trafodion/installer
./trafodion_install --accept_license --config_file my_config
When running installation I obtain following message.
home/trafodion/traf
****INFO: Copying over sqenvcom.sh
***INFO: untarring build file /usr/lib/trafodion/apache-trafodion-1.3.0-incubating-bin/trafodion_server-1.3.0.tgz to home/trafodion/traf
***ERROR: SQ config file cannot be found (home/trafodion/traf/sql/scripts/sqconfig).
***ERROR: Error while running traf_start.
***ERROR: Setup not complete, review logs.
***ERROR: Exiting....
I don't understand why is it saying that. I have defined all configurations in myconfig file.
Any help is appreciated.
See also the discussion on the Trafodion dev mailing list on Apr. 21. Here is the answer Amanda gave on that list:
Amanda Moran via trafodion.incubator.apache.org Apr 22 (4 days ago)
to dev Hi there Devidas-
If you are on a single node instance SQCONFIG should equal "". This
will signal the installer to go grab our standard file located in
$SQ_ROOT/sql/scripts.
Thanks!
Amanda

cx_Oracle running with mod_wsgi environment

I installed cx_Oracle on CentOS 6.2. When I import the library from the shell, it works fine but when I launch it through wsgi, I get the error :
ImportError: libclntsh.so.10.1: cannot open shared object file: No such file or directory
This is an environment variable problem : cx_Oracle does not find the path to the lib.
I have tried the solutions provided here
I have added a link to libclntsh.so.10.1 (with ln) in the /usr/lib directory
I have edited apache configuration and added :
ORACLE_HOME=/usr/lib/oracle/11.2/client64/lib
LD_LIBRARY_PATH=$ORACLE_HOME/
PATH=$ORACLE_HOME/bin:$PATH
I edited /etc/ld.so.conf and added :
/usr/lib/oracle/11.2/client64/lib
done after ldconfig
I tried to use python with :
os.env['ORACLE_HOME']='/usr/lib/oracle/11.2/client64/lib'
I edited the bashrc with :
export ORACLE_HOME=/usr/lib/oracle/11.2/client64/lib
export LD_LIBRARY_PATH=$ORACLE_HOME/
export PATH=$ORACLE_HOME/bin:$PATH
I also edited apachectl with
ORACLE_HOME=/usr/lib/oracle/11.2/client64/lib
export ORACLE_HOME
LD_LIBRARY_PATH=$ORACLE_HOME/
export LD_LIBRARY_PATH
PATH=$ORACLE_HOME/bin:$PATH
export PATH
I am running out of ideas. Any suggestions ?
When you compile the Python module for Oracle, set:
LD_RUN_PATH=/usr/lib/oracle/11.2/client64/lib
user environment variable and export it. This will cause that directory to be embedded in Python extension module .so file and will know where to find it at run time without needing to set LD_LIBRARY_PATH environment variable.
For a standard Apache distribution (Linux distros are often a bit different), the file to set extra environment variables in is called 'envvars' and is in same directory as 'httpd'. For Linux distros often needs to be in a special init.d startup script.
So, lookup what LD_RUN_PATH is all about.
Instead of using yum install on the cx_Oracle rpm, I downloaded the source of the library and run the setup.py build.
I got an error that would point me to the function that was trying to locate the instant client sdk libraries in :
possibleIncludeDirs = ["rdbms/demo", "rdbms/public", "network/public","sdk/include"]
Browsing the Oracle_home folder, i discovered that the sdk file where installed in the lib folder ( I installed the skd using yum install on the rpm from oracle ) and not in the possibleIncludeDirs or in an include folder as suggested in the setup.py :
if not includeDirs:
path = os.path.join(oracleLibDir, "include")
if os.path.isdir(path):
includeDirs.append(path)
if not includeDirs:
path = re.sub("lib(64)?", "include", oracleHome)
if os.path.isdir(path):
includeDirs.append(path)
I downloaded the instant client sdk (the zip file this time) and unziped it to the lib folder.
There was then a sdk folder in the lib folder (/usr/lib/oracle/11.2/client64/lib)
I then launched the setup.py build and setup.py install and it worked.