I'm using bazel to build tensorflow and tensorflow/serving, usually when I built tensorflow/serving with bazel, bazel generated some static library in bazel-bin/tensorflow_serving/core/ for me.
for example, after I successfully built tensorflow/serving, I got bazel-bin/tensorflow_serving/core/libaspired_version_policy.a and so on.
But things changed when building tensorflow/serving on another docker instance, under bazel-bin/tensorflow_serving/core/ I only got _objs, no static libraries anymore, how come?
What should I do to get these static libraries?
Here is my Dockerfile:
FROM tensorflow/tensorflow:1.15.5-py3
LABEL version="1.0.0"
WORKDIR /root/
ENV TMP=/tmp
RUN export https_proxy=http://220.181.102.178:8118 && \
export http_proxy=http://220.181.102.178:8118
RUN ln -s /usr/local/bin/gfortran /usr/bin/gfortran
RUN apt-get update -yq ; exit 0
RUN apt-get install -yq software-properties-common apt-utils && apt-get update -yq; exit 0
RUN apt-get install -yq vim tree clang gdb make git
RUN apt-get install -yq automake bison flex libboost-all-dev libevent-dev
RUN apt-get install -yq libssl-dev ssh libtool pkg-config
RUN apt-get install -yq default-jdk default-jre libunwind8-dev libc-ares-dev
RUN apt-get install -yq python-numpy python-future libleveldb-dev libsnappy-dev libgoogle-perftools-dev
RUN apt-get install -yq librdkafka-dev libapr1-dev libaprutil1-dev texinfo unzip zip
RUN unset TF_NEED_CUDA
## To install go, uncomment below
#RUN add-apt-repository ppa:longsleep/golang-backports -y
#RUN apt-get install -yq golang-go
RUN add-apt-repository ppa:ubuntu-toolchain-r/test -y
RUN apt-get install -yq gcc-8 g++-8
RUN update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-8 80 \
--slave /usr/bin/g++ g++ /usr/bin/g++-8 \
--slave /usr/bin/gcc-ar gcc-ar /usr/bin/gcc-ar-8 \
--slave /usr/bin/gcc-nm gcc-nm /usr/bin/gcc-nm-8 \
--slave /usr/bin/gcc-ranlib gcc-ranlib /usr/bin/gcc-ranlib-8
RUN export https_proxy=http://220.181.102.178:8118 && wget https://github.com/Kitware/CMake/releases/download/v3.18.5/cmake-3.18.5-Linux-x86_64.sh && \
bash cmake-3.18.5-Linux-x86_64.sh --prefix=/usr/ --skip-license && \
rm -rf cmake-3.18.5-Linux-x86_64.sh
RUN export https_proxy=http://220.181.102.178:8118 && wget https://github.com/bazelbuild/bazel/releases/download/0.24.1/bazel-0.24.1-installer-linux-x86_64.sh && \
bash bazel-0.24.1-installer-linux-x86_64.sh && rm -rf bazel-0.24.1-installer-linux-x86_64.sh
RUN mkdir .ssh/
ADD .ssh/ .ssh/
ADD .vimrc .
ADD .bashrc .
RUN export PATH=/usr/local/cuda/bin:$PATH && export LD_LIBRARY_PATH=/usr/local/cuda/lib64:/lib:/lib64:/usr/lib:/usr/lib64:/usr/local/lib:/usr/local/lib64:$LD_LIBRARY_PATH
RUN git config --global credential.helper store
bazel version : 0.24.1
tensorflow/serving version : 1.15.0 downloaded from github release page.
build command :
export https_proxy=http://10.130.48.179:3128
unset TF_NEED_CUDA
# Build tensorflow_model_server
bazel build -c opt --copt=-msse4.1 --copt=-msse4.2 --copt=-mavx --copt=-mavx2 --copt=-mfma --copt=-O3 --copt=-march=native --cxxopt="-fexceptions" --verbose_failures //tensorflow_serving/model_servers:tensorflow_model_server
Please help. thanks.
To build //tensorflow_serving/model_servers:tensorflow_model_server the Bazel need only *.o files. Making static libraries would be wasteful, because they are not needed by that cc_binary rule. Use //tensorflow_serving/core:aspired_version_policy to build that specific library, or just use //... to build everything in the workspace
Put linkstatic=0 in every cc_library call and this should be enough to force bazel to create static libs for every cc_library.
Related
I'm using this version https://github.com/pdf2htmlEX/pdf2htmlEX/releases/tag/v0.18.8.rc1
this debian version pdf2htmlEX-0.18.8.rc1-master-20200630-Ubuntu-focal-x86_64.deb
When I run the conversion I get a bunch of these errors:
Working: 97/100ToUnicode CMap is not valid and got dropped for font: b7
which result in empty files, without any text.
I'm running via docker, this is my dockerfile:
FROM ubuntu:20.04
RUN echo 'debconf debconf/frontend select Noninteractive' | debconf-set-selections
ENV DEBIAN_FRONTEND=noninteractive
RUN dpkg --configure -a
RUN apt-get clean
RUN apt-get update
RUN apt-get install -f -y python3
RUN apt-get install dialog apt-utils -y
RUN apt-get install -f -y python3-pip
RUN apt-get install -f -y python3-setuptools
RUN apt-get install -f -y wget
RUN apt-get install -f -y poppler-utils
RUN apt-get install -f -y poppler-data
RUN apt-get install -f -y jq
RUN apt-get install -f -y zip unzip
RUN apt-get install -f -y pdftk
RUN apt-get install -f -y ffmpeg
RUN apt-get install -f -y libfontforge-dev
RUN DEBIAN_FRONTEND=noninteractive; apt-get install -f -y pdftk-java
RUN apt install -f -y ghostscript
RUN pip3 install --upgrade pip \
&& apt-get clean
RUN pip3 --no-cache-dir install --upgrade awscli
WORKDIR /tmp
COPY lib/pdf2htmlEX-0.18.8.rc1-master-20200630-Ubuntu-focal-x86_64.deb /tmp
RUN apt install -y ./pdf2htmlEX-0.18.8.rc1-master-20200630-Ubuntu-focal-x86_64.deb
RUN wget https://www.imagemagick.org/download/ImageMagick.tar.gz && \
tar -xf ImageMagick.tar.gz && \
cd ImageMagick* && \
./configure && \
make && \
make install && \
ldconfig /usr/local/lib
Please advise how can I resolve this?
I want to try out the develop branch of spaCy in order to test the features of v3. I built it successfully, but get the following message when trying to download a model:
'No compatible models found for v3.0.0 of spaCy'
What can I do? How are contributors supposed to get models for a non-released version?
This is the Dockerfile I used:
FROM python:3.8
RUN apt-get update && apt-get install -y \
git \
make \
&& rm -rf /var/lib/apt/lists/*
RUN pip install --upgrade pip
RUN git clone https://github.com/explosion/spaCy /spaCy
WORKDIR /spaCy
RUN git checkout develop
ENV PYTHONPATH=.
RUN pip install -r requirements.txt
RUN python setup.py build_ext --inplace
RUN python setup.py install
RUN python -m spacy download en_core_web_sm
I am working through the Tensorflow serving_basic example at:
https://tensorflow.github.io/serving/serving_basic
Setup
Following: https://tensorflow.github.io/serving/setup#prerequisites
Within a docker container based off of ubuntu:latest, I have installed:
bazel:
echo "deb [arch=amd64] http://storage.googleapis.com/bazel-apt stable jdk1.8" | sudo tee /etc/apt/sources.list.d/bazel.list
curl https://bazel.build/bazel-release.pub.gpg | sudo apt-key
sudo apt-get update && sudo apt-get install bazel
sudo apt-get upgrade bazel
grpcio:
pip install grpcio
all packages:
sudo apt-get update && sudo apt-get install -y build-essential curl libcurl3-dev git libfreetype6-dev libpng12-dev libzmq3-dev pkg-config python-dev python-numpy python-pip software-properties-common swig zip zlib1g-dev
tensorflow serving:
git clone --recurse-submodules https://github.com/tensorflow/serving
cd serving
cd tensorflow
./configure
cd ..
I've built the source with bazel and all tests ran successfully:
bazel build tensorflow_serving/...
bazel test tensorflow_serving/...
I can successfully export the mnist model with:
bazel-bin/tensorflow_serving/example/mnist_export /tmp/mnist_model
And I can serve the exported model with:
bazel-bin/tensorflow_serving/model_servers/tensorflow_model_server --port=9000 --model_name=mnist --model_base_path=/tmp/mnist_model/
The problem
When I test the server and try to connect a client to the model server with:
bazel-bin/tensorflow_serving/example/mnist_client --num_tests=1000 --server=localhost:9000
I see this output:
root#dc3ea7993fa9:~/serving# bazel-bin/tensorflow_serving/example/mnist_client --num_tests=2 --server=localhost:9000
Extracting /tmp/train-images-idx3-ubyte.gz
Extracting /tmp/train-labels-idx1-ubyte.gz
Extracting /tmp/t10k-images-idx3-ubyte.gz
Extracting /tmp/t10k-labels-idx1-ubyte.gz
AbortionError(code=StatusCode.NOT_FOUND, details="FeedInputs: unable to find feed output images")
AbortionError(code=StatusCode.NOT_FOUND, details="FeedInputs: unable to find feed output images")
Inference error rate is: 100.0%
The "--use_saved_model" model flag is set to default "true"; use the --use_saved_model=false when starting the server. This should work:
bazel-bin/tensorflow_serving/model_servers/tensorflow_model_server --use_saved_model=false --port=9000 --model_name=mnist --model_base_path=/tmp/mnist_model/
I mentioned this on the tensorflow github, and the solution was to remove the original model that had been created. If you're running into this, run
rm -rf /tmp/mnist_model
and rebuild it
I am installing the apache steps are as followecho deb http ://ports.ubuntu.com/ubuntu-ports vivid restricted main multiverse universe >> /etc/apt/sources.list
apt-get install -y git \
openssl \
subversion \
autoconf \
libtool \
libapr \
libapr-util \
make \
libpcre3-dev \
libpcre++-dev \
libxml2-dev \
libexpat1-dev \
python
git clone https ://github.com/apache/httpd.git
cd httpd
svn co http: //svn.apache.org/repos/asf/apr/apr/trunk srclib/apr && cd srclib/apr && ./buildconf && ./build libtool && make && make install
The ./buildconf is giving error as follow
./buildconf
found apr source: srclib/apr
rebuilding srclib/apr/configure
buildconf: checking installation...
buildconf: python version 2.7.9 (ok)
buildconf: autoconf version 2.69 (ok)
buildconf: libtool not found.
You need libtool version 1.4 or newer installed
to build APR from SVN.
./buildconf failed for apr
In above as i have already installed libtool. But the libttol is not found.when i say apt-get install -y libtool i get message as # apt-get install -y libtoo
apt-get install -y libtool
Reading package lists... Done
Building dependency tree
Reading state information... Done
libtool is already the newest version.
0 upgraded, 0 newly installed, 0 to remove and 26 not upgraded.
W: Duplicate sources.list entry http://ports.ubuntu.com/ubuntu-ports/ vivid/restricted ppc64el Packages (/var/lib/apt/lists/ports.ubuntu.com_ubuntu-ports_dists_vivid_restricted_binary-ppc64el_Packages)
W: Duplicate sources.list entry http://ports.ubuntu.com/ubuntu-ports/ vivid/main ppc64el Packages (/var/lib/apt/lists/ports.ubuntu.com_ubuntu-ports_dists_vivid_main_binary-ppc64el_Packages)
W: You may want to run apt-get update to correct these problems
but when i try commands i get $libtool
bash: libtool: command not found
$man libtool
bash: man: command not found
$libttol --help
bash: libttol: command not found
$libtool --version
bash: libtool: command not found
In Ubuntu vivid, /usr/bin/libtool is provided by the libtool-bin package, not by the libtool package. Install libtool-bin:
apt-get update
apt-get install -y libtool-bin
I followed the steps below to install Shibboleth SP on Ubuntu 14.04
Installed the pre-requisite packages
sudo apt-get install --yes wget gcc g++ make libssl-dev libcurl4-openssl-dev apache2-threaded-dev
Created directories
mkdir -p ~/build-shib
sudo mkdir -p /opt/shibboleth-sp/etc/shibboleth
Downloaded and installed the following
cd ~/build-shib
wget http://www.shibboleth.net/downloads/log4shib/1.0.4/log4shib-1.0.4.tar.gz
wget http://www.apache.org/dist/xerces/c/3/sources/xerces-c-3.1.1.tar.gz
wget http://www.apache.org/dist/santuario/c-library/xml-security-c-1.6.1.tar.gz
wget http://www.shibboleth.net/downloads/c++-opensaml/2.4.3/xmltooling-1.4.2.tar.gz
wget http://www.shibboleth.net/downloads/c++-opensaml/2.4.3/opensaml-2.4.3.tar.gz
wget http://www.shibboleth.net/downloads/service-provider/2.4.3/shibboleth-sp-2.4.3.tar.gz
tar xvfz log4shib-1.0.4.tar.gz
tar xvfz xerces-c-3.1.1.tar.gz
tar xvfz xml-security-c-1.7.2.tar.gz
tar xvfz xmltooling-1.4.2.tar.gz
tar xvfz opensaml-2.4.3.tar.gz
tar xvfz shibboleth-sp-2.5.3.tar.gz
cd ~/build-shib/log4shib-1.0.4
./configure --disable-static --disable-doxygen --prefix=/opt/shibboleth-sp && make && sudo make install
cd ~/build-shib/xerces-c-3.1.1
./configure --prefix=/opt/shibboleth-sp && make && sudo make install
cd ~/build-shib/xml-security-c-1.7.2
./configure --without-xalan --disable-static --with-xerces=/opt/shibboleth-sp --prefix=/opt/shibboleth-sp && make && sudo make install
cd ~/build-shib/xmltooling-1.4.2
./configure --with-log4shib=/opt/shibboleth-sp --prefix=/opt/shibboleth-sp -C && make && sudo make install
cd ~/build-shib/opensaml-2.4.3
./configure --prefix=/opt/shibboleth-sp --with-log4shib=/opt/shibboleth-sp -C && make && sudo make install
cd ~/build-shib/shibboleth-2.5.3
./configure --with-saml=/opt/shibboleth-sp --enable-apache-22 --with-log4shib=/opt/shibboleth-sp --with-xmltooling=/opt/shibboleth-sp \
--prefix=/opt/shibboleth-sp -C && make && sudo make install
When I try to check if Shibboleth SP is compiled properly with
ldd /opt/shibboleth-sp/bbin/shibd | grep /opt/shibboleth-sp/lib
I get the following error
ldd: /opt/shibboleth-sp/bin/shibd: No such file or directory
On checking the contents in /opt/shibboleth-sp/bin/, I see that there is no shibd
What am I missing?
In my case the shibd was available at /opt/shibboleth-sp/sbin/ after source compilation. I have a 64-bit CPU architecture with Ubuntu 14.04. I am not sure if that affects.