I am working through the Tensorflow serving_basic example at:
https://tensorflow.github.io/serving/serving_basic
Setup
Following: https://tensorflow.github.io/serving/setup#prerequisites
Within a docker container based off of ubuntu:latest, I have installed:
bazel:
echo "deb [arch=amd64] http://storage.googleapis.com/bazel-apt stable jdk1.8" | sudo tee /etc/apt/sources.list.d/bazel.list
curl https://bazel.build/bazel-release.pub.gpg | sudo apt-key
sudo apt-get update && sudo apt-get install bazel
sudo apt-get upgrade bazel
grpcio:
pip install grpcio
all packages:
sudo apt-get update && sudo apt-get install -y build-essential curl libcurl3-dev git libfreetype6-dev libpng12-dev libzmq3-dev pkg-config python-dev python-numpy python-pip software-properties-common swig zip zlib1g-dev
tensorflow serving:
git clone --recurse-submodules https://github.com/tensorflow/serving
cd serving
cd tensorflow
./configure
cd ..
I've built the source with bazel and all tests ran successfully:
bazel build tensorflow_serving/...
bazel test tensorflow_serving/...
I can successfully export the mnist model with:
bazel-bin/tensorflow_serving/example/mnist_export /tmp/mnist_model
And I can serve the exported model with:
bazel-bin/tensorflow_serving/model_servers/tensorflow_model_server --port=9000 --model_name=mnist --model_base_path=/tmp/mnist_model/
The problem
When I test the server and try to connect a client to the model server with:
bazel-bin/tensorflow_serving/example/mnist_client --num_tests=1000 --server=localhost:9000
I see this output:
root#dc3ea7993fa9:~/serving# bazel-bin/tensorflow_serving/example/mnist_client --num_tests=2 --server=localhost:9000
Extracting /tmp/train-images-idx3-ubyte.gz
Extracting /tmp/train-labels-idx1-ubyte.gz
Extracting /tmp/t10k-images-idx3-ubyte.gz
Extracting /tmp/t10k-labels-idx1-ubyte.gz
AbortionError(code=StatusCode.NOT_FOUND, details="FeedInputs: unable to find feed output images")
AbortionError(code=StatusCode.NOT_FOUND, details="FeedInputs: unable to find feed output images")
Inference error rate is: 100.0%
The "--use_saved_model" model flag is set to default "true"; use the --use_saved_model=false when starting the server. This should work:
bazel-bin/tensorflow_serving/model_servers/tensorflow_model_server --use_saved_model=false --port=9000 --model_name=mnist --model_base_path=/tmp/mnist_model/
I mentioned this on the tensorflow github, and the solution was to remove the original model that had been created. If you're running into this, run
rm -rf /tmp/mnist_model
and rebuild it
Related
I'm using bazel to build tensorflow and tensorflow/serving, usually when I built tensorflow/serving with bazel, bazel generated some static library in bazel-bin/tensorflow_serving/core/ for me.
for example, after I successfully built tensorflow/serving, I got bazel-bin/tensorflow_serving/core/libaspired_version_policy.a and so on.
But things changed when building tensorflow/serving on another docker instance, under bazel-bin/tensorflow_serving/core/ I only got _objs, no static libraries anymore, how come?
What should I do to get these static libraries?
Here is my Dockerfile:
FROM tensorflow/tensorflow:1.15.5-py3
LABEL version="1.0.0"
WORKDIR /root/
ENV TMP=/tmp
RUN export https_proxy=http://220.181.102.178:8118 && \
export http_proxy=http://220.181.102.178:8118
RUN ln -s /usr/local/bin/gfortran /usr/bin/gfortran
RUN apt-get update -yq ; exit 0
RUN apt-get install -yq software-properties-common apt-utils && apt-get update -yq; exit 0
RUN apt-get install -yq vim tree clang gdb make git
RUN apt-get install -yq automake bison flex libboost-all-dev libevent-dev
RUN apt-get install -yq libssl-dev ssh libtool pkg-config
RUN apt-get install -yq default-jdk default-jre libunwind8-dev libc-ares-dev
RUN apt-get install -yq python-numpy python-future libleveldb-dev libsnappy-dev libgoogle-perftools-dev
RUN apt-get install -yq librdkafka-dev libapr1-dev libaprutil1-dev texinfo unzip zip
RUN unset TF_NEED_CUDA
## To install go, uncomment below
#RUN add-apt-repository ppa:longsleep/golang-backports -y
#RUN apt-get install -yq golang-go
RUN add-apt-repository ppa:ubuntu-toolchain-r/test -y
RUN apt-get install -yq gcc-8 g++-8
RUN update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-8 80 \
--slave /usr/bin/g++ g++ /usr/bin/g++-8 \
--slave /usr/bin/gcc-ar gcc-ar /usr/bin/gcc-ar-8 \
--slave /usr/bin/gcc-nm gcc-nm /usr/bin/gcc-nm-8 \
--slave /usr/bin/gcc-ranlib gcc-ranlib /usr/bin/gcc-ranlib-8
RUN export https_proxy=http://220.181.102.178:8118 && wget https://github.com/Kitware/CMake/releases/download/v3.18.5/cmake-3.18.5-Linux-x86_64.sh && \
bash cmake-3.18.5-Linux-x86_64.sh --prefix=/usr/ --skip-license && \
rm -rf cmake-3.18.5-Linux-x86_64.sh
RUN export https_proxy=http://220.181.102.178:8118 && wget https://github.com/bazelbuild/bazel/releases/download/0.24.1/bazel-0.24.1-installer-linux-x86_64.sh && \
bash bazel-0.24.1-installer-linux-x86_64.sh && rm -rf bazel-0.24.1-installer-linux-x86_64.sh
RUN mkdir .ssh/
ADD .ssh/ .ssh/
ADD .vimrc .
ADD .bashrc .
RUN export PATH=/usr/local/cuda/bin:$PATH && export LD_LIBRARY_PATH=/usr/local/cuda/lib64:/lib:/lib64:/usr/lib:/usr/lib64:/usr/local/lib:/usr/local/lib64:$LD_LIBRARY_PATH
RUN git config --global credential.helper store
bazel version : 0.24.1
tensorflow/serving version : 1.15.0 downloaded from github release page.
build command :
export https_proxy=http://10.130.48.179:3128
unset TF_NEED_CUDA
# Build tensorflow_model_server
bazel build -c opt --copt=-msse4.1 --copt=-msse4.2 --copt=-mavx --copt=-mavx2 --copt=-mfma --copt=-O3 --copt=-march=native --cxxopt="-fexceptions" --verbose_failures //tensorflow_serving/model_servers:tensorflow_model_server
Please help. thanks.
To build //tensorflow_serving/model_servers:tensorflow_model_server the Bazel need only *.o files. Making static libraries would be wasteful, because they are not needed by that cc_binary rule. Use //tensorflow_serving/core:aspired_version_policy to build that specific library, or just use //... to build everything in the workspace
Put linkstatic=0 in every cc_library call and this should be enough to force bazel to create static libs for every cc_library.
I want to try out the develop branch of spaCy in order to test the features of v3. I built it successfully, but get the following message when trying to download a model:
'No compatible models found for v3.0.0 of spaCy'
What can I do? How are contributors supposed to get models for a non-released version?
This is the Dockerfile I used:
FROM python:3.8
RUN apt-get update && apt-get install -y \
git \
make \
&& rm -rf /var/lib/apt/lists/*
RUN pip install --upgrade pip
RUN git clone https://github.com/explosion/spaCy /spaCy
WORKDIR /spaCy
RUN git checkout develop
ENV PYTHONPATH=.
RUN pip install -r requirements.txt
RUN python setup.py build_ext --inplace
RUN python setup.py install
RUN python -m spacy download en_core_web_sm
I want to have access to features from TensorFlow federated (tff.python.research) which aren't present with the pip3 install method.
I'm working on a remote server that does not have bazel, thus I cannot build from source. Are there other ways to get and install the latest working version of TFF from its GitHub REPO?
(https://github.com/tensorflow/federated)
To install the latest Tensorflow 2.0 federated, you may follow the steps below.
Install TensorFlow Federated using pip
Install the Python development environment
On Ubuntu:
$ sudo apt update
$ sudo apt install python3-dev python3-pip # Python 3
$ sudo pip3 install --upgrade virtualenv # system-wide install
On macOS:
$ /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
$ export PATH="/usr/local/bin:/usr/local/sbin:$PATH"
$ brew update
$ brew install python # Python 3
$ sudo pip3 install --upgrade virtualenv # system-wide install
Create a virtual environment
$ virtualenv --python python3 "venv"
$ source "venv/bin/activate"
(venv) $ pip install --upgrade pip
Note: To exit the virtual environment, run deactivate.
Install the TensorFlow Federated pip package.
(venv) $ pip install --upgrade tensorflow_federated
(Optional) Test Tensorflow Federated.
(venv) $ python -c "import tensorflow_federated as tff; print(tff.federated_computation(lambda: 'Hello World')())"
Build the TensorFlow Federated pip package
Install the Python development environment.
On Ubuntu:
$ sudo apt update
$ sudo apt install python3-dev python3-pip # Python 3
$ sudo pip3 install --upgrade virtualenv # system-wide install
On macOS:
$ /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
$ export PATH="/usr/local/bin:/usr/local/sbin:$PATH"
$ brew update
$ brew install python # Python 3
$ sudo pip3 install --upgrade virtualenv # system-wide install
Install Bazel
Install Bazel, the build tool used to compile Tensorflow Federated.
Clone the Tensorflow Federated repository.
$ git clone https://github.com/tensorflow/federated.git
$ cd "federated"
Create a virtual environment.
$ virtualenv --python python3 "venv"
$ source "venv/bin/activate"
(venv) $ pip install --upgrade pip
Note: To exit the virtual environment, run deactivate.
Install Tensorflow Federated dependencies.
(venv) $ pip install --requirement "requirements.txt"
(Optional) Test Tensorflow Federated.
(venv) $ bazel test //tensorflow_federated/...
Create a new project.
$ mkdir "/tmp/project"
$ cd "/tmp/project"
$ virtualenv --python python3 "venv"
$ source "venv/bin/activate"
(venv) $ pip install --upgrade pip
Note: To exit the virtual environment run deactivate.
Install the pip package.
(venv) $ pip install --upgrade "/tmp/tensorflow_federated/tensorflow_federated-"*".whl"
Test Tensorflow Federated.
(venv) $ python -c "import tensorflow_federated as tff; print(tff.federated_computation(lambda: 'Hello World')())"
Reference: https://www.tensorflow.org/federated/install
I've created a folder "Model_en" and the path to my saved model is:
"--model_base_path=/Users/tarunkumar/Documents/tensor_models/Model_en/1/"
also, my model name is:
"--model_name=Model_en"
After running the command:
"tensorflow_model_server --rest_api_port=8501 --model_name=Model_en --model_base_path=/Users/tarunkumar/Documents/tensor_models/Model_en/1/
"
I'm getting the error as:
"bash: tensorflow_model_server: command not found"
The installation may not have happen properly. For safer side, first remove the corrupt packages that may have installed,
apt-get remove tensorflow-model-server
Then,
Add TensorFlow Serving distribution URI as a package source:
echo "deb [arch=amd64] http://storage.googleapis.com/tensorflow-serving-apt stable tensorflow-model-server tensorflow-model-server-universal" | sudo tee /etc/apt/sources.list.d/tensorflow-serving.list && curl https://storage.googleapis.com/tensorflow-serving-apt/tensorflow-serving.release.pub.gpg | sudo apt-key add -
To this, you should get OK as output in the terminal.
Install and update TensorFlow ModelServer:
apt-get update && apt-get install tensorflow-model-server
Once installed, upgrade to a newer version of tensorflow-model-server with:
apt-get upgrade tensorflow-model-server
The binary can now be invoked using the command tensorflow_model_server
You may get output,
Failed to start server. Error: Invalid argument: Both server_options.model_base_path and server_options.model_config_file are empty!
This means the installation is successful and you can run the command to start the server.
source
This may help as well, at least it worked for me.
pip install tensorflow-serving-api
You have to install it first. Here is how it is done on Debian/Ubuntu:
Installation
Add TensorFlow Serving distribution URI as a package source (one time setup)
echo "deb [arch=amd64] http://storage.googleapis.com/tensorflow-serving-apt stable tensorflow-model-server tensorflow-model-server-universal" | sudo tee /etc/apt/sources.list.d/tensorflow-serving.list && \
curl https://storage.googleapis.com/tensorflow-serving-apt/tensorflow-serving.release.pub.gpg | sudo apt-key add -
Install and update TensorFlow ModelServer
apt-get update && apt-get install tensorflow-model-server
Once installed, the binary can be invoked using the command tensorflow_model_server.
You can upgrade to a newer version of tensorflow-model-server with:
apt-get upgrade tensorflow-model-server
Installing TensorFlow for object detection is annoying sometimes, especially when wired errors happen after starting one's own object detection project by finetuning pre-trained model.
How to install latest Tensorflow GPU support and latest CUDA/CUDNN without any error?
To install tensorflow-gpu object detection api on Ubuntu 16.04
upgrade system
sudo apt-get update
sudo apt-get upgrade
install basic packages
sudo apt-get install vim curl python-dev gnupg-curl python-tk git
curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
sudo -H python get-pip.py
install tensorflow-gpu (tensorflow version=1.14)
sudo -H pip install tensorflow-gpu
install CUDA (Final CUDA version = 10.1)
// Please turn off your secure boot from BIOS
sudo apt-get install gnupg-curl
// Here we install version 10.0 to avoid other issues. Later we can upgrade it to version 10.1
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64/cuda-repo-ubuntu1604_10.0.130-1_amd64.deb
sudo dpkg -i cuda-repo-ubuntu1604_10.0.130-1_amd64.deb
sudo apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64/7fa2af80.pub
sudo apt-get update
wget http://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1604/x86_64/nvidia-machine-learning-repo-ubuntu1604_1.0.0-1_amd64.deb
sudo apt install ./nvidia-machine-learning-repo-ubuntu1604_1.0.0-1_amd64.deb
sudo apt-get update
install cuDNN (cuDNN version = 7.6.3.30-1, compatible to CUDA 10.1)
sudo apt-get install --no-install-recommends cuda-10-0
// restart your computer here
nvidia-smi.
sudo apt-get install --no-install-recommends libcudnn7=7.6.3.30-1+cuda10.0 libcudnn7-dev=7.6.3.30-1+cuda10.0
sudo apt-get install -y --no-install-recommends libnvinfer5=5.1.5-1+cuda10.0 libnvinfer-dev=5.1.5-1+cuda10.0
sudo apt-get update
// this upgrade command will upgrade your CUDA to version 10.1
sudo apt-get upgrade
sudo apt-get autoremove
install Tensorflow object detection api
sudo -H pip install Cython
sudo -H pip install contextlib2
sudo -H pip install pillow
sudo -H pip install lxml
sudo -H pip install jupyter
sudo -H pip install matplotlib
mkdir tensorflow
cd tensorflow
git clone https://github.com/tensorflow/models
// install protocbuf version 3.0.0
wget -O protobuf.zip https://github.com/google/protobuf/releases/download/v3.0.0/protoc-3.0.0-linux-x86_64.zip
unzip protobuf.zip
sudo cp ./bin/protoc /bin/
sudo cp -r ./include/google /usr/local/include/
cd tensorflow/models/research
protoc object_detection/protos/*.proto --python_out=.
git clone https://github.com/cocodataset/cocoapi.git
cd cocoapi/PythonAPI
make
cp -r pycocotools <path_to_tensorflow>/models/research/
Export Environment
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda-10.1/lib64
export PYTHONPATH=$PYTHONPATH:~/tensorflow/models/research:~/tensorflow/models/research/object_detection/slim
Other
If we install CUDA 10.1 it may have a compatibility issue that libxxxx.so.10.0 is not found when you start training your project.
To solve them:
a. sudo ln -s /usr/lib/x86_64-linux-gnu/libcublas.so.10.1 /usr/local/cuda-10.1/lib64/libcublas.so.10.0
b. sudo ln -s /usr/local/cuda-10.1/lib64/libcudart.so.10.1 /usr/local/cuda-10.1/lib64/libcudart.so.10.0
c. sudo ln -s /usr/local/cuda-10.1/lib64/libcufft.so.10 /usr/local/cuda-10.1/lib64/libcufft.so.10.0
d. sudo ln -s /usr/local/cuda-10.1/lib64/libcurand.so.10 /usr/local/cuda-10.1/lib64/libcurand.so.10.0
e. sudo ln -s /usr/local/cuda-10.1/lib64/libcusolver.so.10 /usr/local/cuda-10.1/lib64/libcusolver.so.10.0
f. sudo ln -s /usr/local/cuda-10.1/lib64/libcusparse.so.10 /usr/local/cuda-10.1/lib64/libcusparse.so.10.0
All Set