Use google coral miniPCIe on Docker :error "Failed to load delegate from libedgetpu.so.1" - python-3.8

I have the problem using google coral miniPCIe on Docker. My docker file is:
FROM python:3.8.13-slim-bullseye
LABEL Author="S"
LABEL version="1.0"
ENV PATH /usr/local/bin:$PATH
# PYTHON 3.8 and libraries
# ffmpeg x counter: # ImportError: libGL.so.1: cannot open shared object file: No such file or directory
# set timezone
RUN apt-get update && \
apt-get install -yq tzdata && \
ln -fs /usr/share/zoneinfo/Europe/Rome /etc/localtime && \
dpkg-reconfigure -f noninteractive tzdata;
RUN set -eux; \
apt-get update; \
apt-get -y install --no-install-recommends \
curl \
gnupg \
bash \
python3-dev \
apt-utils \
libglib2.0-0 \
libsm6 \
libxext6 \
libxrender-dev \
ffmpeg \
net-tools \
iputils-ping \
vim;
# CORAL PCI BOARD
# test with lspci -nn | grep 089a
RUN set -eux; \
echo "deb https://packages.cloud.google.com/apt coral-edgetpu-stable main" | tee /etc/apt/sources.list.d/coral-edgetpu.list ;\
curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add - ; \
apt-get update; \
apt-get -y install --no-install-recommends \
python3-pycoral gcc \
gasket-dkms \
libedgetpu1-std \
pciutils ;
RUN set -eux; \
apt-get -y install --no-install-recommends \
libatlas-base-dev \
mosquitto-clients \
sysstat ; \
rm -rf /var/lib/apt/lists/* ;
COPY ./baseimage.requirements.txt /baseimage/
RUN pip install --no-cache-dir -r /baseimage/baseimage.requirements.txt
RUN python3 -m pip install --extra-index-url https://google-coral.github.io/py-repo/ pycoral~=2.0
CMD ["/bin/bash"]
in the baseimage.requirements.txt files:
asyncua==0.9.94
Cython==0.29.23
filterpy==1.4.5
Flask==1.1.2
Flask_RESTful==0.3.8
geojson==2.5.0
imutils==0.5.4
ipython==8.4.0
keras_unet_collection==0.1.6
msgpack_numpy==0.4.7.1
msgpack_python==0.5.6
numba==0.54.0
numpy==1.19.5
opencv_python==4.1.2.30
Pillow==9.2.0
prometheus_client==0.14.1
pygeoj==1.0.0
pyminizip==0.2.4
PyMySQL==1.0.2
pynmea2==1.18.0
pyproj==3.3.1
pypylon==1.7.4
pyrealsense2==2.42.0.2924
pyserial==3.5
redis==3.5.3
requests==2.22.0
scikit_image==0.18.1
scikit_learn==1.1.1
scipy==1.6.2
setuptools==45.2.0
Shapely==1.8.2
# skimage==0.0
tensorflow==2.5.0
tflite_runtime==2.7.0
paho_mqtt==1.6.1
psutil==5.8.0
wheel==0.36.2
The docker container have mounted: -v /dev/apex_0:/dev/apex_0 and see the coral with command: lspci -nn | grep 089a
I use Python 3.8.13.
In a docker container I execute the coral example:
git clone https://github.com/google-coral/pycoral.git
cd pycoral
bash examples/install_requirements.sh classify_image.py
python3 examples/classify_image.py \
--model test_data/mobilenet_v2_1.0_224_inat_bird_quant_edgetpu.tflite \
--labels test_data/inat_bird_labels.txt \
--input test_data/parrot.jpg
And I have the error:
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/tflite_runtime/interpreter.py", line 160, in load_delegate
delegate = Delegate(library, options)
File "/usr/local/lib/python3.8/site-packages/tflite_runtime/interpreter.py", line 119, in __init__
raise ValueError(capture.message)
ValueError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "examples/classify_image.py", line 121, in <module>
main()
File "examples/classify_image.py", line 71, in main
interpreter = make_interpreter(*args.model.split('#'))
File "/usr/local/lib/python3.8/site-packages/pycoral/utils/edgetpu.py", line 87, in make_interpreter
delegates = [load_edgetpu_delegate({'device': device} if device else {})]
File "/usr/local/lib/python3.8/site-packages/pycoral/utils/edgetpu.py", line 52, in load_edgetpu_delegate
return tflite.load_delegate(_EDGETPU_SHARED_LIB, options or {})
File "/usr/local/lib/python3.8/site-packages/tflite_runtime/interpreter.py", line 162, in load_delegate
raise ValueError('Failed to load delegate from {}\n{}'.format(
ValueError: Failed to load delegate from libedgetpu.so.1
On host machine the same example works.

Related

Call to undefined function Intervention\Image\Gd\imagecreatefromjpeg() Laravel 8 + Docker desktop 4.4.4

This is my docker file:
FROM php:8.0-fpm-buster
# Arguments defined in docker-compose.yml
ARG user
ARG uid
# Install system dependencies
RUN apt-get update && apt-get install -y \
build-essential \
git \
curl \
libpng-dev \
libjpeg-dev \
libfreetype6-dev \
libjpeg62-turbo-dev \
jpegoptim optipng pngquant gifsicle \
libonig-dev \
libxml2-dev \
zip \
sudo \
unzip \
npm \
nodejs \
&& docker-php-ext-configure gd --with-freetype --with-jpeg \
&& docker-php-ext-install -j$(nproc) gd
# Clear cache
RUN apt-get clean && rm -rf /var/lib/apt/lists/*
# Install PHP extensions
RUN docker-php-ext-install pdo_mysql mbstring exif pcntl bcmath gd
but I have an error
Call to undefined function Intervention\Image\Gd\imagecreatefromjpeg()
when i try to upload a jpg image.Even I change it to:
docker-php-ext-configure gd --with-freetype-dir=/usr/include/ --with-jpeg-dir=/usr/include/ --with-gd --with-freetype --with-jpeg \
but I have below error when I build and the above error in Laravel
configure: error: unrecognized options: --with-freetype-dir,
--with-jpeg-dir, --with-gd
------ failed to solve: rpc error: code = Unknown desc = executor failed running [/bin/sh -c apt-get update && apt-get install -y
build-essential git curl libpng-dev libjpeg-dev
libfreetype6-dev libjpeg62-turbo-dev jpegoptim optipng
pngquant gifsicle libonig-dev libxml2-dev zip sudo
unzip npm nodejs && docker-php-ext-configure gd
--with-freetype-dir=/usr/include/ --with-jpeg-dir=/usr/include/ --with-gd --with-freetype --with-jpeg && docker-php-ext-install -j$(nproc) gd]: exit code: 1
how can I fix this?
I fix it by enabling the GD lib,
here is my Dockerfile configs:
FROM php:8.0-fpm-buster
# Arguments defined in docker-compose.yml
ARG user
ARG uid
# Install system dependencies
RUN apt-get update && apt-get install -y \
build-essential \
git \
curl \
libpng-dev \
libjpeg-dev \
libfreetype6-dev \
libjpeg62-turbo-dev \
libmcrypt-dev \
libgd-dev \
jpegoptim optipng pngquant gifsicle \
libonig-dev \
libxml2-dev \
zip \
sudo \
unzip \
npm \
nodejs
# Clear cache
RUN apt-get clean && rm -rf /var/lib/apt/lists/*
# Install PHP extensions
RUN docker-php-ext-configure gd --enable-gd --with-freetype --with-jpeg
RUN docker-php-ext-install pdo_mysql mbstring exif pcntl bcmath gd

AWS Codebuild build stage shows nginx was installed but container doesn't have nginx

I'm new with AWS Codebuild and trying to deploy my docker application in EKS. On my Dockerfile I'm running RUN apk add nginx and build stage shows that it's being installed.
Step 6/24 : RUN apk add nginx
---> Running in cbd7a4e37bb7
(1/2) Installing pcre (8.44-r0)
(2/2) Installing nginx (1.18.0-r15)
Executing nginx-1.18.0-r15.pre-install
Executing nginx-1.18.0-r15.post-install
Executing busybox-1.32.1-r6.trigger
OK: 15 MiB in 34 packages
Removing intermediate container cbd7a4e37bb7
---> a890efd77e97
After the BUILD stage has been completed, I enter my EKS pod but didn't see any nginx under /etc
/etc # ls
ImageMagick-7 fstab issue my.cnf.d periodic securetty sysctl.d
alpine-release group logrotate.d mysql php7 services terminfo
apk group- modprobe.d network pkcs11 shadow udhcpd.conf
ca-certificates hostname modules openldap profile shadow- wgetrc
ca-certificates.conf hosts modules-load.d opt profile.d shells
conf.d init.d motd os-release protocols ssh
crontabs inittab mtab passwd resolv.conf ssl
fonts inputrc my.cnf passwd- rsyncd.conf sysctl.conf
I manually executed apk add nginx inside my pod and that's the only time nginx appeared under /etc
My AWS Codebuild is using the following configurations:
Managed image
OS: Amazon Linux 2
Runtime: Standard
Image: aws/codebuild/amazonlinux2-x86_64-standard:3.0
Image version: Always use the latest image for this runtime version
Privilaged: Enabled
Also,
I noticed that AWS Build doesn't execute the following COPY command in my Dockerfile:
COPY docker/php/php.ini /usr/local/etc/php/ (this works)
COPY docker/nginx/conf.d/default.conf /etc/nginx/conf.d/default.conf (this doesn't, probably because nginx didn't get installed on runtime)
COPY docker/nginx/docker-entrypoint.sh /usr/local/bin/docker-entrypoint-d9 (this doen't work.. still a mystery)
I hope I was able to provide the full details. Please let me know if you need more information. Thank you.
buildspec.yml
version: 0.2
env:
variables:
PHP_VERSION: 7.4
APP_HOME: /var/www/html
phases:
pre_build:
commands:
- echo Logging in to Amazon ECR...test4
- aws --version
- docker --version
- echo $AWS_DEFAULT_REGION
- $(aws ecr get-login --region $AWS_DEFAULT_REGION --no-include-email)
- export KUBECONFIG=$HOME/.kube/config
- export PHP_VERSION=$PHP_VERSION
- REPOSITORY_URI=xxxxxxx.xxx.xxx.xx-xxxxxxx-x.amazonaws.com/development-ecr
- COMMIT_HASH=$(echo $CODEBUILD_RESOLVED_SOURCE_VERSION | cut -c 1-7)
- IMAGE_TAG=$CODEBUILD_BUILD_NUMBER
# - IMAGE_TAG=build-$(echo $CODEBUILD_BUILD_ID | awk -F":" '{print $2}')
build:
commands:
- echo "PHP Version is $PHP_VERSION"
- echo "$APP_HOME"
- echo Running Docker compose..
- echo Build started on `date`
- echo Building the Docker image...
- docker build --build-arg PHP_VERSION=${PHP_VERSION} -t $REPOSITORY_URI:latest test/
- docker tag $REPOSITORY_URI:latest $REPOSITORY_URI:$IMAGE_TAG
post_build:
commands:
- echo Build completed on `date`
- echo Pushing the Docker images...
- docker push $REPOSITORY_URI:latest
- docker push $REPOSITORY_URI:$IMAGE_TAG
- echo "Setting Environment Variables related to AWS CLI for Kube Config Setup"
- CREDENTIALS=$(aws sts assume-role --role-arn $EKS_KUBECTL_ROLE_ARN --role-session-name codebuild-kubectl --duration-seconds 900)
- export AWS_ACCESS_KEY_ID="$(echo ${CREDENTIALS} | jq -r '.Credentials.AccessKeyId')"
- export AWS_SECRET_ACCESS_KEY="$(echo ${CREDENTIALS} | jq -r '.Credentials.SecretAccessKey')"
- export AWS_SESSION_TOKEN="$(echo ${CREDENTIALS} | jq -r '.Credentials.SessionToken')"
- export AWS_EXPIRATION=$(echo ${CREDENTIALS} | jq -r '.Credentials.Expiration')
# Setup kubectl with our EKS Cluster
- echo "Update Kube Config"
- aws eks update-kubeconfig --name $EKS_CLUSTER_NAME
# Apply changes to our Application using kubectl
- echo "Delete current deployment"
- kubectl delete deployment dev-spectrum-app
- sleep 10
- echo "Apply changes to kube manifests"
- kubectl apply -f kube-yaml/
- echo "Completed applying changes to Kubernetes Objects"
pipeline for other stages
- echo Writing image definitions file...
- printf '[{"name":"dev-spectrum","imageUri":"%s"}]' $REPOSITORY_URI:$IMAGE_TAG > imagedefinitions.json
- cat imagedefinitions.json
artifacts:
files: imagedefinitions.json
Dockerfile
ARG PHP_VERSION=7.4
FROM php:${PHP_VERSION:+${PHP_VERSION}-}fpm-alpine
ARG APP_HOME
RUN echo "CHECKING ARGS: $APP_HOME ${PHP_VERSION}"
RUN apk update; \
apk upgrade;
RUN apk add nginx
# Install library extension
RUN apk add libpng libpng-dev libjpeg-turbo libjpeg-turbo-dev freetype-dev libwebp-dev zlib-dev libxpm-dev
# Install MySql
RUN docker-php-ext-install mysqli pdo pdo_mysql
RUN docker-php-ext-install opcache
RUN docker-php-ext-configure gd --with-jpeg --with-freetype \
&& docker-php-ext-install gd
# PHP packages
RUN apk add libressl \
ca-certificates \
openssh-client \
rsync \
git \
curl \
wget \
gzip \
tar \
patch \
perl \
pcre \
imap \
imagemagick \
mariadb-client \
build-base \
autoconf \
libtool \
php7-dev \
pcre-dev \
imagemagick-dev \
php7 \
php7-fpm \
php7-opcache \
php7-session \
php7-dom \
php7-xml \
php7-xmlreader \
php7-ctype \
php7-ftp \
php7-json \
php7-posix \
php7-curl \
php7-pdo \
php7-pdo_mysql \
php7-sockets \
php7-zlib \
php7-mcrypt \
php7-mysqli \
php7-sqlite3 \
php7-bz2 \
php7-phar \
php7-openssl \
php7-posix \
php7-zip \
php7-calendar \
php7-iconv \
php7-imap \
php7-soap \
php7-dev \
php7-pear \
php7-redis \
php7-mbstring \
php7-xdebug \
php7-exif \
php7-xsl \
php7-ldap \
php7-bcmath \
php7-memcached \
php7-oauth \
php7-apcu
# Install Composer
RUN curl -sS https://getcomposer.org/installer | php && \
mv composer.phar /usr/local/bin/composer && \
ln -s /root/.composer/vendor/bin/drush /usr/local/bin/drush
# Install Drush
RUN composer global require drush/drush && \
composer global update
# Copy php.ini memory limit increase
COPY docker/php/php.ini /usr/local/etc/php/
COPY docker/nginx/conf.d/default.conf /etc/nginx/conf.d/default.conf
COPY docker/nginx/docker-entrypoint.sh /usr/local/bin/docker-entrypoint.sh
#COPY docker/nginx/docker-entrypoint.sh /usr/local/bin/docker-entrypoint-d9
RUN echo ${APP_HOME} ${PHP_VERSION}
WORKDIR /var/www/html
RUN mkdir /run/nginx;
#Copy composer files
COPY composer.json composer.lock ./
RUN composer install --prefer-dist --no-interaction
RUN composer dump-autoload
RUN set -eux; \
chmod +x /usr/local/bin/docker-entrypoint.sh
# chmod +x /usr/local/bin/docker-entrypoint-d9
EXPOSE 80 433
CMD ["nginx"]
#RUN nginx
#RUN php-fpm
ENTRYPOINT ["docker-entrypoint.sh"]

Error while installing numpy in alpine image

Error :
ERROR: Command "/usr/local/bin/python -u -c 'import setuptools, tokenize;file='"'"'/tmp/pip-install-2u_c2pqi/numpy/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record /tmp/pip-record-gfsn5nsl/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-install-2u_c2pqi/numpy/
This is the error I am getting in my travis build of my dockerfile.
RUN echo "http://dl-8.alpinelinux.org/alpine/edge/community" >> /etc/apk/repositories \
24 && apk --no-cache --update-cache add postgresql-dev musl-dev linux-headers g++ gfortran py-pip build-base bash libpng-dev openblas-dev w get freetype-dev gcc \
25 && ln -s /usr/include/locale.h /usr/include/xlocale.h \
26 && pip3 install setuptools wheel \
27 && pip3 install numpy pyyaml`
I am installing numpy using the above commands
Some issues have been found in your Dockerfile:
python3-dev and py3-pip packages should be installed to install numpy.
g++ already has gcc and musl-dev in dependencies, so we can simplify a package list.
The final Dockerfile is:
FROM alpine:latest
RUN echo "http://dl-8.alpinelinux.org/alpine/edge/community" >> /etc/apk/repositories \
&& apk --no-cache --update-cache add postgresql-dev g++ linux-headers gfortran build-base bash libpng-dev openblas-dev wget freetype-dev python3-dev py3-pip \
&& ln -s /usr/include/locale.h /usr/include/xlocale.h \
&& pip3 install setuptools wheel \
&& pip3 install numpy pyyaml

ImportError run python project from Google Drive in Google Colab

I mouth my Google Drive with Google Colab (follow the post by https://medium.com/deep-learning-turkey/google-colab-free-gpu-tutorial-e113627b9f5d)
!apt-get install -y -qq software-properties-common python-software-properties module-init-tools
!add-apt-repository -y ppa:alessandro-strada/ppa 2>&1 > /dev/null
!apt-get update -qq 2>&1 > /dev/null
!apt-get -y install -qq google-drive-ocamlfuse fuse
from google.colab import auth
auth.authenticate_user()
from oauth2client.client import GoogleCredentials
creds = GoogleCredentials.get_application_default()
import getpass
!google-drive-ocamlfuse -headless -id={creds.client_id} -secret={creds.client_secret} < /dev/null 2>&1 | grep URL
vcode = getpass.getpass()
!echo {vcode} | google-drive-ocamlfuse -headless -id={creds.client_id} -secret={creds.client_secret}
!mkdir -p drive
!google-drive-ocamlfuse drive
Then I place Tensorflow nmt into drive/colab/ (drive is google drive root directory.
I run the command in Colab Cell
!python drive/colab/nmt/nmt.py\
--attention=scaled_luong \
--src=src --tgt=tgt \
--vocab_prefix=drive/colab/data/vi-vi/vocab \
--train_prefix=drive/colab/data/vi-vi/small_train \
--dev_prefix=drive/colab/data/vi-vi/val \
--test_prefix=drive/colab/data/vi-vi/test \
--out_dir=drive/colab/data/vi-vi/nmt_attention_model \
--num_train_steps=12000 \
--steps_per_stats=100 \
--num_layers=2 \
--num_units=128 \
--dropout=0.2 \
--metrics=bleu
With error
/usr/local/lib/python3.6/dist-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
from ._conv import register_converters as _register_converters
Traceback (most recent call last):
File "drive/colab/nmt/nmt.py", line 31, in <module>
from . import inference
ImportError: cannot import name 'inference'
What should I do ?
It's happening because you are using relative import in nmt.py and your current directory is not the same as where nmt.py is located. And because of that python is not able to find the files. You can fix it by changing your directory to mnt using
%cd drive/colab/nmt/
and then running your command.

How to install xvfb on Scrapinghub for using Selenium?

I use Python-Selenium in my spider (Scrapy), for using Selenium i should install xvfb on Scrapinghub.
when i use apt-get for installing xvfb i have this error message:
E: Could not open lock file /var/lib/dpkg/lock - open (13: Permission denied)
E: Unable to lock the administration directory (/var/lib/dpkg/), are you root?
Is there any other way for installing xvfb on Scrapinghub?
UPDATE 1
I read this, I tried to use docker, I am stuck at this stage
shub-image init --requirements path/to/requirements.txt
i read this
If you are getting an ImportError like this while running shub-image init:
You should make sure you have the latest version of shub installed by
running:
$ pip install shub --upgrade
but i have always this error
Traceback (most recent call last):
File "/usr/local/bin/shub-image", line 7, in <module>
from shub_image.tool import cli
File "/usr/local/lib/python2.7/dist-packages/shub_image/tool.py", line 42, in <module>
command_module = importlib.import_module(module_path)
File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
File "/usr/local/lib/python2.7/dist-packages/shub_image/push.py", line 4, in <module>
from shub.deploy import list_targets
ImportError: cannot import name list_targets
did you try:
sudo apt-get install xvfb
Another way is to compile manually the packages, a sort of:
apt-get source xvfb
./configure --prefix=$HOME/myapps
make
make install
And the third way, is download the .deb from the source web page https://pkgs.org/download/xvfb
after download it, you can mv it to the path of the downloaded sources:
mv xvfb_1.16.4-1_amd64.deb /var/cache/apt/archives/
then you change your directory and do:
sudo dpkg -i xvfb_1.16.4-1_amd64.deb
and that's all!
I resolved my problems ( use selenium in scrapinghub )
1- for xvfb in docker i use
RUN apt-get install -qy xvfb
2- for creating docker image i used this
and for installing geckodriver i use this code
#
# Geckodriver Dockerfile
#
FROM blueimp/basedriver
# Add the Firefox release channel of the Debian Mozilla team:
RUN echo 'deb http://mozilla.debian.net/ jessie-backports firefox-release' >> \
/etc/apt/sources.list \
&& curl -sL https://mozilla.debian.net/archive.asc | apt-key add -
# Install Firefox:
RUN export DEBIAN_FRONTEND=noninteractive \
&& apt-get update \
&& apt-get install --no-install-recommends --no-install-suggests -y \
firefox \
# Remove obsolete files:
&& apt-get clean \
&& rm -rf \
/tmp/* \
/usr/share/doc/* \
/var/cache/* \
/var/lib/apt/lists/* \
/var/tmp/*
# Install geckodriver:
RUN export BASE_URL=https://github.com/mozilla/geckodriver/releases/download \
&& export VERSION=$(curl -sL \
https://api.github.com/repos/mozilla/geckodriver/releases/latest | \
grep tag_name | cut -d '"' -f 4) \
&& curl -sL \
$BASE_URL/$VERSION/geckodriver-$VERSION-linux64.tar.gz | tar -xz \
&& mv geckodriver /usr/local/bin/geckodriver
USER webdriver
CMD ["geckodriver", "--host", "0.0.0.0"]
from here