Error while write the acceptance test with newman Rest APi - testing

I am new to docker and testing through newman
while running test cases the service of /services/c/d/docker/docker-compose.yml
getting following error
getaddrinfo ENOTFOUND
Error while write the acceptance test with newman
I am new to docker and testing through newman
while running test cases the service of /services/c/d/docker/docker-compose.yml
getting following error
getaddrinfo ENOTFOUND
Following is the Snippet
runAcceptanceTests.sh
#!/bin/bash
set -e
if [ "${ENVIRONMENT}" == 'localhost' ]; then
../gradlew --project-dir ../ --build-cache assemble
docker-compose -f docker-compose.yml \
-f docker-compose.localhost.yml \
-f ../services/a/b/docker/docker-compose.yml \
-f ../services/c/d/docker/docker-compose.yml \
build --force-rm api
docker-compose -f docker-compose.yml \
-f docker-compose.localhost.yml \
-f ../services/a/b/docker/docker-compose.yml \
-f ../services/c/d/docker/docker-compose.yml \
kill api
docker-compose -f docker-compose.yml \
-f docker-compose.localhost.yml \
-f ../services/a/b/docker/docker-compose.yml \
-f ../services/c/d/docker/docker-compose.yml \
run --rm \
-e a="${a}" \
-w /opt/code/api \
apibuild ./acceptance-tests/newman/wait-for-api.sh ./acceptance-tests/newman/a.sh
else
docker-compose -f docker-compose.yml \
run --rm \
-e a="${a}" \
-w /opt/code/api \
apibuild ./acceptance-tests/newman/a.sh
fi
a.sh
#!/bin/bash
set -e
readonly NEWMAN_COLLECTION_PATH=${ACCEPTANCE_TEST_PATH:-./acceptance-tests/newman}
readonly CI=${CI:-false}
if $CI && [[ "${ENVIRONMENT,,}" == "abc" ]]; then
echo "Acceptance tests are disabled ."
exit 0
fi
echo "Running :" && \
newman run "${NEWMAN_COLLECTION_PATH}/abc.postman_collection.json" \
--folder=ccc \
-e "${NEWMAN_COLLECTION_PATH}/zzz-${ENVIRONMENT}-acceptance-test.postman_environment.json" \
--global-var="config_key=${a}" -k
wait-for-api.sh
!/bin/sh
set -e
until curl -q -f --max-time 5 'http://localhost:8080/health/ping' &>/dev/null; do
>&2 echo "API is unavailable yet - sleep 5s"
sleep 5
done
>&2 echo " API is up - executing command $#"
exec "$#
docker-compose.yml
version: '3.2'
services:
apibuild:
build: ../ci-cd-pipeline/docker/build-agent
image: api-build-agent:develop
hostname: buildagent
container_name: apibuildagent
volumes:
# mount entire folder
- api/:/opt/code/api/:delegated
environment:
- a=${a:-int}
- JAVA_hostname=`hostname`
security_opt:
- apparmor:unconfined
docker-compose.localhost.yml
version: '3.2'
services:
apibuild:
depends_on:
- api
Trying to add newman test for 2nd service my component consist of two services as follow
1 https://localhost:8080
2 https://localhost:8081enter image description here
i am trying to write the test for 2nd service but getting the error
❏ flow
↳ CheckFlow
POST https://localhost:8081/key/check [errored]
getaddrinfo ENOTFOUND service
could any one help me out regarding it

Related

This GitLab CI configuration is invalid: jobs stage config should implement a script: or a trigger: keyword

i have been facing the above error for the below yaml file
before running the pipeline and getting this error
stage: deploy
stages:
- deploy
Deploy: ~
before_script:
- "command -v ssh-agent >/dev/null || ( apk add --update openssh )"
- "eval $(ssh-agent -s)"
- "echo \"$SSH_PRIVATE_KEY\" | tr -d '\\r' | ssh-add -"
- "mkdir -p ~/.ssh"
- "chmod 700 ~/.ssh"
- "ssh-keyscan $EC2_IPADDRESS >> ~/.ssh/known_hosts"
- "chmod 644 ~/.ssh/known_hosts"
script:
- "mkdir .public"
- "cp -r * .public"
- "mv .public public"
- "zip -r public.zip public"
- "scp -o StrictHostKeyChecking=no public.zip ubuntu#3.129.128.56:/var/www/html"
- "ssh -o StrictHostKeyChecking=no ubuntu#3.129.128.56 \"cd /var/www/html; touch foo.txt; unzip public.zip\""
You can see in this question or this GitLab thread examples of SSH-based pipelines.
In each case, there is no Deploy: ~, so try first without this line.

AWS Codebuild build stage shows nginx was installed but container doesn't have nginx

I'm new with AWS Codebuild and trying to deploy my docker application in EKS. On my Dockerfile I'm running RUN apk add nginx and build stage shows that it's being installed.
Step 6/24 : RUN apk add nginx
---> Running in cbd7a4e37bb7
(1/2) Installing pcre (8.44-r0)
(2/2) Installing nginx (1.18.0-r15)
Executing nginx-1.18.0-r15.pre-install
Executing nginx-1.18.0-r15.post-install
Executing busybox-1.32.1-r6.trigger
OK: 15 MiB in 34 packages
Removing intermediate container cbd7a4e37bb7
---> a890efd77e97
After the BUILD stage has been completed, I enter my EKS pod but didn't see any nginx under /etc
/etc # ls
ImageMagick-7 fstab issue my.cnf.d periodic securetty sysctl.d
alpine-release group logrotate.d mysql php7 services terminfo
apk group- modprobe.d network pkcs11 shadow udhcpd.conf
ca-certificates hostname modules openldap profile shadow- wgetrc
ca-certificates.conf hosts modules-load.d opt profile.d shells
conf.d init.d motd os-release protocols ssh
crontabs inittab mtab passwd resolv.conf ssl
fonts inputrc my.cnf passwd- rsyncd.conf sysctl.conf
I manually executed apk add nginx inside my pod and that's the only time nginx appeared under /etc
My AWS Codebuild is using the following configurations:
Managed image
OS: Amazon Linux 2
Runtime: Standard
Image: aws/codebuild/amazonlinux2-x86_64-standard:3.0
Image version: Always use the latest image for this runtime version
Privilaged: Enabled
Also,
I noticed that AWS Build doesn't execute the following COPY command in my Dockerfile:
COPY docker/php/php.ini /usr/local/etc/php/ (this works)
COPY docker/nginx/conf.d/default.conf /etc/nginx/conf.d/default.conf (this doesn't, probably because nginx didn't get installed on runtime)
COPY docker/nginx/docker-entrypoint.sh /usr/local/bin/docker-entrypoint-d9 (this doen't work.. still a mystery)
I hope I was able to provide the full details. Please let me know if you need more information. Thank you.
buildspec.yml
version: 0.2
env:
variables:
PHP_VERSION: 7.4
APP_HOME: /var/www/html
phases:
pre_build:
commands:
- echo Logging in to Amazon ECR...test4
- aws --version
- docker --version
- echo $AWS_DEFAULT_REGION
- $(aws ecr get-login --region $AWS_DEFAULT_REGION --no-include-email)
- export KUBECONFIG=$HOME/.kube/config
- export PHP_VERSION=$PHP_VERSION
- REPOSITORY_URI=xxxxxxx.xxx.xxx.xx-xxxxxxx-x.amazonaws.com/development-ecr
- COMMIT_HASH=$(echo $CODEBUILD_RESOLVED_SOURCE_VERSION | cut -c 1-7)
- IMAGE_TAG=$CODEBUILD_BUILD_NUMBER
# - IMAGE_TAG=build-$(echo $CODEBUILD_BUILD_ID | awk -F":" '{print $2}')
build:
commands:
- echo "PHP Version is $PHP_VERSION"
- echo "$APP_HOME"
- echo Running Docker compose..
- echo Build started on `date`
- echo Building the Docker image...
- docker build --build-arg PHP_VERSION=${PHP_VERSION} -t $REPOSITORY_URI:latest test/
- docker tag $REPOSITORY_URI:latest $REPOSITORY_URI:$IMAGE_TAG
post_build:
commands:
- echo Build completed on `date`
- echo Pushing the Docker images...
- docker push $REPOSITORY_URI:latest
- docker push $REPOSITORY_URI:$IMAGE_TAG
- echo "Setting Environment Variables related to AWS CLI for Kube Config Setup"
- CREDENTIALS=$(aws sts assume-role --role-arn $EKS_KUBECTL_ROLE_ARN --role-session-name codebuild-kubectl --duration-seconds 900)
- export AWS_ACCESS_KEY_ID="$(echo ${CREDENTIALS} | jq -r '.Credentials.AccessKeyId')"
- export AWS_SECRET_ACCESS_KEY="$(echo ${CREDENTIALS} | jq -r '.Credentials.SecretAccessKey')"
- export AWS_SESSION_TOKEN="$(echo ${CREDENTIALS} | jq -r '.Credentials.SessionToken')"
- export AWS_EXPIRATION=$(echo ${CREDENTIALS} | jq -r '.Credentials.Expiration')
# Setup kubectl with our EKS Cluster
- echo "Update Kube Config"
- aws eks update-kubeconfig --name $EKS_CLUSTER_NAME
# Apply changes to our Application using kubectl
- echo "Delete current deployment"
- kubectl delete deployment dev-spectrum-app
- sleep 10
- echo "Apply changes to kube manifests"
- kubectl apply -f kube-yaml/
- echo "Completed applying changes to Kubernetes Objects"
pipeline for other stages
- echo Writing image definitions file...
- printf '[{"name":"dev-spectrum","imageUri":"%s"}]' $REPOSITORY_URI:$IMAGE_TAG > imagedefinitions.json
- cat imagedefinitions.json
artifacts:
files: imagedefinitions.json
Dockerfile
ARG PHP_VERSION=7.4
FROM php:${PHP_VERSION:+${PHP_VERSION}-}fpm-alpine
ARG APP_HOME
RUN echo "CHECKING ARGS: $APP_HOME ${PHP_VERSION}"
RUN apk update; \
apk upgrade;
RUN apk add nginx
# Install library extension
RUN apk add libpng libpng-dev libjpeg-turbo libjpeg-turbo-dev freetype-dev libwebp-dev zlib-dev libxpm-dev
# Install MySql
RUN docker-php-ext-install mysqli pdo pdo_mysql
RUN docker-php-ext-install opcache
RUN docker-php-ext-configure gd --with-jpeg --with-freetype \
&& docker-php-ext-install gd
# PHP packages
RUN apk add libressl \
ca-certificates \
openssh-client \
rsync \
git \
curl \
wget \
gzip \
tar \
patch \
perl \
pcre \
imap \
imagemagick \
mariadb-client \
build-base \
autoconf \
libtool \
php7-dev \
pcre-dev \
imagemagick-dev \
php7 \
php7-fpm \
php7-opcache \
php7-session \
php7-dom \
php7-xml \
php7-xmlreader \
php7-ctype \
php7-ftp \
php7-json \
php7-posix \
php7-curl \
php7-pdo \
php7-pdo_mysql \
php7-sockets \
php7-zlib \
php7-mcrypt \
php7-mysqli \
php7-sqlite3 \
php7-bz2 \
php7-phar \
php7-openssl \
php7-posix \
php7-zip \
php7-calendar \
php7-iconv \
php7-imap \
php7-soap \
php7-dev \
php7-pear \
php7-redis \
php7-mbstring \
php7-xdebug \
php7-exif \
php7-xsl \
php7-ldap \
php7-bcmath \
php7-memcached \
php7-oauth \
php7-apcu
# Install Composer
RUN curl -sS https://getcomposer.org/installer | php && \
mv composer.phar /usr/local/bin/composer && \
ln -s /root/.composer/vendor/bin/drush /usr/local/bin/drush
# Install Drush
RUN composer global require drush/drush && \
composer global update
# Copy php.ini memory limit increase
COPY docker/php/php.ini /usr/local/etc/php/
COPY docker/nginx/conf.d/default.conf /etc/nginx/conf.d/default.conf
COPY docker/nginx/docker-entrypoint.sh /usr/local/bin/docker-entrypoint.sh
#COPY docker/nginx/docker-entrypoint.sh /usr/local/bin/docker-entrypoint-d9
RUN echo ${APP_HOME} ${PHP_VERSION}
WORKDIR /var/www/html
RUN mkdir /run/nginx;
#Copy composer files
COPY composer.json composer.lock ./
RUN composer install --prefer-dist --no-interaction
RUN composer dump-autoload
RUN set -eux; \
chmod +x /usr/local/bin/docker-entrypoint.sh
# chmod +x /usr/local/bin/docker-entrypoint-d9
EXPOSE 80 433
CMD ["nginx"]
#RUN nginx
#RUN php-fpm
ENTRYPOINT ["docker-entrypoint.sh"]

Why Gitlab CI docker build tagging problem

I have been trying to build my project and deployment to a remote server using gitlab CI runner and using this link as reference
https://www.digitalocean.com/community/tutorials/how-to-set-up-a-continuous-deployment-pipeline-with-gitlab-ci-cd-on-ubuntu-18-04
After runing the pipeline, the publish stage is giving error about the docker tagging
$ docker build -t $TAG_COMMIT -t $TAG_LATEST .
invalid argument "/patch-9:64a25b49" for "-t, --tag" flag: invalid reference format
I have tried changing the docker build tagging in different formats but still could not find out why the error.
I have tried changing the tagging
TAG_LATEST: ${CI_REGISTRY_IMAGE}/${CI_COMMIT_REF_NAME}:latest
TAG_COMMIT:${CI_REGISTRY_IMAGE}/${CI_COMMIT_REF_NAME}:${CI_COMMIT_SHORT_SHA}
but I am still get the error
$ cd $GOPATH/src/$REPO/$NAMESPACE/$PROJECT
$ docker build -t $TAG_COMMIT -t $TAG_LATEST .
invalid argument "/patch-10:fbf4855b" for "-t, --tag" flag: invalid reference format
See 'docker build --help'.
Can anyone help me solve this problem?
My .gitlab-ci.yml file looks
image: golang:1.15.3
variables:
REPO: github.com
NAMESPACE: daniel
PROJECT: danapp
TAG_LATEST: $CI_REGISTRY_IMAGE/$CI_COMMIT_REF_NAME:latest
TAG_COMMIT: $CI_REGISTRY_IMAGE/$CI_COMMIT_REF_NAME:$CI_COMMIT_SHORT_SHA
before_script:
- mkdir -p $GOPATH/src/$REPO/$NAMESPACE/$PROJECT
- cp -r -v $CI_PROJECT_DIR $GOPATH/src/github.com/daniel
- cd $GOPATH/src/$REPO/$NAMESPACE/$PROJECT
stages:
- build
- publish
- deploy
compile:
stage: build
script:
- go build -race -ldflags "-extldflags '-static'" -o $CI_PROJECT_DIR/danapp
artifacts:
paths:
- danapp
publish:
image: docker:latest
stage: publish
services:
- docker:dind
script:
- docker build -t $TAG_COMMIT -t $TAG_LATEST .
- docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN $CI_REGISTRY
- docker push $TAG_COMMIT
- docker push $TAG_LATEST
deploy:
image: alpine:latest
stage: deploy
tags:
- deployment
before_script:
- apk update && apk add openssh-client
- eval $(ssh-agent -s)
- echo "$SSH_PRIVATE_KEY" | tr -d '\r' | ssh-add -
- mkdir -p ~/.ssh
- chmod 700 ~/.ssh
script:
- chmod og= $SSH_PRIVATE_KEY
- apk update && apk add openssh-client
- ssh -i $SSH_PRIVATE_KEY -o StrictHostKeyChecking=no admin#192.168.x.x "docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN $CI_REGISTRY"
- ssh -i $SSH_PRIVATE_KEY -o StrictHostKeyChecking=no admin#192.168.x.x "docker pull $TAG_COMMIT"
- ssh -i $SSH_PRIVATE_KEY -o StrictHostKeyChecking=no admin#192.168.x.x "docker container rm -f danapp || true"
- ssh -i $SSH_PRIVATE_KEY -o StrictHostKeyChecking=no admin#192.168.x.x "docker run -d -p 20005:20005 --name danapp $TAG_COMMIT"
environment: stagging
only:
- master
I suggest you change your $CI_COMMIT_REF_NAME to $CI_COMMIT_REF_SLUG
Maybe this solved.

Extracting ZAP report running in container on Jenkins agent (docker based)

My setup is as follows:
Jenkins pipeline script which triggers Jenkins job which runs inside a dokcer container.
ZAP is in containerzied mode
Commands used:
echo DEBUG - mkdir -p $PWD/out
mkdir -p $PWD/out
echo DEBUG - chmod 777 $PWD/out
chmod 777 $PWD/out
test -d ${PWD}/out \
&& docker run -v $(pwd)/out:/zap/wrk/:rw -t owasp/zap2docker-live zap-api-scan.py -t $TARGET_URL -f openapi -d -r zap_scan_report.html
Also tried: docker run --user $(id -u):$(id -g) -v $(pwd)/out:/zap/wrk/:rw -t owasp/zap2docker-live zap-api-scan.py -t $TARGET_URL -f openapi -d -r zap_scan_report.html
Scan works fine but report is not in the "out" directory.
This works fine on a VM environment
Any suggestions as I guess the mount is not working in a docker container

How to setup an variable ENV inside Dockerfile to be overriden in a Docker container?

I have a container that should be created with the IP user.
This is what i have inside the Dockerfile:
ENV REMOTE_HOST=xxxxxxxxxx
RUN { \
echo '[xdebug]'; \
echo 'zend_extension=/usr/local/lib/php/extensions/no-debug-non-zts-20131226/xdebug.so'; \
echo 'xdebug.remote_enable=1'; \
echo 'xdebug.remote_port=9000'; \
echo 'xdebug.remote_autostart=1'; \
echo 'xdebug.remote_handler=dbgp'; \
echo 'xdebug.idekey=dockerdebug'; \
echo 'xdebug.profiler_output_dir="/var/www/html"'; \
echo 'xdebug.remote_connect_back=0'; \
echo 'xdebug.remote_host=$REMOTE_HOST'; \
} >> /usr/local/etc/php/php.ini
And this is how I create an container derived from that Dockerfile:
dockerrun an image from that Dockerfile:
docker run -e REMOTE_HOST=123456 -p 80:80 -v /Users/myusrname/Documents/mysite:/var/www/html myImage
This is what I have in php.ini inside the container:
root#1713e0a338f9:/var/www/html# cat /usr/local/etc/php/php.ini
...
[xdebug]
zend_extension=/usr/local/lib/php/extensions/no-debug-non-zts-20131226/xdebug.so
xdebug.remote_enable=1
xdebug.remote_port=9000
xdebug.remote_autostart=1
xdebug.remote_handler=dbgp
xdebug.idekey=dockerdebug
xdebug.profiler_output_dir="/var/www/html"
xdebug.remote_connect_back=0
xdebug.remote_host=$REMOTE_HOST
What is the correct way to pass that variable?
If you're attempting to pass in data that will be built with the image then you're looking for ARG and --build-arg; which can be found in the Dockerfile documentation.
Dockerfile:
FROM ubuntu
ARG REMOTE_HOST
RUN echo ${REMOTE_HOST} > /my_file
Then build the file:
➜ docker build -t test_image --build-arg REMOTE_HOST=1.2.3.4 .
Sending build context to Docker daemon 10.24kB
Step 1/3 : FROM ubuntu
---> 20c44cd7596f
Step 2/3 : ARG REMOTE_HOST
---> Using cache
---> f9815e560847
Step 3/3 : RUN echo ${REMOTE_HOST} > /my_file
---> Running in da07d5d060b7
---> cdfdbeac71b9
Run the image and print out the file:
➜ docker run test_image cat /my_file
1.2.3.4
I'll leave this with a note though; that you probably don't want to be hard-coding an IP address to your image; and instead you should set up your image to instead read the environment variable and update that file at runtime; in that case, once your Dockerfile is setup to handle that - you would use docker run -e REMOTE_HOST=1.2.3.4. Do this, you'll want something like:
Dockerfile:
FROM ubuntu
COPY php.ini /usr/local/etc/php/php.ini
COPY entrypoint.sh /usr/local/bin/
RUN chmod +x /usr/local/bin/entrypoint.sh # Or ensure it's +x already
ENTRYPOINT [ "entrypoint.sh" ]
php.ini
{
echo '[xdebug]';
echo 'zend_extension=/usr/local/lib/php/extensions/no-debug-non-zts-20131226/xdebug.so';
echo 'xdebug.remote_enable=1';
echo 'xdebug.remote_port=9000';
echo 'xdebug.remote_autostart=1';
echo 'xdebug.remote_handler=dbgp';
echo 'xdebug.idekey=dockerdebug';
echo 'xdebug.profiler_output_dir="/var/www/html"';
echo 'xdebug.remote_connect_back=0';
echo 'xdebug.remote_host=$REMOTE_HOST';
}
entrypoint.sh
#!/bin/bash
set -e
# Check if our environment variable has been passed.
if [ -z "${REMOTE_HOST}" ]
then
echo "REMOTE_HOST has not been set."
exit 1
else
sed -i.bak "s/\$REMOTE_HOST/${REMOTE_HOST}/g" /usr/local/etc/php/php.ini
fi
exec "$#"
Build the image:
➜ docker build -t test_image .
Sending build context to Docker daemon 4.608kB
Step 1/5 : FROM ubuntu
---> 20c44cd7596f
Step 2/5 : COPY php.ini /usr/local/etc/php/php.ini
---> 1785c0238ce8
Step 3/5 : COPY entrypoint.sh /usr/local/bin/
---> c63c289c411e
Step 4/5 : RUN chmod +x /usr/local/bin/entrypoint.sh # Or ensure it's +x already
---> Running in 09b07f8724a9
---> 66ab090f405a
Removing intermediate container 09b07f8724a9
Step 5/5 : ENTRYPOINT entrypoint.sh
---> Running in 1f5a7ebec98e
---> 2992128843cd
Removing intermediate container 1f5a7ebec98e
Successfully built 2992128843cd
Successfully tagged test_image:lates
Run the image and provide REMOTE_HOST
➜ docker run -e REMOTE_HOST=1.2.3.4 test_image cat /usr/local/etc/php/php.ini
{
echo '[xdebug]';
echo 'zend_extension=/usr/local/lib/php/extensions/no-debug-non-zts-20131226/xdebug.so';
echo 'xdebug.remote_enable=1';
echo 'xdebug.remote_port=9000';
echo 'xdebug.remote_autostart=1';
echo 'xdebug.remote_handler=dbgp';
echo 'xdebug.idekey=dockerdebug';
echo 'xdebug.profiler_output_dir="/var/www/html"';
echo 'xdebug.remote_connect_back=0';
echo 'xdebug.remote_host=1.2.3.4';
}
If you run
docker run --build-arg REMOTE_HOST=123456 ....
Then your docker file should have the following line to retrieve the environment variable.
ARG REMOTE_HOST
ENV REMOTE_HOST=$REMOTE_HOST
And you should be able to print it using a php echo like the following:
echo 'xdebug.remote_host='.getenv('REMOTE_HOST');
However if you just want to pass a host to the container you should be using the --add-host argument when running "docker run". Something like the following:
docker run --add-host=remote_host:10.180.0.1 ....