I'm having come trouble getting my tests to run within CircleCI. Locally the tests run fine, as shown on the right side of the screenshot. But in CircleCI, it will load the SuiteContext file and run the Before/AfterSuite methods, but nothing else. What would cause this sort of behavior? Thanks
default:
autoload: Test/Context
suites:
default:
paths:
features: Test/Features
contexts:
- SettingContext: ~
- LanguageContext: ~
- ActionContext: ~
- FrontendContext: ~
- FileTypeContext: ~
- FieldContext: ~
- ChannelContext: ~
- BrowserContext: ~
- SuiteContext: ~
- RequestContext: ~
- UrlContext: ~
- TemplateContext: ~
- PhraseContext: ~
- CategoryContext: ~
- EntryContext: ~
extensions:
Behat\MinkExtension:
base_url: http://ee300-clean.dev
selenium2: ~
Paths to files in Circle instance:
/home
/ubuntu
/project
/app
/bin
/behat
/behat.yml
/Test
/Context
/Features
circle.yml
test:
override:
- cd /home/ubuntu/project/app && bin/behat
I've changed that override command several times to explicitly set the path to the Features directory, but nothing I've tried works.
This ended up being a case sensitivity issue. Long ago I had renamed my features folder to Features, and locally on my Mac it still appeared as Features, however, it was committed to Git as features. Mac seemed to be more forgiving and still ran the tests even though the config file had "Features" in it, but Ubuntu server on Circle wasn't as forgiving.
Related
Right, so I have a Kotlin Gradle project with the framework SpringBoot. The project is on GitHub and our CI system is Circle CI.
At the build.gradle.kts file we have the following configs:
sonarqube {
properties {
property("sonar.projectKey", "Creditas_recupera-gateway")
property("sonar.organization", "creditas-fintech")
property("sonar.host.url", "https://sonarcloud.io")
property(
"sonar.coverage.jacoco.xmlReportPaths",
"$buildDir/reports/jacoco/jacocoTestReport/jacocoTestReport.xml"
)
}
}
fun ignorePackagesInJacocoReport(classDirectories: ConfigurableFileCollection) {
classDirectories.setFrom(
files(
classDirectories.files.map {
fileTree(it).apply {
exclude(
"**/company/**/*.java",
"**/company/**/*.kts",
"**/company/commons/**",
"**/company/**/config/**",
"**/company**/configuration/**",
"**/company/**/requests/**",
"**/company/**/responses/**",
"**/company/**/dto/**",
"**/company/**/infrastructure/scripts/**"
)
}
}
)
)
}
tasks.jacocoTestReport {
sourceSets(sourceSets.main.get())
executionData(fileTree(project.rootDir.absolutePath).include("**/build/jacoco/*.exec"))
reports {
xml.required.set(true)
html.required.set(true)
xml.outputLocation.set(File("$buildDir/reports/jacoco/jacocoTestReport/jacocoTestReport.xml"))
html.outputLocation.set(layout.buildDirectory.dir("$buildDir/reports/jacoco"))
}
ignorePackagesInJacocoReport(classDirectories)
}
Our config.yml used by Circle CI is like the following:
version: 2.1
orbs:
docker: circleci/docker#0.5.13
jobs:
unit_test:
steps:
- run:
name: Run tests
command: gradle test -i
- run:
name: Save test results
command: |
mkdir -p ~/junit/
find . -type f -regex ".*/build/test-results/.*xml"
find . -type f -regex ".*/build/test-results/.*xml" -exec cp {} ~/junit/ \;
when: always
- persist_to_workspace:
root: ~/
paths:
- project-folder/build/jacoco
coverage:
steps:
- run: ./gradlew dependencies build -x test -x detekt
- run:
name: Run coverage report
command: ./gradlew jacocoTestReport
- persist_to_workspace:
root: ~/
paths:
- recupera-gateway/build/reports/jacoco
sonarqube:
steps:
- restore_cache:
keys:
- app-dependencies-{{ checksum "build.gradle.kts" }}
- run: ./gradlew dependencies build -x test -x detekt
- save_cache:
paths:
- ~/.gradle
key: app-dependencies-{{ checksum "build.gradle.kts" }}
- attach_workspace:
at: ~/
- run:
name: Run sonar analysis
command: ./gradlew sonarqube
workflows:
version: 2.1
- unit_test
- coverage:
requires:
- unit_test
- sonarqube:
requires:
- coverage
Locally, running:
./gradlew jacocoTestReport
returns:
However, on SonarCloud, is the following:
Note that excluded files in Jacoco are present, and the total coverage is completely different.
In SonarCloud General Settings, I am passing JacocoTestReport.xml path to the file:
So, first, I have a question:
Does SonarCloud have the capacity to calculate the coverage by itself? Because it looks like it is taking the coverage results from somewhere else.
And second:
How do I make Jacoco's Coverage Report match with SonarCloud's?
Note that I have omitted many information for security and privacy reasons. If any crucial information is missing, please ask and I will see if it can be provided. Also, it might sound like a duplicate, however other topics found does not have a solution for the problem.
What happened is that, first of all, SonarCloud is unable to check if a file is being ignored or not from Jacoco's report, so it assumes it is 0% coverage. To solve this issue, it would be necessary to add the excluded files to Jacoco as well.
Another thing about the difference between Jacoco's coverage and SonarCloud's is that Jacoco also takes under consideration the instruction coverage, which is not represented on SonarCloud.
For a more complete answer I suggest to see this post on Sonar's forum here, all credits to that page.
I am currently desperate with Shopware 6 - I wrote a plugin that adds a menu item to the administration and makes it possible to create job postings.
Unfortunately, there is a JavaScript error hidden somewhere in the administration. Theoretically yes, no problem, because with the command...
bin/watch-adminstration.sh
...Shopware should generate source maps, which make debugging much easier for me. Here's the problem: the source maps are missing.
So one step back: How exactly do you debug the administration and where do you see the source maps?
I hope someone has a solution, the lack of documentation from Shopware is really getting on my nerves.
It would be great to know what setup you are using in order to achieve admin watching. I & Shopware recommends using the dockware.io docker images for development. You can read their documentation for more details. They also have a Makefile in the /var/www/ folder that can be ran to watch admin. Here is what it does:
watch-admin: ## starts watcher for Shopware 6.4.13.0 Admin at http://localhost:8888
cd /var/www/html && ./bin/build-administration.sh
cd /var/www/html && php bin/console bundle:dump
cd /var/www/html && php bin/console feature:dump
cd /var/www/html && APP_URL=http://0.0.0.0 PROJECT_ROOT=/var/www/html APP_ENV=dev PORT=8888 HOST=0.0.0.0 ENV_FILE=/var/www/html/.env ./bin/watch-administration.sh
As you can see, it starts the Admin on a different port than normal.
Here is an example docker-compose file for such a setup:
version: '3'
services:
shop:
container_name: shop
image: dockware/dev:latest
ports:
- "22:22" # ssh
- "80:80" # apache2
- "443:443" # apache2 https
- "8888:8888" # watch admin
- "9998:9998" # watch storefront proxy
- "9999:9999" # watch storefront
- "3306:3306" # mysql port
#volumes:
# - "./src:/var/www/html"
# - "./src:/var/www/html/custom/plugins"
networks:
- web
environment:
- XDEBUG_ENABLED=0
## ***********************************************************************
## NETWORKS
## ***********************************************************************
networks:
web:
external: false
Just SSH to the docker container after start, cd .. && make watch-admin. Then go to http://localhost:8888, this should load the admin page with source maps visible in the inspector tool.
I really dont know if this is a simple (must be), common or complex task.
I have a buildspec.yml file in my codebuild project, and i am trying to append the version written in package.json file to the output artifact.
I have already seen a lot of tutorials that teach how to append the date (not really useful to me), and others that tell me to execute a version.sh file with this
echo $(sed -nr 's/^\s*"version": "([0-9]{1,}.[0-9]{1,}.*)",$/\1/p' package.json)
and set it in a variable (it doesn't work).
i'm ending up with a build folder called: "my-project-$(version.sh)"
codebuild environment uses ubuntu and nodejs
Update (solved):
my version.sh file:
#!/usr/bin/env bash
echo $(sed -nr 's/^\s*\"version": "([0-9]{1,}\.[0-9]{1,}.*)",$/\1/p' package.json)
Then, i just found out 2 things:
Allow access to your version.sh file:
git update-index --add --chmod=+x version.sh
Declare a variable in any phase in buildspec, i dit in in build phase (just to make sure repository is already copied in environment)
TAGG=$($CODEBUILD_SRC_DIR/version.sh)
then reference it in artifact versioned name:
artifacts:
files:
- '**/*'
name: workover-frontend-$TAG
As result, my build artifact's name: myproject-1.0.0
In my case this script do not want to fetch data from package.json. On my local machine it working great but on AWS doesn't. I had to use chmod in different way, because i got message that i don't have right permissions. My buildspec:
version: 0.2
env:
variables:
latestTag: ""
phases:
pre_build:
commands:
- "echo sed version"
- sed --version
build:
commands:
- chmod +x version.sh
- latestTag=$($CODEBUILD_SRC_DIR/version.sh)
- "echo $latestTag"
artifacts:
files:
- '**/*'
discard-paths: yes
And results in console:
CodeBuild
I also have to mark that when i paste only for example echo 222 into version.sh file i got right answer in CodeBuild console.
I have a code using Fortran modules. I can build it with no problems under normal circumstances. CMake takes care of the ordering of the module files.
However, using a gitlab runner, it SOMETIMES happens that cmake does NOT order the Fortran modules by dependencies, but alphabetically instead, which than leads to a build failure.
The problem seems to occur at random. I have a branch that built in the CI. After adding a commit, that modified a utility script not involved in any way in the build, I ran into this problem. There is no difference in the output of the cmake configure step.
I use the matrix configuration for the CI to test different configurations. I found, that I could trigger this by adding another mpi version (e.g. openmpi/4.1.6). Without that version, it built. With it added in the matrix, ALL configurations showed the problem.
stages:
- configure
- build
- test
.basic_config:
tags:
- hpc_runner
variables:
# load submodules
GIT_SUBMODULE_STRATEGY: recursive
.config_matrix:
extends: .basic_config
# define job matrix
parallel:
matrix:
- COMPILER: [gcc/9.4.0]
PARALLELIZATION: [serial, openmpi/3.1.6]
TYPE: [option1, option2]
BUILD_TYPE: [debug, release]
- COMPILER: [gcc/10.3.0, intel/19.0.5]
PARALLELIZATION: [serial]
TYPE: [option2]
BUILD_TYPE: [debug]
###############################################################################
# setup script
# These commands will run before each job.
before_script:
- set -e
- uname -a
- |
if [[ "$(uname)" = "Linux" ]]; then
export THREADS=$(nproc --all)
elif [[ "$(uname)" = "Darwin" ]]; then
export THREADS=$(sysctl -n hw.ncpu)
else
echo "Unknown platform. Setting THREADS to 1."
export THREADS=1
fi
# load environment
- source scripts/build/load_environment $COMPILER $BUILD_TYPE $TYPE $PARALLELIZATION
# set path for build folder
- build_path=build/$COMPILER/$PARALLELIZATION/$TYPE/$BUILD_TYPE
configure:
stage: configure
extends: .config_matrix
script:
- mkdir -p $build_path
- cd $build_path
- $CMAKE_COMMAND
artifacts:
paths:
- build
expire_in: 1 days
###############################################################################
# build script
build:
stage: build
extends: .config_matrix
script:
- cd $build_path
- make
artifacts:
paths:
- build
expire_in: 1 days
needs:
- configure
###############################################################################
# test
test:
stage: test
extends: .config_matrix
script:
- cd $build_path
- ctest --output-on-failure
needs:
- build
The runner runs on an HPC machine which a complex setup, and I am not to familiar with the exact configuration. I contacted the admin with this problem, but wanted to see if anybody else had run into this before and have solutions or hints on what is going on.
With the help from our admin I figured it out.
The problem comes from cmake using absolute paths. The runner has actually several runners for parallel jobs, with each using a different prefix path, e.g. /runner/001/ or /runner/012/. So when I run configure on a specific runner, cmake saves that prefix path to the configuration.
Now in the build stage, there is no guarantee to have the same configuration run on the same runner. However, since there are absolute paths in the make files, make tries to access the folders in the configure runner's prefix. Now, that can be anything from non-existing, over old files from previous pipelines to the correct files downloaded by another case.
The only fix I currently can see is to run everything on the same runner in one stage, to avoid the roulette of prefix paths. If anybody has a different idea, or if there is a way to fix a specific matrix case to a specific runner prefix, please comment.
We use bitbucket pipeline for our angular 5 to deploy our code to GAE. We are ending up with the following exception. We use bitbucket pipelines the the CI/CD
THis is the pipeline code
image: node:9.11.1
pipelines:
custom:
default:
- step:
script:
- npm install -g #angular/cli#latest
- ng build --prod
- cp app.yaml dist
- ls dist
- cd dist
- curl -o /tmp/google-cloud-sdk.tar.gz https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-sdk-190.0.0-linux-x86_64.tar.gz
- tar -xvf /tmp/google-cloud-sdk.tar.gz -C /tmp/
- /tmp/google-cloud-sdk/install.sh -q
- source /tmp/google-cloud-sdk/path.bash.inc
- echo $GCLOUD_API_KEYFILE | base64 --decode --ignore-garbage > ./gcloud-api-key.json
- gcloud config set project $GCLOUD_PROJECT
- gcloud components install app-engine-java
- gcloud auth activate-service-account --key-file gcloud-api-key.json
- echo $GCLOUD_API_KEYFILE > /tmp/client-secret.json
- gcloud config set project $GCLOUD_PROJECT
- gcloud app update --split-health-checks --project adtecy-ui
- gcloud app deploy app.yaml
I am looking to use node docker image to deploy angular 5 (version 5.2.11) app to GAE flex environment but it takes an unusally long time and the status is still "In Progress" (not sure the usual deploy time)
This is my app.yaml file
env: flex
runtime: python
threadsafe: true
readiness_check:
timeout_sec: 4
check_interval_sec: 5
failure_threshold: 2
success_threshold: 2
app_start_timeout_sec: 3600
I have given a very high timeout period because the previous push failed with a timeout
I believe GAE by default use python and hence we did not install python
Right now the deployment is running for about 20 mins or so but without letting us know any result. Could you guys help in deploying my app to GAE with minimal time?
EDIT:
Now we have got the result after running for 33 minutes
`21df82f90a72: Layer already exists
aeb4b6656589: Pushed
latest: digest:
sha256:c57d3178321c5f2721fc70cd00cb7862d469c74a6bf616ecfda760342c13af7e size: 3255
DONE
--------------------------------------------------------------------------------
Updating service [default] (this may take several minutes)...
.failed.
ERROR: (gcloud.app.deploy) Operation [apps/adtecy-
ui/operations/9c273f87-91a3-495a-b75d-0d6c767dce97] timed out.
This operation may still be underway.`
You can check the status of the deploy operation by running
gcloud app operations "apps/adtecy-ui/operations/9c273f87-91a3-495a-b75d-0d6c767dce97"
there seems to be an issue with running the app is nodejs environment. So I switched to python27 and I was able to successfully deploy it. But when I try to load using the app served in GAE it is throwing an error.
https://adtecy-ui.appspot.com
And here is my app.yaml (And I made some modifications too)
runtime: python27
api_version: 1
threadsafe: true
handlers:
# Routing for bundles to serve directly
- url: /((?:inline|main|polyfills|styles|vendor)\.[a-z0-9]+\.bundle\.js)
secure: always
redirect_http_response_code: 301
static_files: dist/\1
upload: dist/.*
# Routing for a prod styles.bundle.css to serve directly
- url: /(styles\.[a-z0-9]+\.bundle\.css)
secure: always
redirect_http_response_code: 301
static_files: dist/\1
upload: dist/.*
# Routing for typedoc, assets and favicon.ico to serve directly
- url: /((?:assets|docs)/.*|favicon\.ico)
secure: always
redirect_http_response_code: 301
static_files: dist/\1
upload: dist/.*
# Any other requests are routed to index.html for angular to
handle so we don't need hash URLs
- url: /.*
secure: always
redirect_http_response_code: 301
static_files: dist/index.html
upload: dist/index\.html