Azure pipeline : Command Line ##[error]Bash exited with code '1' - npm

I am trying to execute my cypress test using azure pipeline but while executing job facing below issue:
yml file:
# To configure triggers for Azure CI see
# https://docs.microsoft.com/en-us/azure/devops/pipelines/build/triggers?view=azure-devops&tabs=yaml#tags
jobs:
- job: Cypress_tests
pool:
vmImage: 'ubuntu-22.04'
# Runs tests in parallel https://docs.cypress.io/guides/guides/parallelization
# https://learn.microsoft.com/en-us/azure/devops/pipelines/process/phases?view=azure-devops&tabs=yaml
strategy:
parallel: 2
steps:
- task: NodeTool#0
# Caches dependencies using npm lock file as key
# https://docs.cypress.io/guides/continuous-integration/introduction#Caching
- task: CacheBeta#1
inputs:
key: npm | package-lock.json
path: /home/vsts/.npm
restoreKeys: npm | package-lock.json
- task: CacheBeta#1
inputs:
key: cy | package-lock.json
path: /home/vsts/.cache/Cypress
restoreKeys: cy | package-lock.json
- script: npm ci --prefer-offline
# Starts web server for E2E tests - replace with your own server invocation
# https://docs.cypress.io/guides/continuous-integration/introduction#Boot-your-server
- script: npm start &
- script: npx wait-on 'http-get://localhost:3000' # Waits for above
# Runs tests in parallel and records to Cypress Cloud
# https://docs.cypress.io/guides/cloud/projects#Set-up-a-project-to-record
# https://docs.cypress.io/guides/guides/parallelization
- script: npx cypress run --record --parallel --ci-build-id $BUILD_BUILDNUMBER
# For recording and parallelization to work you must set your CYPRESS_RECORD_KEY
# in Azure DevOps → Your Pipeline → Edit → Variables
env:
CYPRESS_RECORD_KEY: $(CYPRESS_RECORD_KEY)
Error:
Starting: CmdLine
==============================================================================
Task : Command line
Description : Run a command line script using Bash on Linux and macOS and cmd.exe on Windows
Version : 2.212.0
Author : Microsoft Corporation
Help : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/command-line
==============================================================================
Generating script.
Script contents:
npm ci --prefer-offline
========================== Starting Command Output ===========================
/usr/bin/bash --noprofile --norc /home/vsts/work/_temp/82c8b231-ced2-40c3-9bca-ec69d4282405.sh
Usage: npm <command>
where <command> is one of:
access, adduser, bin, bugs, c, cache, completion, config,
ddp, dedupe, deprecate, dist-tag, docs, edit, explore, get,
help, help-search, i, init, install, install-test, it, link,
list, ln, login, logout, ls, outdated, owner, pack, ping,
prefix, prune, publish, rb, rebuild, repo, restart, root,
run, run-script, s, se, search, set, shrinkwrap, star,
stars, start, stop, t, tag, team, test, tst, un, uninstall,
unpublish, unstar, up, update, v, version, view, whoami
npm <cmd> -h quick help on <cmd>
npm -l display full usage info
npm help <term> search for help on <term>
npm help npm involved overview
Specify configs in the ini-formatted file:

The NodeTool#0 without any parameters sets the active node version to 6, the ci command probably didn't exist back then. Make sure you select a recent version of Node:
versionSpec - Version Spec
string. Optional. Use when versionSource = spec. Default value: 6.x.
Specifies the version spec of the version to get. Examples: 6.x, 4.x, 6.10.0, >=6.10.0.
- task: NodeTool#0
inputs:
versionSpec: "18.x"

Related

CodeBuild: Always run the 'artifacts' phase regardless of success or failure

I'm using an Jenkins pipeline to trigger AWS CodeBuild, and in my buildspec I run some tests that I wish to publish as artefacts so that they can be downloaded and read by Jenkins.
When all of my tests pass, this works just fine. However, when one or more tests fail, it seems as though the artifacts phase is ignored, so there are no artefacts for Jenkins to download.
Though it's not what I require, I have also attempted to use the reports phase, but that behaves in the exact same way, which I find confusing as it seems crazy to fail on a test and then not publish the reports.
Is it possible to make CodeBuild execute the artifacts phase regardless of success or failure?
version: 0.2
env:
shell: bash
phases:
install:
runtime-versions:
python: latest
commands:
- pip install cfn-lint checkov
- ...
pre_build:
commands:
- cd myproj
- cfn-lint --template cloudformation/template.cfn.yaml --format junit > cfn-lint.xml
- checkov --directory cloudformation --framework cloudformation secrets --output=junitxml > checkov.xml
build:
commands:
- ...
post_build:
commands:
- ...
artifacts:
base-directory: myproj
files:
- cfn-lint.xml
- checkov.xml
The answer here is that the artefacts (and reports) phase is not run if the pre_build phase fails.
https://docs.aws.amazon.com/codebuild/latest/userguide/view-build-details.html#view-build-details-phases
While I've technically answered my question, this means that I've had to move tests into the build phase, which feels wrong because the post_build phase is run regardless of success or failure, so publishing of my artefact (outside of AWS) also fails.

Execute GitLab job only if files have changed in a subdirectory, otherwise use cached artefact in following job

I have a simple pipeline, comparable to this one:
image: docker:20
variables:
GIT_STRATEGY: clone
stages:
- Building - Frontend
- Building - Backend
include:
- local: /.ci/extensions/ci-variables.yml
- local: /.ci/extensions/docker-login.yml
Build Management:
stage: Building - Frontend
image: node:14-buster
script:
# Install needed dependencies for building
- apt-get update
- apt-get -y upgrade
- apt-get install -y build-essential
- yarn global add #quasar/cli
- yarn global add #vue/cli
# Install required modules
- cd ${CI_PROJECT_DIR}/resources/js/management
- npm ci --cache .npm --prefer-offline
# Build project
- npm run build
# Create archive
- tar czf ${CI_PROJECT_DIR}/dist-resources-js-management.tar.gz *
cache:
policy: pull-push
key:
files:
- ./resources/js/management/package-lock.json
paths:
- ./resources/js/management/.npm/
artifacts:
paths:
- dist-resources-js-management.tar.gz
Build Docker:
stage: Building - Backend
needs: [Build Management, Build Administration]
dependencies:
- Build Management
- Build Administration
variables:
CI_REGISTRY_IMAGE_COMMIT_SHA: !reference [.ci-variables, variables, CI_REGISTRY_IMAGE_COMMIT_SHA]
CI_REGISTRY_IMAGE_REF_NAME: !reference [.ci-variables, variables, CI_REGISTRY_IMAGE_REF_NAME]
before_script:
- !reference [.docker-login, before_script]
script:
- mkdir -p {CI_PROJECT_DIR}/public/static/management
- tar xzf ${CI_PROJECT_DIR}/dist-resources-js-management.tar.gz --directory ${CI_PROJECT_DIR}/public/static/management
- docker build
--pull
--label "org.opencontainers.image.title=$CI_PROJECT_TITLE"
--label "org.opencontainers.image.url=$CI_PROJECT_URL"
--label "org.opencontainers.image.created=$CI_JOB_STARTED_AT"
--label "org.opencontainers.image.revision=$CI_COMMIT_SHA"
--label "org.opencontainers.image.version=$CI_COMMIT_REF_NAME"
--tag "$CI_REGISTRY_IMAGE_COMMIT_SHA"
-f .build/Dockerfile
.
I now want the first job to be executed under the following conditions:
Something has changed in the directory ${CI_PROJECT_DIR}/resources/js/management
This job has not yet created an artifact.
The last job should therefore always be able to access an artifact. If nothing has changed in the directory, it does not have to be created anew each time. If it did not exist before, it must of course be created.
Is there a way to map this in the GitLab Ci?
If I currently specify the dependencies and then work with only:changes: for the first job, GitLab complains if the job is not executed. Likewise with needs:.

Vue deployment to Heroku with CircleCI failing: fatal: Not a git repository (or any of the parent directories): .git

Having trouble deploying to HEROKU with CircleCI. I have already tested deploying git push heroku master to heroku manually, which is working. However when I use CircleCI, deployment no longer works.
Github repo url: https://github.com/dulerong/vue-test-circleci
I have set HEROKU environment variables in CircleCI project setting.
HEROKU_API_KEY=my_key
HEROKU_APP_NAME=my_app_name
Error message follows.
#!/bin/bash -eo pipefail
if false;then
force="-f"
fi
heroku_url="https://heroku:$HEROKU_API_KEY#git.heroku.com/$HEROKU_APP_NAME.git"
if [ -n "$CIRCLE_BRANCH" ]; then
git push $force $heroku_url $CIRCLE_BRANCH:main
elif [ -n "$CIRCLE_TAG" ]; then
git push $force $heroku_url $CIRCLE_TAG^{}:main
else
echo "No branch or tag found."
exit 1
fi
fatal: Not a git repository (or any of the parent directories): .git
Exited with code exit status 128
CircleCI received exit code 128
Below is my circleCI config.yml
version: 2.1
orbs:
heroku: circleci/heroku#1.2.5
jobs:
build-job:
working_directory: ~/repo
docker:
- image: circleci/node:12.18.2
steps:
- checkout
- run:
name: Install dependencies
command: npm install
- run:
name: Build
command: npm run build
- save_cache:
key: dependency-cache-{{ checksum "package-lock.json" }}
paths:
- ./node_modules
- run:
name: lint
command: npm run lint
- run:
name: test
command: npm run test:unit
deploy-job:
working_directory: ~/repo
docker:
- image: circleci/node:12.18.2
steps:
- attach_workspace:
at: ~/repo
- heroku/deploy-via-git
workflows:
version: 2.1
deploy:
jobs:
- build-job
- deploy-job:
requires:
- build-job
filters:
branches:
only: master
I have linked CircleCI to my Github repo
I have created .circleci folder config.yml
I have created an app on heroku, which works when I deploy manually
The build part of my CircleCI works, however deployment does not work
Any help is appreciated, thanks in advance.
Found out what I was missing.
- checkout
I was missing this line of code in my deploy-job. Hence after changing the deploy-job yml config code to following, everything worked.
deploy-job:
working_directory: ~/repo
docker:
- image: circleci/node:12.18.2
steps:
- checkout<--- INSERT THIS CODE HERE!!!!
- attach_workspace:
at: ~/repo
- heroku/deploy-via-git
Reason: checkout command leads CircleCI to the root directory of your project. Hence without this line of code, you're looking at a folder directory that's not even the root of your project.
Other useful command include
- run:
name: show directory
command: pwd
- run:
name: look in directory
command: ls -ltr
If you place those commands beneath checkouk, and look into the job progress in your CircleCI project, you can actually see which directory CircleCI is looking at, during that exact moment, very useful to check which directory CircleCI is working in. I put in those two commands and found out that my CircleCI was not looking at the root directory, hence discovering my problem.
Took me a few hours to figure this out!!!

Phoenix framework - execute NPM script in subdirectory

In my Phoenix app, I am running a React frontend that sits in an assets/ directory within the app... it has it's own package.json file, and I would like to be able to run aliased NPM scripts from the Phoenix app's mix.exs file. I've been experimenting with Elixir's System.cmd(), but it doesn't seem to be doing the trick.
Basically, I've set up a lint command for my frontend - in assets/package.json, I've got
"lint": "eslint ./*.js ./js/*.js || exit 0"
and running $ npm run lint from the assets/ directory works as expected... but it would be smashing to just be able to run the command from the app's top-level.
In my mix.exs file, I was experimenting with an alias like so:
defp aliases do
[
"lint": [System.cmd("npm", ["run lint"], cd: "assets")
]
end
but running mix lint produces a pretty wordy error:
** (FunctionClauseError) no function clause matching in Mix.Task.run_alias/3
The following arguments were given to Mix.Task.run_alias/3:
# 1
[{"\nUsage: npm <command>\n\nwhere <command> is one of:\n access, adduser, bin, bugs, c, cache, completion, config,\n ddp, dedupe, deprecate, dist-tag, docs, edit, explore, get,\n help, help-search, i, init, install, install-test, it, link,\n list, ln, login, logout, ls, outdated, owner, pack, ping,\n prefix, prune, publish, rb, rebuild, repo, restart, root,\n run, run-script, s, se, search, set, shrinkwrap, star,\n stars, start, stop, t, tag, team, test, tst, un, uninstall,\n unpublish, unstar, up, update, v, version, view, whoami\n\nnpm <cmd> -h quick help on <cmd>\nnpm -l display full usage info\nnpm help <term> search for help on <term>\nnpm help npm involved overview\n\nSpecify configs in the ini-formatted file:\n /Users/user/.npmrc\nor on the command line via: npm <command> --key value\nConfig info can be viewed via: npm help config\n\nnpm#3.10.10 /Users/user/.nvm/versions/node/v6.11.1/lib/node_modules/npm\n", 1}]
# 2
[]
# 3
:ok
Attempted function clauses (showing 3 out of 3):
defp run_alias([h | t], alias_args, _res) when is_binary(h)
defp run_alias([h | t], alias_args, _res) when is_function(h, 1)
defp run_alias([], _alias_task, res)
(mix) lib/mix/task.ex:331: Mix.Task.run_alias/3
(mix) lib/mix/task.ex:266: Mix.Task.run/2
(mix) lib/mix/cli.ex:75: Mix.CLI.run_task/2
(elixir) lib/code.ex:376: Code.require_file/2
Ok so obviously I'm doing something wrong. Is it possible to execute an NPM command on a sub-directory of a Phoenix app?
Firstly, run and lint should be separate strings in the arguments list.
Secondly, the code you have will immediately run the command instead of when the alias is invoked.
You can use a "run -e ..." alias to execute arbitrary code in an alias like this:
"lint": [~s|run -e 'System.cmd("npm", ["run", "lint"], cd: "assets")'|)
(I'm using ~s|| syntax just to make it easier to type a string with both single and double quotes.)
There is also an extension you can use to do this, but unfortunately it's unable to traverse paths. But maybe it will be patched.
https://github.com/verdammelt/mix_npm
An alternative to the ~s|| notation with a cd, we can also use the --prefix flag on the npm command.
lintjs: ["cmd npm run format:ci --prefix assets"]
If using an umbrella project, define the alias in the app where it runs, and add an alias in the project root.
<root>/mix.exs
defp aliases do
[
# run `mix setup` in all child apps
setup: ["cmd mix setup"],
# app-specific alias defined in onstage_web/mix.exs
lint: ["cmd --app myapp_name mix lintjs"]
]
end
<root>/apps/myapp_name/mix.exs
defp aliases do
[
# run npm install in child app
setup: ["deps.get", "cmd npm install --prefix assets"],,
# simplified/equivalent of accepted answer
lintjs: ["cmd npm run format:ci --prefix assets"]
]
end
Now, we can run mix lint from the project root to lint the JS in a child app.
Also, remember to run mix compile after making changes.

Use GitLab CI to run tests locally?

If a GitLab project is configured on GitLab CI, is there a way to run the build locally?
I don't want to turn my laptop into a build "runner", I just want to take advantage of Docker and .gitlab-ci.yml to run tests locally (i.e. it's all pre-configured). Another advantage of that is that I'm sure that I'm using the same environment locally and on CI.
Here is an example of how to run Travis builds locally using Docker, I'm looking for something similar with GitLab.
Since a few months ago this is possible using gitlab-runner:
gitlab-runner exec docker my-job-name
Note that you need both docker and gitlab-runner installed on your computer to get this working.
You also need the image key defined in your .gitlab-ci.yml file. Otherwise won't work.
Here's the line I currently use for testing locally using gitlab-runner:
gitlab-runner exec docker test --docker-volumes "/home/elboletaire/.ssh/id_rsa:/root/.ssh/id_rsa:ro"
Note: You can avoid adding a --docker-volumes with your key setting it by default in /etc/gitlab-runner/config.toml. See the official documentation for more details. Also, use gitlab-runner exec docker --help to see all docker-based runner options (like variables, volumes, networks, etc.).
Due to the confusion in the comments, I paste here the gitlab-runner --help result, so you can see that gitlab-runner can make builds locally:
gitlab-runner --help
NAME:
gitlab-runner - a GitLab Runner
USAGE:
gitlab-runner [global options] command [command options] [arguments...]
VERSION:
1.1.0~beta.135.g24365ee (24365ee)
AUTHOR(S):
Kamil Trzciński <ayufan#ayufan.eu>
COMMANDS:
exec execute a build locally
[...]
GLOBAL OPTIONS:
--debug debug mode [$DEBUG]
[...]
As you can see, the exec command is to execute a build locally.
Even though there was an issue to deprecate the current gitlab-runner exec behavior, it ended up being reconsidered and a new version with greater features will replace the current exec functionality.
Note that this process is to use your own machine to run the tests using docker containers. This is not to define custom runners. To do so, just go to your repo's CI/CD settings and read the documentation there. If you wanna ensure your runner is executed instead of one from gitlab.com, add a custom and unique tag to your runner, ensure it only runs tagged jobs and tag all the jobs you want your runner to be responsible of.
I use this docker-based approach:
Edit: 2022-10
docker run --entrypoint bash --rm -w $PWD -v $PWD:$PWD -v /var/run/docker.sock:/var/run/docker.sock gitlab/gitlab-runner:latest -c 'git config --global --add safe.directory "*";gitlab-runner exec docker test'
For all git versions > 2.35.2. You must add safe.directory within the container to avoid fatal: detected dubious ownership in repository at.... This also true for patched git versions < 2.35.2. The old command will not work anymore.
Details
0. Create a git repo to test this answer
mkdir my-git-project
cd my-git-project
git init
git commit --allow-empty -m"Initialize repo to showcase gitlab-runner locally."
1. Go to your git directory
cd my-git-project
2. Create a .gitlab-ci.yml
Example .gitlab-ci.yml
image: alpine
test:
script:
- echo "Hello Gitlab-Runner"
3. Create a docker container with your project dir mounted
docker run -d \
--name gitlab-runner \
--restart always \
-v $PWD:$PWD \
-v /var/run/docker.sock:/var/run/docker.sock \
gitlab/gitlab-runner:latest
(-d) run container in background and print container ID
(--restart always) or not?
(-v $PWD:$PWD) Mount current directory into the current directory of the container - Note: On Windows you could bind your dir to a fixed location, e.g. -v ${PWD}:/opt/myapp. Also $PWD will only work at powershell not at cmd
(-v /var/run/docker.sock:/var/run/docker.sock) This gives the container access to the docker socket of the host so it can start "sibling containers" (e.g. Alpine).
(gitlab/gitlab-runner:latest) Just the latest available image from dockerhub.
4. Execute with
Avoid fatal: detected dubious ownership in repository at... More info
docker exec -it -w $PWD gitlab-runner git config --global --add safe.directory "*"
Actual execution
docker exec -it -w $PWD gitlab-runner gitlab-runner exec docker test
# ^ ^ ^ ^ ^ ^
# | | | | | |
# (a) (b) (c) (d) (e) (f)
(a) Working dir within the container. Note: On Windows you could use a fixed location, e.g. /opt/myapp.
(b) Name of the docker container
(c) Execute the command "gitlab-runner" within the docker container
(d)(e)(f) run gitlab-runner with "docker executer" and run a job named "test"
5. Prints
...
Executing "step_script" stage of the job script
$ echo "Hello Gitlab-Runner"
Hello Gitlab-Runner
Job succeeded
...
Note: The runner will only work on the commited state of your code base. Uncommited changes will be ignored. Exception: The .gitlab-ci.yml itself does not have be commited to be taken into account.
Note: There are some limitations running locally. Have a look at limitations of gitlab runner locally.
I'm currently working on making a gitlab runner that works locally.
Still in the early phases, but eventually it will become very relevant.
It doesn't seem like gitlab want/have time to make this, so here you go.
https://github.com/firecow/gitlab-runner-local
If you are running Gitlab using the docker image there: https://hub.docker.com/r/gitlab/gitlab-ce, it's possible to run pipelines by exposing the local docker.sock with a volume option: -v /var/run/docker.sock:/var/run/docker.sock. Adding this option to the Gitlab container will allow your workers to access to the docker instance on the host.
The GitLab runner appears to not work on Windows yet and there is an open issue to resolve this.
So, in the meantime I am moving my script code out to a bash script, which I can easily map to a docker container running locally and execute.
In this case I want to build a docker container in my job, so I create a script 'build':
#!/bin/bash
docker build --pull -t myimage:myversion .
in my .gitlab-ci.yaml I execute the script:
image: docker:latest
services:
- docker:dind
before_script:
- apk add bash
build:
stage: build
script:
- chmod 755 build
- build
To run the script locally using powershell I can start the required image and map the volume with the source files:
$containerId = docker run --privileged -d -v ${PWD}:/src docker:dind
install bash if not present:
docker exec $containerId apk add bash
Set permissions on the bash script:
docker exec -it $containerId chmod 755 /src/build
Execute the script:
docker exec -it --workdir /src $containerId bash -c 'build'
Then stop the container:
docker stop $containerId
And finally clean up the container:
docker container rm $containerId
Another approach is to have a local build tool that is installed on your pc and your server at the same time.
So basically, your .gitlab-ci.yml will basically call your preferred build tool.
Here an example .gitlab-ci.yml that i use with nuke.build:
stages:
- build
- test
- pack
variables:
TERM: "xterm" # Use Unix ASCII color codes on Nuke
before_script:
- CHCP 65001 # Set correct code page to avoid charset issues
.job_template: &job_definition
except:
- tags
build:
<<: *job_definition
stage: build
script:
- "./build.ps1"
test:
<<: *job_definition
stage: test
script:
- "./build.ps1 test"
variables:
GIT_CHECKOUT: "false"
pack:
<<: *job_definition
stage: pack
script:
- "./build.ps1 pack"
variables:
GIT_CHECKOUT: "false"
only:
- master
artifacts:
paths:
- output/
And in nuke.build i've defined 3 targets named like the 3 stages (build, test, pack)
In this way you have a reproducible setup (all other things are configured with your build tool) and you can test directly the different targets of your build tool.
(i can call .\build.ps1 , .\build.ps1 test and .\build.ps1 pack when i want)
I am on Windows using VSCode with WSL
I didn't want to register my work PC as a runner so instead I'm running my yaml stages locally to test them out before I upload them
$ sudo apt-get install gitlab-runner
$ gitlab-runner exec shell build
yaml
image: node:10.19.0 # https://hub.docker.com/_/node/
# image: node:latest
cache:
# untracked: true
key: project-name
# key: ${CI_COMMIT_REF_SLUG} # per branch
# key:
# files:
# - package-lock.json # only update cache when this file changes (not working) #jkr
paths:
- .npm/
- node_modules
- build
stages:
- prepare # prepares builds, makes build needed for testing
- test # uses test:build specifically #jkr
- build
- deploy
# before_install:
before_script:
- npm ci --cache .npm --prefer-offline
prepare:
stage: prepare
needs: []
script:
- npm install
test:
stage: test
needs: [prepare]
except:
- schedules
tags:
- linux
script:
- npm run build:dev
- npm run test:cicd-deps
- npm run test:cicd # runs puppeteer tests #jkr
artifacts:
reports:
junit: junit.xml
paths:
- coverage/
build-staging:
stage: build
needs: [prepare]
only:
- schedules
before_script:
- apt-get update && apt-get install -y zip
script:
- npm run build:stage
- zip -r build.zip build
# cache:
# paths:
# - build
# <<: *global_cache
# policy: push
artifacts:
paths:
- build.zip
deploy-dev:
stage: deploy
needs: [build-staging]
tags: [linux]
only:
- schedules
# # - branches#gitlab-org/gitlab
before_script:
- apt-get update && apt-get install -y lftp
script:
# temporarily using 'verify-certificate no'
# for more on verify-certificate #jkr: https://www.versatilewebsolutions.com/blog/2014/04/lftp-ftps-and-certificate-verification.html
# variables do not work with 'single quotes' unless they are "'surrounded by doubles'"
- lftp -e "set ssl:verify-certificate no; open mediajackagency.com; user $LFTP_USERNAME $LFTP_PASSWORD; mirror --reverse --verbose build/ /var/www/domains/dev/clients/client/project/build/; bye"
# environment:
# name: staging
# url: http://dev.mediajackagency.com/clients/client/build
# # url: https://stg2.client.co
when: manual
allow_failure: true
build-production:
stage: build
needs: [prepare]
only:
- schedules
before_script:
- apt-get update && apt-get install -y zip
script:
- npm run build
- zip -r build.zip build
# cache:
# paths:
# - build
# <<: *global_cache
# policy: push
artifacts:
paths:
- build.zip
deploy-client:
stage: deploy
needs: [build-production]
tags: [linux]
only:
- schedules
# - master
before_script:
- apt-get update && apt-get install -y lftp
script:
- sh deploy-prod
environment:
name: production
url: http://www.client.co
when: manual
allow_failure: true
The idea is to keep check commands outside of .gitlab-ci.yml. I use Makefile to run something like make check and my .gitlab-ci.yml runs the same make commands that I use locally to check various things before committing.
This way you'll have one place with all/most of your commands (Makefile) and .gitlab-ci.yml will have only CI-related stuff.
I have written a tool to run all GitLab-CI job locally without have to commit or push, simply with the command ci-toolbox my_job_name.
The URL of the project : https://gitlab.com/mbedsys/citbx4gitlab
Years ago I build this simple solution with Makefile and docker-compose to run the gitlab runner in docker, you can use it to execute jobs locally as well and should work on all systems where docker works:
https://gitlab.com/1oglop1/gitlab-runner-docker
There are few things to change in the docker-compose.override.yaml
version: "3"
services:
runner:
working_dir: <your project dir>
environment:
- REGISTRATION_TOKEN=<token if you want to register>
volumes:
- "<your project dir>:<your project dir>"
Then inside your project you can execute it the same way as mentioned in other answers:
docker exec -it -w $PWD runner gitlab-runner exec <commands>..
I recommend using gitlab-ci-local
https://github.com/firecow/gitlab-ci-local
It's able to run specific jobs as well.
It's a very cool project and I have used it to run simple pipelines on my laptop.