I'm trying to package a Python and Qt app with pyinstaller. I, however, haven't found a way to run multiple runners in parallel.
You basically would duplicate the job definition and just assign different tags to pick the runners you want. Here's how I build an Electron app on three different runners in parallel:
.build:
stage: build
script:
- npm install --progress=false
- npm run electron:build
build-linux:
extends: .build
tags:
- linux
build-mac:
extends: .build
tags:
- mac
build-windows:
extends: .build
tags:
- windows
This config makes use of hidden jobs and extends.
Since GitLab 14.1 it's possible to use the parallel: matrix: keyword in combination with tags, e.g.
build:
script:
- npm install --progress=false
- npm run electron:build
parallel:
matrix:
- PLATFORM: [linux, macos, windows]
tags:
- ${PLATFORM}
The tags: keyword should then pick up the CI runner for the specified operating system.
Related
This is a GitHub Action
on:
push:
branches:
- '*' # matches every branch that doesn't contain a '/'
- '*/*' # matches every branch containing a single '/'
- '**' # matches every branch
- '!master' # excludes master
jobs:
build:
strategy:
matrix:
os: [ubuntu-latest, windows-latest, macos-latest]
runs-on: ${{ matrix.os }}
steps:
- name: Checkout
uses: actions/checkout#v2
- name: Install Node.js
uses: actions/setup-node#v1
with:
node-version: 16.x
- run: npm install
- run: gulp
- run: xvfb-run -a npm test
if: runner.os == 'Linux'
- run: npm test
if: runner.os != 'Linux'
This is the outcome from a push, which triggers it.
As you can see it works fine on Windows and Linux, but fails on OSX.
Inspecting the Run gulp step we find this message.
As you can see, gulp is not found. It's present in devDependencies, and my expectation was that the npm install step would install it, this being how it gets onto the other two platforms.
I tried explicitly installing gulp-cli but that didn't help.
Installing gulp globally (npm i gulp gulp-cli -g) resolves the problem, but I do not understand why this should be necessary for only MacOS. Perhaps the answer is one of differences between platforms but I do not like mysteries so if anyone can provide clarity that would be highly desirable.
I am trying to use the shard option in jest to run my tests in parallel across multiple runners.
But before I can do this on CI - I have to do some other steps like checkout codebase, install dependencies in the specific job running the test.
Problem statement - I want to avoid installing dependencies for each shard of test running.
I searched around the internet and found that I cannot do this in another preceding job because each job starts fresh in its own container.
Hence I wrote something like this:
run_tests:
runs-on: [self-hosted]
name: Run Tests
continue-on-error: false
strategy:
matrix:
shard: [1, 2, 3, 4]
steps:
- name: Checkout Codebase
uses: actions/checkout-action#v2
- name: Use Node v14.15.1
uses: actions/setup-node#v1
with:
node-version: 14.15.1
- name: Cache NPM dependencies
id: npmCacheStep
uses: actions/cache#v2
with:
path: ~/.npm
key: ${{ runner.OS }}-npm-cache-${{ hashFiles('**/package-lock.json') }}
restore-keys: |
${{ runner.OS }}-npm-cache-
- name: Install Dependencies
if: steps.npmCacheStep.outputs.cache-hit != 'true'
run: npm ci
- name: Run tests
run: npm run test -- --colors --coverage --shard=${{ matrix.shard }}/${{ strategy.job-total }}
To solve my problem statement, I'm saying that if it found a cache of npm dependencies - then don't install npm modules again as recommended at lots of places - https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows
But when I try to run this
the npm install step is skipped (as expected)
I get the following error on github actions workflow logs:
I am trying this solution: Is there a way to speedup npm ci using cache?
I am currently trying to get my E2E tests running on Gitlab with their CI/CD platform.
My issue at the moment is that I cannot both my dev server and cypress to run at the same time so that the E2E tests can run.
Here is my current .gitlab-ci.yml file:
image: node
variables:
npm_config_cache: "$CI_PROJECT_DIR/.npm"
CYPRESS_CACHE_FOLDER: "$CI_PROJECT_DIR/cache/Cypress"
cache:
key: ${CI_COMMIT_REF_SLUG}
paths:
- .npm
- cache/Cypress
- node_modules
stages:
- setup
- test
setup:
stage: setup
image: cypress/base:10
script:
- npm ci
# check Cypress binary path and cached versions
# useful to make sure we are not carrying around old versions
- npx cypress cache path
- npx cypress cache list
cypress:
stage: test
image: cypress/base:10
script:
# I need to start a dev server here in the background
- cypress run --record --key <my_key> --parallel
artifacts:
when: always
paths:
- cypress/videos/**/*.mp4
- cypress/screenshots/**/*.png
expire_in: 7 day
In Cypress's official GitHub page, there is an example .gitlab-ci.yml for running Cypress in continuous integration.
It uses command npm run start:ci & for running dev server in the background.
So, your .gitlab-ci.yml might look like this:
⋮
cypress:
image: cypress/base:10
stage: test
script:
- npm run start:ci & # start the server in the background
- cypress run --record --key <my_key> --parallel
⋮
Or use this utility to start the server, wait for an URL to respond, then run tests and shut down the server https://github.com/bahmutov/start-server-and-test
I have a simple .gitlab-ci.yml file that builds my Vue application. I build once and then deploy the dist folder to my various environments:
stages:
- build
- deploy_dev
- deploy_stg
- deploy_prd
build:
image: node:latest # Pull Node image
stage: build
script:
- npm install -g #vue/cli#latest
- npm install
- npm run build
artifacts:
expire_in: 2 weeks
paths:
- dist/
deploy_to_dev:
image: python:latest
stage: deploy_dev
dependencies:
- build
only:
- master # Only deply master branch automatically to Dev
script:
- export AWS_ACCESS_KEY_ID=$DEV_AWS_ACCESS_ID
- export AWS_SECRET_ACCESS_KEY=$DEV_AWS_ACCESS_KEY
- pip install awscli # Install AWS CLI
- aws s3 sync ./dist s3://$DEV_BUCKET
This all works great, however, I've now introduced some config and build my app differently per environment - for 3 environments I have 3 different build commands. Eg, I have an .env.production so for a production build my command becomes:
npm run build -- --mode production
Is there any way to get around having different builds for each environment but still using the .env files based on a GitLab variable?
You should split your build job to have one per environment and use the environment concept to have something like that for dev and production envs :
.build_template: &build_template
image: node:latest # Pull Node image
script:
- npm install -g #vue/cli#latest
- npm install
- npm run build -- --mode $CI_ENVIRONMENT_NAME
build_dev:
stage: build_dev
<<: *build_template
environment:
name: dev
build_prod:
stage: build_prod
<<: *build_template
environment:
name: production
In this snippet, I used anchors to avoid duplicate lines.
I tried to run XCUITest (Objective-C/swift) on travis ci for a react-native project while there's also node jest unit tests that I'll be running. I was wondering what the best way is to set up the travis.yml file since XCUITest is in Objective-C and jest unit tests are in node_js. I've done some research but not sure what a good way to do it is.
It turns out the travis.yml file can be set up this way
language: objective-c
git:
submodules: false
sudo: required
services:
- docker
node_js:
- "5.10.1"
before_install:
- npm install
env:
- export NODE_VERSION="5.10.1"
script:
- npm test
- cd ios/ && xcodebuild test ...
hopefully it's useful to some people