I just recently started using gitlab CI to automate some build/deploy steps. It works perfectly to build docker images etc, but I was wondering if it's possible to create a folder in the repository during a build step? For example I'm now making an npm utility package, but I'm just importing it in my other projects via a private gitlab repo (using deploy token), but the code of the util package is written in es6 and needs to be transpiled to commonJS to be used in the other packages. Manually I can run npm run build and it will output a dist folder with the transpiled code.
I was trying (and researching) if it's possible to automate this build process using .gitlab-ci but so far I couldn't find anything.
Anyone know how I can achieve this and/or if this is possible?
Thanks in advance!
Not sure if I got your question correctly, so add more details if not.
When your CI build creates new folders or files, they are written to the task runner's file system (no surprise here, I assume).
If you want to access these files from Gitlab's web UI you can define them as artifacts in your build job (see https://docs.gitlab.com/ee/user/project/pipelines/job_artifacts.html)
Your build job would look something like that (pseudo code written by memory, not tested on Gitlab):
build:
script:
- npm run build
artifacts:
paths:
- dist/
expire_in: 1 week
UPDATE If you want to upload the build artifact to an NPM registry, you could just build and push together.
build:
script:
- npm run build
- npm publish <PARAMETERS>
Related
So, I am in the process to integrate Turborepo into our NodeJS(React, Next, Node) Monorepo which uses GitLab CI. The thing is the example in the Docs is quite not up to what I want.
For reference here is what they have in their Docs:
image: node:latest
# To use Remote Caching, uncomment the next lines and follow the steps below.
# variables:
# TURBO_TOKEN: $TURBO_TOKEN
# TURBO_TEAM: $TURBO_TEAM
stages:
- build
build:
stage: build
script:
- npm install
- npm run build
- npm run test
We have a few stages. Beyond the ones in their example:
install
build
package
What I would ideally like to have is to use Turborepo and GitLab Downstream Pipelines to run as follows:
install stage should run when root package.json has changed.
build, package stage should be run just for the affected packages. (i.e, if shared-lib is changed, then shared-lib should run as well as the 2 consumers app-a, app-b. In parallel)
I read the Docs and I can somehow make the Downstread Job run but not for the affected. Instead it does it for all. The main problem is how can I read the affected packages and its consumers, and just run those.
I read with the latest version I can use the --dry commands to read those. But let's say that works reliable which from my testing it doesn't. how can I put those packages, as Downstream Jobs in Gitlab?
I have configured a private agent in VSTS and have installed NPM there globally. When I'm trying to install NPM through my build task, it is still installing NPM packages for every build which is taking an aweful lot of time- approximately 12 minutes.
How can I cache the NPM installations so that the build time is reduced?
We use npm-cache, npm-cache is a node module that will calculate a hash of your package.json file for every hash it will create zip folder on your build server with the content of node_modules, now npm install is reduced to extracting a zip on every build (of course only in case you didn’t actually change package.json).
The idea is: in the first time the tool download the npm packages and save them locally, in the second time if the package.json not changed he takes the packages from the local disk and copy them to the build agent folder, only if the package.json changed he downloads the packages from the internet.
Install the npm-cache on the build machine:
npm install npm-cache -g
In the build definition add Command Line task (Tool: C:\Windows\User\AppData\Roaming\npm\npm-cache (or just npm-cache if you add the tool to environment path variables); Arguments:install npm; Working folder: $(Build.SourcesDirectory) (or where package.json located).
MS has finally implemented this feature (currently in beta) https://learn.microsoft.com/en-us/azure/devops/pipelines/caching/index?view=azure-devops#nodejsnpm
From there:
variables:
npm_config_cache: $(Pipeline.Workspace)/.npm
steps:
- task: CacheBeta#0
inputs:
key: $(Build.SourcesDirectory)/package-lock.json
path: $(npm_config_cache)
displayName: Cache npm
- script: npm ci
Unfortunately we cannot cache the NPM installations as no such a built-in feature for now.
However there's already a user voice submitted to suggest the feature : Improve hosted build agent performance with build caches, and seems the VSTS team are actively working on this now...
For now, you can try to speed Up NPM INSTALL on Visual Studio Team Services
Use Cache task
Caching is added to a pipeline using the Cache pipeline task. This
task works like any other task and is added to the steps section of a
job
With the following configuration:
pool:
name: Azure Pipelines
steps:
- task: Cache#2
inputs:
key: 'YOUR_WEB_DIR/package.json'
path: 'YOUR_WEB_DIR/node_modules/'
- task: Npm#1
inputs:
command: 'install'
workingDir: 'YOUR_WEB_DIR/frontend'
You can use key YOUR_WEB_DIR/package-lock.json too, but be aware that file might be changed by other next step like npm install so hash also will be changed.
I use vue.js + vue-cli + webpack to build my applications. During development I will run npm run dev to have webpack continuously watch my sources, compile everything, and reload the browser. To create production build, I can simply run npm run build. I would like to do this in a way that when I make a git commit, if my sources have changed, the build is created automatically.
My current approach is to simply use git pre and post commit hooks to automatically run npm run build and add the built files to the commit. This has the following downsides:
Even if other parts of the repo are changed, I re-run the build process for the Vue app, and it takes a very long time.
It makes resolving merge conflicts nearly impossible.
It creates a lot of cruft in the repo, ballooning its size
Typically I use a Vue.js frontend with a Django backend in the same repo, and deploy to Heroku or similar via a git push. What other methods are out there for accomplishing this task that don't have the above downsides?
Write a script in the package.json scripts section with something like
build && git commit -m "Build commit"
Does the build have to run on the drone.io server? Can I run the build locally? Since developers need to pass the build first before pushing code to github, I am looking for a way to run the build on developer local machine. Below is my .drone.yml file:
pipeline:
build:
image: node:latest
commands:
- npm install
- npm test
- npm run eslint
integration:
image: mongo-test
commands:
- mvn test
It includes two docker containers. How to run the build against this file in drone? I looked at the drone cli but it doesn't work in my expected way.
#BradRydzewski comment is the right answer.
To run builds locally you use drone exec. You can check the docs.
Extending on his answer, you must execute the command in the root of your local repo, exactly where your .drone.yml file is. If your build relies on secrets, you need to feed these secrets through the command line using the --secret or --secrets-file option.
When running a local build, there is no cloning step. Drone will use your local git workspace and mount it in the step containers. So, if you checkout some other commit/branch/whatever during the execution of the local build, you will mess things up because Drone will see those changes. So don't update you local repo while the build is running.
If I git clone the repo (from mapbox-gl.js repo), how can I build my own standalone version of the files?
I just need the standalone mapbox-gl.js, like this one.
A simple browserify js/mapbox-gl.js -o dist/mapbox-gl.js does not work.
I just added a section to the mapbox-gl-js readme on this topic.
Creating a Standalone Build
A standalone build allows you to turn the contents of this repository
into mapbox-gl.js and mapbox-gl.css files that can be included on
an html page.
To create a standalone build, run bash npm run production
Once that command finishes, you will have a standalone build at
dist/mapbox-gl.js and dist/mapbox-gl.css
I hope that helps! Let me know if you have any follow-up questions.
To create standalone files, run npm run build in root of repo.