Is it possible to ignore the dependency hash validation of just one module (or registry)? - npm

The yarn.lock file saves all the dependencies versions and the hashes of the modules. I know that I can globally disable this hash checking with the option --skip-integrity-check.
We have an internal module that is continually developed. The dependency is really of a snapshot package. When it is updated, it fails in our continuous integration environment because the updated package hash is different of the yarn.lock saved hash.
Is it possible to disable the integrity check just for a specific module?
I'll accept the answer even if it tells how to disable the check for all the modules of a specific registry.
Update: My problem is that my continuous integration server job is breaking when the dependency is updated, even if there's no modification in my code. These are spurious failings and I want to stop them.
Update 2: The accepted solution is really a hack to solve a problem in a usual development workflow. There is an issue open for Yarn in GitHub to fix this problem.

Instead of running
yarn install
You should run it like below
yarn add <specificpackage>#^<versions> --update-checksums
yarn install
This will make sure that the yarn.lock is updated with latest hash for that package and then yarn install will install the rest of the packages with integrity check
Update-1: 20-April
Another possible options is to use the preinstall hook. There are few things you can try here. You can try updating the package. But be aware that launching the yarn command again in preinstall can cause infinite loops.
So better way may be to run a grep, awk or a sed command and get ride of the package entry in the yarn.lock file. This will make sure the yarn install command has no information on the hash and a mismatch can't occur
If you don't want to use awk, sed or grep because of windows compatibility then you should just write a simple nodejs script to get rid of the package from the yarn.lock file. This will cross-os compatible. Below code shows how to do the same
yarn_remove_hash.js
const fs = require('fs')
const content = fs.readFileSync("yarn.lock", "utf-8");
const packageToDelete = "yallist"
let lines = content.split("\n")
for (let [i, line] of Object.entries(lines)) {
if (line.startsWith(packageToDelete + "#")) {
lines[i]="";
let y = i;
while (lines[++y][0] ==" "){
lines[y]= ""
}
}
}
fs.writeFileSync("yarn.lock", lines.join("\n"))
And you will update your scripts section in package.json like below
...
"preinstall": "node yarn_remove_hash.js"
...

If you want to make #Tarun Lalwani's --update-checksums more of a transparent process for you and others, you can add the following to .yarnrc:
--install.update-checksums true
Now when a user runs yarn install it will also update checksums implicitly. This was needed for me because one of my dependencies is linked to a snapshot .tar.gz that changes and NPM/Yarn would assume that it wouldn't, obviously leading us to this integrity issue. I had to move away from NPM because of this and also tried the preinstall hook (I thought I was clever but I guess you guys did the same).
At least Yarn has an option around this. Tarun's updated answer did not work for me either because yarn.lock is checked against before any hooks are ran.

Related

Can't start Framework Qwik project

I have an issue while creating and starting project. I followed the instructions given here https://qwik.builder.io/docs/getting-started/ and used npm, selected Basic App (QwikCity), but when I start the project I'm given the next error:
Error
Terminal output:
[vite] Internal server error: Failed to load url /src/root_component_vgnegdacmce.js (resolved id: C:/Users/JESUS LOPEZ/Documents/Universidad/Pasantías/qwik-app/src/root_component_vgnegdacmce.js). Does the filnt_vgnegdacmce.js). Does the file exist?
File: /C:/Users/JESUS%20LOPEZ/Documents/Universidad/Pasant%C3%ADas/qwik-app/node_modules/vite/dist/node/chunks/dep-5e7f419b.js:39304:21
at loadAndTransform (file:///C:/Users/JESUS%20LOPEZ/Documents/Universidad/Pasant%C3%ADas/qwik-app/node_modules/vite/dist/node/chunks/dep-5e7f419b.js:39304:21)
I'm using Windows 10 and node 18.12.0, I tried with yarn and happened the same. I'm just testing this framework because I was required to create a component library, so I wanted to test the waters with a basic app project and then move on with the component library but even if I select this option, I have a similar error.
This is my repo: https://github.com/luisamlopez/qwik-app but it's literally just a brand new qwik project (npm create qwik#latest) so I haven't touch anything.
Your code works fine without any problems. Maybe some node modules or other dependencies would have not been installed properly because of firewall or network issues.
Clean the node_modules manually delete the folder or by the following command
rm -r node_modules/
npm prune
Note: prune command is optional.
Install the package dependencies by
npm i
Make sure the installation happens successfully without any issues or try to install with a different network or turn off the firewall for a while. Or worst case try with different machines.

Lerna concurrency errors in Docker container

I have a typescript monorepo managed with Lerna.
The tests are done with Jest.
There are tens of packages that have the test script defined, while the jest config is stored in a central location and used by all.
An example test script looks like this:
jest --config ../../../tests/jest.config.json --setupFiles ../../../tests/jest-setup.js --rootDir .
Each package has a different number of "../" depending on its location in the source folder tree.
It works 100% of the time on multiple platforms like Windows, Linux and Mac.
For some reason when we run it inside a docker container, if the concurrency isn't set to 1 we see jest process from one package getting actually the paramters of another one which cause it to fail:
#cmd/example-package: > #cmd/example-package#0.0.0 test:integration
#cmd/example-package: > jest --config ../../tests/jest.config.json --setupFiles ../../tests/jest-setup.js --setupFilesAfterEnv ../../tests/setEnvVars.js --rootDir . --testPathPattern=./integration-tests --runInBand
#cmd/example-package: jest parameters: /src/cmd/example-package , --config,../../../tests/jest.config.json,--setupFiles,../../../tests/jest-setup.js,--setupFilesAfterEnv,../../../tests/setEnvVars.js,--rootDir,.,--testPathPattern=./integration-tests,--runInBand
The last line is printed from code added at the begining of the jest script which prints the current folder and the parameters passed. You can see that the parameters lerna reports passing to jest aren't the one which were actually used.
We saw such errors in the build process as well.
Any idea on how to solve it will be highly appriciated.
Tried multiple nodejs base images (alpine, node, Bullseye), multiple node versions and multiple lerna versions.
Even tried to switch from Lerna to Turborepo but still getting these concurrency errors
In our case, the problem was related to the installation of a different version of the npm on top of the alpine node docker image in one of the container's base images.
Leaving the question here, in case it helps someone.

Yarn script produces different output than running command directly

I have a script defined in package.json:
"scripts": {
"prettierCheck": "./node_modules/.bin/prettier --check ./app/javascript/**/*.js"
}
If I run this script using yarn run prettierCheck, Prettier does not find any formatting issues with my files. However, if I run the Prettier command directly, it finds violating files.
Output of yarn run prettierCheck:
~/Projects/tome $ yarn run prettierCheck
yarn run v1.19.0
$ ./node_modules/.bin/prettier --check ./app/javascript/**/*.js
Checking formatting...
All matched files use Prettier code style!
Done in 0.20s
Output of ./node_modules/.bin/prettier --check ./app/javascript/**/*.js:
~/Projects/tome $ ./node_modules/.bin/prettier --check ./app/javascript/**/*.js
Checking formatting...
{... several files listed here ...}
Code style issues found in the above file(s). Forgot to run Prettier?
Why does this happen? What is the difference between running the command directly vs. through a Yarn script?
I had the same issue or at least I thought I had the same issue. I tried a few things like adding quotes to the file/dir/glob in my command, changing my paths/globs & it started working for me because now I could see some useful output. Try these & maybe it'll work for you guys as well.
This is what prettier CLI documentation has to say about paths & file/dir/globs.
Don't forget the quotes around the globs! The quotes make sure that
Prettier CLI expands the globs rather than your shell, which is
important for cross-platform usage.
So try something like this in your scripts (Note: -c is short for --check)
scripts": {
"pretty-check": "prettier -c 'src/**/*.ts'",
}
Notice the quotes around 'src/**/*.ts'
I'm not sure if this would work for everyone but I'm hoping that it does as it did for me.
And if you are wondering What is file/dir/glob It's basically the part where you define the regex for your files like the part 'src/**/*.ts' in the command prettier -c 'src/**/*.ts'

Understanding NPM vulnerability - tunnel-agent

So I have a vulnerability in a package named tunnel-agent. After running npm audit the packages which depend on this package are listed:
gatsby-plugin-sharp
OK great, I update this and everything is fine? NO.. Still listing as vulnerable, so now I start on the rabbit hole of looking where this leads.
Running npm list tunnel-agent I get to find out who's depending on this package.
So now the vulnerability is fixed in tunnel-agent#0.6.0but I've got one thing saying it's using tunnel-agent#0.4.3. But this is in the same package gatsby-plugin-sharp so why's it not fixed?
I head off to github issues and find that because gatsby-plugin-sharp uses imagemin-mozjpeg > caw#1.2.0 > tunnel-agent#0.4.3 I'm still stuck right?
So what I'm asking is, without relying on plugin authors to update their dependencies, how would you go about using caw#2.0.1 which then uses tunnel-agent#0.6.0 to remove this vulnerability once and for all?
This is all environment variables.
But you could fork the open pull request that have not been merged published. Then create your on npm packages that have the fixes.
https://github.com/request/tunnel-agent/pull/45
yarn negates these errors.
And ‘yarn’ doesn’t have these issue. Since it is designed for local scope.

Ignore packagist.org on composer install | update

I'm using composer internally for managing internal software dependencies. Our repository server is on our private network and we aren't using any other package from any other repository than ours.
Every time you run
composer.phar [install | update]
It checks on packagist.org repositories after check our own repository. Beyond unnecessary, it takes longer when packagist is slow (or even down) or our internet connection is having a bad day.
Is there any way to tell composer to ignore checking for packagist repositories?
Yes, and it is even documented on https://getcomposer.org/doc/05-repositories.md#disabling-packagist-org
You may try to use this command:
$ composer config repositories.packagist false
You probably want to have a look at Satis: http://getcomposer.org/doc/articles/handling-private-packages-with-satis.md
It will make your life easier if you deal with a bit more of local/private packages, because otherwise you'd have to mention EVERY repository that might host required code. And you can use Satis to grab a copy of the versions into a ZIP file that can be hosted locally as well. See http://www.naderman.de/slippy/src/?file=2012-11-22-You-Thought-Composer-Couldnt-Do-That.html#13 for some hints of how to do it (press cursor keys left/right to skip through the presentation)
For extra bonus points, you'd add packagist.org as a Composer repository to Satis, require some needed packages, and set { "require-dependencies": true } to grab their dependencies as well. In your own code, you'd only set your Satis repository and disable Packagist.