Yocto build failed with npm v8 - npm

Question

Issue and Context
I'm currently migrating to Yocto Kirkstone and need some npm packages to be installed on the image.
I was using Dunfell previously which ships with npm v6, I could build it without issue.
But with Kirkstone (and npm v8) it fails during npm pack stating that a command could not be found.
Forcing NodeJs v14 with a custom recipe and defining PREFERRED_VERSION:nodejs work but I would like to stick to v16.
As an example, I will use the runmd package as the one I try to install has much more dependencies and I faced the same issue with runmd.
With this package the first command not found is husky.
What I tried
Generating a new .bb file with devtool add "npm://registry.npmjs.org;package=runmd;version=1.3.6"
=> same issue
Installing other packages
=> same issue
Running the command that fails with npm v6 from the host
=> it worked as before
Checked if the missing package is installed with npm list in the source directory (workspace/sources/runmd)
=> it wasn't
npm list | grep husky
├── UNMET DEPENDENCY husky#8.0.1
npm ERR! code ELSPROBLEMS
npm ERR! missing: husky#8.0.1, required by runmd#1.3.6
...
Forced the installation of husky by running npm install --legacy-peer-deps and run the same command again
=> it went a bit further but then failed by not finding husky in tmp/work/corei7-64-poky-linux/runmd/1.3.6-r0/npm-package this time (see logs)
Tried to force the --legacy-peer-deps in npm.bbclass as EXTRA_OENPM cannot be overwritten and NPM_INSTALL_DEV did not helped (I know, it's bad 😬)
=> still facing the issue
What I suspect
Seems like it has something to do with the new way npm handles dependencies but I can't figure out what to do/try next to fix it.
I'm not an Yocto nor npm v8 expert, so I might have overseen something.
Logs
NOTE: Executing Tasks
NOTE: runmd: compiling from external source tree /yocto/project/nuc-build/workspace/sources/runmd
ERROR: runmd-1.3.6-r0 do_compile: Error executing a python function in exec_func_python() autogenerated:
The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_func_python() autogenerated', lineno: 2, function: <module>
0001:
*** 0002:npm_do_compile(d)
0003:
File: '/yocto/project/sources/poky/meta/classes/npm.bbclass', lineno: 262, function: npm_do_compile
0258: args.append(("target_arch", d.getVar("NPM_ARCH")))
0259: args.append(("build-from-source", "true"))
0260:
0261: # Pack and install the main package
*** 0262: tarball = npm_pack(env, d.getVar("NPM_PACKAGE"), tmpdir)
0263: cmd = "npm install %s %s" % (shlex.quote(tarball), d.getVar("EXTRA_OENPM"))
0264: env.run(cmd, args=args)
0265:}
0266:
File: '/yocto/project/sources/poky/meta/classes/npm.bbclass', lineno: 67, function: npm_pack
0063: cmd = "npm pack %s" % shlex.quote(srcdir)
0064: args = [("ignore-scripts", "true")]
0065: print(cmd)
0066: print(args)
*** 0067: tarball = env.run(cmd, args=args, workdir=workdir).strip("\n")
0068: return os.path.join(workdir, tarball)
0069:
0070:python npm_do_configure() {
0071: """
File: '/yocto/project/sources/poky/bitbake/lib/bb/fetch2/npm.py', lineno: 126, function: run
0122: if args:
0123: for key, value in args:
0124: cmd += " --%s=%s" % (key, shlex.quote(value))
0125:
*** 0126: return _run(cmd)
0127:
0128:class Npm(FetchMethod):
0129: """Class to fetch a package from a npm registry"""
0130:
File: '/yocto/project/sources/poky/bitbake/lib/bb/fetch2/npm.py', lineno: 114, function: _run
0110:
0111: def _run(cmd):
0112: cmd = "NPM_CONFIG_USERCONFIG=%s " % (self.user_config.name) + cmd
0113: cmd = "NPM_CONFIG_GLOBALCONFIG=%s " % (self.global_config_name) + cmd
*** 0114: return runfetchcmd(cmd, d, workdir=workdir)
0115:
0116: if configs:
0117: bb.warn("Use of configs argument of NpmEnvironment.run() function"
0118: " is deprecated. Please use args argument instead.")
File: '/yocto/project/sources/poky/bitbake/lib/bb/fetch2/__init__.py', lineno: 929, function: runfetchcmd
0925: bb.utils.remove(f, True)
0926: except OSError:
0927: pass
0928:
*** 0929: raise FetchError(error_message)
0930:
0931: return output
0932:
0933:def check_network_access(d, info, url):
Exception: bb.fetch2.FetchError: Fetcher failure: Fetch command export PSEUDO_DISABLED=1; export DBUS_SESSION_BUS_ADDRESS="unix:path=/run/user/1000/bus"; export PATH="/yocto/project/nuc-build/tmp/sysroots-uninative/x86_64-linux/usr/bin:/yocto/project/nuc-build/tmp/work/corei7-64-poky-linux/runmd/1.3.6-r0/recipe-sysroot-native/usr/bin/python3-native:/yocto/project/sources/poky/scripts:/yocto/project/nuc-build/tmp/work/corei7-64-poky-linux/runmd/1.3.6-r0/recipe-sysroot-native/usr/bin/x86_64-poky-linux:/yocto/project/nuc-build/tmp/work/corei7-64-poky-linux/runmd/1.3.6-r0/recipe-sysroot/usr/bin/crossscripts:/yocto/project/nuc-build/tmp/work/corei7-64-poky-linux/runmd/1.3.6-r0/recipe-sysroot-native/usr/sbin:/yocto/project/nuc-build/tmp/work/corei7-64-poky-linux/runmd/1.3.6-r0/recipe-sysroot-native/usr/bin:/yocto/project/nuc-build/tmp/work/corei7-64-poky-linux/runmd/1.3.6-r0/recipe-sysroot-native/sbin:/yocto/project/nuc-build/tmp/work/corei7-64-poky-linux/runmd/1.3.6-r0/recipe-sysroot-native/bin:/yocto/project/sources/poky/bitbake/bin:/yocto/project/nuc-build/tmp/hosttools"; export HOME="/tmp/tmp8wc17ixl"; NPM_CONFIG_GLOBALCONFIG=/dev/null NPM_CONFIG_USERCONFIG=/tmp/tmpjceiejg8 npm pack /yocto/project/nuc-build/tmp/work/corei7-64-poky-linux/runmd/1.3.6-r0/npm-package --ignore-scripts=true failed with exit code 127, output:
npm verb cli [
npm verb cli '/yocto/project/nuc-build/tmp/work/corei7-64-poky-linux/runmd/1.3.6-r0/recipe-sysroot-native/usr/bin/node',
npm verb cli '/yocto/project/nuc-build/tmp/work/corei7-64-poky-linux/runmd/1.3.6-r0/recipe-sysroot-native/usr/bin/npm',
npm verb cli 'pack',
npm verb cli '/yocto/project/nuc-build/tmp/work/corei7-64-poky-linux/runmd/1.3.6-r0/npm-package',
npm verb cli '--ignore-scripts=true'
npm verb cli ]
npm info using npm#8.5.0
npm info using node#v16.14.2
npm timing npm:load:whichnode Completed in 1ms
npm timing config:load:defaults Completed in 1ms
npm timing config:load:file:/yocto/project/nuc-build/tmp/work/corei7-64-poky-linux/runmd/1.3.6-r0/recipe-sysroot-native/usr/lib/node_modules/npm/npmrc Completed in 1ms
npm timing config:load:builtin Completed in 1ms
npm timing config:load:cli Completed in 1ms
npm timing config:load:env Completed in 0ms
npm timing config:load:file:/tmp/tmpkm1z714f/.npmrc Completed in 0ms
npm timing config:load:project Completed in 1ms
npm verb config only Use `--omit=dev` to omit dev dependencies from the install.
npm timing config:load:file:/tmp/tmpjceiejg8 Completed in 2ms
npm timing config:load:user Completed in 2ms
npm timing config:load:file:/dev/null Completed in 1ms
npm timing config:load:global Completed in 1ms
npm timing config:load:validate Completed in 1ms
npm timing config:load:credentials Completed in 1ms
npm timing config:load:setEnvs Completed in 0ms
npm timing config:load Completed in 10ms
npm timing npm:load:configload Completed in 10ms
npm timing npm:load:setTitle Completed in 1ms
npm timing config:load:flatten Completed in 2ms
npm timing npm:load:display Completed in 4ms
npm verb logfile /yocto/project/nuc-build/tmp/work/corei7-64-poky-linux/runmd/1.3.6-r0/npm-cache/_logs/2022-06-20T09_50_23_086Z-debug-0.log
npm timing npm:load:logFile Completed in 3ms
npm timing npm:load:timers Completed in 0ms
npm timing npm:load:configScope Completed in 0ms
npm timing npm:load Completed in 19ms
npm timing command:pack Completed in 16ms
npm verb stack Error: command failed
npm verb stack at ChildProcess.<anonymous> (/yocto/project/nuc-build/tmp/work/corei7-64-poky-linux/runmd/1.3.6-r0/recipe-sysroot-native/usr/lib/node_modules/npm/node_modules/#npmcli/promise-spawn/index.js:64:27)
npm verb stack at ChildProcess.emit (node:events:526:28)
npm verb stack at maybeClose (node:internal/child_process:1092:16)
npm verb stack at Socket.<anonymous> (node:internal/child_process:451:11)
npm verb stack at Socket.emit (node:events:526:28)
npm verb stack at Pipe.<anonymous> (node:net:687:12)
npm verb pkgid runmd#1.3.6
npm verb cwd /tmp/tmpkm1z714f
npm verb Linux 5.13.0-1029-azure
npm verb argv "/yocto/project/nuc-build/tmp/work/corei7-64-poky-linux/runmd/1.3.6-r0/recipe-sysroot-native/usr/bin/node" "/yocto/project/nuc-build/tmp/work/corei7-64-poky-linux/runmd/1.3.6-r0/recipe-sysroot-native/usr/bin/npm" "pack" "/yocto/project/nuc-build/tmp/work/corei7-64-poky-linux/runmd/1.3.6-r0/npm-package" "--ignore-scripts=true"
npm verb node v16.14.2
npm verb npm v8.5.0
npm ERR! code 127
npm ERR! path /yocto/project/nuc-build/tmp/work/corei7-64-poky-linux/runmd/1.3.6-r0/npm-package
npm ERR! command failed
npm ERR! command sh -c husky install
npm ERR! sh: 1: husky: not found
npm verb exit 127
npm timing npm Completed in 132ms
npm verb code 127
npm ERR! A complete log of this run can be found in:
npm ERR! /yocto/project/nuc-build/tmp/work/corei7-64-poky-linux/runmd/1.3.6-r0/npm-cache/_logs/2022-06-20T09_50_23_086Z-debug-0.log
ERROR: Logfile of failure stored in: /yocto/project/nuc-build/tmp/work/corei7-64-poky-linux/runmd/1.3.6-r0/temp/log.do_compile.37326
ERROR: Task (/yocto/project/nuc-build/workspace/recipes/runmd/runmd_1.3.6.bb:do_compile) failed with exit code '1'
NOTE: Tasks Summary: Attempted 1074 tasks of which 1067 didn't need to be rerun and 1 failed.
Summary: 1 task failed:
/yocto/project/nuc-build/workspace/recipes/runmd/runmd_1.3.6.bb:do_compile
Summary: There was 1 ERROR message, returning a non-zero exit code.
(The complete log referenced doesn't really help neither)
Thanks a lot in advance !

Answer 1

npm with kirkstone is broken; you can try patches from https://patchwork.yoctoproject.org/project/oe-core/list/?series=4303