How to determine the dependencies of older npm package versions - npm

It appears to me that the dependency linkage on the npm site is only applicable to the current/latest version.
Is there some tidbit of information that I'm not aware of on how to determine what dependency version a package has other than by downloading it and inspecting the package.json file?
I feel like I'm wasting HOURS doing something I would expect to be much easier to do.

"It appears to me that the dependency linkage on the npm site is only applicable to the current/latest version."
Yes that's correct, www.npmjs.com will only show the dependencies for the latest version of a package.
Here are a couple of ways to discover what you want both programmatically and non-programmatically.
Programmatically:
Utilizing the npm view command with the following syntax;
npm view <pkg_name> versions --json
obtains a list of all versions available for a given package in the npm registry.
Note: The <pkg_name> part above should be substituted with the real package name.
For instance; running the following command for let's say the eslint package:
npm view eslint versions --json
prints the following to the console:
[
"0.0.4",
"0.0.5",
"0.0.6",
"0.0.7",
"0.1.0-dev",
"0.1.0",
"0.1.1",
"0.1.2",
...
]
Now we know what versions are available, let's say we want to list the dependencies for eslint version 0.1.2 we can run the following command:
npm show eslint#0.1.2 dependencies --json
This will print:
{
"optimist": "*",
"estraverse": "~1.3.0",
"esprima": "*",
"escope": "1.0.0"
}
Similarly, we can discover the devDependencies for eslint version 0.1.2 by running the following command instead:
npm show eslint#0.1.2 devDependencies --json
This will yield something like this:
{
"vows": "~0.7.0",
"sinon": "*",
"commonjs-everywhere": "~0.9.0",
"mocha": "~1.13.0",
"chai": "~1.8.1",
"grunt": "~0.4.1",
...
}
If you know that a package has a particular dependency in advance.
For instance; retrospectively after running the aforementioned command we now know that eslint version 0.1.2 has escope listed as a dependency.
So, if we wanted to know the version of escope that eslint version 0.1.2 needs, we can run the following command:
npm show eslint#0.1.2 dependencies.escope
^
Note: The package name follows the dot (.), i.e. .escope
This prints the following:
1.0.0
The non-programmatic way
I can't think of a reason why you would want to perform the following non-programmatic way instead of the aforementioned programmatic way when you have a CLI tool available to you. However, if you prefer manual tasks then here goes...
Note: YMMV using the following manual steps as it depends on how the package has been managed/maintained.
Typically, the source code of an npm package will be hosted on GitHub, so you can perform the following manual steps. This will avoid you having to download the package to inspect the package.json file.
For this we'll demonstrate for the eslint package:
Visit npmjs.com and type the name of the package in the "Search Packages" input field. We'll type eslint and hit the return key.
Next click eslint from the list of packages, which will take you to this page.
Click on the github link which typically appears on the right-hand side of the webpage and looks like this:
That will take you to the eslint repo, i.e. this one
On the Github page click the "Branch" button - which appears above the list of source code files, and looks like this:
In the pop-up panel that subsequently appears click the "Tags" button, then locate and click the version tag from the list that you want to discover it's dependencies. (Note: These tag names will typically correspond to the version released/published to npm)
This will then load the source code files in the browser for that particular release/version.
Locate the package.json file from the list of files and click it. This will load the contents of package.json in the browser, upon which you can read it and ascertain its dependencies.
Visualizing the dependency tree
I sometimes utilize this online tool https://npm.anvaka.com which can help you to visualize the complete dependency tree/graph for a given package - however it's for the latest version of a package only.
Here is the complete dependency tree/graph (many levels deep) for the latest version of eslint.

Related

Blackduck Synopsys Yarn Detector cannot find project version name

I'm using Blackduck version 5.6.2 on a Create-React-App application with dependencies installed using yarn v1.22.11.
Blackduck executes as a job in a GitLab CI pipeline. Previously, I used npm to install the packages in the blackduck step before running the scan. Blackduck scanner was able to pick up the project name and version number without any explicit configuration.
I've recently changed from NPM to yarn to take advantage of selective dependency resolution feature in attempts to overcome nested dependencies reported by Blackduck scanner. Once I converted the Blackduck step to use yarn, Blackduck picked up on yarn.lock and package.json files in my repository to use Yarn detector this was seen from the outputs (and the expected behaviour according to
Synopsys' documentation). However, it is no longer able to pick up the project version name correctly.
Here are the messages that indicate this:
2021-09-27 19:13:47 INFO [main] --- Determining project info.
2021-09-27 19:13:47 INFO [main] --- A project name could not be decided. Using the name of the source path.
2021-09-27 19:13:47 INFO [main] --- A project version name could not be decided. Using the default version text.
Once the test completes and I visit the blackduck web console for test results, I now see "Default Detect Version" as the version name instead of the version number that I had before.
I've tried passing -detect.project.version.notes=./package.json argument into the step that starts the blackduck scanner shell script, but it's still falling back onto "Default Detect Version" as the version name.
The docs for yarn detector stops after specifying arguments to include and exclude workspaces, the path to yarn binary, and if production dependencies should be included.
Does anyone have any idea why the Yarn detector is unable to pick up the version number from package.json and how I can address this issue without passing a hardcoded value as the argument to the scanner shell script?
I was able to solve both the project name and version issue as well as blackduck scanner reporting vulnerabilities within nested dependency by performing the following
Switch from using yarn to npm for package installation.
This allowed the NPM detector to utilize package.json and package-lock.json to automatically determine the project name and version.
Add the following flags to blackduck scanner when starting the shell script.
--detect.npm.include.dev.dependencies=false
--detect.npm.include.peer.dependencies=false
Example:
bash <(curl -s https://detect.synopsys.com/detect.sh) --blackduck.url="${BLACKDUCK_HUB_URL}" --blackduck.username="${BLACKDUCK_HUB_USER}" --blackduck.password="${BLACKDUCK_HUB_PASS}" --blackduck.trust.cert=true --detect.policy.check=true --detect.hub.signature.scanner.paths=./ --detect.npm.include.dev.dependencies=false --detect.npm.include.peer.dependencies=false
This will instruct blackduck scanner to not look at dev dependency and nested dependencies.
Additional notes
It is not necessary to use npm-forced-resolutions or yarn's selective dependency resolution; For future reference, npm's "override" function was not mentioned because it has not been implemented yet at the time this answer was written.
If you are using create-react-app, make sure react-scripts is moved from dependencies to devDependencies (as per Dan Abramov's recommendation to address this issue). You can do this by running npm uninstall react-scripts, then running npm install --save-dev react-scripts or npm install --save-dev react-scripts#<specific version>
first off let me point you to the Synopsys Community portal which is the best and quickest place to ask these questions and get answers https://community.synopsys.com/s/. Full disclose, I work at Synopsys and am involved with the Black Duck product.
Can I also ask if the version "5.6.2" you listed refers to the Synopsys Detect scanning tool or Black Duck itself? I ask as the latest version of Black Duck 2021.8.2 and detect 7.5.0 does support extracting the project version name from the yarn.lock and/or package.json, and you should not need to provide any additional parameters to have detect automatically pick this up.
One other thing to note is that generally you should be running the yarn command on your project prior to running detect. This produces more accurate analysis results within Black Duck and resolves the dependencies using your normal package manager tools. Which would mean detect will extract the project name and version information from the yarn.lock file not the package.json file. For more details, you can see this page here https://synopsys.atlassian.net/wiki/spaces/INTDOCS/pages/631275892/Detectors

NPM detect pre-release dependency in package.json/package-lock.json?

Is there a way to detect pre-release dependencies in package.json or package-lock.json?
My context is I would like to create a git "TAG" to indicate release candidates of my code.
I would like to make sure all my in-house developed modules do not have dependencies on any pre-release modules.
I would like to detect presence of a pre-release dependencies( "^1.0.3-NIGHTLY") in my package.json as below example:
"#mydomain/client-mgr-service": "^1.0.3-NIGHTLY", <= I would like to be able to detect this line if there is a NPM feature out-of-box
"#types/country-data": "0.0.0",
"amazon-cognito-identity-js": "^3.0.12",
Thanks in advance for any helps given~
Here is a simple solution I used to detect if there is any pre-release dependencies in my package.json. The idea is to check if there is a "-" right after semver's x.y.z numbers.
grep -P "([0-9]\d*)\.([0-9]\d*)\.([0-9]\d*)-" package.json

NPM5, What is the difference of package-lock.json with package.json?

After updating NPM to version 5, I found package-lock.json file with package.json.
What is the difference between this two files?
What are the advantages of package-lock.json?
A package.json file: lists the packages that your project depends on. allows you to specify the versions of a package that your project can use using semantic versioning rules.
According to the npm docs,
package-lock.json is automatically generated for any operations where npm modifies either the node_modules tree, or package.json . It describes the exact tree that was generated, such that subsequent installs are able to generate identical trees, regardless of intermediate dependency updates.
This file is intended to be committed into source repositories, and serves various purposes:
Describe a single representation of a dependency tree such that teammates, deployments, and continuous integration are guaranteed to install exactly the same dependencies.
Provide a facility for users to "time-travel" to previous states of node_modules without having to commit the directory itself.
To facilitate greater visibility of tree changes through readable source control diffs.
Basically package-lock.json is used to optimize the installation process by allowing npm to skip repeated metadata resolutions for previously-installed packages.
Before npm 5.x.x, package.json was the source of truth for a project. What lived in package.json was law. npm users liked this model and grew very accustomed to maintaining their package file. However, when package-lock was first introduced, it acted contrary to how many people expected it to. Given a pre-existing package and package-lock, a change to the package.json (what many users considered the source of truth) was not reflected in the package-lock.
Example: Package A, version 1.0.0 is in the package and package-lock. In package.json, A is manually edited to version 1.1.0. If a user who considers package.json to be the source of truth runs npm install, they would expect version 1.1.0 to be installed. However, version 1.0.0 is installed, despite the fact that v1.1.0 is listed is the package.json.
Example: A module does not exist in the package-lock, but it does exist in the package.json. As a user who looks to package.json as the source of truth, I would expect for my module to be installed. However since the module is not present in package-lock, it isn’t installed, and my code fails because it cannot find the module.
Read more about package-lock.json in the Official npm Documentation!
package.json records only your direct dependencies and their versions.
package-lock.json records not only your direct dependencies and exact versions, but also those of all dependencies of your dependencies - the entire dependency tree, in other words, with exact versions.
It's the fact that package-lock.json records the exact versions of all dependencies of a project, including sub-dependencies, that ensures that builds will be identical each time. (This is why npm ci bases its build on package-lock.json, not package.json.)
Builds based on package.json (as with npm i) cannot guarantee that all sub-dependencies are the exact same versions each build (e.g., if the subdependency of one of your dependencies releases an update, but the version of your direct dependency doesn't change), even if exact version numbers for direct dependencies are specified in the package.json.

How to solve dependency conflict between js-data-angular and js-data-http

I'm currently working on a project, the project has 2 npm packages installed:
js-data-http and js-data-angular.
The problem is js-data-http requires js-data#>=3.0.0 and js-data-angular requires js-data#>=2.0.0 <3. I've looked through all versions of both packages and couldn't find any version that matches the dependency for js-data.
With that dependency conflict I can't generate a npm-shrinkwrap, has anyone else gone through this?
I've figured it out, I had to set the packages in package.json to these versions:
"jquery": "2.2.1",
"js-data": "2.8.2",
"js-data-angular": "3.1.0"
Then I was able to successfully generate npm-shrinkwrap.json without conflicts, I am not sure why npm wasn't able to automatically install these three packages without conflicts though.

How does npm3 decides to install flat vs. nested?

My project depends on angular2 beta.6 and another project that depends on angular2 beta.0.
package.json for my project
"dependencies": {
"angular2": "2.0.0-beta.6",
"another-project": "0.0.1"
}
package.json for another-project
"dependencies": {
"angular2": "2.0.0-beta.0",
}
When I npm install my project, it installs angular2 twice:
node_modules/angular2 (beta.6)
node_modules/another-project/angular2 (beta.0)
Trying to understand how npm3 decides to nest angular2 beta.0. Is it because both are called angular2 and hence they can not both sit at the top level?
Trying to understand how npm3 decides to nest angular2 beta.0. Is it because both are called angular2 and hence they can not both sit at the top level?
Yes, this is correct. Node code require's a module by name, using code like this:
require('angular2');
Node itself is not aware of the different versions, that's the job of npm, so it just uses whatever module matches in the require path first, relying on matching directory names.
npm accommodates this by installing specific versions in directories for each module when a conflict occurs, so that the require path will include that first.
Yes, it is because of the beta.0. Because npm has found another version of angular2 on the global level, it will install it locally.
npm3 will install globally dependencies only if there is no other versions of the dependencies on a higher level.
Here is a little example I've found :
[node_modules]
dep A v1.0
dep B v1.0
dep A v1.0 (uses root version)
dep C v1.0
dep A v2.0 (this version is different from the root version, so it will be an nested installation)
Flat dependencies were introduced in npm v3. The documentation can be found here https://docs.npmjs.com/how-npm-works/npm3.
To answer your question from the docs
However, since B v1.0 is already a top-level dep, we cannot install B v2.0 as a top level dependency. npm v3 handles this by defaulting to npm v2 behavior and nesting the new, different, module B version dependency under the module that requires it -- in this case, module C.
So order matters. The module you install first will get it's dependency on the top level. Subsequent modules will have nested dependencies.
you can run npm dedupe to remove nested module dependencies if they exist in the top level.