There is a zip file at some http:// | https:// location.
In this case it's a zip of assets, in particular a font, so it's not a js package and has no git repository or other kind of repository.
With bower one could just reference the location and it would download and extract the zip file.
How can I achieve this with npm? Preferably without hacks, because my postinstall script already does a bunch of stuff.
Add this in file package.json:
"scripts": {
"postinstall": "node download.js && <whatever you had here before>"
},
"dependencies": {
"download-file": "0.1.5"
}
And then add this in file download.js next to file package.json:
var download = require('download-file')
const url = "https://YourFileLocation";
const options = {directory: "YourDirName", filename: "YourFileName"};
console.log("Installing " + options.directory + "/" + options.filename);
download(url, options, function(error) {if (error) throw error;});
See: https://www.npmjs.com/package/download-file
Related
When migrated from bower to yarn and ran the command
yarn install
yarn created #bower_components folder and all the bower/front-end components were add inside this folder ../node_modules/#bower_components
"dependencies": {
...
"#bower_components/bootstrap": "twbs/bootstrap#^3.3.7",
"#bower_components/jquery": "jquery/jquery-dist#^3.1.1",
"#bower_components/lodash": "lodash/lodash#^4.17.12",
"#bower_components/moment": "moment/moment#^2.19.1",
... }
If, I need to create migrate from #bower_components to #frontend_components or add to public/lib folder. How do I do it?
yarn --cwd /public/lib add lodash
The way bower-away works is by creating a symlink from /bower_components to /node_modules/#bower_componets:
bower-away gets away with this by resolving all dependencies with
Bower, and adding all of them flattened to package.json
Then, your package.json file with contain bower dependencies as:
{
"dependencies": {
"#bower_components/bootstrap": "twbs/bootstrap#^3.3.7",
"#bower_components/jquery": "jquery/jquery-dist#^3.1.1",
"#bower_components/lodash": "lodash/lodash#^4.17.12",
"#bower_components/moment": "moment/moment#^2.19.1",
}
}
This will work in most scenarios. But in your case, the symlink is not working. I ran the same issue and my fix was to modify my express server and map /bower_components front-end resource into /node_modules/#bower_components backend resource with the following line:
New
app.use("/bower_components", express.static(path.join(__dirname, "/node_modules/#bower_components")));
Original
//app.use("/bower_components", express.static(path.join(__dirname, "/bower_components")));
If this is not your case, you may need to manually update front-end references to new node_modules/#bower_components folder just as advised as the original author: Adam Stankiewicz
But initially, the only change required in code is to change any
reference to bower_components with node_modules/#bower_components
(though you can link it somewhere else in postinstall script).
The workaround which I implemented to resolve this issue was by introducing a simple script and updating the package.json with a new key.
Under package.json (for all the UI related dependencies needed to work with the front-end)
...
...
"scripts": {
"postinstall": "node migrateUI.js"
},
...
"uidependencies": {
...
"bootstrap": "^3.3.7"
"jquery": "^3.1.1",
"lodash": "*",
"moment": "^2.19.1",
...
},
"dependencies": {
....
"bootstrap": "^3.3.7"
"jquery": "^3.1.1",
"lodash": "*",
"moment": "^2.19.1",
....
}
migrateUI.js
const uipackage = require('./package.json');
const packageName = Object.keys(uipackage.uidependencies);
const dir = 'public/libs';
//if the folder already exists, it ignores else creates.
if (!fs.existsSync(dir)) {
fs.mkdirSync(dir);
}
for (let i = 0; i < packageName.length; i++) {
const element = packageName[i];
const source = path.resolve('node_modules/' + element);
const target = path.resolve('public/libs/' + element); //custom lib folder to save all UI dependencies
if (!fs.existsSync(target)) {
fs.symlinkSync(source, target, 'dir', (err) => {
if (err && err.code !== 'EEXIST') {
console.log('Error creating dependecny symlink - ', err);
} else {
console.log('Symlink for dependency created');
}
});
}
}
I have a project that has around 50 packages installed through NPM. After several months of neglect I now need to update most packages, resulting in several mismatching peer dependencies. In order to find the correct combination of package versions I want to see the peer dependencies of all versions of a certain package.
Where on www.npmjs.com can I find a package's peer dependencies?
A package's page shows "dependencies" and "dependents", but I believe these are normal dependencies, not "peer dependencies".
The www.npmjs.com website doesn't expose peer dependencies information. However, this metadata resides at the npm registry endpoint: https://registry.npmjs.org/.
Via your command line you can access it by utilizing the npm view command. For instance:
npm view <pkg_name> peerDependencies
Note: You'll need to substitute the <pkg_name> part with a real package name.
The aforementioned command will list peerDependencies for the latest version of the given package.
I want to see the peer dependencies of all versions of a certain package.
To achieve this you can:
Run the following command to obtain all versions for a given package
npm view babel-loader versions --json
Note Here we check for babel-loader but this could be any valid package name
This will print:
[
"4.0.0",
"4.1.0",
"4.2.0",
"4.3.0",
"5.0.0",
...
]
Then run the following command to obtain the peer dependencies for each version previously listed:
npm view babel-loader#4.0.0 peerDependencies --json
^^^^^^
prints:
{
"babel-core": "^4.0.0",
"webpack": "^1.4.5"
}
Repeat again - similar to before but change the #<version> suffix, i.e. #4.1.0 in the example below.
npm view babel-loader#4.1.0 peerDependencies --json
^^^^^^
This prints:
{
"babel-core": "^4.7.0",
"webpack": "^1.4.5"
}
and so on...
Automate the task:
You may want to consider automating the steps above by creating a node.js script as follows. This utilizes the nodejs execSync() method to shell out the necessary npm commands, however you could change it to utilize the exec() method if you want it to run asynchronously:
script.js
const fs = require('fs');
const sh = require('child_process').execSync;
const PKG_NAME = 'babel-loader'; // <-- Change the package name.
const destFilePath = PKG_NAME.replace(/\//g, '_') + '-peer-deps.json';
const versions = JSON.parse(sh('npm view ' + PKG_NAME + ' versions --json').toString());
const data = versions.map(function(semver) {
const pkgVersion = PKG_NAME + '#' + semver;
console.log('> Fetching peerDependencies info for: ' + pkgVersion);
const peerDeps = sh('npm view ' + pkgVersion +
' peerDependencies --json').toString().replace(/[\r\n]/, '');
return {
name: pkgVersion,
peerDependencies: peerDeps ? JSON.parse(peerDeps) : null
}
});
fs.writeFileSync(destFilePath, JSON.stringify(data, null, 2));
console.log('\nDone !');
Then run the following command to invoke the node.js script:
node ./path/to/script.js
Note: You'll need to redefine the ./path/to/ part as necessary.
Given the value of babel-loader currently assigned to the PKG_NAME variable in script.js you'll see something like the following logged to your console:
> Fetching peerDependencies info for: babel-loader#4.0.0
> Fetching peerDependencies info for: babel-loader#4.1.0
> Fetching peerDependencies info for: babel-loader#4.2.0
...
Upon completion it will write a .json file to disk named babel-loader-peer-deps.json, which includes the following content:
babel-loader-peer-deps.json
[
{
"name": "babel-loader#4.0.0",
"peerDependencies": {
"babel-core": "^4.0.0",
"webpack": "^1.4.5"
}
},
{
"name": "babel-loader#4.1.0",
"peerDependencies": {
"babel-core": "^4.7.0",
"webpack": "^1.4.5"
}
},
{
"name": "babel-loader#4.2.0",
"peerDependencies": {
"babel-core": "^4.7.0",
"webpack": "^1.4.5"
}
},
...
]
EDIT: Reducing the number of https GET requests
If you're wanting to reduce the number of https GET requests to just one then I suggest utilizing the nodejs builtin https.get() to fetch JSON data from the https://registry.npmjs.org/ endpoint.
This example gist below will be much faster.
get-peer-deps.js
const fs = require('fs');
const path = require('path');
const https = require('https');
const pkgName = process.argv[2];
const myName = path.basename(__filename);
// Check the package name has been provided
if (!pkgName) {
console.log('\x1b[40;37m%s\x1b[0m \x1b[40;31m%s\x1b[0m', myName,
'ERR!', 'Missing package name argument');
process.exit(1);
}
const fileName = pkgName.replace(/\//g, '_') + '-peer-deps.json';
const destFilePath = path.join(path.dirname(__filename), fileName);
const endPoint = 'https://registry.npmjs.org/' + encodeURIComponent(pkgName);
// Request JSON from npm registry endpoint.
https.get(endPoint, function(resuest) {
console.log('> Fetching peerDependencies info for: %s...', pkgName);
var response = '';
resuest.on('data', function(chunk) {
response += chunk;
});
resuest.on('end', function() {
processJsonResponse(response);
});
}).on('error', function(err) {
console.error(err);
});
/**
* Processes the JSON response to extract the necessary metadata and saves
* resultant JSOn to disk.
* #param {String} data - The JSON response from the npm registry.
*/
function processJsonResponse(data) {
const versions = JSON.parse(data).versions;
const semvers = Object.keys(versions);
const peerDepsInfo = semvers.map(function(semver) {
const current = versions[semver];
return {
name: current.name + '#' + current.version,
peerDependencies: current.peerDependencies || null
};
});
fs.writeFile(destFilePath, JSON.stringify(peerDepsInfo, null, 2), function(err) {
if(err) {
return console.log(err);
}
console.log('> Done. Saved result to:\n %s', destFilePath);
});
}
Usage:
Then run the following command via your command line tool to invoke get-peer-deps.js:
node ./path/to/get-peer-deps.js babel-loader
^^^^^^^^^^^^
Note: When invoking get-peer-deps.js it's necessary to provide the package name as an argument. In the example above we pass in babel-loader. This can be replaced with whichever valid package name you prefer. For example in the next example we pass in #angular/forms:
node ./path/to/get-peer-deps.js #angular/forms
^^^^^^^^^^^^^^
The resultant .json file will be formatted as described previously and will be save to the same directory where get-peer-deps.js resides.
I am trying to create some tests using K6 framework from LoadImpact, but I am struggelig with including external NPM module following the instructions on their documentation site.
On loadImpacts documentations site they include a detailed example on just what I am after, modules that enable me to parse xml from a soap service response. But, I am unable to get this working! Now, I am a total javascript newbie, but I have been coding for many years and would really like to solve this.
The can be found here: https://docs.k6.io/docs/modules#section-npm-modules
can anyone get this working? I need to run this on servers isolated from the Internet, so I am totaly dependent on creating the packages and transfer the required files.
According to the documentation a package is created like this
-- bundle `cheerio` npm module
git clone git#github.com:cheeriojs/cheerio.git
npm install browserify index.js -s cheerio > cheerio.js
My first question: In the folder I am residing when running this command a 'cheerio.js' file is created along with a a 'cheerio' folder and a 'node_modules' folder.
the cheerio.js in my "root" directory only contains the following:
+ cheerio#0.22.0
+ index.js#0.0.3
+ browserify#16.2.3
updated 3 packages and audited 2829 packages in 2.221s
found 0 vulnerabilities
Back to LoadImpacts example on how to reference this package in a k6 javascript:
import cheerio from "./vendor/cheerio.js";
export default function()
{
const res = http.get("https://loadimpact.com/");
const $ = cheerio.load(res.body);
What file is this, and where in the structure generated by browserify can I find it? I have tried to change this to point to 'index.js' in the 'cheerio' folder or cheerio.js found in 'cheerio/lib'. I will then receive a complaint about the first line in cheerio.js which defines a "parse" variable it cannot find:
var parse = require("./parse'),
if I change this to
var parse = require("./parse.js')
it goes on to complain about missing 'htmlparser2' which I can also find in this structure, but it seems like the entire dependency structure is not working.
Can anybody give me some guidance on how to create a browserify package with dependencies for cheerio and how/what I need to copy to my k6 project to make this work like on the loadImpact site.
The k6 docs for this definitely need some clarification, which I'll later do. The vendor folder currently mentioned there isn't something special, the docs are just missing a step to copy the cheerio.js and xml2js.js files that were generated by browserify to a new vendor folder in your k6 project.
For now, I'll try to offer a simplified explanation on how to achieve the same thing in a simpler way:
Create a new empty folder and go to it in a terminal
Run npm install browserify cheerio there (ignore the npm warnings about missing package.json or description)
Run ./node_modules/.bin/browserify ./node_modules/cheerio/ -s cheerio > cheerio.js in that folder
The resulting cheerio.js file in the folder root should be the file you import from the k6 script:
import http from "k6/http";
import cheerio from "./cheerio.js";
export default function () {
const res = http.get("https://loadimpact.com/");
const $ = cheerio.load(res.body);
console.log($('head title').text())
}
That should be it for a single npm library.
And if you need to use multiple npm packages, it might be better to invest some time into bundling them in a single browserified .js file. For example, if you need both the cheerio and the xml2js libraries mentioned in the k6 docs, you can do something like this:
Create a new empty folder
Add something like the following package.json file in it:
{
"name": "k6-npm-libs-demo",
"version": "0.0.1",
"description": "just a simple demo of how to use multiple npm libs in k6",
"main": "npm-main.js",
"dependencies": {},
"devDependencies": {
"browserify": "*",
"cheerio": "*",
"xml2js": "*"
},
"scripts": {
"install": "./node_modules/.bin/browserify npm-main.js -s npmlibs > vendored-libs.js"
},
"author": "",
"license": "ISC"
}
Of course, if you need different libraries than cheerio and xml2js, you need to adjust the devDependencies options.
Add an npm-main.js file like this (again, adjusting for the libraries you want):
exports.xml2js = require('xml2js');
exports.cheerio = require('cheerio');
Open that folder in a terminal and run npm install. That should result in the creation of a vendored-libs.js file in the root of the folder, which you can use in k6 like this:
import http from "k6/http";
import { cheerio, xml2js } from "./vendored-libs.js";
export default function () {
const res = http.get("https://loadimpact.com/");
const $ = cheerio.load(res.body);
console.log($('head title').text())
var xmlString = '<?xml version="1.0" ?>' +
'<items xmlns="http://foo.com">' +
' <item>Foo</item>' +
' <item color="green">Bar</item>' +
'</items>'
xml2js.parseString(xmlString, function (err, result) {
console.log(JSON.stringify(result));
});
}
I am trying to delete a folder on post compile but cant make rd or del commands to work. Is there a way to delete a folder and subfolders from the scripts events?
Put the commands that you need in a batch or shell file and invoke that file in the post compile step
Use npm and gulp task, that not depends on the OS
Add a package.json file in your project root with gulp and gulp-rimraf dependencies:
{
"version": "1.0.0",
"description": "your description",
"name": "your project name",
"readme": "yuour readme",
"license": "Apache-2.0",
"dependencies": {
},
"devDependencies": {
"gulp": "3.9.1",
"gulp-rimraf": "0.2.0",
},
"scripts": {
"gulp": "gulp",
}
}
Add a gulpfile.js in your project root containing the cleanup task :
/// <binding Clean='clean' />
"use strict";
var gulp = require("gulp"),
rimraf = require("gulp-rimraf");
gulp.task("clean:js", function (cb) {
return rimraf("path to js folder to delete", cb);
});
gulp.task("clean:css", function (cb) {
return rimraf("path to css folder to delete", cb);
});
gulp.task("default", ["clean:js", "clean:css"]);
In your project.json, call the npm script :
{
...
"scripts": {
"precompile": "npm run gulp" // this will call the default gulp task
},
}
For more information, you can read the doc about how to use gulp on the asp.net docs site
You can do the same using grunt instead of gulp if you want
I am publishing a library to NPM.
When I build the library, the resulting artifact is placed in the dist folder located in the root of my project as index.js.
When users install from NPM I would like index.js to be present in the root of the folder created in their node_modules folder. Presently, it remains in a directory named dist.
How can I do this?
My packages.json:
{
"name": "my-package",
"version": "0.0.9",
"files": ["dist/*"],
"main": "index.min.js",
"private": false,
"dependencies": {},
"devDependencies": {},
"repository": "git#github.com:username/my-package.git"
}
I had exactly the same problem.
I solved it not by copying the files up, but by copying the files I needed down into the ./dist/ folder and then doing an npm publish from there; NPM then treats that folder as a complete package and everything works very nicely. The only files I needed to copy from the root folder were:
package.json
README.md
Because we're going to copy these files down into the ./dist/ folder before we do the publish, we do NOT want the package.json file to reference ./dist/. So remove the package.json's files entry completely, because we don't need to tell it which files we'll take - we're going to take everything in the ./dist/ folder. I'm using TypeScript so I also have a typings entry, and again no reference to ./dist/.
{
"name": "my-package",
"version": "0.0.9",
"main": "index.min.js",
"typings": "index.d.ts",
"private": false,
"dependencies": {},
"devDependencies": {},
"repository": "git#github.com:username/my-package.git"
}
Now for the publish step. I built a gulp task that will perform the publish for me, making it nice and automated (except for incrementing the package version #).
From gulp I'll use Node's spawn() to kick-off the npm process. However, because I'm actually working on Windows I used "cross-spawn" rather than the normal built-in Node.js spawn (which I learned the hard way didn't work when I had spaces in my path!).
Here's my gulp file, with the TypeScript bits removed:
var gulp = require('gulp');
var del = require('del');
var spawn = require('cross-spawn'); // WAS: require('child_process').spawn;
var config = {
src: { tsFiles: './src/**/*.ts' },
out: { path: './dist/' }
}
gulp.task('clean', () => {
return del('dist/*');
});
gulp.task('build', ['clean'], () => {
....
});
gulp.task('publish', ['build'], (done) => {
// Copy the files we'll need to publish
// We just use built-in gulp commands to do the copy
gulp.src(['package.json', 'README.md']).pipe(gulp.dest(config.out.path));
// We'll start the npm process in the dist directory
var outPath = config.out.path.replace(/(\.)|(\/)/gm,'');
var distDir = __dirname + '\\' + outPath + '\\';
console.log("dist directory = " + distDir);
// Start the npm process
spawn('npm', ['publish'], { stdio:'inherit', cwd:distDir } )
.on('close', done)
.on('error', function(error) {
console.error(' Underlying spawn error: ' + error);
throw error;
});
});
Notice when we call spawn() we pass in a 3rd argument which is the options. The main entry here is the cwd:distDir, which tells spawn to run the npm process from the ./dist/ directory. Because using spawn can cause problems I've hooked into the spawn error handling. As I was troubleshooting my use of spawn() I found the following StackOverflow article very helpful.
This worked like a charm; my published package has all the files in the root directory and the ./dist/ folder is not published.