Bundle npm module 'cheerio' in K6 test - npm

Question

I am trying to create some tests using K6 framework from LoadImpact, but I am struggelig with including external NPM module following the instructions on their documentation site.
On loadImpacts documentations site they include a detailed example on just what I am after, modules that enable me to parse xml from a soap service response. But, I am unable to get this working! Now, I am a total javascript newbie, but I have been coding for many years and would really like to solve this.
The can be found here: https://docs.k6.io/docs/modules#section-npm-modules
can anyone get this working? I need to run this on servers isolated from the Internet, so I am totaly dependent on creating the packages and transfer the required files.
According to the documentation a package is created like this
-- bundle `cheerio` npm module
git clone git#github.com:cheeriojs/cheerio.git
npm install browserify index.js -s cheerio > cheerio.js
My first question: In the folder I am residing when running this command a 'cheerio.js' file is created along with a a 'cheerio' folder and a 'node_modules' folder.
the cheerio.js in my "root" directory only contains the following:
+ cheerio#0.22.0
+ index.js#0.0.3
+ browserify#16.2.3
updated 3 packages and audited 2829 packages in 2.221s
found 0 vulnerabilities
Back to LoadImpacts example on how to reference this package in a k6 javascript:
import cheerio from "./vendor/cheerio.js";
export default function()
{
const res = http.get("https://loadimpact.com/");
const $ = cheerio.load(res.body);
What file is this, and where in the structure generated by browserify can I find it? I have tried to change this to point to 'index.js' in the 'cheerio' folder or cheerio.js found in 'cheerio/lib'. I will then receive a complaint about the first line in cheerio.js which defines a "parse" variable it cannot find:
var parse = require("./parse'),
if I change this to
var parse = require("./parse.js')
it goes on to complain about missing 'htmlparser2' which I can also find in this structure, but it seems like the entire dependency structure is not working.
Can anybody give me some guidance on how to create a browserify package with dependencies for cheerio and how/what I need to copy to my k6 project to make this work like on the loadImpact site.

Answer 1

The k6 docs for this definitely need some clarification, which I'll later do. The vendor folder currently mentioned there isn't something special, the docs are just missing a step to copy the cheerio.js and xml2js.js files that were generated by browserify to a new vendor folder in your k6 project.
For now, I'll try to offer a simplified explanation on how to achieve the same thing in a simpler way:
Create a new empty folder and go to it in a terminal
Run npm install browserify cheerio there (ignore the npm warnings about missing package.json or description)
Run ./node_modules/.bin/browserify ./node_modules/cheerio/ -s cheerio > cheerio.js in that folder
The resulting cheerio.js file in the folder root should be the file you import from the k6 script:
import http from "k6/http";
import cheerio from "./cheerio.js";
export default function () {
const res = http.get("https://loadimpact.com/");
const $ = cheerio.load(res.body);
console.log($('head title').text())
}
That should be it for a single npm library.
And if you need to use multiple npm packages, it might be better to invest some time into bundling them in a single browserified .js file. For example, if you need both the cheerio and the xml2js libraries mentioned in the k6 docs, you can do something like this:
Create a new empty folder
Add something like the following package.json file in it:
{
"name": "k6-npm-libs-demo",
"version": "0.0.1",
"description": "just a simple demo of how to use multiple npm libs in k6",
"main": "npm-main.js",
"dependencies": {},
"devDependencies": {
"browserify": "*",
"cheerio": "*",
"xml2js": "*"
},
"scripts": {
"install": "./node_modules/.bin/browserify npm-main.js -s npmlibs > vendored-libs.js"
},
"author": "",
"license": "ISC"
}
Of course, if you need different libraries than cheerio and xml2js, you need to adjust the devDependencies options.
Add an npm-main.js file like this (again, adjusting for the libraries you want):
exports.xml2js = require('xml2js');
exports.cheerio = require('cheerio');
Open that folder in a terminal and run npm install. That should result in the creation of a vendored-libs.js file in the root of the folder, which you can use in k6 like this:
import http from "k6/http";
import { cheerio, xml2js } from "./vendored-libs.js";
export default function () {
const res = http.get("https://loadimpact.com/");
const $ = cheerio.load(res.body);
console.log($('head title').text())
var xmlString = '<?xml version="1.0" ?>' +
'<items xmlns="http://foo.com">' +
' <item>Foo</item>' +
' <item color="green">Bar</item>' +
'</items>'
xml2js.parseString(xmlString, function (err, result) {
console.log(JSON.stringify(result));
});
}