How to manage module dependencies in jq - module

Question

Since version 1.5 the jq data processing language has a library module system. A module consists of optional metadata and a set of functions. For instance
module { name: "util", version: "1.0.0" };
def digitsum: tostring|split("")|map(tonumber)|add;
stored as file util.jq can be used like this:
$ echo '789' | jq -L. 'include "util"; digitsum'
24
Modules can use other modules and the dependencies are tracked by the modulemeta directive but how to express and check for a minimum version of a module number? For instance:
module {
name: "math",
version: "0.1.0",
};
include "util"; # TODO: require at least version 1.0.0!
def digitroot:
(.|digitsum) as $sum |
if $sum<10 then $sum else $sum|digitroot end;

Answer 1

The support for modules in jq is currently (June 2019) still very minimal, though on github there is a module management system for jq: https://github.com/joelpurra/jqnpm
Without using such an external module management system, what can be done in jq itself? Extending the given example, the following illustrates one approach to supporting version requirements. Notice the additional key named dependencies in the metadata of the math module. (Currently, this key cannot be named deps as jq overwrites it.)
Files
dependencies.jq
# Recursively check specified version constraints
module { name: "dependencies", version: "0.0.2" };
# parents of a module as defined by its .deps
def parents:
. as $in
| if type == "array" then map(parents) | add
else modulemeta | .deps | map(.relpath)
end ;
# ancestors of a single module or an array of modules.
# The array of "ancestors" of a module includes itself.
def ancestors:
# input and $visited should be arrays of strings
def ancestors($visited):
. as $in
| ($in - $visited) as $new
| if $new == [] then $visited
else $new | parents | ancestors($visited + $new | unique)
end;
if type == "array" then . else [.] end
| ancestors([]) ;
def versionsort:
def parse:
sub("(?<a>(alpha|beta|gamma))"; "\(.a).")
| [splits("[-.]")]
| map(tonumber? // .) ;
sort_by(parse);
# Input: a module name
# Emit empty if the constraints for the given module are satisfied, otherwise raise an error
def dependencies($version):
def le($y): (. == $y) or ([.,$y] | . == versionsort);
modulemeta
| .version as $mv
| if (($mv == null) or ($version | le($mv))) then empty
else ("module \(.name) dependencies version \($version) vs \($mv)" | error)
end ;
# Input: a module name or array of module names
# Check the module-version dependencies in .dependencies, proceeding up the chain as defined by .deps
def dependencies:
def check:
modulemeta
| select(has("dependencies"))
| all( .dependencies | to_entries[];
.key as $m | .value as $v | ($m | dependencies($v) ))
| empty;
ancestors[] | check;
util.jq
module { name: "util", version: "1.0.0" };
def digitsum: tostring|split("")|map(tonumber)|add;
math.jq
module {
name: "math",
version: "0.1.0",
dependencies: {"util": "1.0.0"} };
include "util" ;
def digitroot:
digitsum as $sum
| if $sum<10 then $sum
else $sum|digitroot
end;
Invocation
jq -n -L . '
include "dependencies";
include "math";
"math" | dependencies,
(123|digitroot) '