I was playing around with a plugin thingy that would load things that were available. The docs on the $*REPO is not quite there so I guessed a bit. This seems to work but I have the feeling I'm missing something simpler (besides the regular golfing on the other bits):
my #modules = <Digest::MD5 NotThere PrettyDump>;
my #installed = gather installed-modules( #modules );
put "Already installed: #installed[]";
try require ::( #installed[0] );
# is there a better way to do this without eval
my $digest = ::( #installed[0] ).new;
sub installed-modules ( *#candidates ) {
for #candidates -> $module {
put $module, '-' x 15;
my $ds = CompUnit::DependencySpecification.new:
:short-name($module);
if $*REPO.resolve: $ds {
put "Found $module";
take $module;
}
else {
put "Didn't find $module";
}
}
}
$*REPO.resolve(CompUnit::DependencySpecification.new(:short-name<Test>))
Note that this is only useful to a certain degree as this only tells you if a module can be resolved. What I mean by this is it would also detect a non-installed module being provided by a directory such as -I lib, and you won't know which CompUnit::Repository it came from. You could also grep the results of something like $*REPO.repo-chain.grep(* ~~ CompUnit::Repository::Installable).map(*.installed).flat
Additionally the meaning of an "installed" module is not so simple -- CompUnit::Repository::Installable repositories are what is likely implied, but consider a third party CompUnit::Repository ( such as https://github.com/ugexe/Perl6-CompUnit--Repository--Tar ) -- with this modules are essentially still installed, but the repo itself is not CompUnit::Repository::Installable. All ::Installable really means in rakudo is that rakudo knows how to install it -- it has nothing to do with what rakudo knows how to find and load
Some PRs ( closed, but I will revisit eventually ) that help address some of these problems via a method candidates { ... }:
https://github.com/rakudo/rakudo/pull/1125
https://github.com/rakudo/rakudo/pull/1132
Related
TL;DR
In Puppet Enterprise, how do I run a manifest (testpp.pp) from a task or plan (not Bolt).
plan base_windows::testplan (
TargetSpec $targets,
Optional[String] $contents = undef,
String $filename,
){
$apply_prep($targets)
$apply_results = apply($targets, '_catch_errors' => true) {
class { 'base_windows::testpp': }
}
$apply_results.each | $result | {
notice($result.report)
}
}
apply_prep seems to succeed, but apply is failing with the following error:
{
"msg" : "Evaluation Error: Unknown function: 'report'. (file: /opt/puppetlabs/server/data/orchestration-services/code/environments/development/modules/base_windows/plans/testplan.pp, line: 16, column: 19)",
"kind" : "bolt/plan-failure",
"details" : {
"class" : "Bolt::PAL::PALError"
}
}
If I change the code to:
plan base_windows::testplan (
TargetSpec $targets,
Optional[String] $contents = undef,
String $filename,
){
apply_prep($targets)
$apply_results = apply($targets, '_catch_errors' => true) {
# Is this how to call a class? I cannot find an example.
class { 'base_windows::testpp': }
}
$apply_results.each |$result| {
$target = $result.target.name
if $result.ok {
out::message("${target} returned a value: ${result.value}")
} else {
out::message("${target} errored with a message: ${result.error.message}")
}
}
}
The plan tells me it has failed, but there are no errors in the node's report. In fact, there is no entry for the time the plan was executed.
I cannot find any examples on how to call a class from a plan, so the above apply() is a guess, based on this documentation.
I have installed the puppetlabs_reboot module and successfully ran a plan using it, therefore, I conclude my system is set up correctly, it's just my code that is wrong.
Background
I may be going about this all wrong, so here is some background to the problem. Currently, I have a series of manifests that install various packages from the public Chocolatey repository depending on a node's classification. Package definitions are stored in Hiera data and each package' version is set to latest. At the end of the Package{} resource, some manifests include a reboot.
These manifests are used to provision new nodes and keep existing nodes up-to-date with the latest package version.
The Puppet agent is set to run once per hour and if the source package is updated in the Chocolatey repo, on the next Puppet run, the manifest will update the package, rebooting the node, if required.
Goal
New nodes are provisioned with the latest package version.
Prevent package updates at undetermined times on existing nodes.
Continue to allow Puppet agent runs every hour.
Make use of existing manifests.
Ideas
Split out the package{} code from the profile manifest and place them in tasks / plans, allowing packages to be updated out-of-hours.
Specify the actual package version in Hiera. Although this is more declarative and idempotent, it means keeping an eye on over 100 package version. I guess it would be fairly simple to interrogate the Chocolatey repos with code to pull the latest version number, but even so I am no better off.
Create a task with a script that runs choco upgrade all, however, the next Puppet run would revert package versions according to the version defined in Hiera, meaning Hiera still needs to be kept up-to-date.
Problems
As per the main crux of this question, how do I run manifests (classes) from plans? If I understand correctly, tasks are for ad-hoc scripts, whereas plans can run tasks and manifests. As a lot of time has been invested in writing manifests, I would prefer not to rewrite all my manifests as scripts.
I am confused by the Puppet documentation as it seems to switch between PE and Bolt syntax. I am using Puppet Enterprise where Puppet says they don't recommend using Bolt but their examples seem to site Bolt commands.
No errors in the node' report. apply_prep() reports executed successfully, albeit taking far longer to execute than puppetlabs_reboot module, but apply() results in a failure, but nothing is logged in the node's reports.
Using puppetlabs_reboot module as a reference, it appears their plan uses a bunch of tasks. It appears that they don't use apply() to run their reboot{} class. Is this not duplicating the work?
If anyone has any suggestions or ideas, I'd be grateful if you could share.
I've got it to work. The class I was trying to run, required parameters that I hadn't provided!
plan base_windows::testplan (
TargetSpec $targets,
Optional[String] $contents = undef,
String $filename,
){
apply_prep($targets)
$apply_results = apply($targets, '_catch_errors' => true) {
class { 'base_windows::testpp':
filename => $filename,
contents => $contents,
}
}
}
# Output the whole result_set in the PE console
return $apply_results
I found this out using the logs.
Turn on debug level logging in /etc/puppetlabs/puppetserver/logback.xml (root level="debug")
Tail the following logs:
tail -f /var/log/puppetlabs/bolt-server/bolt-server.log
tail -f /var/log/puppetlabs/puppetserver/puppetserver.log | grep -B 5 -A 5 'testplan'
tail -f /var/log/puppetlabs/orchestration-services/orchestration-services.log
I am seeking help to prevent me tearing my hair out. I have a couple of interdependent modules:
Physics::Unit
Physics::Measure (which uses Physics::Unit)
Now I have created a new module Physics::UnitAffix whose job is to export combinations of SI prefixes and SI units as raku postfix operators (cm, mm, nm and so on) into the script namespace.
Here's a golfed version of the new module:
unit module Physics::UnitAffixQ:ver<0.0.4>:auth<Steve Roe (p6steve#furnival.net)>;
{
use Physics::Unit;
Unit.new( factor => 0.000000001, offset => 0, defn => 'nanometre', type => '',
dims => [1,0,0,0,0,0,0,0], dmix => ("metre"=>1).MixHash,
names => ['nm'], stock => True );
}
sub do-postfix( Real $v, Str $n, Str $t ) {
my $pmt = "Physics::Measure::$t";
return ::($pmt).new(value => $v, units => $n);
}
sub postfix:<nm> (Real:D $x) is export(:DEFAULT) { do-postfix($x,'nm','Length') }
And here's the raku script:
#!/usr/bin/env raku
use lib '../lib';
use Physics::UnitAffixQ; #must be 1st
use Physics::Measure;
my $l = 1nm; say ~$l; #1 nm
I would like users of the modules to be able to write their use statements in any order. However, if they use Physics::Measure first, I either get a SegFault. Or slightly better I have been able to pin down to this error:
No such symbol 'Physics::Measure::Length'
in sub do-postfix at /Users/stephenroe/Dropbox/ThreeMeasures/raku-Physics-Measure/bin/../lib/Physics/UnitAffixQ.rakumod (Physics::UnitAffixQ) line 14
in sub postfix:<nm> at /Users/stephenroe/Dropbox/ThreeMeasures/raku-Physics-Measure/bin/../lib/Physics/UnitAffixQ.rakumod (Physics::UnitAffixQ) line 17
in block <unit> at synopsis-unitaffixq.raku line 6
Can anyone advise on (i) whether I should have to specify the order of use statements and/or (ii) how I can fix this error?
Also to mention this previous SO and note already tried the application of use within curls to limit the scope.
My kit: Welcome to 𝐑𝐚𝐤𝐮𝐝𝐨™ v2020.10. Implementing the 𝐑𝐚𝐤𝐮™ programming language v6.d. Built on MoarVM version 2020.10.
I have created a new branch 'SO64837072' of raku-Physics-Measure on GitHub to help Reproduce this setup, please go:
zef install Physics::Unit
or zef install https://github.com/p6steve/raku-Physics-Unit.git
git clone git clone https://github.com/p6steve/raku-Physics-Measure.git
git checkout SO64837072
cd raku-Physics-Measure/bin
./synopsis-unitaffixq.raku
I am trying to create a cache of POD6 by precompiling them using the CompUnit set of classes.
I can create, store and retrieve pod as follows:
use v6.c;
use nqp;
my $precomp-store = CompUnit::PrecompilationStore::File.new(prefix=>'cache'.IO);
my $precomp = CompUnit::PrecompilationRepository::Default.new(store=> $precomp-store );
my $key = nqp::sha1('test.pod6');
'test.pod6'.IO.spurt(q:to/CONTENT/);
=begin pod
=TITLE More and more
Some more text
=end pod
CONTENT
$precomp.precompile('test.pod6'.IO, $key, :force);
my $handle = $precomp.load($key, )[0];
my $resurrected = nqp::atkey($handle.unit,'$=pod')[0];
say $resurrected ~~ Pod::Block::Named;
So now I change the POD, how do I use the :since flag? I thought that if :since contains a time after the compilation, then the value of the handle would be Nil. That does not seem to be the case.
my $new-handle = $precomp.load($key, :since('test.pod6'.IO.modified));
say 'I got a new handle' with $new-handle;
Output is 'I got a new handle'.
What I am doing wrong?
Here is a pastebin link with code and output: https://pastebin.com/wtA9a0nP
Module loading code caches look ups and essentially start with:
$lock.protect: {
return %loaded{$id} if %loaded{$id}:exists;
}
So the question becomes "how do I load a module and then unload it (so i can load it again)?" to which the answer is: you cannot unload a module. You can however change the filename, distribution longname ( via changing the name, auth, api, or version ), or precomp id -- whatever a specific CompUnit::Repository uses to uniquely identify modules -- to bypass the cache.
What seems to be overlooked is that $key is intended to represent an immutable name, such that it will always point at the same content. What version of foo.pm should be loaded for Module Used::Inside::A if foo.pm is being loaded by Module A and B at the same time, Module A loads foo first, and then Module B modifies foo? The old version Module A loaded, or the ( possibly conflicting with previous version ) Module B loaded version? And how would this differentiation work for precomp file generation themselves ( which again can happen in parallel )?
Of course if we ignore the above we could add code to make expensive .IO.modified calls for every single module load for all CompUnit::Repository types ( slowing down startup speed ) to say "hey this immutable thing changed". But the granularity of file system modified timestamps on some OS's made the check pretty fragile ( especially for multi-threaded module loading with precomp files being generated ), meaning that even more expensive calls to get a checksum would be necessary every single time.
Currently I am having some trouble using an Idris package that was installed with nixpkg in another Idris package that depends on it. Both of these packages have been tested on a Debian system, so the problem is not with the Idris code itself, but rather somewhere in how they are being installed on the NixOS system. I believe I can successfully install the first package by calling nix-env -f math.nix -i idris_math where math.nix is defined as:
with import <nixpkgs> { };
stdenv.mkDerivation rec {
name = "idris_math";
idris = haskellPackages.idris;
buildDepends = [ idris ];
src = ./.;
preHook = ''
ln -sv ${idris}/share/x86_64-linux-ghc-8.0.1/${idris.name}/libs $PWD/libs
export IDRIS_LIBRARY_PATH=$PWD/libs
'';
configurePhase = ''
export TARGET=$out/lib/${idris.name}
'';
buildPhase = ''
${idris}/bin/idris --build *.ipkg
'';
installPhase = ''
${idris}/bin/idris --install *.ipkg
${idris}/bin/idris --clean *.ipkg
'';
}
I can then run nix-env -q and see that idris_math has been installed. The second nixpkg looks identical to math.nix, except the name is changed and the buildDepends line is now buildDepends = [ idris idris_math ]. When I try to build or install this package however, I am met with error: undefined variable 'idris_math'. Does anyone know what I am doing wrong, or of a way to fix this?
When you write with import <nixpkgs> {}, every attribute defined in the Nixpkgs set is essentially made into a local variable. This means that, when you write haskellPackages, for example, you are actually referring to the haskellPackages attribute defined by Nixpkgs. You can see all the packages defined this way by looking at the file top-level/all-packages.nix in your Nixpkgs checkout.
Notice that the idris in your buildDepends is referring to the definition of an idris attribute on the previous line. Your new idris_math package isn't similarly defined in this file, or added to the Nixpkgs set anywhere, so you can't get a reference to it by looking for a local variable or for (import <nixpkgs> {}).idris_math, which is basically what is being attempted when you write idris_math in your second derivation.
Depending on what exactly you want to do with your package, you have a couple of options. If you just want to use idris_math as a dependency for another nearby package (or a few), you can just write buildDepends = [ idris (import /path/to/math.nix) ]. This is a very simple way to do what you want, and very likely a good choice if you want to use this is a library when developing Idris packages.
If you want to make idris_math part of the set that you get from import <nixpkgs> {} anywhere on your machine, and to make it easy to nix-env -i, you can try adding it to an overlay. This would require creating a file like ~/.config/nixpkgs/overlays/<my overlay name>/default.nix with contents something like:
self: super:
with super;
{
idris_math = callPackage /path/to/idris_math.nix {};
}
In this scenario, you probably also want to change your idris_math.nix header to be { stdenv, haskellPackages }:, because callPackage looks for this kind of definition and it is useful when tying the knot to combine all the overlays together.
I am new to play framework. My requirement is very simple. I want to create rest api server in scala using play framework. I could use play2-mini, but it seems it is outdated.
I want play 2.1 to be used in my project. Instead of setting play framework as dependency, I want only core module. So I have few questions -
What is core module of Play ? what is module name ?
Is it sufficient to use core module for creating asynchronous http server ?
This link says I can use core module instead of play-mini. If it's true, where can I get more info about it.
You can just simply setup a route and then point it to a controller that parses the data you send. Here is an example of json parsing and serving back a response with play.
http://www.playframework.com/documentation/2.1.1/ScalaJsonRequests
package controllers
import play.api._
import play.api.mvc._
import play.api.libs.json._
// you need this import to have combinators
import play.api.libs.functional.syntax._
object Application extends Controller {
implicit val rds = (
(__ \ 'name).read[String] and
(__ \ 'age).read[Long]
) tupled
def sayHello = Action { request =>
request.body.asJson.map { json =>
json.validate[(String, Long)].map{
case (name, age) => Ok("Hello " + name + ", you're "+age)
}.recoverTotal{
e => BadRequest("Detected error:"+ JsError.toFlatJson(e))
}
}.getOrElse {
BadRequest("Expecting Json data")
}
}
}
or even simpler...
def sayHello = Action(parse.json) { request =>
request.body.validate[(String, Long)].map{
case (name, age) => Ok("Hello " + name + ", you're "+age)
}.recoverTotal{
e => BadRequest("Detected error:"+ JsError.toFlatJson(e))
}
}
Play Framework is a highly modular project. Internally it consists of around 20 subprojects. Some of them you can include in your project as a library dependency if you need them, for example anorm or jdbc. Other projects (i.e. PlayExceptions, RoutesCompiler, TemplatesCompiler etc.) are essential for any Play application, so you don't need to declare dependency on them. These projects could be called a 'core' of Play Framework.
In other words, if you need a Play application with minimum dependencies, just don't declare dependencies you don't need.
Play sources: https://github.com/playframework/Play20
I don't think there is much problem here. For example, this is my build.sbt used for a very small project, that uses json, in which I wanted to use Play libraries, but not necessarily create a full Play app:
name := "my-small-project"
version := "0.0.1-SNAPSHOT"
resolvers ++= Seq(
"TypeSafe Repo" at "http://repo.typesafe.com/typesafe/releases",
"Sonatype Repo" at "http://oss.sonatype.org/content/repositories/releases"
)
libraryDependencies ++= Seq(
"org.specs2" %% "specs2" % "2.3.7" % "test",
"commons-codec" % "commons-codec" % "1.8",
"com.typesafe.play" % "play_2.10" % "2.2.1",
"com.typesafe.play" % "play-json_2.10" % "2.2.1"
)
And You could still remove some dependencies here, especially if You don't need Base64 encoding. Here I consider play_2.10 as "the core" you're interested in. You should get yourself familiar with sbt though, but it's not that hard.
Also remember, that the difference between "a Play application" and "an application that uses Play libraries" is quite fuzzy, especially if you use sbt. And this is the beauty of it, this shows how Play creators thoughtfully tried not to invent a wheel once again by creating custom system requirements for their project to build. You can enter a Play app dir and type "sbt compile" for example instead of firing a Play console and it should work just fine.