Yes, still going with this. My impression is that there's this powerful facility in Raku, which is not really easy to use, and there's so little documentation for that. I'd like to kind of mitigate that.
In this case, I'm trying to force attributes to be read-only by default, to make immutable classes. Here's my attempt:
my class MetamodelX::Frozen is Metamodel::ClassHOW {
method compose_attributes($the-obj, :$compiler_services) {
my $attribute-container = callsame;
my $new-container = Perl6::Metamodel::AttributeContainer.new(
:attributes($attribute-container.attributes),
:attribute_lookup($attribute-container.attribute_table),
:0attr_rw_by_default
);
$new-container.compose_attributes($the-obj, $compiler_services);
}
}
my package EXPORTHOW {
package DECLARE {
constant frozen = MetamodelX::Frozen;
}
}
I'm calling that from a main function that looks like this:
use Frozen;
frozen Foo {
has $.bar;
method gist() {
return "→ $!bar";
}
}
my $foo = Foo.new(:3bar);
say $foo.bar;
$foo.bar(33);
I'm trying to follow the source, that does not really give a lot of facilities to change attribute stuff, so there seems to be no other way that creating a new instance of the container. And that might fail in impredictable ways, and that's what it does:
Type check failed in binding to parameter '$the-obj'; expected Any but got Foo (Foo)
at /home/jmerelo/Code/raku/my-raku-examples/frozen.raku:7
Not clear if this is the first the-obj or the second one, but any way, some help is appreciated.
I have a class Configuration that reads in environment variables:
class Configuration {
has $.config_string_a;
has $.config_string_b;
has Bool $.config_flag_c;
method new() {
sub assertHasEnv(Str $envVar) {
die "environment variable $envVar must exist" unless %*ENV{$envVar}:exists;
}
assertHasEnv('CONFIG_STRING_A');
assertHasEnv('CONFIG_STRING_B');
assertHasEnv('CONFIG_FLAG_C');
return self.bless(
config_string_a => %*ENV{'CONFIG_STRING_A'},
config_string_b => %*ENV{'CONFIG_STRING_B'},
config_flag_c => Bool(%*ENV{'CONFIG_FLAG_C'}),
);
}
}
my $config = Configuration.new;
say $config.config_string_a;
say $config.config_string_b;
say $config.config_flag_c;
Is there a more concise way to express this? For example, I am repeating the environment variable name in the check and the return value of the constructor.
I could easily see writing another, more generic class that encapsulates the necessary info for a config parameter:
class ConfigurationParameter {
has $.name;
has $.envVarName;
has Bool $.required;
method new (:$name, :$envVarName, :$required = True) {
return self.bless(:$name, :$envVarName, :$required);
}
}
Then rolling these into a List in the Configuration class. However, I don't know how to refactor the constructor in Configuration to accommodate this.
The most immediate change that comes to mind is to change new to be:
method new() {
sub env(Str $envVar) {
%*ENV{$envVar} // die "environment variable $envVar must exist"
}
return self.bless(
config_string_a => env('CONFIG_STRING_A'),
config_string_b => env('CONFIG_STRING_B'),
config_flag_c => Bool(env('CONFIG_FLAG_C')),
);
}
While // is a definedness check rather than an existence one, the only way an environment variable will be undefined is if it isn't set. That gets down to one mention of %*ENV and also of each environment variable.
If there's only a few, then I'd likely stop there, but the next bit of repetition that strikes me is the names of the attributes are just lowercase of the names of the environment variables, so we could eliminate that duplication too, at the cost of a little more complexity:
method new() {
multi env(Str $envVar) {
$envVar.lc => %*ENV{$envVar} // die "environment variable $envVar must exist"
}
multi env(Str $envVar, $type) {
.key => $type(.value) given env($envVar)
}
return self.bless(
|env('CONFIG_STRING_A'),
|env('CONFIG_STRING_B'),
|env('CONFIG_FLAG_C', Bool),
);
}
Now env returns a Pair, and | flattens it in to the argument list as if it's a named argument.
Finally, the "power tool" approach is to write a trait like this outside of the class:
multi trait_mod:<is>(Attribute $attr, :$from-env!) {
my $env-name = $attr.name.substr(2).uc;
$attr.set_build(-> | {
with %*ENV{$env-name} -> $value {
Any ~~ $attr.type ?? $value !! $attr.type()($value)
}
else {
die "environment variable $env-name must exist"
}
});
}
And then write the class as:
class Configuration {
has $.config_string_a is from-env;
has $.config_string_b is from-env;
has Bool $.config_flag_c is from-env;
}
Traits run at compile time, and can manipulate a declaration in various ways. This trait calculates the name of the environment variable based on the attribute name (attribute names are always like $!config_string_a, thus the substr). The set_build sets the code that will be run to initialize the attribute when the class is created. That gets passed various things that in our situation aren't important, so we ignore the arguments with |. The with is just like if defined, so this is the same approach as the // earlier. Finally, the Any ~~ $attr.type check asks if the parameter is constrained in some way, and if it is, performs a coercion (done by invoking the type with the value).
So I mentioned this in a comment but I figured it would be good as an actual answer. I figured this would be useful functionality for anyone building a Docker based system so took Jonanthan's example code, added some functionality for exporting Traits Elizabeth showed me and made Trait::Env
Usage is :
use Trait::Env;
class Configuration {
has $.config_string_a is env;
has $.config-string-b is env(:required);
has Bool $.config-flag-c is env is default(True);
}
The :required flag turns on die if not found. And it plays nicely with the is default trait. Attribute names are upper cased and - is replaced with _ before checking %*ENV.
I have a couple of planned changes, make it throw a named Exception rather than just die and handle Boolean's a bit better. As %*ENV is Strings having a Boolean False is a bit of a pain.
I'm about to choose what language to use for a new project: Perl5 or Perl6. 6 wins so far except that it is missing Moo's lazy attributes. The two implementations I found in modules are missing the key functionality. Hence, my attempt write my own implementation.
Role vs. Class
First problem I've got into is the content of attribute's .package for one declared in a role. Consider the followin:
role HOW1 {
method compose ( Mu $class ) {
note "HOW1.compose";
nextsame;
}
}
role HOW2 {
method compose ( Mu $class ) {
note "HOW2.compose";
nextsame;
}
}
multi trait_mod:<is> (Attribute:D $attr, :$mooish!) {
note "Attribute's package.HOW: ", $attr.package.HOW;
note '$*PACKAGE.HOW: ', $*PACKAGE.HOW;
$attr.package.HOW does HOW1;
$*PACKAGE.HOW does HOW2;
}
class Foo {
has $.bar is mooish;
}
role FooRole {
has $.baz is mooish;
}
The output of the script follows:
Attribute's package.HOW: Perl6::Metamodel::ClassHOW.new
$*PACKAGE.HOW: Perl6::Metamodel::ClassHOW.new
HOW2.compose
HOW1.compose
Attribute's package.HOW: Perl6::Metamodel::GenericHOW.new
$*PACKAGE.HOW: Perl6::Metamodel::ParametricRoleHOW.new
HOW2.compose
As it is clearly seen from the output, applying a role to a metaclass always works for classes and only works for $*PACKAGE.HOW with roles. Use of $*PACKAGE instead of .package could be considered a solution, but not the one I'd really like to use. (Though, if there is no better way...)
Accessor
I would like to provide lazy functionality for private attributes too. Yes, this will be availabe with self!bar syntax only, but this is a sacrifice I'm willing to make. 😉 The problem is that all the examples of custome-made accessor I found so far are using Attribute.set_value() method which is way too low-level. I'd like to have something like this:
role MooishHOW {
method compose ( Mu $class ) {
my $accessor = $class.^add_private_method( 'bar1',
method () is rw {
note self.WHO, ".bar1";
Proxy.new(
FETCH => -> $o {
$!bar1;
},
STORE => method ( $val ) {
note "Storing";
$!bar1 = $val;
}
);
}
);
callsame;
}
}
multi trait_mod:<is> (Attribute:D $attr, :$mooish!) {
$attr.package.HOW does MooishHOW unless $attr.package.HOW ~~ MooishHOW;
}
class Foo {
has $.bar is mooish;
has $!bar1 is mooish;
method to-bar1 {
note "bar1 val:",self!bar1;
}
}
my $inst = Foo.new;
$inst.to-bar1;
But $!bar1 notation doesn't compile because of the scope (MooishRole). Are there a trick I'm missing which would allow referencing a private attribute on self?
Tricky one
Perhaps it is possible to make an attribute to be a Proxy container? This would greatly simplify the overall logic of laziness implementation.
I have answered all my questions by finally achieving the target and released AttrX::Mooish module.
So far, the answer for the first question is: no. $*PACKAGE is currently the only way.
Second question: have no answer, but the final code has to rely on set_value() anyway.
The tricky one happened to be possible: set_value() does binding of an attribue to a container making it possible to bind to a Proxy object. No need to for sacrifices, private attributes can be accessed directly with lazyness working on them.
Thanks everybody, your answers let me work around some rough edges!
I know an alternative of reflection which is using javassist, but using javassist is a little bit complex. And because of lambda or some other features in koltin, the javassist doesn't work well sometimes. So is there any other way to iterate all fields of a data class without using reflection.
There are two ways. The first is relatively easy, and is essentially what's mentioned in the comments: assuming you know how many fields there are, you can unpack it and throw that into a list, and iterate over those. Or alternatively use them directly:
data class Test(val x: String, val y: String) {
fun getData() : List<Any> = listOf(x, y)
}
data class Test(val x: String, val y: String)
...
val (x, y) = Test("x", "y")
// And optionally throw those in a list
Although iterating like this is a slight extra step, this is at least one way you can relatively easy unpack a data class.
If you don't know how many fields there are (or you don't want to refactor), you have two options:
The first is using reflection. But as you mentioned, you don't want this.
That leaves a second, somewhat more complicated preprocessing option: annotations. Note that this only works with data classes you control - beyond that, you're stuck with reflection or implementations from the library/framework coder.
Annotations can be used for several things. One of which is metadata, but also code generation. This is a somewhat complicated alternative, and requires an additional module in order to get compile order right. If it isn't compiled in the right order, you'll end up with unprocessed annotations, which kinda defeats the purpose.
I've also created a version you can use with Gradle, but that's at the end of the post and it's a shortcut to implementing it yourself.
Note that I have only tested this with a pure Kotlin project - I've personally had problems with annotations between Java and Kotlin (although that was with Lombok), so I do not guarantee this will work at compile time if called from Java. Also note that this is complex, but avoids runtime reflection.
Explanation
The main issue here is a certain memory concern. This will create a new list every time you call the method, which makes it very similar to the method used by enums.
Local testing over 10000 iterations also show a general consistency of ~200 milliseconds to execute my approach, versus roughly 600 for reflection. However, for one iteration, mine uses ~20 milliseconds, where as reflection uses between 400 and 500 milliseconds. On one run, reflection took 1500 (!) milliseconds, while my approach took 18 milliseconds.
See also Java Reflection: Why is it so slow?. This appears to affect Kotlin as well.
The memory impact of creating a new list every time it's called can be noticeable though, but it'll also be collected so it shouldn't be that big a problem.
For reference, the code used for benchmarking (this will make sense after the rest of the post):
#AutoUnpack data class ExampleDataClass(val x: String, val y: Int, var m: Boolean)
fun main(a: Array<String>) {
var mine = 0L
var reflect = 0L
// for(i in 0 until 10000) {
var start = System.currentTimeMillis()
val cls = ExampleDataClass("example", 42, false)
for (field in cls) {
println(field)
}
mine += System.currentTimeMillis() - start
start = System.currentTimeMillis()
for (prop in ExampleDataClass::class.memberProperties) {
println("${prop.name} = ${prop.get(cls)}")
}
reflect += System.currentTimeMillis() - start
// }
println(mine)
println(reflect)
}
Setting up from scratch
This bases itself around two modules: a consumer module, and a processor module. The processor HAS to be in a separate module. It needs to be compiled separately from the consumer for the annotations to work properly.
First of all, your consumer project needs the annotation processor:
apply plugin: 'kotlin-kapt'
Additionally, you need to add stub generation. It complains it's unused while compiling, but without it, the generator seems to break for me:
kapt {
generateStubs = true
}
Now that that's in order, create a new module for the unpacker. Add the Kotlin plugin if you didn't already. You do not need the annotation processor Gradle plugin in this project. That's only needed by the consumer. You do, however, need kotlinpoet:
implementation "com.squareup:kotlinpoet:1.2.0"
This is to simplify aspects of the code generation itself, which is the important part here.
Now, create the annotation:
#Retention(AnnotationRetention.SOURCE)
#Target(AnnotationTarget.CLASS)
annotation class AutoUnpack
This is pretty much all you need. The retention is set to source because it has no value at runtime, and it only targets compile time.
Next, there's the processor itself. This is somewhat complicated, so bear with me. For reference, this uses the javax.* packages for annotation processing. Android note: this might work assuming you can plug in a Java module on a compileOnly scope without getting the Android SDK restrictions. As I mentioned earlier, this is mainly for pure Kotlin; Android might work, but I haven't tested that.
Anyways, the generator:
Because I couldn't find a way to generate the method into the class without touching the rest (and because according to this, that isn't possible), I'm going with an extension function generation approach.
You'll need a class UnpackCodeGenerator : AbstractProcessor(). In there, you'll first need two lines of boilerplate:
override fun getSupportedAnnotationTypes(): MutableSet<String> = mutableSetOf(AutoUnpack::class.java.name)
override fun getSupportedSourceVersion(): SourceVersion = SourceVersion.latest()
Moving on, there's the processing. Override the process function:
override fun process(annotations: MutableSet<out TypeElement>, roundEnv: RoundEnvironment): Boolean {
// Find elements with the annotation
val annotatedElements = roundEnv.getElementsAnnotatedWith(AutoUnpack::class.java)
if(annotatedElements.isEmpty()) {
// Self-explanatory
return false;
}
// Iterate the elements
annotatedElements.forEach { element ->
// Grab the name and package
val name = element.simpleName.toString()
val pkg = processingEnv.elementUtils.getPackageOf(element).toString()
// Then generate the class
generateClass(name,
if (pkg == "unnamed package") "" else pkg, // This is a patch for an issue where classes in the root
// package return package as "unnamed package" rather than empty,
// which breaks syntax because "package unnamed package" isn't legal.
element)
}
// Return true for success
return true;
}
This just sets up some of the later framework. The real magic happens in the generateClass function:
private fun generateClass(className: String, pkg: String, element: Element){
val elements = element.enclosedElements
val classVariables = elements
.filter {
val name = if (it.simpleName.contains("\$delegate"))
it.simpleName.toString().substring(0, it.simpleName.indexOf("$"))
else it.simpleName.toString()
it.kind == ElementKind.FIELD // Find fields
&& Modifier.STATIC !in it.modifiers // that aren't static (thanks to sebaslogen for issue #1: https://github.com/LunarWatcher/KClassUnpacker/issues/1)
// Additionally, we have to ignore private fields. Extension functions can't access these, and accessing
// them is a bad idea anyway. Kotlin lets you expose get without exposing set. If you, by default, don't
// allow access to the getter, there's a high chance exposing it is a bad idea.
&& elements.any { getter -> getter.kind == ElementKind.METHOD // find methods
&& getter.simpleName.toString() ==
"get${name[0].toUpperCase().toString() + (if (name.length > 1) name.substring(1) else "")}" // that matches the getter name (by the standard convention)
&& Modifier.PUBLIC in getter.modifiers // that are marked public
}
} // Grab the variables
.map {
// Map the name now. Also supports later filtering
if (it.simpleName.endsWith("\$delegate")) {
// Support by lazy
it.simpleName.subSequence(0, it.simpleName.indexOf("$"))
} else it.simpleName
}
if (classVariables.isEmpty()) return; // Self-explanatory
val file = FileSpec.builder(pkg, className)
.addFunction(FunSpec.builder("iterator") // For automatic unpacking in a for loop
.receiver(element.asType().asTypeName().copy()) // Add it as an extension function of the class
.addStatement("return listOf(${classVariables.joinToString(", ")}).iterator()") // add the return statement. Create a list, push an iterator.
.addModifiers(KModifier.PUBLIC, KModifier.OPERATOR) // This needs to be public. Because it's an iterator, the function also needs the `operator` keyword
.build()
).build()
// Grab the generate directory.
val genDir = processingEnv.options["kapt.kotlin.generated"]!!
// Then write the file.
file.writeTo(File(genDir, "$pkg/${element.simpleName.replace("\\.kt".toRegex(), "")}Generated.kt"))
}
All of the relevant lines have comments explaining use, in case you're not familiar with what this does.
Finally, in order to get the processor to process, you need to register it. In the module for the generator, add a file called javax.annotation.processing.Processor under main/resources/META-INF/services. In there you write:
com.package.of.UnpackCodeGenerator
From here, you need to link it using compileOnly and kapt. If you added it as a module to your project, you can do:
kapt project(":ClassUnpacker")
compileOnly project(":ClassUnpacker")
Alternative source setup:
Like I mentioned earlier, I bundled this into a jar for convenience. It's under the same license as SO uses (CC-BY-SA 3.0), and it contains the exact same code as in the answer (although compiled into a single project).
If you want to use this one, just add the Jitpack repo:
repositories {
// Other repos here
maven { url 'https://jitpack.io' }
}
And hook it up with:
kapt 'com.github.LunarWatcher:KClassUnpacker:v1.0.1'
compileOnly "com.github.LunarWatcher:KClassUnpacker:v1.0.1"
Note that the version here may not be up to date: the up to date list of versions is available here. The code in the post still aims to reflect the repo, but versions aren't really important enough to edit every time.
Usage
Regardless of which way you ended up using to get the annotations, the usage is relatively easy:
#AutoUnpack data class ExampleDataClass(val x: String, val y: Int, var m: Boolean)
fun main(a: Array<String>) {
val cls = ExampleDataClass("example", 42, false)
for(field in cls) {
println(field)
}
}
This prints:
example
42
false
Now you have a reflection-less way of iterating fields.
Note that local testing has been done partially with IntelliJ, but IntelliJ doesn't seem to like me - I've had various failed builds where gradlew clean && gradlew build from a command line oddly works fine. I'm not sure whether this is a local problem, or if this is a general problem, but you might have some issues like this if you build from IntelliJ.
Also, you might get errors if the build fails. The IntelliJ linter builds on top of the build directory for some sources, so if the build fails and the file with the extension function isn't generated, that'll cause it to appear as an error. Building usually fixes this when I tested (with both modules and from Jitpack).
You'll also likely have to enable the annotation processor setting if you use Android Studio or IntelliJ.
here is another idea, that i came up with, but am not satisfied with...but it has some pros and cons:
pros:
adding/removing fields to/from the data class causes compiler errors at field-iteration sites
no boiler-plate code needed
cons:
won't work if default values are defined for arguments
declaration:
data class Memento(
val testType: TestTypeData,
val notes: String,
val examinationTime: MillisSinceEpoch?,
val administeredBy: String,
val signature: SignatureViewHolder.SignatureData,
val signerName: String,
val signerRole: SignerRole
) : Serializable
iterating through all fields (can use this directly at call sites, or apply the Visitor pattern, and use this in the accept method to call all the visit methods):
val iterateThroughAllMyFields: Memento = someValue
Memento(
testType = iterateThroughAllMyFields.testType.also { testType ->
// do something with testType
},
notes = iterateThroughAllMyFields.notes.also { notes ->
// do something with notes
},
examinationTime = iterateThroughAllMyFields.examinationTime.also { examinationTime ->
// do something with examinationTime
},
administeredBy = iterateThroughAllMyFields.administeredBy.also { administeredBy ->
// do something with administeredBy
},
signature = iterateThroughAllMyFields.signature.also { signature ->
// do something with signature
},
signerName = iterateThroughAllMyFields.signerName.also { signerName ->
// do something with signerName
},
signerRole = iterateThroughAllMyFields.signerRole.also { signerRole ->
// do something with signerRole
}
)
I dont think minInvocation or maxInvocation is equivalent to times() in Mockito. Is there?
Please see this questions: Major difference between: Mockito and JMockIt
which has not been answered yet by anyone.
Edit
I found the answer myself: Adding it here for others who need this answered:
The solution is to use DynamicPartialMocking and pass the object to the constructor of the Expectations or NonStrictExpectations and not call any function on that object.
Then in the Verifications section, call any function on the object for which you want to measure the number of invocations and set times = the value you want
new NonStrictExpectations(Foo.class, Bar.class, zooObj)
{
{
// don't call zooObj.method1() here
// Otherwise it will get stubbed out
}
};
new Verifications()
{
{
zooObj.method1(); times = N;
}
};
I found the answer myself: Adding it here for others who need this answered:
The solution is to use DynamicPartialMocking and pass the object to the constructor of the Expectations or NonStrictExpectations and not call any function on that object.
Then in the Verifications section, call any function on the object for which you want to measure the number of invocations and set times = the value you want
new NonStrictExpectations(Foo.class, Bar.class, zooObj)
{
{
// don't call zooObj.method1() here
// Otherwise it will get stubbed out
}
};
new Verifications()
{
{
zooObj.method1(); times = N;
}
};