Static testing for Scala - testing

There are some nice libraries for testing in Scala (Specs, ScalaTest, ScalaCheck). However, with Scala's powerful type system, important parts of an API being developed in Scala are expressed statically, usually in the form of some undesirable or disallowed behavior being prevented by the compiler.
So, what is the best way to test whether something is prevented by the compiler when designing an library or other API? It is unsatisfying to comment out code that is supposed to be uncompilable and then uncomment it to verify.
A contrived example testing List:
val list: List[Int] = List(1, 2, 3)
// should not compile
// list.add("Chicka-Chicka-Boom-Boom")
Does one of the existing testing libraries handle cases like this? Is there an approach that people use that works?
The approach I was considering was to embed code in a triple-quote string or an xml element and call the compiler in my test. Calling code looking something like this:
should {
notCompile(<code>
val list: List[Int] = List(1, 2, 3)
list.add("Chicka-Chicka-Boom-Boom")
</code>)
}
Or, something along the lines of an expect-type script called on the interpreter.

I have created some specs executing some code snippets and checking the results of the interpreter.
You can have a look at the Snippets trait. The idea is to store in some org.specs.util.Property[Snippet] the code to execute:
val it: Property[Snippet] = Property(Snippet(""))
"import scala.collection.List" prelude it // will be prepended to any code in the it snippet
"val list: List[Int] = List(1, 2, 3)" snip it // snip some code (keeping the prelude)
"list.add("Chicka-Chicka-Boom-Boom")" add it // add some code to the previously snipped code. A new snip would remove the previous code (except the prelude)
execute(it) must include("error: value add is not a member of List[Int]") // check the interpreter output
The main drawback I found with this approach was the slowness of the interpreter. I don't know yet how this could be sped up.
Eric.

Related

AST and Language Service Plugin

I would like to create a vscode extension which can help with unit testing and "live" debugging.
Lets say you have the following code:
/*1*/ export class Test {
/*2*/ public add(a: any, b: any): any {
/*3*/ return a + b;
/*4*/ }
/*5*/ }
Live debugging:
New option in the context menu to test the add function (e.g. something similar to current Peek Definition) where you can provide the function input and check the output in a new smaller editor.
Unit test generation:
After you wrote an input and checked the output you can simply generate a new unit test from it.
Now I'm only at the beginning of this project but it seems like getting type information is much more complicated than I expected.
First I would like to get some simple type information (based on cursor position) e.g.:
Line 1 is a class
Line 2-4 is a function
Line 5 is a class
After hours of Google search I think what I have to do is to implement my own TypeScript Language Service Plugin and use it somehow in my extension.
My questions:
Is there any extension similar to my idea which I could use?
Is there an easier way to get the type information I need?
Am I on a right track at all?
Do you have some material about how to use this language service plugin from extension?
P.S. I know that it won't be easy (e.g. several states with different visibility), but good practice for a home project.

<!WHATVER!> syntax in Kotlin? (Angle brackets wrapping exclamation points)

I saw this syntax I'm not familiar with in the Kotlin compiler test suite.
// !DIAGNOSTICS: +UNUSED_LAMBDA_EXPRESSION, +UNUSED_VARIABLE
fun unusedLiteral(){
<!UNUSED_LAMBDA_EXPRESSION!>{ ->
val <!UNUSED_VARIABLE!>i<!> = 1
}<!>
}
What does <!UNUSED_LAMBDA_EXPRESSION!>...<!> mean?
Found in unusedLiteral.kt
The term UNUSED_LAMBDA_EXPRESSION is declared in Errors.kt to be:
DiagnosticFactory0<KtLambdaExpression> UNUSED_LAMBDA_EXPRESSION = DiagnosticFactory0.create(WARNING);
This syntax is not valid Kotlin. It is only used in the test data files of Kotlin's test pipeline. That is, only the test runners recognises this syntax, not the Kotlin compiler. Specifically, the <!DIAGNOSTIC_NAME!>foo<!> syntax denotes a handler. Handlers do checks on things, or output information to a file. In this case, this syntax checks that there is indeed the specified diagnostic being emitted at that point in the file.
Also note that the // !DIAGNOSTICS comment at the top is not just a comment. It denotes a directive. Directives are like the options for running the test.
I highly recommend you read compiler/testData/diagnostics/ReadMe.md, which explains how diagnostic tests work specifically, and if you're really interested in this stuff, check out compiler/test-infrastructure/ReadMe.md too, which tells you all about how the whole test pipeline works in general.

Is there a Kotlin "eval"

I have a bunch of strings like these in my already existing and quit big template file:
Hello $World I say hello to $User, too
I was thinking I could somehow let kotlin parse/search&replace this file as a kotlin string, and I'd just have to set the variables World and User to get an evaluated string... How is this possible?
This is not a kotlin source file, but a file that's beeing read by my kotlin program.
Why I want to do this? I used bash's envsubst before, and had to move away from this, since things were getting too complicated. But now I have no easy way to replace strings in a file anymore...
Thanks
What you want is called a template engine.
You don't really need an eval (which would allow running a full Kotlin application from a String) in this case.
For example, FreeMarker templates use a syntax that's similar to Kotlin template Strings, so a template may look like this:
<h1>Welcome ${user}!</h1>
Then, from Kotlin, you can evaluate the template with a Map holding the template bindings (variables the template can use) like this:
val user = "joe"
val bindings = mapOf("user" to user)
val cfg = new Configuration(Configuration.VERSION_2_3_29)
cfg.directoryForTemplateLoading = File("/where/you/store/templates")
val template = cfg.getTemplate("test.ftlh")
// write the resolved template to stdout
val out = OutputStreamWriter(System.out)
template.process(bindings, out)
See a Java example here: https://freemarker.apache.org/docs/pgui_quickstart_all.html
There are many other template engines, and the KTor site lists a few:
https://ktor.io/docs/working-with-views.html
If you're using KTor, BTW, it makes it much easier to use template engines... your framework may even have similar.
Warning: Please see my other answer first: you probably don't want to run a full Kotlin script just to parse some text file with variables... use a template engine for that.
It's possible to execute Kotlin scripts, i.e. eval Kotlin code, but it requires that you basically ship the Kotlin compiler with your application.
This GitHub project, KtsRunner, for example, does that, so you can do this:
val scriptContent = "5 + 10"
val fromScript: Int = KtsObjectLoader().load<Int>(scriptContent))
println(fromScript)
// >> 15
It requires some hacking though, is very slow and it uses a bunch of Kotlin "internal" APIs to work.
See the full list of libraries this project uses here:
https://github.com/s1monw1/KtsRunner/blob/master/lib/build.gradle.kts#L21
A "proper" KEEP proposal (discussion here) exists to add first-class support for this in Kotlin, but it's not finalized yet.
Currently, it looks something like this:
fun evalFile(scriptFile: File): ResultWithDiagnostics<EvaluationResult> {
val compilationConfiguration = createJvmCompilationConfigurationFromTemplate<MainKtsScript>()
val evaluationConfiguration = createJvmEvaluationConfigurationFromTemplate<MainKtsScript>()
return BasicJvmScriptingHost().eval(scriptFile.toScriptSource(), compilationConfiguration, evaluationConfiguration)
}
But it'll probably change before being released.

"python function decorator" for objective-c to change a method's behavior

I want to modify the behavior of some function without being the author of that function. What I control is that I can ask the author to follow some pattern, e.g. use a base class, use a certain decorator, property etc.
If in python, I would use a decorator to change the behavior of a method.
As an example, My goal: Improve code coverage by automatically testing over multiple input data.
Pseudo code:
#implementation SomeTestSuiteClass
// If in python, I would add a decorator here to change the behavior of the following method
-(void)testSample1 {
input = SpecialProvider();
output = FeatureToTest(input);
SpecialAssert(output);
}
#end
What I want: During test, the testSample1 method will be called multiple times. Each time, the SpecialProvider will emit a different input data. Same for the SpecialAssert, which can verify the output corresponding to the given input.
SpecialProvider and SpecialAssert will be API under my control/ownership (i.e. I write them).
The SomeTestSuiteClass together with the testSample1 will be written by the user (i.e. test writer).
Is there a way for Objective-C to achieve "what I want" above?
You could mock objects and/or its methods using objective-c runtime or some third party frameworks. I discourage it though. That is a sign of poor architecture choices in the 1st place. The main problem in your approach are hidden dependencies in your code directly referencing
SpecialProvider & SpecialAssert symbols directly.
A much better way to this would be like this:
-(void)testSample1:(SpecialProvider*)provider assert:(BOOL (^)(parameterTypes))assertBlock {
input = provider;
output = FeatureToTest(input);
if (assertBlock != nil) {
assertBlock(output);
}
}
Since Objective-c does not support default argument values like Swift does you could emulate it with:
-(void)testSample1 {
[self testSample1:DefaultSpecialProvider() assert:DefaultAssert()];
}
not to call the explicit -(void)testSample1:(SpecialProvider*)provider assert:(BOOL (^)(parameterTypes))assertBlock all the time, however in tests you would always use the explicit 2 argument variant to allow substituting the implementation(s) not being under test.
Further improvement idea:
Put the SpecialProvider and SpecialAssert behind protocols(i.e. equivalent of interfaces in other programming languages) so you can easily exchange the implementation.

Writing a TemplateLanguage/VewEngine

Aside from getting any real work done, I have an itch. My itch is to write a view engine that closely mimics a template system from another language (Template Toolkit/Perl). This is one of those if I had time/do it to learn something new kind of projects.
I've spent time looking at CoCo/R and ANTLR, and honestly, it makes my brain hurt, but some of CoCo/R is sinking in. Unfortunately, most of the examples are about creating a compiler that reads source code, but none seem to cover how to create a processor for templates.
Yes, those are the same thing, but I can't wrap my head around how to define the language for templates where most of the source is the html, rather than actual code being parsed and run.
Are there any good beginner resources out there for this kind of thing? I've taken a ganer at Spark, which didn't appear to have the grammar in the repo.
Maybe that is overkill, and one could just test-replace template syntax with c# in the file and compile it. http://msdn.microsoft.com/en-us/magazine/cc136756.aspx#S2
If you were in my shoes and weren't a language creating expert, where would you start?
The Spark grammar is implemented with a kind-of-fluent domain specific language.
It's declared in a few layers. The rules which recognize the html syntax are declared in MarkupGrammar.cs - those are based on grammar rules copied directly from the xml spec.
The markup rules refer to a limited subset of csharp syntax rules declared in CodeGrammar.cs - those are a subset because Spark only needs to recognize enough csharp to adjust single-quotes around strings to double-quotes, match curley braces, etc.
The individual rules themselves are of type ParseAction<TValue> delegate which accept a Position and return a ParseResult. The ParseResult is a simple class which contains the TValue data item parsed by the action and a new Position instance which has been advanced past the content which produced the TValue.
That isn't very useful on it's own until you introduce a small number of operators, as described in Parsing expression grammar, which can combine single parse actions to build very detailed and robust expressions about the shape of different syntax constructs.
The technique of using a delegate as a parse action came from a Luke H's blog post Monadic Parser Combinators using C# 3.0. I also wrote a post about Creating a Domain Specific Language for Parsing.
It's also entirely possible, if you like, to reference the Spark.dll assembly and inherit a class from the base CharGrammar to create an entirely new grammar for a particular syntax. It's probably the quickest way to start experimenting with this technique, and an example of that can be found in CharGrammarTester.cs.
Step 1. Use regular expressions (regexp substitution) to split your input template string to a token list, for example, split
hel<b>lo[if foo]bar is [bar].[else]baz[end]world</b>!
to
write('hel<b>lo')
if('foo')
write('bar is')
substitute('bar')
write('.')
else()
write('baz')
end()
write('world</b>!')
Step 2. Convert your token list to a syntax tree:
* Sequence
** Write
*** ('hel<b>lo')
** If
*** ('foo')
*** Sequence
**** Write
***** ('bar is')
**** Substitute
***** ('bar')
**** Write
***** ('.')
*** Write
**** ('baz')
** Write
*** ('world</b>!')
class Instruction {
}
class Write : Instruction {
string text;
}
class Substitute : Instruction {
string varname;
}
class Sequence : Instruction {
Instruction[] items;
}
class If : Instruction {
string condition;
Instruction then;
Instruction else;
}
Step 3. Write a recursive function (called the interpreter), which can walk your tree and execute the instructions there.
Another, alternative approach (instead of steps 1--3) if your language supports eval() (such as Perl, Python, Ruby): use a regexp substitution to convert the template to an eval()-able string in the host language, and run eval() to instantiate the template.
There are sooo many thing to do. But it does work for on simple GET statement plus a test. That's a start.
http://github.com/claco/tt.net/
In the end, I already had too much time in ANTLR to give loudejs' method a go. I wanted to spend a little more time on the whole process rather than the parser/lexer. Maybe in version 2 I can have a go at the Spark way when my brain understands things a little more.
Vici Parser (formerly known as LazyParser.NET) is an open-source tokenizer/template parser/expression parser which can help you get started.
If it's not what you're looking for, then you may get some ideas by looking at the source code.