Test failing when executed in a different order - testing

When I execute this program:
use Test;
use NativeCall;
constant LIB = ('gsl', v23);
sub gsl_sf_airy_Ai(num64 $x, uint32 $mode --> num64) is native(LIB) is export { * }
sub Ai(Numeric $x, UInt $mode --> Num) is export { gsl_sf_airy_Ai($x.Num, $mode) }
ok Ai(0, 0) == 0.3550280538878172, 'Ai 1';
ok gsl_sf_airy_Ai(0e0, 0) == 0.3550280538878172, 'Ai 2';
the tests work fine, even if I swap the two "ok" tests this way:
ok gsl_sf_airy_Ai(0e0, 0) == 0.3550280538878172, 'Ai 2';
ok Ai(0, 0) == 0.3550280538878172, 'Ai 1';
If I move the declarations to a module:
unit module mymodule;
use NativeCall;
constant LIB = ('gsl', v23);
sub gsl_sf_airy_Ai(num64 $x, uint32 $mode --> num64) is native(LIB) is export { * }
sub Ai(Numeric $x, UInt $mode --> Num) is export { gsl_sf_airy_Ai($x.Num, $mode) }
and write a test program:
use Test;
use lib '.';
use mymodule;
ok Ai(0, 0) == 0.3550280538878172, 'Ai 1';
ok gsl_sf_airy_Ai(0e0, 0) == 0.3550280538878172, 'Ai 2';
again the two tests are executed without errors, but if I swap the last two lines:
ok gsl_sf_airy_Ai(0e0, 0) == 0.3550280538878172, 'Ai 2';
ok Ai(0, 0) == 0.3550280538878172, 'Ai 1';
I get this error:
Type check failed for return value; expected Num but got Whatever (*) and I don't understand why.
I even suspected a possible memory corruption, so I executed the test program using valgrind, but apparently there's nothing wrong in that department.
Any hint?

Please re-test on the recently released rakudo 2019.11. There's a high chance that I already fixed this when I refactored the NativeCall setup code.

Related

how to make a context aware code evaluator

I was looking at REPL-like evaluation of code from here and here, and tried to make a very small version for it, yet it fails:
use nqp;
class E {
has Mu $.compiler;
has $!save_ctx;
method evaluate(#fragments) {
for #fragments -> $code {
my $*MAIN_CTX;
my $*CTXSAVE := self;
$!compiler.eval($code,
outer_ctx => nqp::ctxcaller(nqp::ctx()));
if nqp::defined($*MAIN_CTX) {
$!save_ctx := $*MAIN_CTX;
}
}
}
method ctxsave(--> Nil) {
say "*in ctxsave*";
$*MAIN_CTX := nqp::ctxcaller(nqp::ctx());
$*CTXSAVE := 0;
}
}
my $e := E.new(compiler => nqp::getcomp("Raku"));
nqp::bindattr($e, E, '$!save_ctx', nqp::ctx());
$e.evaluate: ('say my #vals = 12, 3, 4;', 'say #vals.head');
I pieced together this from the above links without very much knowing what I'm doing :) When run, this happens:
*in ctxsave*
[12 3 4]
===SORRY!=== Error while compiling file.raku
Variable '#vals' is not declared. Did you mean '&val'?
file.raku:1
------> say ⏏#vals.head
with Rakudo v2022.04. First fragment was supposed to declare it (and prints it). Is it possible to do something like this, so it recognizes #vals as declared?
You can do it in pure Raku code, although depending on the not-exactly-official context parameter to EVAL.
# Let us use EVAL with user input
use MONKEY;
loop {
# The context starts out with a fresh environment
state $*REPL-CONTEXT = UNIT::;
# Get the next line of code to run.
my $next-code = prompt '> ';
# Evaluate it; note that exceptions with line numbers will be
# off by one, so may need fixups.
EVAL "\q'$*REPL-CONTEXT = ::;'\n$next-code", context => $*REPL-CONTEXT;
}
Trying it out:
$ raku simple-repl.raku
> my $x = 35;
> say $x;
35
> my $y = 7;
> say $x + $y;
42

Can Golang test scripts issue warnings rather than errors?

I have a set of tests that may not pass due to external 3rd party issues.
I don't want the test to fail when this condition occurs but would like to be made aware.
Issuing a t.Errorf() is not idea because it will stop all subsequent tests. Is there some kind of "Warning" I can trigger that the test script would post and then continue with the remainder of the tests?
The go test tool is like the compiler. To the compiler something either compiles or doesn't, there are no warnings. I think the closest thing you're going to get is to use t.Skip. It will stop execution of the current test but does not mark it as failed. You will not see anything in the output of go test however so you have to use go test -v.
Here's an example package that uses t.Skipf if the addExternal function fails.
package app
import "testing"
func add(a, b int) int {
return a + b
}
func addExternal(a, b int) int {
return 4
}
func divide(a, b int) int {
return a / b
}
func TestThing(t *testing.T) {
got := add(1, 2)
want := 3
if got != want {
t.Errorf("add(1, 2) = %d, want %d", got, want)
}
}
func TestExternalThing(t *testing.T) {
got := addExternal(3, 4)
want := 7
if got != want {
t.Skipf("addExternal(3, 4) = %d, want %d", got, want)
}
}
func TestAnotherThing(t *testing.T) {
got := divide(6, 3)
want := 2
if got != want {
t.Errorf("divide(6, 3) = %d, want %d", got, want)
}
}
And here's the output from running that. Note the return status is 0 and the package is considered to have passed
$ go test -v
=== RUN TestThing
--- PASS: TestThing (0.00s)
=== RUN TestExternalThing
--- SKIP: TestExternalThing (0.00s)
app_test.go:29: addExternal(3, 4) = 4, want 7
=== RUN TestAnotherThing
--- PASS: TestAnotherThing (0.00s)
PASS
ok github.com/jcbwlkr/app 0.006s
$ echo $?
0
Note though that if I change the t.Skipf to t.Errorf or t.Fatalf I get this output
$ go test -v
=== RUN TestThing
--- PASS: TestThing (0.00s)
=== RUN TestExternalThing
--- FAIL: TestExternalThing (0.00s)
app_test.go:29: addExternal(3, 4) = 4, want 7
=== RUN TestAnotherThing
--- PASS: TestAnotherThing (0.00s)
FAIL
exit status 1
FAIL github.com/jcbwlkr/app 0.005s
$ echo $?
1
The other tests in the package are still ran. If I was testing multiple packages such as with go test -v ./... I believe they would also still be ran.

how to deal with the "fmt" golang library package for CLI testing

Disclaimer: I wish you a merry XMas and I hope my question does not disturb you!
sample.go:
package main
import(
"fmt"
"os"
)
type sample struct {
value int64
}
func (s sample) useful() {
if s.value == 0 {
fmt.Println("Error: something is wrong!")
os.Exit(1)
} else {
fmt.Println("May the force be with you!")
}
}
func main() {
s := sample{42}
s.useful()
s.value = 0
s.useful()
}
// output:
// May the force be with you!
// Error: something is wrong!
// exit status 1
I did a lot of research on how to use interfaces in golang testing. But so far I was not able to wrap my head around this completely. At least I can not see how interfaces help me when I need to "mock" (apologies for using this word) golang std. library packages like "fmt".
I came up with two scenarios:
use os/exec to test the command line interface
wrap fmt package so I have control and am able to check the output strings
I do not like both scenarios:
I experience going through the actual command line a convoluted and not-performant (see below). Might have portability issues, too.
I believe this is the way to go but I fear that wrapping the fmt package might be a lot of work (at least wrapping the time package for testing turned out a non-trivial task (https://github.com/finklabs/ttime)).
Actual Question here: Is there another (better/simpler/idiomatic) way?
Note: I want to do this in pure golang, I am not interested in the next testing framework.
cli_test.go:
package main
import(
"os/exec"
"testing"
)
func TestCli(t *testing.T) {
out, err := exec.Command("go run sample.go").Output()
if err != nil {
t.Fatal(err)
}
if string(out) != "May the force be with you!\nError: this is broken and not useful!\nexit status 1" {
t.Fatal("There is something wrong with the CLI")
}
}
Chapter 11 of Kerningham's Book gives a good solution to this question.
The trick is to change the calls to fmt.Printline() to calls to
fmt.Fprint(out, ...) where out is initialised to os.Stdout
This can be overwritten in the test harness to new(bytes.Buffer) allowing the
test to capture the output.
See https://github.com/adonovan/gopl.io/blob/master/ch11/echo/echo.go and
https://github.com/adonovan/gopl.io/blob/master/ch11/echo/echo_test.go
edited by OP...
sample.go:
package main
import(
"fmt"
"os"
"io"
)
var out io.Writer = os.Stdout // modified during testing
var exit func(code int) = os.Exit
type sample struct {
value int64
}
func (s sample) useful() {
if s.value == 0 {
fmt.Fprint(out, "Error: something is wrong!\n")
exit(1)
} else {
fmt.Fprint(out, "May the force be with you!\n")
}
}
func main() {
s := sample{42}
s.useful()
s.value = 0
s.useful()
}
// output:
// May the force be with you!
// Error: this is broken and not useful!
// exit status 1
cli_test.go:
package main
import(
"bytes"
"testing"
)
func TestUsefulPositive(t *testing.T) {
bak := out
out = new(bytes.Buffer)
defer func() { out = bak }()
s := sample{42}
s.useful()
if out.(*bytes.Buffer).String() != "May the force be with you!\n" {
t.Fatal("There is something wrong with the CLI")
}
}
func TestUsefulNegative(t *testing.T) {
bak := out
out = new(bytes.Buffer)
defer func() { out = bak }()
code := 0
osexit := exit
exit = func(c int) { code = c }
defer func() { exit = osexit }()
s := sample{0}
s.useful()
if out.(*bytes.Buffer).String() != "Error: something is wrong!\n" {
t.Fatal("There is something wrong with the CLI")
}
if code != 1 {
t.Fatal("Wrong exit code!")
}
}
Am I missing something here or are you talking of testable examples?
Basically, it works like this: In a *_test.go file, you need to adhere to the convention Example[[T][_M]] where T is a placeholder for the type and M a placeholder for the method you want to display the testable example as example code in the Godoc. If the function is just called Example(), the code will be shown as a package example.
Below the last line of the code of your example, you can put a comment like this
// Output:
// Foo
Now go test will make sure that the testable example function either exactly puts out everything below // Output: (including whitespace) or it will make the test fail.
Here is an actual example for an testable example
func ExampleMongoStore_Get() {
sessionId := "ExampleGetSession"
data, err := ms.Get(sessionId)
if err == sessionmw.ErrSessionNotFound {
fmt.Printf("Session '%s' not found\n", sessionId)
data = make(map[string]interface{})
data["foo"] = "bar"
ms.Save(sessionId, data)
}
loaded, _ := ms.Get(sessionId)
fmt.Printf("Loaded value '%s' for key '%s' in session '%s'",
loaded["foo"],
"foo", sessionId)
// Output:
// Session 'ExampleGetSession' not found
// Loaded value 'bar' for key 'foo' in session 'ExampleGetSession'
}
Edit: Have a look at the output of above example at godoc.org

How to record the time for testing

I wrote two functions and now I would love to find out, which of them is faster. How can I find it out, which one is faster?
How can I find out by testing, which one is faster? Do go provide a timer for that?
Even better, Go provides a built in benchmark and testing functionality!
Create a file named something_test.go (must have the _test part).
func BenchmarkFunc1(b *testing.B) {
for i := 0; i < b.N; i++ {
_ = Func1()
}
}
func BenchmarkFunc2(b *testing.B) {
for i := 0; i < b.N; i++ {
_ = Func2()
}
}
Then simply run: go test -bench=. -benchmem
It will print how long each function took and memory usage / allocations.
Ref:
http://golang.org/pkg/testing/
http://dave.cheney.net/2013/06/30/how-to-write-benchmarks-in-go
You could write a benchmark for each function.
See for instance "How to write benchmarks in Go"
// from source_test.go
func BenchmarkFunction1(b *testing.B) {
// run the Function1 function b.N times
for n := 0; n < b.N; n++ {
Function1(10)
}
}
Repeat for Function2 and check the results with go test -bench=.

Unix C code cp in system()

I have a C code..
i which I have following code for UNIX:
l_iRet = system( "/bin/cp -p g_acOutputLogName g_acOutPutFilePath");
when I am running the binary generated..I am getting the following error:
cp: cannot access g_acOutputLogName
Can any one help me out?
You should generally prefer the exec family of functions over the system function. The system function passes the command to the shell which means that you need to worry about command injection and accidental parameter expansion. The way to call a subprocess using exec is as follows:
pid_t child;
child = fork();
if (child == -1) {
perror("Could not fork");
exit(EXIT_FAILURE);
} else if (child == 0) {
execlp("/bin/cp", g_acOutputLogName, g_acOutPutFilePath, NULL);
perror("Could not exec");
exit(EXIT_FAILURE);
} else {
int childstatus;
if (waitpid(child, &childstatus, 0) == -1) {
perror("Wait failed");
}
if (!(WIFEXITED(childstatus) && WEXITSTATUS(childstatus) == EXIT_SUCCESS)) {
printf("Copy failed\n");
}
}
Presumably g_acOutputLogName and g_acOutPutFilePath are char[] (or char*) variables in your program, rather than the actual paths involved.
You need to use the values stored therein, rather than the variable names, for example:
char command[512];
snprintf( command, sizeof command, "/bin/cp -p %s %s",
g_acOutputLogName, g_acOutPutFilePath );
l_iRet = system( command );