How to get Cargo Test to Pretty Print Failure Outputs - testing

I have a Rust test that is comparing two structs of a type that implements PartialEq and Debug.
According to the help page for the cargo test output, it should be possible to "pretty print" the left and right arguments to each failed assertion:
$ cargo test_foo --lib -- --help
warning: `google2005` (lib test) generated 4 warnings
Finished test [unoptimized + debuginfo] target(s) in 0.07s
Running unittests (target/debug/deps/foo)
Usage: test_foo [OPTIONS] [FILTERS...]
Options:
--include-ignored
Run ignored and not ignored tests
--ignored Run only ignored tests
...
***** snipped *****
...
--format pretty|terse|json|junit
Configure formatting of output:
pretty = Print verbose output;
terse = Display one character per test;
json = Output a json document;
junit = Output a JUnit document
...
***** snipped *****
I would expect this would print my structs with proper indentation, like when you use the {:#?} pretty print marker for Debug output. But it doesn't seem to have any effect:
$ cargo test test_foo --lib -- --format pretty
Finished test [unoptimized + debuginfo] target(s) in 0.07s
Running unittests (target/debug/deps/foo)
running 1 test
test parser::test::test_walk_scrapes_description ... FAILED
failures:
---- parser::test::test_walk_scrapes_description stdout ----
thread 'parser::test::test_walk_scrapes_description' panicked at 'assertion failed: `(left == right)`
left: `Cat { name: "Fluffy", age: 8, breed: "Siamese", color: "Brown", weight: 12, chip_number: 10001001 }`,
right: `Cat { name: "Fluffy", age: 8, breed: "Siamese", color: "Brown", weight: 13, chip_number: 10010001 }`', src/parser.rs:170:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
failures:
parser::test::test_walk_scrapes_description
test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 13 filtered out; finished in 0.00s
I thought it was possible to do achieve something like this:
left: `Cat {
name: "Fluffy",
age: 8,
breed: "Siamese",
color: "Brown",
weight: 12,
chip_number: 10001001
}`,
right: `Cat {
name: "Fluffy",
age: 8,
breed: "Siamese",
color: "Brown",
weight: 13,
chip_number: 10010001
}`',
I've tried both with rustc 1.59.0 (9d1b2106e 2022-02-23) and rustc 1.63.0-nightly (a6b8c6954 2022-06-03).

This is quite problematic since assert_eq!() itself prints the contents and cargo test gets already-formatted string.
It is also impossible to feature-gate the macro and under test pretty-print the variables, because std's code cannot be feature-gated (currently).
You can however write your macro to do that, or use a library like pretty-assertions.
Note, however, that there were RFCs to improve the display of the assert!() macros (it is not generally considered a breaking change), so it is possible that this will also be fixed in the future:
More readable assert_eq proposal
Tracking issue for RFC 2011: nicer assert messages

Related

How can I compile a script that has some dynamic code?

I want to compile this script:
Red [File: %b.red]
context [
a: 123
hidden: 999
set 'global-exports context [
export-a: a
]
]
probe global-exports
But I get an error when trying to compile it with $ ./red-13oct19-a4ee537c -r b.red:
*** Red Compiler Internal Error: Script Error : Invalid path value: global-exports
*** Where: register-object
*** Near: [objects/context/global-exports: make object! [
a: none
hidden: none
]]
In general, you can wrap dynamic code with do [...] function, which treats a block of code like data and uses an interpreter to launch it at runtime:
Red [File: %b.red]
context [
a: 123
hidden: 999
do [ ;-- the code inside is run by interpreter at runtime
set 'global-exports context [
export-a: a
]
]
]
probe get 'global-exports ;-- dynamic retrieval
You also have to retrieve values dynamically, unless you initialize it statically (e.g. global-exports: none) somewhere earlier.
In this particular case also exchanging context with make object! will be sufficient:
Red [File: %b.red]
context [
a: 123
hidden: 999
set 'global-exports make object! [
export-a: a
]
]
probe global-exports
Compiling dynamic code will be possible with JIT compiler, which is planned in future, but not before Red version 1.0.
(thanks #9214 and #hiiamboris for the ideas on red/help gitter chat)

Can Golang test scripts issue warnings rather than errors?

I have a set of tests that may not pass due to external 3rd party issues.
I don't want the test to fail when this condition occurs but would like to be made aware.
Issuing a t.Errorf() is not idea because it will stop all subsequent tests. Is there some kind of "Warning" I can trigger that the test script would post and then continue with the remainder of the tests?
The go test tool is like the compiler. To the compiler something either compiles or doesn't, there are no warnings. I think the closest thing you're going to get is to use t.Skip. It will stop execution of the current test but does not mark it as failed. You will not see anything in the output of go test however so you have to use go test -v.
Here's an example package that uses t.Skipf if the addExternal function fails.
package app
import "testing"
func add(a, b int) int {
return a + b
}
func addExternal(a, b int) int {
return 4
}
func divide(a, b int) int {
return a / b
}
func TestThing(t *testing.T) {
got := add(1, 2)
want := 3
if got != want {
t.Errorf("add(1, 2) = %d, want %d", got, want)
}
}
func TestExternalThing(t *testing.T) {
got := addExternal(3, 4)
want := 7
if got != want {
t.Skipf("addExternal(3, 4) = %d, want %d", got, want)
}
}
func TestAnotherThing(t *testing.T) {
got := divide(6, 3)
want := 2
if got != want {
t.Errorf("divide(6, 3) = %d, want %d", got, want)
}
}
And here's the output from running that. Note the return status is 0 and the package is considered to have passed
$ go test -v
=== RUN TestThing
--- PASS: TestThing (0.00s)
=== RUN TestExternalThing
--- SKIP: TestExternalThing (0.00s)
app_test.go:29: addExternal(3, 4) = 4, want 7
=== RUN TestAnotherThing
--- PASS: TestAnotherThing (0.00s)
PASS
ok github.com/jcbwlkr/app 0.006s
$ echo $?
0
Note though that if I change the t.Skipf to t.Errorf or t.Fatalf I get this output
$ go test -v
=== RUN TestThing
--- PASS: TestThing (0.00s)
=== RUN TestExternalThing
--- FAIL: TestExternalThing (0.00s)
app_test.go:29: addExternal(3, 4) = 4, want 7
=== RUN TestAnotherThing
--- PASS: TestAnotherThing (0.00s)
FAIL
exit status 1
FAIL github.com/jcbwlkr/app 0.005s
$ echo $?
1
The other tests in the package are still ran. If I was testing multiple packages such as with go test -v ./... I believe they would also still be ran.

Tests panic when HTTP server starts after old one is closed

I have some tests written, each function starts iron's HTTP server and closes it after test is done:
extern crate iron;
use iron::prelude::*;
fn hello_world(_: &mut Request) -> IronResult<Response> {
Ok(Response::with((iron::status::Ok, "Hello World")))
}
#[test]
fn test1() {
let mut server = Iron::new(hello_world).http("localhost:3000").unwrap();
server.close().unwrap();
}
#[test]
fn test2() {
let mut server = Iron::new(hello_world).http("localhost:3000").unwrap();
server.close().unwrap();
}
Doing cargo test I'm having:
Running target/debug/lib-f236975fe924352b
running 2 tests
test test1 ... ok
test test2 ... FAILED
failures:
---- test2 stdout ----
thread 'test2' panicked at 'called `Result::unwrap()` on an `Err` value: Io(Error { repr: Os { code: 98, message: "Address already in use" } })', ../src/libcore/result.rs:736
failures:
test2
test result: FAILED. 1 passed; 1 failed; 0 ignored; 0 measured
It seems like the port is still in use by the start of the second test.
Rust's test runner is parallel by default, so yes, I would expect that this would happen. I would pick a different port number for each test, at least to start, since it's so easy.

Serialization misbehave when closing Unity

I'm using ISerializationCallbackReceiver to serialize my custom interrelated data following tutorial http://blogs.unity3d.com/2014/06/24/serialization-in-unity/ regarding serialization of graph nodes adjusting it to my needs.
My serialization is working fine when I'm hot swapping the code when I'm developing scripts. But once I save the scene, close Unity and reload the project again, my data is deserializing from some bogus.
[01] [17:23:43] Quest[The Sculpture and The Box|SLEEP]: Serializing QuestStepData[fqcn=experiment.questsjg.steps.GoToLocation|name=Sculpture|description=F|closedByIndex=-1|go1Present=True|go1=sculptures_01_02 (UnityEngine.GameObject)|go2Present=True|go2=[1] Go To Location Point (Sculpture) (UnityEngine.GameObject)|f1=8]
[02] [17:23:43] Quest[The Sculpture and The Box|SLEEP]: Serializing QuestStepData[fqcn=experiment.questsjg.steps.GoToLocation|name=box|description=S|closedByIndex=-1|go1Present=True|go1=box1 (UnityEngine.GameObject)|go2Present=True|go2=[2] Go To Location Point (box) (UnityEngine.GameObject)|f1=4]
[03] [17:24:09] Quest[The Sculpture and The Box|SLEEP]: Deserializing QuestStepData[fqcn=experiment.questsjg.steps.GoToLocation|name=Sculpture|description=|closedByIndex=-1|go1Present=True|go1= (UnityEngine.GameObject)|go2Present=True|go2= (UnityEngine.GameObject)|f1=8]
[04] [17:24:09] Quest[The Sculpture and The Box|SLEEP]: Deserializing QuestStepData[fqcn=experiment.questsjg.steps.GoToLocation|name=|description=|closedByIndex=-1|go1Present=False|go2Present=False|f1=10]
Here is a log of what I'm de/serializing. Unity got restarted between 17:23:45-17:24:00 and as you can see it did not deserialize (Rows 03 and 04) from the same data I serialized (Rows 01 and 02).
When I save the scene, I typically get Rows 01 and 02 logged.
But when these "rows" gets deserialized when I open Unity, I get Rows 03 and 04 and as you can see Row 03 "description" differs from Row 01 and Row 04 is missing everything except 'fqcn' field when compared with Row 02.
Logs are as precise as they can get (I'm writing them to the file), so these serialization lines 01 and 02 should be the last thing Unity is doing with my data when closing...
Now I'm buffled...
Serialization is working flawlessly when hot swapping the code
Not working when restarting Unity ... (the worst thing is that it "drops" only some infos)
Any hints what I might be doing wrong?
(The same happens for 3+ items... only 1st item is somehow correct)
P.S.: It does not matter whether you have Asset serialization set to "Mixed" or "Force Text".
If I have "MIXED" then data within the scene file looks like this:
The Sculpture and The Box ! NyŻ
stepsData.Array.data[0].fqcn&
experiment.questsjg.steps.GoToLocation ! NyŻ %
stepsData.Array.data[0].closedByIndex -1 ! NyŻ
stepsData.Array.data[0].f1 8 ! NyŻ
stepsData.Array.data[0].name Sculpture ! NyŻ
stepsData.Array.data[0].go1 ^ ! NyŻ "
stepsData.Array.data[0].go1Present 1 ! NyŻ
stepsData.Array.data[0].go2 *¨ ! NyŻ "
stepsData.Array.data[0].go2Present 1 ! NyŻ
stepsData.Array.data[1].fqcn&
experiment.questsjg.steps.GoToLocation ! NyŻ %
stepsData.Array.data[1].closedByIndex -1 ! NyŻ
stepsData.Array.data[1].f1 10 ! NyŻ
As you can see, serialized data are not there, just compare stepsData.Array.data[0] and stepsData.Array.data[1]
If I have "Force Text" then data within the scene file looks like this (it's actually even worse):
- target: {fileID: 11499854, guid: bd017908bdedeb24d9559a160e99a0c1, type: 2}
propertyPath: stepsData.Array.data[0].fqcn
value: quests.steps.GoToLocation
objectReference: {fileID: 0}
- target: {fileID: 11499854, guid: bd017908bdedeb24d9559a160e99a0c1, type: 2}
propertyPath: stepsData.Array.data[0].closedByIndex
value: -1
objectReference: {fileID: 0}
- target: {fileID: 11499854, guid: bd017908bdedeb24d9559a160e99a0c1, type: 2}
propertyPath: stepsData.Array.data[0].f1
value: 10
objectReference: {fileID: 0}
- target: {fileID: 11499854, guid: bd017908bdedeb24d9559a160e99a0c1, type: 2}
propertyPath: stepsData.Array.data[1].fqcn
value: quests.steps.GoToLocation
objectReference: {fileID: 0}
- target: {fileID: 11499854, guid: bd017908bdedeb24d9559a160e99a0c1, type: 2}
propertyPath: stepsData.Array.data[1].closedByIndex
value: -1
objectReference: {fileID: 0}
- target: {fileID: 11499854, guid: bd017908bdedeb24d9559a160e99a0c1, type: 2}
propertyPath: stepsData.Array.data[1].f1
value: 10
objectReference: {fileID: 0}
- target: {fileID: 107138, guid: bd017908bdedeb24d9559a160e99a0c1, type: 2}
propertyPath: m_Name
value: First Quest
objectReference: {fileID: 0}
Hmmmm, what's happening in there?
Aaaa!
The trick is to call EditorUtility.SetDirty(gameObject) EVERY TIME ANYTHING CHANGES as this page states: http://docs.unity3d.com/ScriptReference/EditorUtility.SetDirty.html
Unity internally uses the dirty flag to find out when assets have changed and need to be saved to disk.
E.g. if you modify a prefab's MonoBehaviour or ScriptableObject variables, you must tell Unity that the value has changed. Unity builtin components internally call SetDirty whenever a property changes. MonoBehaviour or ScriptableObject DON'T DO this AUTOMATICALLY so if you want your value to be saved you need to call SetDirty.
Therefore, whenever you're creating your CustomEditor following http://docs.unity3d.com/Manual/editor-CustomEditors.html make sure to use the trick:
GUI.changed = false;
// do you're edit code here
if (GUI.changed) SetDirty(yourGameObjectThatHasJustChanged);
Hope this saves someone's time :-)
Cheers!

How do I force module reload?

The title says it all. When developing a module, how do I force reload, to test new code? I can switch from module to script and introduce binding problems and namespace conflicts, or I can change the version every time I fix a typo. Both are bad options.
What I'm looking for is something like import/force %my-module.reb to reload the module in running session (now I have to restart R3 which is not very fast pattern to use).
I don't know how you're importing your modules, but if you assign the return value of the import function to a variable re-executing the import loads the new code.
For example I have the file mod_a.reb:
REBOL []
forever [
b: import %./mod_b.reb
b/hello
wait 10
]
and the file mod_b.reb:
REBOL []
hello: function [] [
print "Hello"
]
If you run r3 ./mod_a.reb you see the "Hello" string printed every 10 seconds. If you modify the string in mod_b.reb while mod_a.reb is running you see a different string printed.
Currently module does not overwrite existing values (for security reasons).
It works like:
>> m: module [name: test][ export my-func: does[print "hello"]]
>> m/my-func
hello
>> my-func ;<- will error because module was not imported yet
** Script error: my-func has no value
>> import m
>> my-func ;<- now `my-func` is in user's but also `lib` contexts
hello
>> lib/my-func
hello
;-- one must unset it in both contexts
;-- and also remove module from list of existing modules
>> remove/part find system/modules 'test 3 unset 'my-func unset in lib 'my-func
>> import module [name: test][ export my-func: does[print "hello again"]]
>> my-func
hello again
One can simplify it a little bit using private flag and version:
>> import module [name: test version: 1.0.0 options: [private]][export my-func: does[print "hello"]]
>> lib/my-func ;<- will error, because module is private
** Script error: cannot access my-func in path lib/my-func
>> my-func ;<- but it is still in user's context
hello
>> unset 'my-func
>> import module [name: test version: 1.0.1 options: [private]][export my-func: does[print "hello again"]]
>> my-func
hello again
It's also possible to write a function for module unloading (although it may not work in all cases)
unload-module: function[module [word!]][
m: system/modules/:module
unless m [ exit ]
e: select spec-of m 'exports
forall e [
print ["Unsetting:" e/1]
try [unset in system/contexts/user e/1]
try [unset in system/contexts/lib e/1]
]
remove/part find system/modules module 3
m
]
; than:
>> import module [name: test][export my-func: does[print "hello"]]
>> my-func
hello
>> unload-module 'test
Unsetting: my-func
>> import module [name: test][export my-func: does[print "hello again"]]
>> my-func
hello again