I am writing QUnit tests for my JavaScript web app. After reading the QUnit API documentation I was unable to find any standards for labeling modules, tests, and asserts.
The API descriptions and examples are pretty rudimentary:
Module name: Label for this group of tests. Example: module("group a");
Test title: Title of unit being tested. Example: test("hello test", function(){});
Assert message: A short description of the assertion. Example: ok(true, "true succeeds");
I found articles out there regarding Unit Tests at large: What are some popular naming conventions for Unit Tests? and Unit test naming best practices.
However, I was hoping there was an established/generally accepted standard that was specific to QUnit's Module->Test->Assert pattern. Are there any?
No matter the test framework you use, and no matter if you are writing client side or server side unit tests, the main concern on adopt some "name convention" is to make sure that the test name clearly states to everyone what is the system and the behavior under testing.
Let's suppose we want to test this code:
var MyClass = (function () {
function MyClass() {
this.defaultMessage = "Hello person without a name";
}
MyClass.prototype.getHelloMessage = function (firstName, lastName) {
if (!firstName && !lastName) {
return this.defaultMessage;
}
var message = "Hello";
if (lastName) {
message += " " + lastName;
}
if (firstName) {
if (lastName) {
message += ",";
}
message += " " + firstName;
}
return message;
};
return MyClass;
})();
The above code just format a simple message with the given input parameters, if any, and return a default message if no parameters specified.
The module name for me would be the class name and a separator to make the list of test names more readable:
QUnit.module("MyClass tests - ");
One could simple name the test:
test("getHelloMessage test", function () {}
and assert all expectations in one text.
I prefer a more verbose approach, that would be clear to understand the purpose of each test:
test("getHelloMessage without input parameters should return default message", function () {});
test("getHelloMessage with only lastName should not display a comma", function () {});
test("getHelloMessage with only firstName should not display a comma", function () {});
This way each test has a clear purpose, they will be kept as small as needed, and when one test fail it is easy to understand what part of your code has been affected by the latest changes.
The good standard is the one that all members of your team agree that will help everyone to find and solve bugs, and also to help them write better unit tests.
Related
My CasperJS asserts seem to be overly strict. I have a function where I am trying to test the names of client logo images from an array, using Casperjs. However I do not seem to be able to use a variable from a forLoop in casperJS.
I understand there are probably hoisting issues that I am not accounting for, but this does not seem to be the primary problem. I have tried several things to resolve hoisting issues, such as immediately invoked functions, try catch blocks, and using ES6 term "Let" in my loop. None seem to work. Then I notice if I simply hard-code the string my variable should represent, and stick a console.log into my assert of a PASSING test, right before the return, the passing test fails.
Here is my failing code
var clients = 'https://www.google.com/';
var logoArray = ["images/logos/AC.png", "images/logos/Affiny.png", "images/logos/ffintus.png", "images/logos/agileAsset.png"]
function checkClientsArrayTest() {
casper.test.begin('The layout is as expected', 10, function suite(test) {
casper.start(clients, function () {
casper.then(function () {
for (var i = 0; i < logoArray.length; i++) {
try { throw i }
catch (ii) {
console.log(ii);
console.log(i);
test.assertEvalEquals(function () {
return document.querySelectorAll('div.client_logo a img')[ii].getAttribute('src')
.match(logoArray[ii]).toString();
}, logoArray[ii], 'Test searches for Client Logos in DOM.');
}
}
});
}).run(function () {
test.done();
});
});
}
If I change logoArray[ii] to a hardcoded string from the first index of the array, it passes. If I consolelog logoArray[ii], it seems to be what I expect. But if I pass a variable to the assert, or even stick a console.log inside of it, the test fails with the following
Running check for the layout of URL: https://www.google.com
0
0
FAIL Test searches for Client Logos in DOM.
type: assertEvalEquals
file: headlessTester.js
subject: null
fn: undefined
params: undefined
expected: "images/logos/AC.png"
Is this an issue of me getting hoisting wrong (shouldn't fail by sticking in a logger if this is the case afaik), or is this due to strictly structured asserts in CasperJS?
All;
I am just starting learning Jasmine( version 2.0.3 ), when I got to Spies section, the first example confused me:
describe("A spy", function() {
var foo, bar = null;
beforeEach(function() {
foo = {
setBar: function(value) {
bar = value;
}
};
spyOn(foo, 'setBar');
foo.setBar(123);
foo.setBar(456, 'another param');
});
it("tracks that the spy was called", function() {
expect(foo.setBar).toHaveBeenCalled();
});
it("tracks all the arguments of its calls", function() {
expect(foo.setBar).toHaveBeenCalledWith(123);
expect(foo.setBar).toHaveBeenCalledWith(456, 'another param');
});
it("stops all execution on a function", function() {
expect(bar).toBeNull();
});
});
I wonder if anyone could explain why the setBar function does not affect the bar defined inside describe block? How Jasmine spies deal with this?
Thanks
Because you are not actually executing the methods.
If you want this test to fail:
it("stops all execution on a function", function() {
expect(bar).toBeNull();
});
After these calls:
foo.setBar(123);
foo.setBar(456, 'another param');
Then you should call and.callThrough for your spy.
spyOn(foo, 'setBar').and.callThrough();
From the documentation
Spies: and.callThrough
By chaining the spy with and.callThrough, the spy will still track all
calls to it but in addition it will delegate to the actual
implementation.
With regard to your question, 'how jasmine deals with this?'
From here you can read a basic explanation:
Mocks work by implementing the proxy pattern. When you create a mock
object, it creates a proxy object that takes the place of the real
object. We can then define what methods are called and their returned
values from within our test method. Mocks can then be utilized to
retrieve run-time statistics on the spied function such as:
How many times the spied function was called.
What was the value that the function returned to the caller.
How many parameters the function was called with.
If you want all of the implementation details, you can check the Jasmine source code which is Open Source :)
In this source file CallTracker you can see how the gather data about the method calls.
A little more about the proxy pattern.
I want to use something like Cucumber JVM to drive performance tests written for Gatling.
Ideally the Cucumber features would somehow build a scenario dynamically - probably reusing predefined chain objects similar to the method described in the "Advanced Tutorial", e.g.
val scn = scenario("Scenario Name").exec(Search.search("foo"), Browse.browse, Edit.edit("foo", "bar")
I've looked at how the Maven plugin executes the scripts, and I've also seen mention of using an App trait but I can't find any documentation for the later and it strikes me that somebody else will have wanted to do this before...
Can anybody point (a Gatling noob) in the direction of some documentation or example code of how to achieve this?
EDIT 20150515
So to explain a little more:
I have created a trait which is intended to build up a sequence of, I think, ChainBuilders that are triggered by Cucumber steps:
trait GatlingDsl extends ScalaDsl with EN {
private val gatlingActions = new ArrayBuffer[GatlingBehaviour]
def withGatling(action: GatlingBehaviour): Unit = {
gatlingActions += action
}
}
A GatlingBehaviour would look something like:
object Google {
class Home extends GatlingBehaviour {
def execute: ChainBuilder =
exec(http("Google Home")
.get("/")
)
}
class Search extends GatlingBehaviour {...}
class FindResult extends GatlingBehaviour {...}
}
And inside the StepDef class:
class GoogleStepDefinitions extends GatlingDsl {
Given( """^the Google search page is displayed$""") { () =>
println("Loading www.google.com")
withGatling(Home())
}
When( """^I search for the term "(.*)"$""") { (searchTerm: String) =>
println("Searching for '" + searchTerm + "'...")
withGatling(Search(searchTerm))
}
Then( """^"(.*)" appears in the search results$""") { (expectedResult: String) =>
println("Found " + expectedResult)
withGatling(FindResult(expectedResult))
}
}
The idea being that I can then execute the whole sequence of actions via something like:
val scn = Scenario(cucumberScenario).exec(gatlingActions)
setup(scn.inject(atOnceUsers(1)).protocols(httpConf))
and then check the reports or catch an exception if the test fails, e.g. response time too long.
It seems that no matter how I use the 'exec' method it tries to instantly execute it there and then, not waiting for the scenario.
Also I don't know if this is the best approach to take, we'd like to build some reusable blocks for our Gatling tests that can be constructed via Cucumber's Given/When/Then style. Is there a better or already existing approach?
Sadly, it's not currently feasible to have Gatling directly start a Simulation instance.
Not that's it's not technically feasible, but you're just the first person to try to do this.
Currently, Gatling is usually in charge of compiling and can only be passed the name of the class to load, not an instance itself.
You can maybe start by forking io.gatling.app.Gatling and io.gatling.core.runner.Runner, and then provide a PR to support this new behavior. The former is the main entry point, and the latter the one can instanciate and run the simulation.
I recently ran into a similar situation, and did not want to fork gatling. And while this solved my immediate problem, it only partially solves what you are trying to do, but hopefully someone else will find this useful.
There is an alternative. Gatling is written in Java and Scala so you can call Gatling.main directly and pass it the arguments you need to run the Gatling Simulation you want. The problem is, the main explicitly calls System.exit so you have to also use a custom security manager to prevent it from actually exiting.
You need to know two things:
the class (with the full package) of the Simulation you want to run
example: com.package.your.Simulation1
the path where the binaries are compiled.
The code to run a Simulation:
protected void fire(String gatlingGun, String binaries){
SecurityManager sm = System.getSecurityManager();
System.setSecurityManager(new GatlingSecurityManager());
String[] args = {"--simulation", gatlingGun,
"--results-folder", "gatling-results",
"--binaries-folder", binaries};
try {
io.gatling.app.Gatling.main(args);
}catch(SecurityException se){
LOG.debug("gatling test finished.");
}
System.setSecurityManager(sm);
}
The simple security manager i used:
public class GatlingSecurityManager extends SecurityManager {
#Override
public void checkExit(int status){
throw new SecurityException("Tried to exit.");
}
#Override
public void checkPermission(Permission perm) {
return;
}
}
The problem is then getting the information you want out of the simulation after it has been run.
I've just tried to write a simple test for Auth:
use Mockery as m;
...
public function testHomeWhenUserIsNotAuthenticatedThenRedirectToWelcome() {
$auth = m::mock('Illuminate\Auth\AuthManager');
$auth->shouldReceive('guest')->once()->andReturn(true);
$this->call('GET', '/');
$this->assertRedirectedToRoute('general.welcome');
}
public function testHomeWhenUserIsAuthenticatedThenRedirectToDashboard() {
$auth = m::mock('Illuminate\Auth\AuthManager');
$auth->shouldReceive('guest')->once()->andReturn(false);
$this->call('GET', '/');
$this->assertRedirectedToRoute('dashboard.overview');
}
This is the code:
public function getHome() {
if(Auth::guest()) {
return Redirect::route('general.welcome');
}
return Redirect::route('dashboard.overview');
}
When I run, I've got the following error:
EF.....
Time: 265 ms, Memory: 13.00Mb
There was 1 error:
1) PagesControllerTest::testHomeWhenUserIsNotAuthenticatedThenRedirectToWelcome
Mockery\Exception\InvalidCountException: Method guest() from Mockery_0_Illuminate_Auth_AuthManager should be called
exactly 1 times but called 0 times.
—
There was 1 failure:
1) PagesControllerTest::testHomeWhenUserIsAuthenticatedThenRedirectToDashboard
Failed asserting that two strings are equal.
--- Expected
+++ Actual
## ##
-'http://localhost/dashboard/overview'
+'http://localhost/welcome'
My questions are:
Two similar test cases but why the error output differs? First one the mock Auth::guest() is not called while the second one seems to be called.
On the second test case, why does it fail?
Is there any way to write better tests for my code above? Or even better code to test.
Above test cases, I use Mockery to mock the AuthManager, but if I use the facade Auth::shoudReceive()->once()->andReturn(), then it works eventually. Is there any different between Mockery and Auth::mock facade here?
Thanks.
You're actually mocking a new instance of the Illuminate\Auth\AuthManager and not accessing the Auth facade that is being utilized by your function getHome(). Ergo, your mock instance will never get called. (Standard disclaimer that none of the following code is tested.)
Try this:
public function testHomeWhenUserIsNotAuthenticatedThenRedirectToWelcome() {
Auth::shouldReceive('guest')->once()->andReturn(true);
$this->call('GET', '/');
$this->assertRedirectedToRoute('general.welcome');
}
public function testHomeWhenUserIsAuthenticatedThenRedirectToDashboard() {
Auth::shouldReceive('guest')->once()->andReturn(false);
$this->call('GET', '/');
$this->assertRedirectedToRoute('dashboard.overview');
}
If you check out Illuminate\Support\Facades\Facade, you'll see that it takes care of mocking for you. If you really wanted to do it the way that you were doing it (creating an instance of mock instance of Auth), you'd have to somehow inject it into the code under test. I believe that it could be done with something like this assuming that you extend from the TestCase class provided by laravel:
public function testHomeWhenUserIsNotAuthenticatedThenRedirectToWelcome() {
$this->app['auth'] = $auth = m::mock('Illuminate\Auth\AuthManager');
// above line will swap out the 'auth' facade with your facade.
$auth->shouldReceive('guest')->once()->andReturn(true);
$this->call('GET', '/');
$this->assertRedirectedToRoute('general.welcome');
}
Since getting started in Dart I've been watching for a way to execute Dart (Text) Source (that the same program may well be generating dynamically) as Code. Like the infamous "eval()" function.
Recently I have caught a few hints that the communication port between Isolates support some sort of "Spawn" that seems like it could allow this "trick". In Ruby there is also the possibility to load a module dynamically as a language feature, perhaps there is some way to do this in Dart?
Any clues or a simple example will be greatly appreciated.
Thanks in advance!
Ladislav Thon provided this answer on the Dart forum:
I believe it's very safe to say that Dart will never have eval. But it will have other, more structured ways of dynamically generating code (code name mirror builders). There is nothing like that right now, though.
There are two ways of spawning an isolate: spawnFunction, which runs an existing function from the existing code in a new isolate, so nothing you are looking for, and spawnUri, which downloads code from given URI and runs it in new isolate. That is essentially dynamic code loading -- but the dynamically loaded code is isolated from the existing code. It runs in a new isolate, so the only means of communicating with it is via message passing (through ports).
You can run a string as Dart code by first constructing a data URI from it and then passing it into Isolate.spawnUri.
import 'dart:isolate';
void main() async {
final uri = Uri.dataFromString(
'''
void main() {
print("Hellooooooo from the other side!");
}
''',
mimeType: 'application/dart',
);
await Isolate.spawnUri(uri, [], null);
}
Note that you can only do this in JIT mode, which means that the only place you might benefit from it is Dart VM command line apps / package:build scripts. It will not work in Flutter release builds.
To get a result back from it, you can use ports:
import 'dart:isolate';
void main() async {
final name = 'Eval Knievel';
final uri = Uri.dataFromString(
'''
import "dart:isolate";
void main(_, SendPort port) {
port.send("Nice to meet you, $name!");
}
''',
mimeType: 'application/dart',
);
final port = ReceivePort();
await Isolate.spawnUri(uri, [], port.sendPort);
final String response = await port.first;
print(response);
}
I wrote about it on my blog.
Eval(), in Ruby at least, can execute anything from a single statement (like an assignment) to complete involved programs. There is a substantial time penalty for executing many small snippets over most any other form of execution that is possible.
Looking at the problem closer, there are at least three different functions that were at the base of the various schemes where eval might be used. Dart handles at least 2 of these in at least minimal ways.
Dart does not, nor does it look like there is any plan to support "general" script execution.
However, the NoSuchMethod method can be used to effectively implement the dynamic "injection" of variables into your local class environment. It replaces an eval() with a string that would look like this: eval( "String text = 'your first name here';" );
The second function that Dart readily supports now is the invocation of a method, that would look like this: eval( "Map map = SomeClass.some_method()" );
After messing about with this it finally dawned on me that a single simple class can be used to store the information needed to invoke a method, for a class, as a string which seems to have general utility. I can replace a big maintenance prone switch statement that might otherwise be necessary to invoke a series of methods. In Ruby this was almost trivial, however in Dart there are some fairly less than intuitive calls so I wanted to get this "trick" in one place, which fits will with doing ordering and filtering on the strings such as you may need.
Here's the code to "accumulate" as many classes (a whole library?) into a map using reflection such that the class.methodName() can be called with nothing more than a key (as a string).
Note: I used a few "helper methods" to do Map & List functions, you will probably want to replace them with straight Dart. However this code is used and tested only using the functions..
Here's the code:
//The used "Helpers" here..
MAP_add(var map, var key, var value){ if(key != null){map[key] = value;}return(map);}
Object MAP_fetch(var map, var key, [var dflt = null]) {var value = map[key];if (value==null) {value = dflt;}return( value );}
class ClassMethodMapper {
Map _helperMirrorsMap, _methodMap;
void accum_class_map(Object myClass){
InstanceMirror helperMirror = reflect(myClass);
List methodsAr = helperMirror.type.methods.values;
String classNm = myClass.toString().split("'")[1]; ///#FRAGILE
MAP_add(_helperMirrorsMap, classNm, helperMirror);
methodsAr.forEach(( method) {
String key = method.simpleName;
if (key.charCodeAt(0) != 95) { //Ignore private methods
MAP_add(_methodMap, "${classNm}.${key}()", method);
}
});
}
Map invoker( String methodNm ) {
var method = MAP_fetch(_methodMap, methodNm, null);
if (method != null) {
String classNm = methodNm.split('.')[0];
InstanceMirror helperMirror = MAP_fetch(_helperMirrorsMap, classNm);
helperMirror.invoke(method.simpleName, []);
}
}
ClassMethodMapper() {
_methodMap = {};
_helperMirrorsMap = {};
}
}//END_OF_CLASS( ClassMethodMapper );
============
main() {
ClassMethodMapper cMM = new ClassMethodMapper();
cMM.accum_class_map(MyFirstExampleClass);
cMM.accum_class_map(MySecondExampleClass);
//Now you're ready to execute any method (not private as per a special line of code above)
//by simply doing this:
cMM.invoker( MyFirstExampleClass.my_example_method() );
}
Actually there some libraries in pub.dev/packages but has some limitations because are young versions, so that I can recommend you this library expressions to dart and flutter.
A library to parse and evaluate simple expressions.
This library can handle simple expressions, but no blocks of code, control flow statements and so on. It supports a syntax that is common to most programming languages.
There I create an example of code to evaluate arithmetic operations and comparations of data.
import 'package:expressions/expressions.dart';
import 'dart:math';
#override
Widget build(BuildContext context) {
final parsing = FormulaMath();
// Expression example
String condition = "(cos(x)*cos(x)+sin(x)*sin(x)==1) && respuesta_texto == 'si'";
Expression expression = Expression.parse(condition);
var context = {
"x": pi / 5,
"cos": cos,
"sin": sin,
"respuesta_texto" : 'si'
};
// Evaluate expression
final evaluator = const ExpressionEvaluator();
var r = evaluator.eval(expression, context);
print(r);
return Scaffold(
body: Container(
margin: EdgeInsets.only(top: 50.0),
child: Column(
children: [
Text(condition),
Text(r.toString())
],
),
),
);
}
I/flutter (27188): true