I am using ChaiJS with my Casper-Chai plugin, and I am not sure how to go about a particular issue I am encountering.
I would like to be able to write tests such as:
expect(casper).selector("#waldo").to.be.visible;
This is pretty straightforward, with a call such as
utils.addChainableMethod(chai.Assertion.prototype, 'selector',
selectorMethod, selectorChainableMethod);
utils.addMethod(chai.Assertion.prototype, 'visible', visibleMethod);
Where the *Method references are to functions that perform respective tests or chainable calls.
My question is what is the best way to have e.g. 'selector' modify descendants in the chain. Two options come to mind:
use utils.flag(chai, 'object') to change it to the selector; or
create a new flag with e.g. utils.flag(chai, 'casper-selector')
When 'visible' is called, it can read the respective flag. Where modifying the 'object' seems to be useful is when calling e.g. 'length' later-on. However, I am concerned somewhat about unexpected side-effects of changing the 'object'.
I may also want to modify the object, for 'length' tests, down the chain, such as:
// there can be only one!
expect(casper).selector("#waldo").length(1)
// but that one has 4 classes
expect(casper).selector("#waldo").class.to.have.length(4)
Grateful for any thoughts and input.
---- EDIT ----
Okay, so here is the conceptual challenge that gave root to Casper-Chai, that requires a bit of a description of what Casper is and why Casper-Chai should be a Chai plugin and not just an alternative to the existing Casper API. Casper is a wrapper around the PhantomJS headless web browser, and as such Casper runs two effectively two distinct virtual machines: A
"Controller" and a headless web browser.
There is no DOM or "document" or "window" object in the controller; the Controller is in this respect a lot like Node.js, albeit using the WebKit javascript parser. In parallel, PhantomJS starts a headless web browser. The Controller can then communicate through a PhantomJS/Casper API to the headless browser. The Controller can tell the headless browser what pages to load, what javascript to run (which is like typing javascript into the console), and even mimic events such as keyboard input and mouse clicks. The headless browser has a full DOM and javascript stack: it is a web-page loaded in WebKit. You can capture screenshots of what WebKit would render.
Casper-Chai is run in the Controller. Tests created in Mocha + Chai in the Controller are meant to evaluate against the state of the headless browser. Although we can copy state from the browser to the Controller and run tests on that copied state, my limited experimentation with that design revealed problems inherent to the design (i.e. efficiency, race conditions, performance, and potential side-effects). The problem is that the browser state is dynamic, complex, and can be unwieldy.
So taking John's example, expect(casper.find("#waldo")).to.be.visible would not work, since there is no DOM, unless the object returned by Casper did some sort of lazy evaluation/mediation. Even if we serialized and copied the HTML element there there is no CSS in the Controller. Then, even if there were CSS for #waldo, there is no way to test the hierarchy as if it were a web-browser. We would have to copy a substantial portion of the DOM and all the CSS and then replicate a web-browser to test whether #waldo is visible - for every single test. Casper-Chai is meant to avoid this problem by running the tests in the browser.
Just for a little extra illumination, a trivial comparison is getting the number of elements that match a selector. One could write expect(casper.evaluate(function () {return __utils__.findAll('.my_class')}).to.have.length(4), where casper.evaluate runs the given function in the headless browser and returns a list of DOM elements matching the selector as strings; and you can think of __utils__ as Casper's version of jQuery. Alternatively, one could write expect(casper).selector('.my_class').to.have.length(4) where selector becomes the 'object' and it has a getter .length that calls 'casper.evaluate(function () { return utils.findAll('.my_class').length`. Only the integer length is returned. For a short number of tests either works fine, but for larger numbers of tests this performance characteristic becomes impactful (here, in this simplistic form, and potentially to a substantially greater extent in more complex cases).
One could of course write expect(casper.evaluate(function () { __utils__.findAll('.my_class').length }).equal(4), but if one is going to write tests like this, why bother with BDD/Chai? It eliminates the benefit of readability that Chai offers.
It is also worth noting that there may be multiple Casper instances in the Controller, corresponding to multiple PhantomJS pages. Think of them as spooky tabs.
So given Domenic's answer that modifying the 'object' flag is the appropriate way to go about it, this seems the most practical way - subject to any thoughts in light of the above description.
I hope the above describes why Casper-Chai should be a plugin and not just an API extension to Casper. I'll also run this by Casper's author to see if he has any input.
It may not be a perfect relationship, but I am hopeful that Casper & Chai can get along handily. :)
The difficulty stems from the fact that casper has a highly procedural API, with methods like Casper#click(String selector) and Casper#fetchText(String selector). To fit naturally with chai, an object-oriented API is needed, e.g. Casper#find(String selector) (returning a CasperSelection object), CasperSelection#click(), CasperSelection#text(), etc.
Therefore I suggest you extend the casper object itself with a find or selector method which returns an object on which you can base your assertions. Then you will not need to change the object flag.
expect(casper.find("#waldo")).to.be.visible;
expect(casper.find("#waldo")).to.have.length(1)
expect(casper.find("#waldo").class).to.have.length(4)
I mostly agree with #John. You're performing your expectations on some other object, so saying
expect(casper).select("#waldo").to.have.length(1)
is very strange. You're not expecting anything about casper, you're expecting something about casper.find("#waldo"). Consider also the should syntax:
casper.should.select("#waldo").have.length(1)
// vs.
casper.find("#waldo").should.have.length(1)
That said, if you're dead set on this kind of API, this is exactly what the object flag is for. Chai even does this, to make assertions like
myObj.should.have.property("foo").that.equals("bar")
work well:
https://github.com/chaijs/chai/blob/49a465517331308695c3d8262cdad42c3ac591ef/lib/chai/core/assertions.js#L773
Related
I'm writing a new Vue project and want to test my components with vue-test-utils.
So far I've been writing unit-tests with a test case for each component method, testing its expected behavior while mocking other methods that it might call.
Recently I've read on the Vue docs that the unit tests for components should test general behavior and not rely on implemenation details:
https://vuejs.org/guide/scaling-up/testing.html#component-testing
which makes sense, but I still want to test the logic of my component's methods.
I don't want to extract it to different files / composables because they heavily rely on the component's data and other methods, and I don't wanna pass everything as parameters.
What do you recommend regarding this?
I can't mock methods anymore since setMethods is now deprecated in vue-test-utils, which is making it harder to test each method separately.
do you think I should give up on testing each method?
I find it helpful to test methods because usually small changes make those tests fail which help me notice errors in the code, but also makes it harder because every small legitimate change requires unit-test changes as well.
We have done unit test using vue-jest. Here are the steps.
Use the following code to set up the crypto property globally. This will allow Jest to access window.crypto and won't cause any issue.Create a crypto.mock file and paste following code.
const crypto = require('crypto')
Object.defineProperty(global.self, 'crypto', {
value: {
getRandomValues: arr => crypto.randomBytes(arr.length)
}
})
Now import this file in your test file component. Now You can use jasmine.Spy to spy on the elements.
let spyFor: jasmine.Spy; spyFor = spyOn(function.hello, 'hello'); expect(spyFor).toHaveBeenCalledTimes(1);
In this way you can mock on the methods. Pls free to ask if you have anything more.
Is it a good practice to write a EXPECT(something) inside a test double (e.g. spy or mock) method? To ensure the test double is used in a specific way for testing?
If not, what would be a preferred solution?
If you would write a true Mock (as per definition from xUnit Test Patterns) this is exactly what defines this kind of test double. It is set up with the expectations how it will be called and therefore also includes the assertions. That's also how mocking frameworks produce mock objects under the hood. See also the definition from xUnit Test Patterns:
How do we implement Behavior Verification for indirect outputs of the SUT?
How can we verify logic independently when it depends on indirect inputs from other software components?
Replace an object the system under test (SUT) depends on with a test-specific object that verifies it is being used correctly by the SUT.
Here, indirect outputs means that you don't want to verify that the method under test returns some value but that there is something happening inside the method being tested that is behaviour relevant to callers of the method. For instance, that while executing some method the correct behaviour lead to an expected important action. Like sending an email or sending a message somewhere. The mock would be the doubled dependency that also verifies itself that this really happened, i.e. that the method under test really called the method of the dependency with the expected parameter(s).
A spy on the other hand shall just record things of interest that happened to the doubled dependency. Interrogating the spy about what happened (and sometimes also how often) and then judging if that was correct by asserting on the expected events is the responsibility of the test itself. So a mock is always also a spy with the addition of the assertion (expectation) logic. See also Uncle Bobs blog The Little Mocker for a great explanation of the different types of test doubles.
TL;DR
Yes, the mock includes the expectations (assertion) itself, the spy just records what happened and lets the test itself asks the spy and asserts on the expected events.
Mocking frameworks also implement mocks like explained above as they all follow the specified xunit framework.
mock.Verify(p => p.Send(It.IsAny<string>()));
If you look at the above Moq example (C#), you see that the mock object itself is configured to in the end perform the expected verification. The framework makes sure that the mock's verification methods are executed. A hand-written would be setup and than you would call the verification method on the mock object yourself.
Generally, you want to put all EXPECT statements inside individual tests to make your code readable.
If you want to enforce certain things on your test stub/spy, it is probably better to use exceptions or static asserts because your test is usually using them as a black box, and it uses them in an unintended way, your code will either not get compiled, or it will throw and give you the full stack trace which also will cause your test to fail (so you can catch the misuse).
For mocks, however, you have full control over the use and you can be very specific about how they are called and used inside each test. For example in Google test, using GMock matchers, you can say something like:
EXPECT_CALL(turtle, Forward(Ge(100)));
which means expect Forward to be called on the mock object turtle with a parameter equal or greater than 100. Any other value will cause the test to fail.
See this video for more examples on GMock matchers.
It is also very common to check general things in a test fixture (e.g. in Setup or TearDown). For example, this sample from google test enforces each test to finish in a certain amount of time, and the EXPECT statement is in teardown rather than each individual test.
What is the difference between TestCafes t.eval() function and ClientFunctions?
I've been using TestCafe for a while now and didn't stumble across the eval function until now.
I am curious, but I also have a concrete use case, which is not the subject of this question, but might help me understand the difference between both.
I'd like to do simple page manipulation tasks like hiding elements from my tests code.
The documentation clearly says that ClientFunction should not be used for this: "Do not modify the tested webpage within client functions. Use test actions to interact with the page instead."
Is the eval function suitable for this?
In essence, they are the same. The difference is that ClientFunction creates an instance that can be saved to a variable and then reused several times, while t.eval works without saving. Also, t.eval is executed immediately, while ClientFunction is not - it creates an instance that you need to call explicitly in order to run it.
I've been creating PageObjects for WebDriverIO and have been following the ES6 method for Page Object pattern in the WebDriverIO documentation.
However, someone on my team suggested simply making objects of selectors, and then calling those strings in the tests. Is there a good reason why the Page Object pattern returns elements and not string of the selectors?
Page Object returns elements instead of just the selector string to allow actions to be called directly on the elements e.g.
PageObject.Element.waitForDisplayed()
Instead of you doing
Browser.waitForDisplayed(PageObject.Element)
Which may get lengthy and doesn't chain as well. You can find more actions that can be performed on elements here
However, you can also get the string of the selector if you want by doing
PageObject.Element.selector()
Chaining e.g.
PageObject.Element.waitForDisplayed().click()
I think the point is allow you to use the objects directly. So:
MyPageObject.MyElement.click()
versus:
browser.click(MyPageObject.MyElement)
Just a little less verbose
I found, that when I writing unit tests, especially for methods who do not return the value, I mostly write tests in white box testing manner. I could use reflection to read private data to check is it in the proper state after method execution, etc...
this approach has a lot of limitation, most important of which is
You need to change your tests if you rework method, even is API stay
the same
It's wrong from information hiding (encapsulation) point of view -
tests is a good documentation for our code, so person who will read
it could get some unnecessary info about implementation
But, if method do not return a value and operate with private data, so it's start's very hard (almost impossible) to test like with a black-box testing paradigm.
So, any ideas for a good solution in that problem?
White box testing means that you necessarily have to pull some of the wiring out on the table to hook up your instruments. Stuff I've found helpful:
1) One monolithic sequence of code, that I inherited and didn't want to rewrite, I was able to instrument by putting a state class variable into, and then setting the state as each step passed. Then I tested with different data and matched up the expected state with the actual state.
2) Create mocks for any method calls of your method under test. Check to see that the mock was called as expected.
3) Make needed properties into protected instead of private, and create a sub-class that I actually tested. The sub-class allowed me to inspect the state.
I could use reflection to read private data to check is it in the proper state after method execution
This can really be a great problem for maintenance of your test suite
in .Net instead you could use internal access modifier, so you could use the InternalsVisibleToAttribute in your class library to make your internal types visible to your unit test project.
The internal keyword is an access modifier for types and type members. Internal types or members are accessible only within files in the same assembly
This will not resolve every testing difficulty, but can help
Reference