Integration tests with Laika and PhantomJS - testing

I just started using Laika for doing some TDD on my Meteor app. Though, I would like to do some integration tests, as unit tests isn't that valuable to me.
Can I do some screen capturing using PhantomJS through Laika? E.g. I want to click html links and select elements by class/id.
I have a basic (unit) test in coffee:
# tests/players_test.coffee
assert = require 'assert'
suite 'Players', ->
test 'in the server', (done, server) ->
server.eval ->
Players.insert title: 'hello there'
players = Players.find().fetch()
emit('players', players)
server.once 'players', (players) ->
assert.equal 1, players.length
done()
I would like to convert this into a integration test by using an client (added next to (done, server) in the test function) and then manually selecting tags and clicking links, filling in name etc., clicking e.g. 'register', and then checking if that user is to be found in the database.
Thanks!

Yes you can do this.
suite 'Players', ->
test 'in the server', (done, server, client) ->
client.eval ->
// get access to a DOM element (can optionally use jQuery instead)
player = document.querySelector("[data-test='player']")
// now we can call functions on the element
player.value = "Joe blogs"
player.click()
// if you know the element won't exist in the DOM yet use waitForDOM
waitForDOM "[data-test='something-else']", ->
// perform some logic now that the element exists in the DOM
emit('players', players)
server.once 'players', (players) ->
assert.equal 1, players.length
done()
You might also want to check out evalSync here: http://arunoda.github.io/laika/syntax-sugar.html
This lets you write your tests in a synchronous style. It's still being executed asynchronously but it means you don't have to wrap your head around all the different named triggers/subscriptions that the 'eval' tests have. Here are the basics of evalSync...
suite 'evalSync', ->
test 'evalSync for the win', (done, server, client) ->
client.evalSync ->
// perform some logic on the client
emit("return")
server.evalSync ->
// perform some logic on the server
emit("return")
done() // notice this is outside the 'evalSync'
As a side note, I recommend using "data-test" attributes on your elements (or some other custom data attribute). If you select by class or id in your tests and then later refactor your CSS/HTML you'll have to hunt down which classes/ids are being used by your CSS and which are being used by your tests. Using "data-test" makes it clear for you.

Related

Is there a way to test roblox games?

As I started to understand a little bit more about Roblox, I was wondering if there is any possible way to automate the testing. As a first step only on the Lua scripting, but ideally also simulating the game and interactions.
Is there any way of doing such a thing?
Also if there are already best practices on doing testing on Roblox(this includes Lua scripting) I would like to know more about them.
Unit Testing
For lua modules, I would recommend the library TestEZ. It was developed in-house by Roblox engineers to allow for behavior driven tests. It allows you to specify a location where test files exist and will gives you pretty detailed output as to how your tests did.
This example will run in RobloxStudio, but you can pair it with other libraries like Lemur for command-line and continuous integration workflows. Anyways, follow these steps :
1. Get the TestEZ Library into Roblox Studio
Download Rojo. This program allows you to convert project directories into .rbxm (Roblox model object) files.
Download the TestEZ source code.
Open a Powershell or Terminal window and navigate into the downloaded TestEZ directory.
Build the TestEZ library with this command rojo build --output TestEZ.rbxm .
Make sure that it generated a new file called TestEZ.rbxm in that directory.
Open RobloxStudio to your place.
Drag the newly created TestEZ.rbxm file into the world. It will unpack the library into a ModuleScript with the same name.
Move this ModuleScript somewhere like ReplicatedStorage.
2. Create unit tests
In this step we need to create ModuleScripts with names ending in `.spec` and write tests for our source code.
A common way to structure code is with your code classes in ModuleScripts and their tests right next to them. So let's say you have a simple utility class in a ModuleScript called MathUtil
local MathUtil = {}
function MathUtil.add(a, b)
assert(type(a) == "number")
assert(type(b) == "number")
return a + b
end
return MathUtil
To create tests for this file, create a ModuleScript next to it and call it MathUtil.spec. This naming convention is important, as it allows TestEZ to discover the tests.
return function()
local MathUtil = require(script.parent.MathUtil)
describe("add", function()
it("should verify input", function()
expect(function()
local result = MathUtil.add("1", 2)
end).to.throw()
end)
it("should properly add positive numbers", function()
local result = MathUtil.add(1, 2)
expect(result).to.equal(3)
end)
it("should properly add negative numbers", function()
local result = MathUtil.add(-1, -2)
expect(result).to.equal(-3)
end)
end)
end
For a full breakdown on writing tests with TestEZ, please take a look at the official documentation.
3. Create a test runner
In this step, we need to tell TestEZ where to find our tests. So create a Script in ServerScriptService with this :
local TestEZ = require(game.ReplicatedStorage.TestEZ)
-- add any other root directory folders here that might have tests
local testLocations = {
game.ServerStorage,
}
local reporter = TestEZ.TextReporter
--local reporter = TestEZ.TextReporterQuiet -- use this one if you only want to see failing tests
TestEZ.TestBootstrap:run(testLocations, reporter)
4. Run your tests
Now we can run the game and check the Output window. We should see our tests output :
Test results:
[+] ServerStorage
[+] MathUtil
[+] add
[+] should properly add negative numbers
[+] should properly add positive numbers
[+] should verify input
3 passed, 0 failed, 0 skipped - TextReporter:87
Automation Testing
Unfortunately, there does not exist a way to fully automate the testing of your game.
You can use TestService to create tests that automate the testing of some interactions, like a player touching a kill block or checking bullet paths from guns. But there isn't a publicly exposed way to start your game, record inputs, and validate the game state.
There's an internal service for this, and a non-scriptable service for mocking inputs but without overriding CoreScripts, it's really not possible at this moment in time.

Using mocks in Karate DSL feature file with stanalone run

I have REST service, written in language different from Java.
It have few dependencies from other REST services.
For example service under development and testing is A, other services are respectively B and C.
I want to run system test for A, some tests require B or/and C to be online and perform queries from A.
I wrote b-mock.featue and c-mock.feature to represent that services in mock.
Also I wrote some a-test-smth.feature files to run test against A
Is it possible to add some information into a-test-smth.feature to enable some mocks for concrete test?
Now I should run standalone karate.jar twice, first - for mocking. second - for run tests. That approach works, but, I can't ceck that:
some API calls to A not required B or C
can't emulate service B down or for example slow or incorrect response answer fetching
Thanks.
Are you using Java ? If so then the best approach is to perform the set-up of your test in Java code. You can start 2 mocks for B and c and then start the main test for your service A. And at the end do clean-up if needed.
You can refer this as an example: https://github.com/intuit/karate/tree/master/karate-netty#consumer-provider-example
Row 3 shows how you can start a mock and run a Karate test.
If you are not using Java and would like to use only the stand-alone JAR, it is actually possible using Java-interop and quite easy, I just tried it.
EDIT: This API is now built into Karate, so you don't need to write the extra JS code below: https://github.com/intuit/karate/tree/master/karate-netty#within-a-karate-test
(Obsolete)
First create this bit of JavaScript code that is smart enough to start a Karate mock:
function() {
var Mock = Java.type('com.intuit.karate.netty.FeatureServer');
var file = new java.io.File('src/test/java/mock/web/cats-mock.feature');
var server = Mock.start(file, 0, false, null);
return server.port;
}
And this is how it can look in the Background of your main Karate test. You can see how you can do some conditional logic if needed and you have plenty of ways to change things based on your environment.
Background:
* def starter = read('start-mock.js')
* def port = karate.env == 'mock' ? starter() : 8080
* url 'http://localhost:' + port + '/cats'
Does this answer your question ? Let me know and I will add this trick to the documentation !

Running multiple browser instances in the same test spec

If I have a single spec that is using page object model, how do I run multiple browser instance for that same spec?
For example I have spec:
it('should run multi browser', function() {
browser.get('http://example.com/searchPage');
var b2 = browser.forkNewDriverInstance();
b2.get('http://example.com/searchPage');
var b3 = browser.forkNewDriverInstance();
b3.get('http://example.com/searchPage');
SearchPage.searchButton.click();
b2.SearchPage.searchButton.click(); //fails here
b3.SearchPage.searchButton.click();
});
How do I reuse vars declared in the SearchPage page object for the other browser instances?
This is a really interesting question that is not covered in Using Multiple Browsers in the Same Test or in the interaction_spec.js.
The problem with page objects is that page object fields are usually defined with a globally available element or browser which in your case would always point to the first browser instance. But you basically need to call element() using a specific browser:
b2.element(by.id('searchInput'));
instead of just:
element(by.id('searchInput'));
FYI, element is just a shortcut for browser.element.
I am really not sure whether this is a reliable solution and would actually work, but you can redefine global element this way. Think about it as switching the search context to different browser instances:
SearchPage.searchButton.click();
global.element = b2.element;
SearchPage.searchButton.click();
global.element = b3.element;
SearchPage.searchButton.click();
global.element = browser.element;

Ember's App.reset and model.rollback causing buggy behavior during tests

Before each test, I reset the application like so:
QUnit.testStart = ->
App.reset()
For the most part, this works great. However, in one of my routes I've defined a rollback:
App.ProjectsNewRoute = Ember.Route.extend
model: ->
#store.createRecord('project')
setupController: (controller, model) ->
controller.set('content', model)
deactivate: ->
#get('controller.model').rollback()
When the rollback is present, a number of seemingly unrelated integration and route tests fail. If I comment out the rollback, these other tests pass as expected. Alternatively, I can comment App.reset(), and everything works as expected. So why aren't these two playing nicely? As my test suite grows, I'll need to be able to call App.reset() without it breaking things.

Running a GEB test using Intellij

Being a beginner in GEB testing, I am trying to run a simple login program in Intellij. Could you please help me run this test in Intellij? My question is what selections should I make in the edit configurations page? Please help. This example is from the book of geb.
import geb.Browser
Browser.drive {
go "http://google.com/ncr"
// make sure we actually got to the page
assert title == "Google"
// enter wikipedia into the search field
$("input", name: "q").value("wikipedia")
// wait for the change to results page to happen
// (google updates the page dynamically without a new request)
waitFor { title.endsWith("Google Search") }
// is the first link to wikipedia?
def firstLink = $("li.g", 0).find("a.l")
assert firstLink.text() == "Wikipedia"
// click the link
firstLink.click()
// wait for Google's javascript to redirect to Wikipedia
waitFor { title == "Wikipedia" }
}
If you are running this in IntelliJ you should be able to run this as a JUnit test (ctrl+F10). Make sure that this is inside of a Class and in a method.
For ease of syntax, it would be good to use Spock as your BDD framework (include the library in your project; if using Maven, follow the guide on the site but update to Spock 0.7-groovy-2.0 and Geb 0.9.0-RC-1 for the latest libraries
If you want to switch from straight JUnit to Spock (keep in mind you should use JUnit as a silent library) then your test case looks like this:
def "show off the awesomeness of google"() {
given:
go "http://google.com/ncr"
expect: "make sure we actually got to the page"
title == "Google"
when: "enter wikipedia into the search field"
$("input", name: "q").value("wikipedia")
then: "wait for the change to results page to happen and (google updates the page dynamically without a new request)"
waitFor { title.endsWith("Google Search") }
// is the first link to wikipedia?
def firstLink = $("li.g", 0).find("a.l")
and:
firstLink.text() == "Wikipedia"
when: "click the link"
firstLink.click()
then: "wait for Google's javascript to redirect to Wikipedia"
waitFor { title == "Wikipedia" }
}
Just remember: Ctrl + F10 (best key shorcut for a test in IntelliJ!)
The above is close but no cigar, so to speak.
If you want to run bulk standard Gebish test from WITHIN Intellij,
I tried 2 things.
I added my geckodriver.exe to the test/resources under my Spock/Geb tests
I literally in the given: part of my Spok/Geb test did the following:
given:
System.setProperty("webdriver.gecko.driver", "C:\\repo\\geb-example-gradle\\src\\test\\resources" + "\\geckodriver.exe");
Failed
Succeeded
Now the usual deal with answers is, that someone writes something, you try it and then it fails.
So, if it did not work for you, use the reference Geb/Spock project on Github as follows and import it into intellij (remember, New Project, then find the gradle.build script and then intellij will import it nicely)...it also kicks off a build so dont freak out:
https://github.com/geb/geb-example-gradle
Download the driver:
https://github.com/mozilla/geckodriver/releases
and move it to the test/resource folder of the reference project you just imported under test/groovy...(see image)
Now add the above given: clause to the GebishOrgSpec Sock/Geb test:
The test runs nicely from WITHIN Intellij. Evidence the browser open and the test running:
LOVELY JOBBLY :=)