How to avoid duplicated Cucumber steps while multiple team members working on different feature files in parallel?
Sometimes we find some similar steps with different context but are 90%-100% the same. The problem occurs when the tests are complex and need to write new steps for new feature that we didn't have before.
Any tips good tips that could help solving this issue?
Are there good tools that can manage and search steps to avoid duplicated statements?
Thanks,
Everyone writes scenarios in a different way.
One way you could start to avoid duplication is to document your steps and what they do, so that you and your team can look back on them to see if there is an existing step that they can use.
For steps where the functionality is almost the same, you could merge the step definitions: Given I have logged in vs When I log in - Cucumber Expression: "I( have) log(ged) in", Regex: /I(?:| have) log(?:|ged) in/ as an example.
For your example, we would need to know what you're trying to achieve in your step definition. For instance, you could be filling out a form where you have to select an option in a drop down dependant on the user that is logged in:
Psuedo-Code:
// Some stuff here to get to the dropdown
if(user.name === "Bob Ross"){
form.dropdown.select.option(2);
} else if (user.name === "Ellen Ripley"){
form.dropdown.select.option(3);
} else {
form.dropdown.select.option(1);
}
// Some other stuff here to complete this step
Basically, it depends on what you're attempting to do with your semi-duplicate steps.
Can you check what journey you are completing (with a previously ran step and a variable, a Url, a user that is logged in etc) and do something differently? Perhaps write a helper function for it and keep the step definition as just a block of ifs to choose the right one.
Can you merge steps that hold the same functionality?
If you can do either of these, then you should be good to go.
Related
I am testing a large project with long scenarios (some with more than 100 interactions with webpage). I would like to break them down into shorter steps that run in sequence (like in Mocha) but I don't know how to do that.
Example: In a single test, I would like to run
fixture('test1')
test('test1', async (t) => {
...login
...createSubAccount
...modifySubAccount
...activateSubAccount
})
where each of the steps would show in console and in report. Right now, the only thing I know how to do is to put each step into its own test() context, but that means that if e.g. createSubAccount fails, modifySubAccount and activateSubAccount will still run (even though the workflow already failed). Also, there is the unhappy part that each test() clears the browser (but I can deal with that).
In short: How can I split the tests in a way that if a single substep of fixture fails, the whole fixture fails immediately? Or similar thing, but for test()?
Also, I don't want the whole pipeline to end on the first test failure, as would happen with --stopOnFirstFail flag - I want to run all the tests, to find which are failing.
test() is the smallest unit. The idea is it's an independent piece of testing code, e.i. a bunch of test steps. This doesn't change no matter what tool you use (TestCafe, Playwright, Puppeteer, Cypress, mocha, Jest, ...).
And so:
Right now, the only thing I know how to do is to put each step into its own test() context, but that means that if e.g. createSubAccount fails, modifySubAccount and activateSubAccount will still run (even though the workflow already failed).
seems like breaking one of the main principles of tests, that is they are independent. Don't split test steps that belong together between different tests.
If the only drawback now is the length of your test, why don't you do it like you hinted at in the example:
test('test1', async (t) => {
login();
createSubAccount();
modifySubAccount();
activateSubAccount();
});
you can create functions for login, createAccount etc. and then use only such function in your tests, which would make them as short as shown here. You can also easily create various scenarious:
test('activate account without modification', async (t) => {
login();
createSubAccount();
activateSubAccount();
});
test('create account', async (t) => {
login();
createSubAccount();
});
test('create account without login', async (t) => {
createSubAccount();
});
// and so on
It doesn't even look that long.
TestCafe does not support the functionality you require at the moment. The only solution I could think of is, as you proposed, to implement your test as a fixture with steps as tests, use disablePageReloads feature (NOTE: it is experimental), track the number of passed tests manually, and check it at the beginning of each test. It is a bit tedious, but it should work as you need.
Another solution that has not been implemented yet and the easiest way to split the long test into steps is to simply divide it into functions. The only issue that may arise is related to reporting. Even if you implement a custom reporter, there is no possibility to pass information about the steps into it (you can vote for the corresponding feature request).
Also, I would like to draw your attention to Page Model pattern. This can shrink your tests and make them more readable.
Please open a new feature request with a comprehensive description if you have a better idea of how this should be done.
We are trying to automate E2E test cases for an booking application which involves around 60+ steps for each test case. Whenever there is a failure at the final steps it is very much time consuming if we go for traditional retry option since the test case will be executed from step 1 again. On the application we have some logical steps which can be marked somehow through which we would like to achieve resuming the test case from a logical point before the failed step. Say for example, among the 60 steps say every 10th step is a logical point in which the script can be resumed instead of retrying from the step 1. say if the failure is on line number 43 then with the help of booking reference number the test can be resumed from step number 41 since the validation has been completed till step 40 (step 40 is a logical closure point). There might be an option you may suggest to split the test case into smaller modules, which will not work for me since it is an E2E test case for the application which we would want to have in a single Geb Spec. The framework is built using Geb & Spock for Web Application automation. Please share your thoughts / logics on how can we build the recovery scenarios for this case. Thanks for your support.!
As of now i am not able to find out any solution for this kind of problem.
Below are few things which can be done to achieve the same, but before we talk about the solutions, we should also talk about the issues which it will create. You are running E2E test cases and if they fail at step 10 then they should be started from scratch not from step 10 because you can miss important integration defects which are occurring when you perform 10th step after running first 9 steps. For e.g. if you create an account and then immediately search for hotel, you application might through error because its a newly created account, but if you retry it from a step where you are just trying to search for the hotel rooms then it might work, because of the time spent between your test failure and restarting the test, and you will not notice this issue.
Now if you must achieve this then
Create a log every time you reach a checkpoint, which can be a simple text file indicating the test case name and checkpoint number, then use retry analyzer for running the failed tests, inside the test look for the text file with the test case name, if it exists then simple skip the code to the checkpoint mentioned in the text file. It can be used in different ways, for e.g. if your e2e test if going through 3 applications then file can have the test case name and the last passed application name, if you have used page objects then you can write the last successful page object name in the text file and use that for continuing the test.
Above solution is just an idea, because I dont think there are any existing solutions for this issue.
Hope this will give you an idea about how to start working on this problem.
The possible solution to your problem is to first define the way in which you want to write your tests.
I would recommend considering one test Spec (class) as one E2E test containing multiple features.
Also, it is recommended to use opensource Spock Retry project available on GitHub, after implementing RetryOnFailure
your final code should look like:
#RetryOnFailure(times= 2) // times parameter is for retry attempts, default= 0
class MyEndtoEndTest1 extends GebReportingSpec {
def bookingRefNumber
def "First Feature block which cover the Test till a logical step"()
{
// your test steps here
bookingRefNumber = //assign your booking Ref here
}
def "Second Feature which covers a set of subsequent logical steps "()
{
//use the bookingRefNumber generated in the First Feature block
}
def "Third set of Logical Steps"()
{ // your test steps here
}
def "End of the E2E test "()
{ // Your final Test steps here
}
The passing of all the Feature blocks (methods) will signify a successful E2E test execution.
It sounds like your end to end test case is too big and too brittle. What's the reasoning behind needing it all in one script.
You've already stated you can use the booking reference to continue on at a later step if it fails, this seems like a logical place to split your tests.
Do the first bit, output the booking reference to a file. Read the booking reference for the second test and complete the journey, if it fails then a retry won't take anywhere near as long.
If you're using your tests to provide quick feedback after a build and your tests keep failing then I would look to split up the journey into smaller smoke tests, and if required run some overnight end to end tests with as many retries as you like.
The fact it keeps failing suggests your tests, environment or build is brittle.
I got a partial answer here but not exactly what I wanted.
The link describes how to get a list of task futures but what I'd really like to be able to do is list out and cancel individual jobs (that might be hung, long running etc etc). I've seen another post implying that this is not possible but I'd like to confirm (see second link)
Thanks
http://apache-ignite-users.70518.x6.nabble.com/How-can-I-obtain-a-list-of-executing-jobs-on-an-ignite-node-td8841.html
http://apache-ignite-users.70518.x6.nabble.com/Cancel-tasks-on-Ignite-compute-grid-worker-nodes-td5027.html
Yes, this is not possible and actually I'm not sure how this can be done in general case. Imagine there are 5 jobs running and you want to cancel one of them. How are you going to identify it? It seems to be very use case specific to me.
However, you can always implement your own mechanism to do this. One of the possible ways is to use ComputeTaskSession API and task attributes. E.g., set a special attribute that will act as signal for job cancellation and create attribute listener that will stop job execution accordingly.
I'm a QA who decided to use SpecFlow for my test automation after some consideration. I think it's brilliant, but missing one feature which I did use often with other test runners such as NUnit - something similar to the TestCaseSource property from NUnit to specify a potentially dynamic set of data for tests to be ran against at run time.
I would often have different data in each environment the test should run in, so cannot specify hardcoded values for test parameters. A trivial example is for checking that each type of user account is able to login, the user account credentials can be retrieved using a DB query to populate each test case dynamically in NUnit:
public List<User> GetTestData()
{
List<User> testData = new List<User>();
testData = MyDatabase.GetAllUsersInfo().ToList();
return testData;
}
[Test, TestCaseSource("GetTestData")]
public void CallLoginService(User user)
{
var response = LoginController.TryLogin(User.UserName, User.Password);
if (response.Error != null)
{
Assert.Fail("Failed to Login: {0}", response.Error);
}
Assert.AreEqual("Logged in ok", response.Message, "Login message not as expected");
}
Obviously this is a simple example of that feature, but I think it describes it well enough. I know we have the ability in SpecFlow to use a Scenario Outline and table of test run input data, but that is still static, so doesn't fit the bill.
I've been looking for a while and have not found anything in SpecFlow like this yet, does anybody know of anything similar to the above which can be used (or planned if anyone who works on the project reads this)?
Thanks :)
I have no idea if anything like this is planned but for now the problem is that there is a background code generation step when you edit your feature file via Visual Studio.
When it is saved in Visual Studio it is parsed and converted into the feature.cs file and that is the one that is compiled and used for testing.
So your process would become
edit your data source
export to feature file
get specflow's VS plugin to convert to feature.cs
run msbuild
run tests via Nunit or similar
I wouldn't do this. Instead I'd focus on getting my tests to be better examples. It sounds like you are to trying to exhaustively cover every possibility. Don't come up with examples to cover every possible case, but instead cover as much logic as possible with fewer tests.
What's the best strategy to use when writing JMeters tests against a web application where the values of certain query-string and post variables are going to change for each run.
Quick, common, example
You go to a Web Page
Enter some information into a form
Click Save
Behind the scenes, a new record is entered in the database
You want to edit the record you just entered, so you go to another web page. Behind the scenes it's passing the page a parameter with the Database ID of the row you just created
When you're running step 5 of the above test, the page parameter/Database ID is going to change each time.
The workflow/strategy I'm currently using is
Record a test using the above actions
Make a note of each place where a query string variable may change from run to run
Use a XPath or Regular Expression Extractor to pull the value out of a response and into a JMeter variable
Replace all appropriate instances of the hard-coded parameter with the above variable.
This works and can be automated to an extent. However, it can get tedious, is error prone, and fragile. Is there a better/commonly accepted way of handling this situation? (Or is this why most people just use JMeter to play back logs? (-;)
Sounds to me like your on the right track. The best that can be achieved by JMeter is to extract page variables with a regular expression or xpath post processor. However your absolutely correct in that this is not a scalable solution and becomes increasingly tricky to maintain or grow.
If you've reached is point then you may want to consider a tool which is more specialised for this sort of problem. Have a look web testing tool such as Watir, it will automatically handle changing post parameters; but you would still need to extract parameters if you need to do a database update but using Watir allows for better code reuse making the problem less painful.
We have had great success in testing similar scenarios with JMeter by storing parameters in JMeter Variables within a JDBC assertion. We then do our http get/post and use a BSF Assertion and javascript do complex validation of the response. Hope it helps