I am looking into Espresso Framework to adopt the best practices for writing tests for a multi modular Android projects (MVI/MVVM).
Is it a good idea to cover/write all the Espresso tests inside the app module (or) it needs to be isolated specific to each modules?
Is there any other best practices for MVI/MVVM based Android projects? if so, please comment and provide references if any
Language used: Kotlin
I have recently been implementing espresso tests in the multi-modular project where I work. Through some research and help from senior developers, I think I will be able to guide you a bit in the proper direction.
For the first question, Yes, it's essential that all the tests related to a particular activity remain in their corresponding 'test' class. That keeps it modular and easy to debug. So, one should make a test class for a particular activity and that should be in the corresponding module that the 'activity' belongs.
All these test classes can then help you to build a 'TestSuite' which could be used to make a flow for testing the app's specific features. In a test suite, one needs to mention those classes that make that flow.
After all this research and as per knowledge, I guess, this approach is good enough for large projects as well since the app I am working on is really big.
Blocker :
In a multi-modular project, I am still not able to create the test suite since I am not able to import those test classes in the main app module. But this issue will be fixed very soon. I'll definitely update it here when I'll get the method to do so.
Please do comment, if I am doing something wrong. Suggestions are a warm welcome.
Related
We are three testers and going to prepare automation project with selenium and java code so what are the steps for environment setup , scripts integration and running the testcases and getting the results for the whole project suits
So there are a few things we have to use in order to allow multiple engineers to work on the same framework.
Step 1) Creating the framework, assuming you know how to do this already, you have working tests you can skip this stage, however if not please follow the tutorial i link below.
http://toolsqa.com/selenium-webdriver/
Step 2) Creating a REPO, my preference is GitHub, you can use any git repo however i will post the guide to set one up with GitHub, its a similar process for all. This will allow you to merge code properly without causing conflicts.
https://help.github.com/articles/create-a-repo/
Step 3) Source Control program - to push, pull and fetch from your GitHub Repo, you can do this from Command Prompt however i find cloning the repo into a program like 'SourceTree' is really easy, so i've posted that below.
https://confluence.atlassian.com/get-started-with-sourcetree
If you follow these 3 guides, you will be able to have your automation test scripts on GitHub by the end of the day.
If you have any more questions please do not hesitate to ask.
All the best, Jack
The easiest and most logical way to do this would be to create one branch in your CVS (git or SVN, etc) and have each person setup the dev environment in the same way. Work exactly like developers and pull code before you check-in/commit (this will ensure that one small error does not break your framework) and swear to resolve conflicts during merge (to ensure you don't step on each others' toes).
Also, before you kick off, agree on a standard of coding (including package naming, design pattern usage, filename and methodname usage) and if this is in sync with the dev coding standards in your company, even better.
There will be a few hiccups along the way, but experience is the best way to create a process for your development and check-in practices.
Good luck with your new project and happy coding!
You have asked two questions, in my opinion the answer of your questions is.
how multiple automation testers work in same selenium project - You can use any version control system, Git Hub is the best option which gives you a lot of facilities. You all three can work on same project at same time or you can go for any centralized version control system like tortoise svn which is not much likely used now a days. I will suggest Git Hub for that.
what are the steps for environment setup , scripts integration and running the test cases and getting the results for the whole project suits - It depends on various factors like application and the kind of framework you want to use, there are many frameworks which are widely used for automation testing like Modular Framework, Data Driven, Keyword Driven, BDD, Cucumber, TestNg etc or if you have bandwidth and time you can design your custom framework as per the needs.
I hope I put some glimpse on your queries.
Thanks
I think the best practice would be to maintain the test suite in the same repo as the source code to keep the tests in sync with the code changes. But what if the infrastructure or the coding policy doesn't allow adding irrelevant files to source code? Is there a better way to keep sync between both code and tests by having a separate repo for testsuite? Thanks in advance
I think it depends on your goal/team and project. I have worked in both of the models and I found advantages and disadvantages of working with both.
Automation in the same repository:
Advantages:
You can share the code and element locators(for example with Espresso), so it is easy to maintain
It is easy for the developers to help with the maintenance (in case they want/have to)
It is more visible for developers to do the code review and check the PRs
Shared knowlodged about the code and tests between Devs and QAs
Devs can accidentally break automation code (but if qas are doing reviews this should be rare)
Test automation will have the same language as the development code
Disadvantages:
You can share code, so if you have a function with a bug and you are using the same function on your tests, your tests are going to have a bug as well
QAs can accidentally break development code (but if devs are doing reviews this should be rare)
E2E tests between projects will not make sense since the tests are placed in a repo of one project and having integration with others
E2E tests between projects will need to have mocks to test scenarios on the other products/projects otherwise, as said above, it won't make sense to have the test project in a project repo
Can't share code/functions between projects as it will be confusing to have a test project sharing functions with other test projects in different repos (Unless you create a test repo with these shared functions) + you may have a test automation coded in javascript for the web project and for the mobile project you are going to have the same code as the development team, which could be different from the web, like kotlin or swift
Automation in separated repository:
Advantages:
You can share the code between test projects
You will reduce the maintenance cost as the project can be shared between different platform projects
Test automation can have the language that is more known for the team who is going to code or the language which has more advantages when maintaining the code between projects in different platforms
Disadvantages:
Not really visible for the developers as they might not follow
You can proper create an E2E tests involving all the projects in different platforms without mocking them
It is not a big motivation for the developers to follow and maintain the test automation
Anyway, I may be missing something, but I just tried to remember all the key points. In the end the team should decide this together as this again depends if the developers are going to maintain the tests as well and if you are going to perform e2e tests with or without the need of mocking the other projects
In my current Test project, we are using TestNG as a testing framework. We have the Test suites in a separate folder structure, but still they are part of the project.
what if the infrastructure or the coding policy doesn't allow adding irrelevant files to source code
Also are organized in test_suite.xml files (represented by one XML file, as suite is the feature of execution) for different scenarios, since by default they cannot be defined in the testing source code.
The main advantage of this is the flexible configuration of the tests to be run. Also they can be supported by a tester with very little domain knowledge of the Test project.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 8 years ago.
Improve this question
Guys I am looking for Web Testing Generic Automation Framework which can be used to do automation testing of various web based applications .Looking for C# based framework as that is the language I am more familiar with. But any other language framework will also do and it should not use any proprietary/licensed language.
Framework should have some open source and free of cost license model.
I searched for selenium based framework on Google and SO. But could not come with any which have source code available. It will be good if the framework encapsulates all the functionality provided by Selenium WebDriver and/or Selenium RC and empower the functional tester to create and maintain test in human readable scripts.
Requirements of the framework:
The framework code should avoid hard coding of test steps. My idea is to maintain the test scripts outside the automation framework code , so that they can be easily be modified if needed. The framework should read through the step tables and the data tables and run the test accordingly.
If there is no such framework available now right then we can collectively build such a framework in a open source community model.
P.S.
I have read a little about Hermes Framework and Robot Framework, but not yet tried them, any help is welcome.
The good side of this problem: there are a lot of flexible tools and approaches, you can get together and build a flexible, reliable and robust test automation framework.
The hard part is: yes, there is no “out of box” solution, and you’ll need to find and put together lots of tools in order to solve this test automation puzzle.
What I would recommend:
First you need to choose a unit-test test framework. This is a tool which helps to identify separate methods in code as tests, so you can run them together or separately and get the run results, such as pass or fail.
My personal opinion, is that the testing tool – MS-Test – which ships with Visual Studio 2013 (and also Express Edition) is good enough. Another alternatives are: NUnit or Gallio Icarus
All unit-testing frameworks includes a mechanism for doing assertions inside the test. The capability of assertions class depends on given unit-testing framework. Here, I would like to recommend a popular library which works great for the entire unit testing framework.
This is Fluent Assertions (also available from NuGet repository).
That’s a hard moment. You need to decide: are you going to use the PageObject approach in order to build your test automation framework, or you are going to choose simpler approach, without heavy utilization of the Object Oriented Programming.
Properly designed Page Objects makes your test automation code much maintainable. Utilizing the OOP – you can do a magic in your code: write less to do more. Although, such approach requires more skill.
Here are a good articles on this topic:
Maintainable Automated UI Tests
And this one:
Tips to Avoid Brittle UI Tests
The alternative to the PageObject is a scripted approach. This approach can be also successful and requires less time to start.
Coypu is a good and usable example of such framework for Selenium Web Driver.
All the popular unit-testing frameworks support data-driven tests. The best support is in NUnit – you can run/re-run and see the tests generated for individual data row in the tests tree.
MS-Test supports reading data from different data-sources: text files, excel, mssql etc., but it is not possible to re-run the test for individual data row. Although, there is a hack for this – Ms-Test Rows.
For my data-driven tests, I am using a great library – Linq to Excel
I have a lot more to say. There are so many approaches to build test automation framework – and there is no ready solution yet.
I am trying to build one according to my testing methodology – SWD.Starter .
This project is still on its early development stages. But, at least, probably you’ll find a few tips how to build and organize the test automation code.
I've implemented https://github.com/leblancmeneses/RobustHaven.IntegrationTests based on my prior experience on large projects "trying" to implement full end to end testing.
I've been using this and and have a lot of useful extensions for general selenium, angularjs, and kendo ui work. Since this framework is not obtrusive you could just use these extensions without using anything else.
I'm using this on my latest project and everyone is loving it.
There are a lot of bdd/spec frameworks (specflow, mspec, nspec, storyq) to help wire the behavior of your system to tests.
What I've learned:
make it frictionless for any .net developer/tester to begin writing/running tests.
Most fail here because it requires installing additional pluggins into visual studio.
mine uses the standard nunit
Logically you would think that a feature is a class file and scenarios are [Test] methods - to support some of these frameworks they make each scenario a class file.
use the original spec to create stubs of your tests - hopefully readable code
I used spec flow back in 2010 - so things might have changed. I generated my tests from my bdd document. A year later when I went to add more tests and update existing tests, I felt I wasted a lot of time with ceremony than writing code I really wanted - I stopped using it.
My approach uses t4 to generate stubs - developer has a choice to generate from feature file, for a specific scenario or don't use generated code at all.
how is state shared across steps / nested steps
most use dictionary<string,object> to help you separate data from being hardcoded in your tests accessed from a context object.
mine uses viewmodels and pointers to those viewmodels - if your using something like angularjs you are using viewmodels in your server side display/editor templates and in angularjs controller so why not reuse these in your tests!
start early with CI - make development transparent
My project has ResultDiff that given the nunit testresult.xml file, folder location to your gherkin feature files, and output json file; Read description on why this is important on the screenshot: https://github.com/leblancmeneses/RobustHaven.IntegrationTests#step-5-ci-setup-resultdiff
Example:
Modified means business and developers have a mismatch of Gherkin statements - did something change that we need to talk about?
What is missing? a dashboard to render the .json file created by ResultDiff. It's on my backlog.....
With a centralized dashboard that supports multiple environments(branches of your code) this dashboard will serve all stakeholders (business, developers) what is the status of features being developed.
There is a framework named "omelet" which is built in java on top of testng for selenium,
For cross browser multi-parallel testing , it easily blends with your CI tools and have some cool reporting features with step level reports
Running your test cases on BrowserStack and Grid was never so easy as with omelet with few config changes.
if you want to give it a try then do follow the 5 min tutorial available on the website, there is archetype available on maven central + there are many more features available
Stable version is 1.0.4 and we are currently looking for people to contribute to project.
Documentation over here
Github link
I am working on my first MVVMCross application and I am trying to set up a good testing framework. I have looked at the testing Stuart is doing in his TwitterSearch app but I have found nothing that explains his approach or any other approach to testing MVVMCross. Has anyone come across a good post/tutorial on the proper way to test an MVVMCross application? Other than just code that a newb (like myself) may not completely understand...
I'm not sure what you are asking...
What do you mean by a 'proper way'?
Is this a question about mechanics of "how to write a unit test?" Or a question about "how many unit tests to write; Which components to test; How deep to go; etc?"
For the mechanics:
I personally use NUnit for testing (from NuGet)
I include this in a .net4.5 class library project.
I use Moq for most of my Mocking (from NuGet)
There are a few MvvmCross objects I manually mock - as shown in that TwitterSearch message
There's an MvvmCross base test class which provides IoC/ServiceLocation - but I generally only use this when I need to use real MvvmCross classes - e.g. when the class under test inherits from MvxViewModel
I only run tests within Resharper in Visual Studio
There are plenty of other approaches, including some people choose to run tests on devices - e.g. using the excellent MonoTouch Nunit test runner.
There are also plenty of people interested in BDD testing - e.g. things like Frank, Calabash (coming in Xamarin Test Cloud) and the Windows Phone Test Framework that I wrote - https://github.com/Expensify/WindowsPhoneTestFramework :)
For the philosophy, I have no strong opinion, but I like this answer:
https://stackoverflow.com/a/153565/373321
I too get paid to write code
I am in a new project that is looking to include an end to end testing framework.
We want something flexible, I've used Fitnesse before and I think we need something similar to it.
We are also using Hudson CI and are looking for something that would integrate easily with it.
Is there a clear winner?
UPDATED: The system has many components, some of them are web services running on tomcat, there are a couple of NoSQL databases too, but no UI testing is required for the moment.
Please add a comment if further clarification on the project details is needed.
The robot framework is a good keyword driven testing framework that we use for end to end integration testing.
http://code.google.com/p/robotframework/
There is a plugin for jenkins (a fork of hudson) that is very useful. It reportedly works with hudson as well.
Great to read you have decided start using Robot Framework. Its Hudson plugin already has recently got much better (trend charts, available from Hudson directly, ...) and moved to new place:
http://wiki.hudson-ci.org/display/HUDSON/Robot+Framework+Plugin
Can you be more specific, when you say, you want to have an end-to-end testing framework? What kind of application are you developing (standalone, web based, web service, ...)? What language do you program in?
I would also look for previous knowledge in your team or around your team (e.g. other teams). It might be sufficient, to use TestNG or JUnit (I have seen an old paper on UI testing using JUnit). Almost all test frameworks should be fine for Hudson, as long as they can be started in a headless/scripted mode and if they provide the results in a format that Hudson understands (in the worst case HTML is sufficent, though Hudson will only be able to display the pages). So if you have IBM Monitoring tools available, you might want to look at Rational Functional Tester or Rational Performance Tester.
FitNesse is a good tool for functional testing. The business contacts can create the test cases by themselfs (developers have to provide some connector code), which will create more visibility and removes some work from the test engineers. Drawback is, that you can't really do UI testing. If you need a open source plattform for UI, have a look at seam. Even though I like the OSS philosophi, you might need longer ramp up time (higher costs) and possibly more maintainace time for OSS (not true for all OSS). In any case, check how easy it is to get support for your framework. You might even consider paid support for your framework (regardless if open or closed source). Also don't forget, even though you can fix OSS source yourself, you might not have the expertise or time for fixing bugs or adding features.
So give us more info and we can give you better advices.
My experience with jBehave is very good. Recommend.
It's based on BDD - Behavior Driven Development.