So I'm looking to bring web application testing into our .Net environment with a framework such as Selenium. At first, it'll probably be the developers writing the tests, but later it may be just the QA team. I'm wondering where the tests should actually live. Should they live in the same solution that the web application lives or should they live in a completely separate solution that is just for the tests? Please, note these are regression tests that will be done via automating a web browser so access to the web app's assemblies is not required. The answer probably is based on the environment and other factors, but I'm curious about what other people have done in this situation.
Regression Testing covers both Unit and Functional Tests. Functional tests exercise the complete program with various inputs. Unit tests exercise individual functions, subroutines, or object methods.
Unit Tests are part of the solution's code and should live with the Primary Code as with Microsoft MVC. Since Functional Tests examine the whole system and not just components, they can live anywhere. However, since your Functional Test are automated scripts, they should be included inside the solution.
The advantage to having both Functional and Unit tests live with the code is the issue of project management. Having all project related files in one repository links code version with test version. Testing scripts need to be stored in a repository (version control system) just like any other project code, so it is good to keep them with the solution.
That way the test team can do white box testing (testing with access to code) by checking-out the solution just like a developer. Their work can be saved, shared, and documented inside Visual Studio. Microsoft even includes some web based management tools with Team Foundation Server that can be used for managing the testing with open communication between test team and developers.
Related
We are building a shop for a customer on Shopware 6.3.5.2 and want to use tests to
ensure that core functionality is not broken by our customizations (static plugins)
write new tests for new functionality
There is Running End-to-End Tests but this seems to be for core development and uses psh.phar which is not available in the production template.
How should this be done?
edit
This question is meant a bit broader and concerns also Unit Tests.
Actually, you can use the E2E tests of the platform project - as Cypress itself doesn't care where to run the test against. However, as you already noticed you cannot use psh commands to run them. You may run the tests though the basic Cypress commands, setting your shop's url as baseUrl of the tests, for example via this command:
./node_modules/.bin/cypress run --config baseUrl="<your-url>"
It works with cypress open as well.
The only thing what may become troublesome is the setToInitialState command in most of the tests which takes care about the clean up of shopware's database using psh scripts, unfortunately. You may need to adjust it by overriding the command in order to reset the database of the Production template.
I hope I was able to help a bit. 🙏
There are actually two parts here:
ensure that core functionality is not broken by our customizations (static plugins)
write new tests for new functionality
re 1: For regression tests like this I would suggest end-to-end tests. Either test through the UI with tools like selenium or through the HTTP API (I don't know if the shopware API is sufficient for extensive regression tests).
re 2: Since plugins do not run on their own I would extract all relevant functionality into plain old PHP classes that are independent of shopware and test those in isolation. Explore if some of that functionality can be made visible through an API and test the plugin integration through this. Depending on the actual plugin you might have to resort to UI tests again.
I need a automation tool for my web application that develop in unity webplayer. I am a beginner in Unity development, a complete step-by-step tutorial would be awesome.
You can use Integration Test Framework in the Unity Test Tools
Integration Tests allow you to automate the verification process of your assets directly in a scene. They are designed to be used on existing content, directly within the Editor, to build tests which verify the behaviour of single assets or the interaction between them.
How to use the Integration Test Framework you can read here https://bitbucket.org/Unity-Technologies/unitytesttools/wiki/IntegrationTestsRunner
I have a hard time figuring out how you actually do integration tests
Is it still automated test with JUnit, NUnit or whatever, or is it just using the program (it has a gui) an making sure that everything works?
You can refer what-kinds-of-tests-are-there and whats-the-difference-between-unit-functional-acceptance-and-integration-test
GUI testing can be listed in system testing, acceptance testing.
GUI test tool mostly depends on type of UI (Web/Desktop/Mobile). e.g. for web testing you can evaluate good open source tools like selenium.
Our department uses Visual Studio 2008 Team System, and we have a build server that integrates with our TFS source control server. It pulls the source code, builds the solution and runs the unit tests, just as we might do from within VS, and emails a report. The build server is setup using MSBuild and MSTest as the primary tools. All very sweet.
On our development machines we also run a set of selenium unit tests, and I want to include this in the test suite on the build server. I have been told that 'this is not possible using MSBuild/MSTest', but I am at a loss to understand why.
Does any one have experience of running selenium tests (they are just conventional test methods written in C#) who might be able to advise me on whether this is possible and what the gotchas are? Thinking about it, apart from giving the browser access to the desktop when the server is not logged in, once MSBuild has handed off a test list to MSTest it's exactly the same process as on our develop machines.
TIA
I know it's 3 years on, but someone might drop in on this post and not see an answer. This is possible todo.
In a similar fashion how you'd run unit test, in the build definition using default template you need to specify the name of the UnitTest DLL and ensure Run Unit Test is not disabled. Also ensure that the build is building your Automated UI Test solution.
Simples.
I am looking at some options for using a web automated framework. I have so far looked at Selenium and Web Test (part of Visual Studio), Web Test does not test any client side code, so that kind of rules that one out. Selenium has most the features needed although I need to generate the .NET code for the tests, so that I can run the tests under TFS build server.
Is selenium the best automated testing web framework, or is there anything else worth looking at that could integrate with .NET and the TFS build. Ideally I want minimal code changes if any.
Appreciate your thoughts
Cheers
I've been constantly frustrated with Web Test in Visual Studio, though I'm not sure I understand what you mean when you say it can't test client side code.
For developer community support, ease of use, and cross platform access, I'd say Selenium is the hands down winner for programmatic browser automation. Selenium 2 works on every major OS/browser and even mobile.
I've tried WaitN and found some bugs in recent version of IE. There was a recent 2.0 release that has probably worked things out, but there are so many more people using Selenium it's worth relying on that community.
Selenium can generate C#. You can use the Selenium libraries in a C# class to create a web UI test library. Selenium is probably the most common tool.
Another option is WatiN, which is a descendant of Watir for .NET. It's another web UI automation library
This StackOverflow thread discusses using the two to test ASP .NET webforms, and might give you a good set of concepts to start from. My experience with the two is that they were both nice. I would go with Selenium if working with non-programming testers due to its great record-and-playback tools (for Firefox), and WatiN if working with SDETs / developers due to its richer libraries. When in doubt, Selenium is more common and more frequently used.
I've been using Selenium 2.0 (the C# bindings in particular) / NUnit / Hudson. Works well, and Selenium 2.0 is constantly improving and working out the remaining bugs