I come from a configuration management background using tools like chef. I have done quite a bit of code testing. Recently, I have been entrusted with the responsibility to work on the CI-CD of the various applications. But I am noticing a culture that’s quite not in line with my ideology.
Let’s say we have 4 environments in the CI-CD pipeline: Dev, Test, Stage & Prod. The dev environment is used by the developers to deploy and test the app before rolling out to the next stage (test). The test env is for the QA team to run their test. And STAGE is another env for 2nd layer of test by the QA folks before the code goes to prod.
Now, does it make sense for the QA or the CI process to run unit test while progressing/after deploying the code to (TEST or STAGE)? I agree that the unit test should be a part of the automated build. But unit test is mainly for the developers, for their code testing. The QA should be focus on the functional testing, load testing etc.. using their frameworks which may be selenium/protractor or LISA. Why should they be focusing on junit or nunits?
From my experience the developers do the unit testing.
Where I currently work, we follow TDD (Test Driven Development) so developers write the unit tests for their code first and then they write the actual implementation. This work goes on the Dev environment.
The QA people then do the acceptance tests and review stuff.
I hope this helps.
Unit tests are short code fragments created by programmers or occasionally by white box testers during the development process. Unit tests are typically written and run by software developers to ensure that code meets its design and behaves as intended.
Hope you got the answer.
Your concern is absolutely right. QA does not necessarily be doing Unit Testing (If that's what you mean). This testing is done by dev while they were writing the code (if they were in TDD environment or even if in BDD).
However if QA has used Junit to write their Selenium automated test cases then they will be using Junit to update and change their automated test cases.
It's little difficult to automate Selenium test cases using Junit whereas using other framework like TestNG is the best option to write automation test cases for Selenium.
I hope this answer your question.
Related
We are building a shop for a customer on Shopware 6.3.5.2 and want to use tests to
ensure that core functionality is not broken by our customizations (static plugins)
write new tests for new functionality
There is Running End-to-End Tests but this seems to be for core development and uses psh.phar which is not available in the production template.
How should this be done?
edit
This question is meant a bit broader and concerns also Unit Tests.
Actually, you can use the E2E tests of the platform project - as Cypress itself doesn't care where to run the test against. However, as you already noticed you cannot use psh commands to run them. You may run the tests though the basic Cypress commands, setting your shop's url as baseUrl of the tests, for example via this command:
./node_modules/.bin/cypress run --config baseUrl="<your-url>"
It works with cypress open as well.
The only thing what may become troublesome is the setToInitialState command in most of the tests which takes care about the clean up of shopware's database using psh scripts, unfortunately. You may need to adjust it by overriding the command in order to reset the database of the Production template.
I hope I was able to help a bit. 🙏
There are actually two parts here:
ensure that core functionality is not broken by our customizations (static plugins)
write new tests for new functionality
re 1: For regression tests like this I would suggest end-to-end tests. Either test through the UI with tools like selenium or through the HTTP API (I don't know if the shopware API is sufficient for extensive regression tests).
re 2: Since plugins do not run on their own I would extract all relevant functionality into plain old PHP classes that are independent of shopware and test those in isolation. Explore if some of that functionality can be made visible through an API and test the plugin integration through this. Depending on the actual plugin you might have to resort to UI tests again.
I have just started learning about test automation in Selenium and found out that most online tutorials would tell you to run the test suite inside an IDE together with a test framework such as TestNG (with testng.xml) and a build tool such as Maven.
When you are working in a software company and told to build a test framework and run automated tests, I don't believe you actually need to fire up your IDE every time you want to execute your test suite. So, my question is, what is the typical setup a software company follows to 'automate' running your test automation scripts?
Software companies are following agile practices and wanna keep up with industry practices. In real projects, CI & CD are used to continuously integrate, deploy and test the software.
Tests are written by SDET using test automation frameworks. While developing test scripts test developers use IDEs like eclipse. However, tests are executed over Jenkins as a job, after required frequency/event.
For example, after every code deployment, Jenkins can automatically trigger your sanity suite, and run regression bi-weekly.
The process' are automated now-a-days with stakeholders demanding agility.
One can invoke selenium java project from command line via .bat file in Jenkins, or using ant/maven as build tools.
IDEs are seldom used to run tests in real world.
I have a hard time figuring out how you actually do integration tests
Is it still automated test with JUnit, NUnit or whatever, or is it just using the program (it has a gui) an making sure that everything works?
You can refer what-kinds-of-tests-are-there and whats-the-difference-between-unit-functional-acceptance-and-integration-test
GUI testing can be listed in system testing, acceptance testing.
GUI test tool mostly depends on type of UI (Web/Desktop/Mobile). e.g. for web testing you can evaluate good open source tools like selenium.
I know they both are for testing but I did not find much information about PhpRack here. Can we use them together in a project? Or only one can serve the purpose?
PHPUnit is a unit testing framework. Unit testing considers the testing of software from the perspective of discreet units of code, such as a class or method.
PHPRack is an integration testing framework. Integration testing considers the testing of how separate software modules/systems integrate together in a production environment.
Unit testing and integration testing are not the same thing. A development environment can use both.
So I'm looking to bring web application testing into our .Net environment with a framework such as Selenium. At first, it'll probably be the developers writing the tests, but later it may be just the QA team. I'm wondering where the tests should actually live. Should they live in the same solution that the web application lives or should they live in a completely separate solution that is just for the tests? Please, note these are regression tests that will be done via automating a web browser so access to the web app's assemblies is not required. The answer probably is based on the environment and other factors, but I'm curious about what other people have done in this situation.
Regression Testing covers both Unit and Functional Tests. Functional tests exercise the complete program with various inputs. Unit tests exercise individual functions, subroutines, or object methods.
Unit Tests are part of the solution's code and should live with the Primary Code as with Microsoft MVC. Since Functional Tests examine the whole system and not just components, they can live anywhere. However, since your Functional Test are automated scripts, they should be included inside the solution.
The advantage to having both Functional and Unit tests live with the code is the issue of project management. Having all project related files in one repository links code version with test version. Testing scripts need to be stored in a repository (version control system) just like any other project code, so it is good to keep them with the solution.
That way the test team can do white box testing (testing with access to code) by checking-out the solution just like a developer. Their work can be saved, shared, and documented inside Visual Studio. Microsoft even includes some web based management tools with Team Foundation Server that can be used for managing the testing with open communication between test team and developers.