Is there a way to integration test my JUnit Jupiter extension? Of course I can test a good cases of extension usage, but I would like to test things like:
Does it fail tests as expected?
Are the correct reports written on test end?
Is there some built in support for this?
There is no explicit Jupiter extension testing framework -- but with version 1.4 there'll be the Platform Test Kit that you may use to execute the Jupiter engine and your extension in one-go and assert the outcome of a test run.
For details see: https://junit.org/junit5/docs/snapshot/user-guide/#testkit and in addition to that documentation, find usages of the Platform Test Kit within the "platform-tests" project of JUnit 5. For example at: https://github.com/junit-team/junit5/blob/master/platform-tests/src/test/java/org/junit/platform/testkit/engine/ExecutionsIntegrationTests.java
You should not perform integration testing for your own extension of Junit. The Junit 5 API is very, very stable and I bet that your extension will work the same for the entire life of Junit 5. Integration testing is useful in a constantly evolving environment, but Junit only ever adds features and never changes them (unless there is a bug).
If you perform integration testing for your own Junit extension, it's like you're testing Junit itself. Why would you test Junit instead of your extension? Let Junit be tested by Junit developers, unless you think you found a bug. But testing Junit for the sake of testing Junit is a waste of your own time, even if unconsciously you think you're only testing your own extension.
So that kind of tests should be done on the long term by getting your users to report issues to you: "Hey, AwesomeExtension works with Junit 5.A.B but not with 5.X.Y", then find out why they say that and if the issue is not with your extension, then report with Junit.
However, make sure that you get your extension... unit-tested.
Related
This question already has an answer here:
Karate framework and TestNG
(1 answer)
Closed last year.
We have implemented cucumber with rest assured for API automation. Recently we came to know Karate giving advantages when compared with Rest assured. So I thought of recommend to my organization to use Karate, hence I have prepared sample to API automation scripts with Karate. Yes, I can execute very well with JUnit, but same testrunner file if I tried to execute with TestNG am unable to execute at all, displaying Test runs are zero after my execution.
Could you please help me is there any way to run Karate runner file with TestNG framwework. Really your answer going to make us decision.
I have gone through various posts which are related TestNG with Karate. But unluckily am unable to trigger my scripts.
Note: I have extended KarateRunner class and used #KarateOptions as well in my code.
TestNG is needed only for development mode. All teams finally need to do CI and here Karate does not need even JUnit, please refer this: https://github.com/intuit/karate#parallel-execution
Now for development mode - even if you insist on using TestNG, you can mix JUnit into the same project: https://stackoverflow.com/a/19928639/143475
Please give up trying to use TestNG for Karate, it is un-necessary and a waste of your time. Also note that we have IDE support which is what most developers will use. Even the JUnit support is not needed most of the time.
I come from a configuration management background using tools like chef. I have done quite a bit of code testing. Recently, I have been entrusted with the responsibility to work on the CI-CD of the various applications. But I am noticing a culture that’s quite not in line with my ideology.
Let’s say we have 4 environments in the CI-CD pipeline: Dev, Test, Stage & Prod. The dev environment is used by the developers to deploy and test the app before rolling out to the next stage (test). The test env is for the QA team to run their test. And STAGE is another env for 2nd layer of test by the QA folks before the code goes to prod.
Now, does it make sense for the QA or the CI process to run unit test while progressing/after deploying the code to (TEST or STAGE)? I agree that the unit test should be a part of the automated build. But unit test is mainly for the developers, for their code testing. The QA should be focus on the functional testing, load testing etc.. using their frameworks which may be selenium/protractor or LISA. Why should they be focusing on junit or nunits?
From my experience the developers do the unit testing.
Where I currently work, we follow TDD (Test Driven Development) so developers write the unit tests for their code first and then they write the actual implementation. This work goes on the Dev environment.
The QA people then do the acceptance tests and review stuff.
I hope this helps.
Unit tests are short code fragments created by programmers or occasionally by white box testers during the development process. Unit tests are typically written and run by software developers to ensure that code meets its design and behaves as intended.
Hope you got the answer.
Your concern is absolutely right. QA does not necessarily be doing Unit Testing (If that's what you mean). This testing is done by dev while they were writing the code (if they were in TDD environment or even if in BDD).
However if QA has used Junit to write their Selenium automated test cases then they will be using Junit to update and change their automated test cases.
It's little difficult to automate Selenium test cases using Junit whereas using other framework like TestNG is the best option to write automation test cases for Selenium.
I hope this answer your question.
I have a hard time figuring out how you actually do integration tests
Is it still automated test with JUnit, NUnit or whatever, or is it just using the program (it has a gui) an making sure that everything works?
You can refer what-kinds-of-tests-are-there and whats-the-difference-between-unit-functional-acceptance-and-integration-test
GUI testing can be listed in system testing, acceptance testing.
GUI test tool mostly depends on type of UI (Web/Desktop/Mobile). e.g. for web testing you can evaluate good open source tools like selenium.
I just moved into web testing using Selenium and have been learning Selenium WebDriver + Java.
My question is, why do I need to build a test framework when TestNG is there?
What is there that TestNG cannot do as a framework?
Nothing :-)
More seriously, TestNG is very popular with Selenium users for a variety of reasons, among which support for:
Dependencies (very convenient to test pages that follow each other).
Parameterization with #DataProvider (useful to test various combinations of browsers and user agents).
Parallel tests (lets you test on multiple browsers simultaneously).
Check out the Selenium forums for more information and feel free to email the testng-users list if you have questions.
Yes, I think TestNG is enough as a web test framework. If you are familiar of using JUnit, It is easier to switch into TestNG
Building a custom framework may be necessary at times.
And you can use TestNG for that. :)
How to do testing and is there something similar like Rack::Test with ruby frameworks?
You can use Specs, ScalaTest and JUnit (and any other java test framework), theres some examples on the new wiki.
Read this!
I don't know much about ruby, so maybe don't get your point. But what about Jitr?
Jitr (pronounced "jitter") is a JUnit
Integration Test Runner.
It allows your web application
integration tests to easily run
against a lightweight web container in
the same JVM as your tests.
You can use it with JUnit 4's #RunWith annotation. In Scala: #RunWith(classOf[Jitr]).
This may be of help, it's on Lifts wiki pages. There are examples using Junit, Scalatest and and Specs.
How To: Unit test lift snippets with a logged in user