I am very new to Autosar and looking at what needs to be done to test a SUT/DUT (System/Device under test) confirm to Autosar specifications
I read the Autosar conformance test specification (Part 1 to 4)
and they have a separate link on website that talks of Acceptance test
In fact there is no direct link on portal for conformance testing. Only doing a Google search lead to those documents hosted by Autosar.org. So i presume conformance testing is also relevant to device testers
Are Autosar Conformance testing and Acceptance the same ?
When is Conformance test done and when Acceptance test are done ?
Thanks
Related
I am developing WebUI automation tests using Karate 0.9.5.RC5 and it is working wonderfully. Does anyone know how to execute performance testing in Karate for WebUI Automation tests?
That's great to hear and thanks for the feedback. To be honest, we have focused on API perf testing and UI functional test automation so far. Maybe you can help us by experimenting and sharing what you find.
You must be aware of the Gatling integration for API performance testing. So we have some pieces of the puzzle in place.
So maybe a hybrid strategy is best:
identify the API calls being made by the UI, in the future we would like to derive them automatically from the Chrome network / devtools
manually convert the API calls to Karate tests, note that the VS Code plugin has an option to import from cURL
now you can convert the Karate tests to a performance test, and for most teams, this is sufficient
if needed, you can add some Karate calls to load HTML and static resources to make the load profile more realistic
finally, it may be possible to run a Karate UI test in parallel, just to measure "real user" experience and measure the HTML / JS load times etc on the browser side. we don't have this in place yet, but it should be possible to get all the page timings and perf-stats from Chrome
potentially you could look at spinning up multiple Chrome instances in parallel using Docker - but again, this is something yet to be explored
I am developing WebUI automation tests using Karate 0.9.5.RC5 and it is working wonderfully. Does anyone know how to execute performance testing in Karate for WebUI Automation tests?
That's great to hear and thanks for the feedback. To be honest, we have focused on API perf testing and UI functional test automation so far. Maybe you can help us by experimenting and sharing what you find.
You must be aware of the Gatling integration for API performance testing. So we have some pieces of the puzzle in place.
So maybe a hybrid strategy is best:
identify the API calls being made by the UI, in the future we would like to derive them automatically from the Chrome network / devtools
manually convert the API calls to Karate tests, note that the VS Code plugin has an option to import from cURL
now you can convert the Karate tests to a performance test, and for most teams, this is sufficient
if needed, you can add some Karate calls to load HTML and static resources to make the load profile more realistic
finally, it may be possible to run a Karate UI test in parallel, just to measure "real user" experience and measure the HTML / JS load times etc on the browser side. we don't have this in place yet, but it should be possible to get all the page timings and perf-stats from Chrome
potentially you could look at spinning up multiple Chrome instances in parallel using Docker - but again, this is something yet to be explored
Whe are making a proposal to a client for a test automation framework and we are between MTM and Selenium. For what they need (CI process, and manual launch of testing bateryes) is better Selenium.
But in our last meeting the client's head of develop told us that they like the MTM's interface Test Center, because they could unify the automated testing and the manual testing in one interface. So I'm looking for some framework that would provide a similar interface, work with Selenium automated tests and, If possible, interact with their TFS.
Any suggestions?
Thank you a lot.
What's the pro's and cons for both and why should I use either of them for functional testing?
Regards,
Jonas
Selenium is for integration testing, It will test how your web application behaves in an actual browser. This can find things that HTTPUnit cannot, e.g. browser compatibility of css and JavaScript.
HTTPUnit tests web applications by directly calling the web service and manipulating the response. This is a functional test framework as it tests what your web app does, not how it behaves across different platforms.
One advantage of HTTPUnit is that it is much quicker than Selenium. Personally I would (and do) use both. HTTPUnit for complete functional testing and Selenium for selective (not complete) integration testing to check browser compatibility.
HttpUnit is very simple, extremely easy to use, and requires very little to get up and running. It's a good place to start for simple tests, however it will require more coding to create complex tests.
Selenium is more than just a library, like HttpUnit. It equips you better for more advanced, stateful tests.
Selenium will use your browser to automate tests
- Selenium http://seleniumhq.org/
httpunit goes from the html specification and simulates a standards conformant browser
- httpunit http://httpunit.sourceforge.net/
Selenium is more specific and will let you use browser specific behaviour and not limit the application under test.
With httpunit the limits are set by what httpunit supports which is far less than most browser do. E.g. the javascript capabilities are reduced to a very small set. On the other hand if your app passes httpunit tests it will probably run on quite a few browsers and environments.
So if you are interested in supporting a mininimal common set of features httpunit might be for you.
I'm in the middle of picking tools to load test my Ruby on Rails app. So far I'm trying out -
apachebench
autobench
httperf
selenium
trample
Is there anything else worth looking at? I don't have a ton of hardware, so efficiency is a concern.
The famous one (at least for me):
JMeter
The Grinder
OpenSTA
All support simulating concurrent users, can generate decent load, support distributed testing if required (with distributed agent). JMeter and OpenSTA have a recorder and recorded scripts are relatively easy to variablelize. For The Grinder, I'm not sure.
OpenSTA is the most polished one and with the most features (but is not portable).
JMeter is my preferred one mostly because I know it well and because testing can be easily automated (e.g. to be included in a build). Have a look at the user manual to get started. If you need to record over SSL, check BadBoy.
More interesting reading at Shootout: Load Runner vs The Grinder vs Apache JMeter.
Check out JMeter.