Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 4 years ago.
Improve this question
We have an automated regression test setup for Functional tests and
are interested in measuring the test coverage/code coverage for our
project which is based on Linux.
I would like to know if there are any tools which can be used for test
coverage/code coverage measurement without instrumenting the code.
Please suggest any tool or method that can do this.
I am aware that instrumentation provides the best method to measure
the code coverage, but it would suffice for us if the tool can just
measure the functions that were executed for the test coverage
measurement.
Thanks and Regards,
Prashnk
There's only three ways to collect test coverage data:
Instrument the program, and let the instrumentation do to coverage collection
Interrupt the program (assuming native execution) on a periodic basic to determine its PC,
and map the PC to source line numbers
Run the program under an interpreter, and let the interpreter do it (or let the interpreter make calls on an special user-code event handler when certain key program events occur, such as "function entry").
It may be in fact, that an interpreter doesn't have that ability built-in, in which
case you are reduced to the first solution :-{ If you are running an interpreter, any test coverage data collection it may have built-in should be pretty well advertised in its documentation.
You can likely implement the "interrupt to get PC" method yourself, using symbol data from the link map or the object file (or maybe someone has done it already for you.) It has a serious downside: because you are sampling the PC, it is hard to verify that some function actually got executed. It may have, and you simply didn't sample the PC at the right moment. This is the reason this scheme isn't used in practical test coverage tools.
Related
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 3 years ago.
Improve this question
How to decide which type of testing(Manual or automation) required for a project or application to test?
What are the parameters we have to consider to select which type of testing(Manual or automation) to test very new application?
It depends on :-
Size of the project- If the project is large and consist of so many functionalities then automation testing is suggested
How many times you want to test a particular feature- If the requirement is to test regularly then automation test is best
Font size and image- This can not be tested through any automation tool so to test this, one should need to do manual testing
To find bugs- If one needs to find a lot of bugs, Manual testing is suggested.
You shouldn't have to choose between automation testing and manual testing the way you're asking. The way you're asking it gives me the feeling that the product is already waiting to be tested. In this case you would need to resort to manual testing.
Ideally you would want to have both and even more of automation. Some of the questions that you need to ask are:
Is this a new project or an existing one? If it's a new project then it's easier to plan for automation from the start. You could start implementing automation tests from the start. If it's an existing project then you'll need time to set up automation + write scripts etc. Then you have to resort to manual testing initially.
Is there any existing team? If yes, then what are they doing. You need to continue the process instead of suddenly disrupting it for anyone.
How much resources (money+people) do you have? Do you have manual testing resources? Are they busy or do they have bandwidth? How many automation test resources do you have?
What kind of project is it? Who does it go to? Does it have human lives depending upon it? Does it need a legal certificate of some kind for being tested?
There's just too many questions based on how your question is currently stated. I hope that this answers your question when we consider it generally. But if you're looking for a particular answer then please consider adding more context.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 2 years ago.
Improve this question
I am using TestNG and Selenium webdriver via Java.
Is there any tool that can help generate detailed test results, for example, suppose I have a test case that fails more often than not, is there a tool that can statistically report those test cases that fail more often than the others like in a graph, or pie chart, etc?
XL Testview
Have a look at XL Testview from XebiaLabs.
Test analytics and decision support that spans testing tools
See all your test results in one single dashboard
Analyze test results across multiple test tools
Track release metrics and quality trends over time
Use real-time quality data to make the best go/no-go release decisions
I havent used it, but seems to track results over time. Seems pretty interesting.
Test Result Analyzer
Or have a look a the Test Result Analyzer plugin for Jenkins.
Many of us have a requirement of knowing the execution status of a
test package , test class or test-method across multiple builds. This
plugin is an implementation of the said requirement and shows a table
containing the executions status of a package,class or a test-method
across builds.
This plugin supports jUnit and TestNG results sets. Looks like the minimum you want and it is free. :)
Tesults - it handles all this including identifying recurring / continuing fails. In general it's a great central place for a team to view and assign failing tests. Please be aware that I work at Tesults.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 7 years ago.
Improve this question
Can someone please recommend a tool for analyzing, improving, finding "dead code", provide statistics etc. for a source code on COBOL language on VMS/VAX OS and RDB Database?
Thanks.
Some compilers have options for locating dead code, so you may already have the tools you desire. Please keep in mind that there are going to be situations where the code is dead and you cannot tell via static source code analysis.
if a = 1
move 'error' to out-message
end-if
If a can never be 1 then this is dead code. Static source code analysis may not find more complicated instances of this scenario, particularly if the value of a comes from outside the program being analyzed - perhaps a database.
A cursory scan of the static source code analysis tools listed on Wikipedia shows the commercial products are pricy (thousands of euros). The open source tools don't appear to provide much COBOL coverage. You might want to check those out for yourself as I admit I didn't do a thorough evaluation but instead just scanned their documentation.
COBOL is a difficult language to parse.
I have never used it, but the DecSet suite of products includes a product called PCA - Performance and Coverage Analyser. This may be what you are looking for.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 4 years ago.
Improve this question
I wanted to automate the execution of Doxygen whenever I insert my source code to my code repository. Is there any api/interface for doxygen that I can use in my java code to start its execution?
Doxygen itself is started by executing a command-line command, so you only need a way of executing a command when a check in occurs. You don't say anything about the SCC tool, Dev Environment or OS you are working with, so it's hard to give a specific answer.
The usual way to do this would be using a build server running Continuous Integration. That is, when you check in code, the build server will notice the check in, get the new code, and then execute a build (in this case of Doxygen documentation). There are many tools available that will automate a PC to make a build server. Which one to use would depend on things like the OS and Source Control package you're using. (If you don't have/want a dedicated build server you could also run a tool like this on your own PC in the background).
If you just want the build to occur on your own PC when you check in, then you could investigate your source control software to see if you can be notified of the check-in. Some SCCs may offer an ability to execute a script or command for certain events, so it may be possible to get it to execute the command for you.
Lastly, many people just set up a tool/macro within their development environment that executes Doxygen, and then they run it manually. In visual Studio (for example) you could even write a macro that would do both the Check in and execute Doxygen from a single keypress.
If you want more control with doxygen, you have to generate also xml files and create a program that analyze the xml output files to check all your constraints, and stop the build if a constaint is violated.
this technique needs some developement effort but it's the only way that i know to do what you want with doxygen.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 7 years ago.
Improve this question
I am currently doing a lot of regression testing at my job. And like I want to look at some of the best code in the world for regression testing (i.e. Open source). So basically list open source projects that you think show good examples of regression testing.
P.S True regression testing is black box by the way. However the examples so far are a good read for testing
Here is the documentation of KDE4's unit testing development:
http://techbase.kde.org/Development/Tutorials/Unittests
In this blog you can see that QtSoftware/Nokia has opened all the regression testing applications to the world: http://ariya.blogspot.com/2009/06/all-blessings-in-may.html
or direct link to the unit tests+benchmarks:
http://qt.gitorious.org/qt/qt/trees/master/tests/benchmarks
http://qt.gitorious.org/qt/qt/trees/master/tests/auto
This blog is linked from the previous one: http://benjamin-meyer.blogspot.com/2009/05/open-qt-repository-and-hidden-gems.html read it as well :)
Would you like to see where the WebKit regression tests are saved? Or you want to google for it?
jQuery does a lot of regression testing, because it's supporting quite a few browsers and a lot of intricate tests (in particular, supporting css). I'm not sure if it's the best example or not for general regression, but for web development, it's a great case because it handles a VERY difficult task, regression testing against many browsers which are constantly changing, even within versions. Though it is hard to come up with a more pure case where changes in the external world can cause your code to regress :(.
Resig talking about difficulties unit testing javascript.
Resig talking about Fireunit, a tool used in testing jQuery