I was trying out TestCocoon the other day, and everything seemed great. I compiled my code using cscl,cslib and cslink and I was expecting this to take care of all the instrumentation. I get some .csmes files and .exe.csmes files, but when I load them into the CoverageBrowser I cannot see anything relevant. No covered/uncovered lines. All the lines are grey.
Is anything else needed in order for TestCocoon to report coverage? Do I need to modify my source files? I also posted on their forums here, but no result:
http://www.testcocoon.org/forum/viewtopic.php?f=8&t=44
I tried this tool with few projects using Visual Studio 2008, and I found:
Pros:
- it can collect results from multiple runs, you can run your software at different machines and collect results together
- it has useful GUI for browsing results
- you can merge coverage from many modules and anlyse it as whole application
- forum works, I submited two problems and got implemented fixtures in few days
- it works almost without any problems (I found two minor compilation problems) with quite complicated sources, with tons of templates, boost::spirit parsers, other boost stuff (including meta-programming modules etc.), STL, Qt (everything together)
- well documented
- it's free
Cons:
- instrumentation is definitely slow
- multi-process single project compilation using Visual Studio 2008 doesn't work, only one file at a time is compiled which makes building slower (you will get better performance building whole solution with many projects)
At this moment I didn't try to use this tool for continuous coverage measurement.
Either way, in my opinion it's worth to try.
BTW, Tony, PC-Lint is static-analysis tool, isn't it? interesting idea to compare it with dynamic-analysis tool...
TestCocoon (now at 1.6.7) works well with the small C code bases we tend to unit test. The performance impact seems about normal for other instrumentation methods we've used.
We are able to extract coverage information in our makefiles and the coverage browser is very useful.
Dont use testcocoon, I am currently using it, and its shoddy as hell. Pay for something better (it will cost alot). It is the ultimate death sentence, seriously, don't do it. Whatever you do, stay away from testcocoon at all costs. Worst move ever. You might as well sell your kids for drug money.
Related
I'm using Jenkins for CI on iOS projects and want to collect some software metrics on them. But the only tool I was able to find was CLOC which only counts lines of codes (LOCs). Nevertheless it's better than nothing.
What I really want to count are methods, classes, calls to other classes etc. (to do the fancy cyclomatic complexity stuff).
Perhaps I'm missing some tools, let me know, if I do.
OCLint?
From oclint.org:
OCLint is a static code analysis tool for improving quality and
reducing defects by inspecting C, C++ and Objective-C code and looking
for potential problems like:
Possible bugs - empty if/else/try/catch/finally statements
Unused code
unused local variables and parameters
Complicated code - high cyclomatic complexity, NPath complexity and high NCSS
Redundant code -
redundant if statement and useless parentheses
Code smells - long
method and long parameter list
Bad practices - inverted logic and
parameter reassignment ...
Lizard will do it. Check it out at https://github.com/terryyin/lizard.
You can try XClarify, a pretty complete objective-c code analyzer, and it's free for open source contributors.
Beyond lines of code and test coverage, I'm not sure there are any such tools yet for Obj-C. I suspect we'll see some soon given the influx of devs from other platforms who use metrics, but in my 7 years as an Obj-C dev I haven't heard of anyone having a tool for collecting them. Of course it'd be good to be proved wrong :)
ProjectCodeMeter measures flow complexity (similar to McCabe cyclomatic complexity) on Objective-C code, but it doesn't count methods and classes though..
I use few tools for gathering code quality metrics:
OCLint - Gather some metrics, like cyclomatic complexity, and enforce best practice - http://oclint.org
Simian - Similarity Analyser - http://www.harukizaemon.com/simian/
Clang analyzer - Same tool as in Xcode (Product -> Analyze), seems like a bit outdated though useful too. To perform it on CI see that: http://clang-analyzer.llvm.org/scan-build.html
Coveralls - nice tools for visualization of unit test coverage - https://coveralls.io
I've found recently that it exists free plugin for SonarQube - https://github.com/octo-technology/sonar-objective-c but it's not really feature-rich. Official one is here: http://www.sonarsource.com/products/plugins/languages/objective-c/
What I really want to count are methods, classes
nnnot rrreallly.... you can parse the xcode indexes or the output of nm -- or run doxygen.
calls to other classes etc
gcov -- or run doxygen
I just stumbled upon Xcode Statistician (link seems to be dead), but haven't tried it yet. The zip archive can be downloaded directly.
this is my first post, and it covers something which I've been trying to get working on and off for about a year now.
Essentially it boils down to the following: I have a copy of newlib which I'm trying to get working on an LPC2388 (an ARM7TDMI from NXP). This is on a linux box using arm-elf-gcc
The question I have is that I've been looking at a lot of the tutorials talking about porting newlib, and they all talk about the stubs (like exit, open, read/write, sbrk), and I have a pretty good idea of how to implement all of these functions. But where should I put them?
I have the newlib distribution from sources.redhat.com/pub/newlib/newlib-1.18.0.tar.gz and after poking around I found "syscalls.c" (in newlib-1.18.0/newlib/libc/sys/arm) which contains all of the stubs which I have to update, but they're all filled in with rather finished looking code (which does NOT seem to work without the crt0.S, which itself does not work with my chip).
Should I just be wiping out those functions myself, and re-writing them? Or should I write them somewhere else. Should I make a whole new folder in newlib/libc/sys with the name of my "architecture" and change the target to match?
I'm also curious if there's proper etiquette on distribution of something like this after releasing it as an open source project. I currently have a script which downloads binutils, arm-elf-gcc, newlib, and gdb, and compiles them. If I am modifying files which are in the newlib directory, should I hand a patch which my script auto-applies? Or should I add the modified newlib to the repository?
Thanks for bothering to read! Following this is a more detailed breakdown of what I'm doing.
For those who want/need more info about my setup:
I'm building a ARM videogame console based loosely on the Uzebox project ( http://belogic.com/uzebox/ ).
I've been doing all sorts of things pulling from a lot of different resources as I try and figure it out. You can read about the start of my adventures here (sparkfun forums, no one responds as I figure it out on my own): forum.sparkfun.com/viewtopic.php?f=11&t=22072
I followed all of this by reading through the Stackoverflow questions about porting newlib and saw a few of the different tutorials (like wiki.osdev.org/Porting_Newlib ) but they also suffer from telling me to implements stubs without mentioning where, who, what, when, or how!
But where should I put them?
You can put them where you like, so long as they exist in the final link. You might incorporate them in the libc library itself, or you might keep that generic, and have the syscalls as a separate target specific object file or library.
You may need to create your own target specific crt0.s and assemble and link it for your target.
A good tutorial by Miro Samek of Quantum Leaps on getting GNU/ARM development up and running is available here. The examples are based on an Atmel AT91 part so you will need to know a little about your NXP device to adapt the start-up code.
A ready made Newlib porting layer for LPC2xxx was available here, but the links ot teh files appear to be broken. The same porting layer is used in Martin Thomas' WinARM project. This is a Windows port of GNU ARM GCC, but the examples included in it are target specific not host specific.
You should only need to modify the porting layer on Newlib, and since it is target and application specific, you need not (in fact probably should not) submit your code to the project.
When I was using newlib that is exactly what I did, blew away crt0.s, syscalls.c and libcfunc.c. My personal preference was to link in the replacement for crt0.s and syscalls.c (rolled the few functions in libcfunc into the syscalls.c replacement) based on the embedded application.
I never had an interest in pushing any of that work back into the distro, so cannot help you there.
You are on the right path though, crt0.S and syscalls.c are where you want to work to customize for your target. Personally I was interested in a C library (and printf) and would primarily neuter all of the functions to return 0 or 1 or whatever it took to get the function to just work and not get in the way of linking, periodically making the file I/O functions operate on linked in data in rom/ram. Basically without replacing or modifying any other files in newlib I had a fair amount of success, so you are on the right path.
About 2 months ago I overtook building proccess in current company. Even though I don't have much knowledge of it, I was the only with enough time, so I didn't have much choice.
Situation is not that good, and I would like to do following:
Labeling files in SourceSafe with version (example ProjectName PV 1.2)
GetFiles from SourceSafe to specific directory
Build vb6/c++/c# projects(yes, there are all kinds of them)
Build InstallShield setups
This is for now partly done using batch scripts(one for labeling and getting, one for building, etc..). So when building start I pretty much have babysit it.
Good part of this code could be reused.
Any recommendations on how to do it better? One big problem is whole bunch of dependencies between projects. Also labeling has to increment version and if necessary change PV to EV.
I would like to minimize user interaction as much as possible. One click on one build script(Spolsky is god) and all is done, no need to increment version, to set where to get files and similar stuff.
Is the batch scripting best way to go? Should I do some functionality with msbuild. Are there any other options?
Specific code is not need, for now I just need a way how to improve it, even though it wouldn't hurt.
Tnx,
Marko
Since you already have a build system (even though some of it currently "manual"), whatever you do, don't start over from scratch.
(1) Make sure you have a test machine (or Virtual Machine) on which to work. Thus you can make changes and improvements without having to worry about breaking anything.
(2) Put all of your build scripts and tools in version control, not just the source code. Then as you make changes, see if they work. If they do, then save them to version control. If they don't, then roll them back.
(3) Choose one area to work on at a time. Don't try to do everything at once. Going from a lot of manual work to "one-click" will take time no matter what build system you're working with.
Sounds like you want a continuous integration solution, like CC.Net. It has configuration options to do all the things you want and a great community to answer questions.
Also, batch scripting is probably not a good option. Sophisticated build and integration tools will let you feed parameters into the build and create different builds for different environments (test, production, etc.). Batch scripting will involve a lot of hand-coding and glue.
The goal of IDEs is increase productivity. They do a great job at that. Refactoring, navigation, inline documentation, auto completion help increase productivity immensely.
But: Every tool is a weapon. The very same IDE helps to produce chunk code. Some IDE features are an invitation to produce bad code: code generation, code formatting tools, refactoring tools.
IDE overuse tends to isolate developers from the necessary details. It is a good thing that you can start working but at some point in your career you have to be able to figure out how to start a process. You can ignore this detail for some time, in the end they are important to write a working product (vs. bolted together stuff that works 90% of the time).
How do you encourage positive behavior of other developers working with an IDE? This is a question as old as copy and paste.
To get the right impression: developers have to have the maximum freedom to mobilize their maximum creativity and motivation. They may use IDEs and all the related tools as they see fit. Nobody should impose draconian measures on them. I don't want to demotivate and force someone to do something. Good behavior has to be encouraged. It has to itch little a bit if you do the wrong thing. In the same line as the SO "accept rate" metric (and reputation). You can ignore it but life is better if you follow the rules.
(The solution should work in a given setting. You can ignore reviews, changing the staffing or more education as potential solutions.)
Train your IDE, instead of being trained by it.
Set up code formatting the way you (or your team) wants it. Heck, even disable it in cases where it makes sense. I've never seen an IDE align something like this with a sensible combination of tabs and spaces (where \t is obviously the tab character):
{
\tcout << "Hello "
\t << (some + long + expression +
\t to_produce_the_word(world))
\t << endl;
}
In languages like Java, you cannot avoid boilerplate. The best option you have is to check generated code, ensuring that it is the same as what you'd have written by hand. Modify it as necessary. Configure your IDE to generate the exact code that you need, if possible. Eclipse is pretty good at this.
Know what's going on under the hood.
Know that your IDE is actually invoking the compiler. Have some insight into the flags that it passes. Be able to invoke the compiler from the command line.
Know about the runtime system. Be aware of the flags that are used or needed to launch your program. Be able to launch the program from a command line.
I think before anyone uses a RAD tool of any type they should be able to write the application from scratch (scratch being wiring together the framework components) in notepad potentially on a computer that is 10 years older than current technology :P. Not knowing the ins and outs of a paradigm/framework leads to bad code from novice developers who only learn things at a mile high view of the platforms they develop for. Perhaps they should do this in a few technologies -- i.e., GTK programming is completely different to MVC which is then also different to SWING and .NET.
I think the end result should be a developer that thinks of the finer details of a problem before they jump to thinking of how they will write an interface to it in a specific RAD environment.
its an open ended question, but...
We have a Eclipse format file that everyone shares, so that we all format the code in the same manor. (Except the one lone InteliJ guy we have).
Everyone shares a dictionary file. It helps to remove all the red lines from the code. Making it look cleaner and more readable.
I run EMMA over the code to find out who isn't testing their code, and then moan at them.
The main problems we face is that most of the team don't know all the features/power of the IDE (eclipse). The didn't know about CTRL + O (twice), or auto code gen. All I can do as a 'hot key wizard' is keep sharing my knowledge with them to help them become more productive.
I look forward to the day when my problem is that they auto gen as much as possible.
Rather than me finding bugs where the wrong value is returned from a getter method due to a typo.
Attempt development (at least occasionally) using only a text editor and launching the compilation, testing, etc. from the command line.
Typing the commands will get tedious very quickly so create scripts or (even better) learn rake, ant, msbuild.
If the IDE does code generation for you and that code generation is really important (such as generating classes from xsd or proxy classes from wsdl), try to find out how to run the code generation from the command line - then hook the code generation into a build (so you'll never be tempted to edit the generated code).
The idea of autoformatting code is great but it usually just turns your code into a mess. If you have less code, minor formatting inconsistencies are just not a big deal.
Adding code quality tools into your build - style checks, class and method sizes, complexity, code duplication, test coverage, etc (complexian, simian, flog, flay, ndepend, ncover, etc.) will discourage IDE generated code.
Recently (about a month ago) I was trying to introduce new constructs to my company's in-house extension language, and struggling with a couple of reduce-reduce errors. While I eventually solved this problem, digging into the y.output file was no picnic.
As an experiment, I tried using Bison's --graph=<file> option to output a DOT file (note that our standard build uses Byacc, not Bison). As I'm on a 'turnkey' Linux box, I didn't have a Graphviz installation and could not easily install from RPMs (working on Red Hat Enterprise Linux 4). Instead, I built it from source.
As an initial experiment, I tried to run dotty with an output of Postscript. Now our internal language is your average home-grown, Turing-complete, dynamically typed scripting language, but I was unprepared for what followed. The dotty run took over four hours (2GHz dual core AMD64 box)! And when it was done, the graph that was rendered was not what I would call readable.
So, quite simply, I'm looking for advice. Are there a set of switches which would improve the outcome over the 'default' approach I took? I'm looking for experience in
optimizing 'render' time
improving readability of the graph
possible advice on better graphical viewers
I imagine you've already seen this link, but just for completeness, there is a list of viewers etc. at: http://graphviz.org/resources/ or see https://web.archive.org/web/20131005020548/http://graphviz.org/Resources.php for an archived copy.