I have MSbuild task like this to sign all the output modules of our project.
<SignFile Condition="Exists('$(OutputPath)\%(FilesToSign.identity)')"
CertificateThumbprint="$(THUMBPRINT)"
SigningTarget="$(OutputPath)\%(FilesToSign.identity)"
TimestampUrl="http://timestamp.verisign.com/scripts/timestamp.dll" />
It takes quite a while (10 minutes or more) when I have many files. It is possible to run stuff in parallel or in other ways speed it up. (I am trying to sign more than 100 files.. )
Another way of speeding up the signing is to remove the TimeStampUrl parameter. It may not be good enough for release build (to not have a time stamp on the signature), but it is good enough for a development build.
And it speeds up the signing process with 80-90%.
The only way to do parallel build with MsBuild is to have different instances of msbuild, thus different project files, I don't think that's recommended here. You cannot do task or target in parallels, yet you can build project in parallel (but you can create several project files with one target in each). You may have precision here : How to run tasks in parallel in MSBuild .
Moreover, I think you will be limited by your disk access speed and not by your memory.
I don't know the SignFile task enough to give advice on how optimize it, though, sorry.
Related
I'm working on a c++ project built with cmake+ninja with approx 1200 build targets on a 64 thread computer.
There's one translation unit that takes 10min to compile, most others are comparably fast such that a build of all other targets together takes (building on all threads) only about 9 minutes. The slow translation unit is rather independent of the rest, so it doesn't have to be scheduled late, but as it turns out is is scheduled late by default, such that a complete build takes me between 15 and 20 minutes, and at the end there is only one thread working with almost all other targets done. The build would be faster for me, if the slow translation unit would be scheduled first, blocking one thread for about 10 minutes, while all other threads work on the rest of the project and the entire project is built within 10min.
Is there a way in cmake or ninja to shift the priorities for scheduling to point out slow or "please early" targets without messing up dependencies?
As of February 27, 2021, the answer is no. There are some open issues (#232, #376) and an abandoned PR (#1333) on GitHub requesting this feature in base Ninja. CMake does not provide any way to prioritize a target (at least through 3.20) either.
Messing with dependencies (even order-only) doesn't help here (as you likely know) because that would just force smaller targets to start either completely before or completely after the long target. A priority hint is what's truly needed here.
The only workaround I can think of (and it's not a great one), is to split your long target out into a separate ExternalProject and create a superbuild that builds the large target and the independent portion at the same time. This would require significant restructuring and would be a non-starter for a lot of projects. It might be worth the pain if you're losing a lot of development time to this issue, though.
If I Build my solution from VS, it takes less than a minute to do all the checks to tell me all the projects have been built. If I do the same in team foundation build, it takes closer to 20 minutes.
i define as "items to build" only the solution (.sln file) and solution has 32 "projects".
Check the build definition workspace, make sure you are only pulling doen the code you need for the build.
Also TFS does a clean compile, I.e. it pulls down all of the code then rebuilds everything from scratch. How long does a clean build take locally? Less than a minute to build 32 projects seems quick so I suspect that your local build is incremental rather than full.
Run a build with diangnostic loging, this should give you a clue to the parts that are taking the time.
Is there a way to run an ant build such that you get an output of what the build would do, but without actually doing it?
That is to say, it would list all of the commands that would be submitted to the system, output the expansion of all filesets, etc.
When I've searched 'ant' and 'test', I get overwhelming hits for running tests with ant. Any suggestions on actually testing ant build files?
It seems, that you are looking for a "dry run".
I googled it a bit and found no evidence that this is supoorted.
Heres a bugzilla-request for that feature, that explains things a bit:
https://issues.apache.org/bugzilla/show_bug.cgi?id=35464
This is impossible in theory and in practice. In theory, you cannot test a program meaningfully without actually running it (basically the halting problem).
In practice, since individual ant tasks very often depend on each other's output, this would be quite pointless for the vast majority of Ant scripts. Most of them compile some source code and build JARs from the class files - but what would the fileset for the JAR contain if the compiler didn't actually run?
The proper way to test an Ant script is to run it regularly, but on a test system, possibly a VM image that you can restory to the original state easily.
Here's a problem: You have target #1 that builds a bunch of stuff, then target #2 that copies it.
You run your Ant script in test mode, it pretends to do target #1. Now it comes to target #2 and there's nothing to copy. What should target #2 return? Things can get even more confusing when you have if and unless clauses in your ant targets.
I know that Make has a command line parameter that tells it to run without doing a build, but I never found it all that useful. Maybe that's why Ant doesn't have one.
Ant does have a -k parameter to tell it to keep going if something failed. You might find that useful.
As Michael already said, that's what Test Systems - VM's come in handy- are for
From my ant bookmarks => some years ago some tool called "Virtual Ant" has been announced, i never tried it. So don't regard it as a tip but as something someone heard of
From what the site says =
"With Virtual Ant you no longer have to get your hands dirty with XML to create or edit Ant build scripts. Work in a completely virtualized environment similar to Windows Explorer and run your tasks on a Virtual File System to see what they do, in real time, without affecting your real file system*. The actual Ant build script is generated in the background."
Hm, sounds to good to be true ;-)
..without affecting your real file system.. might be what you asked for !?
They provide a 30day trial license so you won't lose no money but only the time to have a look on..
About 2 months ago I overtook building proccess in current company. Even though I don't have much knowledge of it, I was the only with enough time, so I didn't have much choice.
Situation is not that good, and I would like to do following:
Labeling files in SourceSafe with version (example ProjectName PV 1.2)
GetFiles from SourceSafe to specific directory
Build vb6/c++/c# projects(yes, there are all kinds of them)
Build InstallShield setups
This is for now partly done using batch scripts(one for labeling and getting, one for building, etc..). So when building start I pretty much have babysit it.
Good part of this code could be reused.
Any recommendations on how to do it better? One big problem is whole bunch of dependencies between projects. Also labeling has to increment version and if necessary change PV to EV.
I would like to minimize user interaction as much as possible. One click on one build script(Spolsky is god) and all is done, no need to increment version, to set where to get files and similar stuff.
Is the batch scripting best way to go? Should I do some functionality with msbuild. Are there any other options?
Specific code is not need, for now I just need a way how to improve it, even though it wouldn't hurt.
Tnx,
Marko
Since you already have a build system (even though some of it currently "manual"), whatever you do, don't start over from scratch.
(1) Make sure you have a test machine (or Virtual Machine) on which to work. Thus you can make changes and improvements without having to worry about breaking anything.
(2) Put all of your build scripts and tools in version control, not just the source code. Then as you make changes, see if they work. If they do, then save them to version control. If they don't, then roll them back.
(3) Choose one area to work on at a time. Don't try to do everything at once. Going from a lot of manual work to "one-click" will take time no matter what build system you're working with.
Sounds like you want a continuous integration solution, like CC.Net. It has configuration options to do all the things you want and a great community to answer questions.
Also, batch scripting is probably not a good option. Sophisticated build and integration tools will let you feed parameters into the build and create different builds for different environments (test, production, etc.). Batch scripting will involve a lot of hand-coding and glue.
As a best practice, do you run code analysis on both debug and release builds, or just one or the other?
If for some reason the two builds are different (and they really shouldn't be for static analysis purposes), you should ensure that your metrics are running against what's actually going out to production.
Ideally, you should have a CI server, and the commands that developers run to initiate such analysis are no different from what the CI server does.
I usually pick one and that one is the release build. I guess it doesn't really matter but I tend to think that when gather information about what will run in production it is best to test exactly what will go to production (this goes for analysis, profiling, benchmarking, etc.).
Static Code Analysis will show the same results regardless of your build type.
Debug/Release only changes the resulting assembly and the inclusion or exclusion of debugging information at runtime.
I don't have separate ‘debug’ and ‘release’ builds (see Separate ‘debug’ and ‘release’ builds?).
The LLVM folks actually recommend analyzing the DEBUG configuraion:
ALWAYS analyze a project in its "debug" configuration
Most projects can be built in a "debug" mode that enables assertions.
Assertions are picked up by the static analyzer to prune infeasible
paths, which in some cases can greatly reduce the number of false
positives (bogus error reports) emitted by the tool.
In addition, debug builds tend to be faster (no need for optimization), and in the CI world faster is always better (all else being equal).