My incremental CI builds frequently get:
##[error]CMake Error: Target xxx has dependency information when it shouldn't.
##[error]Your cache is probably stale. Please remove the entry xxx_LIB_DEPENDS from the cache.
Is it safe to delete the cache file before each build?
Will most build binaries be still up to date, or will this trigger a full rebuild?
Someone answered here, but it was removed. So i answer it myself with a link to this question in the CMake forums:
if you delete CMakeCache.txt before each build you must make sure to
configure and generate cmake. Without it is not “safe” because the
cached variables are missing in the next build.
Deleting CMakeCache.txt should not affect any binaries, a full rebuild
is not triggered. Otherwise you might have a local variable passed on
before being cached which then leads to inconsistent runs.
The regeneration of the project can trigger rebuilds if the
configuration is different to the previous one.
Thanks #hex
Related
For example, when I change the compiler, the whole cache is deleted and needs to be reconfigured. I understand that things like the compiler flags would need to be reset but why the whole cache?
Turning my comment into an answer
The Message itself
The CMake message
"You have changed variables that require your cache to be deleted. Configure will be re-run and you may have to reset some variables. The following variables have changed: ..."
translates to something like
"CMake tries to behave and is not telling you that for a change like that you have to delete your binary output directory and start from scratch again."
This feature came a long time ago with "ENH: fix for bug 6102, allow users to change the compiler" commit and was improved with "Delete entire CMakeFiles directory when deleting CMakeCache.txt" commit:
Fix this simply by teaching cmCacheManager::DeleteCache to remove the
entire CMakeFiles directory recursively whenever it removes an existing
CMakeCache.txt. This fully resets the build tree to configure with a
fresh compiler.
Why does it have to delete everything?
If you change the compiler or upgrade to a newer version of CMake, your generated build environment could not just simply be adapted.
All cached configurations - as being an evaluation or search results based on the compiler and platform information - have to be deleted and re-run. CMake could simply not know which cached variable is still valid in the new environment.
I am currently trying to swap static libraries in one of my projects which requires swapping find_package() commands for explicit links to static libraries. One of the problems I encountered is that even after deleting find_package() commands reloading the CMake Project the old variables are still present.
Is there a way to explicitly flush the CMake cache other than manually deleting all files from the temporary folder Clion creates for building the project?
Starting with CLion 1.5 144.3143 (2016-01-20), you can clear the cmake cache without loosing the CLion indexes and without restarting (meaning: very fast, also on huge projects):
Tool | CMake | "Reset cache and reload project".
If that for some reasons is not enough, or if you have an older CLion, you can do the more radical:
File | "Invalidate Caches / Restart"
and click on "Invalidate and restart".
EDITS
2016-01-20: add the faster method available with newer CLion.
There is some magic with this cache.
I got bad value for one variable once: by some reason cmake found some wrecked freetype includes inside mono.framework (another reason to dislike mono...) and tried to use them to compile my project. Obviously compilation failed.
I tried to remove CMakeCache.txt and rebuild project but incorrect path come back. Even after I removed mono.framework dir!
The solution was to remove incorrect line from the cache and restart the build. After that cmake got the correct path and build succeded.
UPD:
After I "reload cmake project" incorrect value came back. Deleting everything in $HOME/Library/Caches/clion11/cmake/generated/ dir (for OS X) seems to finally solve the issue.
I am looking for a solution to automatically increment a package version (not to be confused with an assembly version) after each build on CI server (particularly Atlassian Bamboo). Every appx package has a version defined in its manifest file (appxmanifest). Thus in order to increase the version a manifest must be edited before commit. I am considering different approaches to implement this. The first one makes changes in manifest and pushes it back to the repo.
Starts building a plan (in order to lock a build number)
Modifies manifest so that a revision is set to the current build number
Pushes changes to SCM (particularly Atlassian Stash). This step shouldn't trigger the next build.
Continues building the package (invoke MSBuild, UT and other tasks)
Cons
Leads to incorrect workflow on Bamboo: checkout -> push -> build
Each build makes a new commit
Another approach is to setup post receive Stash hook which would modify appxmanifest.
Cons Hard to keep a build number in sync with Bamboo.
Is there any other (cleaner and proper) way to achieve this?
ex-Stash developer here (not that it matters),
I would highly recommend not checking in derived/version information or files. It's going to cause you no end of problems (some of which you have pointed out in your question).
My advice - generate what information you need on the build. I don't know anything about appx packaging, but can you use a placeholder/property (like this) which can be resolved on the Bamboo build? For our builds we use the git hash and timestamp as the version, and in the past I've also used the job/build number (timestamp is better though).
As more food for thought - if that appx version is important for developers to see locally, and it becomes hard to match up with the Git version then you can also attach a Git tag/note to the commit in Bamboo as well. The nice thing about that is that anyone fetching from Git can easily see that extra metadata, but it doesn't result in extra commits for every build. If the appx version need to be based off the previous version then this makes it possible for the build scripts to inspect the previous commit and bump the version appropriately.
I hope that helps.
I have a plan in Bamboo that starts whenever changes are made to the attached repositories (via polling).
Now, on each build, if successful, a CHANGELOG file is updated in the repo, which in turn, triggers another build. How can I omit certain files from Bamboo's polling, so that a build isn't started if changes are found for those files? Because otherwise, I enter in infinite loop, with a change to CHANGELOG triggering another build which in turn updated CHANGELOG and so on.
If this is not possible, what other viable solutions are there? Is it possible to attach a shell script somewhere before the build starts to check whether it's desired to start a new build?
It turned out that this was simpler than I've thought. In Plan Configuration, in the Repositories tab, on each repository, under Advanced, there is an Include / exclude files input where you can Customise what files Bamboo uses to detect changes. By adding a regular expression there, I got everything solved and working as expected.
Bamboo pattern matching reference: https://confluence.atlassian.com/display/BAMBOO/Pattern+matching+reference
The Bamboo Documentation says:
Bamboo will ignore build triggers if the local working copy and the
repository copy have the same revision numbers.
This might not be the best solution, but you might add an additional task at the end of the job/build which updates the repository again to avoid triggering a new build.
I'm not sure if this would then skip builds from repository updates which occur during the current build.
I've just committed some code changes to my repository and all of a sudden (after weeks of being fine). The TC build starts failing as it fails to download the NuGet package for Microsoft.Bcl.Build.1.0.6.
I ended up having to manually copy the contents of the packages directory to the TC build location which utterly defeats the point of NuGet.
What can I check to get to the root cause of this?
Everything about NuGet is enabled in the solution for getting packages.
I've blogged about this issue at http://sedodream.com/2012/12/24/SlowCheetahBuildServerSupportUpdated.aspx. To summarize NuGet package restore (prior to 2.7) is implemented as a part of the MSBuild build process. When MSBuild starts a build it will evaluate the project file and any Import declarations importing other files. This happens before any target gets executed.
Since NuGet pkg restore is a part of the build process the .targets files get restored at a point in time that it is too late for the Import statement to have any impact.
You can work around this by either checking in the .targets file as you stated, or by invoking pkg restore before the build process. I've created a NuGet package, PackageRestore, which can help with the latter approach.
To use PackageRestore just add the NuGet package to your project which will automatically create a file named packageRestore.proj in your projects directory. When configuring your build, you will need to build that item before your .sln/.csproj file.
OK This is a bit of a nasty issue.
If you are having this problem you need to do something rather ugly to your repository.
Make sure you are checking in the packages\repositories.config file.
Then if your build is failing with unresolved references to Microsoft.Bcl.Build you will need to also checkin the .targets file for this package. eg:
package\Microsoft.Bcl.Build.x.x.x\tools\Microsoft.Bcl.Build.targets
Hideous ...
This blog post is the most thorough one I've seen for explaining the workaround options:
http://blogs.msdn.com/b/dotnet/archive/2013/06/12/nuget-package-restore-issues.aspx
None are great IMO - this problem still needs a better solution.
But for the time being, the best recommended option is to check the Bcl.Build .targets file into source control - which means when the version of Bcl.Build is updated, you'll need to add the new .targets file, and remove the old one.
I think (but am not sure, so I created this SO question: What does the Microsoft.Bcl.Build NuGet package do?) that Microsoft.Bcl.Build is only needed for development, and is not needed on a build server. So, I have a Builder.targets file that only exists in the build environment, that is indirectly <import> ed into all of our projects, which includes this bit of MSBuild xml:
<!-- Skip Microsoft.Bcl.Build functionality when building only from Source. Presumably Microsoft.BclBuild is only needed for development. -->
<PropertyGroup>
<BclBuildImported>Ignore</BclBuildImported>
</PropertyGroup>
Since the block of MSBuild logic inserted into your project by the Bcl.Build nuget package is dependent on the BclBuildImported property being empty, this effectively sidesteps the problem in my build environment - the Microsoft.Bcl.Build steps are skipped, and it no longer breaks my CI builds.
Note that since it appears that this package manages binding redirects in your app.config, and ensures that transitive dependencies are included in your projects, it's important to leave in for development. But I'm not currently aware of a need for it in a build server environment.