I'm currently building a product distributed through MSI Windows Installer. That product is being integrated by our customers using different forms such us inside their own MSI, using bootstrapper/chainner like WiX Burn or authoring tools like InstallShield.
Having this scenario in mind, I always wanted to know what are the limitations and/or benefits to use Merge Modules (MSM) instead of keeping an MSI, and also what are the nowadays recommended approach about choosing one against the other.
On paper merge modules are fine, but in the real world I find them clunky to update and hence error prone since they may be merged into many setups before being discovered to be defective. As a result I do not recommend merge modules at all. I prefer a single MSI that can be run as a batch process via a bootstrapper or batch file and that can also be updated easily. This avoids all kinds of problems that are not generally intuitive.
I want to add that merge modules work well for truly shared files installed in locations in the file system that are meant for shared files and that change infrequently. These are generally OS-runtimes. These merge modules are generally heavily tested and work ok. However, often I see people use merge modules for files that end up changing frequently and that they then end up installing in different locations in different flavors in an ad-hoc fashion. This kind of use is a total mess and a hugely wasted effort.
Having said all that - I have indeed used merge modules successfully when I have needed advanced release management with repetitive, non-changing inclusion of a set of files via a merge module into several setups. Even then I ran into a version issue after a while with a couple of files needing update, and subsequent, minor errors with the wrong merge module being used when I left the project to someone else. I also experienced having to rebuild all setups due to a minor merge module bug fix. All setups then had to go through QA again. Very frustrating with such tight coupling.
If your requirements are simple and you are not taking on a huge multi-product release project sharing a bunch of files, use MSI instead of MSM. Easier to comprehend, generally less work to deal with, more atomic updates and less risk of introducing the same error in many setups due to merge module update or design problems.
There's nothing that wrong with merge modules. Their primary use (which hasn't been mentioned) is sharing. If you want the same set of shared files in multiple MSI files, they need the same set of component guids to preserve the sharing rules. Or if you are giving files to clients for them to use (like Microsoft) in their MSI builds then give them merge modules. That's one of the reasons MS and other vendors redistribute merge modules so that everyone can build their MSIs and install them on the same system without file sharing disasters. I've also seen merge modules used as a common UI for MSI files. But mainly they are essential to make sure that shared files are used correctly. I'll tell you from experience that the disaster resulting from incorrect use of shared files is much worse that any perceived difficulty with using merge modules. Note also that they are universal and can be included in all tools that build MSI files.
I've never found merge modules difficult to patch, version, or fix. Major upgrades aren't a problem. The only potential issue I've seen is with build processes that rebuild all the binaries in a merge module during creation of a patch (.msp) build. If only one binary needs a fix but you compile them all, their versions and internals may change enough so that the patch process (the delta between two MSI files and their content) will tell you that they need to be included in the patch because they've changed, but this issue can be avoided if it is in fact an issue.
Related
I've been trying to set up a WiX component such that the user can specify that the installer should not upgrade that component on a MajorUpgrade. I had the following code, but this means that if the condition is met then the new version is not installed, but the old version is also removed.
<Component Id="ExampleComponent" GUID="{GUID here}">
<Condition>NOT(KEEPOLDFILE="TRUE")</Condition>
<File Id="ExampleFile" Name="File.txt" KeyPath="yes" Source="File.txt"/>
</Component>
Ideally, if the user specifies "KEEPOLDFILE=TRUE", then the existing version of "File.txt" should be kept. I've looked into using the Permanent attribute, but this doesn't look relevant.
Is this possible to achieve without using CustomActions?
A bit more background information would be useful, however:
If your major upgrade is sequenced early (e.g. afterInstallInitialize) the upgrade is an uninstall followed by a fresh install, so saving the file is a tricky proposition because you'd save it, then do the new install, then restore it.
If the upgrade is late, then file overwrite rules apply during the upgrade, therefore it won't be replaced anyway. You'd need to do something such as make the creation and modify timestamps identical so that Windows will overwrite it with the new one. The solution in this case would be to run a custom action conditioned on "keep old file", so you'd do the reverse of this:
https://blogs.msdn.microsoft.com/astebner/2013/05/23/updating-the-last-modified-time-to-prevent-windows-installer-from-updating-an-unversioned-file/
And it's also not clear if that file is ALWAYS updated, so if in fact it has not been updated then why bother to ask the client whether to keep it?
It might be simpler to ignore the Windows Installer behavior by setting its component id to null, as documented here:
https://msdn.microsoft.com/en-us/library/windows/desktop/aa368007(v=vs.85).aspx
Then you can do what you want with the file. If you've already installed it with a component guid it's too late for this solution.
There are better solutions that require the app to get involved where you install a template version of this file. The app makes a copy of it that it always uses. At upgrade time that template file is always replaced, and when the app first runs after the upgrade it asks whether to use the new file (so it copies and overwrites the one it was using) or continue to use the existing file. In my opinion delegating these issues to the install is not often an optimal solution.
Setting attributes like Permanent is typically not a good idea because they are not project attributes you can turn on and off on a whim - they apply to that component id on the system, and permanent means permanent.
I tried to make this a comment, it became to long. I prefer option 4 that Phil describes. Data files should not be meddled with by the setup, but managed by your application exe (if there is one) during its launch sequence. I don't know about others, but I feel like a broken record repeating this advice, but hear us out...
There is a description of a way to manage your data file's overwriting or preservation here. Essentially you update your exe to be "aware" of how your data file should be managed - if it should be preserved or overwritten, and you can change this behavior per version of your application exe if you like. The linked thread describes registry keys, but the concept can be used for files as well.
So essentially:
Template: Install your file per-machine as a read-only template
Launch Sequence: Copy it in place with application.exe launch sequence magic
Complex File Revision: Update the logic for file overwrite or preservation for every release as you see fit along the lines as the linked thread proposes
Your setup will "never know" about your data file, only the template file. It will leave your data file alone in all cases. Only the template file it will deal with.
Liberating your data files from the setup has many advantages:
Setup.exe bugs: No unintended accidental file overwrites or file reset problems from problematic major upgrade etc... this is a very common problem with MSI.
Setup bugs are hard to reproduce and debug since the conditions found on the target systems can generally not be replicated and debugging involves a lot of unusual technical complexity.
This is not great - it is messy - but here is a list of common MSI problems: How do I avoid common design flaws in my WiX / MSI deployment solution? - "a best effort in the interest of helping sort of thing". Let's be honest, it is a mess, but maybe it is helpful.
Application.exe Bugs: Keep in mind that you can make new bugs in your application.exe file, so you can still see errors - obviously. Bad ones too - if you are not careful - but you can easily implement a backup feature as well - one that always runs in a predictable context.
You avoid the complicated sequencing, conditioning and impersonation concerns that make custom actions and modern setups so complicated to do right and make reliable.
Following from that and other, technical and practical reasons: it is much easier to debug problems in the application launch sequence than bugs in your setup.
You can easily set up test conditions and test them interactively. In other words you can re-create problem conditions easily and test them in seconds. It could take you hours to do so with a setup.
Error messages can be interactive and meaningful and be shown to the user.
QA people are more familiar with testing application functionality than setup functionality.
And I repeat it: you are always in the same impersonation context (user context) and you have no installation sequence to worry about.
Short version: How can I make sure that my component GUIDs remain stable using Paraffin on a build server?
I am currently working on a project that should be deployed via WiX. As this is a web project, it contains many files (still in early stage and already almost 200 files). Also, during development, files are constantly added and deleted, so maintaining the WiX component lists manually is simply not an option.
Since I read a lot about component rules and that people breaking them go to hell, I decided to go with Paraffin as a harvester. This tool is capable of updating an existing component list, thus not re-creating new GUIDs for existing components.
However, when a new component is created, the tool assigns a new GUID. Even if the component files are identical, then initial GUIDs will be different on different machines or even only at different times.
So, obviously, I need a central authority for fixing the initial GUIDs. My idea was to commit empty component lists, which are then filled by the build server calling Paraffin on build. So when I only distribute the MSIs created by the build server, I can be sure that component rules are being followed.
However, the problem with this approach is, that I have no means of tracking my GUIDs, should the build server crash or empty its local repository. I was thinking about having the build server commit the generated component list to my repository, but that doesn't seem like a clean idea.
Another solution I thought of was having all developers build (and thus call Paraffin) before commiting. Thus, each developer would create the initial GUIDs for their newly added files and commit them to the component list.
The obvious problem with this approach: People (e.g. developer A) will forget to build before they commit. So in these cases the build server will create the initial GUIDs for the new files, but those will also only be stored locally. A few commits later, developer B will come along and build the solution, creating a new GUID for the files created by developer A. He will then commit the component list containing this GUID and the build server will check it out. Now the build server has obtained a GUID (created by developer A) for a package, for which it had previously used a different (self-created) GUID, even though the files didn't change in the meantime.
So, how can I make sure, that my GUIDs remain stable between builds without relying on developers to build their solution before they commit? The approaches outlined above both seem unsatisfying to me, but are all I can think of right now.
As far as I am concerned component rules only really come into play when you have multiple installers that share components with the same guids (which should then be exactly the same resource(s)) or you are using a wixlib or a merge module which is then included as part of different installers.
From what you have said above, to me it doesn't sound like you will so, there is no harm in having different component guids for each build. It will just mean that when you upgrade the website, files that have not changed will be removed and re-installed under a different component guid. IMHO that doesn't really matter as long as the installer correctly installs all files that are required for the site to function and doesn't remove components from other products.
If you use the MajorUpgrade element, the old product will be completely removed before the new one is installed so any component guid's that are shared between the two versions will be removed and then re-installed anyway.
I always just leave my guid elements as Guid='*' that way I know that the there will never^ be any guid clashes in any of my components across my multiple products.
^ I know this is not theoretically true but in this use case it is.
Not entirely true. Changing your component GUIDs from build to build is fine if and only if you schedule RemoveExistingProducts early so that the files are off the system before you reinstall the new GUIDs. This approach works well for smallish installers with not so many files, but as your installer grows you will feel the pinch of having twice as much IO to do, as you remove and then reinstall, rather than just overwrite your files. In short, it's up to you, but you should think carefully about how large your application is likely to get before jumping in with the suggested approach.
We use InstallShield to build installers for a number of very similar products, each of which has shared files parked in fixed directories underneath our vendor directory. We have a DLL hell problem that I thought installers should handle.
Imagine products A and B both have file FOO, installed in directory D. FOO isn't necessarily a DLL; it can be any file.
In an ideal world, both copies of FOO are identical and when both products are installed FOO gets (overwritten) in D and everything is fine. And that's how it seems to work when everything is identical.
In practice, A and B may be from different generations, so in fact A has FOO variant A (FOO_a) and B has variant FOO_b. Now only one of FOO_a or FOO_b will end up in D if both products are installed, and consequently one of A or B will be confused because the wrong file is there.
What I would have expected from installers (and InstallShield) is that for each product P installed on a system, that a reference counter would be kept somewhere (the registry?) regarding each installed file. So when A is installed and FOO(_a) is placed, FOO is marked as belonging to A, and has reference count set to 1. If B is now installed with an FOO_b identical to FOO_a, FOO should be marked as belonging also to B, and the reference count incremented. If B is installed, and FOO_b is different than FOO_a, I'd expect the installer to complain that incompatible files FOO were being proposed for installation and the install should abort. When a product is de-installed, I'd expect reference counters for each installed file to be decremented, and the installed file only removed when the reference count goes to zero.
That's not what InstallShield appears to do. It simply smashes FOO into its target directory during an install. This is in effect the same thing as DLL hell. Is this the intended behavior of installers? Of InstallShield? Of Windows?
We thought we looked for a way to ask InstallShield to manage this, but can't find anything. I'd be pleased to have somebody tell where to look to configure this correctly. Or is it that InstallShield can't/won't do this? [If so, why am I giving them money?]
If variants of FOO are versioned and properly cumulative (this is key to all installation technologies that operate on shared locations), and if your component is shared, shares the same GUID in both products (if MSI), and shares the same installation location, then everything will work. As parts of that list of conditions are made untrue, various bad things can and will happen; some have workarounds, some are unrecoverable.
If the installation location differs, little or none of this matters, unless both products end up searching in the same location for some other reason.
If MSI, and the GUID differs (or the contents of the component differ), you've broken component rules. This may result in files not uninstalling at the final uninstall, unexpected changes in a repair, or other hard to predict results.
If the file is not properly versioned, it is not possible to know when to replace it with the other one. It is likely in some ordering, a new install will end up running with an old version. This can be mitigated in an MSI with companion files, if another file has properly increasing versions.
If the file is not cumulative with respect to versions, it is likely that a new version of the file will break the other consumer of it. In this scenario, you should not install it to a shared location.
One size fits all does get a little messy sometimes, so generally the problem space is simplified to one that can be addressed. In this case the cost is that all installation authors have to follow some rules if they want well-defined behavior.
If you are truly looking for a solution, use separate DLLs. This is simple and reliable, and in fact it's easier to talk about (as you have demonstrated when you mention FOO_a and FOO_b). It also avoids the error message, which is not helpful for your users.
Windows did have a reference counting scheme for shared DLLs, but it was by convention, so it was unreliable. If you've ever had an uninstaller ask if you really want to delete a shared component, now you know why.
No one can possibly begin to answer your question without knowing what installation technology you are using. InstallShield is a product that supports many different frameworks.
Windows only keeps track of shared references on DLL files. ( SharedDllCount ). Windows Installer ( assuming you are even using it ) also keeps track of Compoent reference counts.
However, it should be noted (again IF you are using Windows Installer ) that the file costing process that decides to overwrite a file really has nothing to do with reference counts. Reference counts come into play on uninstall.
Assuming you are using Windows Installer, take a look at:
File Versioning Rules
Also look at:
What happens if the component rules are broken?
This *is something Windows Installer and InstallShield know how to handle it assuming that you implement it correctly. Any programming language can be misused and abused intentionally or not.
if you don't want the files to be common, why in the world are you placing them in the same directory? I would rather put them in different location. Now suppose your FOO_a is being used by many programs at the same version, then for FOO_b which is required by your new program and others that might be added later, add another similar common directory and name them using generation names or something.
I believe you are correct in how the reference counting works if you are dealing with a pure MSI-based installer (it would help to know if A and B are Basic MSI or InstallScript projects). In any case, I would verify the components that contains FOO in the A and B install projects have it set as the key file, and have the Shared property set to Yes. That should cause the reference counting to be managed correctly.
Regarding what should happen when you run the installer and it has a newer version of a shared file, to my knowledge that depends mostly on whether the file has a version or not, whether it's a newer version, and properties like its last changed timestamp. If you are building an installer for B that contains FOO_b that is different from FOO_a, I'm not sure how InstallShield could know that it will break A (I could be wrong, maybe it can). One possible workaround is to use custom actions to check if FOO_b is different and if it is, make a copy of the existing one early in the install, then copy it back at the end of the process.
Visual Studio Installer states that it is a best practice to install each file as an installer component. The heat utility provided with Wix also seems to follow the practice of putting every file in its own component.
InstallShield's component wizard uses InstallShield's setup best practice of placing portable executable files in their own component but groups all other files (e.g. unversioned files) by the common destination folder.
The advantage of practice one (each file in its own component) is that each file is set up as a key file which is important if you want these files to trigger repairs. It also allows automation of creating the components (e.g. heat) easier since you are creating a component for each file.
The disadvantages of practice one include the overhead of managing so many components and the bloating of the registry after the application is installed.
An advantage of practice 2 could be seen in an install that installs hundreds of graphics files to one directory. If you do not care about repair functionality, is there any reason to create hundreds of components for this install?
These 2 different practices are conflicting and I want to know which one that people actually use and why.
I always use the Microsoft approach (something similar to what InstallShield does):
http://msdn.microsoft.com/en-us/library/aa368269(VS.85).aspx
I think it's the best because:
- important files (EXE, DLL etc.) have their own component, so they can be repaired easily
- resource files are grouped together
- it allows an optimum components count (not too many to get a long install, but enough to allow an easy repair)
I also noticed that most commercial setup authoring tools use this approach.
I've written about this in the past and I'll try to find a link to it. I think you already understand the question and it's just time for you to decide what is important to you.
For me, I work on installs with 15,000+ files and we only service with major upgrades. For "Program Executables" we follow 1:1 principals ( a must for COM, Services, ShortCuts and so on anyways ) but for content/data files we actually do a 1 to many with no key file approach to cut down on our number of components. Sure, that means we won't be able to create an MSP that services just one or two content files here and there but for our business needs that's simply not important to us.
Resilency was a bit of a 4 letter word to us so having less key files makes us happier anyways. :-) BTW, VDPROJ also makes every registry key a keyfile of it's own component and that was quite painful for us triggering unneeded repairs.
All of this aside, for anyone who doesn't fully understand all of this, I'd stick to the 1:1 pattern until you come across a situation where you don't want to anymore and you understand the impact of making that choice.
I know there are many questions out there regarding this same information. I have read them all, but my brain is all turned around and I don't know which way to go. Plus the lack of documentation really hurts.
Here is my scenerio. We are trying to use WIX to create an installer for our application that goes out to our dealers for our product information. The app includes about 2000 images and documents of our products and a SQL CE database that are updated via Microsoft Sync Framework. The data changes so often that keeping these 2000 as content files in the app's project is very undesirable. The app relies on .NET Framework 3.5 SP1, SQL Server CE 3.5, Microsoft Sync Framework 1.0 and ADO.NET Sync Services 2.0.
Here are the requirements for the app:
The dealers will be given the app on a CD every year for any updates (app or data updates).
The app must update itself from the internet to get any new images, documents or data.
The prerequisites must be installed if they do not exist on the client machine.
The complete installer should be generated from an MSBuild script with as little human interaction as possible (we don't want to be manually updating the 2000+ file list).
What we have accomplished so far is that we have a Votive project in our solution. We have manually specified the binaries in a .wxs file. Web have modified the .wixproj file to use the HeatDirectory task to gather our data (images and documents and database) from a specified location (This is broken and giving an ICE38 error). This seems all right, but still is a lot of work. We have to manually update our data by running the program in release mode and copying it to the specified directory.
I am looking to see what other people would do in this situation.
How would you arrange your solution with regards to the 2000+ data files? Would you create a custom build script that gets the current data from the server or would you include them as content files in the main project?
How would you get WIX to include all of the project output (including the referenced assemblies) and all of the data files? If you have any complete samples, that would be great. All I have found are little clips here and there and not an entire example from start to finish.
How would you deal with the version numbers? Would you put them as a constant in the build script and reference them through the $(var.VersionNumberName)? Would you have the version number automatically picked up from the project being deployed? If so, How?
If there is any better information than what I am finding, please include. I have read numerous articles, blogs, Stackoverflow questions, the tuturial, the wiki, etc. Everything seems to be in bits and pieces. The tutorial is nice, but doesn't explain anything about MSBuild and Votive. I would like to see a start to finish tutorial on using MSBuild and Votive and all the WIX MSBuild targets. If no one knows of a tutorial like this I may put one together. I have already spent the entire week gathering info and reading. I'm new to MSBuild as well, so if anyone has any great articles on MSBuild, please include them.
The key is to isolate the different types of complexities into separate merge modules and put them altogether into an MSI as part of the build. That way things that change often can change without impacting things that hardly change at all.
1) For the data files:
We use Paraffin to generate the WiX and hence the merge modules for an html + Flash based help system consisting of thousands of files (I can't convince the customer to go to CHM).
Compile these into a merge module all by themselves.
2) Assemblies: assuming that this is a set that changes less often just make a merge module by hand or with WixEdit with the correct files and dependencies.
3) For the version number there a lot of ways to manage this depending on your build system. The AssemblyInfoTask is pretty straight forward way to make sure all your assemblies are versioned appropriately. The MSBuild Extension Pack has some versioning stuff if you are using TFS.
I had a similar scenario and was unable to find a drop in solution so ended up with the following:
I wrote a custom command line program called wixgen.exe for generating wxs manifest files. It is pretty specific to our implementation in that it only knows how to create 2 types of wxs files. One for IIS Website/Virtual Directory deployments and another for Windows Service deployments.
Each time a build is triggered by our continuous integration server a post-build task runs wixgen with the right args to generate a new manifest.wxs for the project being changed. It automatically includes all the files needed for the deployment. These builds also version the dlls using a variation of the technique at: http://richardsbraindump.blogspot.com/2007/07/versioning-builds-with-tfs-and-msbuild.html
A seperate build which is manually triggered is then used to build the wixproj projects containing the generated wxs files and produce the msi's.
I would ditch the CD delivery (so 90's) and got with ClickOnce. This solution seems to fit well since you already use the .NET framework. With ClickOnce you should be able to just keep updating the content of your solution and make updates available to your heart's content. Let me know if you need, sample ClickOnce deployment code.
You can find more ClickOnce information here.
Similar to dkackman's answer, you should seperate your build into several components, isolating build components to be built seperately.
I come from a mainly Java background, however for building MSIs and NET executables we use maven; with the 'maven-wix-plugin' plugin for building the installers, and using the NMaven plugin for compiling any NET code. However, as we're only performing very basic development in NET, with most development in Java, we don't need too much complexity from the NMaven plugin (which is probably a 'good thing' (TM) as it's only at version 0.17).
If you're a purely NET house, you could also look into Blydan (http://www.codeplex.com/byldan), which seems to be the focus of development there at the moment (it's the same team for NMaven and Byldan).
If you do want more information on NMaven or Byldan raise another question and I'll give as much info as I can (which is not a huge amount, as stated I only do very limited NET development).