How can I determine what causes repeated Windows Installer self-repair? - wix

Question

How can I log only the changes causing a MSI file made by Installshield 2008 to reinstall via "self-repair"?
What is the reason behind self-repair?
How do I disable self-repairing of MSI using Installshield 2008?

Answer 1

Self-Repair, Simple & Short Explanation: Why does the MSI installer reconfigure if I delete a file?
Alternative Answer Available
UPDATE:
There is a shorter, more "solution focused" answer available, perhaps try it first. This answer focuses on "understanding self-repair" rather than explaining the steps to take to eliminate the problem. You might want to read the first section of this answer as well.
Unexpected Windows Installer self-repair issues - Quick Fix?
This "article" has gotten large and somewhat unreadable. Here is a newly written preamble - the short "workaround version" for fixing unexpected self-repair (often found in VB6, Visual Studio, MS Office, MS Outlook, AutoCAD, etc...)
If you experience unexpected self-repair, the first thing you can try is to manually create a desktop shortcut directly to the application executable you are launching when the problem occurs. This bypasses the most common trigger of self-repair, "the advertised shortcut". If this works your problem is "solved" (or avoided). Here is a quick, fleshed-out explanation
If the problem still occurs, or your problem is related to the loading of an MS Office, MS Outlook add-in or similar (that you can't launch via a shortcut), then you most likely have a COM registration conflict on your system, and the fix is much more involved. The easiest to try is to disable any addins you don't need in the addins dialog of the application in question and see if this makes the problem go away
If you still see problems, then you most often need to debug a genuine COM registration conflict (or conflicting file/MIME associations, or command verbs). This normally involves (at least) two conflicting applications on your system that "fight it out" updating the registry on each launch after the other application has run (always launching one of the apps will not trigger the self-repair - the conflict surfaces when you alternate between applications). It is also possible that permission problems cause the same application to fail to update the system and it keeps trying endlessly by repeatedly running self-repair. And there are further possibilities, more details below
The "real fix" is to contact both application vendors and ask them for a fix for the problem (since a fix often requires a fix of both vendor MSIs), but in my experience this is rarely successful. Do try it out though - since this is the way to help everyone with a long-standing annoyance! I have personally provided a setup with fixes for a bank deployment and was very happy to have the problem solved in my package
To debug yourself, you need to get hold of a tool to open cached MSI files on the system and you need to "hack" the database - a very involved task requiring expert skills, you would be advised to seek an installation expert for help if the problem is very serious for your desktop environment. It can work, but don't expect miracles.
Please see the section below called "Finding the trigger or culprit for the self-repair" for more details on obtaining a tool to view and modify MSI files
The rest of the "article" describes self-repair problems in depth. There are many other potential causes of self-repair than what is described in this "short" section.
Overall Problem: Developer debugging and self-repair
Windows Installer is a deployment technology, its job is to install the specified files and registry settings and keep them in the specified install locations and to ensure they are the right versions - self-repair or resiliency is a mechanism to that end. Its operation conflicts with a developers need to exchange files on the fly for debugging, development and testing.
Accordingly, many self-repairs (resiliency) are triggered simply by developers trying to debug their installed application and hot-swapping files on the fly. See section 2 in "Some typical self-repair problem scenarios" below for how to handle this. In other cases there are genuine design errors in the MSI that must be corrected or system administration pitfalls that lead into self-repair - and at times the error source can be hard to find.
I have written about the issue of self-repair in an answer on serverfault.com. Slightly different words intended for system administrators, and reading it now it might be a more accessible explanation than this long one (intended for developers). There is also another, shorter answer here on stackoverflow: Why does the MSI installer reconfigure if I delete a file? (this is probably the shortest one and easiest to understand). And finally I found a very nice article on self-repair by Vadim Rapp: How to fix Windows Installer Efforts to Self-Repair. This article is well worth a read.
No self-repair will occur if Windows Installer determines that the product being launched is properly installed. When self-repair occurs something needs to be changed on the system for the application to run properly.
The Primary Causes of Self-Repair
The details are presented below in the section "Some typical self-repair problem scenarios", but as a quick, foreshadowing list - the primary causes are:
1. Badly packaged corporate MSI files or MSI design flaws from the vendor (the MSI package itself is badly designed and triggers self-repair unexpectedly for a variety of reasons)
Excessive or erroneous use of per-user files or per-user registry keys often with erroneous key paths set into the user profile (instead of HKCU). See section 5 below for more details (and a color illustration of such a situation)
Package interference from erroneous COM server registration (particularly VB6 COM files or VBA files and libraries from products such as AutoCAD from Autodesk, and similar products).
Two MSI packages register the same COM file (ActiveX/OCX) from two different locations and "self-repair fight" on each application launch to keep their version correctly registered.
The last application to launch puts the registry right for itself, and it lasts until the other application is launched and does the same. Once you alternate between applications the problem occurs. See section 7 below for a lot more VB / COM self-repair detail
A component key path is set to an empty folder that Windows installer removes on self-repair (triggering an endless loop of removal and subsequent self-repair)
ACL lockdown permission problems (logged on user can not access key file and Windows Installer triggers repair repeatedly). This can also be caused by ACL changes done externally, but is often done by the MSI itself
Here is a serverfault.com work-in-progress describing common MSI design flaws
2. Files or registry keys are deleted by interference from external causes ranging from (logon) scripts to standard OS features, viruses, security software, etc...
Temporary files are deleted automatically by Windows after being erroneously installed to the temp folder by an MSI package
Interference from bad logon- and trigger happy cleanup scripts and cleanup applications
Antivirus applications blocking or deleting files or registry keys so that Windows Installer can no longer detect or access them
Computer viruses changing or deleting files and registry settings
Overactive computer tinkerers and users delete files and settings they don't understand
3. Windows design changes, flaws or restrictions that causes flawed or problematic deployment
An AD-advertised MSI package fails to install (might be cancelled since it takes too long to install) and keeps bugging people. This is strictly speaking not self-repair but an advertised install that is aborted, but the result is the same: endless reinstall
Terminal server complications. Self-repair is generally disabled altogether on terminal servers. This does not normally cause self-repair problems, but application installs without the required per-user files or registry keys that can be added via benign use of self-repair (read below). The user files and user registry keys are then just missing and problems result
UAC interference, certificate validation failure and other problems resulting from Windows design changes. For every version of Windows security features like these are added and normally end up adding new obstacles for reliable deployment
Even certain Windows Updates (updates, security updates, hotfixes, etc...) can make drastic changes to how security is enforced for MSI packages, and hence cause extremely problematic behavior
Though this relates to MSI creation, and not primarily their end-user use, the Windows Update KB3004394 which updates the way Windows checks for revoked root certificates, breaks older version of Installshield's command line build (for setups that were digitally signed). Largely a resolved issue by now, but an illustration of how Microsoft keeps changing core MSI functionality
In a similar manner Installshield crashed for many users after installing Microsoft update MS14-037 “Security update for Internet Explorer versions 6, 7, 8, 9, 10, and 11” (KB2962872)
An extremely problematic change in Windows Installer base functionality occurred after installing kb2918614 (Vista). Suddenly administrator credentials were required for a simple MSI repair operation. This defeated a core benefit of MSI altogether: the ability of regular users to run approved installs with temporary admin rights. There were also other reported MSI problems after installing that fix. It appears another Windows update fixed the issues: kb3008627 (later replaced by kb3072630)
About Self-Repair
Windows Installer is designed to install your application's binaries, settings- and data files and keep them installed and ensure they are the right versions. Self-repair is a mechanism to that end. The overall concept is called resiliency - i.e. a broken installation triggers a self-repair before the application is launched.
Resiliency, or self-repair, is a built-in primary concept of Windows Installer and can not be turned off in any way that is safe. People do the most incredible things sometimes, such as disabling the whole Windows Installer engine to stop their self-repair. This must obviously never be done. The cause of the repair must be identified, and the problem resolved rather than creating new ones, or hacking the system.
Every time you launch an advertised shortcut (essentially a special shortcut that points to a Windows Installer feature and not directly to a file), Windows Installer will verify the installation by checking the "component key paths" for your product. If a discrepancy is found, a repair is triggered to correct the incomplete installation. The "component key paths" are the "key files" specified for the components inside your MSI - there is one per component. Self-repair can also be initiated by someone instantiating a COM server (or attempting to), someone activating a file via its file extension or MIME registration, and a few other ways. Here is a comprehensive article from Symantec on the subject of "self-repair entry points": Initiating Self-Repair and Advertising Features with Entry Point.
If files are deleted, moved or simply overwritten (manually by a user or somehow automatically), self-repair may result (if the file or registry setting isn't set as a key path self-repair is not triggered).
Finding the trigger or culprit for the self-repair
The trigger for the self-repair is generally possible to find in your event viewer on the system where the self-repair took place. Follow these steps to open the event viewer:
Right click "My Computer"
Click Manage
Click continue if you get an UAC prompt
Go to the Event Viewer section, and check the Windows Logs
Alternatively you can do: Start => Run... => eventvwr.exe for just the event viewer. If you don't see run in the start menu, press WINKEY + R.
Look in the "Application section" of the event log and you should find warnings from the event source "MsiInstaller" with IDs 1001 and 1004
In the sample screen shot above the product code is shown inside the red box
In order to determine what product the product code is for, you can look up the product name via the procedure explained here: How can I find the product GUID of an installed MSI setup?
If you actually want to go in deep and check the actual content of the MSI file, you must get hold of a tool capable of viewing a MSI file (such as Orca, Installshield, Advanced Installer or similar). You then open the package listed in the "LocalPackage" path listing as illustrated in the screen shot found in the answer linked to in the previous bullet point.
The actual modification of the system-cached MSI file and/or the registry to remove advertised entry points such as (advertised) shortcuts, COM registration, file associations, MIME associations or command verbs is a specialists job. It is very involved and not good practice, but it is the only "last resort" that I know about.
Finally it is possible for an application to explicitly call Windows Installer itself to trigger self-repair for shared components - for example a spell checker. I believe a few versions of Microsoft Access did this, and this behavior can not be changed or worked around as far as I know.
MSI-expert and MVP Stefan Krüger has an article about the same self-repair issue. And he crucially discusses the actual event log entries and what they mean. Please read about the actual debugging procedure there.
Some typical self-repair problem scenarios:
This is the "verbose explanation" of several self-repair problem scenarios already outlined in the overview above.
A component key path is set to an empty folder that Windows installer removes on self-repair (triggering an endless loop of removal and subsequent self-repair). This is solved by adding the folder to the CreateFolder table instead (Wix equivalent). In my experience this is the most common scenario for unwanted self-repair. Very common.
Many self-repair problems are actually caused by developers trying to debug their applications by replacing files on the fly, deleting files or renaming them. Or they may use cleanup registry scripts and / or batch scripts to unregister and register COM files, COM-Interop, GAC files, file associations, or other common developer debug and development tasks.
This hot-swapping can triggering self-repair when the application is launched via an advertised shortcut.
A top tip for developers struggling with self-repair during application debugging is to not launch the application from an advertised shortcut, but to launch the main EXE directly from Windows Explorer or from a manually created shortcut. This will bypass the most common "self-repair entry point" - the advertised shortcut. Self-repair may still result from broken COM data, advertised file associations and a few other special cases (read this Symantec article for entry point information).
Other applications or rather other MSI packages can break your installation and cause self-repair by interfering with registry data - typically COM settings, but also with other settings and files. These can be some of the hardest cases to solve, since the applications are basically fighting it out and the last one to run will update the registry each time. Typically both MSI files must be redesigned for the applications to operate on the same machine. Or, as is the order of the day, the whole application may be virtualized (for example: Microsoft App-V virtual packages) and run in its own sandbox which seems to be what is done more and more in companies these days. This error scenario is often seen with a suite of badly repackaged applications in a corporate environment. COM fragments from different packages overwrite the COM server's disk path from another package, and self-repair fighting ensues on each application launch via an advertised shortcut. The same file name with different file versions can also be registered from different file locations and share some registry settings that interfere. As far as I recall at least 7 variables or settings in the file system and registry must be in sync for a COM server to be properly instantiable. See section 7 below for a more specialized description of COM interference in the context of VB6 and VBA COM applications.
A component key path is pointing to a temporary file that has been deleted by the application or it will be deleted by the system eventually via some sort of cleanup mechanism (can also be a cleanup tool such as ccleaner). This is common for files in the temp folder itself. This is solved by not installing the temp file, or putting the file somewhere else and making it permanent. I have seen this error most often in the world of corporate application repackaging where a faulty cleanup of the captured image leads to the install of a temporary file that should not have been included in the package at all. Often they may be temporary files waiting for a reboot to be installed to their intended, perhaps protected location, and the reboot was never performed - a common application packaging error. To a lesser degree I have seen it in auto-generated packages coming out of automated build systems.
Permission problems: if a key file for a component is installed to a location that is not accessible for the user who invokes the application. Windows Installer might not "see" the installed file / key path, or be unable to add the file to the folder. These issues can be more exotic to debug, and may not happen that often. There are several variations on this issue:
An example of this is when you install a file to a %USERPROFILE% path and then forget to set a HKCU registry keypath, and instead set the keypath to point to the %USERPROFILE% folder/file. This generally yields an inaccessible hard coded key path that is user-specific: C:\Documents and settings\user1\Desktop. This path will not be found for another user logging on, and self-repair runs in circles. Here is a color illustration.
Another example is key paths set to folders that are not writeable for the System account. This may seem exotic but can result from the MSI's faulty modification of system ACL entries, or from a strange system administrator security setup, or any other non-standard ACL / Security Descriptor.
Another class of self-repair problems emerges in relation to terminal servers and Citrix. The whole windows installer service could be locked down so any self-repair invoked to add per user data could fail and consequently self-repair may fail or more likely not run at all. This is reason enough to not rely on self-repair as a way to add user data like some MSI files do, and such constructs must be replaced with application deployment of user files copied from per-machine locations or the less effective ActiveSetup feature from Microsoft that runs once per user.
VB6 applications and VBA applications, which are heavily COM based with massive potential for COM interference (COM settings overwriting each other and becoming inconsistent), have been known to trigger several mysterious self-repair problems, most of which have not been properly explained. This can also happen on launch of Visual Basic 6 (VB6) or Visual Studio (and many other applications). The common denominator is that some error in the current installation state triggered the self-repair, and you can track down the culprit product and component by following the steps outlined in the section above called "Finding the trigger or culprit for the self-repair". Be sure to report your findings here (I never use VB6 or VBA anymore - your detailed findings could help others with a long-standing annoyance).
Though I have never debugged such VB6 issues in great detail, it would seem that the problems result from applications that install common controls, VB6 COM files, templates and VBA files and libraries that conflict with existing files and registry settings and registrations on the box, or some per-user registry key or userprofile file may need to be added once per user (allow the self-repair to complete once and see if the problem goes away). In particular I have heard of these mysterious self-repair problems when launching AutoCAD (from Autodesk), Visual Basic 6, and several other products (often with VBA automation available in the tool).
Some applications even erroneously install bits and pieces from the VB6 runtime on their own causing these settings to be "ripped out" on uninstall of those applications. This can certainly cause self-repair to be triggered to fix the now (partially?) broken VB6 runtime. There are several variants of this problem, and the "catch all" solution is probably a complete uninstall and reinstall of the VB6 runtime. Here is a description of a very common "specific" problem involving a few COM registry keys. It nicely illustrates what happens in this scenario.
If you experience unexpected self-repair when launching VB6, AutoCAD, Visual Studio or other products, you can first try a workaround to prevent these unexpected self-repairs from happening in the first place (this doesn't solve the problem, but may bypass its symptoms): why does windows installer start up everytime i start up visual basic 6
See my comment to the question in this topic for one of the most typical VB6 style self-repairs: Why does my application triggers the Installer of another application? (ActiveX control registered twice from two different locations on disk).
In my opinion the "general fix" - that should always work - for VB-COM self-repair issues, is to get the vendor to update their project in question to use the latest official and properly installed and shared ActiveX control / OCX available, and not to rely on their own version installed redundantly and registered in the wrong location.
A special case of Windows Installer repair or self-repair that is worth mentioning for completeness, was the issue with Microsoft Office several years ago where a self-repair would be triggered, and you would be asked to insert the Microsoft Office installation media (in those days CD-ROMs or DVDs - today maybe thumb drives). As far as I recall this was related to an erroneous call to the built in Windows Installer standard action "ResolveSource" which unexpectedly (and unnecessarily) triggered the prompt for the installation media. A very common support call back in the day and mentioned here for completeness. It is important to note that this problem can still occur whenever MS Office is installed from any removable media (rather than the better option of a network share). This happens when MS Office detects that it needs to install further, optional (and usually shared) components of the product that were not installed originally. For example unusual spell-checkers, various templates or specific and rarely used tools. It is possible to install these components to "install on first use" (advertised features is the proper Windows Installer term).
There are many other possible scenarios. To mention a few:
a bad logon script may delete things on the system and trigger self-repair
an AD advertised package may fail to install and keep bugging people
two applications may start fighting for the same file associations
computer tinkerers and hackers can manually delete data that trigger self-repair
anti-virus can quarantine files and registry settings that trigger repair
a virus can change or delete things and trigger self-repair
a disk and registry cleanup tool such as ccleaner can delete files and trigger self-repair
and no doubt numerous other scenarios...
Benign uses for self-repair
Finally there are benign uses for self-repair that happen once and do not constitute errors. This is the legal and proper use of self-repair though it may be just as annoying as the design errors, and with user intervention they can pop up again and again:
Self-repair is sometimes used to add per-user data to HKCU and the user profile. This design mostly works, but gets worse for every version of Windows as new obstacles are put in place for deployment. For one thing self-repair typically does not work at all on terminal servers rendering the setup incomplete. Though it is besides the point for this discussion, it is better to have the application copy files to per-user locations. Another issue is UAC. Other problems show up with each new Windows version and even with some Windows Updates as described above (virtual folder redirects, certificate prompts, previously non-existing target path restrictions etc...).
When self-repair is needed to set up user data, it may take so long that the user aborts it and keeps doing so. This causes the self-repair to reappear all the time until it is allowed to finish. A common support call.
It is also possible to install a product with "advertised features" that are designed to be installed "on demand" triggered during application use. Few applications use this, but when it is used a lengthy "self-repair style" installer may run - pulling down the required files and settings. If this process is cancelled the installation of the feature is rolled back and it can be triggered again. This install can be slow for several reasons:
If the installer used large, compressed CAB files that are first downloaded and then extracted locally on a slow disk where the anti virus starts scanning the whole cab and then each extracted file the operation can take a long time.
The operation can also be slow if the network connection is wireless and there are lots of small files to download (high latency), and again the anti virus could slow down things.
If installed from removable media, you could get prompts to insert the source media to allow files to be copied. A very common support call if removable media is used in an office environment (it shouldn't be - use an admin install on a network share)
Etc...