MSI Major Upgrade overwriting rules - wix

Question

I think I read it somewhere, but couldn't find the source now and can't confirm it: when install(Major upgrade) a newer version from MSI, if the file has been modified (either by installer or user), the default rule is that old file wouldn't be replaced by the same file from a new version?
I think also I observed that behavior in the installer I wrote before, but now after a few changes, seems that it will always replace the old modified config files!
Product definition:
<Product Id="*" Name="$(var.ProductName)" Language="1033" Version="$(var.ProductVersion)" Manufacturer="Advanced Software Solution" UpgradeCode="$(var.UpgradeCode)">
<Package Id="*" InstallerVersion="200" Description="The web service installer" Compressed="yes"
InstallScope="perMachine"/>
<MajorUpgrade DowngradeErrorMessage="A newer version of [ProductName] is already installed." />
Component Definition:
<Component Id='WebConfigComp' Guid='GUID'>
<File Id='WebConfigFile' Name='Web.config' Source='$(var.TheWebService.WCF.TargetBinPath)\Web.Distribution.config'
KeyPath='yes'>
</File>
</Component>
InstallExecutesequence
FindRelatedProducts 25
AppSearch 50
LaunchConditions 100
ValidateProductID 700
myScripts_CA 799
CostInitialize 800
FileCost 900
CostFinalize 1000
MigrateFeatureStates 1200
InstallValidate 1400
RemoveExistingProducts 1401
InstallInitialize 1500
BackupCA Installed 1501
ProcessComponents 1600
UnpublishFeatures 1800
SchedSecureObjectsRollback_x64 VersionNT > 400 1801
RemoveFiles 3500
RemoveFolders 3600
CreateFolders 3700
InstallFiles 4000
InstallServices VersionNT 5800
SchedSecureObjects_x64 NOT REMOVE~="ALL" AND VersionNT > 400 5801
ConfigureIIs NOT SKIPCONFIGUREIIS AND VersionNT > 400 5999
RegisterUser 6000
RegisterProduct 6100
PublishFeatures 6300
PublishProduct 6400
InstallFinalize 6600
LunchWCFReadme NOT Installed 6601
Update: I just created a new project for testing, the same behavior observed (the modified file is replaced by the newer version of installer) without change the default InstallExecSequence. Which probably means even though the File Versioning should applies, but it not actually kicked in to affect the result would expected as Remove of old version happened too early be default as Glytzhkof and PhilDW pointed out.
I am using Wix 3.8, the current stable, did I missed something?
Update2:
So far, I can confirm that moving RemoveExistingProducts after InstallFiles will keep the modified unversioned files. But the problem is that seems MajorUpgrade conflict with
<InstallExecuteSequence>
<RemoveExistingProducts After="InstallExecute" />
</InstallExecuteSequence>
I am adding, and the error message is
Error 1 Duplicate symbol
'WixAction:InstallExecuteSequence/RemoveExistingProducts' found. This
typically means that an Id is duplicated. Check to make sure all your
identifiers of a given type (File, Component, Feature) are
unique. C:\TestDev\MySetupTest\MySetupTest\Product.wxs 5 1 MySetupTest
which is not very helpful either.
Final Update:
After digging the web thing for a while, find out what the issue is:
By default, MajorUpgrade schedules RemoveExistingProducts after
InstallValidate. You can change the scheduling using the Schedule
attribute. For example, If you choose to schedule it after
InstallInitialize, it will look like the following:
<MajorUpgrade
Schedule="afterInstallInitialize"
DowngradeErrorMessage="A later version of [ProductName] is already installed. Setup will now exit.">
Source: Wix Toolset website
So including of MajorUpgrade will indeed change RemoveExistingProducts sequence for you, which is a useful feature to have, but was unexpected for me. Thanks for all the help, now things starting make sense to me. An happy ending after all!

Answer 1

When a major upgrade uninstalls an existing installation before the new version gets installed (RemoveExistingProducts before InstallInitialize) it will normally remove all files that were originally installed - this includes files that may have been modified. Then the new version is installed with a fresh bundle of files.
If you schedule RemoveExistingProducts after InstallFinalize the install of the new version's files precedes the removal of obsolete files. In this scenario files are only replaced if they are versioned and newer than installed files, and for unversioned files like txt, pdf, etc... the file replacement rules basically states that the file will only be overwritten if it has not been changed on disk.
It follows that moving RemoveExistingProducts after InstallFinalize may solve your file "replacement problem" which is really a case of the modified files being deleted during uninstalland reinstalled by your current upgrade strategy.

Answer 2

As pointed out, your REP is early in the sequence, so it's essentially an uninstall of all the older product followed by an install of the new one. This is one type of major upgrade.
The other type of major upgrade is when REP is at the "end". In this case the new product is installed on top of the older product, and that will follow file replacement rules, that's the part that's relevant to your question.
This might be useful:
http://msdn.microsoft.com/en-us/library/aa370531(v=vs.85).aspx
and there's different rule if the unversioned files have hash checking. Yes, unversioned means "without a file version in the file's resources".

Answer 3

Behaviour what you are saying is unversioned file replacement logic.
Placing config file in its own component and marking the config file as the key path of the component will ensure that Windows Installer will use unversioned file replacement logic when deciding whether or not to replace this file in the install of your new version.
This post can help you to achieve the same.

Answer 4

I can't comment because of lack of reputation, but if you want to add
<InstallExecuteSequence> <RemoveExistingProducts After="InstallExecute" /> </InstallExecuteSequence>
and if you got duplication error, you should try this instead;
<MajorUpgrade DowngradeErrorMessage="A newer version of [ProductName] is already installed." Schedule="afterInstallExecute" />

Answer 5

My guess is that the change in behavior is because you have moved RemoveExistingProducts early in the InstallExecuteSequence.
A major upgrade is essentially running an uninstall of the original product, and then a reinstall of the new one. If RemoveExistingProducts is late in the InstallExecuteSequence - and component referencing is done correctly - the existing product is not uninstall first, but its "leftover" components are removed after installing the new product. It works like a "diff" if you like. This will preserve installed files on disk that are not removed in the new version. I hope that was comprehensible - a lot at once, and rushed.
Here is a thread that deals with ensuring unversioned files get updated during an update: Forcing an upgrade of a file that is modified during its initial installation