Registering COM EXE with WIX - wix

Question

I created a solution with two new projects: ATLProject1 which is a COM dll and ATLProject2 which is a COM EXE. To both projects added same simple class with single method.
Added both to WIX setup project (this is an existing setup project that already installs other COM dlls. I'm just using it for this test).
Added following lines to product.wxs:
<Component Id="ATLProject1.dll" Guid="{...}">
<File Id="ATLProject1.dll" Name="ATLProject1.dll" KeyPath="yes" SelfRegCost="0" Source="$(var.TargetDir)\ATLProject1.dll" />
</Component>
<Component Id="ATLProject2.exe" Guid="{...}">
<File Id="ATLProject2.exe" Name="ATLProject2.exe" KeyPath="yes" SelfRegCost="0" Source="$(var.TargetDir)\ATLProject2.exe" />
</Component>
and also
<ComponentRef Id="ATLProject1.dll" />
<ComponentRef Id="ATLProject2.exe" />
The file also has these lines:
<EnsureTable Id="PublishComponent"/>
<EnsureTable Id="Condition"/>
<EnsureTable Id="TypeLib"/>
<EnsureTable Id="Class"/>
<EnsureTable Id="Extension"/>
When running the setup I get error: "Module ATLProject2.exe failed to register. HRESULT -2147024769" (hex 0x8007007f the specified procedure cannot be found).
If I remove ATLProject2 from setup, it succeeds and ATLProject1 is correctly registered in registry (this is without generating registry information e.g. using heat, it just works).
Should exe component be treated differently?
I found this 10 year old post suggesting heat.exe does not treat COM exe as COM. If this is the problem, Not sure if this is still the case?

Answer 1

The Windows Installer does not recommend using SelfReg to register at install time. Instead, adding the registration to your .wxs code or capturing the registration at build time is highly recommended.
To add the registration manually, you don't use EnsureTable, you use the COM related elements (like Class, ProgId, TypeLib). It can be tedious but will be far more robust than trying to selfreg during installation.
Unfortunately, the alternative to capture the registration during build using heat.exe (provided in the WiX toolset) does not support capturing from executables. If you are open to a commercial solution, we (FireGiant) developed an alternative to heat.exe that can capture executable registration (and much more). That advanced harvesting solution has more documentation on the FireGiant site.

Answer 2

RegServer Switch: COM EXE files are normally self-registered via the /RegServer switch as in:
MyBinary.exe /RegServer
In other words EXE files are not registered via the normal regsvr32.exe mechanism. This is the tool used to register COM dll's and OCX files, but it does not handle EXE files. There is also an /UnRegServer switch to unregister EXE COM files - for the executables that support /RegServer (which is not all COM EXE files - it could be missing as a feature).
Self-Registration: Self-Registration is not ideal to register COM files, and here is a write-up of why this is the case: MSI register dll - Self-Registration considered harmful. In MSI one extracts the COM registry data and populates a number of special COM-tables to allow the registration of the COM server in a way that supports advanced features such as rollback. I don't like the COM extract either (risk of self-repair problems, more on self-repair problems), but it helps in most cases - especially when there are dependencies that can trigger registration problems. Moreover it is the way COM files are supposed to be registered in MSI. It is the standard. I should note that some COM settings go into the Registry table still - since there are no dedicated COM-related tables for them.
heat.exe: WiX's own heat.exe tool now can extract COM data from dll files and ocx files (32-bit). But it does not seem to work for EXE COM files - I am not sure why:
heat.exe file MyCOMDll.dll -out MyCOMDll.wxs
RegSpy2.exe: There is a tool you can use to extract COM registration information from both DLL, OCX and EXE files. It can be downloaded from here: http://www.installsite.org/files/iswi/RegSpy2.zip. Here is the main page listing numerous tools. The RegSpy tool is written by Phil Wilson - MSI Expert and author of The Definitive Guide to Windows Installer (APress).
Here is how to extract the COM data from a COM executable (if you get no data, try unregistering the file first and then running regspy.exe):
RegSpy.exe MyBinary.exe /RegServer >> RegistryOutput.reg
The exported *.reg file can then be converted to WiX elements. This is not an easy process. Using the WiX tool heat.exe does not populate the proper COM tables, but rather puts everything in the Registry table (which will work though, barring MSI validation errors):
heat.exe reg MyCOMRegistryData.reg -out MyWiXFile.wxs -sfrag -suid
There used to be a tool called Tallow that converted reg files to WiX COM registration, but this tool is no longer anywhere to be found. I am not aware of any other ways to generate it short of writing it yourself, or download another deployment tool and import the COM data or extract it and decompile the generated MSI with dark.exe and take out the WiX markup. Or figure out how heat.exe writes its WiX XML output with COM data and adapt that to process the output from RegSpy.exe.
UPDATE: Throwing in a link to Paraffin: https://github.com/Wintellect/Paraffin. This is supposedly a "better Tallow". I am not sure what it supports in terms of COM-extract. My quick test seemed to indicate it doesn't support COM extraction at all, but supports auto-generating WiX markup and add and remove files for updates.
Custom Actions: It is possible to register your COM EXE by means of a custom action that calls the /RegServer switch as well, though this is not recommended for all the reasons listed in the link above (self-registration considered harmful).
Some Links:
Adding a .reg file to registry WIX
How to generate WiX XML from a .reg file?