How can I get MIDL to search additional include directories for qualified paths - relative-path

Question

Update: Just over six months after opening a support call to Microsoft it has been rejected, they claim it is not a bug (since the documentation doesn't explicitly say that the behaviour seen isn't the correct one). They rejected the DCR saying that since they haven't heard any complaints in the last 10 years this is obviously not a common use case.
This is a call to arms, if you have run into the same issue, please open a support call with Microsoft so they understand that it should be fixed. I know that at least someone has run into the same problem becasue I found this comment in Chrome's source code:
# Building .idl files.
# This is a total mess. MIDL needs to be run from $OPEN_DIR because it's too
# stupid to apply its include paths to a relative path like "ui/ie/bla.idl"
# (it only looks in the current dir). So we have to jump through hoops to fix
# up our relative include paths and output files.
Original question:
I have the following file structure:
C:\first\Foo.idl
C:\second\Bar.idl
Where Bar.idl contains the following line:
import "first/Foo.idl";
How can I get midl to compile Bar.idl when compiling from C:\second?
If I imported Foo.idl directly (without specifying first/) then specifying first as an additional include directory would be enough (midl /I c:\first Bar.idl) and it would find Foo.idl
Alternately if I compiled from C:\ (midl second\Bar.idl) that would be OK too.
The problem is that when compiling from within C:\second with the command line midl /I C:\ Bar.idl, I get the following compilation error:
c1 : fatal error C1083: Cannot open source file: 'first\Foo.idl': No such file or directory
It looks like midl is willing to search relative paths only if they are relative to the current directory and not to one of the specified additional include directories and uses the additional include directories only for unqualified file names, this behaviour is specific to the import keyword, when using include the results are as expected.
I would like to be able to add two different additional include directories so that if I have the file on my local machine midl will take that version, otherwise it will take the file from the server (so chdiring to the root folder is not an option).
Is there a way to get around this?

Answer 1

Its the end of 2020 and MIDL 3.0 is out. However, the problem described by the OP still persists. But if you're using Visual Studio there is a straight forward way to deal with that issue.
If you're adding an .idl file to a project the following MSBuild code gets generated in the project file to which the .idl file is added:
<ItemGroup>
<Midl Include="Folder1\YourCustomFile.idl" />
</ItemGroup>
If you add a second file in another folder and reference the first one this will generate another entry:
<ItemGroup>
<Midl Include="Folder1\YourCustomFile.idl" />
<Midl Include="Folder2\YourSecondCustomFile.idl" />
</ItemGroup>
The problem is that if you compile that code the MIDL compiler will not be aware of any additional include directories. Hence, if you add
#include "YourCustomFile.idl"
at the beginning of YourSecondCustomFile.idl the MIDL compiler will not search Folder1 for any .idl files to be included and compilation will fail.
However, by adding AdditionalIncludeDirectories MSBuild item metadata you can influence which folders are passed to the MIDL compiler as additional include directories.
So, to instruct the MIDL compiler to search Folder1 for include files when compiling YourSecondCustomFile.idl modify the MSBuild code as follows:
<ItemGroup>
<Midl Include="Folder1\YourCustomFile.idl" />
<Midl Include="Folder2\YourSecondCustomFile.idl">
<AdditionalIncludeDirectories>$(ProjectDir);$(ProjectDir)Folder1</AdditionalIncludeDirectories>
</Midl>
</ItemGroup>
ProjectDir is an MSBuild property that points to the directory containing the current project (at least in C++ projects it does). The ; is used to separate different directories. Each of those directories will be passed to the MIDL compiler as a separate directory to search for include files.
This should resolve the compilation problem. I don't think the AdditionalIncludeDirectories item metadata can be added using Visual Studio's user interface so you should edit the Visual Studio project directly in a text editor.
Note that the item metadata is valid per item, i.e. for each individual MIDL file. So, you have to add AdditionalIncludeDirectories to every MIDL file that references other MIDL file.
If you need the same content in multiple AdditionalIncludeDirectories you could define a property somewhere else in the project file like this:
<PropertyGroup>
<AdditionalMidlIncludeDirectories>$(ProjectDir);$(ProjectDir)Folder1;$(ProjectDir)Folder2</AdditionalMidlIncludeDirectories>
</PropertyGroup>
And then you use that property everwhere. You could add the same AdditionalIncludeDirectories statement to every single MIDL which would ensure that the same include directories would be used for every MIDL compiler call:
<ItemGroup>
<Midl Include="Folder1\YourCustomFile.idl">
<AdditionalIncludeDirectories>$(AdditionalMidlIncludeDirectories)</AdditionalIncludeDirectories>
</Midl>
<Midl Include="Folder2\YourSecondCustomFile.idl">
<AdditionalIncludeDirectories>$(AdditionalMidlIncludeDirectories)</AdditionalIncludeDirectories>
</Midl>
</ItemGroup>
Edit:
As mentioned in the comment below the code can be further simplified by applying MSBuild's ItemDefinitionGroup. An ItemDefinitionGroup is used to add metadata to MSBuild items which means that the AdditionalIncludeDirectories metadata can be automatically added to each and every Midl element. The ItemDefinitionGroup is defined as follows:
<ItemDefinitionGroup>
<Midl>
<AdditionalIncludeDirectories>
$(ProjectDir);
$(ProjectDir)Folder1;
$(ProjectDir)Folder2
</AdditionalIncludeDirectories>
</Midl>
</ItemDefinitionGroup>
This simplifies the Midl ItemGroup as follows:
<ItemGroup>
<Midl Include="Folder1\YourCustomFile.idl" />
<Midl Include="Folder2\YourSecondCustomFile.idl" />
</ItemGroup>

Answer 2

As you note, while this is dumb, Microsoft Support have confirmed this is not a bug. The following are possible workarounds.
1. Use the /I switch. A lot.
Use the /I switch to specify both c:\first and c:\second, and specify import "Foo.idl" instead of a relative path.
If the command line becomes too long specify a response file.
2. Use symbolic links
Use symbolic links or junctions to the include directories to reference them all into a single hierarchy under a known directory. Then you can use paths relative to that directory.
A pre-build step might be used to maintain the symbolic links.
MKLINK.exe can create junctions or symbolic links.
3. Use an additional build step
Create an additional build step which copies the required files to known locations, then inport them from there.

Answer 3

This is a call to arms...
12 years, 6 months later, I'm here with my pitchfork, ready to go.
In the meantime, for an easier fix, you can go to Project Properties -> MIDL -> General -> Additional Include Directories. Setting that to "$(MSBuildProjectDirectory)\YourSubFolder" works just fine.