We have an in house compiler and linker tool that we're trying to make MSBUILD compatible (i.e. proper behavior in build/incremental builds/rebuild/clean scenarios)
In the first step we actually call CL task to preprocess our files. The issue is I can't seem to figure out how to set up the tasks properly to do that so it detects if the output is deleted or if one of the inputs is modified.
Second step is call our Compiler with it's proper parameters.
Third step is to call our Linker with it's proper parameters.
I think once step one works making step two and three will be simple; I'm stuck on step one. Example code below. The main MFT contains "#includes" which reference all the other MFT files named in _MFTFiles - so we only need to process the main file; but we need to monitor them all so if we change them incremental builds work properly. If anyone has any idea I'd love to hear it. I have the MSBUILD book and of course scoured here but I don't see an example of what I'm trying to accomplish.
Thanks in advance.
<ItemGroup Label="_MainMFT">
<MainMFT Include="MFTSystem.MFT"/>
</ItemGroup>
<ItemGroup Label="_MFTFiles">
<MFTFiles Include="MFTbject.MFT;DebuggerSupport.MFT;enumerations.MFT;collections.MFT;DataStream.MFT;Version.MFT"/>
</ItemGroup>
<Target Name="_PreprocessFiles"
BeforeTargets="Build"
DependsOnTargets=""
Inputs="#(MFTFiles)"
Outputs="#(MFTFiles->'%(Filename).MFTpp')">
<Message Text="PlatformToolsetVersion is $(PlatformToolsetVersion)" Importance="high"/>
<CL Sources="#(MainMFT)" PreprocessorDefinitions="_DEBUG;EL_DIAG_ENABLED" Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'" PreprocessToFile="true" PreprocessOutputPath="$(ProjectDir)%(Filename).MFTpp" />
<CL Sources="#(MainMFT)" PreprocessorDefinitions="" Condition="'$(Configuration)|$(Platform)'=='Release|Win32'" PreprocessToFile="true" PreprocessOutputPath="$(ProjectDir)%(Filename).MFTpp"/>
</Target>
<Target Name="_ObjectCompiler" AfterTargets="_PreprocessFiles;Build">
<Message Text="Calling ObjectCompiler...." Importance="high"/>
</Target>
<Target Name="_ObjectLinker" AfterTargets="_ObjectCompiler;Link">
<Message Text="Calling ObjectLinker...." Importance="high"/>
</Target>
Related
Say we have the following MSBuild project that defines a target which can be partially run:
<Project DefaultTargets="Foo">
<ItemGroup>
<MyInputs Include="**/*.json"/>
</ItemGroup>
<Target Name="Foo"
Condition="'#(MyInputs)' != ''"
Inputs="#(MyInputs)"
Outputs="#(MyInputs->'%(FileName).cs')">
<MyCustomTask FileToProcess="%(MyInputs.Identity)"/>
<ItemGroup>
<Compile Include="%(MyInputs.FileName).cs"/>
</ItemGroup>
</Target>
</Project>
The problem is that all items are included into ProcessedFiles; even these whose respective MyCustomTasks are not run, due to incremental building. Apparently, MSBuild always processes ItemGroups inside targets.
Is there a way to add an item inside a target, only when the respective target batch is run? I tried using CreateItem, because it is a task and might not get executed just like MyCustomTask, but it didn't work.
My specific problem was that when the source files had already existed, they were included twice in the Compile item, which raised a warning. It was then when I learned about the KeepDuplicates attribute that saved me.
But the question still stands.
I'm trying to create a reusable Target in msbuild, following the basic model outlined in How to invoke the same msbuild target twice?
I'm stuck trying to pass a property that I want interpreted as a list. I haven't found an example online that deals with this situation. As I understand it, the problem is that Properties is already treated as a list item, so it doesn't like having a list item passed in as well. Is there a way to get msbuild to pack and unpack the list correctly here?
Msbuild is complaining with:
error MSB4012: The expression "FilesToZip=#(Scripts)" cannot be used in this context. Item lists cannot be concatenated with other strings where an item list is expected. Use a semicolon to separate multiple item lists.
Here's an example caller:
<Target Name="BuildMigrationZip">
<MSBuild Projects="BuildZip.msbuild"
Targets="BuildZip"
Properties="FilesToZip=#(Scripts);OutputZipFile=$(MigrationPackageFilePath);OutputFolder=$(MigrationPackagePath);Flatten=true"/>
<Message Text="Created database migration zip: $(MigrationPackageFilePath)" Importance="high"/>
</Target>
And the base target:
<Target Name="BuildZip">
<MakeDir Directories="$(OutputFolder)"/>
<Zip Files="#(FilesToZip)"
ZipFileName="$(OutputZipFile)"
Flatten="$(Flatten)"
ParallelCompression="false" />
</Target>
I'm basically at the point of just going back to cut and paste for these, although I want to package up a number of zips here.
UPDATE: The same issue applies to setting Inputs on the reusable target. My question up to this point addresses the raw functionality, but it would be nice to keep dependencies working. So for example:
<Target Name="BuildZip"
Inputs="#(FilesToZip)"
Outputs="$(OutputZipFile)">
<MakeDir Directories="$(OutputFolder)"/>
<Zip Files="#(FilesToZip)"
ZipFileName="$(OutputZipFile)"
Flatten="$(Flatten)"
ParallelCompression="false" />
</Target>
They key is to pass the list around as a property. So when your Scripts list is defined as
<ItemGroup>
<Scripts Include="A"/>
<Scripts Include="B"/>
<Scripts Include="C"/>
</ItemGroup>
then you first convert it into a property (which just makes semicolon seperated items, but msbuild knows how to pass this via the Properties of the MSBuild target) then pass it to the target:
<Target Name="BuildMigrationZip">
<PropertyGroup>
<ScriptsProperty>#(Scripts)</ScriptsProperty>
</PropertyGroup>
<MSBuild Projects="$(MSBuildThisFile)" Targets="BuildZip"
Properties="FilesToZip=$(ScriptsProperty)" />
</Target>
(note I'm using $(MSBuildThisFile) here: you don't necessarily need to create seperate build files for every single target, in fact for small targets like yours it's much more convenient to put it in the same file)
Then in your destination target you turn the property into a list again:
<Target Name="BuildZip">
<ItemGroup>
<FilesToZipList Include="$(FilesToZip)"/>
</ItemGroup>
<Message Text="BuildZip: #(FilesToZipList)" />
</Target>
Output:
BuildZip: A;B;C
Update
When working with Inputs, you cannot pass #(FilesToZip) since that expands to nothing because is not a list: it's a property - which happens to be a number of semicolon-seperated strings. And as such, it is usable for Inputs you just have to expand it as the property it is i.e. $(FilesToZip):
<Target Name="BuildZip"
Inputs="$(FilesToZip)"
Outputs="$(OutputZipFile)">
...
</Target>
Output of second run:
BuildZip:
Skipping target "BuildZip" because all output files are up-to-date with respect to the input files.
I want to write a number in a text file using WriteLinesToFile but the task is putting a line feed at the end which causes me trouble when i want to read or combine in other places
Example:
<WriteLinesToFile File="$(TextFile)" Lines="#(BuildNumber)" Overwrite="true"/>
UPDATE as the user comment below:
The problem that I had was that I was using a very simple command in Property to read the content of a file $([System.IO.File]::ReadAllText("$(TextFile)")) and I really want to use this one but it also included the line feed from WriteLinesToFiles. I ended up using similar solution like yours using ReadLinesFromFile.
There is a slight dissconnect between the title and the description. I would have liked to post this "answer" as an edit, but do not have enough reputation points :)
Do you have a problem with the newline at the end of a file, or do you have a problem ignoring that newline? Could you please clarify?
One way how I suppose you could ignore that newline follows.
This small snippet of code writes a build number to a file, then reads it out and then increments the number read by 1.
<Target Name="Messing_around">
<PropertyGroup>
<StartBuildNumber>1</StartBuildNumber>
</PropertyGroup>
<ItemGroup>
<AlsoStartBuildNumber Include="1"/>
</ItemGroup>
<!-- Use a property
<WriteLinesToFile File="$(ProjectDir)test.txt" Lines="$(StartBuildNumber)" Overwrite="true"/>
-->
<WriteLinesToFile File="$(ProjectDir)test.txt" Lines="#(AlsoStartBuildNumber)" Overwrite="true"/>
<ReadLinesFromFile File="$(ProjectDir)test.txt">
<Output
TaskParameter="Lines"
ItemName="BuildNumberInFile"/>
</ReadLinesFromFile>
<PropertyGroup>
<OldBuildNumber>#(BuildNumberInFile)</OldBuildNumber>
<NewBuildNumber>$([MSBuild]::Add($(OldBuildNumber), 1))</NewBuildNumber>
</PropertyGroup>
<Message Importance="high" Text="Stored build number: #(BuildNumberInFile)"/>
<Message Importance="high" Text="New build number: $(NewBuildNumber)"/>
</Target>
And this is what I see
Output:
1>Build started xx/xx/xxxx xx:xx:xx.
1>Messing_around:
1> Stored build number: 1
1> New build number: 2
1>
1>Build succeeded.
If you attempting to read, in an MSBuild Task, a single line containing only a number from a file with a trailing line feed, then you should not have a problem.
As a side note: With the little information at hand I'd assume that BuildNumber is an Item in an ItemGroup. If you have only one build number to deal with, perhaps Property may have been an option. But then, again, I haven't been tinkering with MSBuild for too long. So, I am open to feedback on the Item vs Property issue.
I would like to minimize repetition in my msbuild xml, particularly inside my Target's Exec Command attribute.
1) Most importantly, in the following Target I would like to un-duplicate the pair of "%(ShaderVertex.Identity)" constructs in the Exec Command attribute (for example, the Property VertexTargetName is defined to be "ShaderVertex", but %($(VertexTargetName).Identity) will not expand $(VertexTargetName)):
<Target Name="ShaderVertexBuild"
Inputs="#($(VertexTargetName))"
Outputs="#($(VertexTargetName)$(OutputRelPathAndFileName))">
<Exec Command="$(FxcPathFull)v$(ShaderArgNoFirstLetter) %(ShaderVertex.Identity)$(ShaderOutputFlagAndRelPath)%(ShaderVertex.Identity)$(OutputFileExtLetterToAppend)">
</Exec>
</Target>
2) It would also be nice not to duplicate the "ShaderVertex" identifier in the ItemGroup that precedes my Target, but again I can't expand a $(VertexTargetName) in the context of an ItemGroup.
<ItemGroup>
<ShaderPixel Include="p*.fx"/>
<ShaderVertex Include="v*.fx"/>
</ItemGroup>
3) Less importantly, it would be nice to be able to further reduce duplicate constructs between my Target's. Using the above example, I'd like to simply pass $(VertexTargetName) and the letter "v" (which tells the HLSL compiler to compile a vertex shader) to a magical Target generator that fills in all the data identical to all Targets -- or even better, just pass $(VertexTargetName) and allow the Target generator to infer from this argument that a "v" needs to be passed (instead of, say, a "p" for PixelShader, which could be inferred from $(PixelTargetName)).
Any ideas?
Caveats: I know that msbuild is not a functional programming language, and therefore may not be able to accommodate my desires here. I also know that VS2012 handles HLSL compilation automatically, but I'm on VS2010 for now, and would like to know enough about msbuild to accomplish this sort of task.
Here is the complete xml listing:
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<!-- TODO NTF: does not take into account *.inc dependency -->
<Target Name="Main">
<CallTarget Targets="ShaderPixelBuild"/>
<CallTarget Targets="ShaderVertexBuild"/>
</Target>
<ItemGroup>
<ShaderPixel Include="p*.fx"/>
<ShaderVertex Include="v*.fx"/>
</ItemGroup>
<PropertyGroup>
<ShaderModelNum>4</ShaderModelNum><!-- Radeon 8600 GTS doesn't support shader model 5 -->
<ShaderArgNoFirstLetter>s_$(ShaderModelNum)_0</ShaderArgNoFirstLetter><!-- first letter defines shader type; v=vertex, p=pixel-->
<FxcPathFull>"%DXSDK_DIR%\Utilities\bin\x64\fxc.exe" /T </FxcPathFull>
<ShaderOutputRelPath>..\..\x64\shadersCompiled\</ShaderOutputRelPath>
<ShaderOutputFlagAndRelPath> /Fo $(ShaderOutputRelPath)</ShaderOutputFlagAndRelPath>
<OutputFileExtLetterToAppend>o</OutputFileExtLetterToAppend>
<OutputRelPathAndFileName>->'$(ShaderOutputRelPath)%(identity)$(OutputFileExtLetterToAppend)'</OutputRelPathAndFileName>
<!-- Note that the content of each of these properties must be duplicated in the Exec Command twice, since %($(PropertyName)) will not expand there -->
<PixelTargetName>ShaderPixel</PixelTargetName>
<VertexTargetName>ShaderVertex</VertexTargetName>
</PropertyGroup>
<Target Name="ShaderPixelBuild"
Inputs="#($(PixelTargetName))"
Outputs="#($(PixelTargetName)$(OutputRelPathAndFileName))">
<Exec Command="$(FxcPathFull)p$(ShaderArgNoFirstLetter) %(ShaderPixel.Identity)$(ShaderOutputFlagAndRelPath)%(ShaderPixel.Identity)$(OutputFileExtLetterToAppend)">
</Exec>
</Target>
<Target Name="ShaderVertexBuild"
Inputs="#($(VertexTargetName))"
Outputs="#($(VertexTargetName)$(OutputRelPathAndFileName))">
<Exec Command="$(FxcPathFull)v$(ShaderArgNoFirstLetter) %(ShaderVertex.Identity)$(ShaderOutputFlagAndRelPath)%(ShaderVertex.Identity)$(OutputFileExtLetterToAppend)">
</Exec>
</Target>
</Project>
My msbuild targets file contains the following section:
<ItemGroup>
<Targets Include="T1">
<Project>A\B.sln"</Project>
<DependsOnTargets>The targets T1 depends on</DependsOnTargets>
</Targets>
<Targets Include="T2">
<Project>C\D.csproj"</Project>
<DependsOnTargets>The targets T2 depends on</DependsOnTargets>
</Targets>
...
</ItemGroup>
<Target Name="T1" DependsOnTargets="The targets T1 depends on">
<MSBuild Projects="A\B.sln" Properties="Configuration=$(Configuration)" />
</Target>
<Target Name="T2" DependsOnTargets="The targets T2 depends on">
<MSBuild Projects="C\D.csproj" Properties="Configuration=$(Configuration)" />
</Target>
As you can see, A\B.sln appears twice:
As Project metadata of T1 in the ItemGroup section.
In the Target statement itself passed to the MSBuild task.
I am wondering whether I can remove the second instance and replace it with the reference to the Project metadata of the target, which name is given to the Target task?
Exactly the same question is asked for the (Targets.DependsOnTargets) metadata. It is mentioned twice much like the %(Targets.Project) metadata.
Thanks.
EDIT:
I should probably describe the constraints, which must be satisfied by the solution:
I want to be able to build individual projects with ease. Today I can simply execute msbuild file.proj /t:T1 to build the T1 target and I wish to keep this ability.
I wish to emphasize, that some projects depend on others, so the DependsOnTargets attribute is really necessary for them.
Target names must be fixed values, so what you have here wouldn't work.
Also I would recommend not using Batching Expressions inside of the DependsOnTargets expression as well. This could lead to strange behavior if you do not fully understand what is happening.
In your case you may be able to just create a "driver" target which uses those items to perform the build. The only difficult part would be the DependsOnTargets that you are trying to perform. I'm not sure about the details on what you are trying to do with that so cannot make any suggestions but as for the other take a look at creating a target similar to.
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<ItemGroup>
<Targets Include="T1">
<Project>A\B.sln"</Project>
<DependsOnTargets>The targets T1 depends on</DependsOnTargets>
</Targets> <Targets Include="T2">
<Project>C\D.csproj"</Project>
<DependsOnTargets>The targets T2 depends on</DependsOnTargets>
</Targets> ...
</ItemGroup>
<Target Name="Build">
<!--
This will be executed once for every unique value of Project in the
Target item group
-->
<MSBuild Projects="%(Targets.Project)"
Properties="Configuration=$(Configuration)"
</Target>
</Project>