MSBuild zip directories task without any external dependencies (no MSBuildCommunityTasks) - msbuild

I have attempted to find a way to zip a build up using MSBuild without using MSBuildCommunityTasks. I did manage to find some code online but it seems to take all the files, even ones in directories and put it in one file (no directories).
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<UsingTask TaskName="Zip" TaskFactory="CodeTaskFactory" AssemblyFile="$(MSBuildToolsPath)\Microsoft.Build.Tasks.v4.0.dll">
<ParameterGroup>
<InputFileNames ParameterType="Microsoft.Build.Framework.ITaskItem[]" Required="true" />
<OutputFileName ParameterType="System.String" Required="true" />
<OverwriteExistingFile ParameterType="System.Boolean" Required="false" />
</ParameterGroup>
<Task>
<Reference Include="System.IO.Compression" />
<Using Namespace="System.IO.Compression" />
<Code Type="Fragment" Language="cs">
<![CDATA[
const int BufferSize = 64 * 1024;
var buffer = new byte[BufferSize];
var fileMode = OverwriteExistingFile ? FileMode.Create : FileMode.CreateNew;
using (var outputFileStream = new FileStream(OutputFileName, fileMode))
{
using (var archive = new ZipArchive(outputFileStream, ZipArchiveMode.Create))
{
foreach (var inputFileName in InputFileNames.Select(f => f.ItemSpec))
{
var archiveEntry = archive.CreateEntry(Path.GetFileName(inputFileName));
using (var fs = new FileStream(inputFileName, FileMode.Open))
{
using (var zipStream = archiveEntry.Open())
{
int bytesRead = -1;
while ((bytesRead = fs.Read(buffer, 0, BufferSize)) > 0)
{
zipStream.Write(buffer, 0, bytesRead);
}
}
}
}
}
}
]]>
</Code>
</Task>
</UsingTask>
</Project>
How can i get this code to zip up my folder and keep the directories intact?

<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<UsingTask TaskName="Zip" TaskFactory="CodeTaskFactory" AssemblyName="Microsoft.Build.Tasks.v12.0">
<ParameterGroup>
<Directory ParameterType="System.String" Required="true" />
</ParameterGroup>
<Task>
<Reference Include="System.IO.Compression.FileSystem" />
<Code>System.IO.Compression.ZipFile.CreateFromDirectory(Directory, Directory + ".zip");</Code>
</Task>
</UsingTask>
<Target Name="Foo">
<Zip Directory="C:\Users\Ilya.Kozhevnikov\Dropbox\Foo" />
</Target>
</Project>

There are many things you could do with directories of files and directories in the archive. As a simplifying assumption, let's pick a base directory and require that all files are below it. In the archive, the entries will be the parts of the file paths up to but not including the base directory name.
So, add a parameter:
<BaseDirectory ParameterType="System.String" Required="true" />
Change the archive path for the entry:
var archiveEntry = archive.CreateEntry(
new Uri(Path.GetFullPath(BaseDirectory) + path.DirectorySeparatorChar)
.MakeRelativeUri(new Uri(Path.GetFullPath(inputFileName))).ToString());
And, as a bonus, fix the too-strict file access sharing:
using (var fs = new FileStream(inputFileName, FileMode.Open, FileAccess.Read, FileShare.Read))

Related

How edit or add properties in inline task

I need invoke msbuild task with properties, whats name can be calculated only in runtime. I try do it by this scripts
Main.xml
<?xml version="1.0" encoding="utf-8"?>
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003" ToolsVersion="14.0" DefaultTargets="Build">
<UsingTask TaskName="GetVars" TaskFactory="CodeTaskFactory" AssemblyFile="$(MSBuildToolsPath)\Microsoft.Build.Tasks.v4.0.dll">
<ParameterGroup>
<Result ParameterType="System.String" Output="true"/>
</ParameterGroup>
<Task>
<Code Type="Fragment" Language="cs">
<![CDATA[
this.Result = "AAA=123;BBB=456;";
]]>
</Code>
</Task>
</UsingTask>
<PropertyGroup>
<Vars></Vars>
</PropertyGroup>
<Target Name="Make">
<GetVars>
<Output TaskParameter="Result" PropertyName="Vars"/>
</GetVars>
<MSBuild Projects="Proj.xml" Targets="make" Properties="$(Vars)"/>
</Target>
</Project>
Proj.xml
<?xml version="1.0" encoding="utf-8"?>
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003" DefaultTargets="Build">
<Target Name="Make">
<Message Text="AAA = $(AAA)"/>
<Message Text="BBB = $(BBB)"/>
</Target>
</Project>
This script give this output:
AAA = 123;BBB=456;
BBB =
I expected this output:
AAA = 123;
BBB = 456;
If you want the inline task to produce several items (the msbuild equivalent of an array or list in other languages), you should state it like that instead of using a property (which is a single key/value pair). This is covered in some of the Inline Task documentation - however that uses full-blown ITaskItems whereas just using a String array will do. So:
output a System.String[] from the inline task
assign it to an Item instead of Property using ItemName =
pass the Item to the MSbuild task (which is what it expects anyway), using #() notation
In code:
<?xml version="1.0" encoding="utf-8"?>
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003" ToolsVersion="14.0" DefaultTargets="Make">
<UsingTask TaskName="GetVars" TaskFactory="CodeTaskFactory" AssemblyFile="$(MSBuildToolsPath)\Microsoft.Build.Tasks.v4.0.dll">
<ParameterGroup>
<Result ParameterType="System.String[]" Output="true"/>
</ParameterGroup>
<Task>
<Code Type="Fragment" Language="cs">
<![CDATA[
this.Result = new System.String[]{"AAA=123", "BBB=456"};
]]>
</Code>
</Task>
</UsingTask>
<Target Name="Make">
<GetVars>
<Output TaskParameter="Result" ItemName="Vars"/>
</GetVars>
<MSBuild Projects="$(MSBuildThisFile)" Targets="Show" Properties="#(Vars)"/>
</Target>
<Target Name="Show">
<Message Text="AAA = $(AAA)"/>
<Message Text="BBB = $(BBB)"/>
</Target>
</Project>
Output:
Show:
AAA = 123
BBB = 456

MSBUILD: How to parse solution file to get project paths

How can I get the list of project files from a solution when using MSBUILD?
For example getting all the .csproj from a .sln.
I was previously using MSBuild Community Tasks's GetSolutionProjects for this but unfortunately it has a dependency on .NET 3.5.
To accomplish this using a CodeTask (available since .NET 4) do the following:
<UsingTask TaskName="GetProjectsFromSolutionCodeTask" TaskFactory="CodeTaskFactory" AssemblyFile="$(MSBuildToolsPath)\Microsoft.Build.Tasks.Core.dll" >
<ParameterGroup>
<Solution ParameterType="System.String" Required="true"/>
<Output ParameterType="Microsoft.Build.Framework.ITaskItem[]" Output="true"/>
</ParameterGroup>
<Task>
<Reference Include="System.Xml"/>
<Reference Include="Microsoft.Build"/>
<Using Namespace="Microsoft.Build.Construction"/>
<Code Type="Fragment" Language="cs">
<![CDATA[
var _solutionFile = SolutionFile.Parse(Solution);
Output = _solutionFile.ProjectsInOrder
.Where(proj => proj.ProjectType == SolutionProjectType.KnownToBeMSBuildFormat)
.Select(proj => new TaskItem(proj.AbsolutePath))
.ToArray();
]]>
</Code>
</Task>
</UsingTask>
and invoke it like so:
<!-- Gets the projects composing the specified solution -->
<Target Name="GetProjectsFromSolution">
<GetProjectsFromSolutionCodeTask Solution="%(Solution.Fullpath)">
<Output ItemName="ProjectFiles" TaskParameter="Output"/>
</GetProjectsFromSolutionCodeTask >
</Target>
This will populate a ProjectFiles item collection with the absolute path of all the projects within the solution.
Please note: path to CodeTaskFactory varies by MSBuild version. Example here is for MSBuild 14.0.

Processing batch items in parallel

I have an ItemGroup, and want to process all its items in parallel (using a custom task or an .exe).
I could write my task/exe to accept the entire ItemGroup and process its items in parallel internally. However, I want this parallelism to work in conjunction with MSBuild's /maxCpuCount param, since otherwise I might end up over-parallelizing.
This thread says there's no way.
My testing shows that MSBuild's /maxCpuCount only works for building different projects, not items (see code below)
How can I process items from an ItemGroup in parallel?
Is there a way to author a custom task to work in parallel in conjunction with MSBuild's Parallel support?
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<Target Name="Build" >
<!-- Runs only once - I guess MSBuild detects it's the same project -->
<!--<MSBuild Projects="$(MSBuildProjectFullPath);$(MSBuildProjectFullPath)" Targets="Wait3000" BuildInParallel="true" />-->
<!-- Runs in parallel!. Note that b.targets is a copy of the original a.targets -->
<MSBuild Projects="$(MSBuildProjectFullPath);b.targets" Targets="Wait3000" BuildInParallel="true" />
<!-- Runs sequentially -->
<ItemGroup>
<Waits Include="3000;2000"/>
</ItemGroup>
<Wait DurationMs="%(Waits.Identity)" />
</Target>
<Target Name="Wait3000">
<Wait DurationMs="3000" />
</Target>
<UsingTask TaskName="Wait" TaskFactory="CodeTaskFactory" AssemblyFile="$(MSBuildToolsPath)\Microsoft.Build.Tasks.v4.0.dll" >
<ParameterGroup>
<DurationMs ParameterType="System.Int32" Required="true" />
</ParameterGroup>
<Task>
<Code Type="Fragment" Language="cs">
Log.LogMessage(string.Format("{0:HH\\:mm\\:ss\\:fff} Start DurationMs={1}", DateTime.Now, DurationMs), MessageImportance.High);
System.Threading.Thread.Sleep(DurationMs);
Log.LogMessage(string.Format("{0:HH\\:mm\\:ss\\:fff} End DurationMs={1}", DateTime.Now, DurationMs), MessageImportance.High);
</Code>
</Task>
</UsingTask>
</Project>
I know this is old, but if you get a few minutes, revisit your attempt to use the MSBuild task. Using the Properties and/or AdditionalProperties reserved item metadata elements* will resolve the issue you described in your code sample ("Runs only once - I guess MSBuild detects it's the same project").
The MSBuild file below processes items from an ItemGroup in parallel via MSBuild's parallel support (including /maxCpuCount). It does not use BuildTargetsInParallel from the MSBuild Extension Pack, nor any other custom or inline task.
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<Target Name="Build" >
<ItemGroup>
<Waits Include="3000;2000"/>
</ItemGroup>
<ItemGroup>
<ProjectItems Include="$(MSBuildProjectFullPath)">
<Properties>
WaitMs=%(Waits.Identity)
</Properties>
</ProjectItems>
</ItemGroup>
<MSBuild Projects="#(ProjectItems)" Targets="WaitSpecifiedMs" BuildInParallel="true" />
</Target>
<Target Name="WaitSpecifiedMs">
<Wait DurationMs="$(WaitMs)" />
</Target>
</Project>
* Well-hidden under "Properties Metadata" on the MSBuild Task reference page.
As you said yourself, you can't parallelize on target or task level, you can yield though.
My custom tasks parallelize heavily using TPL, i.e. my base task wrapper has a ForEach wrapper.
public bool ForEach<T>(IEnumerable<T> enumerable, Action<T> action, int max = -1)
{
return enumerable != null && Parallel.ForEach(enumerable, new ParallelOptions { MaxDegreeOfParallelism = max }, (e, s) =>
{
if (Canceled)
s.Stop();
if (s.ShouldExitCurrentIteration)
return;
action(e);
Interlocked.Increment(ref _total);
}).IsCompleted;
}
Typically limit is omitted and managed by .NET itself, with few exception like non-thread safe operations like MSDeploy, deploying SSRS reports that has a config DoS limit of 20 from single IP, or a zip task that degrades heavily if it's more than CPU count even by 1. It's probably not worth reading maxCpuCount and use Environment.ProcessorCount or %NUMBER_OF_PROCESSORS%, but you can try parsing the command line or reflecting on the host object, e.g. my base task class has this method to get all properties, targets, etc. for various extra special global flags.
private void Engine(object host)
{
var type = host.GetType();
if (type.FullName != "Microsoft.Build.BackEnd.TaskHost")
{
Log.Warn("[Host] {0}", type.AssemblyQualifiedName);
return;
}
var flags = BindingFlags.NonPublic | BindingFlags.Instance;
var taskLoggingContext = type.GetProperty("LoggingContext", flags).GetValue(host, null);
var targetLoggingContext = taskLoggingContext.GetType().GetProperty("TargetLoggingContext", flags).GetValue(taskLoggingContext, null);
ProjectTask = taskLoggingContext.GetType().GetProperty("Task", flags).GetValue(taskLoggingContext, null).To<ProjectTaskInstance>();
ProjectTarget = targetLoggingContext.GetType().GetProperty("Target", flags).GetValue(targetLoggingContext, null).To<ProjectTargetInstance>();
var entry = type.GetField("requestEntry", flags).GetValue(host);
var config = entry.GetType().GetProperty("RequestConfiguration").GetValue(entry, null);
Project = config.GetType().GetProperty("Project").GetValue(config, null).To<ProjectInstance>();
Properties = Project.Properties.ToDictionary(p => p.Name, p => p.EvaluatedValue);
Typical task would look something like this using ForEach:
public class Transform : Task
{
[Required]
public ITaskItem[] Configs { get; set; }
protected override void Exec()
{
//...
ForEach(Configs, i =>
{
//...
}, Environment.ProcessorCount);
//...
}

MSBuild UsingTask Resolve References

I feel like I've fixed this before, but I can't remember how.
I have a tasks file that looks like this (CustomTasks.tasks):
<UsingTask AssemblyFile="CustomTasks.dll" TaskName="MyCustomTask"/>
it references an assembly (namely Ionic.Zip.dll). Ionic.Zip.dll is not in the GAC (and I don't want it to be). It sits right next to my CustomTasks.dll.
I have a directory called MSBuild one level up from my sln file which has CustomTasks.tasks, CustomTasks.dll and Ionic.Zip.dll in it.
I have a csproj that references the tasks file and calls the custom task:
<Import Project="$(ProjectDir)\..\MSBuild\CustomTasks.tasks" />
<MyCustomTask ..... />
at build time, this yields:
The "MyCustomTask" task could not be loaded from the assembly ....MyCustomTasks.dll. Could not load file or assembly 'Ionic.Zip,......' or one of its dependencies.
Got tired and frustrated and took a direct approach...I don't think this is the same way I solved the problem previously...but maybe this will help someone else. Other, more elegant solutions are more than welcome.
<Target Name="BeforeBeforeBuild" BeforeTargets="BeforeBuild">
<HandleAssemblyResolve SearchPath="$(ProjectDir)\..\MSBuild\" />
</Target>
<UsingTask TaskName="HandleAssemblyResolve" TaskFactory="CodeTaskFactory" AssemblyFile="$(MSBuildToolsPath)\Microsoft.Build.Tasks.v4.0.dll">
<ParameterGroup>
<SearchPath ParameterType="System.String" Required="true" />
</ParameterGroup>
<Task>
<Using Namespace="System" />
<Using Namespace="System.IO" />
<Using Namespace="System.Reflection" />
<Code Type="Fragment" Language="cs">
<![CDATA[
AppDomain.CurrentDomain.AssemblyResolve += (sender, e) =>
{
var assemblySearchPath = Path.Combine(SearchPath, e.Name.Split(',')[0]);
if (File.Exists(assemblySearchPath)) return Assembly.LoadFrom(assemblySearchPath);
return null;
};
]]>
</Code>
</Task>
</UsingTask>
This is actually easy to fix. Put your custom build tasks and dependencies in a different folder. Then dependencies are loaded correctly.
For example like so:
<UsingTask AssemblyFile="..\BuildTools\CustomTasks.dll" TaskName="MyCustomTask"/>

Obtain file sizes in MSBuild script

Can I find out the sizes of a set of files from an MSBuild script without writing my own code?
MSBuild itself some metadata on individual items (for example, the last modified time at %(ModifiedTime)
), but no sizes. I can't see anything at http://msbuildextensionpack.com/.
Edit: based on Seva's answer, here's an inline task that returns the total size of an array of items:
<UsingTask TaskName="GetFileSize" TaskFactory="CodeTaskFactory" AssemblyFile="$(MSBuildToolsPath)\Microsoft.Build.Tasks.v4.0.dll">
<ParameterGroup>
<Files ParameterType="Microsoft.Build.Framework.ITaskItem[]" Required="true" />
<TotalSize ParameterType="System.Int64" Output="true"/>
</ParameterGroup>
<Task>
<Using Namespace="System.IO"/>
<Code Type="Fragment" Language="cs"><![CDATA[
long l = 0;
foreach (var item in Files) {
var fi = new FileInfo(item.ItemSpec);
l += fi.Length;
}
TotalSize = l;
]]></Code>
</Task>
</UsingTask>
You can actually use a sub-set of .Net API within msbuild projects using inline tasks.
E.g. the following prints out a file size of a single file:
<UsingTask TaskName="GetFileSize" TaskFactory="CodeTaskFactory" AssemblyFile="$(MSBuildToolsPath)\Microsoft.Build.Tasks.v4.0.dll">
<ParameterGroup>
<FileName Required="true" />
<FileSize ParameterType="System.Int64" Output="true"/>
</ParameterGroup>
<Task>
<Using Namespace="System.IO"/>
<Code Type="Fragment" Language="cs"><![CDATA[
FileInfo fi = new FileInfo(FileName);
FileSize = fi.Length;
]]></Code>
</Task>
</UsingTask>
<Target Name="PrintFileSize" >
<GetFileSize FileName="$(MyFileName)">
<Output TaskParameter="FileSize" PropertyName="MyFileSize" />
</GetFileSize>
<Message Text="file size of $(MyFileName) is $(MyFileSize)" />
</Target>