When i run the u-sql script locally in visual studio.. it is throwing error
E_CSC_SYSTEM_INTERNAL: Internal error! The ObjectManager found an invalid number of fixups. This usually indicates a problem in the Formatter.
#input =
EXTRACT
Address_1 string,
Address_2 string
FROM "/Data_new"
USING Extractors.Text(delimiter : '~', quoting : false, silent : true);
#output2 =
SELECT DISTINCT
Address_1,
Address_2
FROM #input;
OUTPUT #output2
TO "/out.txt"
USING Outputters.Text(delimiter : '~' , quoting : false);
Important things to note before you run U-SQL script locally in visual studio.
U-SQL SDK only support x64 environment, make sure to set build platform target as x64 and set your test environment as x64.
Make sure to copy all dependency files (correct versions) as part of your solution.
Do not change default data root path.
Before running any USQL script from test, make sure it runs successfully. Exclude any audit columns like UpdatedTime, ProcessedOn, etc from data comparison.
For more details, refer "ADLA - Run U-SQL script on your local machine" and "ADLA - U-SQL".
Hope this helps.
I am trying to use OpenCover With XUnit and MSBuild for our project and it works fine with one or two assemblies. But when the number of assemblies goes more than 2 it throws the below error:
EXEC : error : unknown command line option: MyProj.UnitTest.dll
[C:\CMR\Source\trunk\Build\Script\CMR.msbuild]
Committing...
No results, this could be for a number of reasons. The most common reasons are:
1) missing PDBs for the assemblies that match the filter please review the output
file and refer to the Usage guide (Usage.rtf) about filters.
2) the profiler may not be registered correctly,
please refer to the Usage guide and the -register switch.
Thought the problem would be with the 3rd assembly i added, so ran it individually again it worked fine. Below is the script I used:
<Exec Command='$(OpenCoverPath)\OpenCover.Console.exe "-target: $(XUnitPath)\xunit.console.exe" "-targetargs:C:\MyPath\UnitTest1.dll C:\MyPath\UnitTest2.dll C:\MyPath\UnitTest3.dll /noshadow" "-output:c:\OpenCoverReport\coverage.xml"'/>
And this is my assumption, for the purpose of posting here i had put paths of dll as C:\MyPath\UnitTest.dll but indeed the path is so huge and there are multiple assemblies with huge path. Does it has anything to do with this error?
try the -targetdir option of OpenCover
e.g.
<Exec Command='$(OpenCoverPath)\OpenCover.Console.exe -targetdir:"C:\MyPath" "-target: $(XUnitPath)\xunit.console.exe" "-targetargs:UnitTest1.dll UnitTest2.dll UnitTest3.dll /noshadow" "-output:c:\OpenCoverReport\coverage.xml" '/>
I have a typescript project set up that is being built with GruntJS using the typescript plugin. I also have a Visual Studio project that I'd like to be able to invoke the build process from.
My first attempt at doing this was adding an <Exec> task to the BeforeBuild target in visual studio, with the <Exec> task configured like this:
<Exec Command="grunt --no-color typescript" />
This runs the build fine, however, when errors are output from Grunt and they populate the Error List in VS, the filename is incorrectly listed as EXEC.
Looking at the Exec Documentation I see that CustomErrorRegularExpression is a parameter to the command, but I can't quite grasp how to use it to solve my problem.
I messed around with it a bit and managed to change the reported filename to my .jsproj file, which is also incorrect. Looking at this post I tried forming my own regex:
<Exec CustomErrorRegularExpression="\.ts\([0-9]+,[0-9]+\):(.*)" Command="grunt --no-color typescript" IgnoreExitCode="true" />
Does anyone have any experience using this command with this parameter to achieve this sort of thing? I think maybe part of the problem is that grunt is printing out errors in two lines?
You are correct about the Exec task only processing single line messages. In addition it also uses Regex.IsMatch to evaluate the error/warning condition, without making use of the pattern's capture groups.
I was not able to find a way to work around this via MSBuild, but the changes to correct the problem were easy to make directly in the grunt task.
I'm using the grunt-typescript task from: https://www.npmjs.org/package/grunt-typescript.
There were 3 trivial changes necessary to make this work.
1) Replace the output utility methods near the top of tasks/typescript.js:
/* Remove the >> markers and extra spacing from the output */
function writeError(str) {
console.log(str.trim().red);
}
function writeInfo(str) {
console.log(str.trim().cyan);
}
2) Replace Compiler.prototype.addDiagnostic to write the file and error data on the same line:
Compiler.prototype.addDiagnostic = function (diagnostic) {
var diagnosticInfo = diagnostic.info();
if (diagnosticInfo.category === 1)
this.hasErrors = true;
var message = " ";
if (diagnostic.fileName()) {
message = diagnostic.fileName() +
"(" + (diagnostic.line() + 1) + "," + (diagnostic.character() + 1) + "): ";
}
this.ioHost.stderr.Write(message + diagnostic.message());
};
Once these changes are made, you no longer need the CustomErrorRegularExpression to be set on your Exec task, and your build output should display the error text including the correct source file with line and column information.
I have a VS2010 solution file with over 20 projects in it, and some of the projects have dependencies on other projects from within the solution.
I also have multiple build configurations set up for different purposes, but I've trimmed down the projects to be built to just include the bare minimum number of projects.
For example, I have three libraries (A, B, and C), a website project, and a website deployment project (VS2010). The website has references to libraries A and B, and B in turn has a reference to C. My build configuration only has the website and the deployment project checked. When I check the project dependencies from the solution's properties, the website correctly lists the libraries, and B shows C, as expected.
When I run the build against my build config from within VS2010, it works completely fine, but when I run MSBuild on the solution specifying my configuration (as follows), it just results in a bunch of errors.
msbuild.exe mysolution.sln /t:Build /p:configuration=MyConfig
Here's an example of the errors I get:
Services\IService.cs(11,63): error CS0246: The type or namespace name 'Priority' could not be found (are you missing a using directive or an assembly reference?)
I noticed this happening on my build server (TeamCity v7.1.2), but I can reproduce it on multiple machines, and I narrowed it down to an issue with MSBuild, itself.
It only started happening after I installed .NET 4.5 (and 2 security patches), so I uninstalled it, reinstalled .NET 4.0 (with patches) since it was also removed, then tried the same command, and it worked just fine.
This leads me to believe that something was changed or broken in MSBuild with .NET 4.5, but nothing in their documentation seems to talk about this kind of change.
MSBuild 4.5 documentation: http://msdn.microsoft.com/en-us/library/hh162058.aspx
I've even tried passing BuildProjectDependencies=true to MSBuild, and it comes up stating that it skipped the other projects because they were not selected in the configuration manager, which is correct and intentional.
The only way I got it to work with MSBuild 4.5 was to go back and select the projects that were being skipped, but since the actual solution is a bit more complex with the dependency chain, I don't want to have to try to manage the configurations manually each time we update the solution with new projects or dependencies. It SHOULD be automatic.
Any ideas with what I'm doing?
If you still not resolved this issue, lets try some blind shots:
Msbuild have confirmed bug when it sometimes generates wrong build order based on followed dependencies. Try to build not entire solution but exact project you want to build - like msbuild.exe YourWebsiteProject.csproj /t:Clean;Build /p:configuration=MyConfig. Is the problem still persists ?
Ensure that YourWebsiteProject and your libs (B and C) have proper references - on project, not on dll in another project folder (simplest way to fix that - remove from B reference to C and re-add it again, just ensure that you are adding project reference and not browsing to bin\Debug for very C.dll). Is issue still there ?
If you could provide detailed or even diagnostic msbuild log (add to your msbuild command line following switches /ds /v:diag and then share teamcity full build log somewhere or pipe command line log to file) or some sample projects set where I can reproduce this behaviour - it could help a lot with issue resolving.
I thought I'd update my previous answer, as I've spent a lot of time and effort creating my own workaround to this problem. The work around is a bit more comprehensive than simply living with the problem, but I've attempted to both eliminate the issue and insulate ourselves against future shocks like this.
MSBuild has been demoted from working with solutions, configurations or otherwise. MSBuild is just asked to compile projects in isolation. The order this happens is calculated by a Powershell script that parses ours solutions and projects to work out the best Just-In-Time build execution plan.
Key to this (and I think you might find helpful) are the following snippets:
Identifying my solutions
I have a list of all the solutions in my platform, and I essentially iterate over each of these.
$buildPlan = (
#{
solutions = (
#{
name = "DataStorage"
namespace = "Platform.Databases"
},
#{
name = "CoreFramework"
},
#{
namespace = "Platform.Server"
name = "Application1"
},
#{
namespace = "Platform.Server"
name = "Application2"
},
#{
namespace = "Platform.Client"
name = "Application1"
}
)
})
I have some logic that helps to translate this into actual physical paths, but its very bespoke to our needs, so I won't list it here. Sufficed to say, from this list, I can find the .sln file I need to parse.
Parsing the solution file for projects
With each solution, I read the .sln file and attempt to extract all the projects contained within that I will need to build later.
So firstly, identify all projects in my
$solutionContent = Get-Content $solutionFile
$buildConfigurations += Get-Content $solutionFile | Select-String "{([a-fA-F0-9]{8}-([a-fA-F0-9]{4}-){3}[a-fA-F0-9]{12})}\.(Release.*)\|Any CPU\.Build" | % {
New-Object PSObject -Property #{
Name = $_.matches[0].groups[3].value.replace("Release ","");
Guid = $_.matches[0].groups[1].value
}
} | Sort-Object Name,Guid -unique
And then translate this into a nice list of projects that I can iterate over later.
$projectDefinitions = $solutionContent |
Select-String 'Project\(' |
ForEach-Object {
$projectParts = $_ -Split '[,=]' | ForEach-Object { $_.Trim('[ "{}]') };
$configs = ($buildConfigurations | where {$_.Guid -eq $projectParts[3]} | Select-Object Name)
foreach ($config in $configs)
{
$santisiedConfig = if ([string]::IsNullOrEmpty($config.Name)){"Release"}else{$config.Name}
if ($projectParts[1] -match "OurCompanyPrefix.")
{
New-Object PSObject -Property #{
Name = $projectParts[1];
File = $projectParts[2];
Guid = $projectParts[3];
Config = $santisiedConfig
}
}
}
}
Load the Visual Studio project
From my parsing of the solution file, I now have a list of projects per solution, which crucially contains the relative File Path from the solution root to find the project.
$projectDefinition = [xml](Get-Content $csProjectFileName)
$ns = #{ e = "http://schemas.microsoft.com/developer/msbuild/2003" }
$references = #();
1) Identifying external project references
$references += Select-Xml -Xml $projectDefinition -XPath "//e:Project/e:ItemGroup/e:Reference" -Namespace $ns | % {$_.Node} | where {$_.Include -match "OurCompanyPrefix" -and $_.HintPath -notmatch "packages"} | % {$_.Include}
2) Identifying internal project references
$references += Select-Xml -Xml $projectDefinition -XPath "//e:Project/e:ItemGroup/e:ProjectReference" -Namespace $ns | % { $_.Node.Name }
3) Following "Post-Build" events as external references
$references += Select-Xml -Xml $projectDefinition -XPath "//e:Project/e:PropertyGroup/e:PostBuildEvent" -Namespace $ns | where {(!([String]::IsNullOrEmpty($_.Node.InnerText)))} | % {
$postBuildEvents = $_.Node.InnerText.Split("`n")
$projectsReferencedInPostBuildEvents = $postBuildEvents | Select-String "\(SolutionDir\)((\w|\.)*)" | % {$_.Matches[0].Groups[1].Value}
if ($projectsReferencedInPostBuildEvents -ne $null)
{
Write-Output $projectsReferencedInPostBuildEvents | % { $matchedProject = $_; ($releaseConfiguation | ? {$_.File -match $matchedProject}).Name }
}
}
And, since we're at it, get some basic output information too
This is handy when it comes to iterating my list of projects to build, as know where to push the output, or where to find the output of a dependent.
$assemblyName = (Select-Xml -Xml $projectDefinition -XPath "//e:Project/e:PropertyGroup/e:AssemblyName" -Namespace $ns).Node.InnerText
$outputPath = (Select-Xml -Xml $projectDefinition -XPath "//e:Project/e:PropertyGroup[contains(#Condition,'Release|')]/e:OutputPath" -Namespace $ns).Node.InnerText
And at the end of it all
We just need to make sure we don't have any duplicates, so I record just the distinct dependencies of this particular code project:
$dependendents = #();
if ($references -ne $null)
{
$buildAction.project.dependencies += $references | where {(!([string]::IsNullOrEmpty($_))) -and ($_ -match "OurCompanyPrefix\.(.*)")} | % { $_.ToLower()} | Select -unique
}
I would hope this provides you with enough information for parsing your SLN and PROJ files. How you would choose to capture and store this information I think would depend entirely up to you.
I'm in the middle of writing quite an in-depth blog post about this, which will contain all the trimmings and framework I've eluded to above. The post isn't ready yet, but I will be linking to it from an earlier post : http://automagik.piximo.me/2013/02/just-in-time-compilation.html - Since this change by Microsoft nearly derailed this work!
Cheers.
In .NET 4.5, the default value of the OnlyReferenceAndBuildProjectsEnabledInSolutionConfiguration property in C:\Windows\Microsoft.NET\Framework\v4.0.30319\Microsoft.Common.targets was changed from false to true. As its name implies, this property causes MSBuild to ignore references to projects that are excluded from the build configuration.
(It's possible that Microsoft made this change in response to a Connect bug I filed against MSBuild 4.0, even though I warned them that the change breaks the build.)
The workaround is simple: Set the property back to false in the first <PropertyGroup> section of each of your projects:
<PropertyGroup>
...
<OnlyReferenceAndBuildProjectsEnabledInSolutionConfiguration>false</OnlyReferenceAndBuildProjectsEnabledInSolutionConfiguration>
</PropertyGroup>
I've encountered the exact same problem. The two work-around's I've managed to find are both unacceptable for the long term, but they do overcome the initial issue of getting old our build processes working with the new 4.5 stack.
Swap the project references with file references
Create compound build configurations
I've opted for #2, as file references mean the developers would lose out on real-time intellisense etc.
The compound configurations are simply this:
Release Server -> All Server Projects
Release Consumer -> "Release Server" + Client Projects
The problem seems to be, that if a project is not included in the current/active build configuration, that it will not include it as a referenced dependency. So, by adding the dependencies into the configuration, the projects will at least compile.
Both ugly, but at least it get's me out of a tight spot.
Matt
I get the sense this issue has multiple causes. I tried most of the solutions here. I cannot change our build server to use a PS script so that solution was out. Nothing I could try worked.
Finally, I deleted my solution and started a new one. The new solution worked. After diffing the broken solution with the working solution, I found the original solution was was missing lines. Each dependency that would not compile was missing this line:
{A93FB559-F0DB-4F4D-9569-E676F59D6168}.Release|Any CPU.Build.0 = Release|Any CPU
Note 1: The GUID will change from dependency to dependency.
Note 2: You can find lines like this one under the "GlobalSection(ProjectConfigurationPlatforms) = postSolution" part of the solution file.
My build.proj file says build "Release" using the "Any CPU" platform. Because MSBuild could not find this line it did not build this dependency. This results in the "error CS0246: The type or namespace could not be found" message.
If you are curious, someone had set this solution to the "x86" platform (which is wrong for us). I changed it to "Any CPU" (along with several other changes). Visual Studio did not add the corresponding lines to the solution file. Everything built fine in the IDE, but MSBuild started throwing errors.
I found that MSBuild is building projects from a solution in order in which they are declared in the .sln file. So if you reorder them with text editor you can fix the order for MSBuild.
The Story So Far
I've got a nice solution with a desktop application project, a few library projects, and a couple of development tools projects (also desktop applications). At the moment, my build server outputs all of the code into one OutputPath. So we end up with
drop-x.y.z\
Company.MainApplication.exe <-- main application
Company.MainApplicationCore.dll <-- libraries
Helper.exe <-- developer tools
Grapher.exe
Parser.exe
... <-- the rest of the output
But, we're growing up and people outside of our team want access to our tools. So I want to organize the output. I decided that what we would want is a different OutputPath per executable project
drop-x.y.z\
Company.MainApplication\
Company.MainApplication.exe <-- main application
Company.MainApplicationCore.dll <-- libraries
... <-- application specific output
Helper\
Helper.exe <-- developer tools
... <-- tool specific output
Grapher\
Grapher.exe
...
Parser\
Parser.exe
...
What I Did
I found this simple command. I like it because it retains all the Solution working-dir context that makes msbuild a pain.
msbuild /target:<ProjectName>
For example, from my solution root as a working directory, I would call
PS> msbuild /target:Helper /property:OutputPath="$pwd\out\Helper"
I'm testing this from PowerShell, so that $pwd resolves to the full path to my working directory, or the Solution root in this case. I get the output I desire.
However, when I run this command
PS> msbuild /target:Company.MainApplication /property:OutputPath="$pwd\out\Company.MainApplication"
I get the following error output (there's no more information, I ran with /verbosity:diagnostic)
The target "Company.MainApplication" does not exist in the project.
What I Need
The command fails on any project with a dot or dots in the name. I tried with many combinations of working directories and properties. I tried several ways of escaping the property values. I also tried running the command from a <Task> in a targets file.
I need to know either
A) How to fix this command to work property
B) How to achieve the same output with minimal friction
Try using an underscore as an escape character for the dot in the target parameter, e.g.
msbuild /target:Company_MainApplication /property:OutputPath="$pwd\out\Company.MainApplication"
Specify the target after the -target: switch in the format :. If the project name contains any of the characters %, $, #, ;, ., (, ), or ', replace them with an _ in the specified target name.
https://learn.microsoft.com/en-us/visualstudio/msbuild/how-to-build-specific-targets-in-solutions-by-using-msbuild-exe?view=vs-2019
Dan Nolan's answer and comments are correct. Just want to supplement the Microsoft documentation.
The /targets: switch is to identify a <Target to run in the project file. You need to supply your .csproj file as a an argument that is not prefixed by a /xx option marker.
You might also want to work based on the .sln file. In that case, you still dont specify the project in the .sln to build in this manner. I'll leave you to search up the correct syntax in case that's what you end up doing.