I am using a SQL 2008 database project (in visual studio) to manage the schema and initial test data for my project. The atabase project uses a post deployment which includes a number of other scripts using SQLCMD's ":r " syntax.
I would like to be able to conditionally include certain files based on a SQLCMD variable. This will allow me to run the project several times with our nightly build to setup various version of the database with different configurations of the data (for a multi-tenant system).
I have tried the following:
IF ('$(ConfigSetting)' = 'Configuration1')
BEGIN
print 'inserting specific configuration'
:r .\Configuration1\Data.sql
END
ELSE
BEGIN
print 'inserting generic data'
:r .\GenericConfiguration\Data.sql
END
But I get a compilation error:
SQL01260: A fatal parser error occurred: Script.PostDeployment.sql
Has anyone seen this error or managed to configure their postdeployment script to be flexible in this way? Or am I going about this in the wrong way completely?
Thanks,
Rob
P.S. I've also tried changing this around so that the path to the file is a variable, similar to this post. But this gives me an error saying that the path is incorrect.
UPDATE
I've now discovered that the if/else syntax above doesn't work for me because some of my linked scripts require a GO statement. Essentially the :r just imports the scripts inline, so this becomes invalid sytax.
If you need a GO statement in the linked scripts (as I do) then there isn't any easy way around this, I ended up creating several post deployment scripts and then changing my project to overwrite the main post depeployment script at build time depending on the build configuration. This is now doing what I need, but it seems like there should be an easier way!
For anyone needing the same thing - I found this post useful
So in my project I have the following post deployment files:
Script.PostDeployment.sql (empty file which will be replaced)
Default.Script.PostDeployment.sql (links to scripts needed for standard data config)
Configuration1.Script.PostDeployment.sql (links to scripts needed for a specific data config)
I then added the following to the end of the project file (right click to unload and then right click edit):
<Target Name="BeforeBuild">
<Message Text="Copy files task running for configuration: $(Configuration)" Importance="high" />
<Copy Condition=" '$(Configuration)' == 'Release' " SourceFiles="Scripts\Post-Deployment\Default.Script.PostDeployment.sql" DestinationFiles="Scripts\Post-Deployment\Script.PostDeployment.sql" OverwriteReadOnlyFiles="true" />
<Copy Condition=" '$(Configuration)' == 'Debug' " SourceFiles="Scripts\Post-Deployment\Default.Script.PostDeployment.sql" DestinationFiles="Scripts\Post-Deployment\Script.PostDeployment.sql" OverwriteReadOnlyFiles="true" />
<Copy Condition=" '$(Configuration)' == 'Configuration1' " SourceFiles="Scripts\Post-Deployment\Configuration1.Script.PostDeployment.sql" DestinationFiles="Scripts\Post-Deployment\Script.PostDeployment.sql" OverwriteReadOnlyFiles="true" />
</Target>
Finally, you will need to setup matching build configurations in the solution.
Also, for anyone trying other work arounds, I also tried the following without any luck:
Creating a post build event to copy the files instead of having to hack the project file XML. i couldn't get this to work because I couldn't form the correct path to the post deployment script file. This connect issue describes the problem
Using variables for the script path to pass to the :r command. But I came across several errors with this approach.
I managed to work around the problem using the noexec method.
So, instead of this:
IF ('$(ConfigSetting)' = 'Configuration1')
BEGIN
print 'inserting specific configuration'
:r .\Configuration1\Data.sql
END
I reversed the conditional and set NOEXEC ON to skip over the imported statement(s) thusly:
IF ('$(ConfigSetting)' <> 'Configuration1')
SET NOEXEC ON
:r .\Configuration1\Data.sql
SET NOEXEC OFF
Make sure you turn it back off if you want to execute any subsequent statements.
Here's how I am handling conditional deployment within the post deployment process to deploy test data for the Debug but not Release configuration.
First, in solution explorer, open the project properties folder, and right-click to add a new SqlCmd.variables file.
Name the file Debug.sqlcmdvars.
Within the file, add your custom variables, and then add a final variable called $(BuildConfiguration), and set the value to Debug.
Repeat the process to create a Release.sqlcmdvars, setting the $(BuildConfiguration) to Release.
Now, configure your configurations:
Open up the project properties page to the Deploy tab.
On the top dropdown, set the configuration to be Debug.
On the bottom dropdown, (Sql command variables), set the file to Properties\Debug.sqlcmdvars.
Repeat for Release as:
On the top dropdown, set the configuration to be Release.
On the bottom dropdown, (Sql command variables), set the file to Properties\Release.sqlcmdvars.
Now, within your Script.PostDeployment.sql file, you can specify conditional logic such as:
IF 'Debug' = '$(BuildConfiguration)'
BEGIN
PRINT '***** Creating Test Data for Debug configuration *****';
:r .\TestData\TestData.sql
END
In solution explorer, right click on the top level solution and open Configuration Manager. You can specify which configuration is active for your build.
You can also specify the configuration on the MSBUILD.EXE command line.
There you go- now your developer builds have test data, but not your release build!
As Rob worked out, GO statements aren't allowed in the linked SQL scripts as this would nest it within the BEGIN/END statements.
However, I have a different solution to his - if possible, remove any GO statements from the referenced scripts, and put a single one after the END statement:
IF '$(DeployTestData)' = 'True'
BEGIN
:r .\TestData\Data.sql
END
GO -- moved from Data.sql
Note that I've also created a new variable in my sqlcmdvars file called $(DeployTestData) which allows me to turn on/off test script deployment.
I found a hack from an MSDN blog which worked fairly well. The trick is to write the commands to a temp script file and then execute that script instead. Basically the equivalent of dynamic SQL for SQLCMD.
-- Helper newline variable
:setvar CRLF "CHAR(13) + CHAR(10)"
GO
-- Redirect output to the TempScript.sql file
:OUT $(TEMP)\TempScript.sql
IF ('$(ConfigSetting)' = 'Configuration1')
BEGIN
PRINT 'print ''inserting specific configuration'';' + $(CRLF)
PRINT ':r .\Configuration1\Data.sql' + $(CRLF)
END
ELSE
BEGIN
PRINT 'print ''inserting generic data'';' + $(CRLF)
PRINT ':r .\GenericConfiguration\Data.sql' + $(CRLF)
END
GO
-- Change output to stdout
:OUT stdout
-- Now execute the generated script
:r $(TEMP)\TempScript.sql
GO
The TempScript.sql file will then contain either:
print 'inserting specific configuration';
:r .\Configuration1\Data.sql
or
print 'inserting generic data';
:r .\GenericConfiguration\Data.sql
depending on the value of $(ConfigSetting) and there will be no problems with GO statements etc. when it is executed.
I was inspired by Rob Bird's solution. However, I am simply using the Build Events to replace the post deployment scripts based on the selected build configuration.
I have one empty "dummy" post deployment script.
I set up a pre-build event to replace this "dummy" file based on the selected build configuration (see attached picture).
I set up a post-build event to place the "dummy" file back after the build has finished (see attached picture). The reason is that I do not want to generate changes in the change control after the build.
The CopiedFiles parameter is returning all the files that were intended to be copied. But given the fact that SkipUnchangedFiles is set to true and ttask itself is not copying anything as can be seen on command line (no copying message). Why not, then, CopiedFiles is empty?
I need to have CopiedFiles parameter be populated only with files that were actually copied (because they were changed) in order to further copy these files into some other folder. This is to maintain an up-to-date release folder as well as to extract only those files which actually need to be propogated onto UAT/production server.
For reference sake, the copy task code I'm using is given below:
<Copy SkipUnchangedFiles="true"
SourceFiles="#(cfile)"
DestinationFiles="#(cfile->'$(PublishDir)\%(Identity)')">
<Output
TaskParameter="CopiedFiles"
ItemName="Changed" />
</Copy>
<Message Text="changed:#(Changed)" Importance="high" />
Is there a bug in the copy task or this is the intended behavior.
The behavior you are seeing is by design. MSBuild keeps track of file dependencies using task outputs. If it were to do otherwise, anything that relied on the #(Changed) item array as an input would not fully process all of the files it needed in most cases. It will even keep track of properties and items created within targets that don't even execute when Inputs and Outputs are up-to-date, for the same reason. Consider making a different Copy task of your own with an additional output parameter, CopiedFilesCopiedByTask (this naming mirrors the naming and behavior of the ValueSetByTask in the otherwise defunct CreateProperty task).
Is there a way to use MSBuild syntax to skip a specific target? I have a file consisting of a lot of properties (lines containing /property:X=Y) that I want to pass on to a recursively called instance of MSBuild, but this file also contains a /target:X line, that I do not want to have any effect. I don't have the option to modify the file.
I suppose you are able to edit .proj file. You can manage MSBuild targets executing by the Condition. Your target, which you want to exclude, could contain something like this:
<Target
Name="SomeTarget"
Condition="'$(SomeProperty)'=='true'"
DependsOnTargets="SomeAnotherTarget"/>
SomeProperty can be passed in the calling:
MSBuild.exe build.proj /p:SomeProperty=false
Regards
I created the ItemGroup shown in the code snippet. I need to iterate through this ItemGroup and run the EXEC command - also shown in the code snippet. I cannot seem to get it to work. The code returns the error shown below (note - the Message is written 2 times, which is correct), but the EXEC Command is not running correctly. The value is not being set; therefore the EXEC is not executing at all. I need the EXEC to execute twice or by however sections I define in the ItemGroup.
ERROR:
Encrypting WebServer appSettings section
Encrypting WebServer connectionStrings section
C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\aspnet_regiis.exe -pef "" "\gaw\UI" -prov "RSACustomProvider"
Encrypting configuration section...
The configuration section '' was not found.
CODE SNIPPET:
<ItemGroup>
<SectionsToEncrypt Include="Item">
<Section>appSettings</Section>
</SectionsToEncrypt>
<SectionsToEncrypt Include="Item">
<Section>connectionStrings</Section>
</SectionsToEncrypt>
</ItemGroup>
<Target Name="EncryptWebServerWebConfigSections">
<Message Text="Encrypting WebServer %(SectionsToEncrypt.Section) section" />
<Exec Command="$(AspNetRegIis) -pef "%(SectionsToEncrypt.Section)" "$(DropLocation)\$(BuildNumber)\%(ConfigurationToBuild.FlavorToBuild)\$(AnythingPastFlavorToBuild)" -prov "$(WebSiteRSACustomProviderName)""/>
</Target>
The problem is that you are batching on 2 items at a time. What I mean is the you have the statements
%(SectionsToEncrypt.Section)
%(ConfigurationToBuild.FlavorToBuild)
In the same task invocation. When you batch on more than 1 item at a time in the same task invocation, they will be batch independently. That's why you're error is stating The configuration section '' ...
If you your FlavorToBuild just has one value what you should do is to stuff that into a property before you call to Exec and then use the property. So your one liner would then convert to:
<PropertyGroup>
<_FlavToBuild>%(ConfigurationToBuild.FlavorToBuild)<_FlavToBuild>
</PropertyGroup>
<Exec Command="$(AspNetRegIis) -pef "%(SectionsToEncrypt.Section)" "$(DropLocation)\$(BuildNumber)\$(_FlavToBuild)\$(AnythingPastFlavorToBuild)" -prov "$(WebSiteRSACustomProviderName)""/>
If you have multiple values for FlavorToBuild then it's more complicated. You would have 2 options:
Hard code Exec more than once
Use target batching with task batching to perform the foreach/foreach
Batching is one of the most confusing elements of MSBuild. I've put together some online resources at http://sedotech.com/Resources#batching. If you want to know more than that then you can pick up a copy of my book.
I need to construct an MSBUILD script executes .SQL Scripts which have changed since last build.
I initially thought that I could copy all scripts from one folder to another using the <Copy> task and using the CopiedFiles <Output> for the copy task. However the copy task returns All files that it Attempted to copy, not actual copied files.
I am able to get MSBUILD to execute SQL Scripts via MSBUILD.ExtensionPack but Im scratching my head on this one
You can do this with a concept known as incremental building. The idea is that you would create a target and then specify the inputs and outputs, which would be files. MSBuild will compare the timestamps of the input files to the output files. If all outputs were created after all outputs then the target is skipped. If all inputs are newer then all the target will be executed for all files. If only a portion are out of date, then only those will be passed to the target. For more info on this see the section Using Incremental Builds in my article Best Practices For Creating Reliable Builds, Part 2.
Also for more resources on MSBuild I have compiled a list at http://sedotech.com/Resources#MSBuild
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003" DefaultTargets="RunScripts">
<Import Project="$(MSBuildExtensionsPath)\ExtensionPack\MSBuild.ExtensionPack.tasks"/>
<PropertyGroup>
<ConnStr>Server=Example;Database=Example;Trusted_Connection=True</ConnStr>
<BuildFolder>Build\</BuildFolder>
</PropertyGroup>
<ItemGroup>
<Scripts Include="*.sql"/>
</ItemGroup>
<Target Name="RunScripts"
Inputs="#(Scripts)"
Outputs="#(Scripts->'$(BuildFolder)%(Filename)%(Extension)')">
<SqlExecute TaskAction="ExecuteScalar"
Files="#(Scripts)"
ConnectionString="$(ConnStr)"/>
<Copy SourceFiles="#(Scripts)"
DestinationFiles="#(Scripts->'$(BuildFolder)%(Filename)%(Extension)')"/>
</Target>
</Project>
Could it be that you copying into an empty destination?
SkipUnchangedFiles
If true, skips the copying of files that are unchanged
between the source and destination. The Copy task considers
files to be unchanged if they have the same size and the
same last modified time.
In your case i suspect that all files are considered changed since they don't exist at the destination.