Executing all sql files in a folder with ant script (apply task) - sql

I have a problem executing all SQL files in a folder with sqlplus command
I can run one file or first file with this code
<apply executable="sqlplus" dir="." parallel="false">
<!-- <arg value="-S" /> -->
<arg line="${db.user}/${db.pwd}#'${db.sid}'" />
<arg value="#${db.run.build.script}" /> this is working, which is for executing one file
<!-- <arg value="--e source" /> -->
<srcfile/>
<fileset dir="..\..\Documents\test\Database\SQL\common\test"
casesensitive="no" description="take all sql files">
<patternset>
<include name="**/*.sql" />
</patternset>
</fileset>
</apply>
; but cannot proceed further with other files
I have tried various options with apply task; but still not succeed; I thought the second file hasn't "#" in the beginning that causes the problem.
any help will be highly appreciated.

OK, first of all: I've never used sqlplus :-)
But based on the post I've seen here: http://www.zorbathegeek.com/161/run-multiple-sqlplus-queries-via-dos-batch-file.html I understood that it's a must to provide # as a prefix to path to all the files that should be considered.
I see there multiple options, based on restrictions:
First check if you can provide in command line multiple of your files like this:
sqlplus [login details] #C:/<your_path>/<file1>.sql #C:/<your_path>/<file1>.sql
depending if it's possible or not, you should later decide if you need to create 1 file containing all the filenames prefixed with # (as the example provided in the reffered site) or not
Now how to do the prepending? I think it should be simple, just try to use Ant's parhconvert described: http://ant.apache.org/manual/Tasks/pathconvert.html
<fileset dir="..\..\Documents\test\Database\SQL\common\test" id="sql.files">
<include name="**/*.sql"/>
</fileset>
<pathconvert pathsep=" " property="prefixed.sql.files" refid="sql.files">
<map from="c:/" to="#c:/"/>
</pathconvert>
(where you should replace c:/ with the string that your file paths start with. Moreover, if you will dump to file, pathsep=" " should be probably replaced with pathsep="\n" as there might be a need to have one file per line, but this needs test)
then depending on the options of all in one file or all files on cmd line, you:
either need to dump it to tmp file and run using this tmp file name
http://ant.apache.org/manual/Tasks/echo.html
In this case, the file should be later deleted
use property prefixed.sql.files directly in your apply arg value like:
<arg value="${prefixed.sql.files}" />

Related

How to generate files during build using msbuild

Does anyone know how to modify a csproj file in a way to generate code files during build without actually referencing the files?
A process like :
create file,
dynamically reference temporary file during build
compiled assembly has additional members, depending on the files created during build
The purpose of this is to create a way of generating code files using roslyn instead of using t4 templates, which are very awkward to use once you're trying to do something depending on attributes.
Hence i am planning on providing a way to use a special csharp file (for full syntax support) to generate files programatically based on the contents of that special file.
I've spent a couple of weeks looking into resources on the internet (with the topic msbuild), but until now it seems i didn't use the right keywords.
This one has been the most insightful one to me yet:
https://www.simple-talk.com/dotnet/.net-tools/extending-msbuild/
My guess is, that the correct build target for my purpose should be "BeforeCompile" in order to somehow populate the build process with custom code files.
Does anyone have experience with my issue, or is aware of any particular resources which deal with the task?
Solution i got it working with:
<UsingTask TaskName="DynamicCodeGenerator.DynamicFileGeneratorTask" AssemblyFile="..\DynamicCodeGenerator\bin\Debug\DynamicCodeGenerator.dll" />
<Target Name="DynamicCodeGeneratorTarget" BeforeTargets="BeforeBuild;BeforeRebuild">
<DynamicFileGeneratorTask>
<Output ItemName="Generated" TaskParameter="GeneratedFilePaths" />
</DynamicFileGeneratorTask>
<ItemGroup>
<Compile Include="#(Generated)" />
<FileWrites Include="#(Generated)" />
<!-- For clean to work properly -->
</ItemGroup>
</Target>
Unfortunately i did not get it to work with a propertygroup override as suggested
Update: This link is interesting too: https://github.com/firstfloorsoftware/xcc/blob/master/FirstFloor.Xcc/Targets/Xcc.targets
Generating the code file can be achieved by msbuild task or msbuild inline task. It is up to you to generate the proper code. One thing that you must care of is creating output item parameter in order to append it to the #(Compile) item. You can use $(IntDir) location to locate your newly generated file, and add them to the #(FileWrites) item group in order for Clean target work properly.
When you finish writing your task, you must use it in your project like this:
<UsingTask TaskName="TaskTypeFullName" AssemblyFile="YourAssembly.dll"/>
<PropertyGroup>
<!-- Here you need to experiment with [Build/Compile/SomeOther]DependsOn property -->
<BuildDependsOn>
MyCodeGenerator;
$(BuildDependsOn)
</BuildDependsOn>
</PropertyGroup>
<Target Name="MyCodeGenerator">
<YourTaskName>
<Output ItemName="Generated" TaskParameter="GeneratedFiles" />
</YourTaskName>
<ItemGroup>
<Compile Include="#(Generated)" />
<FileWrites Include="#(Generated)" /> <!-- For clean to work properly -->
</ItemGroup>
</Target>
I wanted to use a bash script to generate code for a project using dotnet core on Linux. Here is what worked for me. And thanks #stukselbax, I built this off of your answer.
<Target Name="GenerateProtocolBuffers" BeforeTargets="BeforeBuild;BeforeRebuild">
<Exec Command="./generatecode.sh" Outputs="proto/*.cs">
<Output ItemName="Generated" TaskParameter="Outputs" />
</Exec>
<ItemGroup>
<Compile Include="#(Generated)" />
<FileWrites Include="#(Generated)" />
</ItemGroup>
</Target>
Note that the script I'm using to generate the code is called generatecode.sh. Replace this with your own.

Ant build to operate only on subdirectories

I'm trying to create an Ant build that will run a target in every subfolder in a folder. I need this one in particular to only run at the subfolder level, because if I run it from the top folder and tel it to include subfolders, it scrambles the results.
I put the following together from suggestions I've seen, and it seems close but it's not working. A couple of points about it:
When I run this, I get "C:\Developer\SVN\trunk\DITA\xxx_conversion\test\${subdir} does not exist." (A subdirectory does exist, I have ...\test\testsub\xxx_xxx.dita)
The target named "script" works perfectly by itself. If I point it to a folder, it transforms what's in it and gives me the result I need.
In the foreach task if I change to just <dirset dir="."> as a test then the build succeeds, but it basically just runs the "script" target as is.
In the target named "script", if I change "*.dita" to "**\*.dita" it then includes the top level folder in its operation so the result is not what I need, which is just to take each .dita file in each subfolder and wrap it in a folder of the same name as the dita file:
Starting state: testsub\xxx_xxx.dita
Desired result: testsub\xxx_xxx\xxx_xxx.dita
Scrambled result: testsub\xxx_xxx\testsub\xxx_xxx.dita
If anyone can tell me what's wrong with this (possibly many things) that would be great:
<project name="move_and_wrap_dita_topics" default="script" basedir="C:\Developer\SVN\trunk\DITA\xxx_conversion\test">
<taskdef resource="net/sf/antcontrib/antlib.xml">
</taskdef>
<foreach target="script" param="worksheet" inheritall="true">
<path>
<dirset dir="${subDir}">
<include name="*"/>
</dirset>
</path>
</foreach>
<target name="script">
<copy todir="C:\Developer\SVN\trunk\DITA\xxx_conversion\cleaned" verbose="true">
<fileset dir="C:\Developer\SVN\trunk\DITA\xxx_conversion\test">
<include name="*.dita"/>
</fileset>
<scriptmapper language="javascript">
self.addMappedName(source.replace(source.split('.')[0], source.split('.')[0] + "/" + source.split('.')[0]));
</scriptmapper>
</copy>
</target>
</project>
As an alternative, if there's some way to just write the "include name" in the "script" task so that it skips the top folder completely and only rewrites starting at the subfolders, that would be even simpler. I've tried <include name="/*.dita"/>and all sorts of other variations but no luck.
Starting state: testsub\xxx_xxx.dita
Desired result: testsub\xxx_xxx\xxx_xxx.dita
This should be doable with a single regex mapper, you don't need the foreach at all:
<copy todir="C:\Developer\SVN\trunk\DITA\xxx_conversion\cleaned" verbose="true">
<fileset dir="C:\Developer\SVN\trunk\DITA\xxx_conversion\test"
includes="*/*.dita" />
<!-- use handledirsep to treat \ as if it were / in the source file name -->
<regexpmapper handledirsep="true" from="^(.*)/([^/]*)\.dita$"
to="\1/\2/\2.dita" />
</copy>
The includes="*/*.dita" will match .dita files that are one level below the base directory of the fileset, and the regex will convert testsub/xxx.dita in the source to testsub/xxx/xxx.dita in the destination. If you want to find files that are one-or-more levels deep then use */**/*.dita instead.

Nant delete task doesn't delete files post msbuild

Using Nant 0.92 (and previously 0.85 with same consequence)
I'm trying to call the delete task in NAnt to remove all files except the .dll file after calling msbuild (see the script below .. I'm referring to the delete in the "build" target). The delete task does not seem to execute.
The initial delete task works fine, and behaves as expected, removing all files from the specified build directory. The second delete task, after the compile however doesn't work.
I've tried just deleting everything (not using the exclude tag), tried explicitly setting failonerror and verbose to true. None of these make a difference. I've also tried using sleep to stop the process prior to the delete task, in case something in msbuild was not releasing the files in time for a delete. I've tried putting the delete into a separate target, still no luck.
The command obviously works prior to calling msbuild, and it works after msbuild if trying to delete from a directory other than the msbuild output target (i.e. copy the output files, then delete relevant files).
I'm sure that this is too fundamental a problem to be a bug, but I thought I'd ask anyway. Of course I'll use the workaround in the mean time of just copying the files to a different location delete what I don't need then move appropriately, but I'm curious about this.
I suspect, that unless this behaviour is by design (although I can find nothing in the NAnt documentation to suggest it is), then perhaps the msbuild process is locking the output files until NAnt process completes? This is my best guess. Further insights would be appreciated.
EDIT: also, if I explicitly set the \OutputPath switch of msbuild, then I don't have the same problem. It only appears that when the default OutputPath is used is the problem created.
NAnt build file:
<?xml version="1.0" encoding="utf-8" ?>
<project name="Reports.TestBench.PreBuild" default="postbuild.cleanup" basedir="." xmlns="http://nant.sourceforge.net/release/0.86-beta1/nant.xsd">
<property name="nant.settings.currentframework" value="net-4.0" />
<property name="project.StandardReports" value="${project::get-base-directory()}\Reports.StandardReports\Palladium.Reports.StandardReports.csproj" />
<property name="output.Dir" value="${project::get-base-directory()}\bin\debug\"/>
<property name="build.Type" value="debug"/>
<!--Deletes the pre-existing build files-->
<target name="clean">
<delete>
<fileset basedir="${output.Dir}">
<include name="*.*" />
</fileset>
</delete>
</target>
<!--Builds the projects to the specified build directory-->
<target name="build" depends="clean" description="Build the Palladium Reports Standard Reports application">
<msbuild project="${project.StandardReports}">
<arg value="/p:Configuration=${build.Type}" />
<!--arg value="/p:OutputPath=${path::get-full-path(buildDir.Client)}\Reports" /-->
<arg value="/t:Rebuild" />
</msbuild>
<delete failonerror="true" verbose="true">
<fileset basedir="${output.Dir}">
<include name="*.*" />
<exclude name="Palladium.Reports.StandardReports.dll" />
</fileset>
</delete>
</target>
</project>
Summary of NAnt output showing build success with no further messages:
[msbuild] Build succeeded.
[msbuild] 0 Warning(s)
[msbuild] 0 Error(s)
[msbuild]
[msbuild] Time Elapsed 00:00:03.19
BUILD SUCCEEDED
Total time: 3.5 seconds.
Try poking into NAnt source code - most likely the msbuild task creates an MSBuild engine and does not force it to close the files afterwards. Although looking at the MSDN docs, I don't really see a way to do it - MSBuild constructs lack a Dispose() function. There's a finalizer on classes such as ProjectInstance, so one could force it release the file handles by running in an application domain and closing the whole domain afterwards - that's quite a bit of work.

MSBuild a .dbproj seems to run twice

When we build our Visual Studio 2010 Database Project from the command line using msbuild.exe it can sometimes run twice from the same command line.
We call msbuild from a nAnt script, and just call the 'Build' target. Our database project is quite large so it can take about 4 mins to run through a single time. When it runs through twice our database build takes over 8 minutes.
Here is the exec section we use to call the build. It runs on a .sln file that only has a single .dbproj in it.
<exec program="${framework::get-tool-path('msbuild.exe')}" append="true" failonerror="true" verbose="true">
<arg value="${database.sln}" />
<arg value="/p:OutputPath=${build.output.database}" />
<arg value="/nologo" />
<arg value="/t:Build" />
<arg value="/p:Configuration=Release" />
<arg value="/p:WorkingDir="."" />
<arg value="/verbosity:normal" />
<arg value="/v:m" />
</exec>
The output we get looks like
Creating a model to represent the project...
Loading project references...
Loading project files...
Building the project model and resolving object interdependencies...
Validating the project model...
(x) problems have been detected.
[a list of warnings based on the db analysis]
The results are saved in (y).
Creating a model to represent the project...
Loading project references...
Loading project files...
Building the project model and resolving object interdependencies...
Validating the project model...
(x) problems have been detected.
[a list of warnings based on the db analysis]
The results are saved in (y).
Can anyone help as to why the target seems to be called twice (only sometimes - I haven't figured out why only sometimes). The script always runs on an empty folder structure so there is never a build output left over from a previous run of the build.
First of all try to change the output verbosity to diagnostic for MSBuild:
<arg value="/v:diag" />
And the same for the Ant script. I don't have experience with Ant , but you shpould know how to do it ;)
Hopefully you will find a reason in the diagnostic logs...
BTW:
You have duplicate command line options for MSBuild. Remove one arg from the following:
<arg value="/verbosity:normal" />
<arg value="/v:m" />
/v is abbreviation for /verbosity see MSDN for MSBuild command line options reference.

NAnt <msbuild> custom output directory

I'm new to NAnt and have been able to create a <target> which
1) Deletes any code from the current folder
2) Exports fresh code from SVN
3) Builds the code in the default directory which is the PrecompiledWeb folder (its a web app)
Here it is:
<target name="export" description="export code from svn">
<delete dir="${Delete.Dir}"></delete>
<exec program="svn" commandline="export ${MySVN.Repos} ${MySVN.Dest}" />
<msbuild project="${Solution.Filename}">
<property name="Configuration" value="Release"/>
</msbuild>
</target>
I want to specify a custom output directory (other than "PrecompiledWeb"). Is this possible and could you please show me the necessary tag/property?
Thank you!
EDIT
Thanks to Scott and Si, I'm getting closer to a solution, but I still don't have anything that works. There comments led me to this article on MSBuild's Output Path Property. Using their code:
<?xml version="1.0"?>
<project name="test" default="build" basedir="." xmlns="http://nant.sf.net/schemas/nant-0.84.win32.net-1.0.xsd">
<target name="build">
<exec program="${framework::get-framework-directory('net-3.5')}/msbuild.exe">
<arg value="${Full.Path}\Code\MySolution.sln" />
<arg value="/p:OutputPath=${Full.Path}\Output\" />
</exec>
</target>
</project>
This will sucessfully run; however, in my solution which contains a web site and a class library, it publishes the class library dll to the specified output path, but it still publishes the actual web site to the default PrecompiledWeb folder.
Does anyone have any suggestions for how to change the PrecompiledWeb folder path from NAnt?
Thanks again to everyone who's helped!
Edit 2 Final Solution
Here is what I finally was able to get working (updated for .net 4):
<exec program="${framework::get-framework-directory('net-4.0')}/msbuild.exe">
<arg value="${Full.Path}\Code\MySolution.sln" />
<arg value="/t:Rebuild" />
<arg value="/t:ResolveReferences" />
<arg value="/t:_CopyWebApplication" />
<arg value="/p:OutDir=${Build.Output}bin\" />
<arg value="/p:WebProjectOutputDir=${Build.Output}" />
<arg value="/p:Configuration=Release" />
</exec>
One can specify and override some of properties for msbuild. In order to specify the output directory, override the OutputDir property.
<target name="export" description="export code from svn">
<delete dir="${Delete.Dir}" />
<exec program="svn" commandline="export ${MySVN.Repos} ${MySVN.Dest}" />
<msbuild project="${Solution.Filename}">
<property name="Configuration" value="Release"/>
<property name="OutputDir" value="${Output.Dir}"/>
</msbuild>
</target>
Just had a quick peek at a project, does OutputPath instead of OutputDir help?
Another option might be a web deployment project, which I like because it calls aspnet_compiler as well as the C# compiler, so it picks up issues which you may otherwise miss until deployment.
A build script for one of our projects uses this command to publish a web application:
msbuild.exe /t:_CopyWebApplication /p:Configuration=Release /p:OutDir=.\..\published\ /p:WebProjectOutputDir=.\..\published
(The current directory is set to the web app's project directory at this point, which is why no .csproj file is specified. The entire solution has already been rebuilt earlier in the script.)
By the sound of it, WebProjectOutputDir might be the property you need.
/t:_CopyWebApplication may also be important. I've not used NAnt so I don't know if you can pass this parameter with the msbuild task. You may need to use an exec task, like in this example: http://www.netomatix.com/development/wapwithnant.aspx. This example appears to rebuild and copy all in one go.
When using the task, the correct property name is OutDir, not OutputDir:
<msbuild project="${Solution.Filename}">
<property name="Configuration" value="Release"/>
<property name="OutDir" value="${Output.Dir}"/>
</msbuild>
A source of confusion is that you're blending two distinct build systems. Your NAnt target is delegating all the work of figuring out how to publish your web application to the solution file, hence by extension to the csproj files it references.
csproj files are MsBuild files, so you should probably look there for how to direct your project output. This post by dave^2 might be helpful on that issue.
You can publish your web application wherever you want using NAnt, provided it's doing the publishing. You can do the same with MsBuild. The cause of your quandary is that NAnt is not doing the publishing in this case, and you're letting the csproj file determine the location of your web directory. So either bypass the csproj file and have NAnt (or MsBuild) publish the code; or modify the csproj file to publish the web application where you want; or make a second location for your web application and publish it there as well using your build tool.
AFAIK, those options are exhaustive.
Hmm, don't know how to do it with MSBuild in Nant, but using NAnt, I've done it previously like this:
<solution solutionfile="${build.dir}\solution.sln">
<webmap>
<map url="http://localhost/somdir/project.csproj"
path="c:\inetpub\wwwroot\somelocaldir" />
<webmap>
</solution>
But then, you're using the NAnt 'solution' task offcourse instead of using MSBuild directly.
edit:
I'm also having a look at some msbuild options;
If you set OutDir & OutputPath to the same value, what happens ?
Try something like this:
<property name="nant.settings.currentframework" value="net-3.5"/>
<msbuild project="${Solution.Filename}">
<property name="Configuration" value="Release"/>
<property name="OutDir" value="${Full.Path}\Output\\"/>
<property name="WebProjectOutputDir" value="${Full.Path}\Output\Web\\"/>
</msbuild>