How can I apply all the sql files in a directory? - sql

We currently run a specific SQL script as part of our Ant deploy process.
What we'd like to do is change this so we run all SQL scripts in a given directory. We can't figure out how to get this directory listing in Ant and iterate through the list and run each SQL script. Does anyone know how to do this?
Note: we currently run the sql file by using the Ant exec task that runs "call sqlplus ${id}/${pw}#${db.instance} #${file}"

I would recommend using the Ant SQL task. You can then specify with the following:
<sql
driver="org.database.jdbcDriver"
url="jdbc:database-url"
userid="sa"
password="pass">
<path>
<fileset dir=".">
<include name="data*.sql"/>
</fileset>
<path>
</sql>

i am doing basically the same thing on an oracle database.. but using sqlplus and the apply task. this allows the scripts to contain DDL statements.
would obviously only work where you have a command line program to use.
using ant 1.8.2 and antcontrib.
i have defined a few macros for use.. (see below) and then just call like
<compile_sql connectstring="${db.connect_string}" >
<filelist dir="${db_proc.dir}" files="specific_file.sql" />
<fileset dir="${db_proc.dir}" includes="wildcard*.pks" />
<fileset dir="${db_proc.dir}" includes="wildcard*.pkb" />
</compile_sql>
the macros are as follows
<macrodef name="compile_sql">
<attribute name="connectstring" />
<attribute name="dirtostart" default=""/>
<attribute name="arg1" default=""/>
<element name="sqllist" implicit="true" description="filesetlist of sql to run"/>
<sequential>
<check_missing_files>
<sqllist/>
</check_missing_files>
<apply executable="${sqlplus.exe}" failonerror="true" verbose="true" skipemptyfilesets="true" ignoremissing="false" dir="#{dirtostart}">
<arg value="-L"/>
<arg value="#{connectstring}"/>
<srcfile prefix="#" />
<sqllist/>
<arg value="#{arg1}"/>
<redirector>
<globmapper id="sqlout.mapper"
from="*"
to="*.out"/>
</redirector>
</apply>
</sequential>
</macrodef>
and
<macrodef name="check_missing_files">
<element name="checkfilelist" implicit="true" description="filelist of files to check for existance"/>
<sequential>
<restrict id="missing.files" xmlns:rsel="antlib:org.apache.tools.ant.types.resources.selectors">
<resources>
<checkfilelist/>
</resources>
<rsel:not>
<rsel:exists/>
</rsel:not>
</restrict>
<fail message="These files are missing: ${ant.refid:missing.files}" >
<condition >
<length string="${ant.refid:missing.files}" when="greater" length="0" />
</condition>
</fail>
</sequential>
</macrodef>

Related

ant junit selenium wont run in jenkins

I have junit tests that use selenium to test web server.
When i run the tests using ant from command line, everything is working fine, browser gets opened and tests are going as planed.Browser gets open and i can see tests running.
Recently ive tried to add automatic tests as part of Ci cycle running on jenkins.
I run it as ant build command.
I can see that ant is executing properly (test classes are built i can see output from tests to console) but browser window never gets opened and test fails because of it. here is my ant file
<?xml version="1.0"?>
<project name="JUNIT" default="main" basedir="../../project" xmlns:artifact="antlib:org.apache.maven.artifact.ant">
<!-- Sets variables which can later be used. -->
<!-- The value of a property is accessed via ${} -->
<property name="api.dir" location="src/java" />
<property name="build.api.dir" location="target/classes" />
<property name="test.dir" location="src/test/java" />
<property name="build.test.dir" location="target" />
<!-- Variables used for JUnit testin -->
<property name="test.report.dir" location="testreport" />
<!-- Define the classpath which includes the junit.jar and the classes after compiling-->
<path id="api.class.path">
<pathelement location="${build.api.dir}" />
</path>
<artifact:dependencies cacheDependencyRefs="true" pathId="pomdeps.path">
<pom file="pom.xml"/>
</artifact:dependencies>
<target name="clean">
<delete dir="${test.report.dir}" />
<delete dir="${build.api.dir}" />
<delete dir="${build.test.dir}" />
</target> <!-- Creates the build, docs and dist directory-->
<target name="makedir">
<echo message="Make dir"/>
<mkdir dir="${build.test.dir}" />
<mkdir dir="${build.api.dir}" />
<mkdir dir="${test.report.dir}" />
</target> <!-- Compiles the java code (including the usage of library for JUnit -->
<target name="compile" depends="clean, makedir">
<echo message="Compile"/>
<javac srcdir="${api.dir}" destdir="${build.api.dir}" includeantruntime="false">
<!--classpath refid="junit.class.path" />
<classpath refid="libs.class.path" /-->
<classpath refid="pomdeps.path" />
</javac>
<javac srcdir="${test.dir}" destdir="${build.test.dir}" includeantruntime="false">
<!--classpath refid="junit.class.path" /-->
<classpath refid="api.class.path" />
<classpath refid="pomdeps.path" />
</javac>
</target>
<!-- Run the JUnit Tests --> <!-- Output is XML, could also be plain-->
<echo message="Classes folder ${build.test.dir}"/>
<target name="junit" depends="compile" >
<echo message="junit"/>
<junit printsummary="on" fork="false" haltonfailure="no" showoutput="true">
<classpath refid="pomdeps.path" />
<classpath>
<pathelement location="${build.test.dir}"/>
<pathelement location="${build.api.dir}"/>
</classpath>
<formatter usefile="false" type="plain"/>
<batchtest fork="no" todir="${test.report.dir}">
<fileset dir="${test.dir}">
<include name="**/*Test*.java"/>
</fileset>
</batchtest>
</junit>
</target>
<target name="main" depends="compile, junit">
<description>Main target</description>
</target>
</project>
Jenkins is 1.591 i installed it with default parameters as windows installation downloaded from their site.
Can it be something wrong with jenkins? Do i miss something?
As i mentioned earlier the problem was lack of UI permissions for Jenkins server.
1.configure Jenkins to run as service and make it login with real user name
2.Make sure that windows host that runs Jenkins logs on automaticaly after restart.

Sonar Integration Tests Coverage not show + Selenium + Jboss +Jacoco

I use Sonar 4.1.1, Jboss 6.x, Jacoco 0.6.4, execute tasks with Ant I am not allowed to use Maven. In an eclipse workspace, I have two projects, one is the web application another is selenium test.
I am able to get unit test and code coverage for unit test. But sonar is not able to read the integration test file created by Jacoco. I think there might be something wrong with the way I create jacoco-it.exec file so sonar can't read it. Because sonar does read my jacoco-ut.exec file. And I am able to have both reportPath and itReportPath to read my jacoco-ut.exec file with no problem. Also thinking maybe is something wrong in my build file. I did a lot of research and tried many different ways to create the jacoco-it.exec file, different Jacoco settings and followed different examples from sonar, jacoco, other blogs but still doesn't work. I must be missing something Help!! Thanks!!
I have VM arguments for Jboss like this
-javaagent:/path to jar/jacocoagent.jar=destfile=/path for create/jacoco-it.exec
When I run selenium, the above code create a file with some data, size about 1.3MB
Here is the part of build relate to this issue
<property name="sonar.sourceEncoding" value="UTF-8" />
<property name="sonar.java.coveragePlugin" value="jacoco" />
<property name="sonar.core.codeCoveragePlugin" value="jacoco" />
<property name="sonar.dynamicAnalysis" value="reuseReports" />
<property name="sonar.jacoco.reportsPath" value="${reports.dir}/junit" />
<property name="sonar.jacoco.itReportPath" value="${reports.dir}/jacoco-it.exec" />
<property name="sonar.jacoco.reportPath" value="${reports.dir}/jacoco-ut.exec" />
<target name="unitTest" depends="compile">
<taskdef name="junit" classname="org.apache.tools.ant.taskdefs.optional.junit.JUnitTask">
<classpath>
<path refid="classpath"/>
</classpath>
</taskdef>
<!-- Import the JaCoCo Ant Task -->
<taskdef uri="antlib:org.jacoco.ant" resource="org/jacoco/ant/antlib.xml">
<classpath refid="classpath"/>
</taskdef>
<!-- Run your unit tests, adding the JaCoCo agent -->
<jacoco:coverage destfile="reports/jacoco-ut.exec" xmlns:jacoco="antlib:org.jacoco.ant">
<junit printsummary="yes" haltonfailure="yes" forkmode="once" fork="true" dir="${basedir}" failureProperty="test.failed">
<classpath location="${classes.dir}" />
<classpath refid="classpath"/>
<formatter type="plain" />
<formatter type="xml" />
<batchtest fork="true" todir="${reports.junit.xml.dir}">
<fileset dir="src">
<include name="**/*TestAdd.java" />
</fileset>
</batchtest>
</junit>
</jacoco:coverage>
</target>
<target name="coverageTest" depends="compile">
<taskdef name="junit" classname="org.apache.tools.ant.taskdefs.optional.junit.JUnitTask">
<classpath>
<path refid="classpath"/>
</classpath>
</taskdef>
<taskdef uri="antlib:org.jacoco.ant" resource="org/jacoco/ant/antlib.xml">
<classpath refid="classpath"/>
</taskdef>
<!--Run your unit tests, adding the JaCoCo agent-->
<jacoco:coverage xmlns:jacoco="antlib:org.jacoco.ant" dumponexit="true" >
<junit printsummary="yes" haltonfailure="yes" forkmode="once" fork="true" dir="${basedir}" failureProperty="test.failed">
<classpath location="${classes.dir}"/>
<classpath refid="classpath"/>
<formatter type="plain" />
<formatter type="xml" />
<formatter type="plain" usefile="false"/>
<batchtest todir="${reports.junit.xml.dir}">
<fileset dir="../HelloAppTest/src">
<include name="**/answerTest.java"/>
</fileset>
</batchtest>
</junit>
</jacoco:coverage>
</target>
The reason for that is, may be you are NOT attaching Jacocoagent.jar file to the "TARGET" (ex: JBoss / Tomcat) JVM's scope and stopping it so that it can flush the final code coverage data to the jacoco it exec file.
Once you do that (instead of using Maven/ANT's JVM scope), run your non-Unit (IT) tests and then STOP the target JVM.
After the target JVM is stopped, you'll get the final jacoco .exec file genreated for the IT tests. Use that file for sonar.jacoco.itReportPath variable and it'll work.
For ex: I pass/have this variable to the Tomcat's startup.sh script and while starting tomcat (target JVM), I use this variable within the Tomcat's actual start command.
PROJ_EXTRA_JVM_OPTS=-javaagent:tomcat/jacocoagent.jar=destfile=build/jacoco/IT/jacocoIT.exec,append=false

How to run liquibase on linux system

I am working with Liquibase on Linux, does anyone know how to run the datbasechangelog.xml file from the Linux prompt step by step? And what's the idea behind databasechangelog and how it works?
For our projects, we have set up ant tasks to do this. So, for example, if you want to run the migrations, the ant file may look like the following:
ant-migrations.xml
<project name="Migrations" basedir="." default="update-database">
<property file="./liquibasetasks.properties" />
<path id="master-classpath" description="Master classpath">
<fileset dir="..\lib">
<include name="*.jar" />
</fileset>
</path>
<target name="update-database">
<fail unless="db.changelog.file">db.changelog.file not set</fail>
<fail unless="database.url">database.url not set</fail>
<fail unless="database.username">database.username not set</fail>
<fail unless="database.password">database.password not set</fail>
<taskdef resource="liquibasetasks.properties">
<classpath refid="master-classpath"/>
</taskdef>
<updateDatabase
changeLogFile="${db.changelog.file}"
driver="${database.driver}"
url="${database.url}"
username="${database.username}"
password="${database.password}"
promptOnNonLocalDatabase="${prompt.user.if.not.local.database}"
dropFirst="false"
classpathref="master-classpath"
/>
</target></project>
Make sure your liquibase jar file(s) are referenced in the classpath element.
The properties file contains the references that are particular to your environment:
liquibasetasks.properties
db.changelog.file=YOUR_MIGRATION_FILE.xml
#################################
## DB Settings
#################################
database.driver=
database.username=
database.password=
database.url=
Ok, so now we have the ant task set up and configured.. with all of that saved, you should be able to run the migration by typing the following at the command prompt:
linux>ant -f ant-migrations.xml update-database
Hope that helps!

Getting Nant.MailLogger to work on Windows 2008

I have had a NAnt/NAntContrib build running for a while on one machine:
(MS Windows Server 2003 Standard 32-bit SP2)
And I now need to run the same build script on a newer machine:
(Windows Server Standard 2008)
I have gotten NAnt and NAnt.Config installed and working on the new machine.
I am using NAnt.Core.Maillogger on the original machine, configured as such:
<property name="MailLogger.mailhost" value="mail.server.com" />
<property name="MailLogger.from" value="autobuild#hostredacted.com" />
<property name="MailLogger.failure.notify" value="true" />
<property name="MailLogger.success.notify" value="true" />
<property name="MailLogger.failure.to" value="team#hostredacted.com" />
<property name="MailLogger.success.to" value="team#hostredacted.com" />
<property name="MailLogger.failure.subject" value="AUTOBUILD: Failure on TEST" />
<property name="MailLogger.success.subject" value="AUTOBUILD: Success on TEST" />
<property name="MailLogger.failure.attachments" value="MailLogger.failure.files" />
<property name="MailLogger.success.attachments" value="MailLogger.success.files" />
<fileset id="MailLogger.failure.files">
<include name="build.log" />
</fileset>
<fileset id="MailLogger.success.files">
<include name="build.log" />
</fileset>
I run a very simple test .build file, to test mail functionality:
<target name="test_mail_pass">
<echo message="Test Success:
run by ${environment::get-user-name()}"/>
</target>
<target name="test_mail_fail">
<echo message="Test Fail:
run by ${environment::get-user-name()}"/>
<fail message="Some Failure occurred." />
</target>
The above works on the original machine, and seems to work on the new machine, except for the fact that no mail is sent.
There is no message in the console that indicates that anything went wrong (ignoring the obvious use of the <fail> task).
I don't even know where to begin figuring out what is wrong here, or how to troubleshoot this problem.
Any assistance would be greatly appreciated.
I have solved this problem with much Google-ing.
One of two things solved my problem, and I don't know which it was, but my problem is now solved.
My batch file needed to have the following command-line option:
-logger:NAnt.Core.MailLogger
The file that was referred to in the:
<fileset id="MailLogger.failure.files">
<include name="build.log" />
</fileset>
<fileset id="MailLogger.success.files">
<include name="build.log" />
</fileset>
Need to actually exist.
One post I read (lost the link) told of a problem where if the files to attach do not exist, the Mail will just not get sent.

Do I have a way to check the existence of a directory in Ant (not a file)?

How do I check for the existence of a folder using Ant?
We can check the existence of a file, but can we do the same for a folder as well?
You use the available task with type set to "dir".
For example:
<available file="${dir}" type="dir"/>
The standard way to do conditional processing is with the condition task. In the example below, running doFoo will echo a message if the directory exists, whereas running doBar will echo a message unless the directory exists.
The dir.check target is required by both doFoo and doBar, it sets the dir.exists property to true or false depending on the result of the available task. The doFoo target will only run if that propery is set to true and doBar will only run if it is not set or set to false.
<?xml version="1.0"?>
<project name="test" default="doFoo" basedir=".">
<property name="directory" value="c:\test\directory"/>
<target name="doFoo" depends="dir.check" if="dir.exists">
<echo>${directory} exists</echo>
</target>
<target name="doBar" depends="dir.check" unless="dir.exists">
<echo>${directory} missing"</echo>
</target>
<target name="dir.check">
<condition property="dir.exists">
<available file="${directory}" type="dir"/>
</condition>
</target>
</project>
Antelope provides additional tasks, including an If task that can make the processing simpler (and to me, more intuitive), you can download the Antelope tasks from the download page.
Here's a small example incorporating the available element into an if test.
<!-- Test if a directory called "my_directory" is present -->
<if>
<available file="my_directory" type="dir" />
<then>
<echo message="Directory exists" />
</then>
<else>
<echo message="Directory does not exist" />
</else>
</if>
Warning: you need ant-contrib.jar in your ANT_HOME\lib directory otherwise you won't have access to the if elements, and your script will fail with this error:
Problem: failed to create task or type if
Cause: The name is undefined.
Action: Check the spelling.
Action: Check that any custom tasks/types have been declared.
Action: Check that any <presetdef>/<macrodef> declarations have taken place.
Here's my solution, which doesn't require setting properties and using targets with 'if' or 'unless':
Macro:
<macrodef name="assertDirAvailable">
<attribute name="dir" />
<sequential>
<fail message="The directory '#{dir}' was expected to be available but is not">
<condition>
<not>
<available file="#{dir}" type="dir" />
</not>
</condition>
</fail>
</sequential>
</macrodef>
Usage:
<assertDirAvailable dir="${dirToCheck}" />
My solution using ANT 1.8 version, older versions may not work due if/unless not supporting ${evalTrueOrFalse} syntax.
<?xml version="1.0" encoding="UTF-8"?>
<project name="DoMagic" default="build" basedir=".">
<property environment="env" />
<property name="name" value="Do the ANT Magic" />
<property name="somedir" value="./must_exist_folder"/>
<tstamp><format property="TODAY" pattern="yyyy-MM-dd HH:mm:ss" /></tstamp>
<target name="doMagic" if="${dir.exists}">
<echo message="Do the magic stuff" />
</target>
<target name="doUsage" unless="${dir.exists}">
<echo message="Do usage and help" />
</target>
<target name="build">
<echo message="Do the magic" />
<condition property="dir.exists" else="false"><available file="${somedir}" type="dir" /></condition>
<echo message="Folder found: ${dir.exists}" />
<antcall target="doCustomize"></antcall>
<antcall target="doUsage"></antcall>
</target>
</project>
ANT 1.6 or early ANT 1.7 does not work, upgrade to ANT 1.8 release.
Target attributes if and unless evaluates ${var} syntax to true/false
Condition attribute else value is set to property if available condition was false, without it variable is not set. NotSet value is not same as an explicit false value.
call any target but if/unless attribute defines whether its actually run
http://ant.apache.org/manual/properties.html#if+unless
[If/Unless] In Ant 1.7.1 and earlier, these attributes could only be property names. As of Ant 1.8.0, you may instead use property expansion. Compared to the older style, this gives you additional flexibility.
Here is another approach, allows to call just one task without using ant-contrib.jar.
<target name="my-task" depends="dir-check">
<antcall target="my-task-install"/>
<antcall target="my-task-update"/>
</target>
<target name="my-task-install" unless="dir.exists" >
{some task}
</target>
<target name="my-task-update" if="dir.exists" >
{another task}
</target>
<target name="dir-check">
<condition property="dir.exists">
<available file="my-dir" type="dir" />
</condition>
</target>
Here is another example involving for loop. Fail if a directory does not exist.
<for list="dir1/, dir2/, dir3/" param="local.dir" >
<sequential>
<fail message="Directory #{local.dir} does not exist">
<condition>
<not>
<available file="#{local.dir}" type="dir" />
</not>
</condition>
</fail>
</sequential>
</for>