how to run single test case from jenkins - selenium

I have 10 test cases in testng suite,i need to execute single or individual test cases from jenkinks,can any one help me out how to execute single test case from jenkins,i tried maven surfire commands mvn -Dtest=TestCircle test include test command into golas and executed but its not working,please refer data and correct me how to set goals in jenkins for execute single test case.
"mvn clean compile install -DPICK_CONFIGURATION_FROM="JENKINS" -DEXECUTION_MODE="Remote" -Dtest=TestCircle#EAConsoleLandingPageTest test -DPLATFORM="WEB" -DOPERATING_SYSTEM="WINDOWS" -DBROWSER="chrome" -DCOMPUTERNAME="Test" -DTEST_ENVIRONMENT="RUNFC" -DPLAN_ID="" -DTEST_TYPE="Smoke" -DGRID_URL="http://localhost:4444/wd/hub""
Below is the sample test suite:
classes>
<class
name="com.ea.automation.tests.EAConsoleLandingPageTest">
<methods>
<include name="clickOnGetStartedBtn" />
<include name="selectProjectFromLandingPage" />
<include name="clickOnEAConsoleIcon" />
<include name="verifyRequestProjectAccess" />
<include name="verifyProjectAccess" />
<include name="verifyHomePageRedierecting" />
</methods>
</class>
<class
name="com.ea.automation.tests.EAConsoleSelfServiceTest">
<methods>
<include name="clickOnCreateNewproject" />
<include name="verifyProjectUserAcess" />
<include name="switchProjects" />
<include name="clickOnProjectSettings" />
</methods>
</class>
<class
name="com.ea.automation.tests.EAConsoleNotificationAppTest">
<methods>
<include name="selectProject" />
<include name="clickOnNotification" />
<include name="checkAllWdgetNotifications" />
</methods>
</class>
<class
name="com.ea.automation.tests.EAConsoleSupportAppTest">
<methods>
<include name="clickOnSupportApp" />
<include name="checkCreateTicket" />
<include name="clickOnCloseErrorMessageModel" />
<include name="clickOnSupportFromApp" />
<include name="clickOnSupportEmail" />
</methods>
</class>
<class
name="com.ea.automation.tests.EAConsoleSDKDownloadsAppTest">
<methods>
<include name="selectProject" />
<include name="clickOnSDKDownloads" />
</methods>
</class>

The approach I am suggesting is not preferable. Because the sole purpose of running Jenkins is to run a suite of test cases and not individual ones. But you could implement your idea in the following way.
Add three string parameters to the Jenkins job. One parameter points to the testng xml file name, second parameter points to the class name, and the third parameter points to the test name.
Create and have another testng.xml file in your project repo. Rename this file as small.xml
So the first Build parameter should be small.xml
Jenkins has the capability to execute shell script before the maven build starts.
So we are going to fetch the arguments that we used as Jenkins Build parameters inside the shell script to modify the small.xml file and modify the pom.xml file to point to the small.xml.
sed command in a shell script will help to replace the existing text in pom.xml and small.xml file. You could search this on the internet.
I had implemented this approach in one of the projects that I worked on for picking different suite files based on the Jenkins build parameters. This approach works. So during runtime, based on the Jenkins Build Parameters you pass, the corresponding tests will be run by Jenkins.

Related

TestNG not ordering classes as specified in the testng.xml file

My testng.xml file looks like:
<suite name="Suite" parallel="none">
<test name="Test">
<classes>
<class name="testng.Test1"/>
<methods>
<include name="Browse()"></include>
<include name="Login()"></include>
<include name="Reg()"></include>
</methods>
<class name="testng.Test2"></class>
</classes>
</test> <!-- Test -->
The output of Test1 is:
*Login
Register
Browse*
The output of Test2 is
2
But, when i run the testng.xml file, the output looks like:
*Login Register 2 Browse*.
So, before the Test1 is completed, it is picking the Test2 and printing the output.
How can we run it, so that Test1 is executed first completely, followed by Test2?
I tried using preserve-order="true" but it didn't work.
Not sure what exactly you are asking for here but according to your xml file, you should include your methods in your class before closing it.
<suite name="Suite" parallel="none">
<test name="Test">
<classes>
<class name="testng.Test1">
<methods>
<include name="Browse()"></include>
<include name="Login()"></include>
<include name="Reg()"></include>
</methods>
</class>
<class name="testng.Test2"/>
</classes>
</test> <!-- Test -->
This way all the methods in your "Test1" will run first and then rest of the tests will run.
Learn more here.

TestNG XML with groups and packages not working

I am working on a set of selenium tests I want to run using a TestNG XML.
<suite name="MATS">
<parameter name="browser" value="firefox" />
<parameter name="platform" value="windows" />
<test name="mats_test">
<groups>
<run>
<include name = "mats" />
</run>
</groups>
<packages>
<package name="com.sel.tests.app.set1.*" />
<package name="com.sel.tests.app.set2.*" />
<package name="com.sel.tests.app.set3.*" />
</packages>
</test>
</suite>
Directly under each of those packages there are multiple classes with the #Test(groups = { "mats" }) annotation on top. However, I get this output:
MATS
Total tests run: 0, Failures: 0, Skips: 0
Update: Ok, so cleaning the project didn't work so I imported it again in a new workspace after which it seems to be working. Closing the question. Thanks everyone.
If you're working in IntelliJ Idea try to select in main menu Run -> Edit Configurations. On left side select "TestNG" configuration, on right side select "In whole project" radio button and Working Directory: as $MODULE_DIR$ (or play with that)

Sonar Integration Tests Coverage not show + Selenium + Jboss +Jacoco

I use Sonar 4.1.1, Jboss 6.x, Jacoco 0.6.4, execute tasks with Ant I am not allowed to use Maven. In an eclipse workspace, I have two projects, one is the web application another is selenium test.
I am able to get unit test and code coverage for unit test. But sonar is not able to read the integration test file created by Jacoco. I think there might be something wrong with the way I create jacoco-it.exec file so sonar can't read it. Because sonar does read my jacoco-ut.exec file. And I am able to have both reportPath and itReportPath to read my jacoco-ut.exec file with no problem. Also thinking maybe is something wrong in my build file. I did a lot of research and tried many different ways to create the jacoco-it.exec file, different Jacoco settings and followed different examples from sonar, jacoco, other blogs but still doesn't work. I must be missing something Help!! Thanks!!
I have VM arguments for Jboss like this
-javaagent:/path to jar/jacocoagent.jar=destfile=/path for create/jacoco-it.exec
When I run selenium, the above code create a file with some data, size about 1.3MB
Here is the part of build relate to this issue
<property name="sonar.sourceEncoding" value="UTF-8" />
<property name="sonar.java.coveragePlugin" value="jacoco" />
<property name="sonar.core.codeCoveragePlugin" value="jacoco" />
<property name="sonar.dynamicAnalysis" value="reuseReports" />
<property name="sonar.jacoco.reportsPath" value="${reports.dir}/junit" />
<property name="sonar.jacoco.itReportPath" value="${reports.dir}/jacoco-it.exec" />
<property name="sonar.jacoco.reportPath" value="${reports.dir}/jacoco-ut.exec" />
<target name="unitTest" depends="compile">
<taskdef name="junit" classname="org.apache.tools.ant.taskdefs.optional.junit.JUnitTask">
<classpath>
<path refid="classpath"/>
</classpath>
</taskdef>
<!-- Import the JaCoCo Ant Task -->
<taskdef uri="antlib:org.jacoco.ant" resource="org/jacoco/ant/antlib.xml">
<classpath refid="classpath"/>
</taskdef>
<!-- Run your unit tests, adding the JaCoCo agent -->
<jacoco:coverage destfile="reports/jacoco-ut.exec" xmlns:jacoco="antlib:org.jacoco.ant">
<junit printsummary="yes" haltonfailure="yes" forkmode="once" fork="true" dir="${basedir}" failureProperty="test.failed">
<classpath location="${classes.dir}" />
<classpath refid="classpath"/>
<formatter type="plain" />
<formatter type="xml" />
<batchtest fork="true" todir="${reports.junit.xml.dir}">
<fileset dir="src">
<include name="**/*TestAdd.java" />
</fileset>
</batchtest>
</junit>
</jacoco:coverage>
</target>
<target name="coverageTest" depends="compile">
<taskdef name="junit" classname="org.apache.tools.ant.taskdefs.optional.junit.JUnitTask">
<classpath>
<path refid="classpath"/>
</classpath>
</taskdef>
<taskdef uri="antlib:org.jacoco.ant" resource="org/jacoco/ant/antlib.xml">
<classpath refid="classpath"/>
</taskdef>
<!--Run your unit tests, adding the JaCoCo agent-->
<jacoco:coverage xmlns:jacoco="antlib:org.jacoco.ant" dumponexit="true" >
<junit printsummary="yes" haltonfailure="yes" forkmode="once" fork="true" dir="${basedir}" failureProperty="test.failed">
<classpath location="${classes.dir}"/>
<classpath refid="classpath"/>
<formatter type="plain" />
<formatter type="xml" />
<formatter type="plain" usefile="false"/>
<batchtest todir="${reports.junit.xml.dir}">
<fileset dir="../HelloAppTest/src">
<include name="**/answerTest.java"/>
</fileset>
</batchtest>
</junit>
</jacoco:coverage>
</target>
The reason for that is, may be you are NOT attaching Jacocoagent.jar file to the "TARGET" (ex: JBoss / Tomcat) JVM's scope and stopping it so that it can flush the final code coverage data to the jacoco it exec file.
Once you do that (instead of using Maven/ANT's JVM scope), run your non-Unit (IT) tests and then STOP the target JVM.
After the target JVM is stopped, you'll get the final jacoco .exec file genreated for the IT tests. Use that file for sonar.jacoco.itReportPath variable and it'll work.
For ex: I pass/have this variable to the Tomcat's startup.sh script and while starting tomcat (target JVM), I use this variable within the Tomcat's actual start command.
PROJ_EXTRA_JVM_OPTS=-javaagent:tomcat/jacocoagent.jar=destfile=build/jacoco/IT/jacocoIT.exec,append=false

Selenium: running testsuite in parallel using grid

I am trying to execute the same testsuite in parallel on an arbitrary number of selenium-grid nodes.
The test suite was created with the selenium IDE and exported as testng using the batch-converter
The idea is to create the test-suite once and then launch an arbitrary number of nodes that run that particular suite in parallel
Right now, I got 1 hub running + 2 remote-controls connected to that hub
My testng.xml looks like this
<suite name="mysuite1" verbose="20" annotations="JDK" parallel="tests" thread-count="20" >
<parameter name="selenium.host" value="localhost"></parameter>
<parameter name="selenium.port" value="4444"></parameter>
<parameter name="selenium.browser" value="*firefox"></parameter>
<parameter name="selenium.restartSession" value="false"></parameter>
<test name="mytest1" preserve-order="true">
<parameter name="selenium.port" value="5557"></parameter>
<parameter name="selenium.browser" value="*firefox"></parameter>
<parameter name="selenium.url" value="http://localhost:8080"></parameter>
<classes>
<class name="my.testsuite1" />
<class name="my.testsuite2" />
</classes>
</test>
The target I'm using in the build.xml looks like this
<target name="run-parallel" depends="compile" description="Run-Parallel">
<echo>${host}</echo>
<java classpathref="runtime.classpath" classname="org.testng.TestNG" failonerror="true">
<sysproperty key="java.security.policy" file="lib/testng.policy"/>
<sysproperty key="webSite" value="${webSite}" />
<sysproperty key="seleniumHost" value="${host}" />
<sysproperty key="seleniumPort" value="${port}" />
<sysproperty key="browser" value="${browser}" />
<arg value="-d" />
<arg value="${basedir}/target/reports" />
<arg value="-suitename" />
<arg value="suite1" />
<arg value="-parallel"/>
<arg value="tests"/>
<arg value="-threadcount"/>
<arg value="20"/>
<arg value="testng.xml"/>
</java>
My problem:
When I execute the testsuite above, only one remote-control executes the test while my second remote-control remains idle.
I know that I currently address the remote-controls directly using the "selenium.port", but I am searching for a way to avoid this rigid way of assigning tests to remote-controls
When I add additional elements, all the classes listed within the elements (my.testsuite1-4) are executed in a random order.
<test name="mytest2" preserve-order="true">
<parameter name="selenium.port" value="5558"></parameter>
<parameter name="selenium.browser" value="*firefox"></parameter>
<parameter name="selenium.url" value="http://localhost:8080"></parameter>
<classes>
<class name="my.testsuite3" />
<class name="my.testsuite4" />
</classes>
My question:
How can I define a testsuite properly so that it is scheduled on any number of running remote-controls?
Thanks!
All of your tests should access the Selenium Grid hub. The hub is responsible for dispatching to nodes based upon the requested capabilities. Once you run tests in parallel, you lose the ability to define execution order. Each test should be isolated. This includes any data you may need on your backend, such as DB modifications.

Getting Nant.MailLogger to work on Windows 2008

I have had a NAnt/NAntContrib build running for a while on one machine:
(MS Windows Server 2003 Standard 32-bit SP2)
And I now need to run the same build script on a newer machine:
(Windows Server Standard 2008)
I have gotten NAnt and NAnt.Config installed and working on the new machine.
I am using NAnt.Core.Maillogger on the original machine, configured as such:
<property name="MailLogger.mailhost" value="mail.server.com" />
<property name="MailLogger.from" value="autobuild#hostredacted.com" />
<property name="MailLogger.failure.notify" value="true" />
<property name="MailLogger.success.notify" value="true" />
<property name="MailLogger.failure.to" value="team#hostredacted.com" />
<property name="MailLogger.success.to" value="team#hostredacted.com" />
<property name="MailLogger.failure.subject" value="AUTOBUILD: Failure on TEST" />
<property name="MailLogger.success.subject" value="AUTOBUILD: Success on TEST" />
<property name="MailLogger.failure.attachments" value="MailLogger.failure.files" />
<property name="MailLogger.success.attachments" value="MailLogger.success.files" />
<fileset id="MailLogger.failure.files">
<include name="build.log" />
</fileset>
<fileset id="MailLogger.success.files">
<include name="build.log" />
</fileset>
I run a very simple test .build file, to test mail functionality:
<target name="test_mail_pass">
<echo message="Test Success:
run by ${environment::get-user-name()}"/>
</target>
<target name="test_mail_fail">
<echo message="Test Fail:
run by ${environment::get-user-name()}"/>
<fail message="Some Failure occurred." />
</target>
The above works on the original machine, and seems to work on the new machine, except for the fact that no mail is sent.
There is no message in the console that indicates that anything went wrong (ignoring the obvious use of the <fail> task).
I don't even know where to begin figuring out what is wrong here, or how to troubleshoot this problem.
Any assistance would be greatly appreciated.
I have solved this problem with much Google-ing.
One of two things solved my problem, and I don't know which it was, but my problem is now solved.
My batch file needed to have the following command-line option:
-logger:NAnt.Core.MailLogger
The file that was referred to in the:
<fileset id="MailLogger.failure.files">
<include name="build.log" />
</fileset>
<fileset id="MailLogger.success.files">
<include name="build.log" />
</fileset>
Need to actually exist.
One post I read (lost the link) told of a problem where if the files to attach do not exist, the Mail will just not get sent.