I am wondering about where to start in building a test framework here.
I create a vb.net application to display the list of projects available. Allow user to select the project, time and date when test needs to get executed.
Once the user decides the time and task, I want my system to schedule a task onto a remote machine where the test execution would happen at the specified time.
I am stuck at point two. any pointers or question is much appreciated.
I use testcomplete for automation.
I want my system to schedule a task onto a remote machine where the test execution would happen at the specified time
There's a Windows Task Scheduler and associated API that supports scheduling tasks at specific times. The API is aimed at C++ programmers.
You could use the Task Scheduler Managed Wrapper available on CodePlex for easy interop with VB.Net.
The task to execute could be copied to a network drive so that it is accessible from the remote machine.
For point 2 you'll have to call TestComplete from the command line as per these instructions:
http://support.smartbear.com/viewarticle/55587/
You can also call TestExecute from the command line, it's a cut down version of Test Complete that will run your tests. Your license may or may not include that.
Did you also consider taking a look at Jenkins for scheduling your test runs?
Related
Our client uses an automation software called ActiveBatch (by Advanced Systems Concepts, Inc.). They're currently using ActiveBatch v8 and is now on the the process of migrating the automated jobs to a newer ActiveBatch v11.
Most the jobs have no problems coping with the newer software and they're running OK as of this writing. However, there is one job that is unable to run, rather, initialize in the first place. This job runs OK on v8. Whenever this job is being run on v11, it produces an error message:
%ABAT-W-CREPRCERR, error creating batch process for job %1
Quite self-explanatory; means the process for the particular job was not created. As per checking the user manual, it stated that the job's log file might explain more why the error occurred. Problem is, the log file is not very helpful as it only show magic numbers shown below:

Further readings states that it's Byte Order Mark for UTF-8. I don't know much about this stuff but since the log file only contains those characters, I'm not sure they're helpful at all.
Another thing, if I run the job manually (running EXE via Windows Explorer), no problems will be encountered and it will be a success. The job by the way is a Power Builder 9 application.
I have sequence of steps that an user does, e.g. logging on the a remote UNIX shell, creation of files/directories, changing permission, Running remote Shell scripts and commands, File deletion, File movements,
Run DB queries and basis the query results perform certain tasks exporting the results to a file or run further shell commands/scripts or DB insert statements etc etc.
doing there steps users achieves different processed or data processing and validating.
What is the best way to automate the above schenerio, Should we go for a Workflow tools like Activiti etc. or is there a better framework/way to achieve the requirements.
My requirement is to work with Open-source, and possibly Java based.
I am completely new to this so any help pointers would be appreciated.
The scenario you describe is certainly possible with a workflow tool like Activiti. Apache Camel or Spring Integration would be another possibility (as all the steps you mention are automatic system tasks).
A workflow framework would be a good option if you need one of these
you want to store the history data for 'audit purposes': who did what/when/how long did it take.
you want to visually model your steps, perhaps to discuss it with business people.
there is a need for human interaction between some of the steps
Your description reminds me of a software/account provisioning process.
There are a large number of provisioning tools on the market both Open Source or otherwise (Dell Crowbar is one options).
However, A couple of the comments you made in your response to Joram indicate a more general purpose tool such as Activiti may be an option:
"Swivel Chair" tasks - User tasks that may one day be automated
Visual model of process state
Most provisioning tools dont allow for generic user tasks and dont provide a (good) visual model of the process state.
However, they generally include remote script execution which would need to be cobbled together as a service task if using a BOM tool.
I would certainly expand my research to include provisioning tools as they sound like a better fit, however if you cant find anything that works for you, a BPM platform provides a generic framework to build what you need.
In SSIS package i have multiple scripts running within a job. At any given time i want to read how many scripts have been executed eg, 5/10 (50%) have been completed. Please tell how can i achieve that?
Currently there is no such functionality provided by SSIS to track progress of package execution.
It seems you need to write your own custom utility/application to implement same or use third party one.
There are few ways to do -
Using a switch called /Reporting or /Rep of DTEXEC at the command-line . For example:
DTEXEC /F ssisexample.dtsx /Rep P > progress.txt
2.Implement package logging or customize it.
3 . Implement Event handler on required executable. You can also use OnPipelineRowsSent log of Data Flow Task.
If you want to write your own application then below thread will provide nice starting point.
How do you code the Package Execution Progress window in C#
In my experience, we are using another software to monitor the jobs that are running. http://en.wikipedia.org/wiki/CA_Workload_Automation_AE
You can also try to create your own application that runs on the background that checks that status of your jobs, through checking the logs.
I am trying to create a unit test to run on two machines in Microsoft Test Manager 2010. In this test I want some client and server side test code to run simultaneously; the client side test being dependent on server side test working successfully.
When putting together a Test Suite in Test Manager, I want to be able to set both tests to have the same order value (so they run at the same time) but the validation prevents this; setting the order as shown below:
Is there any way I can achieve the simultaneous test execution I am after?
Sorry for the late answer... I've missed the notification about your answers to my question :-( Sorry for that!
In case you are still looking for solution, here my suggestion.
I suppose you have a test environment consisting of two machines (for server and client).
If so, you will not be able to run tests on both of them, or better to say you will not have enough control over running tests. Check How to Run automated tests on multiple computers at the same time
Actually I posted a related question to "Visual Studio Development Forum", you could check the answers I got here: Is it possible to run test on several virtual machines, which belong to the same environment, using build-deploy-test workflow
That all means you will end up creating two environments each consisting of one machine (one for server and one for client).
But then you will not be able to reference both environment in your build definition it you can only select one environment in DefaultLabTemplate.
That leads to the solution I can suggest:
Create two lab environments
Create three build definitions
the first one will only build your test code
the second one will deploy last successful build from the first one and start tests on the server environment
the third one will deploy last successful build from the first one and start tests on the client environment.
Run the first build definition automatically at night
Trigger the latter two simultaneously later.
It's not really nice, I know...
You will have to synchronize the build definition building the test code with the two build definitions running the tests.
I was thinking about setting up similar tests some months ago and it was the best solution I came up with...
Another option I have not tried yet could be:
Use a single test environment consisting of two machines and use different roles for them (server and client respectively).
In MTM create two Test Settings (one for the server role and one for the client role).
Create a bat file starting tests using tcm.exe tool (see How to: Run Automated Tests from the Command Line Using Tcm for more details).
You will need two tcm.exe calls, one for each Test Settings you have created.
Since a tcm.exe call just queues a test run an returns (more or less) immediately this bath file will start tests (more or less) simultaneously.
Create a build definition using DefaultLabTemplate.
This definition will:
build test code
deploy them to both machines in your environment
run your bath script as the last deployment step
(you will have to make sure this script is located on the build machine or deploy it there or make it accessible from the build machine)
As I've said, I have not tried it yet.
The disadvantage of this approach will be that you will not see the test part in the build log since the tests will not be started by means provided by DefaultLabTemplate. So the build will not fail when tests fail.
But you will still be able to see test outcomes in MTM and will have test results for each machine.
But depending on what is more important to you (having rest results or having build definition that fails if tests fail or having both) it could be a solution for you.
Yes, you can with modified TestSettings file.
http://blogs.msdn.com/b/vstsqualitytools/archive/2009/12/01/executing-unit-tests-in-parallel-on-a-multi-cpu-core-machine.aspx
I'm trying to integrate running Fitnesse tests from MSBuild im my nightly build on TFS.
In an attempt to make it self contained I would like to start the seleniumRC server only when it's needed from fitness.
I've seen that there is a "Command Line Fixture" but it's written in java can I use that?
I think you might be able to. You can call any process easily in MSBuild using the task. However, the problem with doing this is that the exec task will wait for the Selinium process to finish before continuing, which is not the bahaviour you want. You want to run the process, keep it running during your build and then tear it down as your build finishes.
Therefore, I think you are probably going to need to create a custom MSBuild task to do this. See the following post for an example of a tasks that someone has created that will run asynchronously returning control back to the build script:
http://blog.eleutian.com/2007/03/01/AsyncExecMsBuildTask.aspx
And for an example of calling a Java program from MSBuild (but in this case synchronously) take a look at my task that calls Ant from MSBuild here
http://teamprise.com/products/build/
As part of your MSBuild task, you will want to output the process id that you created to an output property so that at the end of your build script you can call another custom MSBuild task that kills the process. It can do this by looking for the process id passed in as a variable in MSBuild and then call Process.Kill method i.e.
Process process = Process.GetProcessById(ProcessId);
process.Kill();
That said, you would need to be careful to ensure that your kill task was always executed in MSBuild by making sure it was included during error paths etc in the build. You could probably make things a bit more resilient by making the selenium RC starter task look for other seleniumRC processes and killing them before starting a new one - that way if a process didn't get closed properly for some reason, it would only run until the next build.
Anyway - my answer sounds like a lot of work so hopefully someone else will come up with an easier way. You might be able to create the seleniumRC process in the test suite start up of the FitNesse tests and kill it in the suite tear down, or you might be able to write a custom task that extends your FitNesse runner tasks and fires up seleiniumRC asynronously before running the test process and then kills it afterwards.
Good luck,
Martin.
Thanks for your replies!
This is how I've done so far.
I made a fit fixture (very simple) that starts a process with the supplied command line, in my case startSelenium.bat. The fixture returns the ProcessID so I can store that in my fitnesse context and close that session later.
I can now make a SuiteSetUp page in my fitnesse test that looks like this.
|RunCommandFixture|
|Commandline|RunCommand?|
|C:\Projects...\startSeleniumRC.bat|>>seleniumprocess|
and a SuiteTearDown like this
|RunCommandFixture|
|ProcessID|StopCommand?|
|<
That works for me. No selenium RC starts by request from my fitnesse test.
What about writing a simple .NET app that does a Process.Start("selenumRC commandline") which gets run by your build script?
If you aren't too far down the Selenium route; might I suggest that you look at similar .NET browser automation tools; specifically WatiN or ArtOfTest. The "stacks" in these are completely .NET, so getting them running on different machines is much easier.