How do I sense if my unit test is a member of an ordered test and, if it is, which position in that ordered test it is at? - automation

Environment:
I have a program - named CSIS - which I need to run a lot of automated tests on in Visual Studio 2010 using C#. I have a series of functions which need to be run in many different orders but which all start and end at the same 'home screen' of CSIS. The tests will either be run on their own as a single CodedUITest (.cs filetype) or as an ordered test (.orderedtest filetype).
Goal:
The goal is to open to the CSIS homepage once no matter which of the unit tests is run first and then, after all CodedUITests are finished, no matter which unit test is last, the automated test will close CSIS. I don't want to create a separate unit test to open CSIS to the homepage and another to close CSIS as this is very inconvenient for testers to use.
Current Solution Development:
UPDATE: The new big question is how do I get '[ClassInitialize]' to work?
Additional Thoughts:
UPDATE: I now just need ClassInitialize to execute code at the beginning and ClassCleanUp to execute code at the end of a test set.
If you would like the actual code let me know.
Research Update:
Because of Izcd's answer I was able to more accurately research the answer to my own question. I've found an answer online to my problem.
Unfortunately, I don't understand how to implement it in my code. I pasted the code as shown below in the 'Code' section of this question and the test runs fine except that it executes the OpenWindow() and CloseWindow() functions after each test instead of around the whole test set. So ultimately the code does nothing new. How do I fix this?
static private UIMap sharedTest = new UIMap();
[ClassInitialize]
static public void ClassInit(TestContext context)
{
Playback.Initialize();
try
{
sharedTest.OpenCustomerKeeper();
}
finally
{
Playback.Cleanup();
}
}
=====================================================================================
Code
namespace CSIS_TEST
{
//a ton of 'using' statements are here
public partial class UIMap
{
#region Class Initializization and Cleanup
static private UIMap sharedTest = new UIMap();
[ClassInitialize]
static public void ClassInit(TestContext context)
{
Playback.Initialize();
try
{
sharedTest.OpenWindow();
}
finally
{
Playback.Cleanup();
}
}
[ClassCleanup]
static public void ClassCleanup()
{
Playback.Initialize();
try
{
sharedTest.CloseWindow();
}
finally
{
Playback.Cleanup();
}
}
#endregion

Microsoft's unit testing framework includes ClassInitialise and ClassCleanUp attributes which can be used to indicate methods that execute functionality before and after a test run.
( http://msdn.microsoft.com/en-us/library/ms182517.aspx )
Rather than try and make the unit tests aware of their position, I would suggest it might be better to embed the opening and closing logic of the home screen within the aforementioned ClassInitialise and ClassCleanUp marked methods.

I figured out the answer after a very long process of asking questions on StackOverflow, Googling, and just screwing around with the code.
The answer is to use AssemblyInitialize and AssemblyCleanup and to write the code for them inside the DatabaseSetup.cs file which should be auto-generated in your project. You should find that there already is a AssemblyInitialize function in here but it is very basic and there is no AssemblyCleanup after it. All you need to do is create a static copy of your UIMap and use it inside the AssemblyInitialize to run your OpenWindow() code.
Copy the format of the AssemblyInitialize function to create an AssemblyCleanup function and add your CloseWindow() function.
Make sure your Open/CloseWindow functions only contains basic code (such as Process.Start/Kill) as any complex variables such as forms have been cleaned up already and won't work.
Here is the code in my DatabaseSetup.cs:
using System.Data;
using System.Data.Common;
using System.Configuration;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using Microsoft.Data.Schema.UnitTesting;
using System.Windows.Input;
using Keyboard = Microsoft.VisualStudio.TestTools.UITesting.Keyboard;
using Mouse = Microsoft.VisualStudio.TestTools.UITesting.Mouse;
using MouseButtons = System.Windows.Forms.MouseButtons;
namespace CSIS_TEST
{
[TestClass()]
public class DatabaseSetup
{
static private UIMap uIMap = new UIMap();
static int count = 0;
[AssemblyInitialize()]
public static void InitializeAssembly(TestContext ctx)
{
DatabaseTestClass.TestService.DeployDatabaseProject();
DatabaseTestClass.TestService.GenerateData();
if(count < 1)
uIMap.OpenWindow();
count++;
}
[AssemblyCleanup()]
public static void InitializeAssembly()
{
uIMap.CloseWindow();
}
}
}

Related

Actionscript, can a class be accessed using a variable name?

I wish to access many classes and variables, I would like to do this by dynamically setting the class name and variable name. Currently I am using
MyClass["myVariable1"]
to dynamically access the variable name
MyClass.myVariable1
I want to also dynanmically acces the class name, something like
["MyClass"]["myVariable1"]
But this does not work.
The purpose is that I have shared object with many user settings, I want to iterate through the shared object and set all the user settings across all the classes. I think if I cant dynamically access the class I must have a statement for each and every class name/variable.
I advise against such a practice. Although technically possible, it is like welcoming a disaster into the app architecture:
You rely on something you have no apparent control of: on the way Flash names the classes.
You walk out of future possibility to protect your code with identifier renaming obfuscation because it will render your code invalid.
Compile time error checks is better than runtime, and you are leaving it to runtime. If it happens to fail in non-debug environment, you will never know.
The next developer to work with your code (might be you in a couple of years) will have hard time finding where the initial data coming from.
So, having all of above, I encourage you to switch to another model:
package
{
import flash.net.SharedObject;
public class SharedData
{
static private var SO:SharedObject;
static public function init():void
{
SO = SharedObject.getLocal("my_precious_shared_data", "/");
}
static public function read(key:String):*
{
// if (!SO) init();
return SO.data[key];
}
static public function write(key:String, value:*):void
{
// if (!SO) init();
SO.data[key] = value;
SO.flush();
}
// Returns stored data if any, or default value otherwise.
// A good practice of default application values that might
// change upon user activity, e.g. sound volume or level progress.
static public function readSafe(key:String, defaultValue:*):*
{
// if (!SO) init();
return SO.data.hasOwnProperty(key)? read(key): defaultValue;
}
}
}
In the main class you call
SharedData.init();
// So now your shared data are available.
// If you are not sure you can call it before other classes will read
// the shared data, just uncomment // if (!SO) init(); lines in SharedData methods.
Then each class that feeds on these data should have an initialization block:
// It's a good idea to keep keys as constants
// so you won't occasionally mistype them.
// Compile time > runtime again.
static private const SOMAXMANA:String = "maxmana";
static private const SOMAXHP:String = "maxhp";
private var firstTime:Boolean = true;
private var maxmana:int;
private var maxhp:int;
// ...
if (firstTime)
{
// Make sure it does not read them second time.
firstTime = false;
maxhp = SharedData.readSafe(SOMAXHP, 100);
maxmana = SharedData.readSafe(SOMAXMANA, 50);
}
Well, again. The code above:
does not employ weird practices and easy to understand
in each class anyone can clearly see where the data come from
will be checked for errors at compile time
can be obfuscated and protected
You can try getting the class into a variable and going from there:
var myClass:Class = getDefinitionByName("MyClass") as Class;
myClass["myVariable1"] = x;

Programmatically execute Gatling tests

I want to use something like Cucumber JVM to drive performance tests written for Gatling.
Ideally the Cucumber features would somehow build a scenario dynamically - probably reusing predefined chain objects similar to the method described in the "Advanced Tutorial", e.g.
val scn = scenario("Scenario Name").exec(Search.search("foo"), Browse.browse, Edit.edit("foo", "bar")
I've looked at how the Maven plugin executes the scripts, and I've also seen mention of using an App trait but I can't find any documentation for the later and it strikes me that somebody else will have wanted to do this before...
Can anybody point (a Gatling noob) in the direction of some documentation or example code of how to achieve this?
EDIT 20150515
So to explain a little more:
I have created a trait which is intended to build up a sequence of, I think, ChainBuilders that are triggered by Cucumber steps:
trait GatlingDsl extends ScalaDsl with EN {
private val gatlingActions = new ArrayBuffer[GatlingBehaviour]
def withGatling(action: GatlingBehaviour): Unit = {
gatlingActions += action
}
}
A GatlingBehaviour would look something like:
object Google {
class Home extends GatlingBehaviour {
def execute: ChainBuilder =
exec(http("Google Home")
.get("/")
)
}
class Search extends GatlingBehaviour {...}
class FindResult extends GatlingBehaviour {...}
}
And inside the StepDef class:
class GoogleStepDefinitions extends GatlingDsl {
Given( """^the Google search page is displayed$""") { () =>
println("Loading www.google.com")
withGatling(Home())
}
When( """^I search for the term "(.*)"$""") { (searchTerm: String) =>
println("Searching for '" + searchTerm + "'...")
withGatling(Search(searchTerm))
}
Then( """^"(.*)" appears in the search results$""") { (expectedResult: String) =>
println("Found " + expectedResult)
withGatling(FindResult(expectedResult))
}
}
The idea being that I can then execute the whole sequence of actions via something like:
val scn = Scenario(cucumberScenario).exec(gatlingActions)
setup(scn.inject(atOnceUsers(1)).protocols(httpConf))
and then check the reports or catch an exception if the test fails, e.g. response time too long.
It seems that no matter how I use the 'exec' method it tries to instantly execute it there and then, not waiting for the scenario.
Also I don't know if this is the best approach to take, we'd like to build some reusable blocks for our Gatling tests that can be constructed via Cucumber's Given/When/Then style. Is there a better or already existing approach?
Sadly, it's not currently feasible to have Gatling directly start a Simulation instance.
Not that's it's not technically feasible, but you're just the first person to try to do this.
Currently, Gatling is usually in charge of compiling and can only be passed the name of the class to load, not an instance itself.
You can maybe start by forking io.gatling.app.Gatling and io.gatling.core.runner.Runner, and then provide a PR to support this new behavior. The former is the main entry point, and the latter the one can instanciate and run the simulation.
I recently ran into a similar situation, and did not want to fork gatling. And while this solved my immediate problem, it only partially solves what you are trying to do, but hopefully someone else will find this useful.
There is an alternative. Gatling is written in Java and Scala so you can call Gatling.main directly and pass it the arguments you need to run the Gatling Simulation you want. The problem is, the main explicitly calls System.exit so you have to also use a custom security manager to prevent it from actually exiting.
You need to know two things:
the class (with the full package) of the Simulation you want to run
example: com.package.your.Simulation1
the path where the binaries are compiled.
The code to run a Simulation:
protected void fire(String gatlingGun, String binaries){
SecurityManager sm = System.getSecurityManager();
System.setSecurityManager(new GatlingSecurityManager());
String[] args = {"--simulation", gatlingGun,
"--results-folder", "gatling-results",
"--binaries-folder", binaries};
try {
io.gatling.app.Gatling.main(args);
}catch(SecurityException se){
LOG.debug("gatling test finished.");
}
System.setSecurityManager(sm);
}
The simple security manager i used:
public class GatlingSecurityManager extends SecurityManager {
#Override
public void checkExit(int status){
throw new SecurityException("Tried to exit.");
}
#Override
public void checkPermission(Permission perm) {
return;
}
}
The problem is then getting the information you want out of the simulation after it has been run.

In MSTest how to check if last test passed (in TestCleanup)

I'm creating web tests in Selenium using MSTest and want to take a screenshot everytime a test fails but I don't want to take one every time a test passes.
What I wanted to do is put a screenshot function inside the [TestCleanup] method and run it if test failed but not if test passed. But how do I figure out if a last test passed?
Currently I'm doing bool = false on [TestInitialize] and bool = true if test runs through.
But I don't think that's a very good solution.
So basically I'm looking for a way to detect if last test true/false when doing [TestCleanup].
Solution
if (TestContext.CurrentTestOutcome != UnitTestOutcome.Passed)
{
// some code
}
The answer by #MartinMussmann is correct, but incomplete. To access the "TestContext" object you need to make sure to declare it as a property in your TestClass:
[TestClass]
public class BaseTest
{
public TestContext TestContext { get; set; }
[TestCleanup]
public void TestCleanup()
{
if (TestContext.CurrentTestOutcome != UnitTestOutcome.Passed)
{
// some code
}
}
}
This is also mentioned in the following post.

What is the most efficient way to handle the lifecycle of an object with COM interop?

I have a Windows Workflow application that uses classes I've written for COM automation. I'm opening Word and Excel from my classes using COM.
I'm currently implementing IDisposable in my COM helper and using Marshal.ReleaseComObject(). However, if my Workflow fails, the Dispose() method isn't being called and the Word or Excel handles stay open and my application hangs.
The solution to this problem is pretty straightforward, but rather than just solve it, I'd like to learn something and gain insight into the right way to work with COM. I'm looking for the "best" or most efficient and safest way to handle the lifecycle of the classes that own the COM handles. Patterns, best practices, or sample code would be helpful.
I can not see what failure you have that does not calls the Dispose() method. I made a test with a sequential workflow that contains only a code activity which just throws an exception and the Dispose() method of my workflow is called twice (this is because of the standard WorkflowTerminated event handler). Check the following code:
Program.cs
class Program
{
static void Main(string[] args)
{
using(WorkflowRuntime workflowRuntime = new WorkflowRuntime())
{
AutoResetEvent waitHandle = new AutoResetEvent(false);
workflowRuntime.WorkflowCompleted += delegate(object sender, WorkflowCompletedEventArgs e)
{
waitHandle.Set();
};
workflowRuntime.WorkflowTerminated += delegate(object sender, WorkflowTerminatedEventArgs e)
{
Console.WriteLine(e.Exception.Message);
waitHandle.Set();
};
WorkflowInstance instance = workflowRuntime.CreateWorkflow(typeof(WorkflowConsoleApplication1.Workflow1));
instance.Start();
waitHandle.WaitOne();
}
Console.ReadKey();
}
}
Workflow1.cs
public sealed partial class Workflow1: SequentialWorkflowActivity
{
public Workflow1()
{
InitializeComponent();
this.codeActivity1.ExecuteCode += new System.EventHandler(this.codeActivity1_ExecuteCode);
}
[DebuggerStepThrough()]
private void codeActivity1_ExecuteCode(object sender, EventArgs e)
{
Console.WriteLine("Throw ApplicationException.");
throw new ApplicationException();
}
protected override void Dispose(bool disposing)
{
if (disposing)
{
// Here you must free your resources
// by calling your COM helper Dispose() method
Console.WriteLine("Object disposed.");
}
}
}
Am I missing something? Concerning the lifecycle-related methods of an Activity (and consequently of a Workflow) object, please check this post: Activity "Lifetime" Methods. If you just want a generic article about disposing, check this.
Basically, you should not rely on hand code to call Dispose() on your object at the end of the work. You probably have something like this right now:
MyComHelper helper = new MyComHelper();
helper.DoStuffWithExcel();
helper.Dispose();
...
Instead, you need to use try blocks to catch any exception that might be triggered and call dispose at that point. This is the canonical way:
MyComHelper helper = new MyComHelper();
try
{
helper.DoStuffWithExcel();
}
finally()
{
helper.Dispose();
}
This is so common that C# has a special construct that generates the same exact code [see note] as shown above; this is what you should be doing most of the time (unless you have some special object construction semantics that make a manual pattern like the above easier to work with):
using(MyComHelper helper = new MyComHelper())
{
helper.DoStuffWithExcel();
}
EDIT:
NOTE: The actual code generated is a tiny bit more complicated than the second example above, because it also introduces a new local scope that makes the helper object unavailable after the using block. It's like if the second code block was surrounded by { }'s. That was omitted for clarify of the explanation.

Using Rhino Mocks, why does invoking a mocked on a property during test initialization return Expected call #1, Actual call #0?

I currently have a test which tests the presenter I have in the MVP model. On my presenter I have a property which will call into my View, which in my test is mocked out. In the Initialization of my test, after I set my View on the Presenter to be the mocked View, I set my property on the Presenter which will call this method.
In my test I do not have an Expect.Call for the method I invoke, yet when I run I get this Rhino mock exception:
Rhino.Mocks.Exceptions.ExpectationViolationException: IView.MethodToInvoke(); Expected #1, Actual #0..
From what I understand with Rhino mocks, as long as I am invoking on the Mock outside the expecting block it should not be recording this. I would imagine the test to pass. Is there a reason it is not passing?
Below is some code to show my setup.
public class Presenter
{
public IView View;
public Presenter(IView view)
{
View = view
}
private int _property;
public int Property
get { return _property;}
set
{
_property = value;
View.MethodToInvoke();
}
}
... Test Code Below ...
[TestInitialize]
public void Initilize()
{
_mocks = new MockRepository();
_view = _mocks.StrictMock<IView>();
_presenter = new Presenter(_view);
_presenter.Property = 1;
}
[TestMethod]
public void Test()
{
Rhino.Mocks.With.Mocks(_mocks).Expecting(delegate
{
}).Verify(delegate
{
_presenter.SomeOtherMethod();
});
}
Why in the world would you want to test the same thing each time a test is run?
If you want to test that a specific thing happens, you should check that in a single test.
The pattern you are using now implies that you need to
- set up prerequisites for testing
- do behavior
- check that behavior is correct
and then repeat that several times in one test
You need to start testing one thing for each test, and that help make the tests clearer, and make it easier to use the AAA syntax.
There's several things to discuss here, but it certainly would be clearer if you did it something like:
[TestMethod]
ShouldCallInvokedMethodWhenSettingProperty()
{
var viewMock = MockRepository.GenerateMock<IView>()
var presenter = new Presenter(viewMock);
presenter.Property = 1;
viewMock.AssertWasCalled(view => view.InvokedMethod());
}
Read up more on Rhino Mocks 3.5 syntax here: http://ayende.com/Wiki/Rhino+Mocks+3.5.ashx
What exactly are you trying to test in the Test method?
You should try to avoid using strict mocks.
I suggest using the Rhino's AAA syntax (Arrange, Act, Assert).
The problem lied with me not understanding the record/verify that is going on with Strict mocks. In order to fix the issue I was having this is how I changed my TestInitilize function. This basicaly does a quick test on my intial state I'm setting up for all my tests.
[TestInitialize]
public void Initilize()
{
_mocks = new MockRepository();
_view = _mocks.StrictMock<IView>();
_presenter = new Presenter(_view);
Expect.Call(delegate { _presenter.View.InvokedMethod(); });
_mocks.ReplayAll();
_mocks.VerifyAll();
_mocks.BackToRecordAll();
_presenter.Property = 1;
}