Determine whether or not web application is running in UI test mode - testing

I am trying to figure out the best way to determine if I am running UI tests for a web application. The reason I am trying to do this is because if I am running UI tests, the only purpose of these tests are to make sure that the UI is working properly and to do that, they should run against mocked APIs (we have a separate set of integration tests to make sure the UI and a true backend API work properly together). Also mocking the API calls will make test test run a lot faster which is another reason to mock them. I consider these "unit tests" for the UI.
I also don't want to have 2 separate copies of the same codebase where everything is the same except the UI test version includes the javascript file that mocks all the required calls needed for the UI tests to run properly. If I where able to figure out that I am running the application in UI test mode then I would be able to know whether or not to include the javascript file to mocks the calls.
Is there any "standard" or "accepted" way to do something like this?

When you start running tests - raise a flag in the DB and have a service you can call to check that flag. make sure to turn that flag off once the tests ended.

The short answer to "Is there any standard or accepted way to do something like this?" would be: no.
This is mainly because you don't want your UI to know this kind of information at all. You want your UI to just be your UI. As soon as your UI starts taking some decisions based on whether it's in "test mode" or "production mode", you embark on a slippery slope that will ultimately lead to a nightmare code-base.
This does not mean your problem cannot be solved; just that the solution should be approached in a different way. I'll first explain the general principles without any language specifics, then provide some guidelines for javascript.
General Principles
The only reason for you to be struggling with this is that your UI is too tightly coupled to the API.
The solution happens to be exactly the same as any situation when you wish to use mocks.
Program to an interface, not an implementation. (Ensure your UI binds only to an abstraction of the API - not the "true/production API".)
Separate instantiation from interaction. (Don't let your UI create any of its API dependencies, because that binds it to a specific implementation - rather provide interface on the UI for it to be given the specific API instance it should use.)
Program to an interface
First note that the above phrase does not mean your language needs to support an "interface" construct. (It's just an unfortunate choice of name by some language implementors.)
Define a base class/object which defines each of the methods/messages that your API should support. (However, none of these will actually be implemented on the base class/object.)
Your UI should have a variable/field/reference to the APIInterface.
Your UI will call the methods it needs from the API via the interface reference. E.g. APIRef.DoMethod1(...) or APIRef->DoMethod1(...) or [APIRef DoMethod1:...] etc.
Separate instantiation from interaction
The thing to avoid here is:
CreateUI {
APIRef = CreateAPI;
}
The above binds your UI to a specific implementation, and forces you to include those files/dependencies in your UI code. You would rather have your UI be told which API to use. E.g.
CreateUI(APIInterface APIToUse) { //NB: Notice that the type use to refer
//to the API is the abstract base type
//defined earlier (keeping to the "Program
//to an interface" principle).
APIRef = APIToUse;
}
//or
SetAPI(APIInterface APIToUse) {
APIRef = APIToUse;
}
Now your production application could look something like this:
API = CreateTrueAPI;
UI = CreateUI(API);
Whereas your test application could look something like this:
API = CreateMockAPI;
UI = CreateUI(API);
Notice how with this solution, your UI doesn't have a clue about "test mode" or "production mode". It just uses the API it is given. The only thing that knows about the "test mode" (in a manner of speaking) and the mock API is the test application.
Applying the principles to Javascript
First, let me state for the record: although I am familiar with the language principles of Javascript, I have never done JS development. So there may be some unforeseen complications. However, in the worst case, with a little tweaking and research, I'm sure you'll figure something out.
Javascript supports duck-typing which basically means you can send any message to any object, and at runtime the object will decide if it can actually process the message. You lose out on compile-time checking that you haven't made any typo errors, but as I understand it: you don't really need to define the abstract base interface at all.
So...
Simply ensure your UI has a reference to an API object.
Ensure your UI doesn't include any API implementation files (neither the true/production version nor the mock version).
In your production host create the true API, create the UI and pass the true API to the UI.
In your test host create the mock API, create the UI and pass the mock API to the UI.

Related

Windows Mobile - Automated Testing Tool for Non-UI Application

I need to automate testing of my windows mobile application. My application does not have any UI. So, normal testing tools which works with random key strokes and mouse clicks will not work here. Are there any tools available for windows mobile to test only background processing?
You have a couple of options depending on what level of testing you want.
Integrated Test
An integrated test aims to test the application in the real world. Therefore you would create all of the "real" things and write code to specifically test to see if your conditions are met. I would believe however, if you're trying to test GPS then this would not be practical. As someone would actually have to move the device around.
Unit Test by mocking
I've done this before for GPS testing. The idea is that you SANDBOX the object being tested. You ensure that all external references (e.g. anything that isn't the object) are interfaces. You then MOCK these interfaces with "test-only" implementations.
For example I worked on a GPS test where we used an interface called: INmeaInterpreter to fire certain events which would be picked up by a class named PositioningService.
The default implementation was a 3rd party component.
However as INmeaInterpreter was an interface we could create an implementation that instead of using the REAL data uses (for example) an NMEA file to read from. This enabled us to test how the PositioningService worked in certain (and sometimes strange) scenarios.
I would then suggest mocking the other external references. The call to the database can just be a dummy object with a counter for the database call that is incremented if it is called. You could then write a test with an NMEA file that should result in a database call and then at the end of the unit test check that dummy object to see if that call occurred.
We did all the above with horrible MSTest but you could use any testing framework (I recommend NUnit). I'm not sure if there are options to specifically test on the device. We ran all of our tests on desktop as we'd split the code nicely so that device specific code was isolated and could easily be replaced with Desktop equivalents.
Obviously the only problem with unit tests is that they dont test the hardware.
I would recommend (depending on the size of the project and the team) to do BOTH types of testing but to place the larger emphasis on the unit tests (as they are easier to run/manage).

How to choose the perfect RESTful framework?

I know this question is too wide to be answered with a simple "use this framework", but I would really appreciate your advice on that one.
I'm looking to make a (quite complex) project than will run over an API. I'm open to any programming language (PHP, Python, Java mostly) and found many frameworks that are more oriented to make a RESTful web server.
The only major constraint I have is that I would have a reusable, simple and not-code-spaghetti independent package in order to improve my API later easily or even switch to an other framework with no pain.
For Python & Java, I thought about making a dedicated package. Each action would call the dedicated method in the package, the package would return object/dict and the action would transform it to the proper format.
After many research, I hesitate between two framework that could be good for my work but I need your advice because I wouldn't make any mistakes here.
Play! Framework (Java)
Pros :
Router are RESTFul oriented (you define the method (GET, POST, etc), the request and the class.method to use)
You don't have to make one class per action
Cons :
The Model is already included. If I later change the framework, maybe I will be stuck with it (but apparently not since Play! seems to use JPA)
Maybe the fact that if I want to send parameters to the action that would be defined in the method signature, I have to adopt the ClassName.properties instead of a json like {ClassName: {properties: 'value'}}
Tornado Web (Python)
Pros :
Seems to be very powerful : used by FriendFeed (at least) !
Auth via major OpenId, OAuth and Facebook already implemented
Very light (could be a problem)
Cons :
Not so popular : you understand better the work by going into the code than the doc
Urls seems to be very basics (As far as I saw it, you have to define all the urls in one file, with all the class included)
One Class per action (that could be heavy)
Decorators for the basic (testing if user is auth, etc) must be made
For using them in production, it would be easily possible with apache & mod_proxy or nginx.
So, my questions is quite simple : what would you choose (between those two or others, I'm not closed to suggestions) and why ?
Thank you really much for your advice!
My favorite RESTful Web App development framework is Restlet. It's a Java framework/library (it can be thought of as either) but it works well with Jython and JRuby, so if you prefer those languages you could still use it. I mostly use it with Groovy.
I prefer Restlet because:
Its API fully embraces and aligns with RESTful paradigms, so it encourages you to work RESTfully. For example, when a Router routes a request to a ServerResource, it creates a new instance of the ServerResource for every request. This encourages the implementation to be stateless. And there's a rich class hierarchy with all the concepts required to implement a RESTful web app: Client, Server, Protocol, VirtualHost, Request, Response, MediaType, Status, etc.
Its API includes classes for writing both servers and clients, and they're very consistent and almost symmetrical. For example, there's a ServerResource class and a ClientResource class. ServerResource.get() and ClientResource.get() both return a Representation. The only difference is that you implement ServerResource.get() and generate a response representation, while you call ClientResource.get() and receive a response representation.
The API is consistent with Java conventions. For example, if a request made with ClientResource.get() receives an error response such as 401, a ResourceException will be thrown. And if you're implementing a ServerResource and want to return an error status, you just throw a ResourceException (which is a RuntimeException, which is nice).
Via its extension mechanism, it plays very nicely with a broad array of the best Java libraries around. Extensions are included for various HTTP client and server libraries, databases, templating libraries, security libs, data libs such as XML, JSON, OAuth, OData, etc., and even OSGI.
Deployment is very flexible. You can embed a Restlet-powered API in an existing Java app, an existing Java Servlet app, or any standard Java Web App (Servlet) server. Or you can build a stand-alone server app with an embedded HTTP server such as Jetty — that's my preferred approach. And because it runs on the JVM, it runs on almost any hardware or OS.
It's mature, reliable, responsibly maintained, steadily improving, and well supported both by the community and commercially.
It's open source, and has very clear and well-structured code. The developers are happy to accept any contributions. I've submitted a few patches and had them committed to trunk quickly with no drama.
Other options I'd suggest would be the Python microframework Bottle and the Ruby microframework Sinatra. They're both simple, straightforward, lightweight, and effective. And because they work with the WSGI and Rack stacks, there's a rich set of "middleware" modules which can easily be used with them.

difference between API and framework

what is the difference between these two terms, thanks in advance for any good simplifications and good examples.
A framework is a group of classes, interfaces and other pre-compiled code upon which or by the use of which applications can be built.
The API is the public face of a framework. A well designed framework only exposes those classes, interfaces, etc that are needed to use the framework. Code that supports the operation of the framework but that is not necessary to users of the framework is kept internal to the framework's assemblies/dlls. This keeps the public face of the framework small and encourages a "pit of success," or the quality of a framework which makes it simple to do the right thing.
(I provide an example from the .NET world)
The SqlConnection class is used to connect to a Sql Server instance. Its public API is pretty simple:
using (SqlConnection connection = new SqlConnection(connectionString))
{
connection.Open();
// Do work here; connection closed on following line.
}
However, this class depends on around 200 methods within the System.Data framework (in this case, an assembly), 3/4 of which are internal and not part of the public API of System.Data. Because the framework's API is kept simple, it becomes easy to use SqlConnection properly. If the user was required to deal with SqlConnectionFactory, SqlDebugContext, DbConnectionPoolGroup or any of the other internal classes required by the SqlConnection class, it would become exponentially more difficult to use SqlConnection properly. Because the API only exposes a small percentage of the framework, it is easier to create and use a connection.
An API is an interface to a (set of) component(s) encapsulating a functionality. For instance, the GoogleMaps API, the DirectX or OpenGL APIs.
A framework is more a set of tools, components aimed at helping the developer to develop his/her project in a given Frame. The framework usually sets some coding standards, provides useful components, ... For instance, Symfony/Cake are PHP web application frameworks. JUnit is a framework for unit tests in Java, ...
Frameworks can often bundle/provide a unified interface to some APIs.
Some APIs can be internally built using a framework.
API - application programming interface -> the contract you must obey when using a library's API
library - a set of classes/modules that solve a specific problem -> has an API
framework - a "bigger" set of libraries with a set of rules on how to use them
Since every library has an API, no point in giving examples.
A popular Java library for time is Joda time.
A popular Java framework is the Spring framework.
You must obey a lot of rules to use Spring well. You don't have to obey as many rules to use Joda time.
An API is something code has, not something it is. A framework has an API, but it is not itself an API.
API "Application Programming Interface" is set of prewritten packages, classes and interfaces with their respective methods. You can use it without much concern about internal implementations. API is used an interface between two or more applications and like REST API.
Framework is a skeleton that contains design patterns, classes, interfaces and libraries that can be used to build applications. Framework provides inversion of control which give the responsibility of program flow to the framework itself, also we can extend the framework without changing its predefined code. For example Spring is a framework that can be used to build web applications.
API's are pre-built-in from SDK (or from which you can include on to). Frameworks are loadable bundles wherein exposed functions of such bundles can be used. You can acquire expose functions of those frameworks by using pointer to functions.
Example:
API:
-stringWithString:
function from framework:
-myExposedMethod:
Framework is use to design an application, ie MVC, MEF. Like a model that you build on, almost a base for a certain set of functionality that you might want in your application.
API is for interaction between applications, your app would use the Facebook API to interact with Facebook.
Hope this is a bit more clear.
Java API simply means ...Application Programming Interface in which all the features describes of product or software.
Java Framework means semi-completed project or code. It provides an architecture to make project . Framework have own classes and methods etc..
An API is simply a library built with a particular language that developers can use to build applications.
Frameworks are a set of libraries, just like APIs however the syntaxes may deffer of the original language. So the developer may be writing a different syntax of PHP for example when using Symphony.
The main or core difference beteen framework and API is that framework allows developer to hook into the life cycle of the objects through lifecycle callback methods mechanism whereas API doesn't do that, API is only intended to perform a functionality only.
Another way to visualize it is this: (true of any programming language)
Any(!) "piece of software that is intended to be used by another piece of software" by-definition must have some "application program interface (API)," which represents the "knobs, switches and dials" that the other piece of software is expected (and, permitted) to use. All of the internal implementation details are not visible and cannot be reached.
"Frameworks" are tools that are designed to make it easier for humans to write a particular, common, type of application – such as a web-page. The framework implements "the stuff that every such application is going to need to be able to do," and does it in one, well-tested way, "precisely so that you (the application author) don't have to." Instead of redundantly writing "the same old thing, one more time, and fretting over whether you did it correctly," you simply leverage what the framework has already done for you.
After all...
Actum Ne Agas: Do Not Do A Thing Already Done.

Books or Articles on Using NUnit to Test Entire Features

Are there any books or articles that show you how to use NUnit to test entire features of a program? Is there a name for this type of testing?
This is different from the typical use of NUnit for unit testing where you test individual classes. This is similar to acceptance testing except that it is written by the developer to discern that the program does what they interpreted as being what the customer wants the program to do. I don't need it to be readable by non-programmers or to produce a readable specification for non-programmers.
The problem I am having is keeping this feature testing code maintainable. I need help in organizing my feature testing code. I also need help organizing the program code to be drivable in this way. I am having a hard time being able to issue commands to the program while still having good code design.
Currently I have a class called Program with a single public method called Run. With every test I start at the beginning of the program like the user would and then get to the desired point in the program where a particular feature would be available. I then use that feature in some way and verify it did what I want. I have a class called Commands that exposes different features of the program as methods. An instance of the Commands object is passed to the program and it eventually gets passed to every Form class. These will subscribe to events from the Commands class that are called by the methods of the command class(one matching event per method). The events are subscribed to by pointed to the method that is called when a certain part of the user interface is used, thus allowing the entire program to be drivable by my tests. If you call a method on the Command object for an event that is currently not subscribed to, a FeatureMissingException is thrown.
All of this works but I don't like the Command class. It is getting too large with too many responsibilities (every feature of the program). The Commands class is also a dependency magnet (all the Form classes have an instance of it but only subscribe to the events that represent features that can be activated through their UI).
It's called integration testing. Integration tests are much more difficult to make automated, and are very often done by hand. Many simpler tests can still be done using NUnit though - you don't have to do anything special, just don't use Mocks (like you should be doing for unit tests) so you can test how the modules actually fit together.
Context/specification is a good way of organizing these tests.
What you want to do is integration testing, like the other answer suggests. This will allows you to functional/feature testing. The most common framework for this for StoryQ or SpecFlow. This allows you to develop your tests in a BDD style and can be mostly be automated against the spec that you want.
Tools like Selenium allow you to do functional testing in a browser to do what the end user would do. All of these can be driven with NUnit since NUnit is purely a framework for running tests be them Unit tests to large functional tests

Is this a reasonable "Application entry point"?

I have recently come across a situation where code is dynamically loading some libraries, wiring them up, then calling what is termed the "application entry point" (one of the libraries must implement IApplication.Run()).
Is this a valid "Appliation entry point"?
I would always have considered the application entry point to be before the loading of the libraries and found the IApplication.Run() being called after a considerable amount of work slightly misleading.
The terms application and system are terms that are so widely and diversely used that you need to agree what they mean upfront with your conversation partner. E.g. sometimes an application is something with a UI, and a system is 'UI-less'. In general it's just a case of you say potato, I say potato.
As for the example you use: that's just what a runtime (e.g. .NET or java) does: loading a set of libraries and calling the application entry point, i.e. the "main" method.
So in your case, the code loading the libraries is doing just the same, and probably calling a method on an interface, you could then consider the loading code to be the runtime for that application. It's just a matter of perspective.
The term "application" can mean whatever you want it to mean. "Application" merely means a collection of resources (libraries, code, images, etc) that work together to help you solve a problem.
So to answer your question, yes, it's a valid use of the term 'application'.
Application on its own means actually nothing. It is often used by people to talk about computer programs that provide some value to the user. A more correct term is application software and this has the following definition:
Application software is a subclass of
computer software that employs the
capabilities of a computer directly
and thoroughly to a task that the user
wishes to perform. This should be
contrasted with system software which
is involved in integrating a
computer's various capabilities, but
typically does not directly apply them
in the performance of tasks that
benefit the user. In this context the
term application refers to both the
application software and its
implementation.
And since application really means application software, and software is any piece of code that performs any kind of task on a computer, I'd say also a library can be an application.
Most terms are of artificial nature anyway. Is a plugin no application? Is the flash plugin of your browser no application? People say no, it's just a plugin. Why? Because it can't run on it's own, it needs to be loaded into a real process. But there is no definition saying only things that "can run on their own" are applications. Same holds true for a library. The core application could just be an empty container and all logic and functionality, even the interaction with the user, could be performed by plugins or libraries, in which case that would be more an application than the empty container that just provides some context for the application to run. Compare this to Java. A Java application can't run on it's own, it must run within a Java Virtual Machine (JVM), does that mean the JVM is the application and the Java Code is just... well what? Isn't the Java code the real application and the JVM just an empty runtime environment that provides nothing to the end user without the loaded Java code?
I think in this context "application entry point" means "the point at which the application (your code) enters the library".
I think probably what you're referring to is the main() function in C/C++ code or WinMain in a Windows app. That is, it's the point where execution is normally started in an app. Your question is pretty broad and vague--for example, which OS are you running this on--but this may be what you're looking for. This might also address the question.
Bear in mind when you're asking questions, details are your friend. People can give you a much better, more informed answer when you provide them with details.
EDIT:
In a broader context consider what has to happen from the standpoint of the OS. When the user specifies that they want to run an app, the OS has to load the app from the hard drive and then when the app is loaded into memory, it has to pass control to some point in the memory blocked occupied by the newly loaded app to continue execution. That would be the "Application Entry Point". When an app is constructed with dynamically linked code the OS has to load all that dynamically linked code in order to get the correct app image into memory. Loading up those shared bits of code does not change the fact that the OS must have a point to which to pass control when the app is loaded into memory.