Looking for an open source web testing tool that can do Windows Authentication.
I've used openSTA and Pylot in very small amounts but found neither to do windows. openSTA says you can do NTLM here: http://www.opensta.org/docs/sclref/build-auth-blob.htm
I could not get it working - probably due to some config on the network or maybe my own mistake.
It would be a shame to limit your testing tool due to a matter of authentication. It is more practical to turn off NTLM / Windows Auth during stress testing. We have done this by adding a configuration parameter that allows the user name to be passed via URL only when configured in a stress / load testing environment. This causes a behavior change in exactly one place where we load and cache the current user. And it works with virtually any testing tool.
If you are testing an ASP.NET app, it is a matter of creating your own user principal and assigning it to the HttpContext.Current.User. A good place to do this is in the AuthenticateRequest event of an HttpModule or Membership Provider that you configure in only in the stress testing environment.
There is not much downside to this technique. This operates under the assumption that Windows authentication is relatively cheap and relatively small compared to the performance of the overall application. We have enjoyed a high confidence in our stress test results without it. The one caveat here is that IE will do a challenge/response with NTLM, so while it is not a perfect test, it is a very good test. And the perfect is the enemy of the good.
Maybe I am missing something about Windows Authentication, but I think that Selenium should be able to do that. Have you tried?
You could try out the Grinder project: NTLM support is a bit tricky but there are some details here:
http://grinder.sourceforge.net/faq.html#spnego (embedd link now dead - see comment from CruiZen)
or
http://bb10.com/java-grinder-user/2007-08/
Related
I have built an automation framework for testing our web app that runs as after each new deploy to our staging environment, as a regression pack. Now the issue is the tests fail whenever there's a new experiment that touches that specific part of the tests, e.g., the home page validation tests fail if there is a new home page experiment. I'd like to know how I can make my tests robust enough to resolve the issue maybe by ignoring experiments altogether or always ensuring the page loads in the current non-experiment group?
I thought maybe a possible solution would be for the web team to write a new cookie than controls the experiments, and then just set that cookie in a hook prior to my tests? Would that work or is there maybe a better way?
The solution with the cookie that controls your A/B experiments will work well with TestCafe. TestCafe allows you to work with cookies using the ClientFunctions mechanism or Client Scripts.
This is my first time playing with cucumber and also creating a suite which tests and API. My questions is when testing the API does it need to be running?
For example I've got this in my head,
Start express server as background task
Then when that has booted up (How would I know if that happened?) then run the cucumber tests?
I don't really know the best practises for this. Which I think is the main problem here sorry.
It would be helpful to see a .travis.yml file or a bash script.
I can't offer you a working example. But I can outline how I would approach the problem.
Your goal is to automate the verification of a rest api or similar. That is, making sure that a web application responds in the expected way given a specific question.
For some reason you want to use Cucumber.
The first thing I would like to mention is that Behaviour-Driven Development, BDD, and Cucumber are not testing tools. The purpose with BDD and Cucumber is to act as a communication tool between those who know what the system should do, those who write code to make it happen, and those who verify the behaviour. That’s why the examples are written in, almost, a natural language.
How would I approach the problem then?
I would verify the vast majority of the behaviour by calling the methods that make up the API from a unit test or a Cucumber scenario. That is, verify that they work properly without a running server. And without a database. This is fast and speed is important. I would probably verify more than 90% of the logic this way.
I would verify the wiring by firing up a server and verify that it is possible to reach the methods verified in the previous step. This is slow so I would do as little as possible here. I would, if possible, fire up the server from the code used to implement the verification. I would start the server as a part of the test setup.
This didn’t involve any external tools. It only involved your programming language and some libraries. The reason for doing it this way is that I want to to be as portable as possible. The fewer tools you use, the easier it gets to work with something.
It has happened that I have done some of the setup in my build tool and had it start a server before running the integration tests. This is usually more heavy weight and something I avoid if possible.
So, verify the behaviour without a server. Verify the wiring with a server. It is important to only verify the wiring in this step. The logic has been verified earlier, there is no need to repeat it.
Speed, as in a fast feedback loop, is very important. Building and testing the entire system should, in a good world, take seconds rather than minutes.
I have a working example if you're interested (running on travis).
I use docker-compose to launch the API & required components such as database, then I run cucumber-js tests against the running stack.
docker-compose is also used for local development & testing.
I've also released a library to help writing cucumber for APIs, https://github.com/ekino/veggies.
I've seen this question has been asked a few times, and lots of solutions get suggested - but none of them seem to work for the RemoteWebDriver (ie: using Selenium Grid). They're usually centered around using the local ChromeDriver/FirefoxDriver/IEDriver classes.
I am using the .NET bindings, by the way :).
What I want to do is fairly simple (in terms of requirement). I have a Selenium Server setup, and am currently using the RemoteWebDriver to perform automated UI tests on various sites. This setup is working fine.
However, some sites use NTLM/Windows Authentication, and we need to start writing automated tests for these. However, as far as I can tell, there is no solution for this.
I have seen the following "solutions", but - unless someone can correct me - they either don't work consistently, or will not work using RemoteWebDriver:
Using the IAlert functionality (like here). However, this isn't implemented in the .NET bindings, and doesn't work for all browsers as far as I can tell.
Using the Robot API to interact with the popup (like here). But this is for running on your local machine, and not supported by RemoteWebDriver.
Using AutoIt to do a similar thing to the Robot API. However, this won't work using RemoteWebDriver.
Passing the credentials in the URL (eg: http://username:password#example.com). However, this doesn't work for Windows Authentication - just normal HTTP Basic Authentication.
I can't actually see any other solutions, unless anyone else can help?
A workaround currently is to log onto the Selenium server, go to the sites in each browser, and save the credentials. But this isn't ideal, and adds a level of manual interaction to each test.
Any help would be appreciated :).
It appears I have found my own solution - use a proxy which adds the NTLM negotiation/authorisation automatically. Pretty simple to setup :).
http://cntlm.sourceforge.net/
I'm in the process of installing an open-source Python based web application to an internal server here at work. The existing code is open - it doesn't require a login to view it - but one of the requirements is that users have to be approved before they can see anything.
Is there a good way (using Apache configuration files for example, but any method would be great) to wrap the application so that any access requires a login? I would like to avoid modifying the open-source code (a maintenance nightmare every time a new release comes out).
Any thoughts or suggestions?
Apache supports Authentication, Authorization and Access Control.
It is a detailed process, and summarising it here would not do it justice. I refer you to the link provided,
We have developed a comet based application for chat (using streaming approach). The application has been developed in ASP .Net 3.5 sp1.
The browser has two connections with the server. One for posting and another for receiving chat messages. While load testing with Jmeter or VSTS the posting is getting recorded and load tested but not the receiving portion. Can some one please suggest any load testing tool which can address this issue.
I've come across the same problem, the top runner for me at the moment is browsermob.com. It has a complete API that allows you to create test scenarios that can "watch and wait" on pages recording every http request made as though they are visiting through a real browser. It gets kind of expensive if you need to test with more than 25 concurrent users (browser users), but seems very reasonably priced from what I have seen so far.
It'd be really interesting to see what tools others who are somewhat technically inept are using.
http://docs.codehaus.org/display/JETTY/Stress+Testing+Cometd