I am using Specflow and firing the nunit-console.exe in TeamCity to run tests as follows:
"C:\Program Files (x86)\NUnit 2.6.4\bin\nunit-console.exe" /labels /include:regression out=TestResultRegression.txt /xml=TestResultRegression.xml /framework=net-4.0 .\MyTests.dll
How can I access the NUnit include tag (/include:regression) so that I can call certain methods or properties for test setup (ex. If include = regression, then run this certain pull these certain test case ids from the app.config file where the key is "regression")
There is no way in NUnit for you to know what runner is running you or how it is doing it. This separation of concerns is by design. You could, of course, access the command line that ran the tests and examine it, but I think that again forces the tests to know too much about their environment.
Best solution is to organize tests hierarchically so that all tests requiring a certain setup are in a namespace or fixture where that type of setup is performed.
Related
I am currently working on a SAM deployment project that includes the use of python for the Lambda. I created tests using pytest and runs great on my terminal. But its somehow hard to read on a terminal. Somehow I would like to have a testing like Visual Studio 2019's Test features, where its clean and neat, easy to review.
Using VS CODE (as I am working on python files), I installed the Test Explorer UI and support for python tests. As soon as I open it, it loads a ton of tests including the tests of the 3rd party libraries that I have on my deployment, and it clutters my test explorer. I do not want any of these tests anyway, but I do not know how to exclude them.
I also would want to only include specified test files manually (if that is possible). I do not have use for tons of tests auto-detected by the test explorer.
I know it's a late reply, but still, there is a solution. Since you're using pytest, I will give details for that test framework.
Python Test Explorer is aware of pytest arguments and most of the pytest arguments can be used to modify test discovery and execution in the same way as if pytest is used from the command line. So, for example, if you want to exclude some folder, you can use --ignore=relative/path/to/some/folder argument. See pytest documentation on --ignore option.
It works pretty much the same if you want only to include some tests or folders. There is no special option for that, just list files and folders you want to include, for example, relative/path/to/some/folder or relative/path/to/some/test_file.py. See pytest documentation on selecting tests.
Now, you have to tell Python Test Explorer what tests you want to include/exclude. This can be done with python.testing.pytestArgs option in settings.json. For example,
"python.testing.pytestArgs": ["--ignore=relative/path/to/some/folder"]
or
"python.testing.pytestArgs": [
"relative/path/to/some/folder",
"relative/path/to/some/test_file.py"
]
Full settings.json for the last example:
{
"python.pythonPath": "python",
"python.testing.pytestEnabled": true,
"python.testing.pytestArgs": [
"relative/path/to/some/folder",
"relative/path/to/some/test_file.py"
]
}
Note: These settings also can be set in pytest.ini or other pytest configuration file. In that case, there is no need to modify settings.json.
I am trying to simplify the way we run cucumbers. We have a runner for each folder and we always change the runner in a run configuration that has some VM options set.
I can right click on a feature, select "Run feature :" but it doesn't run successfully without the parameters from the run configuration (one of them runs the tests with an in memory database for example)
Is there anyway I can add a default configuration so I can run them directly?
Run -> Edit Configurations
In the left-hand panel, there is a category called defaults. You can set the defaults to cucumber there. However, note that you might have to delete your previously existing configurations to get this to work with files you've already tried. They usually appear in a faded gray.
In addition, you could also add a runner class, which can run (a subset of) your tests from a testing framework. You can provide #CucumberOptions in this runner.
Webstorm has great test running support, which I can use to start my test suite by telling it "run the file testStart.js". I can then define testStart.js to do the setup for my test environment (eg. creating a Sinon sandbox) and to bring in all the tests themselves.
That works great, when I run the whole suite. But Webstorm has a feature that let's you re-run just a single failing test, and when I try to use that feature I run in to a problem: my test setup code doesn't get run because the individual test file doesn't invoke the setup code.
So, I'm looking for a solution. The only options I see so far are:
instead of having a separate testStart.js file I could move the setup code in to a testSetup.js file and make every test require it. DOWNSIDE: I have to remember to import the setup file in every single test file (vs. never having to import it in my current scheme)
use Mocha's --require option to run a testSetup.js. DOWNSIDE: Code require-ed in this way doesn't have access to the Mocha code, so I'm not sure how I can call beforeEach/afterEach
use some other Mocha or Webstorm option that I don't know about to run the test setup code. DOWNSIDE: Not sure if such an option even exists
If anyone else has run in to this problem I'd love to hear if any of the above solutions can be made to work (or if there's another solution I hadn't considered).
I wound up just importing testSetup.js in to every test file. It was a pain and violated the DRY principle, but it worked.
If anyone else has a better solution though I'll happily accept it.
I'd like to write one suite of SpecFlow tests that test my web application (using Selenium) in various environments.
so I have a test written like this
Given that I am on the login page
which in turn leads to a step definition that boils down to
driver.Navigate().GoToUrl("http://www.myapp.com/login.aspx");
However, I want my test to be able to run against "http://localhost" or `"http://test.myapp.com" as well, without having to recompile. The best idea I've come up with is to place these sorts of settings in the App.config file, but that has its problems as well.
Does anyone have suggestions on how best to achieve this? Basically I want to pass in environment settings for my tests at runtime.
You can do this by changing the config file through the build process using transforms and there are tools that will let you run the transform outwith the build process (so you don't have to manually change it and you avoid a build) using the command line. This has been talked about already on SO:
Web.Config transforms outside of Microsoft MSBuild?
For example using PowerShell.
I would still question whether you might be better and starting a local instance of the service that you wish to test, rather than connecting to something which is out of the tests explicit control. You could instead use a method similar to self hosting a web api or host a wcf service to do this for you. This way you can inject mocks, modify and reset the database, or perform any other action you want.
If that still isn't what you need, an alternative to config files would be to setup environment variables that can be read at run time, see How to pass Command line argument to specflow test scenario
I have an automated build script which involves unit testing of some GWT Modules in production mode. It appears that when these tests are run, they recompile the GWT module.
However, earlier in the build script, I have already compiled the modules. This is an obvious waste of effort. Does anybody know of any way to test a GWTTestCase to run in production mode, on modules that were already compiled.
I don't mind losing stacktraces or any information, because the build server only informs developers of which tests fails, and expects them to debug in their own environment.
This will be helpful for you
The main class in the test infrastructure is JUnitShell. To control aspects of how your tests execute, you must pass arguments to this class. Arguments cannot be passed directly through the command-line because normal command-line arguments go directly to the JUnit runner. Instead, define the system property gwt.args to pass arguments to JUnitShell.
For example, to run tests in production mode (that is, run the tests afer they have been compiled into JavaScript), declare -Dgwt.args="-prod" as a JVM argument when invoking JUnit. To get a full list of supported options, declare -Dgwt.args="-help" (instead of running the test, help is printed to the console).
Running your test in Production Mode
When using the webAppCreator tool, you get the ability to launch your tests in either development mode or production mode. Make sure you test in both modes - although rare, there are some differences between Java and JavaScript that could cause your code to produce different results when deployed.
If you instead decide to run the JUnit TestRunner from command line, you must add some additional arguments to get your unit tests running in production mode. By default, tests run in development mode are run as normal Java bytecode in a JVM. To override this default behavior, you need to pass arguments to JUnitShell
-Dgwt.args="-prod"
Well, I found a solution, but it is not elagant. I modified JUnitShell.maybeCompileForWebMode method, you should add VM argument -Dcompile=false to prevent compilation while running unit tests . You can get modified version of JUnitShell from here .