I try to setup an integration/API test suite with Karate and consider to use Karate Netty for mocking required services. For the test setup the system under test A (a Spring Boot app) is started up completely. The Karate tests are then executed by a Maven test run against this instance.
The service A depends on multiple other services these needs to be mocked away for the tests. To do so my idea was to configure a running Karate Netty standalone instance as HTTP proxy (done by JVM args of the service A).
Now my idea was to create one test feature file: xyz-test.feature
And the required mocks for this file are defined in an associated mock feature file: xyz-mock.feature
(The test scenarios are rather complex and the responses of the external services could vary)
This means for a full test run I need to load up a couple of mock feature files. So:
What is the matching strategy for multiple mock feature files? Which scenario wins, so to say.
Is there any way to ensure, that the right mock file is used for the associated test file?
(Clearly I can reconfigure the running standalone instance and advice it to use xyz-mock.feature next.
But this would stop me from using parallel execution for my API tests, right?)
I already thought about reusing the Correlation-Id which I can send in for each test and then match against this in the mock file (it is also sent to all called services). But:
Is there a way to define a global matcher per mock file?
It sounds like you need only one mock file. You could boot 2 on different ports if you wanted, but there is no way to "merge" them into one port - if that is what you were looking for.
In my experience, you will be able to have a single mock take care of all your edge cases. This is because Karate's approach is un-conventional: you pretty much write a stateful server. But by keeping variables in memory and some clever JSON-path, you can simulate CRUD with very few lines of code: https://github.com/intuit/karate/tree/master/karate-netty#background
You can use only one at a time, by design
Given the above limitation, here's an interesting idea: add something like an extra pathMatches('/__test/reset') scenario that cleans-up your state and sets the Background variables to things like * def cats = []. Now in each feature, just call the special "reset" URL at the start. The good thing is Karate is thread-safe. Another idea as you said is you can maintain two or three different variables and use some logic to "route" based on a header, again very easy IMO. Use a map of maps, e.g:
def data = { cats1: {}, cats2: {}, cats3: {} }
And you can get the header, e.g. if it is mode: cats1
* def mode = karate.get('requestHeaders.mode[0]')
* def cats = data[mode]
not sure if this answers your question, but if the last Scenario has an "empty" description, it is a "catch all" and can in theory delegate to another server (or mock): https://github.com/intuit/karate/tree/develop/karate-netty#proxy-mode
Your question is a little confusing, so you may have to edit and re-word it if I haven't understood.
EDIT: using multiple mock files should be possible in 1.1.0 onwards: https://github.com/intuit/karate/issues/1566
Related
I'm trying to do something that I don't know if it is even remotely possible or not.
I've a Mock server, and I'd like that when it receives a given request, it "starts another test", calling a test feature. I tried some stuff, including the one bellow. But turns out that this Mockserver scenario do not respond.
Scenario: pathMatches('/ideas')
* def xx = call read('SimpleStart.feature')
* def response = $ideas.*
Is there an elegant way to make this work? AN workaround or a suggestion you can give me?
The use case is:
Perform tests, some tests, make some external services invoke the mockserver, and if the mockserver is requested it triggers other tests.
Thanks in advance.
Yeah Karate certainly isn't designed to do that. The pattern should be set up your mocks and tests from a Java "runner" for maximum control and that's what most teams do.
In short, "orchestrate" things from Java code.
That said, see if this gives you some other creative ideas: https://twitter.com/getkarate/status/1417023536082812935
I have a few Nifi process groups which I want to run integration tests on before promoting to production. The issue is that I can't seem to find any documentation on how to do so.
Data Provenance seems like a promising tool to accomplish what I want, however, over the course of the flowfile's lifecycle, data is published to/from kafka or the file system. As a result, the flowfile UUID changes so I cannot query for it using the nifi-api.
Additionally, I know that Nifi offers a TestRunner library to run tests, however, this seems to only be for processors/processor groups generated via code and not the UI.
Does anyone know of a tool, framework, or pattern for integration and unit testing nifi process groups. Ideally this would be a solution where you can programatically compare input/output of the processor/processor group without modifying the existing workflow.
With the introduction of the Apache NiFi Registry, we have seen users promote flows from a development/sandbox environment to a test/QE environment where there are existing "test harness" flows surrounding the "flow under test" so that they can send repeatable and deterministic (or an anonymized sample of real production data) through the flow and compare the results to an expected value.
As you point out, there is a TestRunner class and a whole testing framework provided for unit tests. While it can be difficult to manually translate a UI-constructed flow to the programmatic construction, you could also create something like a translator to accept a flow template or flow.xml.gz file and convert it into something processable by the test framework.
Maybe plumber will help you with flow testing.
We also wanted to test whole NiFi flows, not just single processor, so we created this library and decided to open-source it.
Simple example in Scala:
// read flow previously exported from NiFi
val template = TemplateDeserializer.deserialize(this.getClass.getClassLoader.getResourceAsStream("exported-flow.xml"))
val flow = NifiTemplateFlowFactory(template).create()
// enqueue some data to any processor
flow.enqueueByName("csv row,12,another value,true", "CsvParserProcessor")
// run entire flow once
flow.run(1)
// get the results from any processor
val records = flow.resultsFromProcessorRelation("LastProcessorInFlow","successRelation")
records should have size 1
This library is still under development so improvements and ideas are welcomed! :)
I'm using WebDriverIO and I want to do the following:
Run a single test before any test is run (createNewUsers)
Use specific capabilities (proxy settings) for that first test
Once done, use a default set of capabilities for everything else
So I can't seem to figure this out:
I've tried to add a second set of capabilities and use the exclude argument to ensure it only applies to that specific spec, however, I don't know whether this is actually possible and then how to call that specific test in my before block - so capabilities I use:
exclude: [ './newUserCreationStage/newStageUsers.js' ],
But then in my before block - how do I say run that (if it's possible):
before: function (capabilities, specs) {
expect = require('chai').expect;
RUN THIS './newUserCreationStage/newStageUsers.js'
},
I would say that your setup needs a bit different approach. First take a look at the xUnit Fixture Setup Patterns. This createNewUsers can actually be achieved via SuiteFixture Setup, Prebuilt Fixture, Setup Decorator and Creation Method. This will set the SUT in the desired state and remove the need to
Run a single test before any test is run
Even better - if you have access, you can seed a DB or call APIs to load all users, data and whatever you need before the test run. This is also known as Back Door Manipulation. Let your CI server to take care of all this as a dedicated step.
Since you are using Mocha, you can utilize tags and organise your suites and specs in a better way. This will allow you to switch drivers with capabilities according to your needs (read test needs, ones may need proxy, others not). I would suggest to also consider mocha-tags. Good fit here is the Strategy pattern. It will allow you to have many related subclasses which only differ in behavior.
I'm writing functional test and having difficulty mocking http:request-config with oauth2. It failed at requesting for token. I tried moving the config to a separate file and create a different config in src/test/resources and include only the test config when testing. Now it complains about "name must be unique" - how do I get around this?
Be sure that your getConfigFiles() override does not include the configuration file that contains the original . This means it will need to be in a separate file from the one containing the flow you are testing.
Another method is to use a mock HTTP server such as sham-http.
In order to test Mule application you can use MUnit:
http://developer.mulesoft.com/docs/display/current/MUnit
It will allow you to mock message processors.
Now, config elements are top level elements. Those can not be mock.
I would suggest you take a look to documentation to see if the tool fit your needs.
HTH
I'd like to test the search functionality of 30 websites that are generated by the same CMS under different domains with different Lucene-indexes. For this purpose I'd like to write a single Page Object which I'd like to be fed by the configuration with those 30 different baseUrls.
I'd be running those tests in the same environment so I'm not sure how to approach this issue. Is there anything I've been missing so far? Looking forward to a push in the right direction and thanks in advance.
You can always override the base url in your test using using browser.config.baseUrl = 'http://example.com'. It will be reverted to the value configured in GebConfig.groovy for the next test.
The question is how many tests do you wish to run this way? If it's just one test then you can get away by using Spock's support for parametrised tests with where: block and this approach. If it's more than one test then you will probably be looking at custom test runners or running your tests multiple times using your build system with different geb environment settings.