I have a few Nifi process groups which I want to run integration tests on before promoting to production. The issue is that I can't seem to find any documentation on how to do so.
Data Provenance seems like a promising tool to accomplish what I want, however, over the course of the flowfile's lifecycle, data is published to/from kafka or the file system. As a result, the flowfile UUID changes so I cannot query for it using the nifi-api.
Additionally, I know that Nifi offers a TestRunner library to run tests, however, this seems to only be for processors/processor groups generated via code and not the UI.
Does anyone know of a tool, framework, or pattern for integration and unit testing nifi process groups. Ideally this would be a solution where you can programatically compare input/output of the processor/processor group without modifying the existing workflow.
With the introduction of the Apache NiFi Registry, we have seen users promote flows from a development/sandbox environment to a test/QE environment where there are existing "test harness" flows surrounding the "flow under test" so that they can send repeatable and deterministic (or an anonymized sample of real production data) through the flow and compare the results to an expected value.
As you point out, there is a TestRunner class and a whole testing framework provided for unit tests. While it can be difficult to manually translate a UI-constructed flow to the programmatic construction, you could also create something like a translator to accept a flow template or flow.xml.gz file and convert it into something processable by the test framework.
Maybe plumber will help you with flow testing.
We also wanted to test whole NiFi flows, not just single processor, so we created this library and decided to open-source it.
Simple example in Scala:
// read flow previously exported from NiFi
val template = TemplateDeserializer.deserialize(this.getClass.getClassLoader.getResourceAsStream("exported-flow.xml"))
val flow = NifiTemplateFlowFactory(template).create()
// enqueue some data to any processor
flow.enqueueByName("csv row,12,another value,true", "CsvParserProcessor")
// run entire flow once
flow.run(1)
// get the results from any processor
val records = flow.resultsFromProcessorRelation("LastProcessorInFlow","successRelation")
records should have size 1
This library is still under development so improvements and ideas are welcomed! :)
Related
I try to setup an integration/API test suite with Karate and consider to use Karate Netty for mocking required services. For the test setup the system under test A (a Spring Boot app) is started up completely. The Karate tests are then executed by a Maven test run against this instance.
The service A depends on multiple other services these needs to be mocked away for the tests. To do so my idea was to configure a running Karate Netty standalone instance as HTTP proxy (done by JVM args of the service A).
Now my idea was to create one test feature file: xyz-test.feature
And the required mocks for this file are defined in an associated mock feature file: xyz-mock.feature
(The test scenarios are rather complex and the responses of the external services could vary)
This means for a full test run I need to load up a couple of mock feature files. So:
What is the matching strategy for multiple mock feature files? Which scenario wins, so to say.
Is there any way to ensure, that the right mock file is used for the associated test file?
(Clearly I can reconfigure the running standalone instance and advice it to use xyz-mock.feature next.
But this would stop me from using parallel execution for my API tests, right?)
I already thought about reusing the Correlation-Id which I can send in for each test and then match against this in the mock file (it is also sent to all called services). But:
Is there a way to define a global matcher per mock file?
It sounds like you need only one mock file. You could boot 2 on different ports if you wanted, but there is no way to "merge" them into one port - if that is what you were looking for.
In my experience, you will be able to have a single mock take care of all your edge cases. This is because Karate's approach is un-conventional: you pretty much write a stateful server. But by keeping variables in memory and some clever JSON-path, you can simulate CRUD with very few lines of code: https://github.com/intuit/karate/tree/master/karate-netty#background
You can use only one at a time, by design
Given the above limitation, here's an interesting idea: add something like an extra pathMatches('/__test/reset') scenario that cleans-up your state and sets the Background variables to things like * def cats = []. Now in each feature, just call the special "reset" URL at the start. The good thing is Karate is thread-safe. Another idea as you said is you can maintain two or three different variables and use some logic to "route" based on a header, again very easy IMO. Use a map of maps, e.g:
def data = { cats1: {}, cats2: {}, cats3: {} }
And you can get the header, e.g. if it is mode: cats1
* def mode = karate.get('requestHeaders.mode[0]')
* def cats = data[mode]
not sure if this answers your question, but if the last Scenario has an "empty" description, it is a "catch all" and can in theory delegate to another server (or mock): https://github.com/intuit/karate/tree/develop/karate-netty#proxy-mode
Your question is a little confusing, so you may have to edit and re-word it if I haven't understood.
EDIT: using multiple mock files should be possible in 1.1.0 onwards: https://github.com/intuit/karate/issues/1566
Can you have automated regression/integration tests for Azure Logic Apps?
And if you can, how? ... especially in the context of CI/CD builds and deployments
... and if you can't, why not!!
There isn't any out-of-the-box tooling yet to provide automated testing of Azure Logic Apps. We have a few customers who have followed one of the following patterns. There is also this article that goes into detail on how to create a Logic App deployment template:
After deployment (using a release management tool like Visual Studio Release Management), a series of unit tests are run (writtin in something like C#) to test the Logic App.
Since a logic app could have any kind of trigger (on queue item, on HTTP request), the code usually performs the action and asserts the result.
A logic app in the resource group that can run a series of basic tests in a workflow. This one requires a bit more chewing on, but idea being you have a workflow that makes use of connectors or "calling nested apps" to perform basic validation tests (ensure connections are active, etc.)
It's something we have had discussions on from time-to-time, but would love to know if you have any thoughts on what types of tooling/configuration you'd want to configure for an app (remember that some apps "trigger" on something like a message in a queue or a file in FTP).
I would like to share one of the approach for LogicApp testing that my team has followed.
First level of validation is the ARM template deployment status (ProvisioningState) which should not have any errors.
After that we have developed test automation using the logic app sdk which does the following
Get auth token.
Execute a specific logic app trigger with a synthetic transaction.
Waits till the execution is completed.
Gets logic app & its action status (succeed, failed or skipped), validates it as per the expected scenario.
Gets the outputs from each action execution, validates them against an expected scenario.
Repeat above steps for all the various cases that logic app might go through.
Hook this all-in CI/CD :)
Deployed an LA, ran a synthetic transaction & validated the results.
Hope this helps.
I want to write some code using the SAP .Net Connector 3 to receive and send data to a SAP System using RFC and iDoc.
How can I setup a simple SAP Test System with RFC to test my code.
Is there a way to mock the SAP System or do I have to install a SAP System?
If so is there any simple tutorial on how to setup an SAP System with a simple "Hello World" RFC?
I was originally going to post a comment. But it was too long.
This isnt a solution, its a warning.
I think you have placed too much emphasis on unit test for this type of solution. Mock the rest of the code all you like. But mocking an interface that may/will behave differently is false confidence.
By all means abstract the infrastructure layer and push dummy data into int to test the rest of the app. But dont plan on mocking the interface in any way that is relevant to stability.
How do you plan to mock:
The sign on process
single sign on, SNC...
gateway connection
connection specific settings
authorizations
load balancing
connection pooling
timeout
Test it against the DEV system, then test again in QA system
and get ready for unexpected issues in PROD.
You can write code to generate TABLE/STRUCTURE content. So you easily mock what that you expect to receive or send to SAP system. Write a dummy that returns that data and mock the call. Dont bother with mock infrastructure. That achieves nothing.
I have a flow with two VM endpoints both configured with the exchange pattern of request/response. I want to evaluate the message at the end of the flow when it reaches the seecond VM endpoint, before the next flow takes off with the message. I thought I might be able to do this with an interceptor inserted before the VM endpoint. Is this possible from within a Mule FunctionalTestCase? Is it possible to programatically add an interceptor to a flow at all..?
Personally, I think that the flows should not really be altered during the testings. In that case you would have another (although just slightly different) version running when you deploy it to a server.
Instead, I would argue that you divide your flows into testable parts and put the endpoint addresses into separate configuration. That way you can test each vm-based flow separated from each other and verify the behaviour using mock flows or similar.
vm://in-flow1 -> process -> vm://mock
vm://mock -> verify payload -> vm://in-flow2
In the "real" configuration, you change "mock" to something pointing to the second vm flow.
You can also elaborate on mocking the first or second VM flows entirely from each other to create distinct unit tests.
However, if you really want to go down the "modify code for testing purposes" rabbit hole, you can likely use some aspect oriented black magic to achieve that.
Look at this blogpost how it's done in mule.
You could try with Munit, and run an spy around the flow (it should work). So you can run assertions after the flow execution
https://github.com/mulesoft/munit
I have Jenkins project that perform some sort of sanity check on couple of independent documents. Check result is written in JUnit XML format.
When one document test fails, entire build fails. Jenkins can be simply configured to send email to commiter in this situation. But I want to notify commiters only when new test failed or any failed test was fixed with the commit. They are not interested in failed tests for documents they have not edited. Email should contain only information of changes in tests, not full test report. Is it possible to send this kind of notification with any currently available Jenkins plugins? What could be the simplest way to achieve this?
I had the same question today. I wanted to configure Jenkins sending notifications only when new tests fail.
What I did was to install email-ext plugin.
You can find there a special trigger that is called Regression (An email will be sent any time there is a regression. A build is considered to regress whenever it hasmore failures than the previous build.)
Regarding fixed tests, there is Improvement trigger (An email will be sent any time there is an improvement. A build is considered to have improved wheneverit has fewer failures than the previous build.)
I guess that this is what you are looking for.
Hope it helps
There's the email-ext plugin. I don't think it does exactly what you want (e.g. sending only emails to committers who have changed a file that is responsible for a failure). You might be able to work around that/extend the plugin though.
Also have a look at the new Emailer, which talks about new email functionality in core hudson that is based on aforementioned plugin.