I am currently developing a DevOps process whose goal is to trigger the execution of a promotion pipeline using the Rest Talend API in the case of promoting tasks from one environment to another.
In the API documentation https://api.eu.cloud.talend.com/tmc/swagger/swagger-ui.html#/promotions32585832executions there is no information on the field ** artifactType ** concerning the tasks .
While researching, I found that for promoting artifacts the keyword is ACTION. However, I unfortunately have no answer for the keyword used to promote the tasks. I have tried TASK, JOB, STEP and ACTION but it shows me an error in the json.
If anyone knows what keyword to use to promote tasks in API Rest, that would help me a lot!
Merci :)
The right answer for promoting tasks using REST API is "FLOW".
Related
I have a few Nifi process groups which I want to run integration tests on before promoting to production. The issue is that I can't seem to find any documentation on how to do so.
Data Provenance seems like a promising tool to accomplish what I want, however, over the course of the flowfile's lifecycle, data is published to/from kafka or the file system. As a result, the flowfile UUID changes so I cannot query for it using the nifi-api.
Additionally, I know that Nifi offers a TestRunner library to run tests, however, this seems to only be for processors/processor groups generated via code and not the UI.
Does anyone know of a tool, framework, or pattern for integration and unit testing nifi process groups. Ideally this would be a solution where you can programatically compare input/output of the processor/processor group without modifying the existing workflow.
With the introduction of the Apache NiFi Registry, we have seen users promote flows from a development/sandbox environment to a test/QE environment where there are existing "test harness" flows surrounding the "flow under test" so that they can send repeatable and deterministic (or an anonymized sample of real production data) through the flow and compare the results to an expected value.
As you point out, there is a TestRunner class and a whole testing framework provided for unit tests. While it can be difficult to manually translate a UI-constructed flow to the programmatic construction, you could also create something like a translator to accept a flow template or flow.xml.gz file and convert it into something processable by the test framework.
Maybe plumber will help you with flow testing.
We also wanted to test whole NiFi flows, not just single processor, so we created this library and decided to open-source it.
Simple example in Scala:
// read flow previously exported from NiFi
val template = TemplateDeserializer.deserialize(this.getClass.getClassLoader.getResourceAsStream("exported-flow.xml"))
val flow = NifiTemplateFlowFactory(template).create()
// enqueue some data to any processor
flow.enqueueByName("csv row,12,another value,true", "CsvParserProcessor")
// run entire flow once
flow.run(1)
// get the results from any processor
val records = flow.resultsFromProcessorRelation("LastProcessorInFlow","successRelation")
records should have size 1
This library is still under development so improvements and ideas are welcomed! :)
Can you have automated regression/integration tests for Azure Logic Apps?
And if you can, how? ... especially in the context of CI/CD builds and deployments
... and if you can't, why not!!
There isn't any out-of-the-box tooling yet to provide automated testing of Azure Logic Apps. We have a few customers who have followed one of the following patterns. There is also this article that goes into detail on how to create a Logic App deployment template:
After deployment (using a release management tool like Visual Studio Release Management), a series of unit tests are run (writtin in something like C#) to test the Logic App.
Since a logic app could have any kind of trigger (on queue item, on HTTP request), the code usually performs the action and asserts the result.
A logic app in the resource group that can run a series of basic tests in a workflow. This one requires a bit more chewing on, but idea being you have a workflow that makes use of connectors or "calling nested apps" to perform basic validation tests (ensure connections are active, etc.)
It's something we have had discussions on from time-to-time, but would love to know if you have any thoughts on what types of tooling/configuration you'd want to configure for an app (remember that some apps "trigger" on something like a message in a queue or a file in FTP).
I would like to share one of the approach for LogicApp testing that my team has followed.
First level of validation is the ARM template deployment status (ProvisioningState) which should not have any errors.
After that we have developed test automation using the logic app sdk which does the following
Get auth token.
Execute a specific logic app trigger with a synthetic transaction.
Waits till the execution is completed.
Gets logic app & its action status (succeed, failed or skipped), validates it as per the expected scenario.
Gets the outputs from each action execution, validates them against an expected scenario.
Repeat above steps for all the various cases that logic app might go through.
Hook this all-in CI/CD :)
Deployed an LA, ran a synthetic transaction & validated the results.
Hope this helps.
I have a mule flow and I want to automate the execution of the application without http listener
I want the mule application execute without enter "localhost:8081/app"
is it a way to do this?
Screenshots of the flow
As I understood from your question, I can suggest the below steps
1) Add Composite source at the start of your flow.
2) Place the existing HTTP inbound endpoint into Composite source scope.
3) As an addition, add the quartz inbound endpoint into composite source scope and configure it at what time you want to run using cron expression.
This approach enables you option to trigger the flow using either HTTP URL or automated execution through quartz component using cron expression.
Please comment on this answer if you feel my understanding is wrong.
Do you simply want the app to run at scheduled intervals? If so, I think the Quartz connector would be you best choice.
Is this the scenario you are after?
Hi I am working with Mule Web Service Consumer and i was trying to call operation with Multiple Parameters it is warning me that
Warning : Operation Messages With More then 1 Part Are Not Supported
I just want to pass multiple parameters to access my SOAP method to achieve the task.
Is this the problem with Web Service Consumer or is their any way to deal with this.
I'm afraid this is a known limitation of the web services consumer. However you can accomplish this with the cxf component.
I having the same issue and found some information around it ...
There is a improvement logged in JIRA, may help if you vote for it :)
This link suggests that you can still use WSConsumer but need to do some hand crafting of the request XML ... I could not understand what that exactly it meant so if anyone has an example on it would be great
PS: The problem I had with using CXF component is that it does not play well with the new Dataweave transformer as the Dataweave needs to be placed within the response block and from there it cannot datasense the response coming out from the CXF component
The Solution here is very simple. You just have to comment other messages and then load metadata for non-commented message (for one which you're trying to load metadata). Repeat this procedure for all the other messages and you're good to go.
Hope this helps !
I need a good example of using the LookbackAPI to get the data for a burn up chart. I see some limited questions and responses on the API but no examples on how I would use it to do so. I need to get the current scope on story points and story points completed.
Sorry for the scarcity of available examples. More and better examples will be coming as the LBAPI beta matures. I'd definitely recommend that you become familiar with the Lookback API (LBAPI) Documentation, as there are good examples there for formulating queries.
For a burnup, let's say you want to get the state Snapshots for an Iteration going from 15-Jan-2013 through 30-Jan-2013, and that the Iteration applies to a Project hierarchy that is four-deep. The following LBAPI query would obtain the PlanEstimate, ToDo, and Schedule State for Stories scheduled into that Iteration:
{
find:
{
_TypeHierarchy:"HierarchicalRequirement",
Children:null,
_ValidFrom:{
$gte:"2013-01-15TZ",
$lt:"2013-01-30TZ"
},
Iteration:{
$in:[
12345678910,
12345678911,
12345678912,
12345678913
]
}
},
fields:[
"PlanEstimate",
"ToDo",
"ScheduleState"
]
}
Where:
$in:[
12345678910,
12345678911,
12345678912,
12345678913
]
Are the ObjectID's of the Iteration called "Iteration 1". It's probably easiest to get these Object ID's from a standard WSAPI query on Iterations: (Name = "Iteration 1"). For Iterations copied into a four-deep project hierarchy, we would see the four Iteration OID's similar to the above.
For charting, the toughest part right now is an easy way to deal with the Time-Series data. The most robust way to query and process LBAPI data currently is by working directly against the REST endpoint and processing the returned JSON results in your own code.
With Javascript apps, for processing the data and turning it into a Chart, the preferred toolkit is AppSDK2, specifically the SnapshotStore.
For Javascript apps, the Lumenize javascript library is separate from LBAPI, but was developed by Rally's director of analytics and is bundled in the SDK. You can find some examples of using LBAPI and Lumenize to produce charting as part of some Rally-internal and Rally-customer Hackathon projects here:
https://github.com/RallyHackathon
Please be cautious with these examples for a couple of reasons:
Several aspects of the Lumenize namespace will be changing/renamed for clarity
There's a bug in the current version of Lumenize where its timeSeriesCalculator does not correctly account for stories deleted or reparented.
Hopefully there will be an updated version of AppSDK2 bundled and released soon to consolidate the Lumenize namespace and resolve the bug, so that there's better glue between AppSDK2 and LBAPI for Javascript App development.
Unfortunately, the .NET, Java and Python toolkits have not yet been updated to support the Lookback API. As a result, you'll have to do an HTTP POST to the Lookback API's REST endpoint directly, with a body similar to the one Mark W listed above and Content-Type 'application/json'.
I'd recommend using the Chrome extension 'XHR Poster' to experiment with what you're sending from a browser:
https://chrome.google.com/webstore/detail/xhr-poster/akdbimilobjkfhgamdhneckaifceicen