I am using Mule's datamapper to write data from a database to a csv file. I am using the streaming option on the database, the datamapper and the file ouput. I want to be able to log the amount of records written by the datamapper. Is there a way to get this data? I am running mule server 3.5.2 and have anypoint studio version 5.2.0.
Not out of the box. You can use an outputArgument and increase a counter if you are NOT using streaming.
If you are using streaming then you can pass an input argument of a counter class. And from the script component of Datamapper you can increment the counter and return the counter as part of the payload to get access to it:
<data-mapper:transform config-ref="Pojo_To_JSON_1" doc:name="Pojo To JSON" stream="true">
<data-mapper:input-arguments>
<data-mapper:input-argument key="counter">#[new Counter()]</data-mapper:input-argument>
</data-mapper:input-arguments>
</data-mapper:transform>
Datamapper script:
//MEL
//START -> DO NOT REMOVE
output.__id = input.__id;
//END -> DO NOT REMOVE
output.text = inputArguments.counter.increment();
I know this is an old thread but still below could help -
<byte-array-to-object-transformer doc:name="Byte Array to Object"/>
<set-variable variableName="Orig_Rows" value="#[payload.length]" doc:name="Variable"/>
Related
I am using Community edition Mule 4.4 ( So my understanding is - cannot use Dataweave ( Transform )
Now due to CE cannot use 'Repetable file store stream' ( which is fine ) ( am using repeatable in memory stream )
Now my problem is after I read in the file - how do I parse the contents ?
The data all shows up as
org.mule.runtime.core.internal.streaming.bytes.ManagedCursorStreamProvider
If I could use Dataweave this was simple enough : i.e.
<ee:transform doc:name="Transform Message">
<ee:message>
<ee:set-payload>
<![CDATA[%dw 2.0
output application/json
---
payload map (value,index)->
{
id:value.column_0
}]]>
</ee:set-payload>
</ee:message>
</ee:transform>
But without using Transform component ( since I am using Community Edition Mule runtime 4.4 ) , how do we handle payload which is really a 'Stream' of data ?
Thanks
Please see above code , I need to convert the file content ( which is a stream ) into JSON
edit1:
Thanks to #aled updating with more details : Below is the file read operation where I am trying to read in a tab delimited file. Was not sure what I should set the outputMimeType so have set it as 'application/csv'
<file:read doc:name="Read Products file" config-ref="File_Config" outputMimeType='application/csv; header=false; separator=|' path="/employee_master/employees.unl" outputEncoding="utf-8">
<repeatable-in-memory-stream />
</file:read>
You are thinking about it the wrong way. First, you are looking at the implementation class in Mule, which in Mule 4 you should not be looking at. No file is really a 'stream of data' when you are trying to process it. Is it a binary file? No, clearly if you use map() DataWeave knows how to parse it. So what is the data format of the input file? JSON, CSV, XML, etc? It looks like the correct format is already set so DataWeave is able to parse the input. Really your problem is that you want to transform the data but you can not use a Transform component in the community edition. Also the editing experience will not be good in the UI.
What you can do in community edition is use a set-payload with an expression. Since the expression language is still DataWeave it will work exactly the same. You will lose the graphical UI of Transform.
My Mule 3.9 app (Munit 1.11) consists of a for-each loop in which a http connector is called in each for-each iteration. The payload with which the connector is called varies in each loop. Now I would like to assert the payload in a corresponding Munit test for each for-each iteration. The idea is to use something comparable to the JavaScript Jest function toHaveBeenNthCalledWith so that a assert payload can be defined for each nth connector call. Is there some build in function for this available or is there a workaround for this?
Trying to figure out the proper way to recursively invoke a Mule flow.
We have a flow that builds an array of work to do as it runs, then recursively calls itself using a "Flow Reference" inside a "For Each" block. Problem is, we haven't figured out the correct way to pass parameters to this recursive flow, so we're not getting the results we expect.
We tried passing parameters using flow properties (setInvocationParameter() in Groovy), but it seems that these are shared across multiple instances of the flow.
For an example, we have the ForEach array iterating through an array containing [2. 3. 4], but depending on timing, some of these values are lost (we typically see 2, then 4 twice - skipping 3).
We've tried different Mule processing strategies without any luck. Mule's default queued-asynchronous has the issues described above. Synchronous doesn't seem to work at all (makes sense since our recursive model probably requires two instances to run at minimum).
Here's the relevant part of the configuration XML (the entire flow is quite large). At the end of the flow is this:
<foreach collection="#[sessionVars['actionArray']]"
counterVariableName="actionIndex"
rootMessageVariableName="actionVar" doc:name="For Each">
<scripting:component doc:name="Run Each Action">
<scripting:script engine="Groovy">
<![CDATA[def aa = message.getSessionProperty('actionArray')
def this_item = aa.get(message.getInvocationProperty('actionIndex'))
// Pass the desired action for the recursive call
message.setInvocationProperty('FlowAction', this_item)
log.info "Running $this_item" // <- Shows the correct item
return]]>
</scripting:script>
</scripting:component>
<flow-ref name="DoAction" doc:name="Do Action"/>
</foreach>
At the front of the flow, there's a logger that displays the "FlowAction" flow variable. When we test with my [2, 3, 4] array, this logger statement is driven three times (as expected), but usually with values 2, 4 and 4.
We're getting the same results on Mule 3.7 and an older 3.4 system we have (both are the Community Edition).
Thanks for any suggestions from the Mule mavens out there...
I'm not sure this is 100% correct, but here's what we did...
After spending a lot of time trying to get the "For Each" and "Flow reference" approach to work reliably, we gave up and switched to a different technique. Our alternative was to delete the For Each block and drive the flow recursively from a short Groovy script:
. . .
// Invoke the flow recursively for every item in the array
Flow flow = muleContext.getRegistry().lookupFlowConstruct("flow name")
actions.each // actions is an array of integers built earlier
{ item->
MuleMessage msg = message.createInboundMessage()
DefaultMuleSession sess = new DefaultMuleSession(flow, muleContext)
DefaultMuleEvent event = new DefaultMuleEvent(msg, MessageExchangePattern.ONE_WAY, sess)
// Copy the current inbound properties to the new message
message.getInboundPropertyNames().each
{
event.getMessage().setProperty(it, message.getInboundProperty(it), PropertyScope.INBOUND)
}
// Copy the current session variables to the new message too
message.getSessionPropertyNames().each
{
event.setSessionVariable(it, message.getSessionProperty(it))
}
// Now set the item we want processed as a flow variable
event.setFlowVariable("Action", item.toString())
// Finally, trigger the flow (which runs asynchronously)
flow.process(event).getMessage()
}
This is working properly in our environment now.
I need to capture current flow name into a variable.
I tried with #[flow.name] but no luck in mule 3.8.0
can anybody please assist me?
Based on the answer in this post: How to get caller flow name in private flow in Mule
There is a simplest way to get the flow name and put it into a variable:
<expression-component doc:name="Expression"><![CDATA[flowVars.flowName = flow.name;]]></expression-component>
Alternately, you can directly use expression #[mule:context.serviceName] in a variable :-
<set-variable variableName="myFlowName" value="#[mule:context.serviceName]" doc:name="Variable"/>
<!-- Print the value of variale in logger -->
<logger message="#[flowVars.myFlowName]" level="INFO" doc:name="Logger"/>
This will set your current flow name directly in variable
In mule 3.8.5 using Groovy script component,
flowVars.currentFlowName = eventContext.getFlowConstruct().getName();
I have been using #[flow.name] in 3.7.3 and just tried in 3.8.0 to make sure it had not been removed and it worked fine for me in logger and setting a flowVars value. I suggest posting up at least a snippet of your flow and maybe we can spot the issue you are having.
PS, not sure why flow.name is not in standard forms or really documented by Mule, and as it is not there continues to be some worries they will remove it. I have seen it stated more than just here that it is not accessible in MEL, but #[flow.name] is a MEL expression and does work. To use if for something like I Parse Template in exception strategies, I use sulthony's form, set a flowVars value in an expression and refer to that flowVars in my template.
You can access flow name in logger by using #[flow.name] but its not accessible in MEL. Use flowconstruct for getting flow name. Refer this answer
Hope this helps.
Any idea what class is "objectX"? Please see below code snippet.
<jdbc-ee:mssql-data-source name="sqlServerDataSource" user="yyy"
password="xxx" url="jdbc:sqlserver://xxxyyy.com;databaseName=testDB"
transactionIsolation="UNSPECIFIED" doc:name="MS SQL Data Source"/>
.
.
.
<scripting:component doc:name="Groovy">
<scripting:script engine="Groovy"><![CDATA[
objectX = muleContext.getRegistry().lookupObject("sqlServerDataSource");
return objectX.getUrl();
]]></scripting:script>
The result of this is:
jdbc:sqlserver://xxxyyy.com;databaseName=testDB
After going through the entire Mule API doc, it seems to be Class AbstractDataSourceFactoryBean ==> refer to Mule API. Am I correct?
You have object there, just ask it to print its class, like so:
Something like
objectX.getClass().getName()