I have a flow which reads a CSV files and perfrorm CRUD operation in database ... My flow is somewhat like :-
<flow name="CsvToFile" doc:name="CsvToFile">
<file:inbound-endpoint path="C:\Data" responseTimeout="10000" doc:name="CSV" connector-ref="File">
<file:filename-wildcard-filter pattern="*.csv" caseSensitive="true"/>
</file:inbound-endpoint>
<jdbc-ee:csv-to-maps-transformer delimiter="," mappingFile="src/main/abc.xml" ignoreFirstRecord="true" doc:name="CSVTransformer"/>
<jdbc-ee:outbound-endpoint exchange-pattern="request-response" queryKey="SelectQuery" queryTimeout="-1" connector-ref="jdbcConnectorGlobal" doc:name="Database">
<jdbc-ee:query key="SelectQuery" value="Select * FROM DBDATA where ID=#[map-payload:ID]"/>
</jdbc-ee:outbound-endpoint>
<set-payload value="#[message.payload]" doc:name="Set Payload"/>
</flow>
Now if I use SELECT SQL statement.. I don't see the result and no data is fetched ... but if I use INSERT like
<jdbc-ee:query key="InsertQuery" value="INSERT INTO DBDATA (ID,NAME,AGE) VALUES(#[map-payload:ID],#[map-payload:NAME],#[map-payload:AGE])"/>
or an UPDATE SQL statement, I can find it's working and data is inserted or updated in dataase.
My question is: how can I read the ID value from .CSV file and use in SELECT query to retrieve values from the database?
You have exchange-pattern="one-way", meaning the jdbc outbound call will not return to the main flow. Use exchange-pattern="request-response" instead to get a return value.
Also <set-payload value="#[message.payload]" doc:name="Set Payload"/> does not make any sense. You need a transformer of some sort to make the return value readable. You can add a logger to see the returned payload.
UPDATE:
Return value of csv-to-maps-transformer is ArrayList, not Map, so you can not use map-payload:ID. Try splitting the ArrayList, or use #[payload[0].ID] if you have just a single entry.
Thanks to Anton..
The final solution is #[payload[0].ID] what I need and working for me
Related
I'm struggling to figure out how to solve this problem in mule using the studio and thought that perhaps reaching out to the good users of SO may be of some help.
I have a simple webservice that takes a request from a client. this request will preform an insert into a database table, effectively using this database as a message queue. A separate process periodically polls this table, performs additional processing on the message, and then writes results to an output table. the database insert and subsequent select will be linked by a a correlationId that I can pass along to ensure I get the result for the message that was sent. Unfortunately, the software this will integrate with requires this pattern to work correctly.
Here's the workflow that is needed:
HttpRequest -> insert record into a table -> wait(or poll/retry/etc?) until a record is written to another table by a separate process(with the same correlationId) -> return data from this other table back to the httpRequest
here's a sample flow that is as close as i've been able to get with this. Oddly enough, this flow does actually return a payload, however it seems to always be "1". i can't quite see how to make this flow retry the database query until a row exists and then return the resulting row.
How should i be synchronizing 2 database calls? is this possible within mule perhaps with a different combination of components?
Thanks.
<flow name="mainFlow">
<http:listener config-ref="HTTP_Listener_Configuration" path="hello" doc:name="HTTP"/>
<cxf:jaxws-service doc:name="CXF" configuration-ref="CXF_Configuration" serviceClass="kansas.MuleTestServiceImpl"/>
<db:insert config-ref="Oracle_Configuration" doc:name="Database">
<db:parameterized-query><![CDATA[insert into tblRequest (id, correlationId) values(#[payload], #[message.correlationId])]]></db:parameterized-query>
</db:insert>
<until-successful objectStore-ref="MyObjectStore" maxRetries="5" millisBetweenRetries="2000" doc:name="Until Successful" > <!-- failureExpression="???" -->
<db:select config-ref="Oracle_Configuration" doc:name="Database">
<db:parameterized-query><![CDATA[select correlationId,msgResponse from tblResponse where correlationId = #[message.correlationId]]]></db:parameterized-query>
</db:select>
</until-successful>
<logger level="INFO" doc:name="Logger" message="#[payload]"/> <!-- why is payload always = 1? -->
</flow>
Mule is great tool but it makes your life too easy. Sometime so easy that you forget simple things.
In your case you forgot that payload is one object which is result of last component. Think about flow as rails with just one cart. Whatever you load on last station is delivered to the next one. And then process repeats. What was originally delivered to the station does not matter. Matters what you load.
In your case first database component has original payload from CXF and stores something in the database. It returns result of the INSERT statement which is 1 - one row is inserted. So our payload keeps deliver new cargo - 1.
But you need original payload from CXF. Where it is? It is gone - we have only one flow, one pair of trails, one cart.
What to do in this situation? Keep required information not in the cart but somewhere else. For example in flow variables. Store original payload in some variable and then restore it when it required again. Like this
<flow name="mainFlow">
<http:listener config-ref="HTTP_Listener_Configuration" path="hello" doc:name="HTTP"/>
<cxf:jaxws-service doc:name="CXF" configuration-ref="CXF_Configuration" serviceClass="kansas.MuleTestServiceImpl"/>
<set-variable variableName="storedPaylod" value="#[payload]" doc:name="Store original payload"/>
<db:insert config-ref="Oracle_Configuration" doc:name="Database">
<db:parameterized-query><![CDATA[insert into tblRequest (id, correlationId) values(#[payload], #[message.correlationId])]]></db:parameterized-query>
</db:insert>
<set-payload value="#[flowVars.storedPaylod]" doc:name="Restore Payload"/>
<until-successful objectStore-ref="MyObjectStore" maxRetries="5" millisBetweenRetries="2000" doc:name="Until Successful" > <!-- failureExpression="???" -->
<db:select config-ref="Oracle_Configuration" doc:name="Database">
<db:parameterized-query><![CDATA[select correlationId,msgResponse from tblResponse where correlationId = #[message.correlationId]]]></db:parameterized-query>
</db:select>
</until-successful>
<logger level="INFO" doc:name="Logger" message="#[payload]"/> <!-- why is payload always = 1? -->
</flow>
Good idea will be to check that first database component really returns 1 - record is inserted. Do this, produce alerts on the error, and then restore original payload and continue your flow.
The best solution to avoid killing the actual value of your payload after the database insert is to make use of the Message Enricher processor.
try this code below:
<flow name="mainFlow">
<http:listener config-ref="HTTP_Listener_Configuration" path="hello" doc:name="HTTP"/>
<cxf:jaxws-service configuration-ref="CXF_Configuration" serviceClass="kansas.MuleTestServiceImpl" doc:name="CXF"/>
<enricher source="#[payload]" target="#[flowVars.insertResponse]" doc:name="Message Enricher">
<db:insert config-ref="Oracle_Configuration" doc:name="Database">
<db:parameterized-query><![CDATA[insert into tblRequest (id, correlationId) values(#[payload], #[message.correlationId])]]></db:parameterized-query>
</db:insert>
</enricher>
<flow-ref name="dbSelectSubFlow" doc:name="dbSelectSubFlow"/>
<logger message="#[payload]" level="INFO" doc:name="Logger"/>
</flow>
<sub-flow name="dbSelectSubFlow">
<until-successful objectStore-ref="MyObjectStore" maxRetries="5" millisBetweenRetries="2000" doc:name="Until Successful">
<db:select config-ref="Oracle_Configuration" doc:name="Database">
<db:parameterized-query><![CDATA[select correlationId,msgResponse from tblResponse where correlationId = #[message.correlationId]]]></db:parameterized-query>
</db:select>
</until-successful>
</sub-flow>
I am trying to insert the contents of an ArrayList to a SQL Server database using Mule ESB. The ArrayList looks like
[
{Id=a1o90000001muvWAAQ, Billing_Number__c=1000005, type=Call_Log__c},
{Id=a1o90000001muvXAAQ, Billing_Number__c=1000006, type=Call_Log__c}
]
How should I insert this into the database in an optimized way?
My mule flow is as below.
<sfdc:query config-ref="Salesforce__Basic_authentication" query="#[flowVars.query]"
doc:name="Salesforce"/>
<set-variable variableName="result" value="#[new java.util.ArrayList()]"
doc:name="Variable"/>
<foreach doc:name="For Each">
<set-variable variableName="itrresult" value="#result.add(message.payload)]"
doc:name="Variable"/>
</foreach>
<splitter expression="#[flowVars.result]" doc:name="Splitter"/>
<logger message="#[payload['Id']]" level="INFO" doc:name="Logger"/>
#[payload['Id']] gives me the Id LoggerMessageProcessor: a1o90000001mzgzAAA in log.
I am not sure what i am doing is the right way though. I am just on the beginning of the learning curve in Mule.
You don't need a Splitter or the Foreach scope. The Database connector has a bulk mode that makes the connector accept a collection as payload.
Enable to submit collections of data with one query, [...]. Enabling bulk mode improves the performance of your applications as it reduces the number of individual query executions. Bulk mode requires a parameterized query with at least one parameter.
Enable it with bulkMode="true" in an insert, update or delete operation.
EDIT: Use this right after your Salesforce Query component:
<sfdc:query config-ref="" query="#[flowVars.query]" doc:name="Salesforce"/>
<db:insert config-ref="" bulkMode="true" doc:name="Database">
<db:parameterized-query>
<![CDATA[INSERT INTO TABLE (ID, BILLING_NUMBER__C, TYPE)
VALUES (#[payload.Id], #[payload.Billing_Number__c], #[payload.type]);]]>
</db:parameterized-query>
</db:insert>
The payload returned by the Database Insert component is an array with the number of rows changed for each query executed. In your case, an array of one's with the size of how many items the Salesforce Query component returned, like [1, 1, 1, ...].
If you still need to log, do this:
<sfdc:query config-ref="" query="#[flowVars.query]" doc:name="Salesforce"/>
<foreach doc:name="For Each">
<logger message="#[payload.Id]" level="INFO" doc:name="Logger"/>
</foreach>
<db:insert config-ref="" bulkMode="true" doc:name="Database">
<db:parameterized-query><!-- the query --></db:parameterized-query>
</db:insert>
Or this:
<sfdc:query config-ref="" query="#[flowVars.query]" doc:name="Salesforce"/>
<foreach doc:name="For Each">
<logger message="#[payload.Id]" level="INFO" doc:name="Logger"/>
<db:insert config-ref="" doc:name="Database"><!-- bulkMode disabled -->
<db:parameterized-query><!-- the query --></db:parameterized-query>
</db:insert>
</foreach>
Use splitter to split your array and insert individually:
<splitter expression="#[payload]" />
<db:insert config-ref="Config" doc:name="Database">
<db:parameterized-query><![CDATA[insert into xx(Id, Billing_Number__c, type) values('#[payload['Id']]','#[payload['Billing_Number__c']]','#[payload['type']]');]]></db:parameterized-query>
</db:insert>
You can iterate over the ArrayList using a for, inserting values one by one using db:dynamic-query.
Mule support the bulk mode to insert a collection of data. I prefer the db:in-param Element with labels within the SQL statement. Configuration has been tested with Mule 3.8.5.
<db:insert config-ref="databaseConfiguration" bulkMode="true" doc:name="Database">
<db:parameterized-query>
<![CDATA[INSERT INTO T_ORDER (ORDER_NO,CUSTOMER_NO) VALUES (:orderNo, :customerNo)]]>
</db:parameterized-query>
<db:in-param name="orderNo" value="#[payload.orderNo]" />
<db:in-param name="customerNo" value="#[payload.customerNo]" />
</db:insert>
The bulk mode offers a much better performance. Make sure your DBMS is configured correctly for this use case.
Collection with huge entries can be partioned, for example to sub collection of 100 entries, before calling the db:insert component.
<!-- Sample: Collection with 1.000 entries -->
<!-- will be chunked into sub collections with 100 entries -->
<foreach batchSize="100" />
I have a SQL UPDATE statement, to update a particular record to the Database within a mule flow. I am having problems with including MEL expression within the statement and the SQL is not getting executed in my flow.
I am trying the next command:
UPDATE CUSTOMERS SET STATUS=#[flowVars['TxnStatus']] WHERE REF_NUM=#[flowVars['ReferenceNumber']]
where TxnStatus and ReferenceNumber are flow variables in my mule Flow. It looks seemingly simple, but the record is not updated.
When I try a similar SELECT statement with MEL expressions it does retrieve the value for me. Please let me know your thoughts.
Thanks,
Added config file:
<?xml version="1.0" encoding="UTF-8"?>
<jdbc-ee:mssql-data-source name="MuleDB_DataSource" user="${MuleDB_User}" password="${MuleDB_Password}" url="${MuleDB_URL}" transactionIsolation="UNSPECIFIED" doc:name="MS SQL Data Source"/>
<jdbc-ee:connector name="Database" dataSource-ref="MuleDB_DataSource" validateConnections="true" queryTimeout="-1" pollingFrequency="10000" doc:name="Database" transactionPerMessage="false">
</jdbc-ee:connector>
<flow name="ExceptionFlowFlow1" doc:name="ExceptionFlowFlow1">
<vm:inbound-endpoint exchange-pattern="one-way" path="Exception" doc:name="VM"/>
<set-variable variableName="ExceptionPayload" value="#[payload]" doc:name="ExceptionPayload"/>
<set-variable variableName="TxnStatus" value="Failure" doc:name="Variable"/>
<logger message="#[variable:TxnStatus]" level="INFO" category="Status" doc:name="Logger"/>
<jdbc-ee:outbound-endpoint exchange-pattern="one-way" queryKey="Update_Status" queryTimeout="-1" connector-ref="Database" doc:name="Update_Status">
<jdbc-ee:query key="Update_Status" value="UPDATE CUSTOMERS SET STATUS=#[flowVars['TxnStatus'] WHERE PAYMENT_REFERENCE_NUMBER=#[flowVars['TxnReference']"/>
</jdbc-ee:outbound-endpoint>
<logger message="#[payload]" level="INFO" doc:name="Logger"/>
<catch-exception-strategy doc:name="Catch Exception Strategy">
<logger message="#[exception]" level="INFO" category="ExceptionFlow failed with the exception --" doc:name="Logger"/>
</catch-exception-strategy>
</flow>
</mule>
I can see difference between value you used in query key and command given by you above
value="UPDATE CUSTOMERS SET STATUS=#[flowVars['TxnStatus'] WHERE PAYMENT_REFERENCE_NUMBER=#[flowVars['TxnReference']"/>
Both are missing end ] in query value. Correct it and try again.
I've had similar issues when trying to use a flow variable as an SQL input value. Try replacing #[flowVars['varName']] with #[variable:varName] like so:
UPDATE CUSTOMERS SET STATUS=#[variable:TxnStatus] WHERE REF_NUM=#[variable:ReferenceNumber]
This has worked for me. Hope that helps!
UPDATE audit_detail
SET
status = '3'
WHERE
request_id = #[flowVars['ses_request_id']
AND
message_id = #[flowVars['ses_message_id']
I have a flow something like this
A 'Database inbound endpoint' which polls(for every 5 mins) to mySQL Database-Server and get result-set by a select-query (automatically this becomes the current payload i.e #[message.payload])
'For each' component and a 'Logger' component in it using a expression as #[message.payload]
Now flow has one more 'Database-out-bound-endpoint' component which executes another select-query and obtains result-set.
'For each' component with a 'Logger' component in it using a expression as #[message.payload]
Note: in the loggers result-set of first DB is printing. I mean second logger is also showing result-set of first query itself.Because the result-set is storing as payload
so, my questions are
what is the MEL to read the result-set of second database-query in the above scenario.
is there any another way to read result-set in the flow
Here is the configuration XML
<jdbc-ee:connector name="oracle_database" dataSource-ref="Oracle_Data_Source" validateConnections="true" queryTimeout="-1" pollingFrequency="0" doc:name="Database"/>
<flow name="testFileSaveFlow3" doc:name="testFileSaveFlow3">
<poll frequency="1000" doc:name="Poll">
<jdbc-ee:outbound-endpoint exchange-pattern="one-way" queryKey="selectTable1" queryTimeout="-1" connector-ref="oracle_database" doc:name="get data from table 1">
<jdbc-ee:query key="selectTable1" value="SELECT * FROM TABLE1"/>
</jdbc-ee:outbound-endpoint>
</poll>
<foreach doc:name="For Each">
<logger message="#[message.payload]" level="INFO" doc:name="prints result-set of table1"/>
</foreach>
<jdbc-ee:outbound-endpoint exchange-pattern="one-way" queryKey="selectTable2" queryTimeout="-1" connector-ref="oracle_database" doc:name="get data from table 2">
<jdbc-ee:query key="selectTable2" value="SELECT * FROM TABLE2"/>
</jdbc-ee:outbound-endpoint>
<foreach doc:name="For Each">
<logger message="#[message.payload]" level="INFO" doc:name="prints result-set of table2"/>
</foreach>
</flow>
thanks in advance.
This is not the issue with the MEL. It is the issue with your flow logic.
The second result set is not available in the message.
The JDBC Outbound Endpoint is one-way. So Mule flow will not wait for the reply (result set) from the second JDBC (outbound ) in the middle of the flow. So the second time also it is printing the first result set.
Type 1:
Try making your JBDC outbound request-response instead of one-way.
Type 2:
Try Mule Enricher to call the JDBC outbound to call the DB and store the result set into a varaible and try looping the varaible.
Hope this helps.
I have requirement for to fetch data from Database and expose the content as Restful service, I am unable to find any useful documents.
Can any one please share the documents or links.
Take a look at http://www.mulesoft.org/extensions/rest-module
You can combine it with JDBC endpoints to fetch data from the DB.
A simple example will be :-
<flow name="testFlow">
<http:listener config-ref="HTTP_InboundRequest" path="/test" doc:name="HTTP"/>
<set-variable doc:name="Variable" value="23" variableName="eventId"/>
<db:select config-ref="Oracle_Configuration" doc:name="Database">
<db:parameterized-query><![CDATA[select ID, NAME where EVENT_ID=#[flowVars['eventId']]]]></db:parameterized-query>
</db:select>
<set-payload value="{"status":"Success"}" doc:name="Set Payload"/>
</flow>
Where you can modify the code as per your requirement and display the Database values as JSON response.
So, if you deploy a service similar to this , it will be exposed as a Rest api