I have the below flow reference: <flow name="getAccountsFlow">
<logger message="Type:"#[message.inboundProperties.'http.query.params'.type]"" level="INFO" doc:name="Logger"/>
<db:select config-ref="MySQL_Configuration" doc:name="Database">
<db:parameterized-query><![CDATA[select * from flights_customers where accountType = "#[message.inboundProperties.'http.query.params'.type]"]]></db:parameterized-query>
</db:select>
</flow>
The logger prints the variable value: INFO 2019-01-07 16:08:51,270 [[assignment-accounts].accounts-api-httpListenerConfig.worker.03] org.mule.api.processor.LoggerMessageProcessor: Type:"business"
But the query returns 0 rows.
But when i use the variable transformer, then value can be accessed in the query as: <flow name="getAccountsFlow">
<logger message="Type:"#[message.inboundProperties.'http.query.params'.type]"" level="INFO" doc:name="Logger"/>
<set-variable variableName="type" value="#[message.inboundProperties.'http.query.params'.type]" doc:name="Variable"/>
<db:select config-ref="MySQL_Configuration" doc:name="Database">
<db:parameterized-query><![CDATA[select * from flights_customers where accountType = :type]]></db:parameterized-query>
<db:in-param name="type" type="VARCHAR" value="#[flowVars.type]" ></db:in-param>
</db:select>
</flow>
why is the #[message.inboundProperties.'http.query.params'.type] not visible in the query?
It’s because of the double quotes around the expression in the query . Try removing those.
But it is safer to use placeholders because of injection anyway, so I would use the second example anyway in my opinion.
I am trying to enrich the data to create an XML file.
The first query does a Group By to obtain the transaction header.
The second query gets all records (details) that match the header from the same file, to enrich the message.
The problem is that it takes about a second to run the query that enriches the data. I will need to run this process for 184,764 headers. At one second per header this job will take too long. Is there a way to accomplish the same thing without having to query the database for details? Can all the records be loaded first and obtain the details from memory instead? Here's the code:
<db:generic-config name="Generic_Database_Configuration" url="${db.url}"
driverClassName="${driver.class.name}" doc:name="Generic Database
Configuration"/>
<data-mapper:config name="List_Map__To_List_Map_"
transformationGraphPath="list_map__to_list_map_.grf"
doc:name="List_Map__To_List_Map_"/>
<data-mapper:config name="List_Map__To_XML_1"
transformationGraphPath="list_map__to_xml_1.grf"
doc:name="List_Map__To_XML_1"/>
<batch:job name="OrceTransactionImportBatch">
<batch:input>
<db:select config-ref="Generic_Database_Configuration"
doc:name="Database">
<db:parameterized-query><![CDATA[SELECT TRANDATED, STORED, REG#D
AS REG_D, TRAN#D AS TRAN_D, VIP#D AS VIP_D, VIP#D AS VIPNO, SUM(RETAIL*QTY)
AS TOTAL,
CONCAT(SUBSTRING(TRANDATED,1,4),
CONCAT('-',CONCAT(SUBSTRING(TRANDATED,5,2),
CONCAT('-',CONCAT(SUBSTRING(TRANDATED,7,2),'T00:00:00'))))) AS
BusinessDayDate
FROM ORCTEXDTLP
WHERE DGROUPID IN (SELECT HGROUPID FROM ORCTEXHDRP WHERE HPRCFLAG = 'P')
GROUP BY STORED, TRANDATED, REG#D, TRAN#D, VIP#D
FETCH FIRST 60 ROWS ONLY]]></db:parameterized-query>
</db:select>
<logger message="before mapper..." level="INFO" doc:name="before
mapper..."/>
</batch:input>
<batch:process-records>
<batch:step name="Batch_Step">
<data-mapper:transform config-ref="List_Map__To_List_Map_"
doc:name="List<Map> To List<Map>"/>
<logger message="before enricher..." level="INFO"
doc:name="before enricher..."/>
</batch:step>
<batch:step name="Batch_Step1">
<logger message="BEFORE FOR EACH..." level="INFO"
doc:name="Logger"/>
<enricher target="#[variable:LineItem]" doc:name="Message
Enricher">
<db:select config-ref="Generic_Database_Configuration"
doc:name="Database">
<db:parameterized-query><![CDATA[SELECT TRANCODED,
CONCAT(SUBSTRING(TRANDATED,1,4),
CONCAT('-',CONCAT(SUBSTRING(TRANDATED,5,2),
CONCAT('-',CONCAT(SUBSTRING(TRANDATED,7,2),'T00:00:00'))))) AS
BusinessDayDate, STORED AS RetailStoreID, TRAN#D AS TransactionNumber, REG#D
AS WorkstationID, RETAIL AS TransactionGrandAmount, VIP#D AS AlternateID,
DISCOUNT, VOUCHER#D AS VOUCHER_D, TRIM(SKU#) AS ItemID, A03K2 AS
UnitCostPrice, RETAIL AS RegularSalesUnitPrice, (RETAIL*QTY) AS
ExtendedAmount, QTY AS Quantity, ROW_NUMBER() OVER () rownumber,
(RETAIL*QTY) AS ActualRetail,
VOUCHERCD AS VoucherCode, VOUCHER#D AS VoucherNumber
FROM FBF02P
LEFT OUTER JOIN KSK2P ON SKUK2 = SKU#
WHERE TRANDATED = #[payload[0]['TRANDATED']] AND STORED = #[payload[0]
['STORED']] AND REG#D = #[payload[0]['REG_D']] AND TRAN#D = #[payload[0]
['TRAN_D']]]]></db:parameterized-query>
</db:select>
</enricher>
<expression-component doc:name="Expression"><![CDATA[#
[payload[0].LineItem=flowVars.LineItem]]]></expression-component>
<logger message="#[payload[0]['TRAN_D']]" level="INFO"
doc:name="Logger"/>
</batch:step>
<batch:step name="Batch_Step2">
<batch:commit streaming="true" doc:name="Batch Commit">
<data-mapper:transform config-ref="List_Map__To_XML_1"
doc:name="List<Map> To XML"/>
<file:outbound-endpoint path="${output.path}"
outputPattern="TranImport#[server.dateTime.format('yyyyMMdd_HHmmss')].xml"
responseTimeout="10000" doc:name="File"/>
</batch:commit>
</batch:step>
</batch:process-records>
<batch:on-complete>
<logger message="DONE..." level="INFO" doc:name="Logger"/>
</batch:on-complete>
</batch:job>
<flow name="OrceTransactionImportFlow">
<poll doc:name="Poll">
<fixed-frequency-scheduler frequency="1" timeUnit="DAYS"/>
<db:update config-ref="Generic_Database_Configuration"
doc:name="Database">
<db:parameterized-query><![CDATA[UPDATE ORCTEXHDRP
SET HPRCFLAG = 'P'
WHERE HPRCFLAG = '' OR HPRCFLAG = 'P']]></db:parameterized-query>
</db:update>
</poll>
<choice doc:name="Choice">
<when expression="#[payload == 0]">
<logger message="Zero payload..." level="INFO"
doc:name="Logger"/>
</when>
<otherwise>
<batch:execute name="OrceTransactionImportBatch"
doc:name="OrceTransactionImportBatch"/>
</otherwise>
</choice>
</flow>
Inside your database connector configuration you should setup a Connection Pooling profile.
Need to poll multiple tables of a database connector. When trying to apply separate poll on tables using composite source
<composite-source>
<poll>
<db:select config-ref="databaseConnector"/> <!--select on table 1-->
</poll>
<poll>
<db:select config-ref="databaseConnector"/> <!--select on table 2-->
</poll>
</composite-source>
getting an error poller already registered on endpoint uri. How can i poll multiple tables for updated data using a database connector.
Use three flows:
<flow name="poll-table-1">
<poll frequency="...">...</poll>
<flow-ref name="table-data-processor" />
</flow>
<flow name="poll-table-2">
<poll frequency="...">...</poll>
<flow-ref name="table-data-processor" />
</flow>
<flow name="table-data-processor">
...
</flow>
You can try the following way:-
<composite-source>
<poll frequency="10000" doc:name="Poll">
<processor-chain >
<db:select config-ref="Oracle_Configuration" doc:name="Database">
<db:parameterized-query><![CDATA[select * from Table1]]></db:parameterized-query>
</db:select>
<logger level="INFO" message="Your Payload from Table1:- ....." doc:name="Logger"/>
<db:select config-ref="Oracle_Configuration" doc:name="Database">
<db:parameterized-query><![CDATA[select * from Table2]]></db:parameterized-query>
</db:select>
<logger level="INFO" message="Your Payload from Table2:- ...." doc:name="Logger"/>
</processor-chain>
</poll>
</composite-source>
<logger level="INFO" message="The remaining flow " doc:name="Logger"/>
This is working fine for me :)
Converting from CSV to JSON using mule datamapper. I want to check if required field is empty. If empty log that field and discard it for further processing.
I know in script option we have if(input.data.length >0).'
But how to discard the whole row if this fails??
You can do this within mule datamapper simply by encapsulating the whole conversion within the if statements opening and closing braces. Something like this:
if ( input.Quantity > 0 ) {
output.id = input.id;
output.Customer = input.Customer;
output.Quantity = input.Quantity;
output.Price = input.Price;
}
However a different, perhaps better, approach would be to let the datamapper transform every row into JSON and then split and filter as seperate steps in the flow.
<flow name="filterindatamapperFlow2" doc:name="filterindatamapperFlow2">
<file:inbound-endpoint path="/tmp/inbox" doc:name="Inbound file"/>
<data-mapper:transform config-ref="CSV_To_UnfilteredJSON" doc:name="CSV To Unfiltered JSON"/>
<request-reply>
<vm:outbound-endpoint path="splittandprocess" exchange-pattern="one-way"/>
<vm:inbound-endpoint path="result"/>
</request-reply>
<json:object-to-json-transformer doc:name="Object to JSON"/>
<file:outbound-endpoint path="/tmp/outbox" doc:name="Outbound file"/>
</flow>
<flow name="splittandprocess">
<vm:inbound-endpoint path="splittandprocess" exchange-pattern="one-way"/>
<json:json-to-object-transformer returnClass="java.util.List" doc:name="JSON to Object"/>
<splitter expression="#[payload]" doc:name="Splitter"/>
<json:json-to-object-transformer returnClass="java.util.Map" doc:name="JSON to Object"/>
<message-filter doc:name="Filter Out Orders With No Quantity" onUnaccepted="handleFilteredMessages">
<expression-filter expression="#[payload['Quantity'] > 0]" />
</message-filter>
<collection-aggregator failOnTimeout="false" timeout="1000"/>
<vm:outbound-endpoint path="result" exchange-pattern="one-way"/>
</flow>
<flow name="handleFilteredMessages">
<logger message="Payload filtered #[payload]" level="ERROR" doc:name="Logger"/>
</flow>
I am facing one Strange issue ... My Mule flow is as follow :-
<jdbc-ee:connector name="Database_Global" dataSource-ref="DB_Source" validateConnections="true" queryTimeout="-1" pollingFrequency="0" doc:name="Database">
<jdbc-ee:query key="InsertQuery" value="INSERT INTO getData(ID,NAME,AGE,DESIGNATION)VALUES(#[flowVars['id']],#[flowVars['name']],#[flowVars['age']],#[flowVars['designation']])"/>
<jdbc-ee:query key="RetriveQuery" value="Select * from getData where ID=#[flowVars['id']] "/>
</jdbc-ee:connector>
<flow name="MuleDbInsertFlow1" doc:name="MuleDbInsertFlow1">
<http:inbound-endpoint exchange-pattern="request-response" host="localhost" port="8082" path="mainData" doc:name="HTTP"/>
<cxf:jaxws-service service="MainData" serviceClass="com.vertu.services.schema.maindata.v1.MainData" doc:name="SOAPWithHeader" />
<component class="com.vertu.services.schema.maindata.v1.Impl.MainDataImpl" doc:name="JavaMain_ServiceImpl"/>
<mulexml:object-to-xml-transformer doc:name="Object to XML"/>
<choice doc:name="Choice">
<when expression="#[message.inboundProperties['SOAPAction'] contains 'retrieveDataOperation']">
<processor-chain doc:name="Processor Chain">
<set-variable variableName="id" value="#[xpath('//id').text]" doc:name="Variable"/>
<logger message="ID from req #[flowVars['id']]" level="INFO" doc:name="Logger"/>
<jdbc-ee:outbound-endpoint exchange-pattern="request-response" queryKey="RetriveQuery" queryTimeout="-1" connector-ref="Database_Global" doc:name="Database (JDBC)"/>
<choice doc:name="Choice">
<when expression="#[message.payload.isEmpty()]">
<processor-chain>
<!-- Data not exists .. We cannot display -->
<logger message="No records found in Database !!!" level="INFO" doc:name="Logger"/>
</processor-chain>
</when>
<otherwise>
<processor-chain>
<!-- Data exists .. We cannotdisplay -->
<logger message="The Data retrieved from the Database" level="INFO" doc:name="Logger"/>
</processor-chain>
</otherwise>
</choice>
Now the issue is whenever I use the query RetriveQuery:- Select * from getData where ID=#[flowVars['id']]
It goes to the choice block where the logger shows No records found in Database !!! .. But you can see I placed a logger before call the SQL query by DB outbound
<logger message="ID from req #[flowVars['id']]" level="INFO" doc:name="Logger"/>
<jdbc-ee:outbound-endpoint exchange-pattern="request-response" queryKey="RetriveQuery" queryTimeout="-1" connector-ref="Database_Global" doc:name="Database (JDBC)"/>
which prints #[flowVars['id']] and I am successfully getting the value ..
But I don't know why it is going to the <when expression="#[message.payload.isEmpty()]">block ...
If I use the following in RetriveQuery : Select * from getData where ID=22
Then it's successfully getting the value of ID in the query ..
Please let me know why it's not getting the value in SQL query if I use a flowVars ..
It's executing successfully for insert and update query but not for Select ..
Pls note :- here the value of ID in Select * from getData where ID is integer ..
This is strange.
Try cleaning and re-building the project: the version that's running is maybe not using the latest config.
Yes, this was a strange and I found cleaning and re building the project in studio as David suggested worked .. may be there was an issue in picking up the latest config and reflecting in the proect