I hear that we con't write two Inbound-endpoints in one flow, but below is my requirement, Second select query needs first query payload value. When I run this giving an exception. Please let me know if there is any alternate way to write this.
<flow name="Some">
<jdbc:inbound-endpoint queryKey="SelectSome"
connector-ref="ProConnector" doc:name="SomeFromPro"
pollingFrequency="10000" queryTimeout="-1">
<jdbc:transaction action="NONE" timeout="10" />
<jdbc:query key="SelectSome"
value="SELECT top 1 * from table1 where IsProcessed = 0" />
<jdbc:query key="SelectSome.ack"
value="update table1 set IsProcessed=1 where ID = #[map-payload:ID] " />
</jdbc:inbound-endpoint>
<jdbc:inbound-endpoint queryKey="SelectSomeBR"
connector-ref="ProConnector" doc:name="SomeBRFromPro"
pollingFrequency="1000" queryTimeout="-1">
<jdbc:transaction action="NONE" timeout="10" />
<jdbc:query key="SelectSomeBR"
value="SELECT * from table2 where IsProcessed = 0 and ParentID = #[map-payload:ID]" />
<jdbc:query key="SelectSomeBR.ack"
value="update table2 set IsProcessed=1 where ParentID = #[map-payload:ID] " />
</jdbc:inbound-endpoint>
.
.
.
.
</flow>
Any help would be appreciated.
Thank you.
You need to use an outbound select query for your second query, like:
<jdbc:outbound-endpoint queryKey="SelectSomeBR"
connector-ref="ProConnector" doc:name="SomeBRFromPro"
pollingFrequency="1000" queryTimeout="-1"
exchange-pattern="request-response">
<jdbc:transaction action="NONE" timeout="10" />
<jdbc:query key="SelectSomeBR"
value="SELECT * from table2 where IsProcessed = 0 and ParentID = #[map-payload:ID]" />
<jdbc:query key="SelectSomeBR.ack"
value="update table2 set IsProcessed=1 where ParentID = #[map-payload:ID] " />
</jdbc:outbound-endpoint>
Reference: https://developer.mulesoft.com/docs/display/current/JDBC+Transport+Reference#JDBCTransportReference-OutboundSELECTQueries
PS. This old expression syntax is obsolete: #[map-payload:ID] Use MEL instead: #[message.payload.ID]" (assuming the message payload is a map with a key named ID).
Related
As ClickHouse can run complex SQL efficiently, we now write hundred-lines-in-one SQL to get the real time analysis results(join lots of source table). And it brings a problem is that it's so hard to maintain and reuse.
And we are using mybatis to write the sql in our Java Application. We try to use Mybatis's global parameter to define some sql for reuse, but not helps a lot in such huge lines SQL.
Is there any way to avoid such long sql or make the sql more readable and make this work more engineerable?
<select id="brandAnalysis" resultType="java.util.HashMap">
SELECT gmvRank.id_code,
gmvRank.rank,
gmvRank.lastRank,
gmvRank.gmv as sale_money,
member.dealerMembers as dealer_turnover,
member.teamMembers,
case
when member.dealerMembers = 0 then 0
when member.teamMembers = 0 then 0
else member.dealerMembers/member.teamMembers end AS dealer_turnover_active
FROM (
SELECT id_code,
<choose>
<when test="monthQuarter == 3">
case
when max(`quarter`) = #{quarter} then argMax(${sqlRankField},`month`)
else null end as rank,
case
when max(`quarter`) = #{quarter} then argMax(${sqlGmvField},`month`)
else null end as gmv,
case
when min(`quarter`) = #{compareQuarter} then argMin(${sqlRankField},`month`)
else null end as lastRank
</when>
<otherwise>
case
when max(`month`) = #{month} then argMax(${sqlRankField},`month`)
else null end as rank,
case
when max(`month`) = #{month} then argMax(${sqlGmvField},`month`)
else null end as gmv,
case
when min(`month`) = #{compareMonth} then argMin(${sqlRankField},`month`)
else null end as lastRank
</otherwise>
</choose>
FROM
tengju.dws_crm_brand_analysis_rank_dd_m
<choose>
<when test="monthQuarter == 3">
PREWHERE quarter in (#{quarter},#{compareQuarter})
and insert_time in (
SELECT MAX(insert_time)
FROM tengju.dws_crm_brand_analysis_rank_dd_m
PREWHERE quarter in (#{quarter},#{compareQuarter})
GROUP BY quarter
)
</when>
<otherwise>
PREWHERE month in (#{month},#{compareMonth})
and insert_time in (
SELECT MAX(insert_time)
FROM tengju.dws_crm_brand_analysis_rank_dd_m
PREWHERE month in (#{month},#{compareMonth})
GROUP BY month
)
</otherwise>
</choose>
<if test="dominationIdCode != null and dominationLevel != null">
and id_code in (
SELECT id_code
FROM tengju.dwd_user_domination_relation_map_all
PREWHERE visit_date >= formatDateTime(yesterday(),'%F')
AND insert_time in (
SELECT max(insert_time)
FROM tengju.dwd_user_domination_relation_map_all
PREWHERE visit_date >= formatDateTime(yesterday(),'%F')
)
AND domination_star_level = #{dominationLevel}
AND domination_id_code = #{dominationIdCode}
<if test="starLevels != null and starLevels.size() > 0">
AND star_level in
<foreach collection="starLevels" item="item" index="index" open="(" close=")" separator=",">
#{item}
</foreach>
</if>
<if test="isDirect != null and isDirect > 0">
AND parent_id_code = #{dominationIdCode}
</if>
)
</if>
<if test="idCodes != null and idCodes.size() > 0">
AND id_code in
<foreach collection="idCodes" item="item" index="index" open="(" close=")" separator=",">
#{item}
</foreach>
</if>
AND brand_id = #{brandId}
GROUP BY id_code
) as gmvRank
GLOBAL LEFT JOIN (
<choose>
<when test="crmRankCrowdPerspective == 4">
<include refid="smallTeamDealer" />
</when>
<when test="crmRankCrowdPerspective == 1">
<include refid="KATeamDealer" />
</when>
<otherwise>
<include refid="ecOrCityTeamDealer" />
</otherwise>
</choose>
) as member
on (gmvRank.id_code = member.id_code)
where 1=1
<if test="saleMoneyFrom != null">
AND gmvRank.gmv >= toDecimal128(#{saleMoneyFrom},2)
</if>
<if test="saleMoneyTo != null">
AND toDecimal128(#{saleMoneyTo},2) >= gmvRank.gmv
</if>
order by ${sortType}
LIMIT #{offset},#{pageSize}
</select>
User common table expressions to get rid of nested operations. Each make and version of database server has its own style of SQL. Not all have common table expressions. All have views though. SQL queries can be insanely verbose, for sure there is no other option.
I have the following XML and would like to extract the PrimaryTeams, SecondaryTeams and OverflowTeams arrays from this and either have them comma separated or one per row.
I have the following xml:
declare #xml xml
set #xml = '<SimpleStrategy xmlns:i="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://schemas.datacontract.org/2004/07/Synthesys.Switch.ACD">
<Id>00000000-0000-0000-0000-000000000000</Id>
<Name>Default</Name>
<AcceptedCLIs xmlns:d2p1="http://schemas.microsoft.com/2003/10/Serialization/Arrays">
<d2p1:string>07811353995</d2p1:string>
</AcceptedCLIs>
<ActiveHours>
<FridayEnd />
<FridayStart />
<MondayEnd />
<MondayStart />
<SaturdayEnd />
<SaturdayStart />
<SundayEnd />
<SundayStart />
<ThursdayEnd />
<ThursdayStart />
<TuesdayEnd />
<TuesdayStart />
<UseIndividualWeekDays>false</UseIndividualWeekDays>
<WednesdayEnd />
<WednesdayStart />
<WeekdayEnd />
<WeekdayStart />
</ActiveHours>
<AgentUserName />
<AllowRouteDuringFinalMessage>false</AllowRouteDuringFinalMessage>
<CRMPrefix />
<DirectDDIMessage />
<DirectDDIPassThrough>false</DirectDDIPassThrough>
<EmergencyBusyBack>false</EmergencyBusyBack>
<EmergencyDivertNumber />
<EmergencyWavFile />
<FinallyDivertNumber />
<FinallyDrop>true</FinallyDrop>
<FinallyMessageFile />
<MaximumQueueLength>0</MaximumQueueLength>
<MaximumQueueWait>0</MaximumQueueWait>
<MinimumRingTime>4000</MinimumRingTime>
<MusicOnHold />
<MusicWhileWaiting />
<NumberOfRings>2</NumberOfRings>
<OutOfHoursDivertNumber />
<OutOfHoursDrop>true</OutOfHoursDrop>
<OutOfHoursMessage />
<OverflowMessage />
<OverflowTeams xmlns:d2p1="http://schemas.microsoft.com/2003/10/Serialization/Arrays" />
<PrimaryTeams xmlns:d2p1="http://schemas.microsoft.com/2003/10/Serialization/Arrays">
<d2p1:int>3</d2p1:int>
<d2p1:int>1</d2p1:int>
</PrimaryTeams>
<Priority>1</Priority>
<RecordAgent>false</RecordAgent>
<RecordCall>true</RecordCall>
<RecordCustomer>false</RecordCustomer>
<RegulatoryMessage>Default.wav</RegulatoryMessage>
<SecondaryOverflowMessage />
<SecondaryTeams xmlns:d2p1="http://schemas.microsoft.com/2003/10/Serialization/Arrays" />
<SendBusyIfQueueTooLong>false</SendBusyIfQueueTooLong>
<SendBusyIfWaitTooLong>false</SendBusyIfWaitTooLong>
<TimeInOverflow>-1</TimeInOverflow>
<TimeWithDirectDDI>20000</TimeWithDirectDDI>
<TimeWithPrimaryTeams>-1</TimeWithPrimaryTeams>
<TimeWithSecondaryTeams>20000</TimeWithSecondaryTeams>
<UseDirectDDI>false</UseDirectDDI>
<UsePAM>false</UsePAM>
<UseSecondaryTeams>false</UseSecondaryTeams>
<WrapTime>40000</WrapTime>
</SimpleStrategy>'
I then created the following SQL Statement to try and extract the Teams
;WITH XMLNAMESPACES ('http://www.w3.org/2001/XMLSchema-instance' as i, 'http://schemas.microsoft.com/2003/10/Serialization/Arrays' as d2p1,
DEFAULT 'http://schemas.datacontract.org/2004/07/Synthesys.Switch.ACD')
SELECT #xml,
#xml.value('(/SimpleStrategy/Name)[1]', 'varchar(255)'),
#xml.value('(/SimpleStrategy/PrimaryTeams)[1]', 'int') as PrimaryTeams,
#xml.value('(/SimpleStrategy/SecondaryTeams)[1]', 'int') as SecondaryTeams,
#xml.value('(/SimpleStrategy/OverflowTeams)[1]', 'int') as OverflowTeams
But all I get is the TeamID's concatenated together.
,PrimaryTeams,SecondaryTeams,OverflowTeams
Default,31,0,0
Any ideas?
Thanks
Matt
Your XML shows two team IDs in <PrimaryTeams>, while both other team nodes are empty... You did not tell us anything about the expected counts in there. However, the following approach will return a kind of entity-value-pairs with all IDs for all Teams. Hope this is what you need:
;WITH XMLNAMESPACES ('http://www.w3.org/2001/XMLSchema-instance' as i, 'http://schemas.microsoft.com/2003/10/Serialization/Arrays' as d2p1,
DEFAULT 'http://schemas.datacontract.org/2004/07/Synthesys.Switch.ACD')
SELECT 'Name' AS Caption
,1 AS RowInx
,#xml.value('(/SimpleStrategy/Name)[1]', 'varchar(255)') AS Content
UNION ALL
SELECT 'Primary Team'
,ROW_NUMBER() OVER(ORDER BY (SELECT NULL))
,t.value('.','varchar(255)')
FROM #xml.nodes('/SimpleStrategy/PrimaryTeams/d2p1:int') A(t)
UNION ALL
SELECT 'Secondary Team'
,ROW_NUMBER() OVER(ORDER BY (SELECT NULL))
,t.value('.','varchar(255)')
FROM #xml.nodes('/SimpleStrategy/SecondaryTeams/d2p1:int') A(t)
UNION ALL
SELECT 'Overflow-Team'
,ROW_NUMBER() OVER(ORDER BY (SELECT NULL))
,t.value('.','varchar(255)')
FROM #xml.nodes('/SimpleStrategy/OverflowTeams/d2p1:int') A(t);
I am using Mule 4 and Anypoint 7 and want to setup the database connector to SELECT all customers from my SQL server database table but if the customerName query parameter is populated in the request then I want to add the WHERE clause to only return customers with the same name as the customerName query parameter otherwise it should just return all customers.
My code is below but I am struggling to get the syntax correct.
<db:select doc:name="Select Customers" doc:id="98a4aa2f-b0b6-4fb5-ab27-d70489fd532d" config-ref="db-config">
<ee:repeatable-file-store-iterable />
<db:sql >SELECT TOP 10 * FROM MYDB.dbo.Customer $(if (attributes.queryParams.customerName != null and isEmpty(attributes.queryParams.customerName) == false) "WHERE Name = :customerName" else "")</db:sql>
<db:input-parameters ><![CDATA[#[{'customerName' : attributes.queryParams.customerName}]]]></db:input-parameters>
</db:select>
How can I do this?
Thanks
You were on the right path. I think you were only missing the evaluation tags around the SQL in the db:sql element.
<db:select doc:name="Select Customers" doc:id="98a4aa2f-b0b6-4fb5-ab27-d70489fd532d"
config-ref="db-config">
<ee:repeatable-file-store-iterable />
<db:sql>#["SELECT TOP 10 * FROM MYDB.dbo.Customer
$(if (isEmpty(attributes.queryParams.customerName) == false) "WHERE Name = :customerName" else "") "]</db:sql>
<db:input-parameters ><![CDATA[#[{'customerName' : attributes.queryParams.customerName}]]]></db:input-parameters>
</db:select>
It is easier to debug things like this with variables, so that you can see the individual values. FWIW heres my test code:
<set-variable variableName="additionalWhereClause"
value='#[if ( isEmpty(attributes.queryParams.email) == false)
"WHERE Email = :emailParm"
else "" ]' />
<set-variable variableName="selectSql"
value="#['SELECT FirstName, LastName, Email
FROM User
$( vars.additionalWhereClause )
ORDER BY Email LIMIT 10']" />
<logger level="INFO" message="queryParams: #[attributes.queryParams]" doc:id="96c62f84-2c98-4df6-829c-e00c9fcec9ca" />
<logger level="INFO" message="additionalWhereClause #[vars.additionalWhereClause]" doc:id="0d3611b4-34ae-4ebb-b931-6d31ce3804c1" />
<logger level="INFO" message="selectSql #[vars.selectSql]" doc:id="5c56342d-9674-4891-9d7e-bb32319f4ad0" />
<db:select doc:name="MySQL Query" doc:id="e60be3e6-9b51-4b3b-9dfa-4ee0af65cb03"
config-ref="mysql-config">
<ee:repeatable-file-store-iterable />
<db:sql>#[ vars.selectSql ]</db:sql>
<db:input-parameters><![CDATA[#[{'emailParm' : attributes.queryParams.email}]]]></db:input-parameters>
</db:select>
<healthcare>
<plans>
<plan1>
<planid>100</planid>
<planname>medical</planname>
<desc>medical</desc>
<offerprice>500</offerprice>
<area>texas</area>
</plan1>
<plan2>
<planid>101</planid>
<planname>dental</planname>
<desc>dental</desc>
<offerprice>1000</offerprice>
<area>texas</area>
</plan2>
</plans>
</healthcare>
<splitter evaluator="xpath" expression="/healthcare/plans" doc:name="Splitter"/>
<transformer ref="domToXml" doc:name="Transformer Reference"/>
<logger level="INFO" doc:name="Logger" message=" plans detils...#[message.payload]" />
i want replace offerprice value with other values during runtime.anyhelp appreciated.I tried various various ways . anyone shed light means it saves me lot
You could use XSLT and use identity templates to replace the one element.Or if your really want to do it with MEL, convert to DOM and use Dom4j APIs to set the value and then convert back to XML if needed:
<expression-component><![CDATA[
node = message.payload.getRootElement().selectSingleNode('//plans/plan1/planid');
node.text = 'newvalue';
]]></expression-component>
<mulexml:dom-to-xml-transformer />
<logger level="ERROR" message=" #[payload]" />
EDIT
Here is an example if you want to update multiple nodes. If the transformation gets any more complex, I would really suggest taking a look at XSLT.
<mulexml:xml-to-dom-transformer returnClass="org.dom4j.Document" />
<expression-component><![CDATA[
plans = message.payload.getRootElement().selectNodes('//plans/*');
foreach (plan : plans){
plan.selectSingleNode('offerprice').text = '3000';
} ]]></expression-component>
<mulexml:dom-to-xml-transformer />
<logger level="ERROR" message=" #[payload]" />
I have a flow in Mule where I want to use a date parameter i get from one query as an input for another query.
<jdbc:connector name="myConnector" transactionPerMessage="false" dataSource-ref="myDataSource">
<jdbc:query key="getPollTimes" value="SELECT to_char(last_poll_start, 'YYYY-MM-DD HH24:MI:SS') as last_poll_start, to_char(last_poll_end, 'YYYY-MM-DD HH24:MI:SS') as last_poll_end FROM db_sources WHERE source_system = 'mySystem'" />
<jdbc:query key="getCustomerIds" value="SELECT id FROM customers WHERE updated < TO_DATE(#[variable:last_poll_end],'YYYY-MM-DD HH24:MI:SS')" />
</jdbc:connector>
<flow name="myFlow">
<enricher>
<jdbc:outbound-endpoint queryKey="getPollTimes" exchange-pattern="request-response" />
<enrich target="#[variable:last_poll_end]" source="#[groovy:payload.last_poll_end]"/>
</enricher>
<logger level="INFO" message="last_poll_end = #[variable:last_poll_end]" />
<jdbc:outbound-endpoint queryKey="getCustomerIds" exchange-pattern="request-response" />
</flow>
When running this I cannot get this to work (note that I am using an Oracle DB). I have included the exception below. Have anyone encountered this?
--------------------------------------------------------------------------------
Exception stack is:
1. Invalid column type(SQL Code: 17004, SQL State: + null) (java.sql.SQLException)
oracle.jdbc.driver.DatabaseError:113 (null)
2. Invalid column type Query: SELECT ID FROM CUSTOMERS WHERE UPDATED < TO_DATE(?,'YYYY-MM-DD HH24:MI:SS') Parameters: [[2000-01-01]](SQL Code: 17004, SQL State: + null) (java.sql.SQLException)
org.apache.commons.dbutils.QueryRunner:540 (null)
3. Failed to route event via endpoint: DefaultOutboundEndpoint{endpointUri=jdbc://getCustomerIds, connector=JdbcConnector
{
name=myConnector
lifecycle=start
this=668e94
numberOfConcurrentTransactedReceivers=4
createMultipleTransactedReceivers=false
connected=true
supportedProtocols=[jdbc]
serviceOverrides=<none>
}
, name='endpoint.jdbc.getCustomerIds', mep=REQUEST_RESPONSE, properties={queryTimeout=-1}, transactionConfig=Transaction{factory=null, action=INDIFFERENT, timeout=0}, deleteUnacceptedMessages=false, initialState=started, responseTimeout=10000, endpointEncoding=UTF-8, disableTransportTransformer=false}. Message payload is of type: ArrayList (org.mule.api.transport.DispatchException)
org.mule.transport.AbstractMessageDispatcher:106 (http://www.mulesoft.org/docs/site/current3/apidocs/org/mule/api/transport/DispatchException.html)
--------------------------------------------------------------------------------
Problem is solved. The issue was partly that the date variable I got back from the first query was stored as an array. To resolve this i pick out the first element. Besides that I removed the to_date() in the second sql query.
This gets the first element in the array:
<enrich target="#[variable:last_poll_end]" source="#[groovy:payload.last_poll_end[0]]"/>
The updated sql:
<jdbc:query key="getCustomerIds" value="SELECT id FROM customers WHERE updated < #[variable:last_poll_end]" />