I am very new to Mulesoft and an entry-level programmer. I have been trying to figure out how to implement Mule 4 custom policies. This is the online documentation: https://docs.mulesoft.com/api-manager/2.x/http-policy-transform#add-headers
Following the examples, I was successfully able to add request and response headers. My main goal is to add the request headers while using variables. I am trying MEL (https://docs.mulesoft.com/mule-runtime/3.7/mule-expression-language-examples) to try to call the variable names but it doesn't work. HOWEVER, whenever I try logging the variables it returns the correct value.
<?xml version="1.0" encoding="UTF-8"?>
<mule xmlns="http://www.mulesoft.org/schema/mule/core"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:http-policy="http://www.mulesoft.org/schema/mule/http-policy"
xmlns:http-transform="http://www.mulesoft.org/schema/mule/http-policy-transform"
xmlns:secure-properties="http://www.mulesoft.org/schema/mule/secure-properties"
xsi:schemaLocation="http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd
http://www.mulesoft.org/schema/mule/http-policy http://www.mulesoft.org/schema/mule/http-policy/current/mule-http-policy.xsd
http://www.mulesoft.org/schema/mule/http-policy-transform http://www.mulesoft.org/schema/mule/http-policy-transform/current/mule-http-policy-transform.xsd
http://www.mulesoft.org/schema/mule/secure-properties http://www.mulesoft.org/schema/mule/secure-properties/current/mule-secure-properties.xsd">
<http-policy:proxy name="{{{policyId}}}-custom-policy">
<http-policy:source propagateMessageTransformations="true">
<try>
<set-variable variableName="header" value="TEST_HEADER"/>
<logger level="INFO" message="#[vars.header]" />
<http-transform:add-headers outputType="request">
<http-transform:headers>#[{'TEST_HEADER': 'TEST'}]</http-transform:headers>
</http-transform:add-headers>
<http-policy:execute-next/>
<http-transform:add-headers outputType="response">
<http-transform:headers>#[{'Header_Added': '#[vars.header]'}]</http-transform:headers>
#this step doesn't work as I hoped it would
</http-transform:add-headers>
<logger level="INFO" message="#[vars.header]" />
#logging prints the value of header
</try>
</http-policy:source>
</http-policy:proxy>
</mule>
I am trying to add #[vars.header] as a request header name and possibly the value? Would I need to create another variable to have the header value? Can someone please guide me the right direction?
Thank you
Mule 4 uses dataweave expressions. Try removing the quotes so its not treated as a static string and remove the nested expression brackets '#[]':
<http-transform:headers>#[{'Header_Added': vars.header}]</http-transform:headers>
Or another tip, if you need to access a var in middle of a string you can use $() syntax:
<http-transform:headers>#[{'Header_Added': '$(vars.header)'}]</http-transform:headers>
I'm still new to Mulesoft's Mule ESB and I just cannot get past this issue.
I'm having trouble understanding why my Mule flow is failing with a sql server insert error.
It appears that the payload has data but for some reason the query parameters are not being populated with that data. I can read from a table or file fine it's just the writing to a db table that goes sideways.
What do I need to do to write my payload to a sql server database?
Below is the error message I get when I run my Mule Application in AnyPoint Studio. I've removed the bulk of the payload from the message due to privacy.
I've also posted my flow xml.
Any help would be appreciated. Thanks
********************************************************************************
Message : Cannot insert the value NULL into column 'AgentId', table 'GARDB1dev.src.AgentList'; column does not allow nulls. INSERT fails. (com.microsoft.sqlserver.jdbc.SQLServerException).
Payload : [{Agent_ID=10032, **REDACTED DATA** [..]]
Payload Type : java.util.ArrayList
Element : /Load-AgentList/processors/2/1/1 # gar-data-load:sharepoint.xml:51
Element XML : <db:insert config-ref="GAR-DB-Connection-SSPI" doc:name="Insert AgentList Data">
<db:parameterized-query>insert into src.AgentList (AgentId,PartyId,PartyType,AgentIdStatus,SalesOrg,AgentIdStatusStart,AgentIdStatusEnd,AgentName,SubOrganization,[Function],NewToOrg,NewToChannel,DaysAgentIdActive,Region,AgentPhone,CertificationSummary)Values (#[payload.Agent_ID],#[payload.Party_ID],#[payload.Party_Type],#[payload.Agent_ID_Status],#[payload.Sales_Organization],#[payload.Agent_ID_Status_Start],#[payload.Agent_ID_Status_End],#[payload.Agent_Name],#[payload.Sub_Organization],#[payload.Function],#[payload.New_To_Org],#[payload.New_To_Channel],#[payload.Days_Agent_ID_Active],#[payload.Region],#[payload.Agent_Phone],#[payload.Certification_Summary])</db:parameterized-query>
</db:insert>
--------------------------------------------------------------------------------
The XML description of my Mule Flow is the following:
<?xml version="1.0" encoding="UTF-8"?>
<mule xmlns:db="http://www.mulesoft.org/schema/mule/db" xmlns:tracking="http://www.mulesoft.org/schema/mule/ee/tracking" xmlns:file="http://www.mulesoft.org/schema/mule/file" xmlns:dw="http://www.mulesoft.org/schema/mule/ee/dw" xmlns:metadata="http://www.mulesoft.org/schema/mule/metadata" xmlns:sharepoint2010="http://www.mulesoft.org/schema/mule/sharepoint2010" xmlns:sharepoint="http://www.mulesoft.org/schema/mule/sharepoint" xmlns="http://www.mulesoft.org/schema/mule/core" xmlns:doc="http://www.mulesoft.org/schema/mule/documentation"
xmlns:spring="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-current.xsd
http://www.mulesoft.org/schema/mule/sharepoint http://www.mulesoft.org/schema/mule/sharepoint/current/mule-sharepoint.xsd
http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd
http://www.mulesoft.org/schema/mule/sharepoint2010 http://www.mulesoft.org/schema/mule/sharepoint2010/current/mule-sharepoint2010.xsd
http://www.mulesoft.org/schema/mule/file http://www.mulesoft.org/schema/mule/file/current/mule-file.xsd
http://www.mulesoft.org/schema/mule/ee/dw http://www.mulesoft.org/schema/mule/ee/dw/current/dw.xsd
http://www.mulesoft.org/schema/mule/db http://www.mulesoft.org/schema/mule/db/current/mule-db.xsd
http://www.mulesoft.org/schema/mule/ee/tracking http://www.mulesoft.org/schema/mule/ee/tracking/current/mule-tracking-ee.xsd">
<file:connector name="input" autoDelete="false" streaming="true" validateConnections="true" doc:name="File" readFromDirectory="C:\Users\rpearso7\_gar-data\input" />
<flow name="Load-AgentList">
<file:inbound-endpoint path="C:\Users\rpearso7\_gar-data\input" responseTimeout="10000" doc:name="File" moveToDirectory="C:\Users\rpearso7\_gar-data\archive" connector-ref="input" moveToPattern="#[function:datestamp]-#[message.inboundProperties['originalFilename']]">
<file:filename-regex-filter pattern="agent_list\.csv" caseSensitive="false"/>
</file:inbound-endpoint>
<dw:transform-message doc:name="Transform Message" metadata:id="fdb1aea0-6e94-425f-9493-ee71983b8eb1">
<dw:input-payload mimeType="application/csv"/>
<dw:set-payload><![CDATA[%dw 1.0
%output application/java
---
payload]]></dw:set-payload>
</dw:transform-message>
<logger message="************** Payload Loaded" level="INFO" doc:name="Log Payload"/>
<scatter-gather doc:name="Scatter-Gather">
<threading-profile maxThreadsActive="1" poolExhaustedAction="WAIT"/>
<processor-chain>
<logger message="***************************** Truncate AgentList Table" level="INFO" doc:name="Logger"/>
<db:update config-ref="GAR-DB-Connection-SSPI" doc:name="Truncate table">
<db:parameterized-query><![CDATA[truncate table src.AgentList;]]></db:parameterized-query>
</db:update>
<logger message="************************************ AgentList table truncated" level="INFO" doc:name="Logger"/>
</processor-chain>
<processor-chain>
<logger message="********************************* Start Insert" level="INFO" doc:name="Logger"/>
<db:insert config-ref="GAR-DB-Connection-SSPI" doc:name="Insert AgentList Data">
<db:parameterized-query><![CDATA[insert into src.AgentList (
AgentId,
PartyId,
PartyType,
AgentIdStatus,
SalesOrg,
AgentIdStatusStart,
AgentIdStatusEnd,
AgentName,
SubOrganization,
[Function],
NewToOrg,
NewToChannel,
DaysAgentIdActive,
Region,
AgentPhone,
CertificationSummary)
Values (
#[payload.Agent_ID],
#[payload.Party_ID],
#[payload.Party_Type],
#[payload.Agent_ID_Status],
#[payload.Sales_Organization],
#[payload.Agent_ID_Status_Start],
#[payload.Agent_ID_Status_End],
#[payload.Agent_Name],
#[payload.Sub_Organization],
#[payload.Function],
#[payload.New_To_Org],
#[payload.New_To_Channel],
#[payload.Days_Agent_ID_Active],
#[payload.Region],
#[payload.Agent_Phone],
#[payload.Certification_Summary])]]></db:parameterized-query>
</db:insert>
</processor-chain>
</scatter-gather>
<logger message="**************** Complete" level="INFO" doc:name="Logger"/>
</flow>
</mule>
I have had a look what happens in the debugger, when I use your transformation on a CSV file.
I have set a breakpoint at the first logger, so I could stop the execution and have a look how different Mule Expression Language (MEL) expressions behave in different steps of your flow.
When I evaluate #[payload] after the transformation; I get an ArrayList with the lines of the CSV file.
This seems to be correct.
However... Since the scather-gather only sends a copy to each the same payload is visible in the debugger after the scather-gather as well.
For details see the following blog post: Scatter-Gather in Mule ESB
Scatter-Gather is a routing message processor in Mule ESB runtime that
sends a request message to multiple targets concurrently. It then
collects the responses from all routes and aggregates them back into a
single response.
According to this the DB INSERT processing-step in your flow gets an ArrayList. The elements of the ArrayList can be indexed with a null-based integer index. You can't use the names of the columns of the CSV file to use them.
First you have to select an entry from the ArrayList and then you can use the column headers from the CSV file on the entries of the ArrayList, like in the following example:
#[payload[0].Agent_id]
This MEL expression returns the Agent ID as expected. But if you try to execute #[payload.Agent_id] ; you get null.
In my opinion you do exactly this in your INSERT. Instead of inserting each of the entries of the ArrayList created from the CSV file, you try to access the columns of the CSV file on the ArrayList and not on its elements.
A potential solution could be to add a foreach scope.
Using this you could iterate through the entries of the ArrayList and insert each of them to your database table.
Please also see the results of different MEL expressions in the different states in a similfied version of you flow:
(Please note, that the Mule Expression Language is case-sensitive. If the "ID" substring is replaced with "Id", then I get "null" back.)
I have been testing with a simple CSV file with the following contents:
Agent_ID,Agent_Name
7,james bond
1,Mr. 47
8,Dr. X
You could consider a solution similar to the following one:
I have just added a foreach scope.
This flow has generated the following console-output:
INFO 2018-03-14 16:08:37,736 [[testproject01].connector.file.mule.default.receiver.01] org.mule.transport.file.FileMessageReceiver: Lock obtained on file: X:\SomePathToTheFile\agent_list.csv
INFO 2018-03-14 16:08:38,485 [[testproject01].Load-AgentList.stage1.02] org.mule.api.processor.LoggerMessageProcessor: ************** Payload Loaded
INFO 2018-03-14 16:08:38,488 [[testproject01].ScatterGatherWorkManager.01] org.mule.api.processor.LoggerMessageProcessor: ***************************** Truncate AgentList Table
INFO 2018-03-14 16:08:38,506 [[testproject01].ScatterGatherWorkManager.01] org.mule.api.processor.LoggerMessageProcessor: ********************************* Start Insert
INFO 2018-03-14 16:08:38,506 [[testproject01].ScatterGatherWorkManager.01] org.mule.api.processor.LoggerMessageProcessor: ********************************* Start Insert
INFO 2018-03-14 16:08:38,507 [[testproject01].ScatterGatherWorkManager.01] org.mule.api.processor.LoggerMessageProcessor: ********************************* Start Insert
I hope this is what you are trying to achieve.
If you have furhter questions or I misunderstood your goal, then please add a comment.
Thank you very much.
We have experienced deadlock issues in a similar scenarion, when another process is running an INSERT statement against that same database table.
Scenario:
TRUNCATE is currently being executed on a table, when the same time, concurrently another process INSERTS some data in the same table of the database.
First prioritize the TRUNCATE and acquire lock on the table. Once it is done then go for the INSERT.
You should request a table lock before you execute the TRUNCATE.
If you do this you can't get a deadlock -- the table lock won't be granted before the INSERT finishes and once you have the lock another INSERT can't occur.
Update from the comment:
You can use the LOCK TABLE command.
I have multiple endpoints for different vendor's and we are differentiating it based on the userId for similar service operation and routing calls accordingly.
mule-app.properties
userIds=123,124,125
123.service.uri=http://google.com
124.service.url=http://yahoo.com
Can someone tell if there is a way to dynamically refer property using MEL and flowVariable holding userId value?
<flow name="test">
<http:listener config-ref="mylistenerconfig" path="test" doc:name="Request Listener" />
<set-variable variableName="userId" value="#[message.inboundProperties.userId]" />
<set-variable variableName="userServiceUri" value="${flowVars['userId'].service.uri}" />
<logger level="INFO" message="******* serviceUri=#[userServiceUri] ****" />
</flow>
I tried directly referring that value from message.inboundProperties.userId, referring it using a seperate variable - nothing works. Can someone suggest on how to achieve this?
Load the properties files with Spring:
<util:properties id="muleAppProps"
location="classpath*:mule-app.properties" />
Then you can dynamically refer to values in it with:
#[app.registry.muleAppProps[userId + '.service.uri']]
Assuming userId is a flow var that contains a value like "123"
I'm using Mule ESB CE 3.4. I have a requirement where I'm reading the configuration information from database and using it as the file name for the file outbound endpoint. Here is an example code (the code may not work as I have only given an outline)
<file:connector name="File-Data" autoDelete="false" streaming="true" validateConnections="true" doc:name="File" />
.....
<!-- Gets the configuration from database using a transformer. The transformer populates the configuration entries in a POJO and puts that in a session. -->
<custom-transformer class="com.test.DbGetConfigsTransformer" doc:name="Get Integration Configs"/>
....<!-- some code to process data -->
<logger message="$$$: #[sessionVars['currentFeed'].getFilePattern()]" doc:name="Set JSON File Name" /> -->
<file:outbound-endpoint path="/temp" outputPattern="#[sessionVars['currentFeed'].getFilePattern()]" responseTimeout="10000" mimeType="text/plain" connector-ref="File-Data" doc:name="Save File"/>
The above code throws the following error:
1. The filename, directory name, or volume label syntax is incorrect (java.io.IOException)
java.io.WinNTFileSystem:-2 (null)
2. Unable to create a canonical file for /temp/Test_User_#[function:datestamp:YYYYMMddhhmmss.sss] (org.mule.api.MuleRuntimeException)
org.mule.util.FileUtils:354 (http://www.mulesoft.org/docs/site/current3/apidocs/org/mule/api/MuleRuntimeException.html)
3. Failed to route event via endpoint: DefaultOutboundEndpoint{endpointUri=file:///temp, connector=FileConnector
In the database table, the field name is called FilePattern and it has the value 'Test_User_#[function:datestamp:YYYYMMddhhmmss.sss]. If I hardcode the value or move this value to the mule configuration file
file.name=Test_User_#[function:datestamp:YYYYMMddhhmmss.sss]
and use the configuration property syntax (for e.g. ${file.name} in the 'outputpattern'), it works. But if I read the same from db and use it, it is not working and throwing the error. The logger displays as (which is read from the db)
$$$: Test_#[function:datestamp:YYYYMMddhhmmss.sss]
Any help is much appreciated.
If your datestamp format does not vary, you should just store the environment prefix in your db and use something like:
outputPattern="#[sessionVars['prefix']+server.dateTime.format('YYYYMMddhhmmss.sss')]"
If you need to use your current database values, you can use basic Java string methods to find the correct substrings. For example:
#[sessionVars['currentFeed'].getFilePattern().substring(0,sessionVars['currentFeed'].getFilePattern().indexOf('function')-2)+server.dateTime.format('YYYYMMddhhmmss.sss')]
If you use different datestamp formats, you can find that part as well using similar String methods. However, I still suggest you come up with an implementation that only stores the environment prefix in the db.
i am doing a Object to string conversion, but when i am doing it is adding one extra question mark at the very begin of string.
below is my flow code
<flow name="jatoSmartWriteFromFTP" doc:name="jatoSmartWriteFromFTP" processingStrategy="synchronous">
<ftp:inbound-endpoint host="delvmpllreap03.sapient.com" port="21" path="/home/jatopoc" user="jatopoc" password="jatopoc" responseTimeout="10000" doc:name="FTP" mimeType="text/xml">
<file:filename-wildcard-filter pattern="filter_data.xml"></file:filename-wildcard-filter>
</ftp:inbound-endpoint>
<logger message="#[message.payload]" level="INFO" doc:name="Logger"></logger>
<object-to-string-transformer doc:name="Object to String" mimeType="text/xml"/>
<mulexml:jaxb-xml-to-object-transformer name="XmlToPerson" jaxbContext-ref="jatoJaxbContext" returnClass="com.jato.speedwing.common.vo.JATOXML"/>
<component doc:name="Java">
<method-entry-point-resolver>
<include-entry-point method="insert"></include-entry-point>
</method-entry-point-resolver>
<spring-object bean="jatoDAO"> </spring-object>
</component>
<jms:outbound-endpoint topic="com.jato.smart.updateInfo" connector-ref="VM_Active_MQ" doc:name="JMS"/>
</flow>
below is my xml file content at ftp
<?xml version="1.0" encoding="UTF-8"?>
<code>
data
</code>
below is the string output after doing object-to-string transformer
?<?xml version="1.0" encoding="UTF-8"?>
<code>
data
</code>
now because of this extra question mark, my jaxb-xml-to-object transformer is not working..
can anybody suggest my what do here. since my task is to read a xml file from ftp > convert that file content to jaxb object(as file content is xml data).
Place a debug breakpoint on Object->String component. Debug your application and determine exactly what you are asking to convert to String (mimeType=text/XML). Mule is interpreting something in the object given and somehow producing the question mark. Some people have seen three question marks before the XML doc declaration and also other character combinations. The 3 question marks represent a byte order mark (BOM) that's a special UNICODE character dealing with endian-ness and byte order. Your case may be related to the character encoding as well. Debug the flow and identify the incoming object. Understand the operation and it's parameters.