Mule object to string transformer adding one extra question mark at the start of string - mule

i am doing a Object to string conversion, but when i am doing it is adding one extra question mark at the very begin of string.
below is my flow code
<flow name="jatoSmartWriteFromFTP" doc:name="jatoSmartWriteFromFTP" processingStrategy="synchronous">
<ftp:inbound-endpoint host="delvmpllreap03.sapient.com" port="21" path="/home/jatopoc" user="jatopoc" password="jatopoc" responseTimeout="10000" doc:name="FTP" mimeType="text/xml">
<file:filename-wildcard-filter pattern="filter_data.xml"></file:filename-wildcard-filter>
</ftp:inbound-endpoint>
<logger message="#[message.payload]" level="INFO" doc:name="Logger"></logger>
<object-to-string-transformer doc:name="Object to String" mimeType="text/xml"/>
<mulexml:jaxb-xml-to-object-transformer name="XmlToPerson" jaxbContext-ref="jatoJaxbContext" returnClass="com.jato.speedwing.common.vo.JATOXML"/>
<component doc:name="Java">
<method-entry-point-resolver>
<include-entry-point method="insert"></include-entry-point>
</method-entry-point-resolver>
<spring-object bean="jatoDAO"> </spring-object>
</component>
<jms:outbound-endpoint topic="com.jato.smart.updateInfo" connector-ref="VM_Active_MQ" doc:name="JMS"/>
</flow>
below is my xml file content at ftp
<?xml version="1.0" encoding="UTF-8"?>
<code>
data
</code>
below is the string output after doing object-to-string transformer
?<?xml version="1.0" encoding="UTF-8"?>
<code>
data
</code>
now because of this extra question mark, my jaxb-xml-to-object transformer is not working..
can anybody suggest my what do here. since my task is to read a xml file from ftp > convert that file content to jaxb object(as file content is xml data).

Place a debug breakpoint on Object->String component. Debug your application and determine exactly what you are asking to convert to String (mimeType=text/XML). Mule is interpreting something in the object given and somehow producing the question mark. Some people have seen three question marks before the XML doc declaration and also other character combinations. The 3 question marks represent a byte order mark (BOM) that's a special UNICODE character dealing with endian-ness and byte order. Your case may be related to the character encoding as well. Debug the flow and identify the incoming object. Understand the operation and it's parameters.

Related

Iterate over a directory and extract only file names without reading the payload

I am using the Mule 4.4 community edition on premise.
Thanks to help, I have been able to read a large file without consuming memory and processing it, which is all good (here).
Now building on this further - my use case is to read all .csv files from within a directory.
And then process them one by one:
\opt\out\
students.csv
teachers.csv
collesges.csv
....
So my plan was to list the files in the directory:
<sftp:list doc:name="List" config-ref="SFTP_Config" directoryPath="/opt/out">
<non-repeatable-iterable />
<sftp:matcher filenamePattern="#['*.csv' ]"
directories="EXCLUDE" symLinks="EXCLUDE" />
</sftp:list>
And then I wanted to only read file names from directory and not read payload.
As per this early access article we are advised to use <non-repeatable-iterable />. However, after the list file operation as per article when I try to extract attributes:
<set-payload doc:name="Set Payload" value="#[output application/json --- payload map $.attributes]"/>
No attributes are available... (my plan is to extract the file names and then run a for loop for each file name and then a choice condition to determine if file name has student, use student transformer, if teacher use teacher transformer, etc.)
However, as attributes are not available, I am not able to pass file names to the for loop (yet to be written).
So I changed from <non-repeatable-iterable /> to <repeatable-in-memory-iterable />
Code below:
<sftp:list doc:name="List" config-ref="SFTP_Config" directoryPath="/opt/out">
<repeatable-in-memory-iterable />
<sftp:matcher filenamePattern="#['*.csv' ]"
directories="EXCLUDE" symLinks="EXCLUDE" />
</sftp:list>
Using the above, I can extract the attributes of file names.
I am confused about the following:
The files to be processed in the above directory will be large (each file 700 MB), so while iterating the directory by using repeatable-in-memory-iterable, will it cause any memory issues? (I do not want to read file content, simply get file names at this stage)
Here is the complete payload till now (note - it does not contain any for loop to iterate over files, which I will plug in...)
<flow name="employee-process-flow">
<http:listener doc:name="Listener" config-ref="HTTP_Listener_config" path="/processFiles"/>
<set-variable value='#[now() as String { format: "ddMMuu" }]' doc:name="Set todays date as ddmmyy" doc:id="c6a91a41-65b1-46df-a720-9c13fe360b6b" variableName="today"/>
<sftp:list doc:name="List" config-ref="SFTP_Config" directoryPath="/opt/out">
<repeatable-in-memory-iterable />
<sftp:matcher filenamePattern="#['*.csv' ]"
directories="EXCLUDE" symLinks="EXCLUDE" />
</sftp:list>
<set-payload doc:name="Set Payload" value="#[output application/json --- payload map $.attributes]"/>
<foreach doc:name="For Each" >
<logger level="INFO" doc:name="Logger" message="we are here"/>
</foreach>
</flow>
The List operation returns a list of messages, and each has a payload and attributes. The content of the files is returned as the payload, in a lazy way, meaning that the file's content is read only if you try to access that element's payload.
It makes sense that if you a non-repeatable-iterator and don't access the payload of each item in the <foreach> then you should not have any memory issues, because the contents are not read.
By using in memory repeatable streaming it is possible that the entire payload is being read into memory. Try reading a file a few gigabytes in size and see what happens there.
I'm not sure what the problem is with the attributes. It should work the same in any streaming mode.
Note that if you plan on doing something with the attributes—other than printing them—then you should output to application/java instead of JSON, to avoid unneeded conversions to and from JSON. For example, in your flow the output is used as input for the <foreach>, so it would be better for it to be Java.
Example:
output application/java --- payload map $.attributes

Adding a Header to a Response [duplicate]

I am very new to Mulesoft and an entry-level programmer. I have been trying to figure out how to implement Mule 4 custom policies. This is the online documentation: https://docs.mulesoft.com/api-manager/2.x/http-policy-transform#add-headers
Following the examples, I was successfully able to add request and response headers. My main goal is to add the request headers while using variables. I am trying MEL (https://docs.mulesoft.com/mule-runtime/3.7/mule-expression-language-examples) to try to call the variable names but it doesn't work. HOWEVER, whenever I try logging the variables it returns the correct value.
<?xml version="1.0" encoding="UTF-8"?>
<mule xmlns="http://www.mulesoft.org/schema/mule/core"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:http-policy="http://www.mulesoft.org/schema/mule/http-policy"
xmlns:http-transform="http://www.mulesoft.org/schema/mule/http-policy-transform"
xmlns:secure-properties="http://www.mulesoft.org/schema/mule/secure-properties"
xsi:schemaLocation="http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd
http://www.mulesoft.org/schema/mule/http-policy http://www.mulesoft.org/schema/mule/http-policy/current/mule-http-policy.xsd
http://www.mulesoft.org/schema/mule/http-policy-transform http://www.mulesoft.org/schema/mule/http-policy-transform/current/mule-http-policy-transform.xsd
http://www.mulesoft.org/schema/mule/secure-properties http://www.mulesoft.org/schema/mule/secure-properties/current/mule-secure-properties.xsd">
<http-policy:proxy name="{{{policyId}}}-custom-policy">
<http-policy:source propagateMessageTransformations="true">
<try>
<set-variable variableName="header" value="TEST_HEADER"/>
<logger level="INFO" message="#[vars.header]" />
<http-transform:add-headers outputType="request">
<http-transform:headers>#[{'TEST_HEADER': 'TEST'}]</http-transform:headers>
</http-transform:add-headers>
<http-policy:execute-next/>
<http-transform:add-headers outputType="response">
<http-transform:headers>#[{'Header_Added': '#[vars.header]'}]</http-transform:headers>
#this step doesn't work as I hoped it would
</http-transform:add-headers>
<logger level="INFO" message="#[vars.header]" />
#logging prints the value of header
</try>
</http-policy:source>
</http-policy:proxy>
</mule>
I am trying to add #[vars.header] as a request header name and possibly the value? Would I need to create another variable to have the header value? Can someone please guide me the right direction?
Thank you
Mule 4 uses dataweave expressions. Try removing the quotes so its not treated as a static string and remove the nested expression brackets '#[]':
<http-transform:headers>#[{'Header_Added': vars.header}]</http-transform:headers>
Or another tip, if you need to access a var in middle of a string you can use $() syntax:
<http-transform:headers>#[{'Header_Added': '$(vars.header)'}]</http-transform:headers>

Mule 4 Custom Policy Adding Request Headers Dynamically

I am very new to Mulesoft and an entry-level programmer. I have been trying to figure out how to implement Mule 4 custom policies. This is the online documentation: https://docs.mulesoft.com/api-manager/2.x/http-policy-transform#add-headers
Following the examples, I was successfully able to add request and response headers. My main goal is to add the request headers while using variables. I am trying MEL (https://docs.mulesoft.com/mule-runtime/3.7/mule-expression-language-examples) to try to call the variable names but it doesn't work. HOWEVER, whenever I try logging the variables it returns the correct value.
<?xml version="1.0" encoding="UTF-8"?>
<mule xmlns="http://www.mulesoft.org/schema/mule/core"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:http-policy="http://www.mulesoft.org/schema/mule/http-policy"
xmlns:http-transform="http://www.mulesoft.org/schema/mule/http-policy-transform"
xmlns:secure-properties="http://www.mulesoft.org/schema/mule/secure-properties"
xsi:schemaLocation="http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd
http://www.mulesoft.org/schema/mule/http-policy http://www.mulesoft.org/schema/mule/http-policy/current/mule-http-policy.xsd
http://www.mulesoft.org/schema/mule/http-policy-transform http://www.mulesoft.org/schema/mule/http-policy-transform/current/mule-http-policy-transform.xsd
http://www.mulesoft.org/schema/mule/secure-properties http://www.mulesoft.org/schema/mule/secure-properties/current/mule-secure-properties.xsd">
<http-policy:proxy name="{{{policyId}}}-custom-policy">
<http-policy:source propagateMessageTransformations="true">
<try>
<set-variable variableName="header" value="TEST_HEADER"/>
<logger level="INFO" message="#[vars.header]" />
<http-transform:add-headers outputType="request">
<http-transform:headers>#[{'TEST_HEADER': 'TEST'}]</http-transform:headers>
</http-transform:add-headers>
<http-policy:execute-next/>
<http-transform:add-headers outputType="response">
<http-transform:headers>#[{'Header_Added': '#[vars.header]'}]</http-transform:headers>
#this step doesn't work as I hoped it would
</http-transform:add-headers>
<logger level="INFO" message="#[vars.header]" />
#logging prints the value of header
</try>
</http-policy:source>
</http-policy:proxy>
</mule>
I am trying to add #[vars.header] as a request header name and possibly the value? Would I need to create another variable to have the header value? Can someone please guide me the right direction?
Thank you
Mule 4 uses dataweave expressions. Try removing the quotes so its not treated as a static string and remove the nested expression brackets '#[]':
<http-transform:headers>#[{'Header_Added': vars.header}]</http-transform:headers>
Or another tip, if you need to access a var in middle of a string you can use $() syntax:
<http-transform:headers>#[{'Header_Added': '$(vars.header)'}]</http-transform:headers>

Mule Server 3.6 > Anypoint Studio > Raw JSON as POST data

Still learning Mulesoft's Anypoint Studio... I am confused as how will I be able to access raw JSON POST data via the HTTP Listener then use the Choice flow control to execute conditions based on a value from a given JSON index. Anyone can show/tell me how to do this?
The JSON HTTP body will automatically become the payload of your message in Mule probably represented as Stream.
Just for demo purposes, try logging the payload after your http:listener using:
<object-to-string-transformer />
<logger level="INFO" message="#[payload]" />
There best way to query JSON is to transform it to a Map suing the JSON module transformers.
<json:json-to-object-transformer returnClass="java.util.HashMap" />
And then query it using MEL like standard MVEL or Java syntax.
For a JSON document like: {"person" : {"name" : "bob"}}
<logger message="#[payload.person.name]" level="INFO" />
You can use these expressions in your choic router also:
<choice>
<when expression="#[payload.person.name == 'bob']">
do something ...
</when>
</choice>

how to replace value using xpath3 expression in mule esb

<babyproducts>
<products>
<product1>
<productid>100</productid>
<productname>towel</productname>
<desc>towel</desc>
<discount>100</discount>
<validlity>10</validlity>
</product1>
<product2>
<productid>101</productid>
<productname>pillow</productname>
<desc>pillow</desc>
<discount>500</discount>
<validlity>5</validlity>
</product2>
</product>
</babyproducts>
I want to split above xml using xpath3 expression and replace the validlity value . after that i want whole set of above xml replaced values. I tried using xpath it is working fine. my requirement insists on doing using xpath3 mule expression
<splitter evaluator="xpath" expression="/babyproducts/products" doc:name="Splitter" />
<set-payload value="#[message.payload]" doc:name="Set Payload" />
<enricher source="#[flowVars.newvalidlity]" target="#[xpath('/products/validlity').text ]" doc:name="Message Enricher">
<logger level="INFO" message=" enricher done ~~~~~.." doc:name="Logger" />
</enricher>
above is my mule flow using xpath. my team wants to do using xpath3 (latest).
my expected outpu is like below with new validlity value 50 (comes from flow variable)
<babyproducts>
<products>
<product1>
<productid>100</productid>
<productname>towel</productname>
<desc>towel</desc>
<discount>100</discount>
<validlity>50</validlity>
</product1>
<product2>
<productid>101</productid>
<productname>pillow</productname>
<desc>pillow</desc>
<discount>500</discount>
<validlity>50</validlity>
</product2>
</product>
</babyproducts>
First of all your XML sample is invalid. There are unclosed tags and your xpath doesn't match the structure of the sample either.
Second of all I can only show the equivalent xpath3 expression of your xpath expression as I have already answered how to update an XML node in other questions of yours.
<splitter evaluator="xpath" expression="/babyproducts/products" doc:name="Splitter" />
Becomes:
<splitter expression="#[xpath3('/babyproducts/products/*', payload, 'NODESET')]"
doc:name="Splitter" />
And
#[xpath('/products/validlity').text]
Becomes:
#[xpath3('/products/validlity')]
No need for .text as by default returns the Stirng value. More info here: http://blogs.mulesoft.org/mule-3-6-xml-xpath-xslt-xquery3/