Mule Requester - Failed to move file - mule

Question

I am trying to move files with the help of MuleRequester. With large files (300MB>x) sometimes the while flow is blocking and I always get the error message below.
The file is locked by the java.exe process all the time.
With small files there is no problem.
The file remains in the folder "C:\itm\main\datafeeds\backup" and the "C:\itm\main\datafeeds\process\" is empty.
I don't have any idea how to fix this.
Error Message:
Message : Failed to move file "C:\itm\main\datafeeds\backup\my_file.xml" to "C:\itm\main\datafeeds\process\my_file.xml". The file might already exist.
Payload : [{numOfRecords=0}]
Payload Type : java.util.LinkedList
Endpoint : DefaultInboundEndpoint{endpointUri=file:///C:/itm/main/datafeeds/backup/?autoDelete=true, connector=FileConnector
{
name=File
lifecycle=start
this=300a38e7
numberOfConcurrentTransactedReceivers=4
createMultipleTransactedReceivers=true
connected=true
supportedProtocols=[file]
serviceOverrides=<none>
}
, name='endpoint..C.itm.main.datafeeds.backup', mep=ONE_WAY, properties={autoDelete=true}, transactionConfig=Transaction{factory=null, action=INDIFFERENT, timeout=0}, deleteUnacceptedMessages=false, initialState=started, responseTimeout=10000, endpointEncoding=UTF-8, disableTransportTransformer=false}
Timeout : 1000
Element : /poller_product_parse_aff_file/processors/0/0/1 # product_2_parse_aff_file_to_db
--------------------------------------------------------------------------------
Root Exception stack trace:
org.mule.api.DefaultMuleException: Failed to move file "C:\itm\main\datafeeds\backup\my_file.xml" to "C:\itm\main\datafeeds\process\my_file.xml". The file might already exist.
at org.mule.transport.file.FileMessageRequester.moveOrDelete(FileMessageRequester.java:222)
at org.mule.transport.file.FileMessageRequester.doRequest(FileMessageRequester.java:188)
at org.mule.transport.AbstractMessageRequester.request(AbstractMessageRequester.java:94)
at org.mule.transport.AbstractConnector.request(AbstractConnector.java:2259)
at org.mule.endpoint.DefaultInboundEndpoint.request(DefaultInboundEndpoint.java:80)
at org.mule.client.DefaultLocalMuleClient.request(DefaultLocalMuleClient.java:184)
at org.mule.module.MuleRequesterModule.request(MuleRequesterModule.java:64)
...
The mule-requester cfg:
<mulerequester:request resource="${file.RequesterUrl}" timeout="1000" doc:name="Mule Requester" />
Environment:
Operation System: Windows Server 2012R2
JRE: v1.7
Mule Requester: v1.5
Mule Anypoint Studio: Mule Server 3.8.0 CE

Answer 1

When working with such big files the problem mostly in memory available for java. Try to increase available (max) java heap size.

Answer 2

Thank you for your answer!
I've had such heap problems in the past and I guess that is not the problem in this case, because heap problems are throwing a specific error message.
My vm arguments are:
-Xms2048m
-Xmx4096m
-XX:MaxPermSize=2048m

Answer 3

I think you are copying the same file again to the same location.
org.mule.api.DefaultMuleException: Failed to move file "C:\itm\main\datafeeds\backup\my_file.xml" to "C:\itm\main\datafeeds\process\my_file.xml". The file might already exist.
Can You post your full xml config to help further.

Answer 4

Other alternative would be to use file outbound and write back the payload into output file. For just moving files there could be many other alternatives like batch/shell script etc.

Answer 5

Thank You for your help!
Here ist the whole flow:
<?xml version="1.0" encoding="UTF-8"?>
<mule xmlns:vm="http://www.mulesoft.org/schema/mule/vm" xmlns:mulerequester="http://www.mulesoft.org/schema/mule/mulerequester" xmlns:db="http://www.mulesoft.org/schema/mule/db" xmlns:metadata="http://www.mulesoft.org/schema/mule/metadata" xmlns:http="http://www.mulesoft.org/schema/mule/http" xmlns:wmq="http://www.mulesoft.org/schema/mule/ee/wmq" xmlns:tracking="http://www.mulesoft.org/schema/mule/ee/tracking"
xmlns:file="http://www.mulesoft.org/schema/mule/file" xmlns="http://www.mulesoft.org/schema/mule/core" xmlns:doc="http://www.mulesoft.org/schema/mule/documentation"
xmlns:spring="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-current.xsd
http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd
http://www.mulesoft.org/schema/mule/file http://www.mulesoft.org/schema/mule/file/current/mule-file.xsd
http://www.mulesoft.org/schema/mule/ee/tracking http://www.mulesoft.org/schema/mule/ee/tracking/current/mule-tracking-ee.xsd
http://www.mulesoft.org/schema/mule/http http://www.mulesoft.org/schema/mule/http/current/mule-http.xsd
http://www.mulesoft.org/schema/mule/ee/wmq http://www.mulesoft.org/schema/mule/ee/wmq/current/mule-wmq-ee.xsd
http://www.mulesoft.org/schema/mule/db http://www.mulesoft.org/schema/mule/db/current/mule-db.xsd
http://www.mulesoft.org/schema/mule/mulerequester http://www.mulesoft.org/schema/mule/mulerequester/current/mule-mulerequester.xsd
http://www.mulesoft.org/schema/mule/vm http://www.mulesoft.org/schema/mule/vm/current/mule-vm.xsd">
<file:connector name="File" autoDelete="true" streaming="false" validateConnections="false" doc:name="File" outputAppend="true" moveToDirectory="${file.workingDir}" />
<flow name="poller_product_parse_aff_file" processingStrategy="synchronous">
<poll doc:name="Poll">
<fixed-frequency-scheduler frequency="15" timeUnit="SECONDS"/>
<db:select config-ref="MYSQL_GENERIC_CFG" doc:name="Database">
<db:parameterized-query><![CDATA[SELECT COUNT(*) AS numOfRecords FROM mule_product]]></db:parameterized-query>
</db:select>
</poll>
<choice doc:name="Choice">
<when expression="#[payload.get(0).numOfRecords < 1000]">
<logger message="#[System.getProperty('line.separator')] ############################ Database is free, so start parsing next xml file ###########################:: #[flowVars.fileRequesterUrl]" level="TRACE" doc:name="Logger"/>
<mulerequester:request resource="${file.RequesterUrl}" timeout="30000" doc:name="Mule Requester"/>
<choice doc:name="Choice">
<when expression="#[payload != null && payload.size()>0]">
<logger message="#[System.getProperty('line.separator')] +++ DB is free and file was found" level="TRACE" doc:name="Logger"/>
<flow-ref name="product_parse_aff_file_to_dbFlow" doc:name="product_parse_aff_file_to_dbFlow"/>
</when>
<otherwise>
<logger message="#[System.getProperty('line.separator')] --- File not found or filesize is zero: #[payload]" level="TRACE" doc:name="Logger"/>
</otherwise>
</choice>
</when>
<when expression="">
<logger message="::::::::::::::: start of parsing the files:no fo records in db :::::>#[payload.get(0).numOfRecords]" level="TRACE" doc:name="Logger"/>
</when>
<otherwise>
<logger message="::::::::::file processing not started as the condition is not met i.e SELECT COUNT(*) AS numOfRecords FROM mule_product >1000" level="TRACE" doc:name="Logger"/>
</otherwise>
</choice>
</flow>
....
</mule>