How to use Move to Pattern & File Age property of File EndPoint in MULE.
I tried to give moveToPattern="#[function:datestamp]-#[message.inboundProperties['originalFilename']]" but it is not working as expected.
For File Age, I provided 50000. According to my understanding, if the last modified date of the file is 22.01.2015:20:07:20, then the file should be moved from this folder to another folder at 22.01.2015:20:12:20. But it is not happening. Please explain by giving an example.
What is the difference between connector reference & endpoint reference.
The 'file age' property defines the time a file must wait before it's processed. Once it's processed, it is going to move to the directory specified in the 'moveToDirectory'.
The file connector is a global connector where you can specify some properties that can be applied to all of your file endpoint configuration.
<file:connector name="File" autoDelete="true"
outputAppend="true" streaming="true" validateConnections="true"
doc:name="File" />
A file endpoint is a generic endpoint that can be referred from all your endpoints. For example, you have different endpoints with the same configuration, thus you can specify all the properties on a global endpoint, so when you need to change something, you only change the global one.
<file:endpoint name="fileEndpoint"
path="${file.path}" outputPattern="${file.outputPattern}" moveToDirectory="${file.moveToDir}"
connector-ref="File" doc:name="File" fileAge="5000"/>
The file inbound/outbound endpoints can then refer the global endpoints.
<file:outbound-endpoint responseTimeout="10000"
ref="fileEndpoint" doc:name="File - Log" />
The documentation for fileAge in the file transport reference is:
Setting this value (minimum age in milliseconds for a file to be processed) is useful when consuming large files, as Mule waits before reading this file until the file last modification timestamp indicates that the file is older than this value
In your case 50000 milliseconds would be 50 seconds, thus a file dropped at 22.01.2015:20:07:20 should be picked as soon as 22.01.2015:20:08:10.
Since it is not happening, there must be something wrong in your configuration. Please share your file transport configuration (if any) and your full endpoint configuration.
Related
we have a flow where we have property file reference as given below
"context:property-placeholder location="httpdemo.${country}.properties"
now we want ${country} value to be replaced by actual value at the time of deployment.
As we know one way to achieve it is setting the value of country as environment variable on the ESB and deploying it. But we don’t want to do that because of below reasons:
We deploy same code base for multiple countries in parallel
Environment properties can be set only during start of mule runtime so if I set env variable as country=UK and have deployed for UK. Later I want to deploy for MY again I need to restart ESB by setting country=MY which we don’t want to do.
Please let me know if there is any other better way
We had a similar situation where we needed to have multiple versions of the same application running parallely. The solution we used for this was to package the property file along with the build and not have the dynamic element (environment based) to it. For eg; in this case we construct httpdemo.usa.properties and packaged it along with the app. This was fairly easy for us since we use Jenkins to manage our builds and releases. When the build is released, we reference a configuration file from Jenkins which contains all the specific "country" related properties. You can even pass this country as a parameter to the build definition. Using a custom maven plugin we replace the property file within the app with the new properties from the Jenkins property file.
Another solution to your problem could be to follow a naming convention to your apps specific to the "country" you want to use and fetch the properties using spring beans. For eg;
<spring:beans>
<spring:bean id="CountryProperties" class="org.springframework.beans.factory.config.PropertiesFactoryBean">
<spring:property name="singleton" value="true"/>
<spring:property name="location" value="${app.name}.properties"/>
</spring:bean>
</spring:beans>
<http:listener-config name="HTTP_Listener_Configuration" host="localhost" port="5000" doc:name="HTTP Listener Configuration"/>
<flow name="dynamic_propsFlow">
<http:listener config-ref="HTTP_Listener_Configuration" path="/test" doc:name="HTTP"/>
<logger message="#[app.registry.CountryProperties['country.full.name']]" level="INFO" doc:name="Logger"/>
</flow>
My properties are like below:
File - dynamic_props_usa.properties
population=10
country.full.name=united.states.of.america
File - dynamic_props_mexico.properties
population=100
country.full.name=mexico
${app.name} gives you the name of the app that is deployed. In case my app is named dynamic_props_usa, it refers the dynamic_props_usa.properties. If it is, dynamic_props_mexico, it refers dynamic_props_mexico.properties. Hope this helps!
Define context property placeholder to have a file reference in the package and have an option to override the same with server if needed at runtime as below,
<context:property-placeholder location="classpath:app-${mule.env}.properties, file:${mule.config.path}/app-${mule.env}.properties" ignore-resource-not-found="true" ignore-unresolvable="true" />
You can run your on premise Mule server with the option -M-Dmule.country=your-value
You may want to take a look at the documentation
I have a flow with file endpoint and groovy.
I want to make the file path configerable.
Please suggest.
I am using file node to trigger the flow. So if anybody can suggest me that how to trigger groovy alone then there will be no need to use file endpoint.
Please suggest any solution.
What you want to do here is configure a file endpoint like so and use this as your inbound endpoint to trigger the flow:
<file:endpoint name="inputFile" path="${input.path}">
<!-- Add any filters (e.g. regex) here -->
</file:endpoint>
Then, you should create a mule-app.properties file inside of src/main/resources that will include something like the following:
# Input Properties
input.path=/path/to/file
Lastly, include the following at the top of your Mule config file (after <mule> but before your flow):
<spring:beans>
<context:property-placeholder location="classpath:mule-app.properties" />
</spring:beans>
Doing this should allow you to achieve what you've outlined above. Hope this helps!
Using Mule 3.2.1, we have encountered a strange problem where we use the SFTP component to grab pgp encrypted report files from a server. We archive the file using the SFTP component and then use a file:endpoint to produce a working copy for further processing.
Our problem is that every once in a while, the working copy of the file ends up corrupted, but the SFTP archived file is good. When looking at the corrupted file in a hex editor, we see good bytes and then all of a sudden, we see null bytes for the remainder of the file. It looks like the underlying file got deleted while Mule was copying.
An additional confusing piece of information is that we have tried downloading a failed file again, and had everything work. This leads me to believe that it is not something in the file that is the problem, but apparently we do have one file that consistently seems to fail. All this stuff is occurring on production servers with files that I have no access to.
Without knowing the inner workings of Mule, I have no idea what conditions could create this problem.
Are there any smart folks out there familiar enough with the inner workings of Mule to venture a guess?
Also, we are not Mule experts and would welcome any critique of our Mule configuration. (BTW, the config below is a modified version of what is in production and polls more frequently, etc)
<sftp:connector name="SftpConnector" validateConnections="true" autoDelete="true">
<file:expression-filename-parser />
</sftp:connector>
<file:connector name="FileConnector" pollingFrequency="1000" fileAge="1000" streaming="false"
autoDelete="false">
<service-overrides messageFactory="org.mule.transport.file.FileMuleMessageFactory" />
<file:expression-filename-parser />
</file:connector>
<sftp:endpoint name="SftpEndpoint" connector-ref="SftpConnector" host="localhost"
port="22" user="tdr" password="password" path="/opt/tdr/outbound" archiveDir="/home/cps/mule/sftp-archive"
responseTimeout="30000" sizeCheckWaitTime="2500" disableTransportTransformer="true">
<file:filename-wildcard-filter pattern="*.pgp,*.gpg" />
</sftp:endpoint>
<file:endpoint name="FileEndpoint" connector-ref="FileConnector" path="/home/cps/mule/input" />
<flow name="DfrFileGrabber">
<quartz:inbound-endpoint jobName="ptDfrGrabber" cronExpression="0/2 * * * * ?">
<quartz:endpoint-polling-job>
<quartz:job-endpoint ref="SftpEndpoint" />
</quartz:endpoint-polling-job>
</quartz:inbound-endpoint>
<file:outbound-endpoint ref="FileEndpoint" outputPattern="#[header:originalFilename]" />
</flow>
I think we got to the bottom of this one. I can not guarantee that my test case duplicates the problem we are seeing in production, but I believe that the root of the problem is that if the file to be copied is sufficiently large, it is possible for the quartz timer fire up before a previous sftp copy was complete causing multiple copies to occur using the same file name.
One solution is to include the tempDir attribute on the SFTP connection. This results in the SFTP connection moving the file that is being retrieved into the tempDir directory on the server while the copy is taking place. Thus, if the quartz timer fires before the first copy is complete, it does not find the same file.
Is there a way to simply "move/rename" a file in Mule as opposed to a copy? The flow below performs a copy, but what I really want is to simply rename the file. The file can be quite large.
<file:connector name="Global-StageToInput" fileAge="10000" autoDelete="true" pollingFrequency="30000"/>
<file:connector name="Global-FileInput" outputPattern="#[header:originalFilename]" />
<flow name="Global-MoveInputFiles">
<file:inbound-endpoint connector-ref="Global-StageToInput" path="${stage}"/>
<file:outbound-endpoint connector-ref="Global-FileInput" path="${input}"/>
</flow>
When you want to move a file from one directory to another the file aparently vanishes from the directory you are moving .
Back up of this file can be taken using the moveToDirectory in Inbound endpoint
In order to rename the backup file you can use moveToPattern in Inbound endpoint
In order to rename in the destination you can use outputPattern in Outbound endpoint
You can use the following links to have more ideas
<file:inbound-endpoint connector-ref="input" path="/tmp/input" moveToDirectory="/tmp/backup" moveToPattern="#[header:originalFilename].backup"/>
<file:outbound-endpoint connector-ref="output" path="/tmp/output" outputPattern="#[function:datestamp]-#[header:originalFilename]"/>
http://www.mulesoft.org/documentation/display/current/File+Transport+Reference
http://www.mulesoft.org/documentation-3.2/display/32X/File+Transport+Reference
http://www.mulesoft.org/connectors/file-connector
If you need only to move a file from a folder to another where you need to rename the file name.
Then use the following code
<file:inbound-endpoint connector-ref="input" path="/tmp/input"/>
<file:outbound-endpoint connector-ref="output" path="/tmp/output" outputPattern="#[function:datestamp]-#[header:originalFilename]"/>
Just only the pattern in the outbound endpoint.
Only the path in the input and output is required .Rest all is not necessary and with mix and match of the option that is there links can be used. Even the connector-ref is not required for a basic requirement
Using Mule 3, how can I pass a java.io.File object payload from an inbound file endpoint to a Groovy script rather than the file's content?
You need to turn streaming off and override the default message factory on the file connector:
<file:connector name="fileConnector" streaming="false" autoDelete="false">
<service-overrides messageFactory="org.mule.transport.file.FileMuleMessageFactory" />
</file:connector>
Note that it is up to you to move / delete the file either before you start or once you've done processing it, otherwise Mule will poll it again and again.