Mule 4 sftp read file - reconnection strategy not working - mule

Am trying to read a file from a SFTP location .
Am using Mule 4.4 Community edition.
If there is any error while connecting to SFTP server or file is not present , would like mule to retry 2 times .
So I configured in sftp connector : 'Reconnection strategy' as 'Standard'
However the logs do not show any retry occurring .
NOTE - ideally this will be kicked off at a scheduled time ( scheduler ) but for purposes of testing am using a HTTP Listener to invoke the flow
Here is the code :
<sftp:config name="SFTP_Config" doc:name="SFTP Config">
<sftp:connection host="abcd" username="xyz" password="pwd" />
</sftp:config>
<flow name="get:employee">
<logger level="INFO" doc:name="Logger" message="starting search" category="get-employee"/>
<sftp:read doc:name="Read" config-ref="SFTP_Config" path="/a/employees.unl">
<repeatable-in-memory-stream />
<reconnect />
</sftp:read>
<error-handler ></error-handler>
</flow>
Am wondering if I am doing something wrong ? I would want the flow or atleast the file reading to be attempted twice before erroring out.
Presently when the file does not exist in the sftp location it simply throws an error :
Message : Path '/a/employees.unl' doesn't exist
Error type : SFTP:ILLEGAL_PATH
when does the reconnection strategy kick in ?
Thanks

Reconnection strategies are for connections, not operations that fail. In your example the connection is working ok. The operation fails because the directory doesn't exists.
For an operations you should put the operation inside an <until-successful> scope. You can use the maxRetries attribute to specify the number of retries.
Documentation: https://docs.mulesoft.com/mule-runtime/4.4/until-successful-scope

Related

Mule requester in a MUnit Test case

I have to use a Mule Requester Inside MUnit test case to read a file under src/test/resources so that i can assert the payload . With the current implementation I have Ftp connector inside MUnits xml which I am referring from mule requester . I clearly see that mule requester is trying to access the connector even before it it initialized . Is there any way i can tell mule to early intialize ftp connector ?
Exception :
********************************************************************************
Message : Cannot perform an action on a connector when it is not started. Connector "no-delete-file-connector-test" is currently in lifecycle phase "initialise"
Element : /no-delete-file-connector-test # app:haz001-test-suite.xml:33 (File)
--------------------------------------------------------------------------------
Exception stack is:
Cannot perform an action on a connector when it is not started. Connector "no-delete-file-connector-test" is currently in lifecycle phase "initialise" (org.mule.api.lifecycle.LifecycleException)
org.mule.transport.AbstractConnector.getRequester(AbstractConnector.java:1178)
org.mule.transport.AbstractConnector.request(AbstractConnector.java:2269)
org.mule.transport.file.FileConnector$$EnhancerByCGLIB$$d6c65eff.CGLIB$request$75(<generated>:-1)
(97 more...)
(set debug level logging or '-Dmule.verbose.exceptions=true' for everything)
********************************************************************************
Following is my configuration :
<flow-ref name="processMainFlow" doc:name="processMainFlow" />
<mulerequester:request resource="file://src/test/resources/expected-discrete.csv?connector=no-delete-file-connector-test" doc:name="Mule Requester" returnClass="java.lang.String"/>
<munit:assert-payload-equals expectedValue="#[flowVars.outputCSV]" doc:name="Assert Payload"/>
<logger message="response for test #[payload] actual data : #[flowVars.outputCSV]" level="INFO"
doc:name="Logger" />
To load a file in your Munit test, you don't need the file connector, use a Set Payload or Set Message, eg:
<set-payload value="#[getResource('expected-discrete.csv').asStream()]" doc:name="Set Payload"/>
The path inside getResource is relative to testResources specified in your pom file (defaults to src/test/munit and src/test/resources).
The file connector is stopped because that is the default behaviour of connectors in Munit tests. There is more info about this in the Mulesoft forums: https://forums.mulesoft.com/questions/44649/munit-config-mock-inbounds-mock-connectors.html
Try to mock the component instead of calling it. If mocked it can avoid this error since FTP requires initiation which is not possible in munit.

How to setup Mule ESB SFTP listener

How do I setup an SFTP (ssh) listener on Mule ESB (CE)?
I could only find the HTTP(S) listener)
Thank you for any hints
There is no separate connector same connector can be used for username-password and publickey. Please refer preferredAuthenticationMethods attribute for more details. Configuration will be like
<sftp:connector name="SFTP" identityFile="ppkOrpemfile_path" preferredAuthenticationMethods="publickey" validateConnections="true" doc:name="SFTP"/>
<flow name="testSFTP_flow">
<sftp:inbound-endpoint connector-ref="SFTP" host="host" port="22" responseTimeout="10000" doc:name="SFTP"/>
</flow>
Hope this helps.
Try to find out wrapper.conf file from your server's conf folder.
Modify the port no
wrapper.java.additional.=-Dmule.mmc.bind.port=7779
MuleESB/ Mule Runtime is not SFTP server. All you can do is using mule SFTP connector pull and push the files (SFTP Client). If you are looking for SFTP server need to host. If your partner company have option to send data over HTTP(S) you could use mule HTTP Listener.

Mulesoft connection timeout issue

I'm getting a Connect timeout exception when I try to run a simple mule flow:
Message : Error sending HTTP request. Payload
: {NullPayload} Payload Type : org.mule.transport.NullPayload
Element : /getunitedflightsFlow/processors/0 #
getunitedflights:getunitedflights.xml:15 (Get Flights Requests)
Element XML :
Root Exception stack trace: java.net.ConnectException: Connect timeout
at
org.glassfish.grizzly.connectionpool.SingleEndpointPool$ConnectCompletionHandler.cancelled(SingleEndpointPool.java:1284)
at
org.glassfish.grizzly.impl.SafeFutureImpl.notifyCompletionHandlers(SafeFutureImpl.java:185)
at
org.glassfish.grizzly.impl.SafeFutureImpl.done(SafeFutureImpl.java:277)
at
org.glassfish.grizzly.impl.SafeFutureImpl$Sync.innerCancel(SafeFutureImpl.java:389)
at
org.glassfish.grizzly.impl.SafeFutureImpl.cancel(SafeFutureImpl.java:247)
at
org.glassfish.grizzly.connectionpool.SingleEndpointPool$ConnectTimeoutWorker.doWork(SingleEndpointPool.java:1352)
at
org.glassfish.grizzly.connectionpool.SingleEndpointPool$ConnectTimeoutWorker.doWork(SingleEndpointPool.java:1344)
at
org.glassfish.grizzly.utils.DelayedExecutor$DelayedRunnable.run(DelayedExecutor.java:158)
at
org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:591)
at
org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:571)
at java.lang.Thread.run(Thread.java:748)
Here is the Mule flow:
<flow name="getunitedflightsFlow">
<http:listener config-ref="HTTP_Listener_Configuration" path="/home" doc:name="HTTP"/>
<http:request config-ref="HTTP_Request_Configuration" path="/essentials/united/flights" method="GET" doc:name="Get Flights Requests"/>
</flow>
I did some googling and it looks like a proxy/firewall issue, I tried adding the proxy settings under preferences and all Native/Direct/Manual options won't work. I also added the following properties/args:
-XX:PermSize=128M -XX:MaxPermSize=256M -Dhttps.proxyHost=proxy.xxxx.com -Dhttps.proxyPort=8080 -Dhttps.proxyUsername=myfirst.mylast -Dhttps.proxyPassword=xxxxxxxxx
it doesn't work, also tried changing the proxy settings in the Http request connector itself but this doesn't work either.
Does anyone know what's going on here,
your help is appreciated!

Mule - session variables lost after sftp:outbound-endpoint

Running the following config, I set a session variable before an sftp outbound endpoint and after it its not available anymore. I thought session variables were meant to be available across transport barriers? Am I doing something wrong?
<set-session-variable variableName="total" value="#[payload.size()]" />
<foreach>
<!-- Omitted for brevity -->
</foreach>
<sftp:outbound-endpoint encoding="UTF-8"
connector-ref="sftp-out" exchange-pattern="request-response" outputPattern="#[flowVars.filename]"
host="${sftp.host}" port="${sftp.port}" path="${sftp.output.path}" user="${sftp.username}"
responseTimeout="60000" identityFile="${app.home}/${sftp.keyPath}" passphrase="${sftp.passphrase}" />
<logger level="INFO" message="#[sessionVars.total]" />
Being available through transport barrier means that are still available when sending the message through a VM outbound endpoint, inside the same server (same app or different apps).
This is useful for sharing behaviour between flows, use transactions, make asynchronous processing or distribute load in a cluster.
Regards.

How to trigger a Mule job from CLI (Command Line Interface)?

I have the below flow that will make a REST request on a periodic basis and then store the data into the PostgreSQL database.
<jdbc:postgresql-data-source name="PostgreSQL_Data_Source" user="postgres" password="*******" url="jdbc:postgresql://localhost:5432/TestDB" transactionIsolation="UNSPECIFIED" doc:name="PostgreSQL Data Source"/>
<jdbc:connector name="PostgreSQL_JDBC_Connector" dataSource-ref="PostgreSQL_Data_Source" validateConnections="true" queryTimeout="-1" pollingFrequency="0" doc:name="Database">
<jdbc:query key="InsertRecord" value="INSERT INTO "tblJSON"("StoreHere") VALUES (CAST(#[message.payload] AS json))"/>
</jdbc:connector>
<flow name="RESTServiceScheduling" doc:name="RESTServiceScheduling">
<!-- Step 1: Generates events at a given interval of time -->
<quartz:inbound-endpoint jobName="RESTServiceJobSchedule" repeatInterval="0" doc:name="Quartz" responseTimeout="10000" cronExpression="0 0 10 ? * *">
<quartz:event-generator-job/>
</quartz:inbound-endpoint>
<!-- Step 2: This will read the REST service data -->
<http:rest-service-component httpMethod="GET" serviceUrl="http://localhost:12186/RestServiceImpl.svc/StorageUsage" />
<!-- Step 3: Transform the HTTP-streamed payload into a java.lang.String -->
<object-to-string-transformer doc:name="Object to String"/>
<!-- Step 4: Dump into the destination Database -->
<jdbc:outbound-endpoint exchange-pattern="one-way" queryKey="InsertRecord" queryTimeout="-1" connector-ref="PostgreSQL_JDBC_Connector" doc:name="Destination"/>
</flow>
This works fine but I need a way from CLI (Command Line Interface) to trigger the job.
How can I do so?
Thanks in advance
Use an HTTP inbound endpoint to trigger the flow and call it with curl from the command line.
I know this is an old question with an accepted answer, but an alternative is to use a file endpoint that deletes the file. Set the file (or its endpoint) to have file age 1 and polling frequency 10 seconds. To trigger the flow, create a file with the right name. I found doing this, though, that Mule would not delete the file until the flow was done. It would then pick up the same file multiple times if the flow took longer than the polling period. To get around that, I have one flow that has just a file inbound endpoint, a logger, and an VM outbound endpoint with a specific path. Use a VM inbound endpoint with the same path as you would use the HTTP inbound endpoint above.
edit: You can then use touch or something similar in your CLI to create the file. I found this question looking for an alternative to the way described above.