WSO2EI 7.1.1 File Processing not working on centos server - apache

I am trying this tutorial from this given link https://ei.docs.wso2.com/en/latest/micro-integrator/use-cases/tutorials/file-processing/
Here is the file proxy definition
<?xml version="1.0" encoding="UTF-8"?>
<proxy name="daily-file-proxy" startOnLoad="true" transports="vfs" xmlns="http://ws.apache.org/ns/synapse">
<target>
<inSequence>
<log level="full"/>
<clone>
<target sequence="daily-file-file-write-sequence"/>
<target sequence="daily-file-mail-notification-sequence"/>
<target sequence="daily-file-db-sequence"/>
</clone>
</inSequence>
<outSequence/>
<faultSequence/>
</target>
<parameter name="transport.PollInterval">15</parameter>
<parameter name="transport.vfs.ActionAfterProcess">MOVE</parameter>
<parameter name="transport.vfs.MoveAfterProcess">$FILE:FILE_DAILY_PROCESS</parameter>
<parameter name="transport.vfs.FileURI">$FILE:FILE_DAILY_DOWNLOAD</parameter>
<parameter name="transport.vfs.MoveAfterFailure">$FILE:FILE_DAILY_FAILURE</parameter>
<parameter name="transport.vfs.FileNamePattern">.*.txt</parameter>
<parameter name="transport.vfs.ContentType">text/plain</parameter>
<parameter name="transport.vfs.ActionAfterFailure">MOVE</parameter>
</proxy>
I also declared all the parameters in the file.properties as well.
This code worked properly from Integration Studio as I drop the txt file in the FILE:FILE_DAILY_PROCESS, it processed in 15 sec and put the data into database.
Then I packaged the project as car file and deployed to micro integerator wso2 in the centos server.
I checked the log and everything deployed properly without error (wso2carbon.log and wso2-error.log not show any error)
I am thinking that the wso2ei could not see the folder declaring
Here is the parameters declaring in file.properties
FILE_DAILY_DOWNLOAD=file:///root/file-processing/daily-download/202107
FILE_DAILY_PROCESS=file:///root/file-processing/daily-process
FILE_DAILY_PROCESS_URI=vfs:file:///root/file-processing/daily-process
FILE_DAILY_FAILURE=file:///root/file-processing/daily-failure

finally, I have figured out the issue. The wso2 could not see the specific folder under ssh user root. It must be under /home/wso2 so the VFS listener only see the folder available under /home/wso2 folder
So use the update settings
FILE_DAILY_DOWNLOAD=file:///home/wso2/file-processing/daily-download/202107
FILE_DAILY_PROCESS=file:///home/wos2/file-processing/daily-process
FILE_DAILY_PROCESS_URI=vfs:file:///home/wso2/file-processing/daily-process
FILE_DAILY_FAILURE=file:///home/wso2/file-processing/daily-failure

Related

How to handle VFS proxy error in WSO2 EI 6.4?

In VFS proxy, How I can handle errors? For example in below cases:
If VFS is not able to connect to SFTP server due to different reasons(like wrong password, directory not exist etc). I am able to see Warning in console log but I want to send a mail or want to do some other stuff if connection is not possible.
If the folder does not exist where we have to move file after reading.
I am able to handle error during processing in FaultSequence but that thing does not work for proxy. I tried to set onError attribute as well but seems proxy does not support that attribute.
Please let me know if someone handle such errors or I am missing something.
<proxy name="FileProxy" startOnLoad="false" statistics="enable" trace="enable" transports="vfs" xmlns="http://ws.apache.org/ns/synapse">
<target faultSequence="Global_Error_Handler">
<inSequence>
<log level="custom">
<property name="status=" value="File received."/>
</log>
</inSequence>
<outSequence/>
</target>
<parameter name="transport.vfs.Streaming">true</parameter>
<parameter name="transport.PollInterval">10</parameter>
<parameter name="transport.vfs.FileURI">vfs:sftp://abc:***#127.0.0.1:22/test/ftp_in?transport.vfs.AvoidPermissionCheck=true</parameter>
<parameter name="transport.vfs.ContentType">text/plain</parameter>
<parameter name="transport.vfs.ActionAfterProcess">MOVE</parameter>
<parameter name="transport.vfs.MoveAfterFailure">vfs:sftp://abc:***#127.0.0.1:22/test/ftp_failure</parameter>
<parameter name="transport.vfs.ActionAfterFailure">MOVE</parameter>
<parameter name="transport.vfs.ReplyFileName">ReplyFileName.csv</parameter>
<parameter name="transport.vfs.FileNamePattern">.*.dat</parameter>
<parameter name="transport.vfs.MoveAfterProcess">file:///home/test/ftp_process</parameter>
</proxy>
Need to handle any error that occurs outside the <target> functionality of proxy for VFS.

The system cannot infer the transport information from the vfs:sftp://user:password#sftpserverhost/in URL

I am using the following code in my proxy to connect ftp server to read file from.
<proxy name="SFTPVFSProxy" transports="vfs" startOnLoad="true" xmlns="http://ws.apache.org/ns/synapse">
<target>
<inSequence>
<send/>
</inSequence>
</target>
<parameter name="transport.vfs.ActionAfterProcess">MOVE</parameter>
<parameter name="transport.PollInterval">15</parameter>
<parameter name="transport.vfs.MoveAfterProcess">vfs:sftp://username:password#ftpserver-host/outbox</parameter>
<parameter name="transport.vfs.FileURI">vfs:sftp://username:password#ftpserver-host/inbox</parameter>
<parameter name="transport.vfs.MoveAfterFailure">vfs:sftp://username:password#ftpserver-host/sent</parameter>
<parameter name="transport.vfs.FileNamePattern">.*.txt</parameter>
<parameter name="transport.vfs.ContentType">application/xml</parameter>
<parameter name="transport.vfs.ActionAfterFailure">MOVE</parameter>
</proxy>
But I am getting below error when I ran the above. VFS transport is eanbled in axis2.xml and I can see vfs transport in wso2 esb 4.9.0 admin console.
The system cannot infer the transport information from the vfs:sftp://user:password#sftpserverhost/in URL
SFTP uses SSH protocol to communicate with the server, so it always logs into the user's home directory and then it takes the path relative to the user's home directory.
You can add "sftpPathFromRoot" to WSO2 VFS transport URI as to get absolute path access as below.
<parameter name="transport.vfs.FileURI">sftp://[ username[: password]#] hostname[: port][ absolute-path]?sftpPathFromRoot=true</parameter>
Alternatively you can use symbolic link.You can find more details here.

How to process reply with InboundEndpoints in WSO2 ESB

Just starting out with InboundEndpoints in version 4.9.0 of the WSO2 ESB.
I would like to create an inbound endpoint that polls for incoming files, forwards this to a service and saves the service reply to a file.
I have read the available docs and examples and this is what I came up with.
<inboundEndpoint xmlns="http://ws.apache.org/ns/synapse"
name="testinbound"
sequence="TestIn"
onError="fault"
protocol="file"
suspend="false">
<parameters>
<parameter name="interval">1000</parameter>
<parameter name="coordination">true</parameter>
<parameter name="transport.vfs.ContentType">text/xml</parameter>
<parameter name="transport.vfs.LockReleaseSameNode">false</parameter>
<parameter name="transport.vfs.AutoLockRelease">false</parameter>
<parameter name="transport.vfs.ActionAfterFailure">DELETE</parameter>
<parameter name="transport.vfs.CreateFolder">true</parameter>
<parameter name="sequential">true</parameter>
<parameter name="transport.vfs.ActionAfterProcess">MOVE</parameter>
<parameter name="transport.vfs.FileURI">//C:/Test/ST/in</parameter>
<parameter name="transport.vfs.MoveAfterFailure">//C:/Test/ST/failure</parameter>
<parameter name="transport.vfs.DistributedLock">true</parameter>
<parameter name="transport.vfs.FileNamePattern">.*\.xml</parameter>
<parameter name="transport.vfs.Streaming">true</parameter>
<parameter name="transport.vfs.MoveAfterProcess">//C:/Test/ST/original</parameter>
<parameter name="transport.vfs.Locking">disable</parameter>
<parameter name="transport.vfs.FileSortAscending">true</parameter>
<parameter name="transport.vfs.FileSortAttribute">NONE</parameter>
<parameter name="transport.vfs.Build">false</parameter>
</parameters>
</inboundEndpoint>
where the "TestIn" sequence looks like:
<sequence xmlns="http://ws.apache.org/ns/synapse"
name="TestIn"
onError="fault"
statistics="enable">
<in>
<send>
<endpoint key="conf:/stock_quote_config/stock_quote_endpoint_reg"/>
</send>
</in>
</sequence>
Now I have a couple of questions about the behavior of this code, and how to achieve my goal.
How to get my hands on the reply of the service? I have tried to add an <out> mediator in the TestIn sequence. I got the following:
OutMediator Current message is a request - skipping child mediators
I also tried the transport.vfs.ReplyFileURI and transport.vfs.ReplyFileName parameters.
Strangely, these options are not available via the UI. I could specify them in the configuration source file, without any effect.
(These parameters are described in the documentation of the 4.9.0 version. See
https://docs.wso2.com/display/ESB490/VFS+Transport )
So how to save the reply? And how to redirect the reply?
I see that after the TestIn sequence the <main> sequence is executed by the ESB on the response path -- this is what I would like to redefine.
I get an error that the file could not be moved from the "in" to the "original" folder. (In fact, after execution the file is moved.)
[2015-11-20 13:36:55,145] ERROR - FilePollingConsumer Error moving
file : file:///c:/Test/ST/in/test.xml to //C:/Test/ST/original
org.apache.commons.vfs2.FileSystemException: Could not delete "file:///c:/Test/ST/in/test.xml".
at org.apache.commons.vfs2.provider.AbstractFileObject.deleteSelf(AbstractFileObject.java:849)
at org.apache.commons.vfs2.provider.AbstractFileObject.moveTo(AbstractFileObject.java:1139)
at org.wso2.carbon.inbound.endpoint.protocol.file.FilePollingConsumer.moveOrDeleteAfterProcessing(FilePollingConsumer.java:842)
at org.wso2.carbon.inbound.endpoint.protocol.file.FilePollingConsumer.directoryHandler(FilePollingConsumer.java:568)
at org.wso2.carbon.inbound.endpoint.protocol.file.FilePollingConsumer.poll(FilePollingConsumer.java:214)
at org.wso2.carbon.inbound.endpoint.protocol.file.FilePollingConsumer.execute(FilePollingConsumer.java:134)
at org.wso2.carbon.inbound.endpoint.protocol.file.FileTask.taskExecute(FileTask.java:45)
at org.wso2.carbon.inbound.endpoint.common.InboundTask.execute(InboundTask.java:44)
at org.wso2.carbon.mediation.ntask.NTaskAdapter.execute(NTaskAdapter.java:90)
at org.wso2.carbon.ntask.core.impl.TaskQuartzJobAdapter.execute(TaskQuartzJobAdapter.java:67)
at org.quartz.core.JobRunShell.run(JobRunShell.java:213)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
Caused by: org.apache.commons.vfs2.FileSystemException: Could not delete "c:\Test\ST\in\test.xml".
at org.apache.commons.vfs2.provider.local.LocalFile.doDelete(LocalFile.java:127)
at org.apache.commons.vfs2.provider.AbstractFileObject.deleteSelf(AbstractFileObject.java:838)
... 16 more
You can use Call mediator instead of send mediator. When the response is received, the mediation flow resumes from the next mediator in the sequence. So you can redirect or process the incoming response.

WSO2 ESB - Proxies do not work when Soap Tracing is turned off

I have downloaded and I am trying out the WSO2 ESB and setting up proxies for WCF services with wsHttpBindings. When I have Soap Tracing enabled everything works fine - when I turn it off I get a 500 internal server error every time. I am completely stuck with this and can't understand why this would be.
I have tried setting up both pass through proxies and WCF based proxies and I always have the exact same issue - with Soap Tracing enabled everything looks great and works perfectly. If I turn it off and otherwise don't touch anything I get a 500 error:
System.ServiceModel.ProtocolException: An HTTP Content-Type header is required for SOAP messaging and none was found. ---> System.Net.WebException: The remote server returned an error: (500) Internal Server Error.
I have tried it with a basicHttpBinding and it seems to work fine with Soap Tracing on or off. The problem seems to just be with wsHttpBindings.
Here is the full source for the proxy:
<?xml version="1.0" encoding="UTF-8"?>
<proxy xmlns="http://ws.apache.org/ns/synapse"
name="ContactService"
transports="http"
statistics="disable"
trace="disable"
startOnLoad="true">
<target>
<outSequence>
<send/>
</outSequence>
<endpoint>
<wsdl service="ContactService"
port="ContactService_WsHttp"
uri="http://localhost/Actions/ContactService.svc?singleWsdl"/>
</endpoint>
</target>
<publishWSDL uri="http://localhost/Actions/ContactService.svc?singleWsdl"/>
<description/>
</proxy>
can you please try with in the inSequence of the proxy that your are using?
thank you,
1G
I am having the same issue, here is the proxy that is having issues:
<proxy name="cx.partner.usps.RateV4"
transports="https http"
startOnLoad="true"
trace="disable">
<description/>
<target>
<inSequence>
<property name="CONTENT_TYPE"
value="application/xml"
scope="axis2-client"
type="STRING"/>
<property name="FORCE_HTTP_1.0" value="true" scope="axis2"/>
<log level="full"/>
<send>
<endpoint key="partner.usps.prd.endpoint"/>
</send>
</inSequence>
<outSequence>
<class name="cx.wso2.mediators.addNamespace">
<property name="soapPartName" value=""/>
<property name="nsToAdd" value="http://xmlns.usps.com"/>
</class>
<send/>
</outSequence>
</target>
<publishWSDL key="conf:/esb-resources/wsdl/usps/RateV4.wsdl">
<resource location="RateV4.xsd" key="conf:/esb-resources/schema/usps/RateV4.xsd"/>
</publishWSDL>
This was working without tracing on 3.x, but after migrating to 4.7 I have to enable the SOAP tracer.
Try this, it worked for us:
<parameter name="enforceWSAddressing">true</parameter>

Can Axis Client-Config.wsdd config be picked up from JBoss?

I have a client for a 3rd party webservice that had to be created using the Azis WSDL2Java tool. Every web request must be encrypted using an Encryption handler provided by the 3rd party and the incoming responses myst be Decrypted in the same format. This all works really well, except that I have the handlers specified in a client-config.wsdd in the src/main/resources of my jar. one of the parameters of the client-config is a username and one is the location to a key file. This will need to be deployed to many different environments where the user and key location will differ, and it's not going to be feasible to change the values each time then rebuild. Is there a way I can either pass in parameters to my wsdd or place the wsdd in the $JBOSS_HOME/server//conf folder etc? (I am using JBoss 5.1.0.GA)
The wsdd looks like this
<?xml version='1.0'?>
<deployment xmlns="http://xml.apache.org/axis/wsdd/" xmlns:java="http://xml.apache.org/axis/wsdd/providers/java">
<globalConfiguration>
<requestFlow>
<handler name="EncryptionHandler" type="java:com.3rdparty.services.util.EncryptionHandler" >
<parameter name="userName" value="myuser"/>
<parameter name="keyFile" value="/jboss-5.1.0.GA/server/<name>/conf/my.key"/>
<parameter name="passwordCallbackClass" value="com.3rdparty.security.client.PWCallback"/>
</handler>
</requestFlow>
<responseFlow>
<handler name="DecryptionHandler" type="java:com.3rdparty.services.util.DecryptionHandler" >
<parameter name="userName" value="myuser"/>
<parameter name="keyFile" value="/jboss-5.1.0.GA/server/<name>/conf/my.key"/>
</handler>
</responseFlow>
</globalConfiguration>
<transport name="http" pivot="java:org.apache.axis.transport.http.HTTPSender"/>
<transport name="local" pivot="java:org.apache.axis.transport.local.LocalSender"/>
<transport name="java" pivot="java:org.apache.axis.transport.java.JavaSender"/>
</deployment>
You can use the system property axis.ClientConfigFile to specify a particular directory of the client-config.wsdd file.