Getting "Connection closed before sending request out" in WSO2 ESB for newly created API - wso2-esb

Question

I'm getting "Connection closed before sending request out" in my WSO2 ESB API when a request is made to an endpoint. I am not using a proxy server.
Here is the source for the API that I have created:
<api xmlns="http://ws.apache.org/ns/synapse" name="WorkspaceONEUser" context="/workspaceone/user" port="8243">
<resource methods="POST" uri-template="*">
<inSequence>
<log>
<property name="Message Flow" value="POST WorkspaceONEUser Search - IN"/>
</log>
<send>
<endpoint key="workspaceOneUserSearchEndpoint"/>
</send>
</inSequence>
<outSequence>
<property name="Access-Control-Allow-Headers" value="authorization,Access-Control-Allow-Origin,Content-Type,X-Requested-With,Accept,Allow,Access-Control-Allow-Credentials" scope="transport"/>
<property name="Access-Control-Allow-Origin" value="HOST_NAME" scope="transport" type="STRING"/>
<property name="Access-Control-Allow-Credentials" value="true" scope="transport" type="STRING"/>
<log>
<property name="Message Flow" value="POST WorkspaceONEUser - OUT"/>
</log>
<log level="full"/>
<send/>
</outSequence>
<faultSequence>
<log>
<property name="message" value="in fault"/>
<property name="text" value="An unexpected error occured"/>
<property name="message" expression="get-property('ERROR_MESSAGE')"/>
<property name="header" expression="get-property('REQUEST_HOST_HEADER')"/>
</log>
</faultSequence>
</resource>
<resource methods="OPTIONS" url-mapping="/">
<inSequence>
<log>
<property name="Message Flow" value="IN OPTIONS"/>
</log>
<log level="full"/>
<property name="Access-Control-Allow-Headers" value="authorization,Access-Control-Allow-Origin,Content-Type,X-Requested-With,Accept,Allow,Access-Control-Allow-Credentials" scope="transport"/>
<property name="Access-Control-Allow-Origin" value="https://HOST_NAME" scope="transport" type="STRING"/>
<property name="Access-Control-Allow-Credentials" value="true" scope="transport" type="STRING"/>
<respond/>
</inSequence>
</resource>
</api>
In the logs, it appears that the connection is being closed before it is sent out:
TID: [-1234] [] [2019-01-28 13:38:29,283] DEBUG {org.apache.synapse.endpoints.AddressEndpoint} - Sending message through endpoint : workspaceOneUserSearchEndpoint resolving to address = https://TARGET_HOST {org.apache.synapse.endpoints.AddressEndpoint}
TID: [-1234] [] [2019-01-28 13:38:29,283] DEBUG {org.apache.synapse.endpoints.AddressEndpoint} - SOAPAction: null {org.apache.synapse.endpoints.AddressEndpoint}
TID: [-1234] [] [2019-01-28 13:38:29,283] DEBUG {org.apache.synapse.endpoints.AddressEndpoint} - WSA-Action: null {org.apache.synapse.endpoints.AddressEndpoint}
TID: [-1234] [] [2019-01-28 13:38:29,283] DEBUG {org.apache.synapse.core.axis2.Axis2FlexibleMEPClient} - Sending [add = false] [sec = false] [mtom = false] [swa = false] [format = null] [force soap11=false] [force soap12=false] [pox=false] [get=false] [encoding=null] [to=https://TARGET_HOST] {org.apache.synapse.core.axis2.Axis2FlexibleMEPClient}
TID: [-1234] [] [2019-01-28 13:38:29,283] DEBUG {org.apache.synapse.core.axis2.Axis2FlexibleMEPClient} - Message [Original Request Message ID : urn:uuid:09116122-22b4-4acf-be8e-31570623bd39] [New Cloned Request Message ID : urn:uuid:b242c096-4348-4977-8048-d70a5ac8c15b] {org.apache.synapse.core.axis2.Axis2FlexibleMEPClient}
TID: [-1234] [] [2019-01-28 13:38:29,283] DEBUG {org.apache.synapse.core.axis2.Axis2FlexibleMEPClient} - Setting Timeout for endpoint : Endpoint [workspaceOneUserSearchEndpoint], URI : https://TARGET_HOST to static timeout value : 120000 {org.apache.synapse.core.axis2.Axis2FlexibleMEPClient}
TID: [-1234] [] [2019-01-28 13:38:29,284] DEBUG {org.apache.synapse.core.axis2.SynapseCallbackReceiver} - Callback added. Total callbacks waiting for : 2 {org.apache.synapse.core.axis2.SynapseCallbackReceiver}
TID: [-1234] [] [2019-01-28 13:38:29,297] DEBUG {org.apache.axis2.transport.http.ApplicationXMLFormatter} - start writeTo() {org.apache.axis2.transport.http.ApplicationXMLFormatter}
TID: [-1234] [] [2019-01-28 13:38:29,297] DEBUG {org.apache.axis2.transport.http.ApplicationXMLFormatter} - end writeTo() {org.apache.axis2.transport.http.ApplicationXMLFormatter}
TID: [-1234] [] [2019-01-28 13:38:29,297] DEBUG {org.apache.synapse.mediators.builtin.SendMediator} - End : Send mediator {org.apache.synapse.mediators.builtin.SendMediator}
TID: [-1234] [] [2019-01-28 13:38:29,297] DEBUG {org.apache.synapse.mediators.base.SequenceMediator} - End : Sequence <anonymous> {org.apache.synapse.mediators.base.SequenceMediator}
TID: [-1] [] [2019-01-28 13:38:34,781] WARN {org.apache.synapse.transport.passthru.TargetHandler} - Connection closed before sending request out Remote Address : wsoidm01pa.jefferson.edu/147.140.23.161:443 {org.apache.synapse.transport.passthru.TargetHandler}
Does anyone know what might be causing this? I suspect that this indicates a connection between the ESB and a downstream server has closed before the ESB was able to completely write the request out.
However, I'm able to POST to the same endpoint from another API that's almost identical. Is there something that's obviously wrong with my API or endpoint that's causing this mysterious problem?

Answer 1

Try to add this property before your <send> mediator.
<property name="ClientApiNonBlocking"
value="true"
scope="axis2"
action="remove"/>
It will remove your Client Api Non Blocking in your <send> mediator (which is enable by default)