Mule file download with HTTP Endpoint - mule

Question

My HTTP Endpoint responsible for downloading the file at the end of the flow is erroring out. It keeps trying to communicate with http://:80/ instead of the URL passed in. What am I doing wrong here?
<?xml version="1.0" encoding="UTF-8"?>
<mule xmlns:scripting="http://www.mulesoft.org/schema/mule/scripting" xmlns:file="http://www.mulesoft.org/schema/mule/file" xmlns:http="http://www.mulesoft.org/schema/mule/http" xmlns:json="http://www.mulesoft.org/schema/mule/json" xmlns:https="http://www.mulesoft.org/schema/mule/https" xmlns="http://www.mulesoft.org/schema/mule/core" xmlns:doc="http://www.mulesoft.org/schema/mule/documentation" xmlns:spring="http://www.springframework.org/schema/beans" version="CE-3.3.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="
http://www.mulesoft.org/schema/mule/json http://www.mulesoft.org/schema/mule/json/current/mule-json.xsd
http://www.mulesoft.org/schema/mule/http http://www.mulesoft.org/schema/mule/http/current/mule-http.xsd
http://www.mulesoft.org/schema/mule/https http://www.mulesoft.org/schema/mule/https/current/mule-https.xsd
http://www.mulesoft.org/schema/mule/file http://www.mulesoft.org/schema/mule/file/current/mule-file.xsd
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-current.xsd
http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd
http://www.mulesoft.org/schema/mule/scripting http://www.mulesoft.org/schema/mule/scripting/current/mule-scripting.xsd ">
<flow name="BingFlow1" doc:name="BingFlow1">
<http:inbound-endpoint exchange-pattern="one-way" host="localhost" port="8081" doc:name="HTTP"/>
<https:outbound-endpoint exchange-pattern="request-response" host="api.datamarket.azure.com" port="443" path="Data.ashx/Bing/Search/v1/Web?Query=%27contract%20california%27&WebFileType=%27PDF%27&$top=50&$format=Json" user="*****" password="*****" doc:name="Bing"/>
<json:json-to-object-transformer returnClass="java.util.Map" doc:name="JSON to Object"/>
<expression-transformer expression="#[message.payload.d.results]" doc:name="Expression"/>
<collection-splitter doc:name="Collection Splitter"/>
<expression-transformer expression="#[org.mule.util.StringUtils.substringAfter(message.payload.Url, 'http://')]" doc:name="Expression"/>
<logger message="Payload is: #[message.payload]" level="INFO" doc:name="Logger"/>
<http:outbound-endpoint exchange-pattern="request-response" host="#[org.mule.util.StringUtils.substringBefore(message.payload, '/')]" port="80" method="GET" doc:name="HTTP" contentType="application/pdf" mimeType="application/pdf" path="#[org.mule.util.StringUtils.substringAfter(message.payload, '/')]"/>
<file:outbound-endpoint responseTimeout="10000" doc:name="File" outputPattern="#[org.mule.util.StringUtils.replace(message.payload, '/','.')]" path="/home/user/Documents/output" mimeType="application/pdf"/>
</flow>
</mule>
When changed to match the first answer, I get this exception:
********************************************************************************
Message : Failed to invoke REST service "http://santaclaraca.gov/modules/ShowDocument.aspx?documentid=108?followRedirects=true". Message payload is of type: LinkedHas
hMap
Code : MULE_ERROR--2
--------------------------------------------------------------------------------
Exception stack is:
1. Failed to invoke REST service "http://santaclaraca.gov/modules/ShowDocument.aspx?documentid=108?followRedirects=true". Message payload is of type: LinkedHashMap (org.mule.transp
ort.http.components.RestServiceException)
org.mule.transport.http.components.RestServiceWrapper:219 (http://www.mulesoft.org/docs/site/current3/apidocs/org/mule/transport/http/components/RestServiceException.html)
--------------------------------------------------------------------------------
Root Exception stack trace:
org.mule.transport.http.components.RestServiceException: Failed to invoke REST service "http://santaclaraca.gov/modules/ShowDocument.aspx?documentid=108?followRedirects=true". Mess
age payload is of type: LinkedHashMap
at org.mule.transport.http.components.RestServiceWrapper.doInvoke(RestServiceWrapper.java:219)
at org.mule.component.AbstractComponent.invokeInternal(AbstractComponent.java:126)
at org.mule.component.AbstractComponent.access$000(AbstractComponent.java:61)
+ 3 more (set debug level logging or '-Dmule.verbose.exceptions=true' for everything)
********************************************************************************
--- SOLVED ---
Big thanks to David. This is my final solution:
<?xml version="1.0" encoding="UTF-8"?>
<mule xmlns:vm="http://www.mulesoft.org/schema/mule/vm" xmlns:file="http://www.mulesoft.org/schema/mule/file"
xmlns:http="http://www.mulesoft.org/schema/mule/http" xmlns:json="http://www.mulesoft.org/schema/mule/json"
xmlns:https="http://www.mulesoft.org/schema/mule/https" xmlns="http://www.mulesoft.org/schema/mule/core"
xmlns:doc="http://www.mulesoft.org/schema/mule/documentation"
xmlns:spring="http://www.springframework.org/schema/beans" version="CE-3.3.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="
http://www.mulesoft.org/schema/mule/json http://www.mulesoft.org/schema/mule/json/current/mule-json.xsd
http://www.mulesoft.org/schema/mule/vm http://www.mulesoft.org/schema/mule/vm/current/mule-vm.xsd
http://www.mulesoft.org/schema/mule/http http://www.mulesoft.org/schema/mule/http/current/mule-http.xsd
http://www.mulesoft.org/schema/mule/https http://www.mulesoft.org/schema/mule/https/current/mule-https.xsd
http://www.mulesoft.org/schema/mule/file http://www.mulesoft.org/schema/mule/file/current/mule-file.xsd
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-current.xsd
http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd ">
<configuration doc:name="Configuration">
<expression-language>
<import class="org.mule.util.StringUtils" />
</expression-language>
</configuration>
<flow name="BingQuery" doc:name="BingQuery">
<http:inbound-endpoint exchange-pattern="one-way"
host="localhost" port="8082" doc:name="HTTP" />
<https:outbound-endpoint exchange-pattern="request-response"
host="api.datamarket.azure.com" port="443"
path="Data.ashx/Bing/Search/v1/Web?Query=%27california%20school%20district%20contract%27&WebFileType=%27PDF%27&$top=10&$format=Json"
user="*****" password="*****"
doc:name="Bing" />
<json:json-to-object-transformer
returnClass="java.util.Map" doc:name="JSON to Object" />
<expression-transformer expression="#[message.payload.d.results]"
doc:name="Expression" />
<collection-splitter doc:name="Collection Splitter" />
<vm:outbound-endpoint exchange-pattern="one-way"
doc:name="VM" path="fileWriter" />
</flow>
<flow name="RestProcessor" doc:name="RestProcessor">
<vm:inbound-endpoint exchange-pattern="one-way"
doc:name="VM" path="fileWriter" />
<set-variable variableName="fileName" value="#[message.payload.ID].pdf"
doc:name="Variable" />
<http:rest-service-component
serviceUrl="#[joinChar=message.payload.Url.contains('?')?'&':'?' ; StringUtils.join(new String[]{message.payload.Url,(String)joinChar,'followRedirects=true'})]"
httpMethod="GET">
<http:error-filter>
<expression-filter
expression="#[Integer.valueOf(message.inboundProperties['http.status']) >= 400]"></expression-filter>
</http:error-filter>
</http:rest-service-component>
<file:outbound-endpoint responseTimeout="10000"
doc:name="File" outputPattern="#[flowVars.fileName]" path="/home/ken/Documents/output"
mimeType="application/pdf" />
</flow>
</mule>

Answer 1

I can't spot the error: your flow looks good (except a problem I'll detail below) so, alternatively to using an http:outbound-endpoint, which can be finicky at times, you could use the http:rest-service-component.
Your next problem will be in the outputPattern of the file endpoint: remember at this point message.payload will be the response from the HTTP endpoint not the (partial) URL anymore. You need to pre-compute the file name and store it in a flow variable, then use it.
This would give:
<configuration>
<expression-language>
<import class="org.mule.util.StringUtils" />
</expression-language>
</configuration>
<flow name="BingFlow1" doc:name="BingFlow1">
<http:inbound-endpoint exchange-pattern="one-way" host="localhost" port="8081" doc:name="HTTP"/>
<https:outbound-endpoint exchange-pattern="request-response" host="api.datamarket.azure.com" port="443" path="Data.ashx/Bing/Search/v1/Web?Query=%27contract%20california%27&WebFileType=%27PDF%27&$top=50&$format=Json" user="*****" password="*****" doc:name="Bing"/>
<json:json-to-object-transformer returnClass="java.util.Map" doc:name="JSON to Object"/>
<expression-transformer expression="#[message.payload.d.results]" doc:name="Expression"/>
<collection-splitter doc:name="Collection Splitter"/>
<set-variable variableName="fileName" value="#[org.mule.util.StringUtils.substringAfter(message.payload.Url, 'http://').replace('/','.')]" />
<http:rest-service-component serviceUrl="#[joinChar=message.payload.Url.contains('?')?'&':'?' ; StringUtils.join(new String[]{message.payload.Url,(String)joinChar,'followRedirects=true'})]" httpMethod="GET">
<http:error-filter>
<expression-filter expression="#[Integer.valueOf(message.inboundProperties['http.status']) >= 400]" />
</http:error-filter>
</http:rest-service-component>
<file:outbound-endpoint responseTimeout="10000" doc:name="File" outputPattern="#[flowVars.fileName]" path="/home/user/Documents/output" mimeType="application/pdf"/>
</flow>