xmlHttp to download file - vba

Question

Using google chrome i fill in a form and upon pressing the submit button a spreadhsheet automatically starts downloading. I have identified the POST sent to the server but somehow when i send the same request from my script the response is not a file but the original web form. Please see below the request as shown in Chrome :
Remote Address: ****** -- Request URL:* --
Request Method: POST -- Status Code: 200 OK --**
Request Headers
-Accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
-Accept-Encoding:gzip,deflate,sdch
-Accept-Language:en-GB,en-US;q=0.8,en;q=0.6
-Cache-Control:max-age=0
-Connection:keep-alive
-Content-Length:207
-Content-Type:application/x-www-form-urlencoded
-Cookie:JSESSIONID=*******; ssoLang=en
-Host:*******
-Origin:******
-Referer:*******
-User-Agent:Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.114 Safari/537.36
Query String Parameters
Form Data : REQUEST
Response Headers
-Content-Disposition: filename="xxx.xlsx"
-Content-Type:application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
-Date:Sat, 31 May 2014 14:28:25 GMT
-Server:Apache-Coyote/1.1
-Transfer-Encoding:chunked
The response headers I am getting are different :
-Cache-Control: no-cache, no-store
-Date: Sat, 31 May 2014 14:55:38 GMT
-Pragma: no-cache Transfer-Encoding: chunked
-Content-Type: text/html;charset=UTF-8 Expires: Thu, 01 Jan 1970 00:00:00 GMT
-Server: Apache-Coyote/1.1
The script i attempted to do the above is below, can anybody please point out what is wrong here ?
Sub Download_File()
Dim localFile As String, URL As String, request As String
localFile = ThisWorkbook.Path & "\test.xlsx"
URL = ****SENSITIVE***
Dim XMLreq As Object
Set XMLreq = CreateObject("MSXML2.XMLHTTP")
request = ****REQUEST*****
Dim fileNum As Integer, bytes() As Byte
With XMLreq
.Open "POST", URL, False
.setRequestHeader "Host", ****SENSITIVE***
.setRequestHeader "Connection", "keep-alive"
.setRequestHeader "Content-Length", "207"
.setRequestHeader "Cache-Control", "max-age=0"
.setRequestHeader "Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8"
.setRequestHeader "Origin", ****SENSITIVE***
.setRequestHeader "User-Agent", "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.114 Safari/537.36"
.setRequestHeader "Content-Type", "application/x-www-form-urlencoded"
.setRequestHeader "Referer", ****SENSITIVE***
.setRequestHeader "Accept-Encoding", "gzip,deflate,sdch"
.setRequestHeader "Accept-Language", "en-GB,en-US;q=0.8,en;q=0.6"
.setRequestHeader "Cookie", "JSESSIONID=****SENSITIVE***; ssoLang=en"
.send (Escape(request))
While .readyState <> 4: DoEvents: Wend
If .Status = 200 Then
Debug.Print .getAllResponseHeaders
fileNum = FreeFile
Open localFile For Binary Access Write As #fileNum
bytes = .responseBody
Put #fileNum, , bytes
Close #fileNum
Else
MsgBox "XMLhttp POST error " & .statusText & vbCrLf & "Status = " & .Status & vbCrLf & "URL = " & URL
End If
End With
End Sub
Private Function Escape(ByVal URL As String) As String
Dim i As Integer, BadChars As String
BadChars = "<>%=&!##£$^()+{[}]|\;:'"",/?"
For i = 1 To Len(BadChars)
URL = Replace(URL, Mid(BadChars, i, 1), "%" & Hex(Asc(Mid(BadChars, i, 1))))
Next i
URL = Replace(URL, " ", "+")
Escape = URL
End Function

Answer 1

This is not a complete answer but more like a comment. I do not have enough reputation to leave a comment.
The response looks like you are redirected to the web form when some checks or authentication fails. Did you check the JSESSIONID cookie? That should be different for each time you logon to the server. Once you manually login and then capture the JSESSIONID cookie (either using Fiddler as Alex mentioned) or by typing javascript:alert(document.cookie); in the browser address bar and pressing enter, you might be able to reuse the same session token.
Also check the content length to ensure that the url-encoded content length matches this value. You can use the composer feature in Fiddler to test POST queries to the server to get the request parameters correct.
BTW, Fiddler installation folder can be copied to a USB stick and used as a portable installation. You might try that as well.
Edit: Looking further, the issue seems to be that the site requires a correct Referer header for sending out the excel file. As unsafe headers like Referer cannot be set for XMLHttp request, this might cause server side authentication failure and it might send you the initial landing page which is the web form here. Check this answer here