Upload large file to JBoss Issue from Asp.net Client - file-upload

Question

All , I had a JBoss application which uses apache-common-fileupload component, I had tested it can be uploaded large file from a asp.net Client which use HttpWebRequest in post method. But after several times of successful upload.It failed with a exception below in JBoss console. Please help to check it .Thanks.
org.apache.commons.fileupload.FileUploadBase$IOFileUploadException: Processing of multipart/form-data request failed. Stream ended unexpectedly
at org.apache.commons.fileupload.FileUploadBase.parseRequest(FileUploadBase.java:359)
at com.accela.deploy.servlet.RequestHandler.getUploadFileFromRequest(RequestHandler.java:86)
at com.accela.deploy.servlet.DeployServlet.doPost(DeployServlet.java:122)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:710)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at com.accela.commom.filter.AuthFilter.doFilter(AuthFilter.java:99)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:96)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:182)
at org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:84)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at org.jboss.web.tomcat.service.jca.CachedConnectionValve.invoke(CachedConnectionValve.java:157)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:615)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
at org.apache.coyote.http11.Http11AprProcessor.process(Http11AprProcessor.java:877)
at org.apache.coyote.http11.Http11AprProtocol$Http11ConnectionHandler.process(Http11AprProtocol.java:594)
at org.apache.tomcat.util.net.AprEndpoint$Worker.run(AprEndpoint.java:1733)
at java.lang.Thread.run(Thread.java:722)
Here is the server.xml content. Please check it .
<!--APR library loader. Documentation at /docs/apr.html -->
<Listener className="org.apache.catalina.core.AprLifecycleListener" SSLEngine="on" />
<!--Initialize Jasper prior to webapps are loaded. Documentation at /docs/jasper-howto.html -->
<Listener className="org.apache.catalina.core.JasperListener" />
<Service name="jboss.web">
<!-- A HTTP/1.1 Connector (Http11Protocol Http11NioProtocol Http11AprProtocol)-->
<Connector protocol="org.apache.coyote.http11.Http11AprProtocol"
address="${jboss.bind.address}" port="${av.http.port}"
scheme="http" secure="false"
URIEncoding="UTF-8"
redirectPort="${av.https.port}"
maxHttpHeaderSize="8192"
maxThreads="${av.http.maxThreads:500}"
acceptCount="${av.http.acceptCount:200}"
enableLookups="false"
disableUploadTimeout="true"
/>
<!-- Java SSL Coyote HTTP/1.1 Connector using Apache Portable Runtime (APR) -->
<Connector protocol="org.apache.coyote.http11.Http11AprProtocol"
address="${jboss.bind.address}" port="${av.https.port}"
scheme="https" secure="true"
URIEncoding="UTF-8"
maxHttpHeaderSize="8192"
maxThreads="${av.https.maxThreads:500}"
acceptCount="${av.https.acceptCount:200}"
enableLookups="false"
disableUploadTimeout="true"
SSLEnabled="true"
SSLVerifyClient="none"
SSLProtocol="all"
SSLCipherSuite="!aNULL:!ADH:!eNULL:!LOW:!EXP:!SSLv2:RC4+RSA:+HIGH:+MEDIUM:ALL"
SSLCertificateFile="${jboss.server.home.dir}/conf/certs/${av.host}.crt"
SSLCertificateKeyFile="${jboss.server.home.dir}/conf/certs/${av.host}.key"
/>
<Engine name="jboss.web" defaultHost="localhost">
<Realm className="org.jboss.web.tomcat.security.JBossSecurityMgrRealm"
certificatePrincipal="org.jboss.security.auth.certs.SubjectDNMapping"
allRolesMode="authOnly"
/>
<Host name="localhost"
autoDeploy="false" deployOnStartup="false" deployXML="false"
configClass="org.jboss.web.tomcat.security.config.JBossContextConfig">
<Valve className="org.apache.catalina.valves.AccessLogValve"
prefix="av.web.${jboss.bind.address}.access."
suffix=".log"
pattern="%a %t %T "%r" %s %b %u %S %p"
directory="${jboss.server.log.dir}"
resolveHosts="false" />
<Valve className="org.jboss.web.tomcat.service.sso.ClusteredSingleSignOn" />
-->
<!-- Check for unclosed connections and transaction terminated checks
in servlets/jsps.
Important: The dependency on the CachedConnectionManager
in META-INF/jboss-service.xml must be uncommented, too
-->
<Valve className="org.jboss.web.tomcat.service.jca.CachedConnectionValve"
cachedConnectionManagerObjectName="jboss.jca:service=CachedConnectionManager"
transactionManagerObjectName="jboss:service=TransactionManager" />
</Host>
</Engine>
</Service>
Edited and Added Asp.net client code .
public static bool SendSingleChunk(string sRequestUrl,int iOffset, int iChunkSize)
{
string sPackFullUriName = "d:\\2GB.zip";
string boundary = "---------------------------" + DateTime.Now.Ticks.ToString("x");
byte[] boundarybytes = System.Text.Encoding.ASCII.GetBytes("--" + boundary + "\r\n");
string headerTemplate = "Content-Disposition: form-data; name=\"{0}\"; filename=\"{1}\"\r\n"
+ "Content-Type: application/octet-stream\r\n\r\n";
sFName = Path.GetFileName(sPackFullUriName);
string header = string.Format(headerTemplate, sFName, sPackFullUriName);
byte[] headerbytes = System.Text.Encoding.UTF8.GetBytes(header);
byte[] endBoundarybytes = System.Text.Encoding.ASCII.GetBytes("\r\n--" + boundary + "--\r\n");
Stream requestStream = null;
HttpWebRequest request = null;
byte[] bData = null;
try
{
bData = StorageHelper.ChunkReadFromDiskFile(sPackFullUriName, iOffset, iChunkSize);
Log.Write("This is the " + iChunkindex + " times to send chunk, size is " + bData.Length.ToString());
long lContentLen = bData.Length + boundarybytes.Length + headerbytes.Length + endBoundarybytes.Length;
string sContentType = "multipart/form-data; boundary=" + boundary;
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(sRequestUrl);
request.Method = WebRequestMethods.Http.Post;
request.KeepAlive = false;
request.Timeout = 300 * 60 * 1000; //300 mins;
request.ContentLength = lContentLen;
request.AllowAutoRedirect = false;
request.ContentType = sContentType;
using (requestStream = request.GetRequestStream())
{
requestStream.Write(boundarybytes, 0, boundarybytes.Length);
requestStream.Write(headerbytes, 0, headerbytes.Length);
if (bData != null)
{
requestStream.Write(bData, 0, bData.Length);
}
else
{
return false;
}
//requestStream.Write(formitembytes, 0, formitembytes.Length);
requestStream.Write(endBoundarybytes, 0, endBoundarybytes.Length);
requestStream.Flush();
requestStream.Close();
}
string returnedContent = string.Empty;
using (var response = (HttpWebResponse)request.GetResponse())
{
using (var responseStream = response.GetResponseStream())
{
using (StreamReader reader = new StreamReader(responseStream))
{
returnedContent = reader.ReadToEnd();
reader.Close();
}
}
response.Close();
if (DeployConfig.ResultSucceed.Equals("OK", StringComparison.InvariantCultureIgnoreCase))
{
return true;
}
else
{
return false;
}
}
}
catch (Exception ex)
{
return false;
}
finally
{
bData = null;
}
}
My idea:
I believe the root reason of the Stream ended unexpectedly exception is the length of the stream I post to server is not exactly equal to the length i specified in the HttpWebRequest.ContentLength.
My Questions:
1.What if the the length of the HttpWebRequest.ContentLength is larger or smaller than the actual size of I sent to server ?
2.What if I continue to execute SendSingleChunk after a failed post ? It seems the server recognized these two request streams as the same one.

Answer 1

The error message says:
.. Stream ended unexpectedly
This is most probably because the asp.net client application closes the socket unexpectedly after several successful uploads or may be there is a connection problem. Please see the below thread for a full discussion on the possible reasons for this error.
Why did I get "FileUploadException: Stream ended unexpectedly" with Apache Commons FileUpload?

Answer 2

All, For the upload large file issue , Here are some tips we should care about .
Whatever the webserver you used, actually in the process of uploading file from a certain of client to it, The first thing after receive data which it have to do is persist the data to some temporary directory. so, firstly make sure the application in the server have the enough privilege to write data to the disk.
Secondly. If we upload the file with underlying html protocol just like what I did.
We should check the protocol carefully, make sure every steps following the protocol. here is the doc for it .
The last thing I want to say is if you use the asp.net as a web client for some purpose, you must carefully check out what these property of HttpWebRequest use for. thanks.
request.KeepAlive = false;
request.Timeout = 300 * 60 * 1000; //300 mins;
request.ContentLength = lContentLen;
request.AllowAutoRedirect = false;