I want to run code when a HTTP client has finished downloading a file from Ktor Server. The most simple approach does not work:
routing {
get("/download") {
call.response.header(
ContentDisposition,
Attachment.withParameter(FileName, file.fileName).toString()
)
call.respondFile(file)
// client has not finished downloaded when we reach this point
}
}
I already tried intercepting pipelines, writing out the file in byte arrays myself, but all that detects a completed download too soon. If I for example stop the webserver and related services too soon, the client isn't able to actually complete the download.
So is there a way to reliably detect when the client has ACKed the last bytes of the file?
Related
I was making a simple file upload&download service and found out that, as far as I understand, Netty doesn't release direct buffers until request processing is over. As a result, I can't upload bigger files.
I was trying to make sure that the problem is not inside my code, so I created the most simple tiny Ktor application:
routing {
post("upload") {
call.receiveMultipart().forEachPart {}
call.respond(HttpStatusCode.OK)
}
}
The default direct memory size is about 3Gb, to make test simpler I limit it with:
System.setProperty("io.netty.maxDirectMemory", (10 * 1024 * 1024).toString())
before starting the NettyApplicationEngine.
Now if I upload a large file, for example with httpie, I got "Connection reset":
http -v --form POST http://localhost:42195/upload file#/tmp/FileStorageLoadTest-test-data1.tmp
http: error: ConnectionError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer')) while doing POST request to URL: http://localhost:42195/upload
On the server side there is no information about the problem except for the "java.io.IOException: Broken delimiter occurred" exception. But if I put the breakpoint in NettyResponsePipeline#processCallFailed, the real exception is:
io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 65536 byte(s) of direct memory (used: 10420231, max: 10485760)
It is a pity that this exception is not logged.
Also, I found out that the same code works without problems if I use Jetty engine instead.
Environment:
Ubuntu Linux
Java 8
Ktor=1.2.5
netty-transport-native-epoll=4.1.43.Final
(but if Netty started without native-epoll support, the problem is the same)
I made a simple spring-boot application that returns a static json response for all requests.
When the app gets a request with a large payload (~5mb json, 1 TP ), the client receives the following error:
java.net.SocketException: Broken pipe (Write failed)
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:111)
at java.net.SocketOutputStream.write(SocketOutputStream.java:155)
I have tried increasing every limit i could - here are my tomcat settings:
spring.http.multipart.max-file-size=524288000
spring.http.multipart.max-request-size=524288000
spring.http.multipart.enabled=true
server.max-http-post-size=10000000
server.connection-timeout=30000
server.tomcat.max-connections=15000
server.tomcat.max-http-post-size=524288000
server.tomcat.accept-count=10000
server.tomcat.max-threads=200
server.tomcat.min-spare-threads=200
What can I do to make this simple spring boot with just one controller, to handle such payloads successfully?
This springboot application and the client sending the large payload run on an 8-core machine with 16gb ram. So resources shouldn't be a problem.
This was because the controller was returning a response without consuming the request body.
So the server closes the connection as soon as it receives the request, without consuming the full request body. The client still hadn't finished sending the request and the server closed the connection before that.
Solution:
1. Read the full request body in your code
2. Set tomcat's maxSwallowSize to a higher value (default : 2mb)
server.tomcat.max-swallow-size=10MB
My application servers .wav files, which are downloadable at certain URLs. I have to change the logic so that they will be streamed instead of being downloaded - so I will remove Content-Disposition header that was being explicitly set.
Piece of code:
// removed
//response.setHeader("Content-Disposition", "attachment;filename=" + fileName);
bis = new BufferedInputStream(inputStream);
bos = new BufferedOutputStream(sOutputStream);
byte[] buff = new byte[10000];
int bytesRead = 0;
while(-1 != (bytesRead = bis.read(buff))) {
bos.write(buff, 0, bytesRead);
}
bos.flush();
2nd or 3rd call to bos.write causes
ClientAbortException: java.net.SocketException: socket write error: Connection aborted by peer
at org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:402)
at org.apache.tomcat.util.buf.ByteChunk.flushBuffer(ByteChunk.java:449)
at org.apache.tomcat.util.buf.ByteChunk.append(ByteChunk.java:349)
at org.apache.catalina.connector.OutputBuffer.writeBytes(OutputBuffer.java:425)
at org.apache.catalina.connector.OutputBuffer.write(OutputBuffer.java:414)
at org.apache.catalina.connector.CoyoteOutputStream.write(CoyoteOutputStream.java:89)
at java.io.BufferedOutputStream.write(BufferedOutputStream.java:105)
When I debug the code, at the time write method fails, the browser opens a player and another, identical request is being generated and succeeds.
When Content-Disposition is set eveything works fine. Any ideas?
It's because the client aborts the request after noticing that it's actually a media file and switches via client's media player to streaming mode via HTTP Range requests in order to improve buffering speed. The client will then fire multiple HTTP requests on different parts of the file (obviously, this works only efficiently if your servlet also really supports it ... a lot of homegrown file servlets don't and may eventually perform much worse).
As to those client abort exceptions in the server log, your best bet is to filter out and suppress them, or at least log with an DEBUG/INFO oneliner instead of with a whole stack trace.
See also:
How to stream audio/video files such as MP3, MP4, AVI, etc using a Servlet
ClientAbortException at application deployed at jboss with IE8 browser
I'm debugging a hybrid Worklight (6.1.0.01) application in Xcode (5.1.1). I'm seeing the following message in my log:
[DEBUG] [OCLogger] Max file size exceeded for log messages.
I"m assuming that this log file is managed by Worklight, as I do not have any custom native code running or explicit references to OCLogger. I do have calls to WL.Logger.debug.
How can I manage this log file? And where is it on my file system? (Truncating it will probably do the trick at this point.)
It is enabled by default. But you can disable using native or Javascript API Calls.
[OCLogger setCapture: NO];
For more info client side log capture
client side logging in client apps
I'm having a hard time trying to get my task to stay persistent and run indefinitely from a WCF service. I may be doing this the wrong way and am willing to take suggestions.
I have a task that starts to process any incoming requests that are dropped into a BlockingCollection. From what I understand, the GetConsumingEnumerable() method is supposed to allow me to persistently pull data as it arrives. It works with no problem by itself. I was able to process dozens of requests without a single error or flaw using a windows form to fill out the request and submit them. Once I was confident in this process I wired it up to my site via an asmx web service and used jQuery ajax calls to submit request.
The site submits request based on a url that is submitted, the Web Service downloads the html content from the url and looks for other urls within the content. It then proceeds to create a request for each url it finds and submits it to the BlockingCollection. Within the WCF service, if the application is Online (i.e. Task has started) - it pulls the request using the GetConsumingEnumerable via a Parallel.ForEach and Processes the request.
This works for the first few submissions, but then the task just stops unexpectedly. Of course, this is doing 10x more request than I could simulate in testing - but I expected it to just throttle. I believe the issue is in my method that starts the task:
public void Start()
{
Online = true;
Task.Factory.StartNew(() =>
{
tokenSource = new CancellationTokenSource();
CancellationToken token = tokenSource.Token;
ParallelOptions options = new ParallelOptions();
options.MaxDegreeOfParallelism = 20;
options.CancellationToken = token;
try
{
Parallel.ForEach(FixedWidthQueue.GetConsumingEnumerable(token), options, (request) =>
{
Process(request);
options.CancellationToken.ThrowIfCancellationRequested();
});
}
catch (OperationCanceledException e)
{
Console.WriteLine(e.Message);
return;
}
}, TaskCreationOptions.LongRunning);
}
I've thought about moving this into a WF4 Service and just wire it up in a Workflow and use Workflow Persistence, but am not willing to learn WF4 unless necessary. Please let me know if more information is needed.
The code you have shown is correct by itself.
However there are a few things that can go wrong:
If an exception occurs, your task stops (of course). Try adding a try-catch and log the exception.
If you start worker threads in a hosted environment (ASP.NET, WCF, SQL Server) the host can decide arbitrarily (without reason) to shut down any worker process. For example, if your ASP.NET site is inactive for some time the app is shut down. The hosts that I just mentioned are not made to have custom threads running. Probably, you will have more success using a dedicated application (.exe) or even a Windows Service.
It turns out the cause of this issue was with the WCF Binding Configuration. The task suddenly stopped becasue the WCF killed the connection due to a open timeout. The open timeout setting is the time that a request will wait for the service to open a connection before timing out. In certain situations, it reached the limit of 10 max connection and caused the incomming connections to get backed up waiting for a connection. I made sure that I closed all connections to the host after the transactions were complete - so I gave in to upping the max connections and the open timeout period. After this - it ran flawlessly.