How to stream text data with JAX-RS continuously - jax-rs

I read multiple questions that are similar to mine and found this: https://stackoverflow.com/a/34358215/12550134
But I am not able to do this. I use plain JAX-RS API and Open Liberty as my server. Unfortunately the ResourceConfig cannot be found, so I cannot disable the buffer, as described in the answer above.
This is my code:
#GET
#Produces(MediaType.TEXT_PLAIN)
public Response sayHelloStream() {
LOGGER.debug("calling sayHelloStream");
StreamingOutput out = outputStream -> {
Writer writer = new BufferedWriter(new OutputStreamWriter(outputStream));
for (int i = 0; i < 999999999; i++) {
writer.write("Hello\n");
writer.flush();
try {
LOGGER.debug("before sleep");
TimeUnit.SECONDS.sleep(3);
LOGGER.debug("after sleep");
} catch (InterruptedException e) {
LOGGER.error("error with the timer", e);
}
}
};
return Response.ok(out).build();
}
When calling it in the browser nothing happens. To my understanding due to the buffer. How am I able to stream text data like this using plain JAX-RS?

I would use the SSE extension. AFAIK it's part of the JAX-RS API, allthough you might need an extra module to enable it server-side:
https://eclipse-ee4j.github.io/jersey.github.io/documentation/latest/sse.html
...
import javax.ws.rs.sse.Sse;
import javax.ws.rs.sse.SseEventSink;
import javax.ws.rs.sse.OutboundSseEvent;
...
#Path("events")
public static class SseResource {
#GET
#Produces(MediaType.SERVER_SENT_EVENTS)
public void getServerSentEvents(#Context SseEventSink eventSink, #Context Sse sse) {
new Thread(() -> {
for (int i = 0; i < 10; i++) {
// ... code that waits 1 second
final OutboundSseEvent event = sse.newEventBuilder()
.name("message-to-client")
.data(String.class, "Hello world " + i + "!")
.build();
eventSink.send(event);
}
}).start();
}
}
It streams text data to the client in chunks in the SSE format, so it can easily be handled in the browser using e.g. the HTML5 <eventsource> element or the EventSource JavaScript API.
var source = new EventSource('.../events');
source.addEventListener('message-to-client', function(e) {
console.log(e.data);
}, false);

Related

How to detect when client has closed stream when writing to Response.Body in asp.net core

I'm trying to write an infinite length response body and detect when a client disconnects so I can stop writing. I'm used to getting socket exceptions or similar when a client closes the connection but that doesn't seem to be happening when writing directly to Response.Body. I can close the client applications and the server side just keeps on writing. I've included the relevant code below. It's entirely possible there is a better way to do it but this came to mind. Basically I have a live video feed which should go on forever. I'm writing to ResponseBody as chunked content (No content length, flushing after each video frame). The video frames are received via an event callback from elsewhere in the program so I'm subscribing to the events in the controller method and then forcing it to stay open with the await Task.Delay loop so the Response stream isn't closed. The callback for H264PacketReceived is formatting the data as a streaming mp4 file and writing it to the Response Stream. This all seems to work fine, I can play the live stream with ffmpeg or chrome, but when I close the client application I don't get an exception or anything. It just keeps writing to the stream without any errors.
public class LiveController : ControllerBase
{
[HttpGet]
[Route("/live/{cameraId}/{stream}.mp4")]
public async Task GetLiveMP4(Guid cameraId, int stream)
{
try
{
Response.StatusCode = 200;
Response.ContentType = "video/mp4";
Response.Headers.Add("Cache-Control", "no-store");
Response.Headers.Add("Connection", "close");
ms = Response.Body;
lock (TCPVideoReceiver.CameraStreams)
{
TCPVideoReceiver.CameraStreams.TryGetValue(cameraId, out cameraStream);
}
if (this.PacketStream == null)
{
throw new KeyNotFoundException($"Stream {cameraId}_{stream} not found");
}
else
{
connected = true;
this.PacketStream.H264PacketReceived += DefaultStream_H264PacketReceived;
this.PacketStream.StreamClosed += PacketStream_StreamClosed;
}
while(connected)
{
await Task.Delay(1000);
}
}
catch (Exception e)
{
Console.WriteLine(e.ToString());
}
finally
{
connected = false;
this.PacketStream.H264PacketReceived -= DefaultStream_H264PacketReceived;
this.PacketStream.StreamClosed -= PacketStream_StreamClosed;
}
}
private bool connected = false;
private PacketStream PacketStream;
private Mp4File mp4File;
private Stream ms;
private async void PacketStream_StreamClosed(PacketStream source)
{
await Task.Run(() =>
{
try
{
Console.WriteLine($"Closing live stream");
connected = false;
ms.Close();
}
catch (Exception e)
{
Console.WriteLine(e.ToString());
}
});
}
private async void DefaultStream_H264PacketReceived(PacketStream source, H264Packet packet)
{
try
{
if (mp4File == null && packet.IsIFrame)
{
mp4File = new Mp4File(null, packet.sps, packet.pps);
var _p = mp4File.WriteHeader(0);
await ms.WriteAsync(mp4File.buffer, 0, _p);
}
if (mp4File != null)
{
var _p = mp4File.WriteFrame(packet, 0);
var start = mp4File._moofScratchIndex - _p;
if (_p > 0)
{
await ms.WriteAsync(mp4File._moofScratch, start, _p);
await ms.FlushAsync();
}
}
return;
}
catch (Exception ex)
{
connected = false;
Console.WriteLine(ex.ToString());
}
}
Answering my own question.
When the client disconnects mvc core sets the cancellation token HttpContext.RequestAborted
By monitoring and/or using that cancellation token you can detect a disconnect and clean everything up.
That said, the entire design can be improved by creating a custom stream which encapsulates the event handling (producer/consumer). Then the controller action can be reduced to.
return File(new MyCustomStream(cameraId, stream), "video/mp4");
The File Method already monitors the cancellation token and everything works as you'd expect.

How to solve a FolderClosedIOException?

So I am new to Apache Camel. I know that most of this code is probably not the most efficient way to do this, but I have made a code that uses Apache Camel to access my gmail, grab the new messages and if they have attachments save the attachments in a specified directory. My route saves the body data as a file in that directory. Everytime the DataHandler tries to use the getContent() method, whether its saving a file or trying to print the body to System.out, I get either a FolderClosedIOException or a FolderClosed Exception. I have not clue how to fix it. The catch reopens the folder but it just closes again after getting another message.
import org.apache.camel.*;
import java.io.*;
import java.util.*;
import javax.activation.DataHandler;
import javax.mail.Folder;
import org.apache.camel.builder.RouteBuilder;
import org.apache.camel.impl.DefaultCamelContext;
import com.sun.mail.util.FolderClosedIOException;
public class Imap {
public static void main(String[] args) throws Exception {
CamelContext context = new DefaultCamelContext();
context.addRoutes(new RouteBuilder() {
public void configure() {
from("imaps://imap.gmail.com?username=********#gmail.com&password=******"
+ "&debugMode=false&closeFolder=false&mapMailMessage=false"
+ "&connectionTimeout=0").to("file:\\EMAIL");
}
});
Map<String,String> props = new HashMap<String,String>();
props.put("mail.imap.socketFactory.class","javax.net.ssl.SSLSocketFactory");
props.put("mail.imap.auth", "true");
props.put("mail.imap.host","imap.gmail.com");
props.put("mail.store.protocol", "imaps");
context.setProperties(props);
Folder inbox = null;
ConsumerTemplate template = context.createConsumerTemplate();
context.start();
while(true) {
try {
Exchange e = template.receive("imaps://imap.gmail.com?username=*********#gmail.com&password=***********", 60000);
if(e == null) break;
Message m = e.getIn();
Map<String, Object> s = m.getHeaders();
Iterator it = s.entrySet().iterator();
while(it.hasNext()) {
Map.Entry pairs = (Map.Entry)it.next();
System.out.println(pairs.getKey()+" === "+pairs.getValue()+"\n\n");
it.remove();
}
if(m.hasAttachments()) {
Map<String,DataHandler> att = m.getAttachments();
for(String s1 : att.keySet()) {
DataHandler dh = att.get(s1);
String filename = dh.getName();
ByteArrayOutputStream o = new ByteArrayOutputStream();
dh.writeTo(o);
byte[] by = o.toByteArray();
FileOutputStream out = new FileOutputStream("C:/EMAIL/"+filename);
out.write(by);
out.flush();
out.close();
}
}
} catch(FolderClosedIOException ex) {
inbox = ex.getFolder();
inbox.open(Folder.READ_ONLY);
}
}
context.stop();
}
}
Please somebody tell me whats wrong!!
The error occurs here:
dh.writeTo(o);
We were was solving a similar problem in akka-camel
The solution i believe was to use manual acknowledgement and send an acknowledgement after we were done with the message.

Windows Store App UI - not responsive

I'm trying to get my head around designing a UI that remains responsive while a long running task is being executed.
To that end, I created a simple app in VS2012 and added the following class to it:
using System.Threading.Tasks;
namespace TaskTest
{
class Class1
{
public async Task<int> Async()
{
//simulate a long running process
for (long x = 0; x < long.MaxValue; x++) { }
return 1;
}
}
}
I then modified the main page's LoadState() method thusly:
protected override async void LoadState(Object navigationParameter, Dictionary<String, Object> pageState)
{
await DoLongRunningProcess();
}
private async Task DoLongRunningProcess()
{
var id = 0;
id = await new Class1().Async();
await new MessageDialog(id + "").ShowAsync();
}
I want the page to remain responsive while that process executes. However, when I run this code, the page takes a long time to load. What am I doing wrong?
TIA
async isn't magic; it just gives you the capability to write asynchronous code. In particular, async does not execute code on a background thread. You can use Task.Run to do this.
You may find my async/await intro or the MSDN documentation helpful.
That was helpful. I made the following changes and I got the result I was looking for:
class Class1
{
public int Launch()
{
//throw new Exception("Class1 exception");
for (var i = 0; i < int.MaxValue / 2; i++) ;
return 1;
}
}
...
protected async override void LoadState(Object navigationParameter, Dictionary<String, Object> pageState)
{
var task = DoLongRunningProcess();
await task;
await new MessageDialog(task.Result + "").ShowAsync();
}
private Task<int> DoLongRunningProcess()
{
return Task.Run<int>(() => new Class1().Launch());
}
The page continues to load and after a short pause the message dialog is displayed. Now however, I need to know how to catch exceptions. If I uncomment the //throw new Exception ... line in method Launch(), it is reported as an unhandled exception. I want to catch this exception in the main UI thread (i.e., in the body of method LoadState) but I can't seem to manage it.

Maximum threads issue

To begin with, I checked the discussions regarding this issue and couldn't find an answer to my problem and that's why I'm opening this question.
I've set up a web service using restlet 2.0.15.The implementation is only for the server. The connections to the server are made through a webpage, and therefore I didn't use ClientResource.
Most of the answers to the exhaustion of the thread pool problem suggested the inclusion of
#exhaust + #release
The process of web service can be described as a single function.Receive GET requests from the webpage, query the database, frame the results in XML and return the final representation. I used a Filter to override the beforeHandle and afterHandle.
The code for component creation code:
Component component = new Component();
component.getServers().add(Protocol.HTTP, 8188);
component.getContext().getParameters().add("maxThreads", "512");
component.getContext().getParameters().add("minThreads", "100");
component.getContext().getParameters().add("lowThreads", "145");
component.getContext().getParameters().add("maxQueued", "100");
component.getContext().getParameters().add("maxTotalConnections", "100");
component.getContext().getParameters().add("maxIoIdleTimeMs", "100");
component.getDefaultHost().attach("/orcamento2013", new ServerApp());
component.start();
The parameters are the result of a discussion present in this forum and modification by my part in an attempt to maximize efficiency.
Coming to the Application, the code is as follows:
#Override
public synchronized Restlet createInboundRoot() {
// Create a router Restlet that routes each call to a
// new instance of HelloWorldResource.
Router router = new Router(getContext());
// Defines only one route
router.attach("/{taxes}", ServerImpl.class);
//router.attach("/acores/{taxes}", ServerImplAcores.class);
System.out.println(router.getRoutes().size());
OriginFilter originFilter = new OriginFilter(getContext());
originFilter.setNext(router);
return originFilter;
}
I used an example Filter found in a discussion here, too. The implementation is as follows:
public OriginFilter(Context context) {
super(context);
}
#Override
protected int beforeHandle(Request request, Response response) {
if (Method.OPTIONS.equals(request.getMethod())) {
Form requestHeaders = (Form) request.getAttributes().get("org.restlet.http.headers");
String origin = requestHeaders.getFirstValue("Origin", true);
Form responseHeaders = (Form) response.getAttributes().get("org.restlet.http.headers");
if (responseHeaders == null) {
responseHeaders = new Form();
response.getAttributes().put("org.restlet.http.headers", responseHeaders);
responseHeaders.add("Access-Control-Allow-Origin", origin);
responseHeaders.add("Access-Control-Allow-Methods", "GET,POST,DELETE");
responseHeaders.add("Access-Control-Allow-Headers", "Content-Type");
responseHeaders.add("Access-Control-Allow-Credentials", "true");
response.setEntity(new EmptyRepresentation());
return SKIP;
}
}
return super.beforeHandle(request, response);
}
#Override
protected void afterHandle(Request request, Response response) {
if (!Method.OPTIONS.equals(request.getMethod())) {
Form requestHeaders = (Form) request.getAttributes().get("org.restlet.http.headers");
String origin = requestHeaders.getFirstValue("Origin", true);
Form responseHeaders = (Form) response.getAttributes().get("org.restlet.http.headers");
if (responseHeaders == null) {
responseHeaders = new Form();
response.getAttributes().put("org.restlet.http.headers", responseHeaders);
responseHeaders.add("Access-Control-Allow-Origin", origin);
responseHeaders.add("Access-Control-Allow-Methods", "GET,POST,DELETE"); //
responseHeaders.add("Access-Control-Allow-Headers", "Content-Type");
responseHeaders.add("Access-Control-Allow-Credentials", "true");
}
}
super.afterHandle(request, response);
Representation requestRepresentation = request.getEntity();
if (requestRepresentation != null) {
try {
requestRepresentation.exhaust();
} catch (IOException e) {
// handle exception
}
requestRepresentation.release();
}
Representation responseRepresentation = response.getEntity();
if(responseRepresentation != null) {
try {
responseRepresentation.exhaust();
} catch (IOException ex) {
Logger.getLogger(OriginFilter.class.getName()).log(Level.SEVERE, null, ex);
} finally {
}
}
}
The responseRepresentation does not have a #release method because it crashes the processes giving the warning WARNING: A response with a 200 (Ok) status should have an entity (...)
The code of the ServerResource implementation is the following:
public class ServerImpl extends ServerResource {
String itemName;
#Override
protected void doInit() throws ResourceException {
this.itemName = (String) getRequest().getAttributes().get("taxes");
}
#Get("xml")
public Representation makeItWork() throws SAXException, IOException {
DomRepresentation representation = new DomRepresentation(MediaType.TEXT_XML);
DAL dal = new DAL();
String ip = getRequest().getCurrent().getClientInfo().getAddress();
System.out.println(itemName);
double tax = Double.parseDouble(itemName);
Document myXML = Auxiliar.getMyXML(tax, dal, ip);
myXML.normalizeDocument();
representation.setDocument(myXML);
return representation;
}
#Override
protected void doRelease() throws ResourceException {
super.doRelease();
}
}
I've tried the solutions provided in other threads but none of them seem to work. Firstly, it does not seem that the thread pool is augmented with the parameters set as the warnings state that the thread pool available is 10. As mentioned before, the increase of the maxThreads value only seems to postpone the result.
Example: INFO: Worker service tasks: 0 queued, 10 active, 17 completed, 27 scheduled.
There could be some error concerning the Restlet version, but I downloaded the stable version to verify this was not the issue.The Web Service is having around 5000 requests per day, which is not much.Note: the insertion of the #release method either in the ServerResource or OriginFilter returns error and the referred warning ("WARNING: A response with a 200 (Ok) status should have an entity (...)")
Please guide.
Thanks!
By reading this site the problem residing in the server-side that I described was resolved by upgrading the Restlet distribution to the 2.1 version.
You will need to alter some code. You should consult the respective migration guide.

JavaFX: File upload to REST service / servlet fails because of missing boundary

I'm trying to upload a file using JavaFX using the HttpRequest. For this purpose I have written the following function.
function uploadFile(inputFile : File) : Void {
// check file
if (inputFile == null or not(inputFile.exists()) or inputFile.isDirectory()) {
return;
}
def httpRequest : HttpRequest = HttpRequest {
location: urlConverter.encodeURL("{serverUrl}");
source: new FileInputStream(inputFile)
method: HttpRequest.POST
headers: [
HttpHeader {
name: HttpHeader.CONTENT_TYPE
value: "multipart/form-data"
}
]
}
httpRequest.start();
}
On the server side, I am trying to handle the incoming data using the Apache Commons FileUpload API using a Jersey REST service. The code used to do this is a simple copy of the FileUpload tutorial on the Apache homepage.
#Path("Upload")
public class UploadService {
public static final String RC_OK = "OK";
public static final String RC_ERROR = "ERROR";
#POST
#Produces("text/plain")
public String handleFileUpload(#Context HttpServletRequest request) {
if (!ServletFileUpload.isMultipartContent(request)) {
return RC_ERROR;
}
FileItemFactory factory = new DiskFileItemFactory();
ServletFileUpload upload = new ServletFileUpload(factory);
List<FileItem> items = null;
try {
items = upload.parseRequest(request);
}
catch (FileUploadException e) {
e.printStackTrace();
return RC_ERROR;
}
...
}
}
However, I get a exception at items = upload.parseRequest(request);:
org.apache.commons.fileupload.FileUploadException: the request was rejected because no multipart boundary was found
I guess I have to add a manual boundary info to the InputStream. Is there any easy solution to do this? Or are there even other solutions?
Have you tried just using the InputStream from HttpServletRequest like so
InputStream is = httpRequest.getInputStream();
BufferedInputStream in = new BufferedInputStream(is);
//Write out bytes
out.close();
is.close();