Keep connection alive for streaming in ASP.NET Core - asp.net-core

I'm making a small web application which is built on ASP.NET Core. My application is for streaming video from clients to clients through service.
I've followed this post :
http://www.strathweb.com/2013/01/asynchronously-streaming-video-with-asp-net-web-api/
I've implemented the application of tutorial successfully, but, that was for streaming Video from server to clients.
What I wanna do now is :
Clients register to service for streaming. (using video or audio tag)
Service receives client submitted data (submit through POSTMAN)
Service broadcast the data to its every registered clients.
Here is what I've implemented:
(Index.cshtml)
<div>
<video width="480"
height="320"
controls="controls"
autoplay="autoplay">
<source src="/api/video/initiate"
type="video/mp4">
</source>
</video>
</div>
StreamingService
public class StreamingService: IStreamingService
{
public IList<Stream> Connections {get;set;}
public StreamingService()
{
Connections = new List<Stream>();
}
public byte[] AnalyzeStream(Stream stream)
{
long originalPosititon = 0;
if (stream.CanSeek)
{
originalPosititon = stream.Position;
stream.Position = 0;
}
try
{
var readBuffer = new byte[4096];
int bytesReader;
while ((byteRead = stream.Read(readBuffer, totalBytesRead, readBuffer.Length - totalBytesRead)) > 0)
{
totalBytesRead += byteRead;
if (totalBytesRead == readBuffer.Length)
{
var nextByte = stream.ReadByte();
if (nextByte != -1)
{
var temp = new byte[readBuffer * 2];
Buffer.BlockCopy(readBuffer, 0, temp, 0, readBuffer.Length);
Buffer.SetByte(temp, totalBytesRead, (byte)nextByte);
readBuffer = temp;
totalBytesRead++;
}
}
}
var buffer = readBuffer;
if (readBuffer.Length != totalBytesRead)
{
buffer = new byte[totalBytesRead];
Buffer.BlockCopy(readBuffer, 0, buffer, 0, totalBytesRead);
}
return buffer;
}
finally
{
if (stream.CanSeek)
stream.Position = originalPosititon;
}
}
}
VideoController
public class VideoController: Controller
{
private readonly IStreamingService _streamingService;
private readonly IHostingEnvironment _hostingEnvironment;
public VideoController(IStreamingService streamingService, IHostingEnvironment hostingEnvironment)
{
_streamingService = streamingService;
_hostingEnvironment = hostingEnvironment;
}
[HttpGet("initiate")]
public IActionResult Initiate()
{
_streamingService.Connections.Add(Response.Body);
}
[HttpPost("broadcast")]
public async Task<IActionResult> Broadcast()
{
// Retrieve data submitted from POSTMAN.
var data = _streamingService.AnalyzeStream(Request.Body);
foreach (var stream in _streamingService.Connections)
{
try
{
await stream.WriteAsync(data, 0, data.Length);
}
catch (Exception exception)
{
stream.Dispose();
_streamingService.Connections.Remove(stream);
}
}
}
}
When I send data from POSTMAN through api/video/broadcast . For loop ran and I got an exception said the stream has been disposed.
My question is:
How can I keep the stream alive for streaming ?
(Stream created in api/video/initiate is kept alive and when a client calls api/video/broadcast , all initiated stream will update its date without having disposed)
Thank you,

Is it an option to keep the stream in cache?
You can read more about it here. The simplest way it to add the cache services to the dependency injection container and the request the concrete implementation of IMemoryCache through constructor injection in your VideoController (as you've done with IStreamingService and IHostingEnvironment).
Just add the stream to the cache and use the cached stream the next time api/video/broadcast is hit.
Be aware though that if you are on a webfarm or hosted in the cloud it is recommended to use Distributed Cache like Redis Cache, or else your cache could disapear unexpected. I use Azure Redis Cache for instance which works great!

Related

Cannot Interrupt HttpClient.GetStreamAsync by means of CancellationToken in ASP.NET Core

I'm trying to send an http request to an AXIS Camera in order to receive a stream.
Everything works fine except that I can't get to use CancellationToken to cancel the request when it is no more needed. I've the following architecture:
Blazor client:
// LiveCamera.razor
<img src="CameraSystem/getStream" onerror="[...]" alt="">
ASP.NET Core Server:
// CameraSystemController.cs
[ApiController]
[Route("[controller]")]
public class CameraSystemController : Controller
{
[HttpGet("getStream")]
public async Task<IActionResult> GetStream()
{
Stream stream = await Device_CameraStandard.GetStream();
if (stream != null) {
Response.Headers.Add("Cache-Control", "no-cache");
FileStreamResult result = new FileStreamResult(stream, _contentTypeStreaming) {
EnableRangeProcessing = true
};
return result;
} else {
return new StatusCodeResult((int)HttpStatusCode.ServiceUnavailable);
}
}
}
Class accessing the camera:
// Device_CameraStandard.cs
internal class Device_CameraStandard
{
private HttpClient _httpClient;
private static CancellationTokenSource _tokenSource;
private System.Timers.Timer _keepAliveTimer;
internal Device_CameraStandard() {
_keepAliveTimer = new System.Timers.Timer();
_keepAliveTimer.Interval = 3000;
_keepAliveTimer.Elapsed += KeepAliveTimeout;
_tokenSource = new CancellationTokenSource();
[...]
}
internal async Task<Stream> GetStream()
{
return await _httpClient.GetStreamAsync("http://[...]/axis-cgi/mjpg/video.cgi?&camera=1", _tokenSource.Token);
}
// Invoked periodically by client from LiveCamera.razor.cs, not included here
internal void KeepAlive()
{
LLogger.Debug("KeepAlive!");
_keepAliveTimer.Stop();
_keepAliveTimer.Start();
}
private void KeepAliveTimeout(object sender, ElapsedEventArgs e)
{
LLogger.Debug("Timeout!");
_keepAliveTimer.Stop();
_tokenSource.Cancel();
_tokenSource.Dispose();
_tokenSource = new CancellationTokenSource();
}
}
However, even if all clients leave LiveCamera.razor page and the _keepAliveTimer elapses and the CancellationTokenSource is canceled, the request is not canceled. I can see it from the fact that bandwidth usage does not decreases (the "receiving" bandwitdh, indicating that Server is still receiving data from camera), it only decreases if I close the browser tab.
Could you please help me to understand what am I doing wrong? Thanks
EDIT: In the end, even after following the suggestion of observing the token in all code parts where the returned stream was used, included the controller, I ended up discovering that the tag
// LiveCamera.razor
<img src="CameraSystem/getStream" onerror="[...]" alt="">
was causing the client to never stop sending requests. Thus I had to use a workaround to force client to stop sending requests before leaving LiveCamera.razor page.

Asp .Net Core Socket

I made a backend with ASP .Net Core, first i made some Controllers for HTTP requests and they all worked fine, but i wanted to integrate a Socket to connect with with my Unity Game. I got the Socket working, but since i added the Socket the HTTP Controllers dont work anymore.
I think it has something to do with the Port but i m not able to figure out how to fix that.
Here is the Socket Code
public static void ServerSocket()
{
Console.WriteLine("TEXT");
TcpListener listener = new TcpListener(System.Net.IPAddress.Any, 1302);
listener.Start();
while (true)
{
Console.WriteLine("Waiting for a connection");
TcpClient client = listener.AcceptTcpClient();
Console.WriteLine("Client Accepted");
NetworkStream stream = client.GetStream();
StreamReader sr = new StreamReader(client.GetStream());
StreamWriter sw = new StreamWriter(client.GetStream());
try
{
byte[] buffer = new byte[1024];
stream.Read(buffer, 0, buffer.Length);
int recv = 0;
foreach (byte b in buffer)
{
if (b != 0)
{
recv++;
}
}
string request = Encoding.UTF8.GetString(buffer, 0, recv);
Console.WriteLine("request received");
sw.WriteLine("Hello darkness my old");
sw.Flush();
}
catch (Exception e)
{
Console.WriteLine("Something went wrong");
sw.WriteLine(e.ToString());
Console.WriteLine(sr.ToString());
}
}
}
I call that function in Startup.
public void Configure(IApplicationBuilder app, IHostingEnvironment env)
{
if (env.IsDevelopment())
{
app.UseDeveloperExceptionPage();
}
else
{
app.UseHsts();
}
app.UseHttpsRedirection();
app.UseMvc();
Thread newThread = new Thread(ServerSocketApp.ServerSocket);
newThread.Start();
}
You have a while(true) { ... }, which will cause execution to spin and never return back to Startup(), thus causing Startup() to halt as well.
Consider moving this socket code to a background thread. See Thread.Start in System.Threading as one way you can make this happen.
TcpListener.AcceptTcpClient() is a blocking method, which will prevent the rest of the code from executing.
Please try using TcpListener.AcceptTcpClientAsync instead.
Edit: the other guy is also right, putting that code in a different Thread is a good idea.

How to detect when client has closed stream when writing to Response.Body in asp.net core

I'm trying to write an infinite length response body and detect when a client disconnects so I can stop writing. I'm used to getting socket exceptions or similar when a client closes the connection but that doesn't seem to be happening when writing directly to Response.Body. I can close the client applications and the server side just keeps on writing. I've included the relevant code below. It's entirely possible there is a better way to do it but this came to mind. Basically I have a live video feed which should go on forever. I'm writing to ResponseBody as chunked content (No content length, flushing after each video frame). The video frames are received via an event callback from elsewhere in the program so I'm subscribing to the events in the controller method and then forcing it to stay open with the await Task.Delay loop so the Response stream isn't closed. The callback for H264PacketReceived is formatting the data as a streaming mp4 file and writing it to the Response Stream. This all seems to work fine, I can play the live stream with ffmpeg or chrome, but when I close the client application I don't get an exception or anything. It just keeps writing to the stream without any errors.
public class LiveController : ControllerBase
{
[HttpGet]
[Route("/live/{cameraId}/{stream}.mp4")]
public async Task GetLiveMP4(Guid cameraId, int stream)
{
try
{
Response.StatusCode = 200;
Response.ContentType = "video/mp4";
Response.Headers.Add("Cache-Control", "no-store");
Response.Headers.Add("Connection", "close");
ms = Response.Body;
lock (TCPVideoReceiver.CameraStreams)
{
TCPVideoReceiver.CameraStreams.TryGetValue(cameraId, out cameraStream);
}
if (this.PacketStream == null)
{
throw new KeyNotFoundException($"Stream {cameraId}_{stream} not found");
}
else
{
connected = true;
this.PacketStream.H264PacketReceived += DefaultStream_H264PacketReceived;
this.PacketStream.StreamClosed += PacketStream_StreamClosed;
}
while(connected)
{
await Task.Delay(1000);
}
}
catch (Exception e)
{
Console.WriteLine(e.ToString());
}
finally
{
connected = false;
this.PacketStream.H264PacketReceived -= DefaultStream_H264PacketReceived;
this.PacketStream.StreamClosed -= PacketStream_StreamClosed;
}
}
private bool connected = false;
private PacketStream PacketStream;
private Mp4File mp4File;
private Stream ms;
private async void PacketStream_StreamClosed(PacketStream source)
{
await Task.Run(() =>
{
try
{
Console.WriteLine($"Closing live stream");
connected = false;
ms.Close();
}
catch (Exception e)
{
Console.WriteLine(e.ToString());
}
});
}
private async void DefaultStream_H264PacketReceived(PacketStream source, H264Packet packet)
{
try
{
if (mp4File == null && packet.IsIFrame)
{
mp4File = new Mp4File(null, packet.sps, packet.pps);
var _p = mp4File.WriteHeader(0);
await ms.WriteAsync(mp4File.buffer, 0, _p);
}
if (mp4File != null)
{
var _p = mp4File.WriteFrame(packet, 0);
var start = mp4File._moofScratchIndex - _p;
if (_p > 0)
{
await ms.WriteAsync(mp4File._moofScratch, start, _p);
await ms.FlushAsync();
}
}
return;
}
catch (Exception ex)
{
connected = false;
Console.WriteLine(ex.ToString());
}
}
Answering my own question.
When the client disconnects mvc core sets the cancellation token HttpContext.RequestAborted
By monitoring and/or using that cancellation token you can detect a disconnect and clean everything up.
That said, the entire design can be improved by creating a custom stream which encapsulates the event handling (producer/consumer). Then the controller action can be reduced to.
return File(new MyCustomStream(cameraId, stream), "video/mp4");
The File Method already monitors the cancellation token and everything works as you'd expect.

ASP.NET Core 2.2 - SignalR How to join a group and send message from the server?

I have a set up a basic solution for SignalR where I can send and receive messages to all clients through the browser.
Required solution:
I need to implement the use of groups, BUT with the ability to join the group and send a message from a background service class that is running on the server.
Currently I have only been successful in sending a message to ALL clients from the server.
My background service class (basically a comms server) will be running multiple instances, so each instance will have a unique name such as CommsServer 1, CommsServer 2 etc. Each instance of the comms server will need to output messages to a specific SignalR group of recipients.
In the browser, the user will select which SignalR group they wish to join from a dropdown list that is pulled from the server. The server also has knowledge of this same list and therefore each comms server instance will represent an item from the list.
My code so far:
MessageHub Class:
public class MessageHub : Hub
{
public Task SendMessageToAll(string message)
{
return Clients.All.SendAsync("ReceiveMessage", message);
}
}
Message.js File:
"use strict";
var connection = new signalR.HubConnectionBuilder()
.withUrl("/messages")
.build();
connection.on("ReceiveMessage", function (message) {
var msg = message.replace(/&/g, "&").replace(/</g, "<").replace(/>/g, ">");
var div = document.createElement("div");
div.innerHTML = msg + "<hr/>";
document.getElementById("messages").appendChild(div);
});
connection.start().catch(function (err) {
return console.error(err.toString());
});
document.getElementById("sendButton").addEventListener("click", function (event) {
var message = document.getElementById("message").value;
connection.invoke("SendMessageToAll", message).catch(function (err) {
return console.error(err.toString());
});
e.preventDefault();
});
Razor Page:
<textarea name="message" id="message"></textarea>
<input type="button" id="sendButton" value="Send Message" />
<div id="messages"></div>
<script src="~/lib/signalr/dist/browser/signalr.js"></script>
<script src="~/js/message.js"></script>
Comms server class:
public class TcpServer
{
private readonly IHubContext<MessageHub> _hubContext;
public TcpServerTcpServer(IHubContext<MessageHub> hubContext)
{
_hubContext = hubContext;
}
public string inboundIpAddress = "127.0.0.1";
public int inboundLocalPortNumber = 10001;
public void TcpServerIN(string serverName)
{
Task.Run(() =>
{
IPAddress localAddrIN = IPAddress.Parse(inboundIpAddress);
TcpListener listener = new TcpListener(localAddrIN, inboundLocalPortNumber);
listener.Start();
while (true)
{
TcpClient client = listener.AcceptTcpClient();
// Get a stream object for reading and writing
NetworkStream stream = client.GetStream(); // Networkstream is used to send/receive messages
//Buffer for reading data
Byte[] bytes = new Byte[4096];
String data = null;
int i;
// Loop to receive all the data sent by the client.
while ((i = stream.Read(bytes, 0, bytes.Length)) != 0)
{
// Translate data bytes to a ASCII string.
data = System.Text.Encoding.ASCII.GetString(bytes, 0, i);
_hubContext.Clients.All.SendAsync(data);
byte[] msg = System.Text.Encoding.ASCII.GetBytes(data);
stream.Write(msg, 0, data.Length);
}
// Shutdown and end connection
client.Close();
}
});
}
}
I need to select a specific SignalR group name e.g. "CommsServer 1" this will form the name of the SignalR group that the user can select from a dropdown list in the browser so they can monitor events from just this one particular server instance. Other user may wish to monitor events from a different server instance.
One option I was considering is simply send events from all servers instances to ALL browser connected clients and then somehow filter the events on the client side, but this would seem an inefficient way of handling things and not good for network traffic.
If you want your client to decide which group to join (by selecting from a list of dropdowns), then your client should probably notify the server of its subscription to messages aimed at the group via the hub:
Hub:
public class MessageHub : Hub
{
public async Task SendMessageToAll(string message)
{
await Clients.All.SendAsync("ReceiveMessage", message);
}
public async Task Subscribe(string groupName)
{
await Groups.AddToGroupAsync(Context.ConnectionId, groupName);
}
}
Background service:
public class TcpServer
{
private readonly IHubContext<MessageHub> _hubContext;
private readonly string _serviceInstanceName;
public TcpServerTcpServer(IHubContext<MessageHub> hubContext, string serviceInstanceName)
{
_hubContext = hubContext;
_serviceInstanceName = serviceInstanceName;
}
public async Task SendMessageToGroup(string message)
{
await _hubContext.Clients.Group(_serviceInstanceName).SendAsync("ReceiveMessage", message);
}
}
And somewhere in your client, perhaps after user chooses an option from your dropdown:
connection.invoke("subscribe", "CommsServer 1, or whatever").catch(function (err) {
return console.error(err.toString());
});

Send many byte array from server to UWP as a stream

I have a WCF service that updates the byte array from camera continuous like this:
private void ImageGrabbedCamera(object sender, ImageGrabbedEventArgs e)
{
try
{
if (e.GrabResult.GrabSucceeded)
{
//This variable is updated continuous from the camera
result = e.GrabResult.Clone();
}
}
catch (Exception ex)
{
Debug.WriteLine(ex.Message);
}
}
And I call this method from the UWP client app to get the byte array from the server
public Stream GetStreamCamera()
{
MemoryStream ms;
if (result != null)
{
ms = new MemoryStream(ObjectToByteArray(result.PixelData as byte[]));
ms.Position = 0;
return ms;
}
else
{
return new MemoryStream();
}
}
In client app, I call while(true) to GetStreamCamera() method to get the frame but it not OK because the capacity is very big and it's not only 1 camera, it about 10 cameras with resolution (1280 * 960). So do we have any protocol that UWP support to stream the image from the server to UWP client?
I don't want to call while(true) to get 1 frame/call anymore.