I have a WCF service that updates the byte array from camera continuous like this:
private void ImageGrabbedCamera(object sender, ImageGrabbedEventArgs e)
{
try
{
if (e.GrabResult.GrabSucceeded)
{
//This variable is updated continuous from the camera
result = e.GrabResult.Clone();
}
}
catch (Exception ex)
{
Debug.WriteLine(ex.Message);
}
}
And I call this method from the UWP client app to get the byte array from the server
public Stream GetStreamCamera()
{
MemoryStream ms;
if (result != null)
{
ms = new MemoryStream(ObjectToByteArray(result.PixelData as byte[]));
ms.Position = 0;
return ms;
}
else
{
return new MemoryStream();
}
}
In client app, I call while(true) to GetStreamCamera() method to get the frame but it not OK because the capacity is very big and it's not only 1 camera, it about 10 cameras with resolution (1280 * 960). So do we have any protocol that UWP support to stream the image from the server to UWP client?
I don't want to call while(true) to get 1 frame/call anymore.
Related
I read multiple questions that are similar to mine and found this: https://stackoverflow.com/a/34358215/12550134
But I am not able to do this. I use plain JAX-RS API and Open Liberty as my server. Unfortunately the ResourceConfig cannot be found, so I cannot disable the buffer, as described in the answer above.
This is my code:
#GET
#Produces(MediaType.TEXT_PLAIN)
public Response sayHelloStream() {
LOGGER.debug("calling sayHelloStream");
StreamingOutput out = outputStream -> {
Writer writer = new BufferedWriter(new OutputStreamWriter(outputStream));
for (int i = 0; i < 999999999; i++) {
writer.write("Hello\n");
writer.flush();
try {
LOGGER.debug("before sleep");
TimeUnit.SECONDS.sleep(3);
LOGGER.debug("after sleep");
} catch (InterruptedException e) {
LOGGER.error("error with the timer", e);
}
}
};
return Response.ok(out).build();
}
When calling it in the browser nothing happens. To my understanding due to the buffer. How am I able to stream text data like this using plain JAX-RS?
I would use the SSE extension. AFAIK it's part of the JAX-RS API, allthough you might need an extra module to enable it server-side:
https://eclipse-ee4j.github.io/jersey.github.io/documentation/latest/sse.html
...
import javax.ws.rs.sse.Sse;
import javax.ws.rs.sse.SseEventSink;
import javax.ws.rs.sse.OutboundSseEvent;
...
#Path("events")
public static class SseResource {
#GET
#Produces(MediaType.SERVER_SENT_EVENTS)
public void getServerSentEvents(#Context SseEventSink eventSink, #Context Sse sse) {
new Thread(() -> {
for (int i = 0; i < 10; i++) {
// ... code that waits 1 second
final OutboundSseEvent event = sse.newEventBuilder()
.name("message-to-client")
.data(String.class, "Hello world " + i + "!")
.build();
eventSink.send(event);
}
}).start();
}
}
It streams text data to the client in chunks in the SSE format, so it can easily be handled in the browser using e.g. the HTML5 <eventsource> element or the EventSource JavaScript API.
var source = new EventSource('.../events');
source.addEventListener('message-to-client', function(e) {
console.log(e.data);
}, false);
I'm trying to write an infinite length response body and detect when a client disconnects so I can stop writing. I'm used to getting socket exceptions or similar when a client closes the connection but that doesn't seem to be happening when writing directly to Response.Body. I can close the client applications and the server side just keeps on writing. I've included the relevant code below. It's entirely possible there is a better way to do it but this came to mind. Basically I have a live video feed which should go on forever. I'm writing to ResponseBody as chunked content (No content length, flushing after each video frame). The video frames are received via an event callback from elsewhere in the program so I'm subscribing to the events in the controller method and then forcing it to stay open with the await Task.Delay loop so the Response stream isn't closed. The callback for H264PacketReceived is formatting the data as a streaming mp4 file and writing it to the Response Stream. This all seems to work fine, I can play the live stream with ffmpeg or chrome, but when I close the client application I don't get an exception or anything. It just keeps writing to the stream without any errors.
public class LiveController : ControllerBase
{
[HttpGet]
[Route("/live/{cameraId}/{stream}.mp4")]
public async Task GetLiveMP4(Guid cameraId, int stream)
{
try
{
Response.StatusCode = 200;
Response.ContentType = "video/mp4";
Response.Headers.Add("Cache-Control", "no-store");
Response.Headers.Add("Connection", "close");
ms = Response.Body;
lock (TCPVideoReceiver.CameraStreams)
{
TCPVideoReceiver.CameraStreams.TryGetValue(cameraId, out cameraStream);
}
if (this.PacketStream == null)
{
throw new KeyNotFoundException($"Stream {cameraId}_{stream} not found");
}
else
{
connected = true;
this.PacketStream.H264PacketReceived += DefaultStream_H264PacketReceived;
this.PacketStream.StreamClosed += PacketStream_StreamClosed;
}
while(connected)
{
await Task.Delay(1000);
}
}
catch (Exception e)
{
Console.WriteLine(e.ToString());
}
finally
{
connected = false;
this.PacketStream.H264PacketReceived -= DefaultStream_H264PacketReceived;
this.PacketStream.StreamClosed -= PacketStream_StreamClosed;
}
}
private bool connected = false;
private PacketStream PacketStream;
private Mp4File mp4File;
private Stream ms;
private async void PacketStream_StreamClosed(PacketStream source)
{
await Task.Run(() =>
{
try
{
Console.WriteLine($"Closing live stream");
connected = false;
ms.Close();
}
catch (Exception e)
{
Console.WriteLine(e.ToString());
}
});
}
private async void DefaultStream_H264PacketReceived(PacketStream source, H264Packet packet)
{
try
{
if (mp4File == null && packet.IsIFrame)
{
mp4File = new Mp4File(null, packet.sps, packet.pps);
var _p = mp4File.WriteHeader(0);
await ms.WriteAsync(mp4File.buffer, 0, _p);
}
if (mp4File != null)
{
var _p = mp4File.WriteFrame(packet, 0);
var start = mp4File._moofScratchIndex - _p;
if (_p > 0)
{
await ms.WriteAsync(mp4File._moofScratch, start, _p);
await ms.FlushAsync();
}
}
return;
}
catch (Exception ex)
{
connected = false;
Console.WriteLine(ex.ToString());
}
}
Answering my own question.
When the client disconnects mvc core sets the cancellation token HttpContext.RequestAborted
By monitoring and/or using that cancellation token you can detect a disconnect and clean everything up.
That said, the entire design can be improved by creating a custom stream which encapsulates the event handling (producer/consumer). Then the controller action can be reduced to.
return File(new MyCustomStream(cameraId, stream), "video/mp4");
The File Method already monitors the cancellation token and everything works as you'd expect.
I'm making a small web application which is built on ASP.NET Core. My application is for streaming video from clients to clients through service.
I've followed this post :
http://www.strathweb.com/2013/01/asynchronously-streaming-video-with-asp-net-web-api/
I've implemented the application of tutorial successfully, but, that was for streaming Video from server to clients.
What I wanna do now is :
Clients register to service for streaming. (using video or audio tag)
Service receives client submitted data (submit through POSTMAN)
Service broadcast the data to its every registered clients.
Here is what I've implemented:
(Index.cshtml)
<div>
<video width="480"
height="320"
controls="controls"
autoplay="autoplay">
<source src="/api/video/initiate"
type="video/mp4">
</source>
</video>
</div>
StreamingService
public class StreamingService: IStreamingService
{
public IList<Stream> Connections {get;set;}
public StreamingService()
{
Connections = new List<Stream>();
}
public byte[] AnalyzeStream(Stream stream)
{
long originalPosititon = 0;
if (stream.CanSeek)
{
originalPosititon = stream.Position;
stream.Position = 0;
}
try
{
var readBuffer = new byte[4096];
int bytesReader;
while ((byteRead = stream.Read(readBuffer, totalBytesRead, readBuffer.Length - totalBytesRead)) > 0)
{
totalBytesRead += byteRead;
if (totalBytesRead == readBuffer.Length)
{
var nextByte = stream.ReadByte();
if (nextByte != -1)
{
var temp = new byte[readBuffer * 2];
Buffer.BlockCopy(readBuffer, 0, temp, 0, readBuffer.Length);
Buffer.SetByte(temp, totalBytesRead, (byte)nextByte);
readBuffer = temp;
totalBytesRead++;
}
}
}
var buffer = readBuffer;
if (readBuffer.Length != totalBytesRead)
{
buffer = new byte[totalBytesRead];
Buffer.BlockCopy(readBuffer, 0, buffer, 0, totalBytesRead);
}
return buffer;
}
finally
{
if (stream.CanSeek)
stream.Position = originalPosititon;
}
}
}
VideoController
public class VideoController: Controller
{
private readonly IStreamingService _streamingService;
private readonly IHostingEnvironment _hostingEnvironment;
public VideoController(IStreamingService streamingService, IHostingEnvironment hostingEnvironment)
{
_streamingService = streamingService;
_hostingEnvironment = hostingEnvironment;
}
[HttpGet("initiate")]
public IActionResult Initiate()
{
_streamingService.Connections.Add(Response.Body);
}
[HttpPost("broadcast")]
public async Task<IActionResult> Broadcast()
{
// Retrieve data submitted from POSTMAN.
var data = _streamingService.AnalyzeStream(Request.Body);
foreach (var stream in _streamingService.Connections)
{
try
{
await stream.WriteAsync(data, 0, data.Length);
}
catch (Exception exception)
{
stream.Dispose();
_streamingService.Connections.Remove(stream);
}
}
}
}
When I send data from POSTMAN through api/video/broadcast . For loop ran and I got an exception said the stream has been disposed.
My question is:
How can I keep the stream alive for streaming ?
(Stream created in api/video/initiate is kept alive and when a client calls api/video/broadcast , all initiated stream will update its date without having disposed)
Thank you,
Is it an option to keep the stream in cache?
You can read more about it here. The simplest way it to add the cache services to the dependency injection container and the request the concrete implementation of IMemoryCache through constructor injection in your VideoController (as you've done with IStreamingService and IHostingEnvironment).
Just add the stream to the cache and use the cached stream the next time api/video/broadcast is hit.
Be aware though that if you are on a webfarm or hosted in the cloud it is recommended to use Distributed Cache like Redis Cache, or else your cache could disapear unexpected. I use Azure Redis Cache for instance which works great!
how i get buffering status while media player trying to connect audio streaming link(ie 2%,4%...100% then online radio start to play) in android.
this is my code.
but i have no idea how i solve my problem.thanks is advance to any kind of help.
player.setOnBufferingUpdateListener(new OnBufferingUpdateListener() {
public void onBufferingUpdate(MediaPlayer mp, int percent) {
playSeekBar.setSecondaryProgress(percent);
Log.i("Buffering", "" + percent);
}
});
i solve this problem. here is the link. http://coderfriend.blogspot.com/
as per request here i share blog content..
when user click play button to play radio then i want to show connecting status(buffering 1%,2%.. 99%). when status will be 100% radio start to play. i was face problem to solve this. so here i share this code for all.
//at first create this class
public class StreamingMediaPlayer {
private static final int INTIAL_KB_BUFFER = 96*10/8;//assume 96kbps*10secs/8bits per byte
private TextView textStreamed;
private ImageButton playButton;
private ProgressBar progressBar;
// Track for display by progressBar
private long mediaLengthInKb, mediaLengthInSeconds;
private int totalKbRead = 0;
// Create Handler to call View updates on the main UI thread.
private final Handler handler = new Handler();
private MediaPlayer mediaPlayer;
private File downloadingMediaFile;
private boolean isInterrupted;
private Context context;
private int counter = 0;
public StreamingMediaPlayer(Context context,TextView textStreamed, ImageButton playButton, Button streamButton,ProgressBar progressBar)
{
this.context = context;
this.textStreamed = textStreamed;
this.playButton = playButton;
this.progressBar = progressBar;
}
/**
* Progressivly download the media to a temporary location and update the MediaPlayer as new content becomes available.
*/
public void startStreaming(final String mediaUrl, long mediaLengthInKb, long mediaLengthInSeconds) throws IOException {
this.mediaLengthInKb = mediaLengthInKb;
this.mediaLengthInSeconds = mediaLengthInSeconds;
Runnable r = new Runnable() {
public void run() {
try {
downloadAudioIncrement(mediaUrl);
} catch (IOException e) {
Log.e(getClass().getName(), "Unable to initialize the MediaPlayer for fileUrl=" + mediaUrl, e);
return;
}
}
};
new Thread(r).start();
}
/**
* Download the url stream to a temporary location and then call the setDataSource
* for that local file
*/
public void downloadAudioIncrement(String mediaUrl) throws IOException {
URLConnection cn = new URL(mediaUrl).openConnection();
cn.connect();
InputStream stream = cn.getInputStream();
if (stream == null) {
Log.e(getClass().getName(), "Unable to create InputStream for mediaUrl:" + mediaUrl);
}
downloadingMediaFile = new File(context.getCacheDir(),"downloadingMedia.dat");
if (downloadingMediaFile.exists()) {
downloadingMediaFile.delete();
}
FileOutputStream out = new FileOutputStream(downloadingMediaFile);
byte buf[] = new byte[16384];
int totalBytesRead = 0, incrementalBytesRead = 0;
do {
int numread = stream.read(buf);
if (numread <= 0)
break;
out.write(buf, 0, numread);
totalBytesRead += numread;
incrementalBytesRead += numread;
totalKbRead = totalBytesRead/1000;
testMediaBuffer();
fireDataLoadUpdate();
} while (validateNotInterrupted());
stream.close();
if (validateNotInterrupted()) {
fireDataFullyLoaded();
}
}
private boolean validateNotInterrupted() {
if (isInterrupted) {
if (mediaPlayer != null) {
mediaPlayer.pause();
//mediaPlayer.release();
}
return false;
} else {
return true;
}
}
/**
* Test whether we need to transfer buffered data to the MediaPlayer.
* Interacting with MediaPlayer on non-main UI thread can causes crashes to so perform this using a Handler.
*/
private void testMediaBuffer() {
Runnable updater = new Runnable() {
public void run() {
if (mediaPlayer == null) {
// Only create the MediaPlayer once we have the minimum buffered data
if ( totalKbRead >= INTIAL_KB_BUFFER) {
try {
startMediaPlayer();
} catch (Exception e) {
Log.e(getClass().getName(), "Error copying buffered conent.", e);
}
}
} else if ( mediaPlayer.getDuration() - mediaPlayer.getCurrentPosition() <= 1000 ){
// NOTE: The media player has stopped at the end so transfer any existing buffered data
// We test for < 1second of data because the media player can stop when there is still
// a few milliseconds of data left to play
transferBufferToMediaPlayer();
}
}
};
handler.post(updater);
}
private void startMediaPlayer() {
try {
File bufferedFile = new File(context.getCacheDir(),"playingMedia" + (counter++) + ".dat");
// We double buffer the data to avoid potential read/write errors that could happen if the
// download thread attempted to write at the same time the MediaPlayer was trying to read.
// For example, we can't guarantee that the MediaPlayer won't open a file for playing and leave it locked while
// the media is playing. This would permanently deadlock the file download. To avoid such a deadloack,
// we move the currently loaded data to a temporary buffer file that we start playing while the remaining
// data downloads.
moveFile(downloadingMediaFile,bufferedFile);
Log.e(getClass().getName(),"Buffered File path: " + bufferedFile.getAbsolutePath());
Log.e(getClass().getName(),"Buffered File length: " + bufferedFile.length()+"");
mediaPlayer = createMediaPlayer(bufferedFile);
// We have pre-loaded enough content and started the MediaPlayer so update the buttons & progress meters.
mediaPlayer.start();
startPlayProgressUpdater();
playButton.setEnabled(true);
} catch (IOException e) {
Log.e(getClass().getName(), "Error initializing the MediaPlayer.", e);
return;
}
}
private MediaPlayer createMediaPlayer(File mediaFile)
throws IOException {
MediaPlayer mPlayer = new MediaPlayer();
mPlayer.setOnErrorListener(
new MediaPlayer.OnErrorListener() {
public boolean onError(MediaPlayer mp, int what, int extra) {
Log.e(getClass().getName(), "Error in MediaPlayer: (" + what +") with extra (" +extra +")" );
return false;
}
});
// It appears that for security/permission reasons, it is better to pass a FileDescriptor rather than a direct path to the File.
// Also I have seen errors such as "PVMFErrNotSupported" and "Prepare failed.: status=0x1" if a file path String is passed to
// setDataSource(). So unless otherwise noted, we use a FileDescriptor here.
FileInputStream fis = new FileInputStream(mediaFile);
mPlayer.setDataSource(fis.getFD());
mPlayer.prepare();
return mPlayer;
}
/**
* Transfer buffered data to the MediaPlayer.
* NOTE: Interacting with a MediaPlayer on a non-main UI thread can cause thread-lock and crashes so
* this method should always be called using a Handler.
*/
private void transferBufferToMediaPlayer() {
try {
// First determine if we need to restart the player after transferring data...e.g. perhaps the user pressed pause
boolean wasPlaying = mediaPlayer.isPlaying();
int curPosition = mediaPlayer.getCurrentPosition();
// Copy the currently downloaded content to a new buffered File. Store the old File for deleting later.
File oldBufferedFile = new File(context.getCacheDir(),"playingMedia" + counter + ".dat");
File bufferedFile = new File(context.getCacheDir(),"playingMedia" + (counter++) + ".dat");
// This may be the last buffered File so ask that it be delete on exit. If it's already deleted, then this won't mean anything. If you want to
// keep and track fully downloaded files for later use, write caching code and please send me a copy.
bufferedFile.delete();
moveFile(downloadingMediaFile,bufferedFile);
// Pause the current player now as we are about to create and start a new one. So far (Android v1.5),
// this always happens so quickly that the user never realized we've stopped the player and started a new one
mediaPlayer.pause();
// Create a new MediaPlayer rather than try to re-prepare the prior one.
mediaPlayer = createMediaPlayer(bufferedFile);
mediaPlayer.seekTo(curPosition);
// Restart if at end of prior buffered content or mediaPlayer was previously playing.
// NOTE: We test for < 1second of data because the media player can stop when there is still
// a few milliseconds of data left to play
boolean atEndOfFile = mediaPlayer.getDuration() - mediaPlayer.getCurrentPosition() <= 1000;
if (wasPlaying || atEndOfFile){
mediaPlayer.start();
}
// Lastly delete the previously playing buffered File as it's no longer needed.
oldBufferedFile.delete();
}catch (Exception e) {
Log.e(getClass().getName(), "Error updating to newly loaded content.", e);
}
}
private void fireDataLoadUpdate() {
Runnable updater = new Runnable() {
public void run() {
if((totalKbRead>19)&&(totalKbRead<120))
textStreamed.setText((totalKbRead-19 + "% Buffering"));//show buffering status.. ie 1%,2%. in ui
else if(totalKbRead<19)
textStreamed.setText(("Connecting..."));
else
textStreamed.setText((""));
float loadProgress = ((float)totalKbRead/(float)mediaLengthInKb);
progressBar.setSecondaryProgress((int)(loadProgress*100));
}
};
handler.post(updater);
}
private void fireDataFullyLoaded() {
Runnable updater = new Runnable() {
public void run() {
transferBufferToMediaPlayer();
// Delete the downloaded File as it's now been transferred to the currently playing buffer file.
downloadingMediaFile.delete();
textStreamed.setText(("Audio full loaded: " + totalKbRead + " Kb read"));
}
};
handler.post(updater);
}
public MediaPlayer getMediaPlayer() {
return mediaPlayer;
}
public void startPlayProgressUpdater() {
float progress = (((float)mediaPlayer.getCurrentPosition()/1000)/mediaLengthInSeconds);
progressBar.setProgress((int)(progress*100));
if (mediaPlayer.isPlaying()) {
Runnable notification = new Runnable() {
public void run() {
startPlayProgressUpdater();
}
};
handler.postDelayed(notification,1000);
}
}
public void interrupt() {
playButton.setEnabled(false);
isInterrupted = true;
validateNotInterrupted();
}
/**
* Move the file in oldLocation to newLocation.
*/
public void moveFile(File oldLocation, File newLocation)
throws IOException {
if ( oldLocation.exists( )) {
BufferedInputStream reader = new BufferedInputStream( new FileInputStream(oldLocation) );
BufferedOutputStream writer = new BufferedOutputStream( new FileOutputStream(newLocation, false));
try {
byte[] buff = new byte[8192];
int numChars;
while ( (numChars = reader.read( buff, 0, buff.length ) ) != -1) {
writer.write( buff, 0, numChars );
}
} catch( IOException ex ) {
throw new IOException("IOException when transferring " + oldLocation.getPath() + " to " + newLocation.getPath());
} finally {
try {
if ( reader != null ){
writer.close();
reader.close();
}
} catch( IOException ex ){
Log.e(getClass().getName(),"Error closing files when transferring " + oldLocation.getPath() + " to " + newLocation.getPath() );
}
}
} else {
throw new IOException("Old location does not exist when transferring " + oldLocation.getPath() + " to " + newLocation.getPath() );
}
}
}
//now copy the below code in activity
StreamingMediaPlayer audioStreamer =
new StreamingMediaPlayer(this,textStreamed,playButton,
streamButton,progressBar);
audioStreamer.startStreaming("your streaming station name",5208, 216);
i think this helps you :)
My project is about Remoting and i want to add a webcam component to it. Here it goes: I have 3 project in my solution... Client, Server, Remote.dll. In Remote.dll is a common class which has methods works in server machine. When i call these methods from Client it executes in server side. So now my question is i put the code of Webcam in remote.dll and it has an event called "video_NewFrame" which it works everytime when webcam catch an image. But i cant reach to the images from my Client side because when code drops to this event it executes infinitely
and my timer in Client side doesnt work as well. I tried to assing image to my global variable but whenever code goes to client and comes to Remote.dll again my variable is null...
How can i reach simultaneously captured images from my client? here is my code:
(i use AForge framework for webcam)
private bool DeviceExist = true;
private FilterInfoCollection videoDevices;
private VideoCaptureDevice videoSource = null;
public bool WebCamStart(int DeviceIndex)
{
if (DeviceExist)
{
videoDevices = new FilterInfoCollection(FilterCategory.VideoInputDevice);
//string myDevice = videoDevices[0].Name;
videoSource = new VideoCaptureDevice(videoDevices[0].MonikerString);
videoSource.NewFrame += new NewFrameEventHandler(video_NewFrame);
CloseVideoSource();
videoSource.DesiredFrameSize = new Size(640, 480);
//videoSource.DesiredFrameRate = 10;
videoSource.Start();
return true;
}
else return false;
}
public Bitmap lastImg;
private void video_NewFrame(object sender, NewFrameEventArgs eventArgs)
{
Bitmap img = (Bitmap)eventArgs.Frame.Clone();
//in executes infinitely when execution comes here and i cant reach from Cliend side...
}
public string getFPS()
{
return videoSource.FramesReceived.ToString();
}
public void CloseVideoSource()
{
if (!(videoSource == null))
if (videoSource.IsRunning)
{
videoSource.SignalToStop();
videoSource.Stop();
videoSource = null;
}
}
public string getCamList()
{
string result = "No Device Found";
try
{
videoDevices = new FilterInfoCollection(FilterCategory.VideoInputDevice);
//comboBox1.Items.Clear();
if (videoDevices.Count == 0)
throw new ApplicationException();
DeviceExist = true;
foreach (FilterInfo device in videoDevices)
{
//comboBox1.Items.Add(device.Name);
result = device.Name;
return result;
}
//comboBox1.SelectedIndex = 0; //make dafault to first cam
}
catch (ApplicationException)
{
DeviceExist = false;
//comboBox1.Items.Add("No capture device on your system");
return "No capture device on your system";
}
return result;
}
// and my client side...
private void timerWebCam_Tick(object sender, EventArgs e)
{
//lblFPS.Text ="Device Running... " + remObj.getFPS() + " FPS";
pictureBox1.Image = remObj.lastImg;
}