public class WowzaTest extends ModuleBase {
public void onConnect(IClient client, RequestFunction function, AMFDataList params) {
// how do i get stream name in this method or this class
}
}
The player is wowza's test player. Rtmp request is rtmp://myip:1935/play/myStream.
You cannot get stream name in onConnect because RTMP streaming is actually splitted into 2 parts:
connect to RTMP url (onConnect will happen here), where RTMP url is
rtmp://ip/app[/appinst]
send play command to play some stream
(in your case appinst is not present in your URL, so _definst_ will be used instead automatically)
you may want to handle play command:
http://www.wowza.com/forums/content.php?150-How-to-override-play-to-remap-a-stream-name
or do smth like this:
http://www.wowza.com/forums/content.php?329-How-to-use-IMediaStreamActionNotify3-interface-to-listen-for-RTMP-stream-events-%28includes-codec-info%29
Related
I need to implement a server that gets video from some source for example IPCamera
then preprocess the image and streams it down to the client (if requested).
I already implemented the part with processing, it accepts a single frame (bitmap) and returns the processed bitmap. What I'm struggling with is the part of receiving video from the camera and then streaming it to the client.
What would be the right way to do it?
What libraries do you recommend using?
I use ASP.NET Core for the Server part, and Angular/React for the Client.
I tried to implement gRPC but a gRPC-Web client for typescript seems to be a pain in the ass.
Edit: 02.08.2022
What I achieved so far:
I figured out how to receive image output from the camera.
I found out RTSP Client for C#. Source: C# RTSP Client for .NET
It works pretty fine. I can receive output with small to no delay, and I use my phone to emulate the RTSP camera/server.
So RTSP Client receives raw frames (in my case H.264 IFrame/PFrame). The problem is I need to decode those frames preferably to Bitmap because I use YoloV4 ONXX Model for object detection.
Here's how I set up YoloV4 with ML.Net. Source: Machine Learning with ML.NET – Object detection with YOLO
To decode raw frames I use FFMpeg (sadly I didn't find any working FFMpeg package that would work with .NET Core, I tried AForge.Net, Accord but in both packages, the FFMPEG namespace is missing after installing for some reason, so I dug through Github and took this project FrameDecoderCore). It's not the best solution but it works. Now I can receive the output and decode it to Bitmap.
Now I'm facing three major issues:
How to detect objects without delaying the process of receiving camera output. And how to properly build an onnx model just to predict without training.
How to convert processed bitmaps back to a video stream. I also need to be able to save part of it as a video file on disk (video format doesn't matter) whenever the desired object was detected.
How to stream processed or unprocessed output to the client when the client wants to see the camera output. - I'm thinking of gRPC here and sending bitmaps and then displaying it on HTML Canvas.
Here's how my service looks at the moment:
public class CCTVService : BackgroundService
{
private readonly RtspClient _rtspClient;
private readonly ILogger<CCTVService> _logger;
private const int streamWidth = 480;
private const int streamHeight = 640;
private static readonly FrameDecoder FrameDecoder = new FrameDecoder();
private static readonly FrameTransformer FrameTransformer = new FrameTransformer(streamWidth, streamHeight);
public CCTVService(ILogger<CCTVService> logger)
{
_logger = logger;
_rtspClient = new RtspClient(new ConnectionParameters(new Uri("rtsp://192.168.0.99:5540/ch0")));
}
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
using (_rtspClient)
{
try
{
await _rtspClient.ConnectAsync(stoppingToken);
_logger.LogInformation("Connecting to RTSP");
}
catch(RtspClientException clientException)
{
_logger.LogError(clientException.Message);
//throw;
}
_rtspClient.FrameReceived += (obj, rawFrame) =>
{
if (rawFrame is not RawVideoFrame rawVideoFrame)
return;
var decodedFrame = FrameDecoder.TryDecode(rawVideoFrame);
if (decodedFrame == null)
return;
using var bitmap = FrameTransformer.TransformToBitmap(decodedFrame);
_logger.LogInformation($"Timestamp: {new DateTimeOffset(rawFrame.Timestamp).ToUnixTimeSeconds()} Timestamp-diff: {new DateTimeOffset(DateTime.Now).ToUnixTimeSeconds() - new DateTimeOffset(rawFrame.Timestamp).ToUnixTimeSeconds()}");
// save bitmaps | Test
//var t = new Thread(() =>
//{
// using var bitmap = FrameTransformer.TransformToBitmap(decodedFrame);
// var name = "./test/" + new DateTimeOffset(rawFrame.Timestamp).ToUnixTimeMilliseconds().ToString() + " - " + new Random().NextInt64().ToString() + ".bmp";
// bitmap.Save(name);
//});
//t.Priority = ThreadPriority.Highest;
//t.Start();
};
try
{
await _rtspClient.ReceiveAsync(stoppingToken);
}
catch
{
// swallow
}
}
}
}
So I can't really help with part 2 and 3 of your question but with ML.NET, one of the things you might consider is batching the predictions. Instead of preprocessing them one at a time, you could collect 10-20 frames and then instead of using PredictionEngine, use the Transform passing it in an IDataView instead of a Bitmap.
Here are some samples of using ONNX models inside applications. The WPF sample might be of interest since it uses a webcam to capture inputs. I believe it uses the native Windows APIs, so different than how you'd do it for web but it might be worth looking at anyway.
https://github.com/dotnet/machinelearning-samples/tree/main/samples/csharp/end-to-end-apps/ObjectDetection-Onnx
I should send and wait for response using an Udp server from a Play controller. Where is the correct place to create de Actor? Local Variable or Object property?
This question come to my mind becouse working with play, when I change code in the actor play recompiles (I'm using play debug ~run), but changes are not applied, so I think I may be doing something wrong.
My actual code:
public class Application extends Controller {
static final ActorRef server = Akka.system().actorOf(Props.create(Listener.class));
public static Result index() {
// ActorRef sender = Akka.system().actorOf(Props.create(SimpleSender.class));
I run a web service where I convert a file from one file format into another. The conversion logic is already functioning but now, I want to query this logic via Jersey. Whenever file upload via Jersey is addressed in tutorials / questions, people describe how to do this using multipart form data. I do however simply want to send and return a single file and skip the overhead of sending multiple parts. (The webservice is triggered by another machine which I control so there is no HTML form involved.)
My question is how would I achieve something like the following:
#POST
#Path("{sessionId"}
#Consumes("image/png")
#Produces("application/pdf")
public Response put(#PathParam("sessionId") String sessionId,
#WhatToPutHere InputStream uploadedFileStream) {
return BusinessLogic.convert(uploadedFile); // returns StreamingOutput - works!
}
How do I get hold of the uploadedFileStream (It should be some annotation, I guess which is of course not #WhatToPutHere). I figured out how to directly return a file via StreamingOutput.
Thanks for any help!
You do not have to put anything in the second param of the function; just leave it un-annoted.
The only thing you have to be carefull is to "name" the resource:
The resource should have an URI like: someSite/someRESTEndPoint/myResourceId so the function should be:
#POST
#Path("{myResourceId}")
#Consumes("image/png")
#Produces("application/pdf")
public Response put(#PathParam("myResourceId") String myResourceId,
InputStream uploadedFileStream) {
return BusinessLogic.convert(uploadedFileStream);
}
If you want to use some kind of SessionID, I'd prefer to use a Header Param... something like:
#POST
#Path("{myResourceId}")
#Consumes("image/png")
#Produces("application/pdf")
public Response put(#HeaderParam("sessionId") String sessionId,
#PathParam("myResourceId") String myResourceId,
InputStream uploadedFileStream) {
return BusinessLogic.convert(uploadedFileStream);
}
I want to send an image from java server (Restful Jax-rs).
My client is Android.
#GET
public Response getUserImage() {
byte[] image =new byte[1024];
return Response.ok(image, MediaType.APPLICATION_OCTET_STREAM).header("content-attachment; filename=image_from_server.png") .build();
But here one download box is coming.
So I want to download without download box, when I run the request URL on browser it should open automatically.
Thanks.
I believe that's because you specified application/octet-stream.
I think you should use image/jpeg or image/png.
#GET
#Produces({"image/png"})
public Response getUserImage() {
final byte[] image = ...;
// say your image is png?
return Response.ok().entity(new StreamingOutput(){
#Override
public void write(OutputStream output)
throws IOException, WebApplicationException {
output.write(image);
output.flush();
}
}).build();
}
You can encode the image with base 64 encoding, wrap the base 64 encoded string in xml or json and send that over rest. On the rest client extract the base 64 encoded string and decode it to get the final image. This preserves all the file meta data as well and but the only down side is 30% increase in image size.
I have tried the same with Restlet rest api, am pretty sure its possible over JAX-RS as well.
I tring to stream shoutcast stream in my window phone 7 app
I start an async HttpWebRequest like this
//Init Request
HttpWebRequest myHttpWebRequest = (HttpWebRequest)WebRequest.Create("http://ACommonoShoutCastUrl:8000");
myHttpWebRequest.Headers["Icy-MetaData"] = "1";
myHttpWebRequest.UserAgent = "WinampMPEG/5.09";
myHttpWebRequest.AllowReadStreamBuffering = true;
//Call
// Create an instance of the RequestState and assign the previous myHttpWebRequest object to its request field.
RequestState myRequestState = new RequestState();
myRequestState.request = myHttpWebRequest;
// Start the asynchronous request.
IAsyncResult result = (IAsyncResult)myHttpWebRequest.BeginGetResponse(new AsyncCallback(RespCallBack), myRequestState);
The problem is that the CallBack->RespCallBack is never called...
This code worked for me normally in other environments but not on the phone...
I tired also to use WebClient that seems to stream data,
the problem in this case is that it never call the end OpenReadCompleted because of endelss shoutcast stream
Thanks for support
any help would be appreciated
SHOUTcast implements its own protocol so you can't directly access and play it. You can use DownloadStringAsync (you will need a WebClient instance for this) to download the PLS file and read the URL with the help of RegEx.
When you get the URL, you can read the raw audio data by implementing MediaStreamSource and then use a MediaElement to play the contents.
You can find a sample implementation of MediaStreamSource here.
I just put the following on a page and the callback was called in repsonse to the button click. (I set a break point on the throw statement and it was hit.)
private HttpWebRequest myHttpWebRequest;
public MainPage()
{
InitializeComponent();
}
private void Button_Click(object sender, RoutedEventArgs e)
{
//Init Request
//The following URI was chosen at random
myHttpWebRequest = (HttpWebRequest)WebRequest.Create("http://yp.shoutcast.com/sbin/tunein-station.pls?id=1377200");
myHttpWebRequest.Headers["Icy-MetaData"] = "1";
myHttpWebRequest.UserAgent = "WinampMPEG/5.09";
myHttpWebRequest.AllowReadStreamBuffering = true;
// Start the asynchronous request.
myHttpWebRequest.BeginGetResponse(RespCallBack, myHttpWebRequest);
}
private void RespCallBack(IAsyncResult ar)
{
throw new NotImplementedException();
}
I am trying to implement the Shoutcast streaming to my MediaElement via my own MediaStreamSource class. I have a loop in which I am downloading data from Shoutcast server and then set the data to the MediaStreamSource class - this works not perfectly for yet, but I discovered another more important issue. I made a test. I have downloaded a stream to a mp3 (stream is in mp3) file, then put this file to my application and set it to my MediaStreamSource. Here's the code for this:
private void Button_Click(object sender, RoutedEventArgs e)
{
var assembly = System.Reflection.Assembly.GetExecutingAssembly().GetManifestResourceNames();
var res = System.Reflection.Assembly.GetExecutingAssembly().GetManifestResourceStream("Demo1.sample.mp3");
byte[] data = new byte[res.Length];
res.Read(data, 0, data.Length);
MemoryStream ms = new MemoryStream();
ms.Write(data, 0, data.Length);
ms.Position = 0;
ShoutcastMediaStreamSource ss = new ShoutcastMediaStreamSource(ms);
player.SetSource(ss);
}
my ShoutcastMediaStreamSource based on MenagedMediaHelpers. So when I put this stream to my ShoutcastMediaStreamSource in debbuger I can see that the method OpenMediaAsync() is called properly, then the GetSampleAsync() is called in a loop by MediaElement, here also everything is fine, but when I run this app there is no sound ! Neither on the emulator nor on the device, and there are no errors. I think, the GetSampleAsync() method is running too fast, bacause for the file (and also stream) duration is about 30 sec., and this app ends after ca. 10 sec. But nevertheless, there should be a sound (scratch).
BUT whats suprising - this app works in Silverlight as Web Page ! The music is playing. I am confused.
Here is allso an app
http://timheuer.com/blog/archive/2010/08/16/download-and-store-media-for-playback-in-windows-phone-7-using-mediastreamsource.aspx
and there is a comment:
If Mp3MediaStreamSource is set as a
source for MediaElement then
MediaElement doesn't play that file
and donesn't show any error in Windows
phone 7 sdk RTM version. In previsios
version it was working but it's not
working with Windows phone 7 sdk final
release.
The callback gets called if you disable read stream buffering:
webRequest.AllowReadStreamBuffering = false;