My React Native (Expo) app is receiving a stream of raw RTP bytes into a UDP socket. How can I display the data as a video?
This is a follow-up to my previous question, now with more accurate phrasing after making some progress.
Related
Here, Data transfer is for controlling video pause and video record. We are using iMX8Mini Eval board - for streaming video to Android Mobile via USB-OTG. We would like to know, whether video stream does not affected with any command sent over same USB-OTG.
I use getUserMedia to the the audio stream, and pass the stream into web audio using createMediaStreamSource. I then want to stream the raw audio data into a webrtc data channel.
There isn’t a data channel destination node in web audio. I’ve only been able to access the raw audio data from inside an audio worklet, but I don’t know how to get that data into a data channel. How should I go about streaming raw audio from getUserMedia data into a data channel?
I have a device, which can't use WebRTC. For real time audio streaming i am using udp socket. This socket is sending audio chunks. I need to create a playable buffer which can append chunks dynamically, any thoughts about that?
I tried to join this chunks and than save it to file and play, but it is not real time at all :(
this.audioSocket.on('message', (data, rinfo) => {
}
Here we get data (Uint8Array) from our device
Is there any playable memory buffer?
I have a camera that sends mjpeg frames as UDP packets over wifi that I would like to display in my max os-x application. My application is written in objective-c and I am trying to use the AVFoundation classes to display the live stream. The camera is controlled using http get & post requests.
I would like the camera to be recognized as a AVCaptureDevice as I can easily display streams from different AVCaptureDevices. Since the stream is over wifi, it isn't recognized as a AVCaptureDevice.
Is there a way I can create my own AVCaptureDevice that I can use to control this camera and display the video stream?
After much research into the packets sent from the camera, I have concluded that it does not communicate in any standard protocol such as RTP. What I ended up doing is reverse-engineering the packets to learn more about their contents.
I confirmed it does send jpeg images over UDP packets. It takes multiple UDP packets to send a single jpeg. I listened on the UDP port for packets and assemble them into a single image frame. After I have a frame, I then created an NSImage from it and displayed it in an NSImageView. Works quite nicely.
If anyone is interested, the camera is an Olympus TG-4. I am writing the components to control settings, shutter, etc.
when video and audio are made into RTP package ,and then sending-end sends RTP packages to receiving-end. I want to know that which end is this synchronized process made? sending-end or receiving-end ?
Audio and video are sent in separate RTP sessions each with their own random timestamp offset. They are synchronised at the receiver using RTCP Sender Reports (SR) sent by the sender. RTCP SRs map the RTP timestamps of an RTP session (e.g. video or audio) to an NTP timestamp allowing the receiver to synchronise audio and video. This is all specified in RFC3550. The book by Colin Perkins "RTP: Audio and Video for the Internet" has an excellent description of many aspects of RTP including synchronisation.