Screen sharing with WebRTC? - webrtc

We're exploring WebRTC but have seen conflicting information on what is possible and supported today.
With WebRTC, is it possible to recreate a screen sharing service similar to join.me or WebEx where:
You can share a portion of the screen
You can give control to the other party
No downloads are necessary
Is this possible today with any of the WebRTC browsers? How about Chrome on iOS?

The chrome.tabCapture API is available for Chrome apps and extensions.
This makes it possible to capture the visible area of the tab as a stream which can be used locally or shared via RTCPeerConnection's addStream().
For more information see the WebRTC Tab Content Capture proposal.
Screensharing was initially supported for 'normal' web pages using getUserMedia with the chromeMediaSource constraint – but this has been disallowed.
EDIT 1 April 2015: Edited now that screen sharing is only supported by Chrome in Chrome apps and extensions.

You guys probably know that screencapture (not tabCapture ) is avaliable in Chrome Canary (26+) , We just recently published a demo at; https://screensharing.azurewebsites.net
Note that you need to run it under https:// ,
video: {
mandatory: {
chromeMediaSource: 'screen'
}
You can also find an example here; https://html5-demos.appspot.com/static/getusermedia/screenshare.html

I know I am answering bit late, but hope it helps those who stumble upon the page if not the OP.
At this moment, both Firefox and Chrome support sharing entire screen or part of it( some application window which you can select) with the peers through WebRTC as a mediastream just like your camera/microphone feed, so no option to let other party take control of your desktop yet. Other that that, there another catch, your website has to be running on https mode and in both firefox and chrome the users are gonna have to install extensions.
You can give it a try in this Muaz Khan's Screen-sharing Demo, the page contains the required extensions too.
P. S: If you do not want to install extension to run the demo, in firefox ( no way to escape extensions in chrome), you just need to modify two flags,
go to about:config
set media.getusermedia.screensharing.enabled as true.
add *.webrtc-experiment.com to media.getusermedia.screensharing.allowed_domains flag.
refresh the demo page and click on share screen button.

To the best of my knowledge, it's not possible right now with any of the browsers, though the Google Chrome team has said that they're eventually intending to support this scenario (see the "Screensharing" bullet point on their roadmap); and I suspect that this means that eventually other browsers will follow, presumably with IE and Safari bringing up the tail. But all of that is probably out somewhere past February, which is when they're supposed to finalize the current WebRTC standard and ship production bits. (Hopefully Microsoft's last-minute spanner in the works doesn't screw that up.) It's possible that I've missed something recent, but I've been following the project pretty carefully, and I don't think screensharing has even made it into Chrome Canary yet, let alone dev/beta/prod. Opera is the only browser that has been keeping pace with Chrome on its WebRTC implementation (FireFox seems to be about six months behind), and I haven't seen anything from that team either about screensharing.
I've been told that there is one way to do it right now, which is to write your own webcamera driver, so that your local screen appeared to the WebRTC getUserMedia() API as just another video source. I don't know that anybody has done this - and of course, it would require installing the driver on the machine in question. By the time all is said and done, it would probably just be easier to use VNC or something along those lines.

navigator.mediaDevices.getDisplayMedia(constraint).then((stream)=>{
// todo...
})
now you can do that, but Safari is different from Chrome in audio.

it is Possible I have worked on this and built a Demo for Screen share. During this watcher can access your mouse and Keyboard. If he moves his mouse then Your mouse also moves and if he types from his Keyboard, it will be typed into your pc.
View this code this code is for Screen share...
Right now in this days you can share screen with this, you not need any extentions.
const getLocalScreenCaptureStream = async () => {
try {
const constraints = { video: { cursor: 'always' }, audio: false };
const screenCaptureStream = await navigator.mediaDevices.getDisplayMedia(constraints);
return screenCaptureStream;
} catch (error) {
console.error('failed to get local screen', error);
}
};

Related

Does Agora support screen share on Safari?

i dont have a straight answer whether Agora supports screen sharing on SAFARI. This 4x api page does not seem to list Safari at all, and there is some chatter on Stack overflow to that effect (at least for 3.x api) https://docs.agora.io/en/Interactive%20Broadcast/screensharing_web_ng?platform=Web
This is a show stopper for me, so appreciate a straight answer YES or NO whether Agora supports screen sharing on safari
When i tried it, I got a getDisplayMedia error:
"getDisplayMedia must be called from a user gesture handler" on Safari 13+. I do indeed create the new client, join and publish the local video upon an actual user click on a button, so not sure why we get this error. Only happens with screen share, camera/mic work
Looks like you answered your own question on the Agora RTE Dev Slack, I'll relay it here for anyone looking for a solution.
How Sri did it was essentially:
AgoraRTC.createScreenVideoTrack(..).then(() => client.join( ..)

safari 13.1 navigator.mediaDevices.enumerateDevices() return only audio devices

Im facing with an issue on desktop Safari 13.1 version. If I open the console in the web inspector (with a regular macbook which has webcam and mic) and execute this command on any kind of website:
navigator.mediaDevices.enumerateDevices()
First time it will return in the Promise result with a videoinput and an audioinput.
Second time it will return only 2 audioinput. Videoinput is disappear.
Unfortunately I call this method several times while checking the available devices on my solution.
Any idea why does it happen and how could I get the accurate information about the devices even If I call it more than once?
See the results here
I've found the same issue, also on my iPad running iOS 13.
It seems you need to request camera access first in order to see the correct device list.
navigator.mediaDevices.getUserMedia({ video: true })
This will prompt you for access to the camera (you need to be on HTTPS or localhost).
Grant permission, then run this again and you should see the videoinput device(s) listed in the returned promise:
navigator.mediaDevices.enumerateDevices()
I guess this makes sense as a privacy feature that a website cannot check if a camera exists without first asking your permission.

Capybara Selenium Navigate To URL Hangs With Popup Alert on Safari

At the end of my tests Capybara automatically navigates to "about:blank" in order to set up the next test. Sometimes the application I'm testing will throw a popup alert if the user leaves the page (which is expected). I have some code to handle this:
begin
page.driver.browser.navigate.to("about:blank")
page.driver.browser.switch_to.alert.accept
rescue Selenium::WebDriver::Error::NoAlertPresentError
# No alert was present. Don't need to do anything
end
This works fine on Firefox, Chrome, and IE. But for some reason on Safari the navigate command hangs, I assume because of the popup. Anyone know a workaround for this?
There is no simple workaround for this at this time in any version of Selenium language bindings. It is a known issue the Selenium team is not interested in resolving. Fundamentally, it is due to the architecture of Safari and consequently the architecture of the Safari Driver.
The JavaScript of the Safari Driver extension does not know about most of the alerts and popups and dialogs that appear as modal Cocoa layer windows.
It also cannot interact with them.
There is a way but it won't be easy and nobody's done it.
You would need to use Cocoa.
So you would want to use RubyCocoa in this case.
(or PyObjC if you used Python)
You would then possibly also want a sidecar app actually written in Objective-C.
The trick would be to use the AX (Accessibility API) and a separate process to observe if there is an alert as the front window and poke at its labels and buttons' text as visible to the AX APIs.
AX APIs are probably exposed in RubyCocoa via the ScriptingBridge.
However, you would need to add your 'app' to the Security preference pane's list of things allowed to control the computer.
With that, you could detect the window and handle it.
It could be fairly brittle across web sites, but if built well, you could handle expected conditions.
You could try to confirm like this which I believe should work across browsers
# click ok to confirm
page.evaluate_script('window.confirm = function() { return true; }')

Download and display PDF with Trigger.io and jQuery Mobile

We are going to develop a mobile app (iOS and Android), which should provide downloading and storing several user-selected PDF files and viewing them within the app (iOS with WebView) or with an external PDF Viewer (Android).
My Question is: Is this possible with Trigger.io? I didn't find anything concerning this in the official documentation. Can we do something with the file or the child browser / tabs module? If yes, do you have any examples?
Note: We will use Backbone.js and jQuery Mobile as additional libraries.
The Android webview doesn't allow for inline opening of PDFs - you can test that by opening e.g. http://trigger.io/cross-platform-application-development-blog/wp-content/uploads/2012/05/trigger.io-whitepaper.pdf in your stock browser.
On iOS, you can use:
forge.tools.getURL('my_file.pdf',
function (file) {
forge.tabs.open(file);
}
);
But that won't work on Android (tested on 2.3, 3 and 4.0).
Short answer - yes.
Downloading, storing, then showing/referencing later can definitely be done. Check out the forge.file documentation.
I have not tried the child browser feature yet. Although the tabs module will work, I think its best to just let the device (and its user settings) decide how to display/render the PDF. I am only saying this because my devices (especially the Android ones... 2.3 and 4.x) tend to behave differently. Either way... the device's "back button" always gets me back to where I left off in my trigger.io app.

Safari 6's New Developer Toolbar doesn't show Form Data in XHR/AJAX Requests

Safari recently went to version 6 (Lion/Mtn Lion) and they've changed over from the standard webkit dev tools to one that's much more XCode looking, my problem other than the OCD of not liking things change is that in the resource tab (or anywhere you can track down the DataService.aspx/AJAX calls) I can no longer see the form data that I am passing.
Can anyone point me to where I can find that data so I don't have to console out my params when I'm testing new data service/backend calls?
I've logged a bug with Apple, they've marked it as a duplicate so hopefully they've received enough requests to fix this, until then I'm continuing to use chrome as the webkit developer is the same as safari's old version.
You can find this info in Instrument tab (stop-watch icon). In the left sidebar thers Timelines row, click the grey circle (record button) on the right. Then click to Network Requests where you see all reqs, and you have to click small icon on the right oc request to display response headers and all form data are available in right panel. Panel can be hidden same as left one (in case you dnt see it).
Unfortunately there are no query pamaters listed, according to this disscussion. I belive its a bug in safari
Edit 15.May 2013: This bug was fixed in Safari 6.0.3.
As far as I can tell, there's no way to show the request parameters.
This goes even further. I can't see the JSON response data either (no clickable arrows to show the containing Javascript objects within the JSON, just pure text)
I think we have to switch to Firefox /w Firebug or regular Webkit in order to get XHR monitoring...
Guys if you want to see post data in safari 6 which is not possible right now, install the firebug lite extension and there you go you have the post data.
I used it and it works great with safari 6
Actually the request headers, response headers and query parameters are in the details sidebar on the right when using the resources view or if you click to see the content of a request in the Timelines/Network Requests view. Took me a few minutes to find that too.
If you need to see what the device is actually sending and your server is on a Windows Machine I use http://www.Wireshark.org and check on the server side of things. No interpretation by any WebKit stuff and very valuable (such as issue with iOS and the 'Blob' data). Similar network snooping should exist on Mac as well.