What determines which capabilities are enumerated by the MediaStreamTrack.getCapabilities() method? - getusermedia

I have been working on an application of getUserMedia and WebRTC that requires me to make use of information returned by the getConstraints(), getCapabilities(), and getSettings() methods of a MediaStreamTrack.
In testing, using getCapabilities() on the track corresponding to an external webcam might return something like...
aspectRatio: {max: 1920, min: 0.000925925925925926}
deviceId: ""
exposureCompensation: MediaSettingsRange {max: 2047, min: 3, step: 1}
exposureMode: (2) ["manual", "continuous"]
facingMode: []
focusMode: (2) ["manual", "continuous"]
frameRate: {max: 30, min: 1}
groupId: ""
height: {max: 1080, min: 1}
whiteBalanceMode: ["manual"]
width: {max: 1920, min: 1}
zoom: MediaSettingsRange {max: 5, min: 1, step: 1}
...but if focusDistance or colorTemperature (or other potential members listed here) are not in the capabilities list, then what determines the controllable settings for a camera or other media device?
Would the answer to this also apply to mobile devices, such as Android or iOS tablets or phones, that have integrated cameras and microphones?

The capabilities returned by track.getCapabilities() are the intersection of four things:
The track source. getUserMedia and getDisplayMedia tracks have different capabilities.
The track.kind. E.g. audio capabilities are not returned for video tracks, and vice versa.
The capabilities supported by the browser.
The capabilities supported by the device.
You learn what capabilities the browser supports like this:
console.log(navigator.mediaDevices.getSupportedConstraints());
You then typically rely on documentation and specification to know which capabilities apply to which method and kind.
Newer versions of browsers tend to support more capabilities. Not all browsers support getCapabilities() yet. Firefox, for instance, does not yet, even though it otherwise supports constraints.
For this reason, it is always best to check what's supported—as shown above—before relying on a capability.
To the specific capabilities you mention, focusDistance and colorTemperature, they are defined for tracks returned by getUserMedia, but defined in the mediacapture-image extension spec, so they should show up if they're supported. Their absence means they're not supported.

Related

sendDevToolsCommand in Selenium 4 alpha

According to this article, Selenium 4 alpha has a sendDevToolsCommand that sends an arbitrary DevTools command to the browser and returns a promise that will be resolved when the command has finished:
Added “sendDevToolsCommand” and “setDownloadPath” for chrome.Driver.
But I can't seem to find how to use it. It sounds a bit like using JavaScript executor in Selenium.
Can someone provide an example usage? I'm using Selenium + Java.
The command to call the devtool api was added a few years back in the Chrome driver.
You can already use it with Selenium even if the method is not yet present:
Take full page screenshot
Print To PDF
Inject some Javascript before the page loads
Block a network URL
Save/restore the cookies for all domains
Get transparent screenshot
This command gives you access to the devtools api, which is used by ChromeDriver internally to drive the browser.
The method takes the name of the command as first argument and a dictionary of parameters as second argument. To figure out how to call a command, add puppeteer in your searches. For instance puppeteer set download location.
Note that executeCdpCommand is implemented in the Java master branch, so it should be available in the next release.
I couldn't find the sendDevToolsCommand in the Selenium documentation yet, but the source actually has the setDownloadPath that you also mentioned above defined right below, which actually uses the sendDevToolsCommand. Based on that usage, it seems like you could do something like:
const { Builder } = require("selenium-webdriver");
const driverInstance = await new Builder()
.withCapabilities({ browserName: "chrome" })
.build();
driverInstance.sendDevToolsCommand('Page.setDownloadBehavior', {
behavior: 'allow',
downloadPath: path
})
or for a visually obvious example:
await driverInstance.sendDevToolsCommand("Emulation.setDefaultBackgroundColorOverride", {
color: { r: 0, g: 255, b: 0, a: 1 } // watch out, it's bright!
});
where the first argument is a Chrome Devtools Protocol Domain method (e.g. or Page.setDownloadBehavior or Emulation.setCPUThrottlingRate) and the second argument is an object containing the options for that Domain method (as described in the same protocol docs).
Edit: just tested and the above works :)
I'm excited that this was added because it means that, in addition to network throttling, it should be pretty trivial to add cpu throttling to Selenium tests now! Something like:
driverInstance.sendDevToolsCommand('Emulation.setCPUThrottlingRate', {
rate: 4 // throttle cpu 4x
}
Selenium 4 release will have a user friendly API for Chrome DevTools protocol.
I just finished implementing Network and Performance domains for the Selenium Java client.
https://github.com/SeleniumHQ/selenium/pull/7212
In addition, there is a generic API for all domains in Java client that was merged a while ago.
All those new features will be released probably in the next Alpha release.

How do I change frequency of telemetry of Azure IoT Edge simulated tempSensor module

I would like to change the frequency of the simulated tempSensor module so that it sends telemetry to the IoT Hub every 60 seconds, not every 5 seconds. I cannot find the code in order to create my own C# module or to modify the one that Azure has as a sample. How do I change the frequency of the telemetry of this simulated temperature sensor module used in the IoT Edge walkthrough?
https://hub.docker.com/r/microsoft/azureiotedge-simulated-temperature-sensor/
I have an Azure Iot Hub with 8000 messages per day and Iot Edge device simulated and a tempSensor and filterModule through Docker setup etc... All is working well.
I could write another C# module similar to the filterModule which would function like a time filter and averages 12 readings together and sends a message every 60 seconds and then the filterModule does it's job after. So the route would go from tempSensor to timeFilter to filterModule. Along this line of thinking.
SIMULATED TEMPERATURE SENSOR in Docker
https://azure.microsoft.com/en-us/resources/samples/?service=iot-hub&sort=0&term=simulated
Where do I find the code for that example? I looked here:
https://github.com/Azure-Samples
I am trying to learn how the simulated temperature sensor module works in the code in order to build off that to create my own IoT modules.
I found something close but am not sure how to go between this code and the modules used in IoT Edge.
https://learn.microsoft.com/en-us/dotnet/api/overview/azure/iot?view=azure-dotnet
I don't see API reference here:
https://learn.microsoft.com/en-us/azure/iot-edge/
Here are some other places I looked:
https://marketplace.visualstudio.com/items?itemName=vsciot-vscode.azure-iot-edge
https://marketplace.visualstudio.com/items?itemName=vsciot-vscode.azure-iot-toolkit
The source code is not currently open source.
Update:
Currently (April 2018) the code is open-sourced under the MIT licence and can be found on GitHub here:link
To solve your immediate need, there is an undocumented setting for adjusting the delay. All the available settings for the Simulated Temperature sensor module (and default values):
"MessageDelay": "00:00:05",
"machineTempMin": 21,
"machineTempMax": 100,
"machinePressureMin": 1,
"machinePressureMax": 10,
"ambientTemp": 21,
"ambientHumidity": 25
This may be set as environment variables in the "createOptions" section, like this:
"tempSensor": {
"version": "1.0",
"type": "docker",
"status": "running",
"restartPolicy": "always",
"settings": {
"image": "microsoft/azureiotedge-simulated-temperature-sensor:1.0-preview",
"createOptions": "{\"Env\":[\"MessageDelay=00:01:00\"]}"
}
}

Values returned by webdrivers

After performing a search using POST /session/{session id}/element, I get this from the Chrome webdriver:
{ sessionId: '3241e7da289f4feb19c1f55dfc87024b',
status: 0,
value: { ELEMENT: '0.12239552668870868-1' } }
Is this what the specs demand?
I am asking because I couldn't find anywhere a spot where it said clearly "ELEMENT" in capital letters. All I can find in the specs is that a key called value is set (which it is: it's set as { ELEMENT: '0.12239552668870868-1' }
Can I always always expect this response, from other browsers' webdrivers? That is, are status and sessionId always returned?
Is that { ELEMENT: '0.12239552668870868-1' } the way chromium makes up the object? Or is this true for any webdrivers? Of not, what do other webdrivers return?
The WebDriver-W3C Candidate Recommendation clearly mentions the following points:
The Find Element command is used to find an element in the current browsing context that can be used for future commands.
Let location strategy be the result of getting a property called "using".
Let selector be the result of getting a property called "value".
The result of getting a property with argument name is defined as being the same as the result of calling Object.GetOwnProperty(propertyName).
GetOwnProperty(propertyName) in ECMAScript® Language Specification is defined as :
String objects use a variation of the GetOwnProperty internal method used for other native ECMAScript objects. This special internal method provides access to named properties corresponding to the individual characters of String objects.
So invoking GetOwnProperty are internal method used for other native ECMAScript objects and are resolved within the internal scope of the Browser Drivers and Browser Clients.
Mozilla have well documented Object.getOwnPropertyNames() and getOwnPropertyDescriptors().
Browser Specific Implementation
I did a small test with all the info you have provided with Search Box of Google Home Page i.e. https://www.google.co.in with all the major variants of WebDrivers and here is the result :
ChromeDriver - OSS :
[[ChromeDriver: chrome on XP (0d24fd038bde751b1e411711271c3e69)] -> name: q]
[[ChromeDriver: chrome on XP (0d24fd038bde751b1e411711271c3e69)] -> name: q]
FirefoxDriver - W3C :
[[FirefoxDriver: firefox on XP (e7a56813-97c5-466e-9c35-24c9f89af6ed)] -> name: q]
[[FirefoxDriver: firefox on XP (e7a56813-97c5-466e-9c35-24c9f89af6ed)] -> name: q]
InternetExplorerDriver - W3C :
[[InternetExplorerDriver: internet explorer on WINDOWS (367257db-cdbc-4be7-aeac-805a21ad9d2d)] -> name: q]
[[InternetExplorerDriver: internet explorer on WINDOWS (367257db-cdbc-4be7-aeac-805a21ad9d2d)] -> name: q]
So as you can observe from the field details of the concerned value field returned is in similar pattern and till the WebDriver variant passes the correct reference to user it shouldn't be a obstacle.
Finally, it is worth to mention at this point of time that like FirefoxDriver and InternetExplorerDriver (both being W3C compliant), ChromeDriver is almost W3C compliant and may vary from a few functional aspects.
Update A
As per your question and update, you are pretty right about ChromeDriver and Chrome communication protocol. Getting more granular we can find some difference in the webdriver call as follows :
Firefox :
1516626575533 webdriver::server DEBUG <- 200 OK {"value":{"element-6066-11e4-a52e-4f735466cecf":"6e35faa4-233f-400c-a6c7-6a66b54a69e5"}}
So, Firefox Browser returns :
"value":{"element-6066-11e4-a52e-4f735466cecf":"6e35faa4-233f-400c-a6c7-6a66b54a69e5"}
Chrome :
[14.921][DEBUG]: DEVTOOLS RESPONSE Runtime.evaluate (id=25) {
"result": {
"type": "object",
"value": {
"status": 0,
"value": {
"ELEMENT": "0.7086986861512812-1"
}
}
}
}
So, Chrome Browser returns :
"value": {"ELEMENT": "0.7086986861512812-1"}
What matters most to we user is the value of the element returned by the browser object which is always referred by an user and correctly identified by the webdriver instance. All these inner logic becomes abstract to the end user.
Update B
Adding some significant bytes from #FlorentB. 's comments :
The earlier versions of Selenium i.e. Selenium v2.x used the keyword ELEMENT to store the reference of a DOM Tree element. This key was changed within the recent versions of Selenium i.e. Selenium v3.x to element-6066-11e4-a52e-4f735466ce. Most of the implementation of the current ChromeDriver is still from the Selenium 2.x specifications.
I have just encountered this same issue, and found the change was made around 3.5 of the Selenium server and related images.
I found this comment the most specific to understand the change and identify which version it changed in:
https://github.com/SeleniumHQ/selenium/issues/4773#issuecomment-333092149
I am using Docker images like selenium/node-firefox:3.4.0-actinium, and have found v3.4.0 returns the ELEMENT key from the older JSonWire spec, whereas v3.9 returns the format element-6066-11e4-a52e-4f735466cecf from the new WebDriver spec. (I haven't checked any other versions in between).
It's part of their gradual migration to WebDriver, but it is a bit confusing that they did this breaking change at 3.5 (or thereabouts) and not v3.0.0 which I think everyone would have been OK with.
Also there's a mix of implementations in the "native" drivers like Gecko which is produced by the Firefox team now, and Chrome, as they will have different development roadmaps.
Furthermore, I've found the client-side library I'm using hasn't even implemented the new response yet, so I'll have to hang back for a while (or patch and PR it myself). I've seen similar conversations in other clients (like the Java client 2 years ago).
You can see the differences between the two protocols' definitions of the Element response:
https://github.com/SeleniumHQ/selenium/wiki/JsonWireProtocol#webelement-json-object
https://www.w3.org/TR/webdriver/#elements

Preloading audio formats for SoundJS

I am setting up a manifest to load a number of images and an audio file that will be played back using SoundJS.
I understand the concept of using createjs.Sound.alternateExtensions to play the supported audio format, but I can't tell if I need to preload both file formats as part of the load manifest. Obviously, this would be undesirable since only one file will be used for playback.
Is it necessary to load both?
manifest = [
{
src: "./assets/voiceover.mp3",
id: "vo"
}, {
src: "./assets/voiceover.ogg",
id: "vo"
}];
Just specify it once in the list. The "alternateExtensions" list is what SoundJS will try, depending on the current browser support.
For example, if you specify a "file.mp3", and the current browser (eg, Opera) doesn't support mp3, then it will swap out the extension for one it does support. You do not have to specify "file.ogg", it will do that for you.
This approach assumes you have a .ogg file in the same folder as the mp3. If you do not, consider using a complex "src" object, which has full paths to each file.
One note: If you are relying on alternateExtensions, it is recommended to define the file as ".ogg", and put the "mp3" and other extensions as alternates. This is because certain versions of Firefox report that it supports MP3, but will fail. Starting with ogg will get around this.

Sony A6000 failing to set exposure mode

I'm attempting to set the exposure mode of a Sony A6000 using smart remote version 2.0.1 by posting the following command: {'method': 'setExposureMode', 'params': ['Shutter'], 'id': 12, 'version': '1.0'}, but I'm getting back the following response: {u'id': 12, u'error': [500, u'Set operation failed.']}. I can't seem to find a description for error code 500 anywhere in the API documentation. I've used the same command to successfully set the exposure modes of NEX-5, NEX-6, and A5000 cameras. Is there something different I need to be doing for the A6000?
Nothing different as far as I am aware of. Please check code and other general things like current camera mode, also if camera has latest firmware, latest "Smart Remote Control" app, etc,.
Best Regards, Prem, Developer World team