I am trying to implement perfect WebRTC negotiation for my small video conferencing application by considering the examples from the following page:
https://blog.mozilla.org/webrtc/perfect-negotiation-in-webrtc/
Unortunately I did not manage to make it fully work, especially mobile safari seems to handle rollback behavior its own way, here is the code that handles the rollback behavior:
if (description) {
const offerCollision = description.type == 'offer' && (makingOffer || pc.signalingState != 'stable');
this.ignoreOffer = !this.polite && offerCollision;
if (this.ignoreOffer) {
return;
}
if (offerCollision) {
await Promise.all([pc.setRemoteDescription({ type: 'rollback' }), pc.setRemoteDescription(description)]);
So when on mobile safari an offer collision (offerCollision === true) is detected and pc.setRemoteDescription({ type: 'rollback' }) is called as it's implemented in my code, it throws an error of type InvalidStateError. Taking a closer look at the documentation about this type of error in MDN (https://developer.mozilla.org/en-US/docs/Web/API/RTCPeerConnection/setRemoteDescription#Browser_compatibility) shows:
"The RTCPeerConnection is closed, or it's in a state which isn't compatible with the specified description's type. For example, if the type is rollback and the signaling state is one of stable, have-local-pranswer, or have-remote-pranswer, this exception is thrown, because you can't roll back a connection that's either fully established or is in the final stage of becoming connected."
Checking the peer connections signaling state just before rolling back shows that it is in the state "have-local-offer" which should be ok since MDN says that rollback is not possible (throws InvalidStateError) in the states stable, have-local-pranswer, or have-remote-pranswer.
For the other case when my Desktop Chrome browser runs in an offer collision everything just works as expected with the same signaling state just before rollback is initiated.
Does someone here have an idea what's potentially wrong or to be handled differrently for mobile Safari.
As mentioned in the comments, Safari (both iOS/mobile and macOS) has a known bug with { type: 'rollback' }. It also does not yet support optional descriptions in setLocalDescription/setRemoteDescription, the latest perfect negotiation recommendation in the spec.
This can be fixed by discarding the colliding peer connections and retrying (as suggested). To do this, handle the error when setting a remote description with the following steps:
Reset any state variables (e.g. makingOffer, isSettingRemoteAnswerPending)
Close the peer connection by calling peerConnection.close()
Tear down any peer connection event listeners (e.g. negotiationneeded, icecandidate, track)
Signal to the other peer to do the same e.g. via websocket messaging
On receiving the signal to restart, the other can go through the same steps, and finally create a new offer to kick off the negotiation process again.
An example of this flow can be seen in this demo.
Good news: Safari Tech Preview (15.4) supports implicit rollback in setRemoteDescription directly¹, which appears to also have fixed the explicit {type: 'rollback'} method you were using in the more backwards-compatible polyfill (though I've asked for clarification in the bug).
Bad news: At the time of writing, it's still not available in regular Safari (at 15.3 now) or Safari for iOS. But that should be only a matter of time.
Implicit rollback was designed specifically for perfect negotiation, letting you use the standard pattern directly. Here's a fiddle using it.
1. I've submitted a PR to update MDN to reflect this.
In case anyone finds themselves back here, I found that I could make perfect negotiation work with Safari if I make 2 changes:
in onnegotiationneeded: if we are on safari, don't do the createOffer() or setLocalDescription() if the instance is polite.
on incoming messages, in addition to the existing 'ignore offer' functionality, ignore type=='offer' messages if the instance is NOT polite and we are running on safari.
My assumption is that Safari webkit doesn't like being presented with offers if it's made an offer to the peer. I don't have anything to back that up with beyond the fact it seems to work - the errors disappear and I get video.
Related
I have been working on a project using the ESP32 with the ESP-IDF that will check it's NVS memory for wifi credentials before starting the network stack. If it has said credentials, it will connect to the wifi network in STA mode, if it lacks them, it will launch as it's own AP to allow the user to send it the credentials over HTTP.
After manually putting my test credentials into NVS, I started working on the AP code. Once all the AP code and logic was complete, I manually wiped the flash memory with esptool to force the board to launch in that mode. Doing so worked fine, and I was able to send it the updated credentials over HTTP.
At this point, the board attempted to connect as STA upon reset, however, the SYSTEM_EVENT_STA_WPS_ER_PIN event kept being caught by the wifi event loop. The board has since only experienced this event and has been completely unable to connect to wifi since. To make matters stranger, even after rolling back to a previous version with git, the problem still persists.
main.c
void app_main() {
// Start NVS
initNVS();
// Init Wifi Controller
initWifiController();
// Get Credentials to send to wifi
Creds creds = getCreds();
// Start wifi in STA mode with gathered creds
beginWifi(creds);
initializePins();
initializeTimers();
}
wifiController.c
void initWifiController(){
// * NVS must be initialized before wifi work can be done
// Handle when connected to the network
connectionSemaphore = xSemaphoreCreateBinary();
// Begin network stack
ESP_ERROR_CHECK(esp_netif_init());
// Create event loop for handling callbacks
ESP_ERROR_CHECK(esp_event_loop_create_default());
}
void beginWifi(Creds creds){
if(creds.status == ESP_OK){
ESP_LOGI(TAG, "Connection credentials have been found, connecting to network");
connectSTA(creds);
}
else if(creds.status == ESP_ERR_NVS_NOT_FOUND){
ESP_LOGW(TAG, "Missing credentials, starting as AP");
connectAP();
}
else{
ESP_LOGE(TAG, "ESP failed with error %s, not starting wifi", esp_err_to_name(creds.status));
}
void connectSTA(Creds creds){
ESP_LOGI(TAG, "Attempting to connect to wifi with following creds: %s | %s", creds.ssid, creds.pass);
// Set netif to sta
esp_netif_create_default_wifi_sta();
// Prepare and initialize wifi_init_config_t
wifi_init_config_t wifi_init_config = WIFI_INIT_CONFIG_DEFAULT();
ESP_ERROR_CHECK(esp_wifi_init(&wifi_init_config));
// Register tracked events for event_handler
ESP_ERROR_CHECK(esp_event_handler_register(WIFI_EVENT, ESP_EVENT_ANY_ID, event_handler, NULL));
ESP_ERROR_CHECK(esp_event_handler_register(IP_EVENT, IP_EVENT_STA_GOT_IP, event_handler, NULL));
// TODO: Check if this can be used to avoid havng to use NVS manually
esp_wifi_set_storage(WIFI_STORAGE_RAM);
// Config struct for wifi details
wifi_config_t wifi_config = {};
// Copy casted info into wifi_config
// * https://www.esp32.com/viewtopic.php?f=13&t=14611
// * See above link for details on this
strcpy((char *)wifi_config.sta.ssid, creds.ssid);
strcpy((char *)wifi_config.sta.password, creds.pass);
esp_wifi_set_config(ESP_IF_WIFI_STA, &wifi_config);
ESP_ERROR_CHECK(esp_wifi_start());
// ? Is this required to avoid a memory leak?
free(creds.pass);
free(creds.ssid);
}
void connectAP(){
// ? How important is it that these be called in this order?
ESP_LOGI(TAG, "Starting in AP Mode");
esp_netif_create_default_wifi_ap();
// TODO: maybe move this creation to initWifiController to avoid making it twice
wifi_init_config_t wifi_init_config = WIFI_INIT_CONFIG_DEFAULT();
ESP_ERROR_CHECK(esp_wifi_init(&wifi_init_config));
// TODO: Consider moving this to init Wifi Controller to avoid running it twice as well
ESP_ERROR_CHECK(esp_event_handler_register(WIFI_EVENT, ESP_EVENT_ANY_ID, &event_handler, NULL));
// Configuration for AP
wifi_config_t wifi_config = {
.ap = {
.ssid = "Grow System",
.password = "Password",
.ssid_len = strlen("Grow System"),
.max_connection = 4,
.authmode = WIFI_AUTH_WPA_WPA2_PSK
}
};
// TODO: Enable password support on AP configuration
ESP_ERROR_CHECK(esp_wifi_set_mode(WIFI_MODE_AP));
ESP_ERROR_CHECK(esp_wifi_set_config(WIFI_IF_AP, &wifi_config));
ESP_ERROR_CHECK(esp_wifi_start());
ESP_LOGI(TAG, "Wifi connectAP finished");
registerEndPoints();
}
The code essentially checks NVS for the credentials, if it finds them both it returns a struct containing both of them as well as ESP_OK. If one of them was not found, the struct instead contains ESP_ERR_NVS_NOT_FOUND. wifiController.c then receives this creds struct and calls beginWifi() which then either calls connectSTA() or connectAP() based on the status of the struct. When the credentials are present, connectSTA() is called, but for unknown reasons the event loop consistently only receives the SYSTEM_EVENT_STA_WPS_ER_PIN event. As I mentioned earlier, even after rolling my code back to a version that did not have the connectAP() function, this behavior persists.
Because of this, I have a hunch that the issue may be related to when I wiped the flash manually, as opposed to the code.
The header file contains the following line in it's typedef to define this event.
SYSTEM_EVENT_STA_WPS_ER_PIN, /*!< ESP32 station wps pin code in enrollee mode */
I do not know what that means, as I have not intentionally included anything regarding wps in this project.
So far my research hasn't returned anything incredibly useful, so if anyone has any ideas what SYSTEM_EVENT_STA_WPS_ER_PIN is or what could be causing my issue, I would be very appreciative. If you think that anymore detail would be useful to solve this issue, please let me know and I will be more than happy to provide it.
Useful to solve the problem.
I'm in the proces of learning to use the ESP32 wifi, read your message checked, the ESP-idf and esp-32 technical manual not much info there I found this URI
https://www.wi-fi.org/downloads-public/Wi-Fi_Protected_Setup_Best_Practices_v2.0.2.pdf/8188
Kind regards
Update: check the sdkconfig file in your projectmap against the one in the example map for wifi config settings.
I have been digging into this issue for several days now and I have not found the root cause but have managed to discovery a workaround and some details about the issue. the SYSTEM_EVENT_STA_WPS_ER_PIN event that kept arising actually is caused by the chip trying to use WPS Enrolle Pin mode to connect to the router. Supposedly this generates an eight digit long pin that can be put into your router which should allow it to connect. I really do not know why it was doing this, but I believe it may have had something to do with how I had AP mode set up. My primary confusion now is why rolling back the code did not fix it.
For the "workaround" I found that flashing Espressif's example STA wifi connection code managed to solve it. The chip properly connected to my network once it was flashed, and I was able to reflash my own code without any issues arriving. This seems extremely strange to me, so part of me thinks that maybe the chip just couldn't connect to my network for some reason and an edge case in the code caused it to go into enrollee mode.
Here is a link to the code that I used to "fix" the issue: https://github.com/espressif/esp-idf/blob/master/examples/wifi/getting_started/station/main/station_example_main.c
I will mark this answer as correct and continue looking for the root problem. In the event that someone else know what actually could have caused this, feel free to post and I will update the correct answer. In the event that I find what the root problem is, I will update this answer as well.
EDIT: After continuing to dig, I believe that the problem was actually do to a multitude of errors in my code. Particularly, I was calling esp_netif_create_default_wifi_sta() and then not setting the WI-FI mode. I needed to add esp_wifi_set_mode(WIFI_MODE_STA), which was absent in my program. I believe setting the network stack to sta without changing the wifi mode was what caused my issue.
tl;dr: what's the logic behind browsers (Chrome, FF, Safari) behaving as an app that's online after clicking offline and not simply... go offline?
CPP, FOP, STP
I have a small socket.io app that fetches from Twitter's API to make an image gallery.
I wanted to style the the divs that create a frame around the photos, but while the app is running found out that when selecting the elements in the dev tools, whenever a new image was added, Chrome emits a "purple
pulsing" (hereafter referred to as CPP) that kicked me out of the div I wanted to style and (rudely) put me at its parent div (the Gallery proper, if you will).
Voilà:
I started by shutting off my WiFi, which solved the problem with two drawbacks:
remembered the offline option in the network panel
needed a connection to read the socket.io docs :~)
Next I tried the offline option and found that, like the production version, CPP reëmerged, the image requests logging net::ERR_INTERNET_DISCONNECTED.
I realized that I could probably set the option reconnection: false in the socket.io bit but alas, this novella of question (which contains multitudes) still beckoned:
The Actual Question(s)
What chez Google (and Firefox (Orange Pulse), Safari (Transparent Pulse), et. al) is the logic to this behavior?
Why not Truly Sever the relevant tab's connection?
Better yet, why not let the poor developer both hold fast to their element and acknowledge visually that new elements are being thrown in?
The images are still fetched (!) which makes the Offline option seem even more misleading.
The docs from Google reference PWAs and those with service workers... does
Check the Offline checkbox to simulate a completely offline network experience.
apply only to them?
The Code that Kinda Could:
Here are the ~20 relevant lines at play (and here's the whole gig):
// app.js
var T = new Twit(config)
var stream = T.stream('statuses/filter', { track: '#MyHashtag', })
stream.on('tweet', function(tweet) { io.sockets.emit('tweet', tweet) })
function handler(request, response) {
var stream = fs.createReadStream(__dirname + '/index.html')
stream.pipe(response)
}
... and the index.html's relevant script:
// index.html
var socket = io.connect('/');
socket.on('tweet', function(tweet) {
if (someConditions = foo) {
tweet_container.innerHTML = '<img src="'
+ tweet.user.profile_image_url +
'" />'
}
}, 1000)
Nota Bene: I realize this question contains questions germane to polling, streams, networking, and topics whose names I'm not even familiar with, but my primary curiosity is what's the logic behind behaving as an app that's online after clicking offline and not simply... go offline" (and behave as it does when disconnecting from WiFi).
P.S.S here's a quote from knee-deep in the socket.io docs
If a certain client is not ready to receive messages (because of network slowness or other issues, or because they’re connected through long polling and is in the middle of a request-response cycle), if it doesn’t receive ALL the tweets related to bieber your application won’t suffer.
In that case, you might want to send those messages as volatile messages.
currently i develop an Ionic2 application which commuicates with a Firebase database. While updating a node sometimes it works and sometimes it doesn't.
So i tired to handle the error with the following code:
this.db.list("/Events/" + this.eventID+ "/teilnehmer").push(this.userID)
.then(resolve => {
console.log('success');
}, reject => {
console.log('error');
})
.catch(reject => {
console.log('catch');
});
But even if i disconnect my internet connection there is no error thrown.
Does someone of you know how i could handle an error if the push was not successful?
I had the same situation that push does not return promises sometime, so i raised issue on github FirebaseListObservable push() takes too long to add but unfortunately it was not resolved, I contacted firebase support through email, the support team reviewed my issue and checked the code i sent, and replied there is nothing to do in the code , and advised me to clear the phone's storage cache, i did the same and issue got resolved,
here is the mail from firebase support
Hi xxxx,
I suggest that you try clearing your phone's cache before proceeding
again, its memory might be too high. To do this, go to: Settings ->
Storage -> Cached Data. Select it then then choose ok (for clearing
the cache). Also please check this the same issue raised on Github
and answered by one of our engineers.
If the issue persists after trying the proposed suggestions, please
get back to me with a minimal and runnable code that we can use to
simulate this, possibly a plunkr or jsfiddle code, or a from the
scratch code that can demonstrate the issue.
Regards, xxxx
If you would like to handle this you could consider setting a timeout because the Promise will just stay in 'pending' if Firebase doesn't return anything and resolve, reject / catch will never be triggered.
You could do this for example with Promise.race(): Example Promise timeout with Promise.race() also check this thread: More Examples
When working with a large-sized data response from an HTTP Adapter, the size of the response appears to cause our challenge handler to fire a handleChallenge() method.
My question is, why would the size of the response cause the mobilefirst security challenge handler to fire, when the session is still valid?
More Details:
Our application uses an ISAM security appliance with Header based authentication. When an HTTP adapter call we make comes back with a content-length greater than 20,377, the adapter call triggers the handleChallenge() method of our challenge handler. When inspecting the response, we have seen that the responseJSON is actually populated with our required data, so really the handleChallenge should never have fired.
When we ping the adapter directly through the browser with the same parameters, it works fine. We've been able to isolate that this problem is occurring in the worklight.js / mobilefirst realm.
Does anyone have any idea if, or why, the Challenge Handler in worklight.js would not function as expected with a very large response size?
The bottom line is that it should. There is no reason for it not to.
If you have not been able to resolve this otherwise, my suggestion is to open an IBM PMR (support ticket) to have the development team investigate the issue.
We ended up ( sort of ) diagnosing it. At a certain payload size, the "/*secure {" fell off the response ( of which we're still not certain why. Our loginChallengeHandler function was based off of some example we found in some IBM documentation, and would improperly mark the response as a login form if the /*secure wasn't present. Once we tightened up the challenge handler, it worked.
Worklight 6.1 on both Windows (colleague) and Mac (me), building an a Hybrid app destined for Android device but to speed up development we do initial testing as Mobile Web App in Chrome browser on desktop.
We get a weird symptom that I'm trying to fine-down to a reproducible test case. I think I see different behaviours when stepping in debugger and just letting it run. Want to check whether a certain coding pattern could be the cause of the symptom before I go any further.
Fundamental question: should we wait for the resolution of a promise returned by a JSONSTore request for an action on a collection before issuing another request? more explanation below.
The overall intent is to load some data into the JSONStore, with some intelligent replace/merge action if a record is already present. Pseudo code:
for each record retrieved from back-end
if ( record already present in Store )
do some data merging
replace record
else
add record
The application code actually works like this, just considering the add() case, the problem manifests when the store is empty, all records need to be added
for each record to add
addPromise = store.get().add(record);
listOfPromises.insert(addPromise);
examine the list of promises recording any errors
That is there is no "wait" for add to finish before issuing the next add request. Hence in effect we've initiated a set of adds "in parallel" whatever that might mean in JavaScript in Chrome.
The code appears to run just fine, no errors reported. On android device it works reliably. In Chrome under normal running (no stepping in debugger) we end up with no reported errors but only one record inserted - indeed as though a snapshot of the initial "empty" store had been taken and each add is working on that "empty" copy.
After writing this I'm now pretty convinced that the coding pattern described above is vulnerable to a kind of race and that the better approach is build a list of documents to be added and insert them in a single operation.
A more detailed answer will be coming later, but I now know that this
the coding pattern described above is vulnerable to a kind of race and
that the better approach is build a list of documents to be added and
insert them in a single operation.
is true. In the browser the JSONStore does require that we wait for the result of one request before issuing another one. The recommended approach is
var dataToAdd = buildArrayOfDataToAdd(responseFromServer);
var dataToReplace = buildArrayOfDataToReplace(responseFromServer);
jsonstore.add( dataToAdd ).then( function() { jsonstore.replace( dataToReplace); })