Besides "no connection", what other failure modes should I test for? How do I simulate a high-latency link, an unreliable link, or all the other sorts of crazy stuff that will undoubtedly happen "in the wild"?
How about wireless applications? How do I test the performance in a less-than-ideal WL environment?
To add to TimK's answer, if you have a router, test pulling the upstream link on the router, this will test a bad connection without your system knowing that you lost the physical link.
Also if you plug it back in after a few seconds it's possible that the connection won't be lost*. This can simulate a very high latency.
*this depends on your ISP and your router.
If you're using Linux, try Virtual Distributed Ethermet (VDE).
VDE gives you virtualised switches/hubs and Ethernet cables. You can tune network characteristics such as latency, delay, MTU, errored bits per MB, bandwidth, duplicates, etc on individual cables - all in real time!
You definitely want to test physically pulling the cable out. Lots of networking code will throw different exceptions in that scenario vs when the connection has just been lost.
To add to TimK's answer, if you have a router, test pulling the upstream link on the router, this will test a bad connection without your system knowing that you lost the physical link.
Our network/server closet is a spaghetti-mess of wires; I'm not going to walk in there and start unplugging stuff lest I hit something mission-critical. (At least I have access to it; I'm sure many readers don't even know where their routers are.) Similarly, both ends of the ethernet cable require a hands-and-knees adventure to reach.
I tested enabling/disabling the network adapter, and I'm going to test from my cable internet connection from home as well. Also, I had the idea of installing Tor to create a high latency connection.
For wireless connections, I have a metal box to test what happens when the signal dies, but I notice that network connection behavior is very different depending on how I test:
put the transmitter/reciever in a metal box
go stand next to the microwave in the kitchen and turn it on
go stand in a little closet which has concrete walls
Related
I'm facing issues when communicating with devices over USB hub. When enumerating devices directly to host port, it does work, some devices over usb hub have issues.
Setup: STM32F103C8 - MAX3421E - LUFA (usb stack) (ported to MAX3421E (host) and STM32F103C8T6 (device)) - USB Full-Speed setup
Scenario:
When I attach device directly to host, I don't experience any issues enumerating almost all (some devices seems to be faulty and have weird/nonstandard behavior) devices. But when I try to enumerate over usb hub, devices starts to behave very strangely. I'm receiving much more NAK's from devices than when connected directly to host. Some devices are able to return Device Descriptor, but retrieving Configuration Descriptor fail. Some devices return Toggle Error after several NAK's, this could be remedied so far by delaying retry IN token. Also there is different behavior of devices when connected over different hubs. I.e. one device has no problems when connected to HUB1, but have issues when connected to HUB2. Then I have HUB3 (7 port) which internally acts as HUB in HUB. On this HUB3 device working fine on port behind secondary internal hub, but not on primary ports exposed over "root" hub.
I'm in suspicion that hub's TT could be somehow interfering with usb communication, but according to information I have found, TT should not be enabled under Full-Speed setup.
I have checked (many times) that I'm setting correct device address assigned during SetAddress phase (which is proved by returning Device Descriptor). When I step debug it seems that I can get Configuration Descriptor also, but while in normal system run, it isn't retrieved, but only over hub.
Does anyone has any ideas, what to look after? I've run out of ideas here after week of trying to find a root cause.
Thanks
so...
- as usual after searching for root cause, solution after days of trying comes naturally after asking on somewhere (this is hapenning to me always, but I do try prior asking always)
- when using hubs, make sure you don't suspend SOF generation during control transfers. LUFA just resume bus inside control transfer routines, so make sure you don"t stops and reenable SOF within (my fault as I'm using ported version to MAX)
- if you have tight main loop make sure you don"t reinitialize usb transfer without completion of previous try, but if you do so, check you don't owerwrite data which haven"t been processed yet fully (especially when using interrupt-driven transfer complete processing) [things seems to work when you have quite some debug output, as it delay that time critical transfers]
Enumeration over hub isuues are now second to none. Small glitches are subject for tweaking.
Unfortunately as I was in question for electrical issues, I had to unsolder usb host shield and soldered another one, which in light of new information seems unneeded. Nevermind, I have trained my soldering skills.
We’ve been using the Tokbox platform for several months now with a Javascript web-client as well as an Android phone client, where sessions and connections are managed by a Python server. While integration and bring-up went well on both ends (client and server), we continue to encounter problems with the in-session audio and video experience.
Sessions are always routed and always between two participants only, with much use of a collaborative editor.
The in-session experience is like a coin toss: we never know how it’s going to go, and that’s becoming a business threat.
Web-Client: A/V Resources
The most common problem is the acquisition of audio and/or video: at the beginning of a session, one or the other participants may have problems hearing or seeing the other. Allocating a new connection to establish new streams does not fix that, nor does restarting the browser.
Question: What’s the recommended way to detect possible resource locks (e.g. does another application hog the camera/microphone)?
Web-Client: Network
Bandwidth and packet loss are a challenge, for example this inspector graph:
Audio and video of both participants is all over the place, and while we can not control the network connections the web-client should be able to reliably give useful information.
Question: Other than continuous connection monitoring with getStats() and maybe the experimental navigator.connection property, how can the web-client monitor network connectivity?
Pre-Call Test
We recommend to customers to run a pre-call test and have implemented it on our site as well. However, results of that test often times do not reflect the in-session connectivity. Worse, a pre-call test may detect a low (no video) bandwidth while Skype works just fine.
Question: How can that be?
I'm a member of the TokBox development team. I remember you reported an issue with the Python SDK, thanks for that!
Web-Client: A/V Resources
Most acquisition issues are detected by the JS SDK and if they aren't then we'd really like to hear about it! Please report reproduction steps or affected session IDs to TokBox support (referencing this StackOverflow question): https://support.tokbox.com/hc/en-us/requests/new
Most acquisition errors appear as OT_HARDWARE_UNAVAILABLE or OT_MEDIA_ERR_ABORTED errors. Are you detecting and surfacing these errors to your users? There is also the special OT_CHROME_MICROPHONE_ACQUISITION_ERROR error which is due to a known issue with Chrome that has been mostly fixed since Chrome 63 (see https://bugs.chromium.org/p/webrtc/issues/detail?id=4799).
Web-Client: Network
Unfortunately this is one of the more difficult issues to troubleshoot. Yes, Subscriber#getStats() is the best tool we have at our disposal and is a wrapper around the native RTCPeerConnection#getStats() function. Unfortunately we don't have much control over the values returned by the native function and if you think our SDK is returning incorrect values when compared with values from RTCPeerConnection#getStats() then please let us know!
It would be worthwhile confirming whether the issue is reproducible in all browsers or only a particular one. If you have detailed data regarding the inaccuracy of the native RTCPeerConnection#getStats() function then we could work together to report it to the browser vendor(s).
Fortunately we have just released the new Publisher#getStats() function which lets you get the publisher side of the stats. This should help you narrow down the cause of a connectivity issue to either a publisher or subscriber side. Please let us know if this helps with tracking down these issues.
Pre-Call Test
Again, these tests are based on Subscriber#getStats() which in turn are based on RTCPeerConnection#getStats(), the accuracy of which is out of our hands, but we'd love any reproduction steps to either fix a bug in our client SDK or report a bug to the browser vendors.
Just to confirm though, when you say you've implemented a pre-call test in your site, did you use the official JavaScript network test module? https://github.com/opentok/opentok-network-test-js This is actually what's used by the TokBox pre-call test.
#Aiham, thanks for responding, I've been looking at the the new Publisher#getStats() you linked to (thank you!), so we too can give our users some way of visibly seeing the network conditions that might be affected the quality of their call (and who's causing it). However, it seems as though bytes / packets sent goes up sharply as the number of subscribers increases, even though we're in a routed session.
Am I wrong to expect the Publisher#getStats() statistics to stay fairly stable regardless of the number of subscribers then receiving that stream in a routed session? I expected the nature of a routed call to mean it's sent once to the OpenTok Media Servers, and the statistics would end there.
I have been working on Reachability class for a while and have tried both the one from Apple sample and the one from ddg. I wonder whether the Reachability class keep sending / receiving data after starting the notifier.
As I'm developing an app which connect to different hosts quite often, I decided to write a singleton and attach the reachability classes I need on it. The reacability classes would be initiated and start their notifiers once the app start. I use the singleton approach as I want this singleton class to be portable and can be applied to other apps without much rewriting. I am not sure if it is good idea to implement like this but it worked quite well.
However, someone reported that the battery of his device drain significantly faster after using the app and someone reported more data usage. My app does not send / receive data on background so I start wondering if it is related to the reachability.
I tried profiling the energy usage with Instrument and I notice that there are continuous small data (few hundred bytes in average) coming in via the network interfaces even I put my app in idle. However, there are almost no data sending out.
I know that Reachability requires data usage when initiate (resolving DNS etc) but I am not sure that whether it still keep using data after starting notifier. Does anyone can tell?
I am not familiar with the low-level programming, it would be nice if someone could explain how does the Reachability work.
I use Reachability, and while I haven't monitored the connections, I have browsed the code, and I can't see any reason why it would keep sending ( or receiving).
If you have a ethernet connection to your Mac, it is quite easy to check. Enable sharing over wifi of your ethernet connection. Install little snitch, it will run in demo mode for three hours after every boot. Turn off the data connection on the test device and connect it to your mac over wifi.
This will allow you to see any network access your test device is making.
If this isn't possible, you can also run your app in the simulator as the network side should be the same, so you should be able to check.
There are also a ton of other tools to track network activity, but I think little snitch is the easiest to use.
I want to test an iPad app that needs an internet connection. I want to sit at my desk next to my router and test it against a poor wifi connection. How can I replicate a really poor wifi connection? Wrap the router in tin foil? Try and limit the router connection speed to the internet?
Thanks.
In the end I tested the app using the iOS simulator in Xcode on a Mac, then used the excellent Network Link Conditioner tool (that comes with Xcode) to tune my network speed as required. Worked a treat.
Better ways to do these tests are to use RF isolation chambers.
Did you try reducing the power levels on Access Point to minimum?
On a different note, if you want to have a bad Wi-Fi environment,create lot of interference on same channel from another Access Point.
I'm trying to create an embedded outdoor display of bus arrival times at my university. I'd like the device to utilize my school's secured WiFi network to show arrival time updates determined from a server script I have running.
I was hoping to get some advice on the high-level operation of this thing -- would it be better for the display board to poll a hosted database via the WiFi network or should I have a script try to communicate with the board directly over 802.11? (Push or Pull?)
I was planning to use a Wifly or WIZnet ethernet board in combination with a wireless access hub. Mostly inspired by this project: http://www.circuitcellar.com/Wiznet/winners/001166.html Would anyone recommend something else over one of the WIZnet boards? I saw SPI/UART options and thought these boards could work with an AVR platform.
And out of curiosity -- if you were to 'cold start' this device (ie, request a bus arrival time by pushing the display's on button) you might expect it to take 10-20 seconds to get assigned an IP and successfully connect to the database, does that sound right?
I'd go pull. In fact, I'd have outdoor display make http or https requests of the server. That way the server could tell it how long to show a given set of data before polling for a new one using standard http page expiration.
I think pull would make it easier to have multiple displays, and to test your server as well. I've also got a gut feeling that this would make your display more secure. Someone would have to hack your server to hijack your display.
There's a very cool looking Arduino-targetted product called the WiShield. Seems super easy to use and he provides some source code. It uses SPI for host communication. If you're not interested in going the Arduino route, I'm sure the source code wouldn't be too hard to port to something like avr-gcc. Check it out, might save you some time and headaches for $55. Worth checking out anyway.