I'm making a mobile client for a web site now. And information exchange between my app and server is in JSON (searching users and data on server,sending messages, conversation threading, etc.) But all these features work too slow. I click on the button "send" and then wait for some second before the message will be sent, the same thing with searching, authorization, etc. So I have such a questions:
1. Why it's such a performance overhead?
2. Can it be troubles with the server side or it's JSON parser troubles or may be something else?
3. How can i fix/optimize this? All solutions, advices etc. will be helpful!
I would use Xcode to debug the app to see whether the majority of time is spent loading the data from the server or parsing the JSON once the data is received.
If it is the first, try loading the data from a PC over the same wireless connection and see if it is slow on that too. If so, clearly your server side code needs optimising.
If it is the second and the parsing is slow, you may want to look into using JSONKit instead of the native JSON parser as testing shows it is faster. You may also want to review the structure of your JSON.
One thing I have noticed however is that connections are slower on my iPad than on other machines. I've noticed this when comparing apps I've developed in the simulator to on the device on the same network and when conducting speedtests. As for why this happens, I am not sure - some form of additional overhead in iOS perhaps.
I can save you some time - it has nothing to do with JSON. It has to do with how the your app handles requests in general. It obviously needs optimization on the server.
EDIT:
I suppose it could also be that you might be experiencing high-latency on your phone, but again, that has nothing to do with your app.
Debug it using a regular browser and chrome dev tools (in the network tab) - you'll see that the requests take long even on a desktop at which point you'll have to start fishing around in the server-side code to see what's making it go slow (hint: unoptimized database queries are a big bottleneck....but then again, so is crappy hardware).
Sorry that I couldn't be of more help, but without seeing the entire setup of the server and the code that's going slow (not the client requests, but the server code), that's the best I can do.
Best of luck.
Related
Good morning!
I have this old Flash app (no worries the question is not about this!) which receives data from .NET server.
The data is a two thousand rows table and basically what I do is: query the DB with vb.net, create a long querystring with the data and send it back to Flash with a response.write method via GET. Then the Flash client parses it accordingly. The app is a parking lot GPS map that our employees uses to locate the vehicles. Believe it or not it has worked fine for the last 12 years and counting!
Long story short now my boss asked me to start over and remake entirely the app in HTML5. One major change is that for the sake of "standartization" the data will be converted from regular table columns to XML format so the chunk of data will grow in size.
Also I confess that I never feel completely happy on moving data back and forth via GET. I can't remember exactly WHY I did it this dirty. Probably by the time we were in a rush to get the app running so it just worked and among a lot of other things to do it was put in the backburner and the rest is history.
Anyway, since we are restarting it fresh I'd like to do it the right way this time. So questions are:
What would you recommend for sending data from .NET server to AJAX client? The POST method is the obvious alternative or there is a newest and best way of doing it?
Should I send the whole XML as a big unique chunk of data and parse it entirely in client or would be better to send it in array format (each item node as an array entry) and parse the array entries? My question here is what would be less CPU intensive for client, considering that machines are tablets and not PCs.
Stream the data would be an option or this is a silly idea?
I appreciate suggestions and examples!
Thanks!
First off, I would suggest using JSON over XML. There are two libraries you can use to serialize/deserialize JSON data: either Newtonsoft or System.Text.Json.
What would you recommend for sending data from .NET server to AJAX client? The POST method is the obvious alternative or there is a newest and best way of doing it?
You should definitely be doing this via a POST request.
Should I send the whole XML as a big unique chunk of data and parse it entirely in client or would be better to send it in array format (each item node as an array entry) and parse the array entries? My question here is what would be less CPU intensive for client, considering that machines are tablets and not PCs.
This really depends. If I were writing this I would add support for server-side pagination so that you know how many total records would be returned, but you're only returning however many records are currently visible. This would dramatically improve speed.
Stream the data would be an option or this is a silly idea?
Just return a JSON response.
What would you recommend for sending data from .NET server to AJAX client? The POST method is the obvious alternative or there is a newest and best way of doing it?
There is no real difference between a GET and a POST, certainly not one that matters to your context anyway; GET would be fine.
A GET might look like this:
GET /api/parkinglot/1234 HTTP/1.1
Host: somehost.com
A POST might look like this:
GET /api/parkinglot HTTP/1.1
Host: somehost.com
{ "id":1234 }
It's a text file, in essence, sent to the server. The server responds. It's not something that is "the way we do things now" or "more modern", POST doesn't "perform better".. It uses trivially more bytes, and is interpreted slightly differently by the server.. That's about it. For what you're describing, GET would be every bit as valid
Should I send the whole XML as a big unique chunk of data and parse it entirely in client or would be better to send it in array format (each item node as an array entry) and parse the array entries? My question here is what would be less CPU intensive for client, considering that machines are tablets and not PCs.
It doesn't really matter. An array isn't necessarily magically more or less of anything than XML; it's all just text, interpreted by the client. You could write a really wasteful array based solution or a lean XML one. What you ought to be throwing away is the idea of sending massive blocks of data to the client. Clients are limited in resource; don't send 2000 anything; what possible use could the user of the device have for 2000 items of data? You can't show it on screen and meaningfully interpret it; if it's a tabular block of data they'll end up panning around it, scrolling, zooming, searching.. Think about redesigning the app so that it sends the data they need when they need it. You might consider that sending 2000 points of data to be rendered as 1000 pins on a map, lat and long, might be a great idea, the client might have a really good rendering engine that can cope with it and make it quick and a pleasure to use.. but really? It sounds like the server needs to do a lot more of the work here
Stream the data would be an option or this is a silly idea?
This is all streaming. Every download or upload is a stream of data. Data gets from A to B in a serial flow so that it pops out the other end the same order it went in. You need to mentally move away from the concept of streaming vs downloading vs sending vs whatever else you think of in terms of getting data around the place. These are not distinct things; start focusing on being really efficient with the data you request, the time it takes to process and emit from the server, and the processing that happens on the client. Decide where it's best to do various calculations; there's no point the client searching for all users called smith, the server sending a million people to the client and the client parsing and searching the data. The server should do most of that. If you want to draw a triangle on screen, you can send 3 points and have the client render it instead of having the server render a 2 million pixel image, sending it and having the client draw the image. In one of these examples the server does a lot, in the other the client does a lot. In both the problem is that there is an excess of data flowing. Focus on the strengths of each resource
I appreciate suggestions and examples!
It isn't really what stackoverflow is for; we don't design your programs for you or write them - you have to do that and we tell you how to fix issues you hit along the way. Questions that ask "what is the best" are typically off topic because they attract opinionated answers.
In writing this answer I haven't really answered any of the questions you've asked in the way you want, because it simply isn't permitted. Instead I've tried to keep to factual observations and points you should consider when forming your own solution. When you hit problems with that solution, we can help but "design and implement my solution for me" is not a problem
We’ve been using the Tokbox platform for several months now with a Javascript web-client as well as an Android phone client, where sessions and connections are managed by a Python server. While integration and bring-up went well on both ends (client and server), we continue to encounter problems with the in-session audio and video experience.
Sessions are always routed and always between two participants only, with much use of a collaborative editor.
The in-session experience is like a coin toss: we never know how it’s going to go, and that’s becoming a business threat.
Web-Client: A/V Resources
The most common problem is the acquisition of audio and/or video: at the beginning of a session, one or the other participants may have problems hearing or seeing the other. Allocating a new connection to establish new streams does not fix that, nor does restarting the browser.
Question: What’s the recommended way to detect possible resource locks (e.g. does another application hog the camera/microphone)?
Web-Client: Network
Bandwidth and packet loss are a challenge, for example this inspector graph:
Audio and video of both participants is all over the place, and while we can not control the network connections the web-client should be able to reliably give useful information.
Question: Other than continuous connection monitoring with getStats() and maybe the experimental navigator.connection property, how can the web-client monitor network connectivity?
Pre-Call Test
We recommend to customers to run a pre-call test and have implemented it on our site as well. However, results of that test often times do not reflect the in-session connectivity. Worse, a pre-call test may detect a low (no video) bandwidth while Skype works just fine.
Question: How can that be?
I'm a member of the TokBox development team. I remember you reported an issue with the Python SDK, thanks for that!
Web-Client: A/V Resources
Most acquisition issues are detected by the JS SDK and if they aren't then we'd really like to hear about it! Please report reproduction steps or affected session IDs to TokBox support (referencing this StackOverflow question): https://support.tokbox.com/hc/en-us/requests/new
Most acquisition errors appear as OT_HARDWARE_UNAVAILABLE or OT_MEDIA_ERR_ABORTED errors. Are you detecting and surfacing these errors to your users? There is also the special OT_CHROME_MICROPHONE_ACQUISITION_ERROR error which is due to a known issue with Chrome that has been mostly fixed since Chrome 63 (see https://bugs.chromium.org/p/webrtc/issues/detail?id=4799).
Web-Client: Network
Unfortunately this is one of the more difficult issues to troubleshoot. Yes, Subscriber#getStats() is the best tool we have at our disposal and is a wrapper around the native RTCPeerConnection#getStats() function. Unfortunately we don't have much control over the values returned by the native function and if you think our SDK is returning incorrect values when compared with values from RTCPeerConnection#getStats() then please let us know!
It would be worthwhile confirming whether the issue is reproducible in all browsers or only a particular one. If you have detailed data regarding the inaccuracy of the native RTCPeerConnection#getStats() function then we could work together to report it to the browser vendor(s).
Fortunately we have just released the new Publisher#getStats() function which lets you get the publisher side of the stats. This should help you narrow down the cause of a connectivity issue to either a publisher or subscriber side. Please let us know if this helps with tracking down these issues.
Pre-Call Test
Again, these tests are based on Subscriber#getStats() which in turn are based on RTCPeerConnection#getStats(), the accuracy of which is out of our hands, but we'd love any reproduction steps to either fix a bug in our client SDK or report a bug to the browser vendors.
Just to confirm though, when you say you've implemented a pre-call test in your site, did you use the official JavaScript network test module? https://github.com/opentok/opentok-network-test-js This is actually what's used by the TokBox pre-call test.
#Aiham, thanks for responding, I've been looking at the the new Publisher#getStats() you linked to (thank you!), so we too can give our users some way of visibly seeing the network conditions that might be affected the quality of their call (and who's causing it). However, it seems as though bytes / packets sent goes up sharply as the number of subscribers increases, even though we're in a routed session.
Am I wrong to expect the Publisher#getStats() statistics to stay fairly stable regardless of the number of subscribers then receiving that stream in a routed session? I expected the nature of a routed call to mean it's sent once to the OpenTok Media Servers, and the statistics would end there.
In several interviews I have been asked about handling of connection, web service calls, server responses and all. Even now I am not clear about many things.Could you please help me to get a better idea about the following scenarios?
What is the advantage of using NSURLSessionDataTask instead of NSURLConnection-I have an idea like data loss will not happen even if the connection breaks for NSURLSessionDataTask but not for the latter.But how it works?
If the connection breaks after sending the request to a server or while connecting to server , How can we handle the code at our end in case of NSURLConnection and NSURLSessionDataTask?-My idea is to use Reachability classes and check when it becomes online.
The data we are sending got updated at the server side. But we don't get the response from server. What can we do at our side to handle this situation?- Incrementing timeOutInterval is the only thing that we can do?
Please help me with these scenarios. Thank you very much in advance!!
That's multiple questions, really, but I'll try to answer them all briefly.
Most failure handling is the same between NSURLConnection and NSURLSession. The main advantages of the latter are support for background downloads and cancelling groups of related requests.
That said, if you're doing a large download that you think might fail, NSURLSession does provide download tasks that let you resume the download if your network connection fails, similar to what NSURLDownload used to do on OS X (never available on iOS). This only helps for downloading large files, though, not for large uploads (which require significant server-side support to resume) or other requests.
Your intuition is correct. When a connection fails, create a reachability object monitoring that particular hostname to see when it would be a good time to try the request again. Then, try the request again.
You might also display some sort of advisory UI to say that you have no Internet connection. (By advisory, I mean something that the user doesn't have to click on and that does not impact offline use of the app any more than necessary; look at the Facebook app for a great example.)
Provide a unique identifier when you make the request, and store that on the server along with the server's response until the client acknowledges receipt of the response (or purge it anyway after some reasonable number of days). When the upload finishes, the server gives you back its response if it can.
If something goes wrong, the client asks the server to resend the response associated with that unique identifier. Once your client has the data, it acknowledges receipt and the server deletes the response. If you ask the server for the response and it doesn't have one, then the upload didn't really complete.
With some additional work, this approach can make it possible to support long-running uploads more reliably. If an upload fails, ask the server how much data it got for that identifier, then tell the server that you're going to upload new data starting at the next byte. On the server side, overwrite the old data starting at that byte (just in case some data was still being written when you asked for the length).
Hope that helps.
I have been working on Reachability class for a while and have tried both the one from Apple sample and the one from ddg. I wonder whether the Reachability class keep sending / receiving data after starting the notifier.
As I'm developing an app which connect to different hosts quite often, I decided to write a singleton and attach the reachability classes I need on it. The reacability classes would be initiated and start their notifiers once the app start. I use the singleton approach as I want this singleton class to be portable and can be applied to other apps without much rewriting. I am not sure if it is good idea to implement like this but it worked quite well.
However, someone reported that the battery of his device drain significantly faster after using the app and someone reported more data usage. My app does not send / receive data on background so I start wondering if it is related to the reachability.
I tried profiling the energy usage with Instrument and I notice that there are continuous small data (few hundred bytes in average) coming in via the network interfaces even I put my app in idle. However, there are almost no data sending out.
I know that Reachability requires data usage when initiate (resolving DNS etc) but I am not sure that whether it still keep using data after starting notifier. Does anyone can tell?
I am not familiar with the low-level programming, it would be nice if someone could explain how does the Reachability work.
I use Reachability, and while I haven't monitored the connections, I have browsed the code, and I can't see any reason why it would keep sending ( or receiving).
If you have a ethernet connection to your Mac, it is quite easy to check. Enable sharing over wifi of your ethernet connection. Install little snitch, it will run in demo mode for three hours after every boot. Turn off the data connection on the test device and connect it to your mac over wifi.
This will allow you to see any network access your test device is making.
If this isn't possible, you can also run your app in the simulator as the network side should be the same, so you should be able to check.
There are also a ton of other tools to track network activity, but I think little snitch is the easiest to use.
I have rewritten web application from using mod_python to using mod_wsgi. Problem is that now it takes at least 15 seconds before any request is served (firebug hints that almost all of this time is spent by receiving data). Before the rewrite it took less than 1 second. I’m using werkzeug for app development and apache as a server. Server load seems to be minimal and same goes for memory usage. I’m using apache2-mpm-prefork.
I’m using the default setting for mod_wsgi - I think it’s called the ‘embedded mode’.
I have tested if switching to apache2-mpm-worker would help but it didn’t.
Judging from app log it seems that app is done with request quite fast - less than 1 second.
I have changed the apache logging to debug, but I can’t see anything suspicious.
I have moved the app to run on a different machine but it was all the same.
Thank in advance for any help.
Sounds a bit like your response content length doesn't match how much data you are actually sending back, with content length returned being longer. Thus browser waits for more data until possibly times out.
Use something like:
http://code.google.com/p/modwsgi/wiki/DebuggingTechniques#Tracking_Request_and_Response
to verify what data is being sent back and that things like content length match.
Otherwise it is impossible to guess what issue is if you aren't showing small self contained example of code illustrating problem.