Can anyone who's following the specification more closely and has experience with how these things usually work estimate when Cross-Site XMLHttpRequest will become a Recommendation and start getting supported by the browsers?
I haven't been following the specification.
Given that IE6 is still in common use 5 years after IE7 came out, it'll take a long time (10 years?) before this can be reliably counted on to exist in browsers.
In the mean time, consider using jsonp for non-sensitive data. Granted, you shouldn't be transmitting sensitive information without encrypting it to begin with...
when Cross-Site XMLHttpRequest will become a Recommendation
When:
It is ready
There are two independent, compatible implementations
The wheels of bureaucracy have turned far enough
start getting supported by the browsers
Ages ago. Firefox 3.5 supports it, for instance.
Related
I'm working on developing both web-client and API server. I've been doing some research regarding default timeout, some are at 800ms, others 1200ms. However, I can't find the reason behind the arbitrary number. Can someone help me regarding this? An explanation behind the arbitrary number would be a great help.
Thanks,
TLDR: Please see paragraph starting with "The arbitrary number" in bold below. The rest is just extra info on the topic.
Although you might know this or have already read this in your research, I can share the following ideas:
Typically the timeout is set depending on the expected complexity of a query, the amount of data to be processed, and the expected load of the system when the query occurs (or any other expected operation that may require attention in terms of modifying a timeout). Also, this can be based on something like the number of requests an API makes to other APIs to handle an incoming request(s) and what those expectations might be.
The arbitrary number ("best guess" of whoever developed the software) would typically be expected when planning for a "most requests should complete in some fraction of this time if there is no issue regardless of what happens" or "this isn't anything to worry about" type of scenario. Hence the default values for timeouts are pretty much based on the assumption that they represent the vast majority of "acceptable" completed requests where no issue is present. It is typically set somewhere between "this should be plenty of time" and "there is most likely something terribly wrong with this request, let's end it" and most successful requests pass this test by "default".
In the case that you have operations that may take several minutes and you expect that this can occur without an actual issue being present, you may want to set the timeout higher than the default so your requests don't timeout when there is no actual problem (for example, most commercial APIs have constraints on the number of requests and time in which they must complete so problematic requests don't clog up the system and other reasons as seen by their developers).
Thus, there really isn't a great answer or standard to this aside from just taking a look at the amount of data/requests to be processed, planning for a reasonable ebb and flow of server load, level of optimization of your code compared to the expected load, and so on... It's almost like error-handling but for things that you don't know might happen yet (such as unexpected bugs) but based on things you already know about your system and its expected usage.
Generally, you won't have many scenarios where the timeout really matters all that much but you always want to have one (at least the default) to prepare for the unexpected.
I found the following article that talks about the topic and some of what I mentioned as well if you haven't seen it already:
https://medium.com/#masnun/always-use-a-timeout-for-http-requests-de4da538b9e3
tl;dr - According to SLA [ Service Level Agreement ] mostly. If not, try to optimize the code as much as possible to bring down the time it takes to give out the response in terms of milliseconds
I'll put the answer in layman's terms since it really depends on various factors.
Let's assume you have an API and it performs some operation and gives the result back. It's quite simple and you'll get the response for that under some milliseconds if they don't perform any complex operations.
And when we move into a more and more complex system where one API talks to another, it adds up the time, and the worst-case scenario, first API which started the request might get the final response after 5 seconds, 30 seconds, or even 60 seconds depending on the number of API calls and how good the system is designed.
And we are only considering the happy flow. What if something goes wrong in one of the APIs that gets called internally?
To avoid this bad experience, the clients will make an SLA that requires the company/developers to design the code in such a way that it gives the response within a certain acceptable range.
I came across this conversation once on Google Groups conversation and it might provide some insight.
So to answer the question about the acceptable range, If you don't have an SLA, try to optimize the code as much as possible to bring down the time it takes to give out the response in terms of milliseconds.
Generally 1 Second is considered acceptable. The reason for this and why the suggested numbers vary so much is most APIs have a lockout if you send requests to fast. However, some APIs will let you send requests faster. In my experience all of the APIs I have seen request a 1s(1000ms) delay between requests to prevent overload/accidental DDOS and have a timeout of 30-60sec.
Edit: It is important to mention to not let another request from the same IP be answered if the first one is still waiting as this would make a DDoS easy
This spec http://www.w3.org/TR/webdatabase/ says:
This document was on the W3C Recommendation track but specification work has stopped. The specification reached an impasse: all interested implementors have used the same SQL backend (Sqlite), but we need multiple independent implementations to proceed along a standardisation path.
Does this mean that HTML5 database is going away, and for some time we will have a de-facto standard using SQLite, possibly with browser differences? Or has the W3C published a plan of attack for finishing the standard?
According to this article:
[...] we think it is worth explaining our design choices, and why we think IndexedDB is a better solution for the web than Web SQL Database.
In another article, we compare IndexedDB with Web SQL Database, and note that the former provides much syntactic simplicity over the latter. IndexedDB leaves room for a third-party JavaScript library to straddle the underlying primitives with a BTree API, and we look forward to seeing initiatives like BrowserCouch built on top of IndexedDB. Intrepid web developers can even build a SQL API on top of IndexedDB. We’d particularly welcome an implementation of the Web SQL Database API on top of IndexedDB, since we think that this is technically feasible. Starting with a SQL-based API for use with browser primitives wasn’t the right first step, but certainly there’s room for SQL-based APIs on top of IndexedDB.
I'm not personally swayed by the arguments put forth in the article, but it seems clear that (for the time being) Mozilla has decided that Web SQL Database is dead.
Further interesting comments about this article may be found on Hacker News.
My understanding is that this is now called "IndexedDB"
http://www.w3.org/TR/IndexedDB/
Apparently the Firefox team has started implementing this:
http://hacks.mozilla.org/2011/01/indexeddb-in-firefox-4/
I don't know if anyone knows the answer. Mozilla doesn't like the dependence upon SQLite and has decided to go a different way. However, all WebKit based browsers already have it implemented and I don't see them removing it as any websites built to take advantage of the spec would be broken.
This means that at least in certain contexts, mostly within the mobile sphere where most browsers have a webkit implementation, it can still makes sense to use the HTML5 Web SQL spec. I see this as especially true for developers who are looking to create mobile applications using a framework like phonegap.
There are some times where as an application developer you want to provide users with access to data even if they aren't connected to the internet or if the connection is slow and some types of data is just more efficiently stored in a database than in a cookie or JSON cashe. For example, if you have data that has relationships it is much easier and quicker to do a join query to pull the data you need than it is to search a json map.
I don't think the spec is dead, and I actually hope that Mozilla will reverse their stance so that developers can use it to solve problems outside of the mobile webkit world.
I am nearly finished a web application. I need to test it and find the security issues before it release. Is there any methods / guideline to do this kind of testing? Or is there any tools to help me check my application is ready to go online? Thank you.
I would say:
check that there are no warnings or errors even in strict mode (error report).
In case you store any sensitive data (as passwords, credit cards, etc.) be sure they are encrypted with non-standard algorithms. Use SSL and try to be somehow paranoid with it.
Set your database with specific accesses by action and hosts, and do not use root account.
Perform exhaustive testing (use unit test when possible). Involve as many people you can.
Test it under the main browsers (Firefox, Chrome, Opera, Safari, IE) and if have time in others.
Validate all your HTML/CSS against standards (W3C). (recommendable)
Depends on the platform you are using, there are profilers which can help you identify bottlenecks in your code. (can be done in later stages).
Tune settings for your web server / script language.
Be sure it is search-engine friendly.
Pray once is online :)
This is not a complete list as it depends in:
which language/platform/web server you are using.
what kind of application you developed (social, financial, management, etc.)
who will use that application (the entirely world, an specific company, your family or just you).
are you going to sell it? then you must have at least most of the previous points.
is your application using very sensitive information (as credit cards)? if so, you should pay for some professional (company?) to check your code, settings and methods.
This is just my opinion, take it as it is. I would also like to hear what other people suggests.
Good Luck
As well as what's already been suggested, depending on what type of application it is, you can use a vulnerability scanner to scan your application for any vulnerabilities that could lead to hackers gaining entry.
There are quite a few good scanners out there, but note when using them that the results may or may not be 100%. It's hard to say.
For a list of scanners, commercial and free, see: http://projects.webappsec.org/Web-Application-Security-Scanner-List
For more information on scanners: http://en.wikipedia.org/wiki/Web_Application_Security_Scanner
Good luck.
Here you can find a practical checklist to use before launching a website
http://launchlist.net/
And here is a list of all the stuff you forgot to test
http://www.thebraidytester.com/downloads/YouAreNotDoneYet.pdf
Slightly unorthodox question here:
I'm currently trying to break an Apache with a handful of custom modules.
What spawned the testing is that Apache internally forwards requests that it considers too large (e.g. 1 MB trash) to modules hooked in appropriately, forcing them to deal with the garbage data - and lack of handling in the custom modules caused Apache in its entirety to go up in flames. Ouch, ouch, ouch.
That particular issue was fortunately fixed, but the question's arisen whether or not there may be other similar vulnerabilities.
Right now I have a tool at my disposal that lets me send a raw HTTP request to the server (or rather, raw data through an established TCP connection that could be interpreted as an HTTP request if it followed the form of one, e.g. "GET ...") and I'm trying to come up with other ideas. (TCP-level attacks like Slowloris and Nkiller2 are not my focus at the moment.)
Does anyone have a few nice ideas how to confuse the server's custom modules to the point of server-self-immolation?
Broken UTF-8? (Though I doubt Apache cares about encoding - I imagine it just juggles raw bytes.)
Stuff that is only barely too long, followed by a 0-byte, followed by junk?
et cetera
I don't consider myself a very good tester (I'm doing this by necessity and lack of manpower; I unfortunately don't even have a more than basic grasp of Apache internals that would help me along), which is why I'm hoping for an insightful response or two or three. Maybe some of you have done some similar testing for your own projects?
(If stackoverflow is not the right place for this question, I apologise. Not sure where else to put it.)
Apache is one of the most hardened software projects on the face of the planet. Finding a vulnerability in Apache's HTTPD would be no small feat and I recommend cutting your teeth on some easier prey. By comparison it is more common to see vulnerabilities in other HTTPDs such as this one in Nginx that I saw today (no joke). There have been other source code disclosure vulnerablites that are very similar, I would look at this and here is another. lhttpd has been abandoned on sf.net for almost a decade and there are known buffer overflows that affect it, which makes it a fun application to test.
When attacking a project you should look at what kind of vulnerabilities have been found in the past. Its likely that programmers will make the same mistakes again and again and often there are patterns that emerge. By following these patterns you can find more flaws. You should try searching vulnerablites databases such as Nist's search for CVEs. One thing that you will see is that apache modules are most commonly compromised.
A project like Apache has been heavily fuzzed. There are fuzzing frameworks such as Peach. Peach helps with fuzzing in many ways, one way it can help you is by giving you some nasty test data to work with. Fuzzing is not a very good approach for mature projects, if you go this route I would target apache modules with as few downloads as possible. (Warning projects with really low downloads might be broken or difficult to install.)
When a company is worried about secuirty often they pay a lot of money for an automated source analysis tool such as Coverity. The Department Of Homeland Security gave Coverity a ton of money to test open source projects and Apache is one of them. I can tell you first hand that I have found a buffer overflow with fuzzing that Coverity didn't pick up. Coverity and other source code analysis tools like the open source Rats will produce a lot of false positives and false negatives, but they do help narrow down the problems that affect a code base.
(When i first ran RATS on the Linux kernel I nearly fell out of my chair because my screen listed thousands of calls to strcpy() and strcat(), but when i dug into the code all of the calls where working with static text, which is safe.)
Vulnerability resarch an exploit development is a lot of fun. I recommend exploiting PHP/MySQL applications and exploring The Whitebox. This project is important because it shows that there are some real world vulnerabilities that cannot be found unless you read though the code line by line manually. It also has real world applications (a blog and a shop) that are very vulnerable to attack. In fact both of these applications where abandoned due to security problems. A web application fuzzer like Wapiti or acuentix will rape these applications and ones like it. There is a trick with the blog. A fresh install isn't vulnerable to much. You have to use the application a bit, try logging in as an admin, create a blog entry and then scan it. When testing a web application application for sql injection make sure that error reporting is turned on. In php you can set display_errors=On in your php.ini.
Good Luck!
Depending on what other modules you have hooked in, and what else activates them (or is it only too-large requests?), you might want to try some of the following:
Bad encodings - e.g. overlong utf-8 like you mentioned, there are scenarios where the modules depend on that, for example certain parameters.
parameter manipulation - again, depending on what the modules do, certain parameters may mess with them, either by changing values, removing expected parameters, or adding unexpected ones.
contrary to your other suggestion, I would look at data that is just barely short enough, i.e. one or two bytes shorter than the maximum, but in different combinations - different parameters, headers, request body, etc.
Look into HTTP Request Smuggling (also here and here) - bad request headers or invalid combinations, such as multiple Content-Length, or invalid terminators, might cause the module to misinterpret the command from Apache.
Also consider gzip, chunked encoding, etc. It is likely that the custom module implements the length check and the decoding, out of order.
What about partial request? e.g requests that cause a 100-Continue response, or range-requests?
The fuzzing tool, Peach, recommended by #TheRook, is also a good direction, but don't expect great ROI first time using it.
If you have access to source code, a focused security code review is a great idea. Or, even an automated code scan, with a tool like Coverity (as #TheRook mentioned), or a better one...
Even if you don't have source code access, consider a security penetration test, either by experienced consultant/pentester, or at least with an automated tool (there are many out there) - e.g. appscan, webinspect, netsparker, acunetix, etc etc.
Our testing system is pretty rudimentary; fire up a browser, see if it works. Recently we ran into problems, found by our client, with our application where the number of users created a slow-down in the application. The application is basically a huge Word document with people editing their own versions all at the same time. Part of the problem came from not knowing how to test multiple instances at the same time. My partner and I thought about how to test this; one idea was to hire out an internet cafe and hire students for an hour to bang on the app.
What are other ways that people have tried to emulate concurrency in testing their web-based application? Most of the advice here is for specific methodology; I'm asking, how do you test it to make sure that it works?
If you have never checked out Selenium, then you need to. It will allow you to do automated web testing through the browser. Ok, so first problem solved.
Now ideally you could use that same script and load it up on a bunch of boxes and run them all at once to get some sort of load testing right? Luckily for you someone has already figured this out, although it is a paid service: Browser Mob. But, it looks like you were willing to spend a little money to do this anyway, and would probably net you better, more repeatable results.
We usually answer the question "can the web application do more than one thing at a time" by using JMeter to produce a simulated HTTP load on the web server.
I find that it helps to consider distinguish several different types of testing; concurrency (what happens when two events in the system collide), capacity (what happens when there are many overlapping requests), volume (what happens as data accumulates in the system)...
Huge general slow down, evidenced by response times that fall outside of the SLA, are usually related to capacity problems (with contention as a common cause) or volume (many users, much data, and the system gets slower over time). The former usually requires some sort of multi-threaded request stream; the latter you can usually manage by preloading the volume, and then measuring the response times experienced by a single user.
I generally find that separating the load generator from the actual measurement/instrumentation is a good idea. That can be as simple as having a black box over there to generate a typical load, and sitting here with a stop watch measuring the responsiveness of a typical use case.
JMeter http://jmeter.apache.org/