We use a custom HTTP module in IIS as a reverse proxy for web applications. Generally this works well and has done for some time, but we've come across an issue with Windows Authentication (WA). We're using IE 11, IIS 10 and Server 2016.
When accessing the target site directly, WA works fine - we get a browser login dialog when the initial HTML page is requested and the subsequent requests (CSS, JS, etc) go through fine.
When accessing via our proxy, the same (correct behaviour) happens for the initial html page, the first CSS/JS request authenticates ok too, but the subsequent ones cause a browser login to popup.
What seems to happen on the 'bad' requests (i,.e. those that cause the login dialog) is:
1) Browser decides it needs to authenticate, so sends an Authorization header (Negotiate, with an NTLM token)
2) Server responds (401) with a WWW-Authenticate: Negotiate response with a full NTLM token
3) Browser re-requests with an Authorization header (Negotiate, with a full NTLM token)
4) Server responds (401) with a WWW-Authenticate: Negotiate (with no token), which causes the browser to show the login dialog
5) With login credentials entered, Browser sends the same request as in (1) - identical NTLM token, server responds as in (2), Browser re-requests as in (3), but this time it works!
We've set up a test web site with one html page, requesting 3 JS and 2 CSS files to replicate this. On our test server we've got two sites, one using our reverse proxy and one using ARR. The ARR site works fine. Also, since step (5) above works, we believe that the proxy pass-through is fundamentally working, i.e. NTLM tokens are not being messed up by dodgy encoding, etc.
One thing that does work, is that if we use Fiddler and put breakpoints on each request, we're able to hold back on the 5 sub-requests (JS & CSS files), letting one go through at a time. If we let each sequence (i.e. NTLM token exchange for each URL/file, through to the 200 response), then it works. This made us think that there is some inter-leaving effect (e.g. shared memory corruption) in our proxy, this is still a possibility.
So, we put code at the start of BeginRequest and end of EndRequest with a Synclock and a shared var to store the Path (AppRelativeCurrentExecutionFilePath). This was for our code to 'Single Thread' each of these request/exchanges. This does what we expected, i.e. only allowing one auth exchange to happen and resulting in a 200 before allowing the next. However, we still have the same problem of the server rejecting the first exchange. So, does this indicate something happening in/before BeginRequest, where if we hold the requests back in Fiddler then they work, but not if we do it in our http module?
Or is there some sort of timing issue where the manual breakpoints in Fiddler also mean we’re doing it at ‘human’ speed and therefore allowing things to work better?
One difference we can see is the ‘Connection: Keep-Alive’. That header is in the request from the browser to our proxy site, but not passed from our proxy to the base site, yet the ARR site does pass that through... It’s all using HTTP 1.1. and so we can't find a way to set Keep-Alive on our outgoing request - could this be it?
Regarding 'things to try', we think we've eliminated things like having the site in the Intranet Zone for IE by having the ARR site work ok, and having the same IE settings for that site. Clearly, something is not right, so we could have missed something here!
In short, we've been working on this for days, and have tried most of what we can find on SO and elsewhere, but can't figure out what the heck is going on.
Any suggestions - let me know if you want any further info. All help will be very gratefully received!
I am trying to build a model for how frequently users make web requests. I am interested in the timing between each new page they visit. I want to build a load simulator which then uses this model.
To do this I've been analyzing Squid access logs and looking at the timing between http requests by user IP. Squid captures all the requests associated with a web site request and I am only interested in the top level page requests. There are numerous starting pages for a request eg. not just *.html so it seems challenging to only capture the starting page for each session.
Is there a way to only capture the initial request for the top level page, like for when a user a page on Amazon, and then they jump to another page, etc.
You can use Squid Analysis Report Generator it will read log files and generate reports in HTML format with detailed information like access and denied website,daily and weekly report.
I'm investigating a problem a user is having with a web application that is built using Yii.
The user is not seeing the Yii 'flash' session-based user-feedback messages. These messages are shown once to a user and then destroyed (so they're not shown on subsequent page loads).
I took a look at the server access logs and I noticed something weird.
When this user requests a page there is a second identical request but from a different IP and with a different User Agent string. The second request is often at the same time or is sometimes (at most) a couple of minutes later. A bit of googling leads me to the conclusion that the user is browsing the web using a HTTP Proxy.
So, is this likely to be a HTTP Proxy? Or could it be something more suspicious? And if it is a HTTP Proxy, does this explain why they're not seeing the flash session messages? Could it be that the messages are being 'shown' to the Proxy and then destroyed?
For example: Once a user logged in, server side then needs to route the http traffics to a certain tomcat instance because all his/her files(not in DBMS but in local HD) stored on that machine and there will be interactions between the web app and underlying FS. Anyway to do this?
Thanks!
This is often done by redirecting to a server specific URL at or following login. The users session then continues with that URL. I frequently find www replaced with www1, www2, etc, after login on sites I use.
I am making app that takes a screenshot of a URL requested by the user. I want to make it as transparent as possible when sites that require username and passwords are in question.
For instance, if user wants to screenshot its iGoogle page, he will send the server URL but, the screenshot will not be the same as what he sees on his screen.
Is there any way to do this ? I guess that in such cases I will have to actually request screenshot from the user. Perhaps user can even deliver me his cookie for that domain.
Any thoughts ?
Ty.
Yes, in most cases you'll need user's cookies.
If site uses regular cookies, you can create bookmarklet that reads document.cookie. This will not work with httpOnly cookies which are used increasingly often for sessions.
Some sites limit sessions to certain IP, and in that case you can't take screenshot without proxying request through user's computer.
If you can get user to use bookmarlet, an interesting trick would be to read and send DOM to your server:
image.src = 'http://example.com?source=' +
escape(document.documentElement.innerHTML);
For HTTP authentication easiest solution would be to ask user for login/password.