Why in the world is this the webkit user agent:
Mozilla/5.0 (Windows NT 6.0; WOW64) AppleWebKit/534.27+ (KHTML, like Gecko) Version/5.0.4 Safari/533.20.27
Why not:
Webkit/5.04 (Windows NT 6.0; WOW64) AppleWebKit/534.27+ (KHTML, like Gecko) Safari/533.20.27
Thanks for clearing this up :)
It really is a left over from the early days of Web. Many sites were only compatible with Netscape Navigator, which was the dominant browser at that point, and so what they did was to sniff the User-Agent for the "Mozilla/*" part. When IE showed up, MS wanted those websites to work in their browser as well, so they went for pretending they are Mozilla as well. And so did all the browsers that popped up later on, including WebKit-based ones. And it doesn't seem like that artifact is going away anywhere soon as still many old sites do that type of sniffing and for browsers dropping this convention would probably mean breaking thousands of sites.
It's a throwback to the browser wars, the browser is identifying as a 'Mozilla Compatible' agent.
Related
I'm scraping house rental listings with Selenium using Firefox on a Windows 7 box and have run into a captcha on a website where I have three pages to traverse. Some searching tells me that one of the many techniques I'll have to implement in order to avoid detection is to change the user agent for each request.
I have found Python code on stackoverflow posted as recently as five months ago for doing the same sort of thing in Google Chrome using the following user agent string:
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.97 Safari/537.36
But if I go to https://www.whatsmyua.info/ from Firefox on my desktop I see the user agent string is:
Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:83.0) Gecko/20100101 Firefox/83.0
And if I do the same thing using the Tor browser I get this:
Mozilla/5.0 (Windows NT 10.0; rv:78.0) Gecko/20100101 Firefox/78.0
Those two user agent strings don't resemble the Python-style code at all!
Wouldn't putting reference to Windows, AppleWebKit, Chrome and Safari into the user agent string be an obvious tipoff to the bot detector that this is no ordinary browser access?
For error reporting etc I would like to know the version of chromium used by a chromium based web browser. Can I find that somewhere in the users web browser?
execute "navigator.userAgent" in javascript
it will return a string similar to
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.86 Safari/537.36"
with Chromium as 3rd
Thanks for suggestions about using navigator.userAgent. They might be useful (not sure), but a more safe way seems to use something like
if (typeof chrome === "undefined") {
// Not chromium based, probably Firefox then
}
Any comments on this? #Asesh?
I am running some headless tests with Selenium & firefox to login to some website. If I switch between separate firefox profiles in the same browser to login to separate accounts on the website is there any way besides IP tracking to track my connections and find out it's all on the same machine (on which the Selenium script is running)?
What I am doing:
profile = webdriver.FirefoxProfile('path/to/profile1')
driver = webdriver.Firefox(profile)
driver.get("website.com")
#connect as user1 on website & do stuff...
driver.quit()
profile = webdriver.FirefoxProfile('path/to/profile2')
driver = webdriver.Firefox(profile)
driver.get("website.com")
#connect as user2 on website & do stuff...
driver.quit()
EDIT: In other words, Given the code above and excluding IP tracking can website.com figure out that the accounts user1 and user2 are actually on the same machine(hence one person with 2 accounts) ?
Multiple Firefox profiles
A profile in Firefox is the collection of settings, customizations, add-ons, and other personalizations that a user has made or installed into their copy of Firefox.
Reasons to have multiple profiles
The casual user may want to have different profiles for different family members. Having different profiles would allow each person to have his or her own set of bookmarks, settings, and add-ons.
Web developers might want a secondary profile for testing websites, apps, or other projects on different Firefox channels. For example, you might want to have some extensions installed for Web development, but not for general-purpose Web browsing.
For QA, testing, and bug triaging contributors, you may want to have multiple development versions of Firefox installed, each with its own profile. Creating new profiles for testing can keep you from losing your preferences, bookmarks, and history. It takes little time to set up a new profile, and once it is complete, all of your Firefox versions will update separately and can be run simultaneously.
Bursting the Myth
So multiple instances of Firefox Browser Clients with multiple Firefox Profiles can be run simultaneously. So your assumtion of ...If I switch between separate firefox profiles in the same browser to login to separate accounts... is incorrect. The Web Browsing Client variant might be same i.e. Firefox but each of them will inherit different set of preferences, bookmarks, and history.
User Agent
Your User Agent tells every site you visit what browser you're using. Sites are able to detect your UA and redirect to a more fitting page for your browser if desired. In other words, if you go to a site with this detection in place from a mobile phone the site might detect you are a mobile user via the UA and redirect you to their mobile site if available. This doesn't work on all sites becuase not all sites have UA detection in place.
UAs also tell the sites you're visiting some browser related software which you may have installed. For example, this UA:
(Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.0; Trident/4.0; GTB6; Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1) ; SLCC1; .NET CLR 2.0.50727; Media Center PC 5.0; InfoPath.2; .NET CLR 3.5.21022; .NET CLR 1.1.4322; .NET CLR 3.5.30729; OfficeLiveConnector.1.3; OfficeLivePatch.0.0;)
Tells us that the visitor is using Internet Explorer 8, Windows Vista, has Media Center PC 5.0 installed, Office Live Connector, etc. So if we had a special section for Vista users, we could redirect this person based on the Windows NT 6.0 string to a sub section in our site related strictly to Vista. On the other hand, this User Agent:
Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3
Tells us the visitor is using FireFox 3.5.3 and Windows Vista. Not much else.
Having the knowledge from above, User Agent Spoofing is used by many programmers that don't want people to know how they're getting to the site(s). Some programmers will find legitimate UAs and program their script to use such UAs. UAs can be blank, or contain custom information like a URL to a site that might promote a particular browser or script.
Update
As per your question update ...can website.com figure out that the accounts user1 and user2 are actually on the same machine... I am not sure if User Agent carries the Browser Profile information. However based on the different User Agents different parameters can be extracted. As an example, for a user-agent string e.g.:
Mozilla/5.0 (Windows NT 6.2; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.102 Safari/537.36
Different User-Agent detection library thinks as follows:
According to useragent v2.1.9:
ua
rawUa: Mozilla/5.0 (Windows NT 6.2; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.102 Safari/537.36
string:
family: Chrome
major: 70
minor: 0
patch: 3538
device: Other 0.0.0
os
string: Windows 8 0.0.0
family: Windows 8
major: 0
minor: 0
patch: 0
According to ua-parser-js v0.7.11:
ua
ua: Mozilla/5.0 (Windows NT 6.2; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.102 Safari/537.36
browser
name: Chrome
version: 70.0.3538.102
major: 70
engine
name: WebKit
version: 537.36
os
name: Windows
version: 8
device
model: undefined
vendor: undefined
type: undefined
cpu
architecture: amd64
According to platform.js v1.3.3:
ua
name: Chrome
version: 70.0.3538.102
layout: Blink
os
os: Windows 8 64-bit
device
product:
manufacturer:
description: Chrome 70.0.3538.102 on Windows 8 64-bit
I couldn't find any relevant post about this mysterious behaviour anywhere else so excuse me if there's an obvious answer.
I am testing an application in various browsers and IE 11 has some strange behaviour.
When I visit a site that tells me my user agent I get:
Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko
Which is standard for IE11.
However when I run my application the user agent is different only on IE11.
I tried debugging the application on IE11 by adding in Global.asax:
protected void Application_BeginRequest()
{
var ua1 = Request.Headers["User-Agent"];
var ua2 = Request.UserAgent;
}
Both of them show:
Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; WOW64; Trident/7.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; InfoPath.3; .NET4.0C; .NET4.0E)
Chrome shows the same on both cases but IE 11 shows different user agents depending whether I am in my application or any other tab. So my application seems to affect the user agent somehow but only when viewing it with IE11.
Can someone explain this and tell me how to get the same user agent even via my application? Any idea why this is happening?
Ideally I want when debugging to get the expected user agent:
Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko
Do you by any chance have the F12 Developer Tools window open and the browser emulation set to IE7, or the user agent spoofing set to Internet Explorer 7?
You can find it in the Emulation tab of the tool.
Alternatively, if that is not set, have you checked if IE is running in "Compatibility Mode" - as it may be set to do this for intranet sites. If so, this question is a likely duplicate of:
IE11 User-Agent - Wrong one when pointing towards localhost - Right one when going towards my PC name?
and also:
IE 11 sends different User-Agent header to different subdomains
When someone accesses a site via a WebView in an iPhone app, is there a reliable way (JavaScript or server-side) to tell what app is being used? It seems like the user agent string can tell me if it's from Safari or not, but not what precise app is being used if it's not Safari, unless a custom user agent has been specified by the app's programmer (which is not often).
I'd like to be able to tell more precisely what exact app is being used in order to serve [slightly] customised content. The identifier doesn't need to be human-readable, just app-sepcific (so e.g. a bundle ID would be fine).
Example (and not useful) user agents:
From Safari: "Mozilla/5.0 (iPhone; U; CPU iPhone OS 3_1_2 like Mac OS X; en-us) AppleWebKit/528.18 (KHTML, like Gecko) Version/4.0 Mobile/7D11 Safari/528.16"
Most Apps:
"Mozilla/5.0 (iPhone; U; CPU iPhone OS 3_1_2 like Mac OS X; en-us) AppleWebKit/528.18 (KHTML, like Gecko) Mobile/7D11"
I tried to write just "Nope", but answers are required to be at least 15 characters long...