I am trying to automate login to my app which uses among others, google sso authentication.
However login form return error "This browser or app may not be secure.". I set my google account options to allow less secure apps but still nothing.
I browsed few topics:
GMail is blocking login via Automation (Selenium)
Selenium Google Login Block
Automation Google login with python and selenium shows ""This browser or app may be not secure""
And it seems that google is blocking this way at all in favor of oauth.
People write in these topics that solutions stopped working recently
So is it currently possible, to set ChromeDriver somehow using capabalities, to be able to login through SSO?. I need a simple solution, that will run headless with other scripts on cloud (not something that would require me to manually login first on another instance as one anwser suggests).
If its not possible or extremly complicated please tell me I will not waste time on it.
If you want to use chrome capabilities, what you can do is set the user-data-dir to a chrome profile that has already been signed in using SSO.
You should look up how to reuse chrome profiles with selenium.
If your accounts have 2 steps verifications, google believe it's safer and allows you to get login. Then the issue will be how to handle the 2 steps verifications. Working on that :/
Related
I maintain an application written in C++Builder 2009. Part of it involves using a TWebBrowser control (based on Internet Explorer) to send users to a Google login page in order to obtain an OAuth key. This has worked well for a while, but now Google, bless their hearts, has implemented some kind of security upgrade, and now my users get to a page that says "Couldn't sign you in, this browser or app may not be secure". FYI, I am already setting a Registry key that is supposed to make IE run in version 11 emulation mode.
I do have a couple of workarounds: If the user runs IE first in admin mode, signs on, leaves it up while running my application, we don't get the problem. Second, I can start up the default browser - Chrome, IE, whatever - and send them to the URL for OAuth, then it avoids the error message.
The problem with this solution is that without being able to hook into TWebBrowser events, I don't have any way to automatically retrieve the OAuth key - it is necessary for the user to cut/paste it into my application. I'd like to avoid these clunky solutions.
I should also mention, this problem occurs only for certain Gmail accounts. I have no idea what the difference is between accounts that work and don't work. Any ideas on that?
So, is there any way to configure IE or TWebBrowser so this security issue is bypassed? Or, if I was to update to a modern version of C++Builder and use TWebBrowser (or something else?), would this problem be avoided? Any other ideas to fix this problem?
The latest C++Builder supports Google's Chromium engine, it's probably safe to say it'll be compatible with Google's security upgrades.
Powerful Chromium Based WebView Component To Host Web Content In Your Delphi/C++ Builder FireMonkey Apps
I am trying to access some services that require a google login. I thought it would be a really neat idea to automate it and have been trying to do so. Whenever I use my script to try and log in, however, I get this message:
This browser or app may not be secure. Learn more
Try using a different browser. If you’re already using a supported browser, you can refresh your screen and try again to sign in.
How do you go about getting around this? I tried researching it but came up with blanks. Is it possible to be logged in prior to running your script?
Logging in to Google is still possible (by exploiting a bypass).
By default, Google detects and effectively blocks all logins from Selenium webdriver.
The following link comes from the Google OAuth Playground, ensuring the practicality of the link. It most likely won't expire any time soon.
https://accounts.google.com/o/oauth2/v2/auth/oauthchooseaccount?redirect_uri=https%3A%2F%2Fdevelopers.google.com%2Foauthplayground&prompt=consent&response_type=code&client_id=407408718192.apps.googleusercontent.com&scope=email&access_type=offline&flowName=GeneralOAuthFlow
You can use that link (ie. driver.get()) to log into your Google account.
This bypasses the automation checks allows you to log into Google using Selenium. At least for now.
UPDATE: As of January 2021, this no longer works.
I'm curious how reCAPTCHA v3 works. Specifically the browser fingerprinting.
When I launch an instance of Chrome through Selenium/chromedriver and test against reCAPTCHA 3 (https://recaptcha-demo.appspot.com/recaptcha-v3-request-scores.php) I always get a score of 0.1 when using Selenium/chromedriver.
When using incognito with a normal instance, I get 0.3.
I've beaten other detection systems by injecting JavaScript and modifying the web driver object and recompiling webdriver from source and modifying the $cdc_ variables.
I can see what looks like some obfuscated POST back to the server, so I'm going to start digging there.
What might it be looking for to determine if I'm running Selenium/chromedriver?
reCaptcha
Websites can easily detect the network traffic and identify your program as a BOT. Google have already released 5(five) reCAPTCHA to choose from when creating a new site. While four of them are active and reCAPTCHA v1 being shutdown.
reCAPTCHA versions and types
reCAPTCHA v3 (verify requests with a score): reCAPTCHA v3 allows you to verify if an interaction is legitimate without any user interaction. It is a pure JavaScript API returning a score, giving you the ability to take action in the context of your site: for instance requiring additional factors of authentication, sending a post to moderation, or throttling bots that may be scraping content.
reCAPTCHA v2 - "I'm not a robot" Checkbox: The "I'm not a robot" Checkbox requires the user to click a checkbox indicating the user is not a robot. This will either pass the user immediately (with No CAPTCHA) or challenge them to validate whether or not they are human. This is the simplest option to integrate with and only requires two lines of HTML to render the checkbox.
reCAPTCHA v2 - Invisible reCAPTCHA badge: The invisible reCAPTCHA badge does not require the user to click on a checkbox, instead it is invoked directly when the user clicks on an existing button on your site or can be invoked via a JavaScript API call. The integration requires a JavaScript callback when reCAPTCHA verification is complete. By default only the most suspicious traffic will be prompted to solve a captcha. To alter this behavior edit your site security preference under advanced settings.
reCAPTCHA v2 - Android: The reCAPTCHA Android library is part of the Google Play services SafetyNet APIs. This library provides native Android APIs that you can integrate directly into an app. You should set up Google Play services in your app and connect to the GoogleApiClient before invoking the reCAPTCHA API. This will either pass the user through immediately (without a CAPTCHA prompt) or challenge them to validate whether they are human.
reCAPTCHA v1: reCAPTCHA v1 has been shut down since March 2018.
Solution
However there are some generic approaches to avoid getting detected while web-scraping:
The first and foremost attribute a website can determine your script/program is through your monitor size. So it is recommended not to use the conventional Viewport.
If you need to send multiple requests to a website keep on changing the User Agent on each request. Here you can find a detailed discussion on Way to change Google Chrome user agent in Selenium?
To simulate human like behavior you may require to slow down the script execution even beyond WebDriverWait and expected_conditions inducing time.sleep(secs). Here you can find a detailed discussion on How to sleep webdriver in python for milliseconds
Outro
Some food for thought:
Selenium webdriver: Modifying navigator.webdriver flag to prevent selenium detection
Unable to use Selenium to automate Chase site login
Confidence Score of the request using reCAPTCHA v3 API
Selenium and Puppeteer have some browser configurations that is different from a non-automated browser. Also, since some JavaScript functions are injected into browser to manipulate elements, you need to create some override to avoid detections.
There are some good articles explaining some points about Selenium and Puppeteer detection while it runs on a site with detection mechanisms:
Detecting Chrome headless, new techniques - You can use it to write defensive code for your bot.
It is not possible to detect and block Google Chrome headless - it explains in a clear and sound way the differences that JavaScript code can detect between a browser launched by automated software and a real one, and also how to fake it.
GitHub - headless-cat-n-mouse - Example using Puppeteer + Python to avoid detection
I'm trying to use robot framework as a ui test tooling for a website we use internal.
To test different user roles I open the browser with basic authentication (http://user:ww#url). Unfortunately this methode is removed from chrome and chromedriver (http://www.chromestatus.com/feature/5669008342777856) (for the test I use PhantomJS).
because of this issue subresource requests are blocked. See image attached.
Because of this issue also js files are blocked and therefore my UI tests don't work properly.
Does anybody have an idee on how to solve this or another way of testing?
This issue is being encountered by all browser automation frameworks. This SO answer describes an approach to take a two step approach:
Go to the url with http://user:pass#hostname.ext
Go to the url with http://hostname.ext
The username and password are cached and subsequent visits will reuse it.
I have a strange request for a project - we have a text file containing a list of login credentials for Dropbox for around 10 users. I have to validate their credentials automatically and notify the users if login fails with their credentials.
I've looked at the dropbox API, but that uses OAuth, which requires the user to manually fill in credentials, but that doesn't do the trick for me.
Is there a way for me to check if a bunch of logins? I've looked at the DropboxUpload repository on github, but it works fine for single user logins, but fails to achieve what I want when it comes to checking the same thing for multiple users.
Any help/leads on how I should go about proceeding with this would be greatly appreciated!
Many thanks,
John
You can use dropbox webpage to check user credentials doing normal login/logout but in an automated way.
To create a "bot" which will login and logout and check if credentials are correct you can use Selenium.
Selenium is a framework for testing web page ui but it can also be used to create automated checker/tester/bot for what you need.
It is also possible to setup selenium in a such way that it will use renderless browser engine so no browser will be visible during test/check. This also speeds up the process of testing.
I hope this will help you if you'll not find any better way and can be used as a final solution since it is not the best way to check credentials