Override geolocation using robot framework and seleniumlib - selenium

I've trying to override geolocation using robot + seleniumLib, all docs mention this python code, using cdp commands, but how do i import this code in this case?
I read something about using the keyword "Create Webdriver", but really dont know how it works.
Can anyone help me with this?
.py and .robot file are in same folder.
.robot file
***Settings***
Library SeleniumLibrary
Library geo.py
Suite Teardown End
***Variables***
***Test Cases***
Test Location
Should appear the overrided location
***Keywords***
End
Close All Browsers
Should appear the overrided location
Open Browser https://browserleaks.com/geo Chrome
Wait Until Page Contains HTML5 Geolocation API
Scroll Element Into View xpath://*[#id="footer"]
Page Should Not Contain Element xpath://*[#alt="BR"] #flag img from my current country location
Sleep 5
Close Browser
.py file
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
def geoLocation_Test():
driver = webdriver.Chrome()
Map_coordinates = dict({
"latitude": 41.8781,
"longitude": -87.6298,
"accuracy": 100
})
driver.execute_cdp_cmd("Emulation.setGeolocationOverride", Map_coordinates)

You shouldn't create a new instance of the browser. You can get SeleniumLibrary's instance of the driver through the driver attribute of the library.
Here's how to define a function to set the geolocation in chrome:
from robot.libraries.BuiltIn import BuiltIn
def set_geolocation(latitude, longitude, accuracy=100):
map_coordinates = {
"latitude": float(latitude),
"longitude": float(longitude),
"accuracy": int(accuracy),
}
selib = BuiltIn().get_library_instance("SeleniumLibrary")
selib.driver.execute_cdp_cmd("Emulation.setGeolocationOverride", map_coordinates)
The following test passes, showing the location to be Chicago, IL even though that's not my physical location nor the location of my VPN connection.
*** Settings ***
Library geo.py
Library SeleniumLibrary
Suite Setup open browser about:blank chrome
Suite Teardown close browser
*** Test Cases ***
Geolocation Example
[Setup] set geolocation 41.8781 -87.6298 100
Go to https://my-location.org
Element should contain id=latitude 41.8781
Element should contain id=longitude -87.6298
capture page screenshot

Related

Download file through Google Chrome RemoteWebDriver- headless mode in Linux using Selenium Java [duplicate]

I'm do me code in Cromedrive in 'normal' mode and works fine. When I change to headless mode it don't download the file. I already try the code I found alround internet, but didn't work.
chrome_options = Options()
chrome_options.add_argument("--headless")
self.driver = webdriver.Chrome(chrome_options=chrome_options, executable_path=r'{}/chromedriver'.format(os.getcwd()))
self.driver.set_window_size(1024, 768)
self.driver.command_executor._commands["send_command"] = ("POST", '/session/$sessionId/chromium/send_command')
params = {'cmd': 'Page.setDownloadBehavior', 'params': {'behavior': 'allow', 'downloadPath': os.getcwd()}}
self.driver.execute("send_command", params)
Anyone have any idea about how solve this problem?
PS: I don't need to use Chomedrive necessarily. If it works in another drive it's fine for me.
First the solution
Minimum Prerequisites:
Selenium client version: Selenium v3.141.59
Chrome version: Chrome v77.0
ChromeDriver version: ChromeDriver v77.0
To download the file clicking on the element with text as Download Data within this website you can use the following solution:
Code Block:
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.chrome.options import Options
options = Options()
options.add_argument("--headless")
options.add_argument("--window-size=1920,1080")
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option('useAutomationExtension', False)
driver = webdriver.Chrome(chrome_options=options, executable_path=r'C:\Utility\BrowserDrivers\chromedriver.exe', service_args=["--log-path=./Logs/DubiousDan.log"])
print ("Headless Chrome Initialized")
params = {'behavior': 'allow', 'downloadPath': r'C:\Users\Debanjan.B\Downloads'}
driver.execute_cdp_cmd('Page.setDownloadBehavior', params)
driver.get("https://www.mockaroo.com/")
driver.execute_script("scroll(0, 250)");
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button#download"))).click()
print ("Download button clicked")
#driver.quit()
Console Output:
Headless Chrome Initialized
Download button clicked
File Downloading snapshot:
Details
Downloading files through Headless Chromium was one of the most sought functionality since Headless Chrome was introduced.
Since then there were different work-arounds published by different contributors and some of them are:
Downloading with chrome headless and selenium
Python equivalent of a given wget command
Now the, the good news is Chromium team have officially announced the arrival of the functionality Downloading file through Headless Chromium.
In the discussion Headless mode doesn't save file downloads #eseckler mentioned:
Downloads in headless work a little differently. There's the Page.setDownloadBehavior devtools command to set a download folder. We're working on a way to use DevTools network interception to stream the downloaded file via DevTools as well.
A detailed discussion can be found at Issue 696481: Headless mode doesn't save file downloads
Finally, #bugdroid revision seems to have nailed the issue for us.
[ChromeDriver] Added support for headless mode to download files
Previously, Chromedriver running in headless mode would not properly download files due to the fact it sparsely parses the preference file given to it. Engineers from the headless chrome team recommended using DevTools's "Page.setDownloadBehavior" to fix this. This changelist implements this fix. Downloaded files default to the current directory and can be set using download_dir when instantiating a chromedriver instance. Also added tests to ensure proper download functionality.
Here is the revision and commit
From ChromeDriver v77.0.3865.40 (2019-08-20) release notes:
Resolved issue 2454: Headless mode doesn't save file downloads [Pri-2]
Solution
Update ChromeDriver to latest ChromeDriver v77.0 level.
Update Chrome to Chrome Version 77.0 level. (as per ChromeDriver v76.0 release notes)
Note: Chrome v77.0 is yet to be GAed/pushed for release so till then you can download and install a development build and test either from:
Chrome Canary
Latest build from the Dev Channel
Outro
However Mac OSX users have a wait for their pie as On Chromedriver, headless chrome crashes after sending Page.setDownloadBehavior on MacOSX.
Chomedriver Version: 95.0.4638.54
Chrome Version 95.0.4638.69
from selenium.webdriver.chrome.options import Options
options = Options()
options.add_argument("--headless")
options.add_argument("--start-maximized")
options.add_argument("--no-sandbox")
options.add_argument("--disable-extensions")
options.add_argument('--disable-dev-shm-usage')
options.add_argument("--disable-gpu")
options.add_argument('--disable-software-rasterizer')
options.add_argument("user-agent=Mozilla/5.0 (Windows Phone 10.0; Android 4.2.1; Microsoft; Lumia 640 XL LTE) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.135 Mobile Safari/537.36 Edge/12.10166")
options.add_argument("--disable-notifications")
options.add_experimental_option("prefs", {
"download.default_directory": "C:\\link\\to\\folder",
"download.prompt_for_download": False,
"download.directory_upgrade": True,
"safebrowsing_for_trusted_sources_enabled": False,
"safebrowsing.enabled": False
}
)
What seemed to work was that I used "\\" instead of "/" for the address. The latter approach didn't throw any error, but didn't download any documents either. But, using double back slashes did the job.
For javascript use below code:
const chrome = require('selenium-webdriver/chrome');
let options = new chrome.Options();
options.addArguments('--headless --window-size=1500,1200');
options.setUserPreferences({ 'plugins.always_open_pdf_externally': true,
"profile.default_content_settings.popups": 0,
"download.default_directory": Download_File_Path });
driver = await new webdriver.Builder().setChromeOptions(options).forBrowser('chrome').build();
Then switch tabs as soon as you click the download button:
await driver.sleep(1000);
var Handle = await driver.getAllWindowHandles();
await driver.switchTo().window(Handle[1]);
This C# works for me
Note the new headless option https://www.selenium.dev/blog/2023/headless-is-going-away/
private IWebDriver StartBrowserChromeHeadlessDriver()
{
var chromeOptions = new ChromeOptions();
chromeOptions.AddArgument("--headless=new");
chromeOptions.AddArgument("--window-size=1920,1080");
chromeOptions.AddUserProfilePreference("download.default_directory", downloadFolder);
var chromeDownload = new Dictionary<string, object>
{
{ "behavior", "allow" },
{ "downloadPath", downloadFolder }
};
var driver = new ChromeDriver(driverFolder, chromeOptions, TimeSpan.FromSeconds(timeoutSecs));
driver.ExecuteCdpCommand("Browser.setDownloadBehavior", chromeDownload);
return driver;
}
import pathlib
from selenium.webdriver import Chrome
driver = Chrome()
driver.execute_cdp_cmd("Page.setDownloadBehavior", {
"behavior": "allow",
"downloadPath": str(pathlib.Path.home().joinpath("Downloads"))
})
I don't think you should be using the browser for downloading content, leave it to Chrome developers/testers.
I believe you should rather get href attribute of the element you want to download and obtain it using requests library
If your site requires authentication you could fetch cookies from the browser instance and pass them to requests.Session.

Cannot locate element by Xpath for chrome download page

I am trying to write code to check download is completed by selenium and chrome driver. My idea is
1.Go to download page("chrome://downloads/")
2.Check the url to ensure we have downloaded file from that site (locate http://xxxxxxxx)
3.Check download status( If I found "show in folder", it implies download success. if not, download failed)
I am stucking in step2, when I try to locate the url. I used developer tools, move cursor to the url to locate the element,and then I right click and copy the xpath. The xpath is like this
//*[#id="file-link"]
And then I try to click ctrl+F in developer tool and paste the xpath again. I cannot locate the element. Why? Checked there is no frame.
It shadow DOM and need to select /deep/ using CSS selector
downloads-manager/deep/downloads-item/deep/a[id="file-link"]
Another way of doing this could be writing a small Java utility for the same using File Class.
Something like this:
File f = new File("C:\\Users\\username\\Downloads\\Users_" + fielName+ ".xls");
Assert.assertTrue(f.exists());
This work for me on python:
from selenium import webdriver
from selenium.webdriver.support.wait import WebDriverWait
# disable file preview
options = webdriver.ChromeOptions()
options.add_experimental_option(
'prefs', {
'profile.default_content_settings.popups': 0,
'download.default_directory': '/some/download/dir',
'download.prompt_for_download': False,
'download.directory_upgrade': True,
'plugins.always_open_pdf_externally': True,
'plugins.plugins_disabled': 'Chrome PDF Viewer',
'disable-popup-blocking': True
}
)
driver = webdriver.Chrome(options=options)
# start load file
driver.get('url-to-file')
def condition_load_file(dr: webdriver.Chrome):
return dr.execute_script("""
try {
var el = document.querySelector("body > downloads-manager").shadowRoot.querySelector("#frb0").shadowRoot.querySelector("#show");
return el !== null;
} catch(exc) {
return false;
}
""")
# open new window and go to chrome://downloads/
driver.execute_script("window.open('');")
driver.switch_to.window(driver.window_handles[1])
driver.get('chrome://downloads/')
# waiting for download to finish
WebDriverWait(driver, 120).until(condition_load_file)

Geb test ignoring GebConfig.groovy file launched in IntelliJ

Running in IntelliJ IDEA.
GebConfig.groovy is in /src/test/resources.
I am using the Chrome driver.
When I type
System.setProperty("webdriver.chrome.driver", "my/path")
inside my spec file, and I right click and select run, the test works, meaning it opens Chrome and loads the page.
When I don't do that in the spec file, but just leave it in the GebConfig.groovy file, I get the error message "the page to the driver executable must be set".
There's an air-gap, so I can't copy-paste; I'll type as much as I can here:
GebConfig.groovy:
import org.openqa.selenium.chrome.ChromeDriver
...
environments {
chrome {
System.setProperty("webdriver.chrome.driver", "my/path")
driver = {new ChromeDriver()}
}
}
The spec file is really simple, like the example on GitHub
import LoginPage
import geb.spock.GebReportSpec
class LoginSpec extends GebReportSpec
{
// Works when I put this here, but I should not have to do this!
System.setProperty("webdriver.chrome.driver", "my/path")
def "user can log in" () {
when: "log in as me"
def loginPage = to LoginPage
loginPage.login("me")
then:
....
}
}
To fix your problem if you want to keep the path in the geb config, setting the path outside of the environment section like so should work:
import org.openqa.selenium.chrome.ChromeDriver
System.setProperty("webdriver.chrome.driver", "my/path")
//You can also set the driver up here as a default and running with an environment set will override it
driver = {new ChromeDriver()}
environments {
chrome {
driver = {new ChromeDriver()}
}
}
Personally I would avoid adding the driver path to the geb config and create a run configuration in intelliJ for running locally.
Right click the spec file > Click "Create 'nameOfMySpec'".
Now add your driver path to the VM parameters:
-Dgeb.env=chrome -Dwebdriver.chrome.driver=my/path
It's also worth considering a shell script that could then also be called via Jenkins etc:
mvn test -Dgeb.env=chrome -Dwebdriver.chrome.driver=my/path

to set MutationObserver, How to inject javascript before page-loading using Selenium

I'm trying to set MutationObserver for observing page mutation while loading.
In order to do that, MutationObserver should be configured before page loading.
With selenium-chromedriver, couldn't find the way to inject JS for such purpose.
I know chrome extension can do that but extensions won't work on headless mode.
That's the problem.
It's possible via the DevTool API by calling Page.addScriptToEvaluateOnNewDocument
from selenium import webdriver
from selenium.webdriver.remote.webdriver import WebDriver
import json
def send(driver, cmd, params={}):
resource = "/session/%s/chromium/send_command_and_get_result" % driver.session_id
url = driver.command_executor._url + resource
body = json.dumps({'cmd': cmd, 'params': params})
response = driver.command_executor._request('POST', url, body)
if response['status']:
raise Exception(response.get('value'))
return response.get('value')
def add_script(driver, script):
send(driver, "Page.addScriptToEvaluateOnNewDocument", {"source": script})
WebDriver.add_script = add_script
# launch Chrome
driver = webdriver.Chrome()
# add a script which will be executed when the page starts loading
driver.add_script("""
if (window.self === window.top) { // if main document
console.log('add script');
}
""")
# load a page
driver.get("https://stackoverflow.com/questions")
We can now use execute_cdp_cmd(cmd, cmd_args) to execute Chrome Devtools Protocol command in Selenium
from selenium import webdriver
driver = webdriver.Chrome()
driver.execute_cdp_cmd(
"Page.addScriptToEvaluateOnNewDocument",
{
"source": """// Your JavaScript here"""
}
)
driver.get("https://stackoverflow.com")
driver.quit()
The argument for "source" is just a string. So you can actually write your script in a .js file (for syntax highlighting) and read it using Python

Take multiple browser snapshots

I am an ASP.NET web developer and need to submit artifacts of the developed page, to the testers.
These include:
Snapshots of the page when opened in different browsers.
Same thing, with different versions of the browsers.
I do this manually (one by one) and also from different systems.(because IE8 is available only at a particular system...so on..). Yes of course time consuming.
Is there a way that could simplify my life in this.(something like browserstack.com)
I am looking at Selenium grid. But this requires some coding too and using eclipse etc. Not sure yet if taking screenshots is possible too and I have no knowledge of selenium.
Appreciate any help in this. Thank you
Create a config file for the Selenium Hub
The file name should be for example hubConfig.json and save it in the same folder where you will start the Hub. The host contains the IP of the HUB and the port contains the port number ... :)
{
"host": "127.0.0.1",
"port": 4444
}
Run your Selenium Hub, and load the configuration file too
Create a new file with the following name: startSeleniumHub.cmd and add the following command in it:
java -jar selenium-server-standalone.jar -role hub -hubConfig hubconfig.json
You can download the latest version Selenium Server Standalone on the Selenium HQ website:
http://www.seleniumhq.org/download/
Create a config file for the Node
The file name should be for example nodeConfig.json and save it in the same folder on the same PC where you will start the Node.
{
"capabilities":[
{
"platform":"WINDOWS",
"browserName":"firefox",
"firefox_binary":"C:\\Program Files (x86)\\Mozilla Firefox\\firefox.exe",
"maxInstances":5,
"seleniumProtocol":"WebDriver"
},
{
"platform":"WINDOWS",
"browserName":"chrome",
"maxInstances":5,
"seleniumProtocol":"WebDriver"
},
{
"platform":"WINDOWS",
"browserName":"internet explorer",
"maxInstances":5,
"version":11,
"seleniumProtocol":"WebDriver"
}
],
"configuration":{
"port":5555,
"host":"IP_OF_THE_NODE_PC",
"register":true,
"hubHost":"IP_OF_THE_HUB",
"hubPort": 4444,
"maxSession":1
}
}
If something is not understandable for you in this config, you can read more details about it on the following page:
https://code.google.com/p/selenium/wiki/Grid2
If you want, you can add more browsers like Edge, Safari, Opera, etc...
Connect your PCs to the Selenium Hub as a Node with the configuration file and with the WebDrivers as well
Create a new file with the following name: startSeleniumNode.cmd and add the following command in it:
java -jar "selenium-server-standalone-X.XX.X.jar" -role node -nodeConfig "nodeConfig.json" -Dwebdriver.chrome.driver="chromedriver.exe" -Dwebdriver.ie.driver="IEDriverServer.exe"
You can download the latest Chrome and IE WebDriver on the same page where you downloaded the Selenium Server Standalone:
http://www.seleniumhq.org/download/
Write some JAVA code :)
It is just an example, I'm sure that there are better solutions. For example you can use TestNG to create different tests, add parameters to your tests, or run your tests in parallel, etc... You can find more information on the TestNG site: http://testng.org/doc/index.html
import org.apache.commons.io.FileUtils;
import org.openqa.selenium.OutputType;
import org.openqa.selenium.Platform;
import org.openqa.selenium.TakesScreenshot;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.openqa.selenium.remote.RemoteWebDriver;
import java.io.File;
import java.io.IOException;
import java.net.URL;
public class ScreenshotMakerTest
{
public static void main(String [] args) throws IOException
{
// Init a new DesiredCapabilites which will setup the WebDriver to open a specific browser.
DesiredCapabilities dc = new DesiredCapabilities();
// Set the browser. If you want to open a Chrome, you can modify it to: DesiredCapabilites.chrome(); etc...
dc = DesiredCapabilities.internetExplorer();
// Set the Platform. It must be the same what you defined in the nodeConfig.json.
dc.setPlatform(Platform.WINDOWS);
// Set the version of the browser. If you didn't set any version in the nodeConfig.json, you can skip this line.
dc.setVersion("11");
// Create the WebDriver which will open the given browser on a Node
WebDriver driver = new RemoteWebDriver(new URL("IP_OF_THE_HUB:4444/wd/hub"), dc);
// Open a page
driver.get("http://google.com");
// Create screenshot about the current page
File scrFile = ((TakesScreenshot)driver).getScreenshotAs(OutputType.FILE);
// Save the screenshot into a file
FileUtils.copyFile(scrFile, new File("c:\\screenshots\\screenshot.png"));
// Close the browser
driver.quit();
}
}
In this example, your code will open an Internet Explorer 11 on a Windows Node, open the http://google.com url and create a screenshot about it, and save it to c:/screenshots/screenshot.png.
Hope it helps.