I have this code:
async def acess_all():
def acess_localhost():
option = Options()
option.add_argument("--no-sandbox")
option.debugger_Address="127.0.0.1:8081"
driver = webdriver.Chrome(options=option)
driver.get("http://127.0.0.1:8081")
wait = WebDriverWait(driver, 5)
wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, 'a.item'))).click()
try:
t = threading.Thread(target=get_infos)
t.start()
os.system("chromium --remote-debugging-port=8081 https://google.com")
except:
print("Error!")
What I need:
Guys, as you can see, this code opens the Chrome browser on the Google page, and my Selenium code opens the browser on localhost because it's accessing where the remote-debbuging address points to, but I can't access the console tab of devtools, I need to access this tab using Selenium and run a javascript command to copy the cookie in json format, but I can't, how can I access the devtools console tab?
I believe I've figured out a way to run a javascript command in the console and get a return value (using selenium, as in the question).
Note: I'm running this on a Windows computer, but the idea should remain the same on a different operating system. You might have to tweak a few things in this code. Also, all Chrome sessions have to be closed beforehand to get this to work.
Ok, unless I interpreted your question wrong (tell me if I did), this is the basic order of the things you want you want to happen when:
Open the regular chrome browser (by regular, I mean not selenium, but regular chrome.exe or Google Chrome.app) to google.com and set the debugging port (which I assume is what you're doing when you run this command: chromium --remote-debugging-port=8081 https://google.com)
Open Chromedriver (Selenium) and go to the locally-hosted debugger window at 127.0.0.1:8081
Select the "Google" window option in the list of available windows at 127.0.0.1:8081
Once the devtools window opens, move to the Console tab
Finally, run some Javascript in the Console's input box and somehow get a return value for a cookie
You already did the first few items (1-3) in your code but you needed help figuring out the rest. I think I've found a way to do it.
So assuming that you already opened the google.com window on the Chrome browser and the 127.0.0.1:8081 window on localhost, all you need to do now is access the Console.
Here's what my chromedriver (selenium) browser screen looks like at this point, just for reference.
We'll start by waiting until a specific element (devtools-elements-breadcrumbs) has loaded on the page. We wait for this so we are sure that the page is loaded enough to navigate. I found this element by looking at the driver page_source. Here's how we wait for it:
wait.until(EC.presence_of_element_located((By.TAG_NAME, "devtools-elements-breadcrumbs")))
Once the breadcrumb element is located we can move to the console window by sending a shortcut to the chromedriver browser telling it to move right one tab in the Inspector window (to Console). The command is Control + ] (Windows/Linux) or Command + ] (Mac), to go to the next panel. This is how we send that shortcut to the selenium window (once again, using Windows):
click_console = ActionChains(driver)
click_console.key_down(Keys.CONTROL).send_keys(']').key_up(Keys.CONTROL).perform()
or on a Mac,
click_console = ActionChains(driver)
click_console.key_down(Keys.COMMAND).send_keys(']').key_up(Keys.COMMAND).perform()
After moving to the Console, we now have to wait for the console area to load, so we wait for all the CodeMirror class elements (once again, found using driver.page_source)
# wait for console area to open
wait.until(EC.presence_of_all_elements_located((By.CLASS_NAME, "CodeMirror")))
Now the last thing to do is to send the javascript command to the console. In this example, I'm searching for the APISID cookie.
cookie_name = "SEARCH_SAMESITE"
# javascript find the cookie
# https://stackoverflow.com/a/59603055/11073064
js_command = '(\"; \" + document.cookie).split(\"; {}\").pop().split(\';\').shift();'.format(cookie_name)
# send the command to the selenium webbrowser
send_js_command = ActionChains(driver)
send_js_command.send_keys(js_command, Keys.ENTER).perform()
and to get the value outputted in the console after you run that command:
# wait for return value span tag to be found
element = wait.until(EC.presence_of_element_located((By.CSS_SELECTOR, 'span.object-value-string.source-code')))
value = element.text
driver.close()
Here's the full code I used (on Windows).
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
import threading
from selenium.webdriver.chrome.options import Options
from selenium.webdriver import ActionChains
from selenium.webdriver.common.keys import Keys
import os
def start_chrome_browser_with_debug():
# this is the path to your regular google chrome browser, not selenium chromedriver
# note the chrome.exe, not chromedriver.exe.
# point the path to your regular chrome browser
os.system('"C:\Program Files (x86)\Google\Chrome\Application\chrome.exe" --remote-debugging-port=8081 https://google.com')
cookie_name = "SEARCH_SAMESITE"
js_command = '(\"; \" + document.cookie).split(\"; {}\").pop().split(\';\').shift();'.format(cookie_name)
chrome_browser = threading.Thread(target=start_chrome_browser_with_debug)
chrome_browser.start()
option = Options()
option.add_argument("--no-sandbox")
option.debugger_Address = "127.0.0.1:8081"
driver = webdriver.Chrome(options=option)
driver.get("http://127.0.0.1:8081")
wait = WebDriverWait(driver, 5)
wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, 'a.item'))).click()
wait.until(EC.presence_of_element_located((By.TAG_NAME, "devtools-elements-breadcrumbs")))
click_console = ActionChains(driver)
click_console.key_down(Keys.CONTROL).send_keys(']').key_up(Keys.CONTROL).perform()
wait.until(EC.presence_of_all_elements_located((By.CLASS_NAME, "CodeMirror")))
send_js_command = ActionChains(driver)
send_js_command.send_keys(js_command, Keys.ENTER).perform()
element = wait.until(EC.presence_of_element_located((By.CSS_SELECTOR, 'span.object-value-string.source-code')))
value = element.text
driver.close()
os.system('taskkill /F /IM chrome.exe /T')
print(value)
Actually, you can execute the javascript command by selenium itself :
driver.execute_script("some javascript code here");
You can't use Selenium to interact with the dev tools console, however you don't need it to get the cookies, just use get_cookies() from the webdriver and convert it to json using json.dumps()
import json
cookies = driver.get_cookies()
js = json.dumps(cookies)
I couldn't make this work on a Mac using only Python and Selenium. However, I found a solution that works if I use pyautogui in addition.
I'm working on a file, duals7.svg, saved locally. I want to be able to open it, play with it and watch what happens at the Javascript developer console in Chrome.
The script below does exactly what I want:
# chrome.py
# https://www.browserstack.com/guide/python-selenium-to-run-web-automation-test
# https://www.tutorialspoint.com/how-do-i-pass-options-to-the-selenium-chrome-driver-using-python
# https://stackoverflow.com/questions/36311191/how-to-open-chrome-developer-console-in-selenium-webdriver-using-java
# https://towardsdatascience.com/controlling-the-web-with-python-6fceb22c5f08
from selenium import webdriver
from selenium.webdriver.chrome.options import Options as ChromeOptions
import pyautogui
op = webdriver.ChromeOptions ()
op.add_argument ('--allow-file-access-from-files')
op.add_argument ('--auto-open-devtools-for-tabs')
driver = webdriver.Chrome (options=op)
driver.get ('file:///Users/murray/projects/warp/dual7.svg')
console = (850, 175)
pyautogui.moveTo (console [0], console [1], duration = 0.5)
pyautogui.click (console [0], console [1])
(I don't think the pyautogui.moveTo command is needed. Including it allows me to watch the mouse move, which I like.)
When I run it (python chrome.py), I see my svg "app" in Chrome with the Javascript dev console open:
The coordinates of the console button were determined by trial and error. Please forgive the aesthetics of the app: it'll look nice when it's done.
When I interact with my "app" I see my console.log () debugging messages.
If you're interested, the app is being modified from
/* elliptic-mesh.js
*
* generates a structured mesh bounded by four splines by
* solving the elliptic Laplace equation
*
* For more info see:
* http://www.particleincell.com/2012/online-meshing-example/
* http://www.particleincell.com/2012/bezier-splines/
*
* Lubos Brieda, Particle In Cell Consulting LLC, 2012
* you may freely use this algorithm in your codes but whenever possible
* please include a link/reference to the source article
*/
My version generalizes Dr. Brieda's code a bit, adds some debugging statements, and will add save and load options.
I am using dotnet core C# and selenium to load pages and run some tests.
I want to get the details that are displayed in the chrome developer tool network tab.
The list of resources that are loaded and the size and path for each resource, and be able to calculate the total load time and transfer size for the web page and all it's resources.
I know I could get the list of resources with javascript
var list = performance.getEntriesByType('resource');
But that would not return CORS items and so isn't an exact list of all the items.
Is there a way to get the full list of resources with Selenium? or can I generate the HAR file with selenium and get the list of resources from that?
In Selenium 4 there will be such option, to get the Chrome DevTools.
Till then, there're no better solutions in C# I am aware of. You could try get the data, using JS:
String scriptToExecute = "var performance = window.performance || window.mozPerformance || window.msPerformance || window.webkitPerformance || {}; var network = performance.getEntries() || {}; return network;";
String netData = ((JavascriptExecutor)driver).executeScript(scriptToExecute).toString();
Is there any way to switch from headless to headed in the middle of a task in Selenium?
For example, I'm running some workflow headless, but in the middle I want the user to be able to push a button to see what the actual browser window looks like. How do I do that?
Not possible.
But it is possible to hide browser window from the screen :) :
driver.Manage().Window.Position =new Point(-2000, 0);
Unfortunately, switching between UI (headed) and HeadLess execution through selenium is not possible.
For example, Selenium creates an instance of browser of type WebDriver (either Firefox/Chrome/IE/HtmlUnit). Selenium can operate only on the instance it has created, and during creation time it will be defined if it is a UI test (Firefox/Chrome/IE) or Headless (HtmlUnit).
So, the design of Selenium prevents to switch between headless/UI execution in the middle of the test, as per your requirement.
Using xvfb you can do this. You can develop your tests using any driver, so, after configured the xvfb, you just need to use
xvfb-run tests
to run headless and if you don't want, you can run the tests without xvfb parameter.
A linux machine is required to use xvfb.
I've done this for one of my daily tasks - signin into firewall. It was a 2FA process, which required combo of browser interaction and a (random generated) token. For the first part a simple powershell script was running in headless mode, so when the site prompts for your token, it turns IE to visible and waits for user input. If such is provided, again switches to headless and finishes the process. The user is informed for success/fail of the operation with dialog message box.
You can start with such simple IE automation and optimize your .ps1 script later. The solution I was using back in the days:
$ie = new-object -ComObject "InternetExplorer.Application"
$requestUri = http://-your-site-here
# switch the browser headless mode on/off
#$ie.visible = $true
$ie.silent = $true
$ie.navigate($requestUri)
while($ie.Busy) { Start-Sleep -Milliseconds 100 }
# get DOM in order to interact
$doc = $ie.Document
$userInputId = "your-uname-id-here"
$doc.getElementsByTagName("input") | % {
if ($_.id -ne $null){
if ($_.id.Contains($userInputId)) { $_.value = "your-username-here" }
}
}
# can call JS events on elements
# like $btn.click()
Write-Verbose "Login Complete"
I'm trying to automate a scenario where I login to a site with two users on two different browsers where further interaction will then occur in the same test. It runs up to the point where Chrome has launched the URL and the recorded steps fail giving an error that web page cannot be accessed. The recorded steps do work on Chrome if they were separated out into its testmethod of its own.
BrowserWindow window;
window = BrowserWindow.Launch(new Uri("http://example.com"));
this.UIMap.ClickOnSignInLink();
this.UIMap.TypeInValidUserCredential();
this.UIMap.ClickOnSignInButton();
this.UIMap.AssertUserIsLoggedIn();
BrowserWindow window2;
BrowserWindow.CurrentBrowser = "Chrome";
window2 = BrowserWindow.Launch(new Uri("http://example.com"));
this.UIMap.ClickOnSignInLink();
this.UIMap.TypeInValidUserCredential();
this.UIMap.ClickOnSignInButton();
Test method FeaturesTest.LiveBidding threw exception:
Microsoft.VisualStudio.TestTools.UITest.Extension.UITestControlNotAvailableException: The web page could not be accessed. If the page is refreshing, please wait until the page refreshes and then perform actions on it.
Only way i managed to do this is by closing the first browser. To do this you can do either of these below. Add this after IE job is finished and you want to switch to Chrome.
window.Close();
or initialize a new playback
if (Playback.IsInitialized)
{
Playback.Cleanup();
Playback.Initialize();
}
I am working on writing a story for a bdd framework which uses jbehave/selenium/webdriver and am having a problem where the test encounters an error while running the story but appears to be fine when running manually. I'm having a problem where javascript for the functionality I'm testing behaves slightly different when I'm running tests manually on firefox vs through selenium web driver on the same system/version of firefox and this difference is causing a js error.
I've debugged and basically the root of the problem appears to be that var request_XML_container = $('div_appendpoint_id'); returns something different when I'm running the test manually vs when I run through the bdd framework.
var request_XML_container = $('div_appendpoint_id');
request_XML_container.innerHTML = encoded_xml_from_request;
var pos = method_to_get_position('id_of_place_div_should_be_appended_to');
// JS exception is thrown saying that style is not defined **ONLY**
// when running through web driver. Running test manually on
// same system and same browser works fine.
request_XML_container.style.left = (pos[0] - 300) + 'px';
request_XML_container.style.top = (pos[1] + 25) + 'px';
request_XML_container.style.display = "block";
Why this would work fine when running manually that var request_XML_container = $('div_appendpoint_id'); would return an item with style defined, but when running through webdriver that the style attribute of the element would not be defined?
UPDATE: I had originally thought that this was updating an iframe, but I read the markup wrong and the iframe I saw is a sibling - not a parent - of the element where the response is being appended to. I'm trying to simply append the response to a div. To be honest, this only makes things more confusing as grabbing a div by id should be pretty straight forward and I'm now sure why webdriver would be producing a different return element in this situation.
UPDATE 2: Steps to reproduce and information about the system I'm on:
Use webdriver to navigate to this url: http://fiddle.jshell.net/C3VB5/11/show/
Have webdriver click the button. It should not work
Run your test again, but pause put a breakpoint at your code to click the driver
Click the button on the browser that webdriver opened. It should not work
Refresh the browser page on the browser that webdriver opened. Now, it should work.
System details:
OS : OS X 10.8.5 (12F37)
IDE : Eclipse Kepler: Build id: 20130614-0229
Browser (used manually and by webdriver) : Firefox 23.0.1
Selenium version: 2.35.0
UPDATE 3: I have provided this maven project on github to aid in reproducing: https://github.com/dkwestbr/WebdriverBug/tree/master/Webdriver
Synopsis/tl:dr; Basically, in certain situations it appears as though webdriver is overwriting the '$()' javascript method with a method that does not return an HTMLElement with innerHTML or style defined (among other things). This post details the issue and how to reproduce.
I have opened this ticket to track the issue: https://code.google.com/p/selenium/issues/detail?id=6287&thanks=6287&ts=1379519170
I have confirmed that this is a bug with the Thucydides framework (understandable since they still aren't at a 1.0 release).
Issue can be tracked here: https://java.net/jira/browse/THUCYDIDES-203