Can you "get" multiple pages in parallel with chromedriver?
I am using Python, and as far as I understand selenium provides windowing API but does not allow opening a new window for a new driver.get() action. Trying to fetch a page while another is in process has proven problematic for me although I guess it might have been my wrong usage.
Currently I am opening a number of chromedriver sessions in parallel, which in turn results with X 5 times chrome processes open - This can get intimidating although it seems to work. I am just wandering now if calling driver.get(url) on an existing session (after a previous page was retrieved) might open extra tabs/windows in the "internal" chrome process and bloat memory?
You can open links in a new window using selenium by forcing a context click:
A couple suggestions:
https://stackoverflow.com/a/19152396/1387701
https://stackoverflow.com/a/45582818/1387701
You can then use the switch_to command to switch between these windows.
See more at https://www.browserstack.com/guide/how-to-switch-tabs-in-selenium-python
For some unknown reasons ,my browser open test pages of my remote server very slowly. So I am thinking if I can reconnect to the browser after quitting the script but don't execute webdriver.quit() this will leave the browser opened. It is probably kind of HOOK or webdriver handle.
I have looked up the selenium API doc but didn't find any function.
I'm using Chrome 62,x64,windows 7,selenium 3.8.0.
I'll be very appreciated whether the question can be solved or not.
No, you can't reconnect to the previous Web Browsing Session after you quit the script. Even if you are able to extract the Session ID, Cookies and other session attributes from the previous Browsing Context still you won't be able to pass those attributes as a HOOK to the WebDriver.
A cleaner way would be to call webdriver.quit() and then span a new Browsing Context.
Deep Dive
There had been a lot of discussions and attempts around to reconnect WebDriver to an existing running Browsing Context. In the discussion Allow webdriver to attach to a running browser Simon Stewart [Creator WebDriver] clearly mentioned:
Reconnecting to an existing Browsing Context is a browser specific feature, hence can't be implemented in a generic way.
With internet-explorer, it's possible to iterate over the open windows in the OS and find the right IE process to attach to.
firefox and google-chrome needs to be started in a specific mode and configuration, which effectively means that just
attaching to a running instance isn't technically possible.
tl; dr
webdriver.firefox.useExisting not implemented
Yes, that's actually quite easy to do.
A selenium <-> webdriver session is represented by a connection url and session_id, you just reconnect to an existing one.
Disclaimer - the approach is using selenium internal properties ("private", in a way), which may change in new releases; you'd better not use it for production code; it's better not to be used against remote SE (yours hub, or provider like BrowserStack/Sauce Labs), because of a caveat/resource drainage explained at the end.
When a webdriver instance is initiated, you need to get the before-mentioned properties; sample:
from selenium import webdriver
driver = webdriver.Chrome()
driver.get('https://www.google.com/')
# now Google is opened, the browser is fully functional; print the two properties
# command_executor._url (it's "private", not for a direct usage), and session_id
print(f'driver.command_executor._url: {driver.command_executor._url}')
print(f'driver.session_id: {driver.session_id}')
With those two properties now known, another instance can connect; the "trick" is to initiate a Remote driver, and provide the _url above - thus it will connect to that running selenium process:
driver2 = webdriver.Remote(command_executor=the_known_url)
# when the started selenium is a local one, the url is in the form 'http://127.0.0.1:62526'
When that is ran, you'll see a new browser window being opened.
That's because upon initiating the driver, the selenium library automatically starts a new session for it - and now you have 1 webdriver process with 2 sessions (browsers instances).
If you navigate to an url, you'll see it is executed on that new browser instance, not the one that's left from the previous start - which is not the desired behavior.
At this point, two things need to be done - a) close the current SE session ("the new one"), and b) switch this instance to the previous session:
if driver2.session_id != the_known_session_id: # this is pretty much guaranteed to be the case
driver2.close() # this closes the session's window - it is currently the only one, thus the session itself will be auto-killed, yet:
driver2.quit() # for remote connections (like ours), this deletes the session, but does not stop the SE server
# take the session that's already running
driver2.session_id = the_known_session_id
# do something with the now hijacked session:
driver.get('https://www.bing.com/')
And, that is it - you're now connected to the previous/already existing session, with all its properties (cookies, LocalStorage, etc).
By the way, you do not have to provide desired_capabilities when initiating the new remote driver - those are stored and inherited from the existing session you took over.
Caveat - having a SE process running can lead to some resource drainage in the system.
Whenever one is started and then not closed - like in the first piece of the code - it will stay there until you manually kill it. By this I mean - in Windows for example - you'll see a "chromedriver.exe" process, that you have to terminate manually once you are done with it. It cannot be closed by a driver that has connected to it as to a remote selenium process.
The reason - whenever you initiate a local browser instance, and then call its quit() method, it has 2 parts in it - the first one is to delete the session from the Selenium instance (what's done in the second code piece up there), and the other is to stop the local service (the chrome/geckodriver) - which generally works ok.
The thing is, for Remote sessions the second piece is missing - your local machine cannot control a remote process, that's the work of that remote's hub. So that 2nd part is literally a pass python statement - a no-op.
If you start too many selenium services on a remote hub, and don't have a control over it - that'll lead to resource drainage from that server. Cloud providers like BrowserStack take measures against this - they are closing services with no activity for the last 60s, etc, yet - this is something you don't want to do.
And as for local SE services - just don't forget to occasionally clean up the OS from orphaned selenium drivers you forgot about :)
OK after mixing various solutions shared on here and tweaking I have this working now as below. Script will use previously left open chrome window if present - the remote connection is perfectly able to kill the browser if needed and code functions just fine.
I would love a way to automate the getting of session_id and url for previous active session without having to write them out to a file during hte previous session for pick up...
This is my first post on here so apologies for breaking any norms
#Set manually - read/write from a file for automation
session_id = "e0137cd71ab49b111f0151c756625d31"
executor_url = "http://localhost:50491"
def attach_to_session(executor_url, session_id):
original_execute = WebDriver.execute
def new_command_execute(self, command, params=None):
if command == "newSession":
# Mock the response
return {'success': 0, 'value': None, 'sessionId': session_id}
else:
return original_execute(self, command, params)
# Patch the function before creating the driver object
WebDriver.execute = new_command_execute
driver = webdriver.Remote(command_executor=executor_url, desired_capabilities={})
driver.session_id = session_id
# Replace the patched function with original function
WebDriver.execute = original_execute
return driver
remote_session = 0
#Try to connect to the last opened session - if failing open new window
try:
driver = attach_to_session(executor_url,session_id)
driver.current_url
print(" Driver has an active window we have connected to it and running here now : ")
print(" Chrome session ID ",session_id)
print(" executor_url",executor_url)
except:
print("No Driver window open - make a new one")
driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()),options=myoptions)
session_id = driver.session_id
executor_url = driver.command_executor._url
Without getting into why do you think that leaving an open browser windows will solve the problem of being slow, you don't really need a handle to do that. Just keep running the tests without closing the session or, in other words, without calling driver.quit() as you have mentioned yourself. The question here though framework that comes with its own runner? Like Cucumber?
In any case, you must have some "setup" and "cleanup" code. So what you need to do is to ensure during the "cleanup" phase that the browser is back to its initial state. That means:
Blank page is displayed
Cookies are erased for the session
While running my load test for 1 or 5 or 10 or so concurrent users, the test runs far more users in parallel. it starts new users before the first thread finishes even though Start thread count is set to some number.
Can I avoid it by setting proper delays? If so how should the delay be set?
I also tried solving it by calling driver.quit() at the end of the test in a try catch block so threads close after exdcution but looks like it doesn't closes the threads (Chrome browser windows in this case) with errors in it.
try{
WebElement myDynamicElement3 = (new WebDriverWait(driver, 10))
.until(ExpectedConditions.presenceOfElementLocated(By.xpath("//*[text()='hello']")));
}catch(Exception e) {
throw new AssertionError("Element not found....", e);
}
driver.quit();
Please help which approach should I try and how.
Unfortunately it is not possible to provide the answer without seeing your Thread Group configuration, the way you instantiate the WebDriver, the way you're handling WebDriver instances between Thread Group iterations, etc.
In the meantime I can only recommend considering using WebDriver Sampler Plugin or at least look into its source code - this way you should learn how to properly instantiate and shut down the browser instances.
And last but not the least, using real browsers for load testing is not the best idea as modern browsers are very resource intensive, each browser instance requires full CPU core and at least 1 GB of RAM and you can simulate at least 100 virtual users using HTTP Request samplers at cost of one browser instance.
I'm automating functionality of My web project . I've created one test suit using selenium Webdriver which is working fine on Local Server(created on same). But while I do execute the same on Azure server(obviously slow as compared to local environment), The script get failed and I need to debug the whole script and need to put some wait where its taking time to locate the element. And after all changes on azure server some time it failed on local server too.
How to deal with slow website response ?
Is there any effective way of scripting which required less effort to execute the script on different environment ?
You can set Implicit wait (less desired) or Explicit waits.
var driver = new ChromeDriver();
driver.Manage().Timeouts().ImplicitlyWait(TimeSpan.FromSeconds(1000));
driver.FindElement(By.XPath("your xpath")).Click();
In this case, you're 'telling' Selenium to wait the specified timeout before it will search the element you want to be found. If the element was not found, you will get the Timeout error.
var driver = new ChromeDriver();
driver.FindElement(By.XPath("your xpath")).Click();
In this case you're telling Selenium to get the element automatically, so in case it will not be found, you will get a NoSuchElementException, with no timeouts because there's no time to wait before the element to be present.
And if we're expanding it to the Explicit wait:
var driver = new ChromeDriver();
var wait = new WebDriverWait(driver, new TimeSpan(0, 0, 1000));
wait.Until(d => d.FindElement(By.XPath("your xpath"))).Click();
In this case you're telling Selenium to keep searching for the element until it is found but no longer than the maximum timeout specified as a param to WebDriverWait.
Let's take for example the case where on your local environment a page loads on 1 minute and on Azure it takes 10 minutes (far stretched example), if you're going ahead with the Explicit wait, then you could set the timeout to let's say 11 minutes.
Those 11 minutes are the maximum timeout in which Selenium is polling for a specific element to be found. However, if you run your tests on the local environment and the element is found in 30 seconds, then your test will continue with the execution. It will not wait for the maximum timeout to complete before doing that.
It's most likely not time to locate the element, it's probably the speed of pages loading. Use WebDriverWaits for page transitions and any dynamic page changes. Once you wait for an element or dynamic section of the page to load, you can interact with the page without having to worry about speed. That will probably take care of most of your issues but without more specifics, it's hard to say what else might be going on.
I was doing selenium test behind on dev server behind the proxy.
Since the pages has some resources (i.e ads) that blocked by the proxy, the pages are in waiting state until it got network timeout.
So, when I navigate to another page using click, the selenium is waiting for the page to be fully loaded before continue the test.
Is there any way to stop selenium waiting for the page fully loaded, since the resources blocked by the proxy isn't actually needed for the test.
Thank you.
This question actually brough up multiple times. Because it often does not make sense to stop loading resources since you always do not know what resource needs to be stopped from loading. In case you really want to do, it can be accomplished though a simple javascript hack.
JavascriptExecutor js = (JavascriptExecutor) driver;
js.executeScript("return window.stop;");
Alrternatively,
You also can use Actions
Actions action = new Actions(driver);
action.sendKeys(Keys.ESCAPE);
Depending on the language binding it may change. My example is in java