Selenium WebDriver Consumes 19,000 MB of Memory.. Takes 10 min to Launch Firefox - selenium

I'm using the following code to launch Selenium WebDriver:
public static WebDriver launchWebDriver(String ipAddress, String port)
{
System.setProperty("webdriver.gecko.driver", "PATH_TO_GECKO_DRIVER");
String firefoxInstallationFolder = "PATH_TO_FIREFOX_INSTALLATION_FOLDER";
File firefoxProfileFile = new File(firefoxInstallationFolder);
FirefoxProfile firefoxProfile = new FirefoxProfile(firefoxProfileFile);
firefoxProfile.setPreference("datareporting.policy.dataSubmissionPolicyAccepted", false);
firefoxProfile.setPreference("dom.max_chrome_script_run_time", 0);
firefoxProfile.setPreference("dom.max_script_run_time", 0);
firefoxProfile.setPreference("media.peerconnection.enabled", false);
FirefoxOptions firefoxOptions = new FirefoxOptions();
firefoxOptions.setProfile(firefoxProfile);
firefoxOptions.addPreference("dom.popup_maximum", 200);
firefoxOptions.addPreference("dom.webnotifications.enabled", false);
firefoxOptions.addPreference("media.peerconnection.enabled", false);
firefoxOptions.addPreference("general.useragent.override", "CUSTOM_UA");
DesiredCapabilities dc = DesiredCapabilities.firefox();
if (ipAddress != null && port != null) {
Proxy proxy = new Proxy();
proxy.setHttpProxy(ipAddress + ":" + port);
proxy.setFtpProxy(ipAddress + ":" + port);
proxy.setSslProxy(ipAddress + ":" + port);
dc.setCapability(CapabilityType.PROXY, proxy);
}
firefoxOptions.merge(dc);
WebDriver driver = new FirefoxDriver(firefoxOptions);
return driver;
}
Before running above code Task Manager showed Java Platform Binary consuming 212 MB of memory. After running above code (nothing else was running on my machine at the time) the memory usage starting creeping up to 492 MB then to 636 MB then to 1,946 MB and to 13,000 MB. It hit a high of over 19,000 MB. I'm not running anything other than the default code from Selenium V3.
Below are some screenshots I captured showing how the memory is increasing rapidly.
Note: It started at less than 1,000 MB, but I only started capturing at 12,000 MB but you can see how it zooms from 12,000 MB to over 17,000 MB in screenshots below. Nothing else is running on my PC.
This is true whether or not I pass actual ipAddress and port as argument or whether I pass null for both arguments. The only thing different I can think of is that I've manually set proxy under Options -> General -> Network Settings -> Settings -> Manual Proxy Configuration. I'm obviously running a specific Firefox Profile as well. But I have no idea why this should cause such long launch times and consume so much memory. We're dealing with almost 20 GB of memory!
What's causing such a big memory hog and long launch times?
Thanks

This happens than you reuse Firefox Profile with cached images (may be it is GecoDriver bug I don't know). To fix the problem:
Clean up cache in you Firefox Profile. You can simply delete everything in folder <your_firefox_profile>/cache2
Turn off disk caching in your Firefox browser running under Selenium. May be you can do it with FirefoxOptions but I prefer to set browser.cache.disk.enable to false in about:config

Related

SpecRun.exe hangs for 60 seconds after test execution

posted this to google groups SpecFlow but there is little or no activity there so here we go.
I have a SpecFlow/Selenium/MSBuild project and I am running one simple scenario through
the command line, something like this:
SpecRun.exe run Default.srprofile "/filter:#%filter%"
The browser instance fires up, the assert is done, and the browser instance closes. This
takes about 5-10 seconds.
However: after this, I have to wait for 60 seconds until the SpecRun process closes and gives me the result like:
Discovered 1 tests
Thread#0:
0% completed
Thread#0: S
100% completed
Done.
Result: all tests passed
Total: 1
Succeeded: 1
Ignored: 0
Pending: 0
Skipped: 0
Failed: 0
Execution Time: 00:01:01.1724989
I am currently assuming this is because it is writing the test execution report to disk.. but I can not figure out how to turn this OFF... http://www.specflow.org/documentation/Reporting/
And, I can not figure out why this would take 60 seconds, or how to further debug this.
I have removed the AfterScenario and checked the selenium driver quit/close and verified that is not what is causing the problem.
Can anyone shed some light on this ?
Thank you
Jesus. There was something seriously wrong with the BaseStepDefinitions. Did some more debugging and found that the BeforeScenario was hit 25 times on one single test. 25 instances were launched and closed per 1 single scenario. Fixed by starting all over again with a clean file like:
[Binding]
public class BaseStepDefinitions
{
public static IWebDriver Driver;
private static void Setup()
{
Driver = new ChromeDriver();
}
[BeforeFeature]
public static void BeforeFeature()
{
Setup();
}
[AfterFeature]
public static void AfterFeature()
{
Driver.Dispose();
}
}
I will not post my original file because it is embarrassing.
This is a similar problem that helped me https://groups.google.com/forum/#!topic/specflow/LSt0PGv2DeY

Firefox WebDriver hangs waiting for page to load

sometimes in my test done with Selenium 2.41 and tested with Firefox 28 the execution hangs waiting for page to load.
This is the wait condition:
int time = 30;
WebDriverWait wait = new WebDriverWait(webDriver, time);
ExpectedCondition<Boolean> pageLoadCondition = new ExpectedCondition<Boolean>() {
public Boolean apply(WebDriver driver) {
return ((JavascriptExecutor)driver).executeScript("return document.readyState").equals("complete");
}
};
wait.until(pageLoadCondition);
It's supposed that after 30 seconds this method will throw a TimeoutException, but it's not, sometimes hangs forever.
This is the stacktrace produced in these situations:
java.lang.Thread.State: RUNNABLE at
java.net.SocketInputStream.socketRead0(Native Method) at
java.net.SocketInputStream.read(Unknown Source) at
org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
at
org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
at
org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
at
org.apache.http.impl.conn.LoggingSessionInputBuffer.readLine(LoggingSessionInputBuffer.java:116)
at
org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
at
org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
at
org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:260)
at
org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
at
org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
at
org.apache.http.impl.conn.AbstractClientConnAdapter.receiveResponseHeader(AbstractClientConnAdapter.java:223)
at
org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:271)
at
org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:123)
at
org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:682)
at
org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:486)
at
org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:863)
at
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:72)
at
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:57)
at
org.openqa.selenium.remote.HttpCommandExecutor.fallBackExecute(HttpCommandExecutor.java:322)
at
org.openqa.selenium.remote.HttpCommandExecutor.execute(HttpCommandExecutor.java:301)
at
org.openqa.selenium.firefox.internal.NewProfileExtensionConnection.execute(NewProfileExtensionConnection.java:165)
at
org.openqa.selenium.firefox.FirefoxDriver$LazyCommandExecutor.execute(FirefoxDriver.java:362)
at
org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:568)
at
org.openqa.selenium.remote.RemoteWebDriver.executeScript(RemoteWebDriver.java:504)
at
es.tao.commonservices.selenium.tests.TAORobotWebDriver$1.apply(TAORobotWebDriver.java:6227)
at
es.tao.commonservices.selenium.tests.TAORobotWebDriver$1.apply(TAORobotWebDriver.java:1)
at
org.openqa.selenium.support.ui.FluentWait.until(FluentWait.java:208)
at
es.tao.commonservices.selenium.tests.TAORobotWebDriver.waitToLoad(TAORobotWebDriver.java:6230)
at
es.tao.commonservices.selenium.tests.TAORobotWebDriver.handleWaitToLoad(TAORobotWebDriver.java:6110)
I have set this preference for firefox profile, but it's still not working:
ffProfile = new FirefoxProfile();
ffProfile.setPreference("webdriver.load.strategy", "unstable");
Also have this properties set:
webDriver.manage().timeouts().pageLoadTimeout(30, TimeUnit.SECONDS);
webDriver.manage().timeouts().setScriptTimeout(30, TimeUnit.SECONDS);
You may want to try Firefox 27.01. I upgraded to Firefox 28.0 and it seemed to break some tests I was doing using watir-webdriver. I went back to 27.01 and the tests ran again(if you go back download the whole install package as the setup only, does not seem to let you turn off auto-update so it updates itself to 28.0).
The fails were using hover and find_element.
Found an unresolved bug: https://code.google.com/p/selenium/issues/detail?id=6955 - if you can, please provide a test case, primarily a reduced host page with minimal scripts where the problem still occurs so it can be repeated reliably and traced down.
Sometimes I question myself if Google uses their own tools at all.... they should have run into that bug ages ago considering how huge that company is.

How to stop loading Selenium chromeDriver if it loads for more than 10 sec?

Is there any way to set a property for ChromeDriver to stop loading if time exceeds 10 seconds?
Something like the following pseudocode:
WebDriver dr = new ChromeDriver();
dr.loadLimit(10 SECONDS);
PS implicit wait is not going to work because it waits for Chrome response and only then starts counting 10 seconds.
In Python: driver.set_page_load_timeout(10)
Since version 2.1 of selenium chromedriver it supports a page load timeout. Before that version it was missing. (See the discussion about this here)
This is how you would set this in Java:
int pageLoadTimeout = 10;
driver.manage().timeouts().pageLoadTimeout(pageLoadTimeout, TimeUnit.SECONDS);
(see the webdriver spec and the selenium resources about this)
It work for me
from selenium import webdriver
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
import time
desired_capabilities = DesiredCapabilities().CHROME
desired_capabilities['pageLoadStrategy'] = 'none'
d = webdriver.Chrome(desired_capabilities=spider.desired_capabilities)
d.get('http://www.google.com')
time.sleep(10)

Why https is faster than http in my apache server?

Server information:
$ httpd -v
Server version: Apache/2.2.24 (Unix)
Server built: May 8 2013 15:17:37
I create a self-signed SSL Certificate with openssl.
Test Code(Java with selenium webdriver):
long startTime, useTime = 0, t;
int count = 10;
for (int i = 0; i < count; i++) {
ChromeDriver driver = new ChromeDriver(capabilities);
startTime = System.nanoTime();
driver.get("https://*.*.*.*/pic.html");
//When testing Http,it will be:driver.get("http://*.*.*.*/pic.html");
//pic.html is a simple page with many images.
t = System.nanoTime() - startTime;
useTime += t;
driver.quit();
}
System.out.println("Average Time: " + useTime/1000000.0/count +" ms");
Result:
HTTPs:Average Time: 1718.13659 ms
HTTP:Average Time: 2484.122677 ms
Thanks in advance.
It might be that using https also enables transparent compression of the content. The time added for compression and encryption (and back of course) might be less than the time saved by transferring less content over a slow link.
You can verify this by:
Using incompressible content (e.g. a large JPEG image)
Speeding up the transfer link significantly (e.g. by using "localhost")
Because Apache and chrome (I see you're using chromedriver) both support http2.0 which is faster for reasons other than encryption but only works with encryption.

Selenium test in Internet Explorer always times out?

I'm trying to run a basic test in Internet Explorer via Selenium-RC/PHPUnit, and it always returns with
# phpunit c:\googletest.php
PHPUnit 3.4.15 by Sebastian Bergmann.
E
Time: 35 seconds, Memory: 4.75Mb
There was 1 error:
1) Example::testMyTestCase
PHPUnit_Framework_Exception: Response from Selenium RC server for testComplete()
.
Timed out after 30000ms.
C:\googletest.php:17
FAILURES!
Tests: 1, Assertions: 0, Errors: 1.
Paul#PAUL-TS-LAPTOP C:\xampp
#
The last command in command history is waitForPageToLoad(30000). The same test runs fine and completes in firefox. How can I get this test to run and complete in internet explorer?
Thanks
There's an open bug in selenium that causes waitForPageToLoad to sometimes timeout on IE.
http://jira.openqa.org/browse/SRC-552
It's marked as occurring on IE6, but I'm experiencing the same error in at least IE9.
A workaround is to wait for e.g. a specific DOM-element on the page that is loading instead of using waitForPageToLoad. For example: waitForVisible('css=#header')
Try going into Internet Options and turn off Protected mode under the security tab. You may also want to decrease the security level for the Internet zone.
I've turned off protected mode and looks like it helped.
If it is acceptable to customize the client driver, here is the Python implementation for your refernece:
def open(self):
timeout = self.get_eval('this.defaultTimeout')
self.set_timeout(0)
self.do_command("open", [url,ignoreResponseCode])
self.set_timeout(timeout)
self.wait_for_page_to_load(timeout)
def wait_for_page_to_load(self,timeout):
# self.do_command("waitForPageToLoad", [timeout,])
import time
end = time.time() + int(float(timeout) / 1000)
while time.time() < end:
if self.get_eval('window.document.readyState') == 'complete': return
time.sleep(2)
raise Exception('Time out after %sms' % timeout)
I just use DOM attribute document.readyState to determine if the page is fully loaded.
IE 9+ intermittently throws a timeout error even the page is fully loaded, for more details.