BrowserMob Proxy not injecting headers with new Chrome version - selenium

I recently upgraded my Chrome/Chromedriver from v69 to the latest v76, and my suite of Selenium tests - which use BrowserMob Proxy to inject HTTP headers - have stopped working. The ChromeDriver will open the browser OK, but the headers are no longer being injected.
Chrome, Selenium and BrowserMob Proxy are now all on the latest versions:
Chrome: 76.0.3809.132
BrowserMob: 2.1.5
Selenium: 3.141.59
Does anyone know whether there are compatibility issues with the latest Chrome and BrowserMob? Or, can anyone spot an issue in my code?
Any help is much appreciated!
Here is my code:
Proxy seleniumProxy = null;
try {
String hostIp = Inet4Address.getLocalHost().getHostAddress();
proxy.setTrustAllServers(true);
proxy.start(0, Inet4Address.getByName(hostIp));
proxy.addRequestFilter((request, contents, messageInfo) -> {
headerMap.forEach((key, value) -> {
if (key != null && value != null) {
request.headers().add((String) key, value);
}
});
return null; //continue regular processing...
});
seleniumProxy = ClientUtil.createSeleniumProxy(proxy, InetAddress.getByName(hostIp));
seleniumProxy.setHttpProxy(hostIp + ":" + proxy.getPort());
seleniumProxy.setSslProxy(hostIp + ":" + proxy.getPort());
System.out.println("Created proxy on port " + proxy.getPort());
} catch (UnknownHostException e) {
LOG.error("unable to create BrowerMob proxy", e);
}
return seleniumProxy;

Related

Selenium getting stuck on data:, (MS Edge)

I want to use Selenium to navigate to a website. Selenium opens up the browser but does not navigate further to the specified website URL, but gets stuck on the "data:," url. After some time time I get the following exception:
"The HTTP request to the remote WebDriver server for URL http://localhost:58504/session timed out after 60 seconds"
Note: I did not specify the 58504 port anymore, so I guess it is the default port that Selenium use?
I am programming in C# and using the following Nuget Packages:
https://www.nuget.org/packages/Selenium.WebDriver/4.1.1
https://www.nuget.org/packages/WebDriverManager/2.13.0
Here is the code:
using OpenQA.Selenium;
using OpenQA.Selenium.Edge;
using WebDriverManager;
using WebDriverManager.DriverConfigs.Impl;
using WebDriverManager.Helpers;
public void VisitWebsite()
{
IWebDriver driver = null;
try
{
new DriverManager().SetUpDriver(new EdgeConfig(), VersionResolveStrategy.MatchingBrowser);
EdgeOptions options = new EdgeOptions();
options.AddArgument("--no-sandbox);
options.AddArgument("--disable-infobars");
options.AddArgument("--disable-dev-shm-usage");
options.AddArgument("--disable-browser-side-navigation");
options.AddArgument("--disable-extensions");
options.AddArgument("--dns-prefetch-disable");
options.AddArgument("--disable-gpu");
options.AddArgument("--disable-software-rastersizer");
driver = new EdgeDriver(options);
}
catch (Exception ex)
{
throw ex;
}
driver.Navigate().GoToUrl("https://www.google.com");
}
Where does it go wrong? Thanks!
I found the answer. It seems the DeveloperToolsAvailability policy for the MSEdge browser had a value that "blocks" Selenium from working. You can read more about it here: https://learn.microsoft.com/en-us/microsoft-edge/webdriver-chromium/?tabs=c-sharp#developer-tools-availability-policy

Laravel dusk + docker-compose + selenium + firefox: Undefined index ELEMENT

I have docker-compose with 3 images - webserver, db and selenium.
I up it then exec into webserver and run php artisan dusk.
My app is a simple https page that returns Welcome in a few div.
But I got ErrorException: Undefined index: ELEMENT.
Selenium image: selenium/standalone-firefox.
test:
...
$this->browse(function (Browser $browser) use ($user) {
$browser
->visit('/home')
->assertSee('Welcome');
});
DuskTestCase.php
...
protected function driver() {
return RemoteWebDriver::create(
'http://selenium:4444/wd/hub', DesiredCapabilities::firefox()->setCapability(
'acceptInsecureCerts', true
)
);
}
This is due to an incompatibility between the geckodriver and Selenium:
https://github.com/facebook/php-webdriver/issues/492
For Firefox and Selenium you need to add enablePassThrough=false to the Selenium capabilities.
Something like:
protected function driver() {
return RemoteWebDriver::create(
'http://selenium:4444/wd/hub', DesiredCapabilities::firefox()
->setCapability('acceptInsecureCerts', true)
->setCapability('enablePassThrough', false)
);
}
should probably do the trick.

ERR_EMPTY_RESPONSE in Chrome when use BrowserMobProxy for selenium test

I've created simple method for get network traffic from Chrome:
public void saveNetworkTraffic() {
System.setProperty("webdriver.chrome.driver", System.getProperty("user.dir") + "/bin/chromedriver");
String sFileName = "networklog.xar";
BrowserMobProxy proxy = new BrowserMobProxyServer();
proxy.start(0);
Proxy seleniumProxy = ClientUtil.createSeleniumProxy(proxy);
DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.setCapability(CapabilityType.PROXY, seleniumProxy);
WebDriver driver = new ChromeDriver(capabilities);
WebDriverRunner.setWebDriver(driver);
proxy.enableHarCaptureTypes(CaptureType.REQUEST_CONTENT, CaptureType.RESPONSE_CONTENT);
proxy.newHar("google.com");
driver.get("http://google.com/");
Har har = proxy.getHar();
File harFile = new File(sFileName);
try {
har.writeTo(harFile);
} catch (IOException ex) {
System.out.println(ex.toString());
System.out.println("Could not find file " + sFileName);
}
}
When browser opens a page it shows an error "ERR_EMPTY_RESPONSE"
on this step driver.get("http://google.com/") instead of usual google page in Chrome.
I've tried to figure out the reason for the error, but according to the https://github.com/lightbody/browsermob-proxy#using-with-selenium, my code should work fine.
Add this code, try it:
proxy.addFirstHttpFilterFactory(new ResponseFilterAdapter.FilterSource(new ResponseFilter() {
#Override
public void filterResponse(HttpResponse response, HttpMessageContents contents, HttpMessageInfo messageInfo) {}}, Integer.MAX_VALUE));
Your code look fine, try adjusting the dependencies like:
<dependency>
<groupId>net.lightbody.bmp</groupId>
<artifactId>browsermob-core</artifactId>
<version>2.1.5</version>
</dependency>

WebdriverIO & Browsermob

I'm currently trying to use Browsermob with WebdriverIO and I found this code on another answer, but when I run it, the firefox browser comes up and I see activity in the console windows I have selenium and browsermob-proxy running, but it does not go to the search.yahoo.com page. It just sits at a blank page and the tests ends (which says it passed, but that's something else)
I'm running the latest WebdriverIO and Browsermob on a Mac
Here's the code
var Proxy = require('browsermob-proxy').Proxy
, webdriverio = require('webdriverio')
, fs = require('fs')
, proxy = new Proxy()
;
proxy.cbHAR('search.yahoo.com', doWebio, function(err, data) {
if (err) {
console.error('ERR: ' + err);
} else {
fs.writeFileSync('stuff.har', data, 'utf8');
}
});
function doWebio(proxy, cb) {
var browser = webdriverio.remote({
host: 'localhost'
, port: 4444
, desiredCapabilities: { browserName: 'firefox', seleniumProtocol: 'WebDriver', proxy: { httpProxy: proxy } }
});
browser
.init()
.url("http://search.yahoo.com")
.setValue("#yschsp", "javascript")
.submitForm("#sf")
.end().then(cb);
}
have you tried using chrome. Maybe it'll work. To do so:
Add chromedriver from here to your /usr/bin
make change to above code like below (note upper case P in proxy)
start selenium server and browserMob as usual and run the test
desiredCapabilities: { browserName: 'chrome', seleniumProtocol: 'WebDriver', Proxy: { httpProxy: proxy } }
For those who come to this, with FireFox, you now need GeckoDriver installed to use FireFox with Selenium. https://github.com/mozilla/geckodriver/releases
Also, the BrowserMob proxy hasn't had a release since 2016. The BrowserUp Proxy is an actively maintained drop-in replacement https://github.com/browserup/browserup-proxy with support up to Java 11, active development, brotli support, security fixes, and more.

Appium TouchAction/ MultiTouchAction don't work when testing Mobile Web

I'm trying to set up an automated test for a mobile website using Appium 1.2.4, Java-client 2.0.0, Selenium 2.42.2 and a real device (Samsung Galaxy Tab-3).
The problem is that, whenever I'm trying to use Touch Actions (e.g. driver.swipe() or driver.pinch()) I'm getting a Not Yet Implemented … error.
I've tried to improvise by switching the driver context to NATIVE_APP before executing the TouchAction and yet it still didn't work (I get a different error: An unknown server-side error occurred while processing the command).
Would appreciate your kind advice.
Configuration:
Selenium 2.42.2 Java-client 2.0.0 Appium 1.2.4
Device:Galaxy Tab3
OS: Android 4.4.2
Browser: Chrome 38
Thanks and Best Regards,
Daniel.
DesiredCapabilities capability = new DesiredCapabilities();
capability.setCapability("device","ANDROID");
capability.setCapability("app", getValidBrowserName(browser));
capability.setCapability("newCommandTimeout",150);
capability.setCapability("platformName", "Android");
capability.setCapability("deviceName", "GalaxyTab3");
capability.setCapability("platformVersion", "4.4.2");
RemoteWebDriver driver = null;
Platform currentPlatform = desiredCapabilities.getPlatform();
boolean isMobilePlatform = currentPlatform.equals(Platform.ANDROID) || currentPlatform.equals(Platform.MAC);
URL hubURL = new URL(hubHost + "/wd/hub");
try {
driver = isMobilePlatform ? new AndroidDriver(hubURL, desiredCapabilities) : new RemoteWebDriver(hubURL, desiredCapabilities);
driver.manage().timeouts().implicitlyWait(waitTimeout, TimeUnit.SECONDS);
driver.manage().timeouts().setScriptTimeout(scriptTimeout, TimeUnit.SECONDS);
driver.setFileDetector(new LocalFileDetector());
}
catch (Exception e) {
throw new Exception("Failed to create Selenium WebDriver instance on hub: "
+ hubURL.toString() + " with capabilities: "
+ desiredCapabilities.toString() + "\n"
+ e.getMessage());
}
WebElement buttonTest = driver.findElement(By.id("configBtn"));
int buttonX = buttonTest.getLocation().getX();
int buttonY = buttonTest.getLocation().getY();
((AppiumDriver)driver).swipe(buttonX, buttonY, buttonX + 70, buttonY + 70, 1000);