How to add „From” request header in Firefox-Selenium? - selenium

I’m looking for a way to add the “From” request header to disclose an email address of the requesting user.
I am using Selenium with Firefox, alternatively I can switch to PhantomJS or Chrome.
It could be some preference to set in selenium.webdriver.FirefoxProfile. I checked Firefox’s about:config documentation, but can’t find any indication how to implement this header. Any help is appreciated especially that it is difficult to google this issue given the name of the header.

Still haven’t found an interface how to do this with Selenium. There is however a workaround with intercepting the request using selenium-wire:
from seleniumwire import webdriver
def from_request_header_interceptor(request):
del request.headers['From']
request.headers['From'] = 'email#example.com'
driver = webdriver.Firefox(**kwargs)
driver.request_interceptor = from_request_header_interceptor
driver.get('https://www.httpbin.org/headers')
The httpbin outputs i.a. headers “From”: “email#example.com”. Seems to work.

Related

Loading local TileFile layer in pydeck

Am trying to load a local TileFile layer in pydeck(0.5) in jupyter notebook
I am using the following code:
import pydeck as pdk
data ='https://localhost:/home/user/myfolder/tiles/{z}/{x}/{y}.png`
layer = pdk.Layer(
'TileLayer', # `type` positional argument is here
data=data
)
# Set the viewport location
view_state = pdk.ViewState(
longitude=50,
latitude=50,
zoom=0,
min_zoom=0,
max_zoom=5,
pitch=40.5,
bearing=-27.36)
# Combined all of it and render a viewport
r = pdk.Deck(layers=[layer], initial_view_state=view_state)
r.to_html('TileLayer-example.html'
This just creates a blank view window in Jupyter where it should show an image.
If I take launch the TileLayer-example.html independently into a browser I also get a blank window, However browser (Firefox) console output is as follows:
Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at https://localhost:/home/user/myfolder/tiles/0/0/0.png. (Reason: CORS request did not succeed).
TypeError: NetworkError when attempting to fetch resource. tile-layer.js:18:56
value tile-layer.js:18
value tile-layer.js:136
t tile-2d-header.js:90
u runtime.js:45
_invoke runtime.js:271
e runtime.js:97
Babel 2
r
u
I am not sure if the problem is CORS related (calling a local file) or if my data path is just plain wrong? I have tried may variants on path but all seem to have the same console warning issues.
Any suggestions for simple safe ways around the CoRs problem, or suggestions for accessing local TileFiles in PyDeck would be much appreciated
For other folk currently looking to implement TileLayer in Pydeck. It appears TileLayer is not yet documented or currently supported in pydeck and can only be achieved with a custom layer see this github issue and this suggested workaround with a custom layer
With respect to the CoRs issue of trying to display a local file resource in the browser when called from a URL. if you are looking for a quick fix to do a test/debug then in Firefox: the suggestions here help. For Firefox specifically they suggest the following:
go to about:config
search for privacy.file_unique_origin
set it to false.
Note this is a security risk so reset it to true after debugging.
The only workable solution currently seems to be to put the tilefile in the working directory from which your deck project is launched

Automate a button click on chrome://extensions page using selenium webdriver

I'm trying to write an automated test that will automate the process of updating a google chrome extension. I'm not aware of another method of doing this automatically so here is what I'm currently trying to do:
Open the chrome extensions page (as far as I'm aware this is just an html page unless I'm missing something).
Click on the "Update extensions" button
Here is what I have tried having opened the chrome extensions page:
IwebElement UpdateButton = driver.findelement(By.Id("update-extensions-now"));
UpdateButton.Click();
For some reason the button click is not registering. I have tried some other locators such as CSS path and Xpath but they don't work either. Also, when I debug this test, it passes fine so I know it's not an issue with any of my locators. I have (as a test) tried to automate clicks on the other elements on this page and it's the same issue. I can't get a handle on any elements on the chrome://extensions page at all.
Has anyone encountered this or have any ideas as to what's going on?
You can use the Chrome extensions API to auto-update required extension.
Find the file "manifest.json" in the default Google Chrome
C:\Users\*UserName*\AppData\Local\Google\Chrome\User Data\Default\Extensions
There find the update URL of your extension:
{
"name": "My extension",
...
"update_url": "http://myhost.com/mytestextension/updates.xml",
...
}
The returned XML by the Google server looks like:
<?xml version='1.0' encoding='UTF-8'?>
<gupdate xmlns='http://www.google.com/update2/response' protocol='2.0'>
<app appid='yourAppID'>
<updatecheck codebase='http://myhost.com/mytestextension/mte_v2.crx' version='2.0' />
</app>
</gupdate>
appid
The extension or app ID, generated based on a hash of the public key, as described in Packaging. You can find the ID of an extension or Chrome App by going to the Extensions page (chrome://extensions).
codebase
A URL to the .crx file.
version
Used by the client to determine whether it should download the .crx file specified by codebase. It should match the value of "version" in the .crx file's manifest.json file.
The update manifest XML file may contain information about multiple extensions by including multiple elements.
Another option is to use the --extensions-update-frequency command-line flag to set a more frequent interval in seconds. For example, to make checks run every 45 seconds, run Google Chrome like this:
chrome.exe --extensions-update-frequency=45
Note that this affects checks for all installed extensions and apps, so consider the bandwidth and server load implications of this. You may want to temporarily uninstall all but the one you are testing with, and should not run with this option turned on during normal browser usage.
The request to update each individual extension would be:
http://test.com/extension_updates.php?x=id%3DyourAppID%26v%3D1.1
You can find even more detailed information on exntesions developers site: https://developer.chrome.com/extensions
If you look at the HTML of the "chrome://extensions" page you will notice that the "Update extensions now" button is contained within an iframe. You need to switch to the iframe before trying to register a button click. i.e:
(This is in c#. Note that this code is written from memory so it may not be 100% accurate. Also, you will want to write more robust method. This code just quickly demonstrates that by switching to the iframe, it will work ok)
String ChromeExtensionsPage = "chrome://extensions";
driver.Navigate().GoToUrl(ChromeExtensionsPage);
driver.Switchto().Frame("DesiredFrame");
IwebElement UpdateButton = driver.findelement(By.Id("DesiredButtonID"));
UpdateButton.Click();

Mink and Sahi timing out on getContent

I've written a website scraper for use in a project.
I'm controlling Firefox through Sahi using Mink to visit each site and interact with any elements where necessary. I've managed to get this working perfectly on all sites I've tried except for one...
I'm trying to get the markup from https://www.o2.co.uk/shop/phones/
I'm using the exact same code for this page, as I have for all others:
// Configure driver
$this->driver = new \Behat\Mink\Driver\SahiDriver('firefox',
new \Behat\SahiClient\Client(
new \Behat\SahiClient\Connection(null, CRAWL_SERVER, 9999)
)
);
// Init session:
$this->session = new \Behat\Mink\Session($this->driver);
// Start session:
$this->session->start();
// Open the url
$this->session->visit($config['url']);
// Get the markup from the page
$markup = $this->session->getPage()->getContent();
When I use this code to attempt to get the markup from https://www.o2.co.uk/shop/phones/ Mink just seems to hang, waiting for something to happen.
It would seem that maybe something on this page is preventing either Sahi or Mink from returning the markup. I've also tried running other functions instead of getContent(), such as $this->session->wait(2000); and attempting to search through getPage using the find command.
If anyone has any idea as to why this is happening I would be very interested in finding out why and how I can make this work.
tl;dr
Why is Mink/Sahi timing out on this site?

phpunit selenium getting header information of the url open

I am using phpunit and selenium to write some functional testing for my project.
What I am looking for is the way to retrieve Response-header
There are some URLs which dont have a .js extension but return javascript as the code.
What I am looking for is the header information (especially content-type) of the URL opened with $this->open(); function
For eg:
//if I do
$this->open('http://stackoverflow.com');
// or
$this->open('http://ajax.googleapis.com/ajax/libs/jquery/1.7.1/jquery.min.js')
// and then use some function like
$this->getResponseHeaders();
// returns me array of header information or
// some function just for content type
$this->getContentType();
// which return text/html or text/javascript
Is there any way selenium provides this functionality or a hack or a trick to get the details. I went through the selenium documentation and the TestCase files and didnt find anything close to it.
It's sad, but you cannot. It does not allowed by phpunit because it does not allowed by WebDriver.
We will not be adding this feature to the WebDriver API as it falls outside of our current
scope (emulating user actions).
If I understand your question you need these two steps:
$this->open('http://stackoverflow.com');
$htmltext = $this->getHtmlSource();
and then you can get the inforamtion by splitting the htmltext...

Secured and unsecured items message in IE

I'm getting "This page contains bothe Secure and Non secure items"message in IE. When I commented the following piece of code from dojo.js.uncompressed.js file, the message is gone.
if(dojo.isIE){
if(!dojo.config.afterOnLoad){
document.write('<scr'+'ipt defer src="//:" '
+ 'onreadystatechange="if(this.readyState==\'complete\'){' + dojo._scopeName + '._loadInit();}">'
+ '</scr'+'ipt>'
);
}
Is that an issue with the dojo? I would like to move the commented code to another custom file so that the dojo framework is not affected. Can you suggest a better way of implementing it.
Thanks.
You would get that error if you're using frames or have external files where some of the files have https URLs while some have http URLs. Assuming, your main page loads up through https, you could try changing:
src="//:"
to:
src="https//:"
the //: is most likely the problem, as I ran into a similar issue with a chunk of javascript code... In internet explorer, the locaiton //: is not secure, so when your page (presumably on an https:// url) loads, IE notes that you've got your main code loading from a secure location, and another script being loaded in from an unsecure location.
The workaround that I came to was to create an empty file in my web root named "blank.html" (though "blank.js" would probably work better in your case) and replace the //: link with "/blank.html". This results in another hit to your webserver, but browser caching will probably make that impact minimal.