I'm trying to write a test for a simple API, which always fails because of a strange browser behaviour.
The response coming from the API is just some plain text:
foo-bar-123
I can see exactly that in the browser window and also as response in the network tab.
Okay so far, but when I look at the Inspector, I see something like that:
<html><head></head><body>foo-bar-123</body></html>
If I control the browser with selenium, the result of webdriver.page_source is the same.
For reasons I don't understand, the browser adds some HTML tags to the content.
Is this some strange kind of "feature"? Can this be switched off?
I don't think it's a bug because both Firefox and Chrome are showing this behaviour.
I just want to get the real content without any fancy stuff the browser thinks I need.
Related
I'm trying to figure out how to hide the border (including the address bar, tabs, title bar... everything that isn't the browser viewport) of my Firefox instance instantiated by Selenium.
If there's some way to have it use a userChrome.css, that would be straightforward enough. I've tried loading a profile folder that included a userChrome.css using this answer as a guide, but it seemed to ignore the styles. I've also looked through Firefox's about:config to see if there's some preference that would hide the frame of the window, but I haven't found anything yet.
Any solution that allows me to hide all or some of these elements when creating the instance with Selenium would be helpful. I know it's silly, but that's how it goes sometimes, you know?
-edit-
I don't think the title bar needs to be hidden. But everything else should be hidden.
-another edit to clarify a few things-
I mentioned kiosk mode in the comments as an example of the sort of thing I'm going for. Kiosk mode isn't exactly what I'm looking for, though. The windows aren't meant to be fullscreen, but they should still lack the elements of a common browser window. Think of it as like an Electron app. Out of the box, Electron lacks an address bar, tabs, etc. That's basically what we have for our app, but it's with regular-old Firefox. Again, whether these elements are displayed or not doesn't typically impact the test, but we want them hidden anyway.
Finally, I a friend of mine tried achieving this goal using a userChrome.css wrapped in a Firefox profile and was able to get Selenium to use the userChrome. So perhaps I need to figure out what I'm doing wrong. The biggest difference between how he did it and how I'm doing it is I must use a remote web driver for testing. But even still, it should be able to load the userChrome.css file. I'll try to update this question with more details as I fiddle with it some more.
-edit-
I think the reason userChrome isn't working when specifying a profile is because of the version(s) of Selenium/Geckodriver/Firefox being used.
The geckodriver version I started with was 0.15. 0.17 behaved exactly the same. 0.18 didn't respect the profile I passed along to it at all and instead had Firefox open the profile selection window (not very useful, but I was able to at least select the correct profile and see the userChrome.css get applied). 0.24 is no different.
Firefox is 52.9.0. Not much I can do about that.
We're using selenium (standalone) server 3.8.1. Switching out for 3.141.59 Didn't change anything.
Unless there's a version combination that will work with Firefox 52, I think the only thing I can do is wait until there's an update.
At last I have figured it out. In order to get Selenium to use my custom profile, I needed to do the following:
FirefoxProfile profile = new FirefoxProfile(new File(path_to_profile));
FirefoxOptions options = new FirefoxOptions().setProfile(profile);
RemoteWebDriver driver = new RemoteWebDriver(options.toCapabilities());
driver.get(url_of_webpage);
Thanks to avinesh09 on Github for the info I needed to solve the problem. It's so simple, but this has to be the only way that I neglected to try to load the profile.
If fullscreen (kiosk) mode is what you ask for (as then all you see is the viewport) it is as simple as:
driver.manage().window().fullscreen();
It is the same user experience as pressing "F11" in your browser.
Import.io is working pretty fine, but there is one website I would like to extract datas, but when I start the extractor, then enter the URL http://restaurant.michelin.fr/restaurants/france/75000-paris/restaurants-michelin/page-4/ which is loaded. Then I press the ON button, but the page won't load, nothing is displayed.... blank page and looks like it's still loading... In that case, how can I do ? I've also tried with the crawler, but same result. I restarted the program and computer but always the same issue. Thanks a lot.
The import.io desktop app browser uses firefox24. Few websites aren't compatible with the browser and this appears to be what is happening in this case.
It does however work in Magic! https://magic.import.io/
Once you have published the Magic API, you can then use the tools in MyData such as Bulk and Chain to add more URLs.
I have just tried to save a Magic API and it worked a treat. The only disadvantage here is that you won't be able to edit the columns until after you have extracted the data.
I have been testing an app using Firefox Web Driver but it is really slow comparing with HtmlUnitdriver, so I decided to translate to last one. All works fine but I have problems with some kind of links that loads information async.
In my case it is not problem of "wait" because I do and as i said before, using Firefox driver works very well. But with HtmlUnit doesnt work well.
In my scene I have following code:
Send
Viewing the app on a browser such as Firefox, when I click on that link, the system return me a result. But when I use HtmlUnitdriver, when I do click to the element, the driver goes directly to the href and then I have a bad result.
I don't really know how to do make works with it. I'm really interested on this way because is really fast.
Safari recently went to version 6 (Lion/Mtn Lion) and they've changed over from the standard webkit dev tools to one that's much more XCode looking, my problem other than the OCD of not liking things change is that in the resource tab (or anywhere you can track down the DataService.aspx/AJAX calls) I can no longer see the form data that I am passing.
Can anyone point me to where I can find that data so I don't have to console out my params when I'm testing new data service/backend calls?
I've logged a bug with Apple, they've marked it as a duplicate so hopefully they've received enough requests to fix this, until then I'm continuing to use chrome as the webkit developer is the same as safari's old version.
You can find this info in Instrument tab (stop-watch icon). In the left sidebar thers Timelines row, click the grey circle (record button) on the right. Then click to Network Requests where you see all reqs, and you have to click small icon on the right oc request to display response headers and all form data are available in right panel. Panel can be hidden same as left one (in case you dnt see it).
Unfortunately there are no query pamaters listed, according to this disscussion. I belive its a bug in safari
Edit 15.May 2013: This bug was fixed in Safari 6.0.3.
As far as I can tell, there's no way to show the request parameters.
This goes even further. I can't see the JSON response data either (no clickable arrows to show the containing Javascript objects within the JSON, just pure text)
I think we have to switch to Firefox /w Firebug or regular Webkit in order to get XHR monitoring...
Guys if you want to see post data in safari 6 which is not possible right now, install the firebug lite extension and there you go you have the post data.
I used it and it works great with safari 6
Actually the request headers, response headers and query parameters are in the details sidebar on the right when using the resources view or if you click to see the content of a request in the Timelines/Network Requests view. Took me a few minutes to find that too.
If you need to see what the device is actually sending and your server is on a Windows Machine I use http://www.Wireshark.org and check on the server side of things. No interpretation by any WebKit stuff and very valuable (such as issue with iOS and the 'Blob' data). Similar network snooping should exist on Mac as well.
I'm trying to get screenshots from a web page with multiple browsers. Just experimenting with Selenium RC, I wrote code like this:
var sel = new DefaultSelenium(server, 4444, target, url);
sel.Start();
sel.Open(url);
sel.WaitForPageToLoad("30000");
var imageString = sel.CaptureScreenshotToString();
This basically works, but in most cases the screenshot is of a blank browser window, because the page is not yet ready for display. It kind of works if I add a sleep just after the WaitForPageToLoad, but that slows down the fast browsers and/or may be to short for the slower browsers (or under load).
A typical solution for this seems to be to wait for the presence of a certain element. However, this is meant as a simple generic solution to get a screenshot of a local web page with as many browsers as possible (to test the layout) and I don't want to have to enter certain element names or whatever. It's a simple tool where you just enter the Selenium Server URL and the URL you want to test, and get the screenshots back.
Any advice?
I use Selenium-RC to capture screenshots of remote pages where the waiting time is variant. In such cases, checking the title of the page and using time.sleep(n seconds) usually does it for me.
May be you can make use of Browser status bar to verify whether that page is loaded fully or not. I think this is the best solution.