I'm working on a monitoring tool for my website to log data. The actual logging is made on server. My goal is to calculate stats based on how long the user stays on the website.
Main question: I used chrome headless command --remote-debugging-port=80. I got logs for up to 10 minutes. Works perfectly. But how long will it work if left working? Is there a default timeout? If yes, how can I change it? If I want to run it exactly 30 minutes after page finished loading?
I'm trying to do the same on firefox (tried using PhantomJS but it wasn't loading the page correctly even though useragent was set to firefox) but firefox just throws an bank page when I'm trying to start a headless mode. I used "firefox -headless" and tried capturing an screenshot. It was just exiting my currently open firefox tabs without capturing any image. Any idea?
Using firefox quantum 59.0. I don't want to use selenium.
Also PhantomJS solution would be great. Currently I just want to collect logs. So, it only have to run all javascript (an jquery) code on the page which then sends the data using ajax. I tried page.onLoadFinished and then a wait function to make it stay on the page for the exact time after page loading.
Since no one answered, I will try to answer my own question after even more research and logical thinking.
Main question: Seems that there is no timeout but if need can be used --timeout X. Even though it's not perfect because it runs independently if the page if fully loaded or not.
As for the firefox, it's buggy. -new-instance (make headless run while you are already on firefox) is not working and -no-remote didn't help. Firefox is only working if running only one instance. So, if it's the PC you are working on and you want to run tests too, firefox is not for you. Headless runs only when no other instances of firefox are running, while chrome runs fine.
PhantomJS didn't work even though tried multiple solutions.
Best solution? Use chrome. Need portable? Use chromium and use headless. Or write your soft to use cefsharp which is based on chromium. Your browser with all libs will be around 120-200MB. Pretty big for portable but do it's work. Same as portable chrome or chromium. CefSharp have a privilege of integrating whatever you like into the browser since it's a... browser.
Related
I'm trying to scrape my own banking information by automating the process using Selenium in Ruby.
I'm running into a bizarre situation where performing the exact same sequence in the browser (whether just the normal browser or private/incognito) works fine, but when I try to log in under a Selenium-controlled browser I get back a strange 500 error from the server.
I've noticed the browser console logs also look different in terms of certain logging messages related to cookies, JS errors, libraries being loaded, etc.
I have found an answer on SO mentioning one possible difference in Chrome being a specific "cdc" string that might be detectable, but is there some kind of corresponding difference in Firefox/Geckodriver that could be used to detect the fact that I'm trying to automate the browser?
I'm not really sure where to look, because my understand was that running via Selenium should basically have identical behaviour to running via the browser itself.
Would love some guidance on what mechanisms may be in play to explain the difference in behaviour!
When I run my selenium tests using a chrome browser all my tests cases run fine. When using the phantomjs browser it would appear that the browser session does not get reset after each test case. In my tests cases, I log in as a user to then navigate to certain pages and then logout. A problem occurs when a test case happens to fail. The browser session is not reset so when the next test case begins, the test that failed was unable to logout. This causes all test cases after a single failure to fail.
When searching the internet for a solution to this issue it been known sine 2013. I can't seem to find anything regarding this issue that's recent. Is there any up to date workarounds?
Manually trying to delete the cookies before or after each test case does not appear to work. webDriver.manage().deleteAllCookies();
I'm using phantomjs ver 2.1.1.
First of all PhantomJS is dead, you are better off switching to Headless Chrome or Headless Firefox.
Secondly PhantomJS is a port of Webkit which is not thread safe. This means that if you try and run more than one test in parallel you will see threading issues, to fix this you would need to start multiple instances of PhantomJS and have each GhostDriver instance connect to a different instance of PhantomJS.
The particular problem that you are seeing is that PhantomJS doesn't clear itself down properly, again the only solution would be to kill the initial PhantomJS instance you are running after your test finished and then start up a clean new one, unfortunately that is not supported by GhostDriver.
The final problem is that GhostDriver is dead as well, there was no point in continuing development when PhantomJS died.
TLDR; Use Chrome/Firefox Headless mode instead.
I am using selenium WebDriver using RobotFramework. The major problem we are facing is, my tests are timing out even after setting timeout as high as 10 minutes. It happen with any browser I use. These thing works much faster If I run test manually (with all browser cache/data/cookie cleared). These are the other things I have observing for few months.
Some component are never loaded (I check the call trace using BrowserMob Proxy and we found nothing unusual)
"Click Element" does not work in many cases. Clicking on element triggers some action but that action is not always trigerred in automation. Manually it works all the time.
Notes:
This is happening on FF and Chrome. (IE is not working for me at all)
App server and automation suite is in LAN so latency is not an issue
No other heavy process is running at this time
Issue persist even if with firefox default profile
I tried it with different selenium versions. (2.45 - 2.52)
I took latest driver for chrome. Broweser: FF 40+ and GC 48
This does not look application issue as we spent 2 month confirming that. Let me know if you need any other detail.
I am doing a lot of testing with the Chrome Web Driver within Selenium. The problem is that every time I run it, it has to re-download all the site images which takes time. It does not keep these images cached.
So I would like to set which files to render and which not too. Obviously I want Javascript and CSS files to still download. But I particularily want to turn off images.
Is this possible? If not, is there a way to enable caching? So the next time I run the progran it can get the images from the local cache.
The solution is to load the same chrome profile again, it should (may not) ensure that images & other similar things are cached.
Here is how to load a particular profile :
https://code.google.com/p/selenium/wiki/ChromeDriver
DesiredCapabilities capabilities = DesiredCapabilities.chrome();
capabilities.setCapability("chrome.switches", Arrays.asList("--user-data-dir=/path/to/profile/directory"));
WebDriver driver = new ChromeDriver(capabilities);
A similar search on SO, gave this result - Load Chrome Profile using Selenium WebDriver using java you might bother to take a look.
Caching from one session to another is not possible as far as I know.
However, it is possible to remove rendering the page if you run it headless. This will load the page, but not render it (make it visible, load images).
You can use HTMLUnitDriver, which is the standard, but somewhat outdated, or you can use PhantomDriver, which has a more modern version of Javascript.
"You can use HTMLUnitDriver" >>> careful with this!
Please note that since HTMLUnitDriver is not the same implementation as the real browser, you may or may not get the same results in both.
So if you start hitting weird issues, like running the test with Chrome passes but running the test with HTMLUnitDriver does not, consider just running your tests with the browser driver. I've heard of looong troubleshooting stories from colleagues and they had to give up on running their suites in headless mode (ie, using HTMLUnitDriver).
NOTE: since I cannot leave a comment on the selected answer, I am "forced" to leave it as an answer. If somebody can help me convert it in a comment, I will appreciate! thanks
For caching images you would need something like squid or haproxy, and then proxy your selenium browser through it.
I'm executing selenium tests with testng, that are started on a remote system with Selenium RC via hudson (with ssh connection). The remote system is windows xp with MKS Toolkit installed, hence ssh. Tests are NOT executed as a windows service.
I've tried using both captureScreenshot and captureEntirePageScreenshot methods. The first one always produces a black image. The second one creates the correct screen shot but it only works on Firefox and our tests usually pass on Firefox and fail in other browsers, so it is crucial to capture screen shots for the other browsers (mainly IE and Safari). The tests are ran in parallel, with many browser windows open at the same time. I'm not certain if this is what's causing the problem. Any thoughts will be appreciated.
Unfortunately screenshots in Selenium have been problematic from the start in browsers that are not Firefox. This is something that we Selenium Developers have been working on for a while to correct.
The latest work has been updating Snapsie to work in IE. There is a blog post at http://blog.codecentric.de/en/2010/02/remote-screenshots-mit-selenium-und-dem-robot-framework/ that explains what has happened.
I have noticed that if the screen isn't active, i.e. the screensaver has kicked in, it can produce black screenshots.
Edit:
I just had a thought. You can always run Castro to video record your tests and then watch it play back. This is something SauceLabs use to run Selenium in the cloud.
Write a method for this and call that whereever you need to take the screenshot. Use the java.awt package which has been used in selenium. For example, check this site
After setting Windows Auto-Logon, and launching process not as Windows Service, I found how to solve the Remote Desktop with Black Screenshots problem of IEDriverServer.exe, by creating a batch file that disconnects RDP, instead of closing the RDP session with the regular X button:
%windir%\system32\tscon.exe %SESSIONNAME% /dest:console
See more details here:
https://stackoverflow.com/a/24529629/658497
(Although, I would prefer there was a way to run it as the default action, when terminating RDP session with X Windows button).