selenium chrome NET BLOCKED BY CLIENT - selenium

selenium chrome If a page is not operated for a long time, for example, time.sleep 15 minutes, and then again, it can not open the web page, how to refresh the browser debugging found that it is blocked by client did not install any advertising plug-ins after this situation, if you enter the target URL in a new tab enter, it can be opened normally, who knows the reason? I don't think it has anything to do with cookies, thank you.
chromedriver version is also correct, I do not know if everyone's selenium chrome will not have this problem if the open page time.sleep longer time
***The reason for this is partly to do with the html http-equiv attribute of the target site. If the page does not change for a long time, any further execution of the url of the target site in the same tab will not be considered as "re-fetching from the server" but from the cache, which will cause a js exception. Any subsequent get requests of the xhr type will be directed to chrome-extension://invalid
The value after chrome-extension:// will be BLOCKED by chrome if it is unrecognisable, which will happen in any chrome speculation.
A not-so-good solution is to pywin32 simulate re-opening a new tab by typing in the url, but this causes some tab over-closing and handles switching exceptions, so I'm considering seeing if I can fix this from some chrome option parameter settings***
Also if I don't use selenium chrome but normal manual normal browsing of the target site, I don't manipulate the page for a long time and I don't have the above problem, not quite sure if this is an ISSUE with chrome, don't want to switch to firefox as there is too much code to change.

Related

Causes of Selenium Chrome to Many Redirects Error When Chrome Works Manually

I am writing a selenium test which generates to many redirects error; yet, I can manually perform the test. The test fills in a userid and password and presses a login button. I had implemented this test on Chrome 96; however, it doesn't seem to work on chrome 99. I am using selenium 4.1.1. Additionally, the site switched to http/2 during this period. I appreciate any ideas on what the issue might be.

Selenium doesn't load redirect page

I have a tool which load a website, fill in the information, and save the result.
It was working fine until last week.
After debugging, I found out the reason is because the site URL "became" redirected.
The redirected sequence is like below. (for example, site url is google.com)
google.com` → google.com\somethingbetween → google.com (yes, the final redirected url is exactly the same as the original url)
If I open chrome and navigate to the url manually, the page is loaded fine. But if opened by selenium chromedriver, it will stop as google.com\somethingbetween. The strange thing is if I manually enter the url inside the address bar of the browser which was opened by ChromeDirver (will have a little popup saying that the browser is currently controlled by automation tool), the browser will also stop at the second page ( google.com\somethingbetween).
I tried to use the Chrome beta version 79 (with chrome driver v79) but the problem didn't go away.
Use Chrome driver of stable version which may help you to get rid of this problem.Because under this version there is an issue of "issue 3133: window.navigator.webdriver is undefined when "enable-automation" is excluded in non-headless mode (should be true) [Pri-2].So do try with the stable versions like v78,v77.
The redirection issue occurs when "driver.get()" method is used. Please try using "driver.navigate().to()" method to navigate to the url. This may solve the issue.
Linking a related question:
How do I switch to redirected url using selenium web driver

Chrome Headless - Firefox

I'm working on a monitoring tool for my website to log data. The actual logging is made on server. My goal is to calculate stats based on how long the user stays on the website.
Main question: I used chrome headless command --remote-debugging-port=80. I got logs for up to 10 minutes. Works perfectly. But how long will it work if left working? Is there a default timeout? If yes, how can I change it? If I want to run it exactly 30 minutes after page finished loading?
I'm trying to do the same on firefox (tried using PhantomJS but it wasn't loading the page correctly even though useragent was set to firefox) but firefox just throws an bank page when I'm trying to start a headless mode. I used "firefox -headless" and tried capturing an screenshot. It was just exiting my currently open firefox tabs without capturing any image. Any idea?
Using firefox quantum 59.0. I don't want to use selenium.
Also PhantomJS solution would be great. Currently I just want to collect logs. So, it only have to run all javascript (an jquery) code on the page which then sends the data using ajax. I tried page.onLoadFinished and then a wait function to make it stay on the page for the exact time after page loading.
Since no one answered, I will try to answer my own question after even more research and logical thinking.
Main question: Seems that there is no timeout but if need can be used --timeout X. Even though it's not perfect because it runs independently if the page if fully loaded or not.
As for the firefox, it's buggy. -new-instance (make headless run while you are already on firefox) is not working and -no-remote didn't help. Firefox is only working if running only one instance. So, if it's the PC you are working on and you want to run tests too, firefox is not for you. Headless runs only when no other instances of firefox are running, while chrome runs fine.
PhantomJS didn't work even though tried multiple solutions.
Best solution? Use chrome. Need portable? Use chromium and use headless. Or write your soft to use cefsharp which is based on chromium. Your browser with all libs will be around 120-200MB. Pretty big for portable but do it's work. Same as portable chrome or chromium. CefSharp have a privilege of integrating whatever you like into the browser since it's a... browser.

Slow/incomplete page load in the browser launched by selenium webdriver

I am using selenium WebDriver using RobotFramework. The major problem we are facing is, my tests are timing out even after setting timeout as high as 10 minutes. It happen with any browser I use. These thing works much faster If I run test manually (with all browser cache/data/cookie cleared). These are the other things I have observing for few months.
Some component are never loaded (I check the call trace using BrowserMob Proxy and we found nothing unusual)
"Click Element" does not work in many cases. Clicking on element triggers some action but that action is not always trigerred in automation. Manually it works all the time.
Notes:
This is happening on FF and Chrome. (IE is not working for me at all)
App server and automation suite is in LAN so latency is not an issue
No other heavy process is running at this time
Issue persist even if with firefox default profile
I tried it with different selenium versions. (2.45 - 2.52)
I took latest driver for chrome. Broweser: FF 40+ and GC 48
This does not look application issue as we spent 2 month confirming that. Let me know if you need any other detail.

IE 10 not using AppCache after browser closed

I wrote an application using the HTML5 Cache Manifest and I'm having a problem using it in IE 10.
I used Fiddler to witness the manifest file being downloaded and all resources fetched on the initial load of the application. If I disable my network adapter to force the machine offline, the application continues to work as expected as long as I don't close the browser window.
However, when I close the browser window, then attempt to re-open the page from a favorite, IE 10 tells me "You're not connected to a network". Obviously I know that, I'm trying to use the app offline. These exact steps work in Chrome.
Is this behavior by design? Is there a workaround? I can't test with IE 11 right now...is this different in IE 11?
Hearing of some issues of the appcache clearing if your company utilizes gpo settings and has "empty temporary internet files folder when browser is closed" enabled.
Did you find the answer to this? I have the same problem. I did get a bit further though. I found that if you go to the IE10 File menu option and tick Work Offline then try and access your cached app it loads the page but I still have an issue as it does not appear to be using the javascript file that should also be cached. All works ok on Google Chrome but our clients are restricted to IE so Chrome is not an option.