When 2 tests are running in Chrome, i have observed that too many Google Chrome(32 Bit) processes are running in Task manager, Is this a correct behavior of Chome Driver
When multiple automated tests are getting executed through Google Chrome you must have observed that there are potentially dozens of Google Chrome processes running which can be observed through Windows Task Manager's Processes tab.
Snapshot:
As per the article SOLVED: Why Google Chrome Has So Many Processes for a better user experience Google Chrome initiates a lot of windows background processes for each tab that have been opened by your Automated Tests. Google tries to keep the browser stable by separating each web page into as many processes as it deems fit to ensure that if one process fails on a page, that particular process(es) can be terminated or refreshed without needing to kill or refresh the entire page.
However, from 2018 onwards Google Chrome was actually redesigned to create a new process for each of the following entities:
Tab
HTML/ASP text on the page
Plugin those are loaded
App those are loaded
Frames within the page
In a Chromium Blog Multi-process Architecture it is mentioned:
Google Chrome takes advantage of these properties and puts web apps and plug-ins in separate processes from the browser itself. This means that a rendering engine crash in one web app won't affect the browser or other web apps. It means the OS can run web apps in parallel to increase their responsiveness, and it means the browser itself won't lock up if a particular web app or plug-in stops responding. It also means we can run the rendering engine processes in a restrictive sandbox that helps limit the damage if an exploit does occur.
As a conclusion, the many processes you are seeing is pretty much in line with the current implementation of google-chrome
Outro
You can find a relevant discussion in How to quit all the Firefox processes which gets initiated through GeckoDriver and Selenium using Python
Related
I was wondering if anyone had more information on what the specific risks for using chromedriver as was concerned by this statement.
"If possible, run ChromeDriver with a test account that has no access to sensitive local or network data. ChromeDriver should never be run with a privileged account."
Would like to know what the specific risks are when using a privileged account and what if any preventative measures can be taken to protect against them.
Thank you in advance!
How Google Chrome Browser Works
In the article Chrome Browser Security #STEPHANIE CRAWFORD mentioned, Google has leveraged its power as a search engine by creating its Safe Browsing technology which will automatically warn you if Chrome detects that a site you're visiting contains malware or phishing.
Chrome deploys this security measure through a unique security feature termed as Sandboxing. Sandboxing implies, separating each process out into independent spaces to see how they function individually. Chrome handles its workload as a series of multiple processes rather than as part of one large browser process. Each time you open a Web page, Chrome launches one or more new processes to run the scripts on that page. Also, each Chrome extension and app runs in its own process. Chrome implements sandboxing through its multi-process architecture. The security advantage in sandboxing comes with Chrome being able to control the access token for each process. These access token for a process allows that process access to important information about your system, like its files and registry keys. Chrome intercepts each access token from the processes launched from the browser, and it modifies that token to limit its access to that information. So, Chrome's sandboxing helps block web pages that try to install malware, capture your personal information or obtain data from your hard drive. The drawback of sandboxing is that, it can't catch everything. A sandboxed process might still be able to access less secure file systems. It's also likely to miss protecting registry keys and files managed by third party software, like a game or chat program that isn't native to the system.
WebDriver driven Chrome
While initiating a WebDriver controled Chrome Browsing Context using Selenium recently we had been advocating to use a certain command line argument:
--no-sandbox: Disables the sandbox for all process types that are normally sandboxed.
See:
WebDriverException: unknown error: DevToolsActivePort file doesn't exist while trying to initiate Chrome Browser
How to configure ChromeDriver to initiate Chrome browser in Headless mode through Selenium?
unknown error: session deleted because of page crash from unknown error: cannot determine loading status from tab crashed with ChromeDriver Selenium
No Sandbox
There are a couple of more Sandbox related flags available which enables the sandboxed processes to run without a job object assigned to them. This flag is required to allow Chrome to run in RemoteApps or Citrix. This flag can reduce the security of the sandboxed processes and allow them to do certain API calls like shut down Windows or access the clipboard. Also we lose the chance to kill some processes until the outer job that owns them finishes.
--allow-no-sandbox-job: Disables usage of sandbox job.
--allow-sandbox-debugging: Allows debugging of sandboxed processes.
--disable-gpu-sandbox: Disables the GPU process sandbox.
--disable-namespace-sandbox: Disables usage of the namespace sandbox.
--disable-seccomp-filter-sandbox: Disable the seccomp filter sandbox (seccomp-bpf) (Linux only).
--disable-setuid-sandbox: Disable the setuid sandbox (Linux only).
--disable-win32k-lockdown: Disables the Win32K process mitigation policy for child processes.
--enable-audio-service-sandbox: enable the audio service sandbox.
--gpu-sandbox-allow-sysv-shm: Allows shmat() system call in the GPU sandbox.
--gpu-sandbox-failures-fatal: Makes GPU sandbox failures fatal.
--no-sandbox-and-elevated: Disables the sandbox and gives the process elevated privileges (Windows only).
Sandbox
Sandbox leverages the OS-provided security to allow code execution that cannot make persistent changes to the computer or access information that is confidential. The architecture and exact assurances that the sandbox provides are dependent on the operating system.
windows implementation principles:
Do not re-invent the wheel: It is tempting to extend the os kernel with a better security model. Don't. Let the operating system apply its security to the objects it controls. On the other hand, it is just okay to create application-level objects (abstractions) that have a custom security model.
Principle of least privilege: This should be applied both to the sandboxed code and to the code that controls the sandbox. In other words, the sandbox should work even if the user cannot elevate to super-user.
Assume sandboxed code is malicious code: For threat-modeling purposes, we consider the sandbox compromised (that is, running malicious code) once the execution path reaches past a few early calls in the main() function. In practice, it could happen as soon as the first external input is accepted, or right before the main loop is entered.
Be nimble: Non-malicious code does not try to access resources it cannot obtain. In this case the sandbox should impose near-zero performance impact. It's ok to have performance penalties for exceptional cases when a sensitive resource needs to be touched once in a controlled manner. This is usually the case if the OS security is used properly.
Emulation is not security: Emulation and virtual machine solutions do not by themselves provide security. The sandbox should not rely on code emulation, code translation, or patching to provide security.
linux implementation
macos implementation
I write UI automation tests using Selenium. And I noticed that when I create an instance of chromedriver.exe -> ~8 chrome.exe processes appear in the Task Manager.
Task Manager screenshot when running 1 test:
So, when I run in parallel, let's say, 8 tests there are a lot of chrome.exe instances in the Task Manager that use some ports and load a CPU and a memory.
Does it work by design? Why so much chrome.exe instances are needed for one chromedriver.exe? Is this configurable?
In my code, I just have a "Chrome" class. Its constructor creates a new chromedriver.exe instance:
Chrome chrome = new Chrome();
public class Chrome
{
public OpenQA.Selenium.Chrome.ChromeDriver Driver;
public Chrome(bool incognitoMode = false, bool maximizeWindow = true)
{
ChromeOptions options = new ChromeOptions();
if (incognitoMode)
{
options.AddArguments("--incognito");
}
Driver = new ChromeDriver(options);
Driver.Manage().Timeouts().PageLoad = TimeSpan.FromSeconds(360);
if (maximizeWindow)
Driver.Manage().Window.Maximize();
}
/////////////////////////////////////////////
}
When automated tests are executed using Google Chrome you must have observed that there are potentially dozens of Google Chrome processes running which can be observed through Windows Task Manager's Processes tab.
Snapshot:
As per this article for a better user experience Google Chrome initiates a lot of background processes for each of the TABS that have been opened by your #Tests. Google tries to keep the chrome browser stable by separating each web page into as many processes as it deems fit to ensure that if one process fails on a page, that particular process(es) can be terminated or refreshed without needing to kill or refresh the entire page.
However, from 2018 onwards Google Chrome was actually redesigned to create a new process for each of the following entities:
Tab
HTML/ASP text on the page
Plugin those are loaded
App those are loaded
Frames within the page
Further, in a Chromium Blog it is mentioned:
Google Chrome takes advantage of these properties and puts web apps and plug-ins in separate processes from the browser itself. This means that a rendering engine crash in one web app won't affect the browser or other web apps. It means the OS can run web apps in parallel to increase their responsiveness, and it means the browser itself won't lock up if a particular web app or plug-in stops responding. It also means we can run the rendering engine processes in a restrictive sandbox that helps limit the damage if an exploit does occur.
As a conclusion, the many processes you are seeing is as designed and in-line with the current implementation of Google Chrome.
I want to know selenium with firefox question，
I used Firefox version 56.0.2,selenium3.5.1,and geckodriver 0.19.1,server ubuntu(x64) os,firefox --headless mode
I find when I run my app for a long time, the firefox memory will increase a lot, such 400MB or more, and when I let firefox open about:blank, the memory don not decrease
I want to know how to decrease the firefox memory (do not kill the firefox process), only use the selenium to control firefox or start firefox with some config
I want to open "about:blank" or other URL to reduce memory,but I find it doesn't work;
No,Selenium itself doesn't uses any memory. It's the WebDriver and Web Browser processes which consumes the memory. For an example when you create a new instance of any WebDriver variant to start a relevant Web Browsing Session, both the process consumes memory.
Now the different Browser Client variants will follow different and distinct methods and style to initiate, manage and teardown the browser internal process. So memory consumption will be different for the different browsers.
Answering your questions :
When I run my app for a long time, the firefox memory will increase a lot : In-coarse of handling a active browsing session, the browser binary have to keep track a lot of memory (stack memory / heap memory) resources time to time. Hence memory consumption can go up/down depending on situation.
I want to know how to decrease the firefox memory : No, you have no control over the memory consumption of a browser.
Solution
Web Browsers have evolved a lot recently. Each of the Web Browser variants e.g. Mozilla, Chrome and Internet Explorer are continuously working on a more momery efficient browser process. You can take the following steps for your Automated Tests to consume the optimum memory :
Keep the JDK Version updated to the latest versions as each release aims towards optimum memory utilization.
Keep the Selenium Version updated to the latest versions as each release aims towards better memory management.
Keep the WebDriver Version updated to the latest versions as each release aims towards better memory management.
Keep the Web Browser Version updated to the latest versions as each release aims towards better memory utilization.
If the base version of your Web Browser is too older, you may consider to uninstall the Web Browser through Revo Uninstaller and install a recent stable and GA version of the Web Browser .
Use CCleaner tool before and after your Test Execution to wipe off the OS system chores.
Perform the Test Execution in an isolated system free from Manual Intervention
Keep the Test System within the Test Lab well equipped with the Hardware Requirements to execute the Test Suites.
I am using selenium WebDriver using RobotFramework. The major problem we are facing is, my tests are timing out even after setting timeout as high as 10 minutes. It happen with any browser I use. These thing works much faster If I run test manually (with all browser cache/data/cookie cleared). These are the other things I have observing for few months.
Some component are never loaded (I check the call trace using BrowserMob Proxy and we found nothing unusual)
"Click Element" does not work in many cases. Clicking on element triggers some action but that action is not always trigerred in automation. Manually it works all the time.
Notes:
This is happening on FF and Chrome. (IE is not working for me at all)
App server and automation suite is in LAN so latency is not an issue
No other heavy process is running at this time
Issue persist even if with firefox default profile
I tried it with different selenium versions. (2.45 - 2.52)
I took latest driver for chrome. Broweser: FF 40+ and GC 48
This does not look application issue as we spent 2 month confirming that. Let me know if you need any other detail.
I'm executing selenium tests with testng, that are started on a remote system with Selenium RC via hudson (with ssh connection). The remote system is windows xp with MKS Toolkit installed, hence ssh. Tests are NOT executed as a windows service.
I've tried using both captureScreenshot and captureEntirePageScreenshot methods. The first one always produces a black image. The second one creates the correct screen shot but it only works on Firefox and our tests usually pass on Firefox and fail in other browsers, so it is crucial to capture screen shots for the other browsers (mainly IE and Safari). The tests are ran in parallel, with many browser windows open at the same time. I'm not certain if this is what's causing the problem. Any thoughts will be appreciated.
Unfortunately screenshots in Selenium have been problematic from the start in browsers that are not Firefox. This is something that we Selenium Developers have been working on for a while to correct.
The latest work has been updating Snapsie to work in IE. There is a blog post at http://blog.codecentric.de/en/2010/02/remote-screenshots-mit-selenium-und-dem-robot-framework/ that explains what has happened.
I have noticed that if the screen isn't active, i.e. the screensaver has kicked in, it can produce black screenshots.
Edit:
I just had a thought. You can always run Castro to video record your tests and then watch it play back. This is something SauceLabs use to run Selenium in the cloud.
Write a method for this and call that whereever you need to take the screenshot. Use the java.awt package which has been used in selenium. For example, check this site
After setting Windows Auto-Logon, and launching process not as Windows Service, I found how to solve the Remote Desktop with Black Screenshots problem of IEDriverServer.exe, by creating a batch file that disconnects RDP, instead of closing the RDP session with the regular X button:
%windir%\system32\tscon.exe %SESSIONNAME% /dest:console
See more details here:
https://stackoverflow.com/a/24529629/658497
(Although, I would prefer there was a way to run it as the default action, when terminating RDP session with X Windows button).