Loading very large pages in Chromium - chromium

Question

how can I enable latest versions of CefSharp/Cef to utilize more of the available memory on a computer?
Here is a test case:
I load an infinite page, like https://www.facebook.com/Google, then run a script that scrolls down the page, as I want to load as much as possible of that page.
With CefSharp/Cef 79 and earlier, I am able to scroll down to dates back in year 2010
With latest CefSharp/Cef the render process crashes when reaching some point in 2015
Here is the script:
async function wait(intervalInMilliseconds) {
return new Promise((resolve) => setInterval(resolve, intervalInMilliseconds));
}
async function unlimitedScroll() {
for (let i = 0; ; ++i) {
await window.scrollTo(0, document.body.scrollHeight);
await wait(2000);
await console.log(`Scroll: ${i}, total: ${window.performance.memory.totalJSHeapSize.toLocaleString()}, used: ${window.performance.memory.usedJSHeapSize.toLocaleString()}, limit: ${window.performance.memory.jsHeapSizeLimit.toLocaleString()}`);
}
}
unlimitedScroll();
The browser crashes around the time the totalJSHeapSize reaches 4GB - but I am running on a 32GB machine, so I have memory to spare
I tried setting values with --js-flags for the settings --max_heap_size, --max_old_space_size and --max_semi_space_size, but it does not help. As a matter of fact, setting a large value for --max_semi_space_size makes the browser crash even sooner than with the defaults.

Answer 1

As it turned out, it was the introduction of the V8 Pointer compression (https://v8.dev/blog/pointer-compression) that is imposing the hard limit of 4GB JavaScript heap space and preventing the browser from loading very large pages.
When using the Chromium Embedded Framework, this can be solved by making a custom build with pointer compression turned off.
I did this by following the instructions at https://bitbucket.org/chromiumembedded/cef/wiki/MasterBuildQuickStart.md, and before running the build process, manually edited the file file chromium_git\chromium\src\v8\BUILD.gn and commented out the setting of the V8_COMPRESS_POINTERS flag.
The resulting libraries have the drawback of consuming significantly more memory, but the limit is gone, and when running on a machine with 32GB RAM or more, we can load those very large pages successfully.