Chromium/Chrome headless - file download not working? - chromium

Question

I downloaded the latest version of chromium, to test out the headless feature.
When I run (as root, because I'm still testing things):
./chrome --no-sandbox http://cp7.awardspace.com/speed-test/awardspace-data1mb.zip
In the GUI terminal, it opens Chromium and downloads the file.
If I'm trying to run it headless, I enter the following:
./chrome --no-sandbox --headless http://cp7.awardspace.com/speed-test/awardspace-data1mb.zip
The terminal outputs some information, no window gets opened, but also: I don't have the file downloaded anywhere.
I have been scouting the internet and discussion groups for more information, but cannot find anything.
Is file downloading not working in headless mode for Chromium?

Answer 1

That's a reported bug in headless implementation:
https://bugs.chromium.org/p/chromium/issues/detail?id=696481

Answer 2

Use ChromeDriverService and POST session/{sessionId}/chromium/send_command
JSON for POST:
{
"cmd": "Page.setDownloadBehavior",
"params": {
"behavior": "allow",
"downloadPath": "C:\\Download\\Path"
}
}
C# Solution
Add reference to System.Net.Http and use NuGet to install Newtonsoft.JSON.
public static IWebDriver Driver { get; private set; }
public void SetDriver()
{
var chromeOptions = new ChromeOptions();
chromeOptions.AddArguments("--headless", "--window-size=1920,1080");
var driverService = ChromeDriverService.CreateDefaultService();
Driver = new ChromeDriver(driverService, chromeOptions);
Task.Run(() => AllowHeadlessDownload(driverService));
}
static async Task AllowHeadlessDownload(ChromeDriverService driverService )
{
var jsonContent = new JObject(
new JProperty("cmd", "Page.setDownloadBehavior"),
new JProperty("params",
new JObject(new JObject(
new JProperty("behavior", "allow"),
new JProperty("downloadPath", #"C:\Download\Path")))));
var content = new StringContent(jsonContent.ToString(), Encoding.UTF8, "application/json");
var sessionIdProperty = typeof(ChromeDriver).GetProperty("SessionId");
var sessionId = sessionIdProperty.GetValue(Driver, null) as SessionId;
using (var client = new HttpClient())
{
client.BaseAddress = driverService.ServiceUrl;
var result = await client.PostAsync("session/" + sessionId.ToString() + "/chromium/send_command", content);
var resultContent = await result.Content.ReadAsStringAsync();
}
}

Answer 3

In Java use following code :
System.setProperty("webdriver.chrome.driver", "/usr/local/bin/chromedriver");
ChromeOptions options = new ChromeOptions();
options.addArguments("--test-type");
options.addArguments("--headless");
options.addArguments("--disable-extensions"); //to disable browser extension popup
ChromeDriverService driverService = ChromeDriverService.createDefaultService();
ChromeDriver driver = new ChromeDriver(driverService, options);
Map<String, Object> commandParams = new HashMap<>();
commandParams.put("cmd", "Page.setDownloadBehavior");
Map<String, String> params = new HashMap<>();
params.put("behavior", "allow");
params.put("downloadPath", "//home//vaibhav//Desktop");
commandParams.put("params", params);
ObjectMapper objectMapper = new ObjectMapper();
HttpClient httpClient = HttpClientBuilder.create().build();
String command = objectMapper.writeValueAsString(commandParams);
String u = driverService.getUrl().toString() + "/session/" + driver.getSessionId() + "/chromium/send_command";
HttpPost request = new HttpPost(u);
request.addHeader("content-type", "application/json");
request.setEntity(new StringEntity(command));
httpClient.execute(request);
driver.get("http://www.seleniumhq.org/download/");
driver.findElement(By.linkText("32 bit Windows IE")).click();

Answer 4

The following code works in C# using ChromeDriver 2.46
private ChromeDriver GetDriver()
{
var options = new ChromeOptions();
options.AddArguments("headless");
options.AddUserProfilePreference("download.prompt_for_download", "false");
options.AddUserProfilePreference("download.directory_upgrade", "true");
options.AddUserProfilePreference("download.prompt_for_download", "false");
options.AddUserProfilePreference("safebrowsing.enabled", "false");
options.AddUserProfilePreference("safebrowsing.disable_download_protection", "true");
options.AddArguments("--disable-web-security");
var curr = Directory.GetCurrentDirectory();
options.AddUserProfilePreference("download.default_directory", curr);
var driver = new ChromeDriver(options);
Log.Info($"Started Chrome Driver with options: {options.ToJsonNoTypes()}");
var param = new Dictionary<string, object>();
param.Add("behavior", "allow");
param.Add("downloadPath", curr);
driver.ExecuteChromeCommand("Page.setDownloadBehavior", param);
return driver;
}

Answer 5

Note: Not exactly answer to the question, but solves the problem
I researched a lot on making headless chrome download with different parameters/options/preferences, but nothing worked. Then I used standard Java way of downloading file using Apache Commons's FileUtils
FileUtils.copyURLToFile(URI, FILE);

Answer 6

I was able to download files with chrome headless thanks to Chrome Remote Interface
public void TryEnableFileDownloading(string downloadPath)
{
TrySendCommand("Page.setDownloadBehavior", new Dictionary<string, object>()
{
["behavior"] = "allow",
["downloadPath"] = downloadPath
});
}
Full code for integration with selenium could be found here
https://github.com/cezarypiatek/Tellurium/blob/master/Src/MvcPages/SeleniumUtils/ChromeRemoteInterface/ChromeRemoteInterface.cs
More info about setDownloadBehavior and Chrome Remote interface
https://chromedevtools.github.io/devtools-protocol/tot/Page/#method-setDownloadBehavior

Answer 7

I tried today in IdeaJ editor, Java and Maven on Windows 10, and it's working fine:
ChromeOptions ds = getDesiredCapabilities(browserName);
ChromeDriverService driverService = ChromeDriverService.createDefaultService();
ChromeDriver driver = new ChromeDriver(driverService, ds);
Map<String, Object> commandParams = new HashMap<>();
commandParams.put("cmd", "Page.setDownloadBehavior");
Map<String, String> params = new HashMap<>();
params.put("behavior", "allow");
params.put("downloadPath", System.getProperty("user.home") + File.separator + "Downloads");
commandParams.put("params", params);
ObjectMapper objectMapper = new ObjectMapper();
HttpClient httpClient = HttpClientBuilder.create().build();
String command = objectMapper.writeValueAsString(commandParams);
String u = driverService.getUrl().toString() + "/session/" + driver.getSessionId() + "/chromium/send_command";
HttpPost request = new HttpPost(u);
request.addHeader("content-type", "application/json");
request.setEntity(new StringEntity(command));
httpClient.execute(request);
User this configured "driver" instance to navigate to download file either by URL or by Json/AngularJs download action.
Also it sometimes behaved to corrupt file while downloading due to internet slowness.
More on this, same configuration will work for both Google Chrome UI or Headless execution in latest chrome driver 77~