Issue 6391 - WebDriver: Fix runtime problen with Gecko and Firefox

test/runners/firefox_process.js

 /*
  * This file is part of Adblock Plus <https://adblockplus.org/>,
  * Copyright (C) 2006-present eyeo GmbH
  *
  * Adblock Plus is free software: you can redistribute it and/or modify
  * it under the terms of the GNU General Public License version 3 as
  * published by the Free Software Foundation.
  *
  * Adblock Plus is distributed in the hope that it will be useful,
  * but WITHOUT ANY WARRANTY; without even the implied warranty of
  * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
  * GNU General Public License for more details.
  *
  * You should have received a copy of the GNU General Public License
  * along with Adblock Plus.  If not, see <http://www.gnu.org/licenses/>.
  */
 
 "use strict";
 
 const {Builder} = require("selenium-webdriver");
 const firefox = require("selenium-webdriver/firefox");
 require("geckodriver");
 
 const {executeScript} = require("./webdriver");
 const {ensureFirefox} = require("./firefox_download");
 
+// Firefox 57 seems to be the minimum to reliably run with WebDriver
+// on certain system configurations like Debian 9, TravisCI.
 const FIREFOX_VERSION = "57.0";

[Sebastian Noack, 2018/08/22 17:02:04]

From the issue description it isn't clear to me why this change bumps up the
Firefox version. Can you elaborate?

FWIW, at some point we'd would want to run the tests on the oldest and newest
supported version, out of scope for now, but is there a chance that this will
ever work with Firefox 51 (current oldest supported version)?

[hub, 2018/08/22 17:36:55]

On *some* systems (Debian 9, TravisCI, even some macOS) Firefox 56 fail. Bumping
to 57 works. That's what Tristan did for
https://codereview.adblockplus.org/29860555/

BTW https://adblockplus.org/en/requirements show Firefox 60 as the minimum,
which contradict the manifest that say 51.

WebDriver support is buggy in older releases.

I'll add a comment to explain.

[Sebastian Noack, 2018/08/22 18:29:05]

There are two level of support. We discourage users to not use anything older
than the previous release version of their browser, and this is what the
requirements page reflects (it is auto generated based on real-time release
information of those browsers). However, technically we support a wider range of
browser versions (sometimes exposing limitations of older browser versions but
with graceful degradation), as reflected in the metadata.* files. This is also
the minimum version our testers do manual testing on, and ideally our automated
tests would run as well on the oldest and newest support browser version. It's
not a priority for now, but we might have to revisit this.

 
 function runScript(firefoxPath, script, scriptName, scriptArgs)
 {
   let binary = new firefox.Binary(firefoxPath);
   binary.addArguments("-headless");
 
   const options = new firefox.Options()
         .setBinary(binary);
 
   const driver = new Builder()
         .forBrowser("firefox")
         .setFirefoxOptions(options)
         .build();
 
   return executeScript(driver, "Firefox", script, scriptName, scriptArgs);
 }
 
 module.exports = function(script, scriptName, ...scriptArgs)
 {
   return ensureFirefox(FIREFOX_VERSION).then(firefoxPath =>
   {
     return runScript(firefoxPath, script, scriptName, scriptArgs)
       .then(result => result)
       .catch(error =>
       {
         throw error;
       });
   });
 };