Issue 5383 - Add tests for the Chrome and Firefox packagers

tests/README.md

Index: tests/README.md
diff --git a/tests/README.md b/tests/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..b0e19d632e4d757b239627bf621c2d9ba175fda9
--- /dev/null
+++ b/tests/README.md
@@ -0,0 +1,73 @@
+# Buildtools unit tests
+
+## Introduction
+
+Unit tests for buildtools, using the pytest framework.
+
+## Approach
+
+In order to test the buildtools capability of creating WebExtension-packages
+for Chrome, Edge and Firefox, an example configuration for each platform is
+provided.
+Running the tests calls the same API as the cli would, except for releases
+(we don't want to trigger our releaseAutomation during tests) - in this case,
+a build is manually created and verified)

[kzar, 2017/10/16 11:17:28]

Nit: Missing full stop, same below.

[tlucas, 2017/10/17 12:45:42]

Done - Although i couldn't find the one below.

+
+Each extension is build with different parameters, the resulting package and
+it's content is compared against provided expected results:
+
+- Release or build-only, with or without a specific build-number (Edge)
+- Release, build-only or developement environment, with or without a specifc
+  build-number (Chrome, Firefox), with or without a predefined signing key
+  (Chrome)
+
+The expected results for each manifest are provided with the files in
+`expecteddata/`.
+
+## Test cases
+
+_(covered platforms are referred to as C=Chrome, E=Edge, F=Firefox)_

[kzar, 2017/10/16 11:17:29]

Nit: Please capitalise "Covered".

[tlucas, 2017/10/17 12:45:43]

Done.

+
+- Metadata inheritance (EF)
+- correct package filename (CEF)

[kzar, 2017/10/16 11:17:26]

Nit: "Correct"

[tlucas, 2017/10/17 12:45:46]

Done.

+- Printed warning about non-square icons (CF)
+- Presence of javascript unit test files in developement environment (CF) 

[kzar, 2017/10/16 11:17:27]

Nit: "JavaScript", same below.

[tlucas, 2017/10/17 12:45:45]

Done.

+- Absence of javascript unit test files in build-only or release (CF)
+- Inlcusion of defined contentScripts into the manifest (CF)
+- Packaging (and moving) of included icons / scripts / htmls (CEF)

[kzar, 2017/10/16 11:17:25]

Nit: "HTML files"

[tlucas, 2017/10/17 12:45:44]

Done.

+- Presence of files for all configured locales (CEF)
+- Correct import of translations with or without placeholders, with or without
+  access keys (CEF)
+- Translation presence and validity for the Chrome Web Store (C)
+- Packaging of modularized script files (CEF)
+- Adherence to provided build-number in the manifest (CEF)
+- Other content in the manifest (CEF)
+- Correct encrypted signature of the package (C)
+
+## Requirements
+
+- [Tox](https://pypi.python.org/pypi/tox) (>= 2.7.0)
+
+_(Tox will take care of installing the other dependencies such as flake8 and
+flake8-abp inside a virtualenv.)_
+
+## Usage
+
+To run the tests simply run
+
+```
+$ tox
+```
+
+in the buildtools' root folder.
+
+## Coverage
+
+Pytest will create a coverage report as output to the terminal, using the
+pytest plugin `pytest-cov`.
+
+In order to get an html report you can add `--cov-report=html` to the
+pytest-command in tox.ini.
+
+For more information please refer to the
+[coverage documentation](https://coverage.readthedocs.io/)