Issue 3985 - Unified test runner for sitescripts

tox.ini

+[tox]
+envlist = py27
+skipsdist = true
+
+[pytest]
+# The names of the test files here don't start with 'test_' so we need
+# to instruct pytest to load tests from all files.
+python_files = *.py
+
+[run]
+# Exclude tests from coverage.
+omit = */test/*
+
+[testenv]
+basepython=python2
+deps =
+    pytest
+    pytest-cov
+    mock
+    jinja2
+setenv =
+    SITESCRIPTS_CONFIG = ./.sitescripts.test
+whitelist_externals =
+    bash
+    rm
+commands =
+    bash -c "sed -e 's/sitescripts\.[a-zA-Z_]*\.web.*/# &/' .sitescripts.example >.sitescripts.test"

[Sebastian Noack, 2016/05/25 07:51:36]

Perhaps we should only exclude the web handlers that will cause an ImportError.
For the other handlers, it's a nice side effect to have implicitly tested
whether they can be imported at least.

[Vasily Kuznetsov, 2016/05/25 15:17:40]

Done.

+    python ensure_dependencies.py
+    py.test \

[Sebastian Noack, 2016/05/25 07:51:36]

I think we should also run flake8, probably with a quite long ignore list. But
at least the whitespaces should comply now, and we should make sure that we
don't regress there.

[Vasily Kuznetsov, 2016/05/25 15:17:40]

Done.

Not sure if this is very meaningful at this point given the length of the ignore
list but at least we can sort of measure progress now. We should probably start
by removing the less common errors and the more "bad" errors.

+        --cov-config tox.ini --cov-report term \
+        --cov sitescripts.hg \
+        --cov sitescripts.notifications \
+        --cov sitescripts.stats \
+        sitescripts/hg/test \
+        sitescripts/notifications/test \
+        sitescripts/stats/test
+    rm .sitescripts.test