README.md - Issue 29342955: Issue 3985 - Unified test runner for sitescripts

Delta Between Two Patch Sets: README.md

Issue 29342955: Issue 3985 - Unified test runner for sitescripts (Closed)

Left Patch Set: Created May 23, 2016, 9:50 p.m.

Right Patch Set: Measure test coverage of everything in sitescripts Created May 25, 2016, 3:37 p.m.

Left:
Right:

Use n/p to move between diff chunks; N/P to move between comments.

Jump to:

Left: Side by side diff | Download
Right: Side by side diff | Download

LEFT	RIGHT
1 # Sitescripts	1 # Sitescripts

2	2

3 ## Introduction	3 ## Introduction

4	4

5 The sitescripts repository contains many of the server side Python scripts that	5 The sitescripts repository contains many of the server side Python scripts that

6 we use. There are both web handlers, which can be used via the multiplexer, and	6 we use. There are both web handlers, which can be used via the multiplexer, and

7 scripts that perform other tasks.	7 scripts that perform other tasks.

8	8

9 The scripts are often unrelated and as such will not all be documented here. For	9 The scripts are often unrelated and as such will not all be documented here. For

10 more information about the individual scripts you will need to look at their	10 more information about the individual scripts you will need to look at their

(...skipping 69 matching lines...) Expand 10 before \| Expand all \| Expand 10 after Loading...
80 `~/.sitescripts`.	80 `~/.sitescripts`.

81 3. Type `python multiplexer.py`, it will start a web server at	81 3. Type `python multiplexer.py`, it will start a web server at

82 http://localhost:5000/ . This web server will use any URL handlers that have	82 http://localhost:5000/ . This web server will use any URL handlers that have

83 been defined in the modules you are testing to respond to requests.	83 been defined in the modules you are testing to respond to requests.

84	84

85	85

86 ## Testing	86 ## Testing

87	87

88 There are tests for some parts of the functionality of sitescripts. They are	88 There are tests for some parts of the functionality of sitescripts. They are

89 located in `test` directories and can be run via	89 located in `test` directories and can be run via

90 [Tox](https://tox.readthedocs.org/). There are two parts of the test suite:	90 [Tox](https://tox.readthedocs.org/):

91 the tests that only depend on the packages that can be automatically installed

92 by tox and the tests that have other dependencies (a database). The first part

93 is run by tox by default:

94	91

95 $ tox	92 $ tox

96

97 The tests dependent on the database are covering the packages `filterhits` and
Sebastian Noack 2016/05/24 07:58:36 Nice idea to use Vagrant for tests with non-python Nice idea to use Vagrant for tests with non-python dependencies. Just some thoughts on that approach: 1. The way you intend those tests to run, couldn't one simply setup the machine with the infrastructure repository, rather than having a separate Vagrantfile for testing in here? 2. If we integrate Vagrant for the tests here, couldn't we do it the other way around, that people just have to run tox, which then setups the VM and runs tests in there if necessary?
98 `notifications`. They can be run in a standard prepared environment using

99 [vagrant](https://www.vagrantup.com://www.vagrantup.com/):

100

101 $ vagrant up

102 Bringing machine 'default' up with 'virtualbox' provider...

103 ==> default: Importing base box 'debian/contrib-jessie64'...

104 ...

105 $ vagrant ssh

106 ...

107 vagrant@debiancontrib-jessie:~$ cd /vagrant

108 vagrant@debiancontrib-jessie:/vagrant$ tox -e filtehits,notifications
mathias 2016/05/23 22:25:32 Do you really want a binary image from an un-trust Do you really want a binary image from an un-trusted source being able to write back files you could potentially execute on your host system? Even if not malicious by intention: I do not see how one would trust a test environment with potentially broken code and potentially just slightly different executable objects that much.
109

110 It's also possible to run the other tests inside of vagrant environment by not

111 giving any arguments to `tox`.

112

113 Please note that since the subpackages of sitescripts are not installed via

114 `setup.py` but are instead imported directly from the working directory, the

115 `.pyc` files that are created next to the `.py` files might be obsolete and or

116 not matching the environment. This might cause the tests to fail and Tox/Pytest

117 normally hints at the source of the problem in these cases. The problem can be

118 fixed by deleting all `.pyc` files:

119

120 $ find . -name '*.pyc' \| xargs rm
mathias 2016/05/23 22:25:32 After addressing my comment above, you could trans After addressing my comment above, you could transform this section into a lecture about not mixing up the environments in the first place, and what to do if it happened by accident.
LEFT	RIGHT