Issue 3996 - Make sure to install current versions of Markdown and Jinja2 modules on web servers

Note that old packages have to be removed manually before provisioning the web servers.

Repository: hg.adblockplus.org/infrastructure

[mathias, 2016-04-27 12:35:29]

Obviously we tackled the very same issue at the same time, you just happen to be
faster with the updates in the associated tools. In fact both our patch-sets are
pretty similar (https://codereview.adblockplus.org/29340860/), yet according to
my tests the manual removal of any packages is not necessary..?

https://codereview.adblockplus.org/29340860/diff/29340861/modules/web/manifes...
File modules/web/manifests/server.pp (right):

https://codereview.adblockplus.org/29340860/diff/29340861/modules/web/manifes...
modules/web/manifests/server.pp:44: package {'python-pip':
Please use ensure_packages() instead of an explicit resource definition.

https://codereview.adblockplus.org/29340860/diff/29340861/modules/web/manifes...
modules/web/manifests/server.pp:49: ensure => 'present',
In order to improve consistency between environments and hosts please use
specific version numbers as soon as a requirement for such (or a min. version)
has been communicated.

[Wladimir Palant, 2016-04-27 13:00:42]

On 2016/04/27 12:35:29, mathias wrote:
> Obviously we tackled the very same issue at the same time, you just happen to
be
> faster with the updates in the associated tools.

Well, that's why we set owners on the issue ;). And I also mentioned on IRC that
I'm looking into this issue...

> In fact both our patch-sets are
> pretty similar (https://codereview.adblockplus.org/29340860/),

You meant https://codereview.adblockplus.org/29340857/ apparently.

> yet according to
> my tests the manual removal of any packages is not necessary..?

Provisioning succeeds, however the old package version is still being used. I
got the correct version by running `sudo apt-get remove python-jinja2
python-markdown` and provisioning then.

https://codereview.adblockplus.org/29340860/diff/29340861/modules/web/manifes...
File modules/web/manifests/server.pp (right):

https://codereview.adblockplus.org/29340860/diff/29340861/modules/web/manifes...
modules/web/manifests/server.pp:44: package {'python-pip':
On 2016/04/27 12:35:28, mathias wrote:
> Please use ensure_packages() instead of an explicit resource definition.

Done.

https://codereview.adblockplus.org/29340860/diff/29340861/modules/web/manifes...
modules/web/manifests/server.pp:49: ensure => 'present',
On 2016/04/27 12:35:28, mathias wrote:
> In order to improve consistency between environments and hosts please use
> specific version numbers as soon as a requirement for such (or a min. version)
> has been communicated.

Done. Note that there seems to be an ugly side-effect: puppet will try to
install markdown every time things are provisioned, it seems that it cannot
detect the version correctly.

[mathias, 2016-04-27 13:21:28]

Indeed, obviously the native package location has precedence in sys.path...
Well, in that case we need to ensure the native package being held
($ensure='held' with the default provider), in order to avoid ugly side-effects
in case any other native package declares (or ever will declare) a dependency on
python-markdown.

https://codereview.adblockplus.org/29340860/diff/29340861/modules/web/manifes...
File modules/web/manifests/server.pp (right):

https://codereview.adblockplus.org/29340860/diff/29340861/modules/web/manifes...
modules/web/manifests/server.pp:49: ensure => 'present',
On 2016/04/27 13:00:43, Wladimir Palant wrote:
> On 2016/04/27 12:35:28, mathias wrote:
> > In order to improve consistency between environments and hosts please use
> > specific version numbers as soon as a requirement for such (or a min.
version)
> > has been communicated.
> 
> Done. Note that there seems to be an ugly side-effect: puppet will try to
> install markdown every time things are provisioned, it seems that it cannot
> detect the version correctly.

Acknowledged.

[Wladimir Palant, 2016-04-27 13:39:51]

On 2016/04/27 13:21:28, mathias wrote:
> Indeed, obviously the native package location has precedence in sys.path...
> Well, in that case we need to ensure the native package being held
> ($ensure='held' with the default provider), in order to avoid ugly
side-effects
> in case any other native package declares (or ever will declare) a dependency
on
> python-markdown.

From what I can tell, ensure=>absent will already do? This will make sure that
nothing can install these packages - and have the nice side-effect of removing
them from our servers automatically, so the manual step should no longer be
necessary.

[mathias, 2016-04-27 13:43:56]

> From what I can tell, ensure=>absent will already do? This will make sure that
> nothing can install these packages - and have the nice side-effect of removing
> them from our servers automatically, so the manual step should no longer be
> necessary.

No, actually ensure=>'absent' only takes effect at provisioning time, and only
in the step where the respective package resource is taken care of. Any
subsequently installed package (and even in the same Puppet run) that may define
a dependency on the 'absent' one will again cause it to be installed. Marking a
package as 'held', on the other hand, will cause APT itself to ensure the
package not being installed, and even an explicit installation command will not
affect the system (but return EXIT_SUCCESS).

[Wladimir Palant, 2016-04-27 13:52:20]

Well, ensure=>'held' it is then...

[mathias, 2016-04-27 13:57:30]

LGTM.