Rietveld Code Review Tool
Help | Bug tracker | Discussion group | Source code

Side by Side Diff: README.md

Issue 4810150141493248: Issue 122 - Puppet ENC via Hiera (Closed)
Patch Set: Puppet ENC via Hiera Created March 4, 2015, 5:59 p.m.
Left:
Right:
Use n/p to move between diff chunks; N/P to move between comments.
Jump to:
View unified diff | Download patch
« no previous file with comments | « .hgignore ('k') | Vagrantfile » ('j') | Vagrantfile » ('J')
Toggle Intra-line Diffs ('i') | Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
OLDNEW
1 Adblock Plus infrastructure 1 Adblock Plus infrastructure
2 =========================== 2 ===========================
3 3
4 The Adblock Plus infrastructure uses [Puppet](http://puppetlabs.com/) 4 The Adblock Plus infrastructure uses [Puppet](http://puppetlabs.com/)
5 to set up servers, and to have a realistic development environment. 5 to set up servers, and to have a realistic development environment.
6 6
7 Our Puppet manifests are only tested with Ubuntu 12.04 right now. 7 Our Puppet manifests are only tested with Ubuntu 12.04 right now.
8 8
9 Private files 9 Environment specific setup
10 ------------- 10 --------------------------
11 11
12 Some parts of our infrastructure are, obviously, confidential. We have 12 Some infrastructure parts are specific to the environment (such as e.g.
13 htpasswd files, SSH keys and SSL certificates that we need to be 13 *development*, *test* and *production*) whilst passwords, for example,
14 careful with. 14 are confidential. In order to allow for such specific configuration, the
15 repository requires a set of manual operations during the initial setup:
15 16
16 That's why _modules/private_ is missing, and needs to be placed there 17 ### `modules/private`
17 manually. We provide stub versions of all those files in 18
18 _modules/private-stub_, so just linking or copying that to 19 The `private` module is destined to store confidential information such as
19 _modules/private_ will make everything work locally. 20 [RSA](http://en.wikipedia.org/wiki/RSA_%28cryptosystem%29) keys, `htpasswd`
21 files and so on. The repository provides a `private-stub` module containing
22 defaults suitable for development and testing purpose. One can create a
Felix Dahlke 2015/03/09 23:14:51 Should be: "testing purposeS"
mathias 2015/03/16 12:07:11 Done.
23 symbolic link to start using the resource:
24
25 #### UNIX-like
26
27 ln -s private-stub modules/private
28
29 #### Windows
30
31 MKLINK /D modules\private private-stub
32
33 When creating a custom version, one may inspect the `modules/private-stub`
34 directory to determine which resources have to be provided.
35
36 ### `hiera/private`
37
38 Analogous to `modules/private`, [Hiera](https://docs.puppetlabs.com/hiera/1/)
39 configuration files specific to the current environment are expected to be
40 found in `hiera/private`. Default resources for development (and testing)
41 purpose are provided within `modules/private-stub/hiera`:
Felix Dahlke 2015/03/09 23:14:51 "purposeS" again
mathias 2015/03/16 12:07:11 Done.
42
43 #### UNIX-like
44
45 ln -s ../modules/private-stub/hiera hiera/private
46
47 #### Windows
48
49 MKLINK /D ..\modules\private-stub\hiera hiera\private
50
51 Note that custom versions are recommended to be tracked together with the
52 custom `private` module, if any.
20 53
21 Development environment 54 Development environment
22 ----------------------- 55 -----------------------
23 56
24 As with our other projects, all changes to our infrastructure should 57 As with our other projects, all changes to our infrastructure should
25 be made in a local development environment, and reviewed before 58 be made in a local development environment, and reviewed before
26 deployment. Thanks to Puppet, we can easily set up local VMs that 59 deployment. Thanks to Puppet, we can easily set up local VMs that
27 mirror our production environment. 60 mirror our production environment.
28 61
29 The most convenient way to do this is to use Vagrant, as described 62 The most convenient way to do this is to use Vagrant, as described
30 below. 63 below.
31 64
32 ### Requirements 65 ### Requirements
33 66
34 * [VirtualBox](https://www.virtualbox.org/) 67 * [VirtualBox](https://www.virtualbox.org/)
35 * [Vagrant](http://vagrantup.com/) 68 * [Vagrant](http://vagrantup.com/)
36 * _modules/private_ exists (see above) 69 * Both `modules/private` and `hiera/private` exist (see above)
37 70
38 ### Start a VM 71 ### Start a VM
39 72
40 For each production server, we have a Vagrant VM with the same host 73 For each production server, we have a Vagrant VM with the same host
41 name. 74 name.
42 75
43 To start the _filter1_ VM: 76 To start the _filter1_ VM:
44 77
45 vagrant up filter1 78 vagrant up filter1
46 79
(...skipping 10 matching lines...) Expand all
57 90
58 vagrant ssh server5 91 vagrant ssh server5
59 92
60 If you want to test "real" SSH access you can use the test user account defined 93 If you want to test "real" SSH access you can use the test user account defined
61 in _private-stub_: 94 in _private-stub_:
62 95
63 ssh -i modules/private/files/id_rsa test@10.8.0.100 96 ssh -i modules/private/files/id_rsa test@10.8.0.100
64 97
65 The default password for this user (required for the _sudo_ command) is "test". 98 The default password for this user (required for the _sudo_ command) is "test".
66 99
67 Adding a server 100 Adding a host
68 --------------- 101 -------------
69 102
70 To set up a new server, you should first add it to the development 103 To set up a new host, extend the custom `hiera/private/host.yaml` by another
71 environment and test the setup, then set up a corresponding production 104 `servers:` item, e.g.:
72 server.
73 105
74 ### Development environment 106 # ...
107 custom1:
108 ip: [10.8.0.254]
109 dns: foobar.example.com
110 ssh_public_key: AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAA...
111 role: codereviewserver
75 112
76 1. Add entries in _Vagrantfile_ and _manifests/vagrant.pp_ 113 See `modules/base/manifests/init.pp`, especially the `explicit_host_record()`
Felix Dahlke 2015/03/09 23:14:51 sounds weird, I'd say "especially the type named `
mathias 2015/03/16 12:07:11 Fair enough, though it's actually a "named type" a
114 named type within class `base`, for more information on the possible option
115 keys and values.
77 116
78 2. Add the host name to one of the manifests imported by 117 In development, this is all that needs to be done before the new box can be
79 _manifests/nodes.pp_ 118 started using `vagrant up ...`. Production servers, however, need a working
119 Puppet configuration first (see below).
80 120
81 3. Make sure the server uses the _nagios::client_ class and add a 121 Configuring Puppet
82 _nagios\_host_ to _manifests/monitoringserver.pp_ 122 ------------------
83 123
84 ### Production environment 124 ### Prerequisites
85 125
86 1. Install Ubuntu Server 12.04 LTS 126 1. Install Ubuntu Server 12.04 LTS
87 2. Perform an update and install Puppet 127 2. Run `hiera/install-precise.py` as user `root` to install Puppet and Hiera
88
89 » apt-get -y update && apt-get -y upgrade && apt-get -y install puppet
90
91 3. Enable pluginsync (Add the following to the _main_ section in 128 3. Enable pluginsync (Add the following to the _main_ section in
92 _/etc/puppet/puppet.conf_) 129 _/etc/puppet/puppet.conf_)
93 130
94 pluginsync=true 131 pluginsync=true
95 132
96 4. Configure the master address (Add the following to the bottom of 133 4. Configure the master address (Add the following to the bottom of
97 _/etc/puppet/puppet.conf_) 134 _/etc/puppet/puppet.conf_)
98 135
99 [agent] 136 [agent]
100 server = puppetmaster.adblockplus.org 137 server = puppetmaster.adblockplus.org
101 138
102 Now you can either set it up as a pure agent or as a master. The 139 Now you can either set it up as a pure agent or as a master. The
103 master provides the configuration, agents fetch it from the master and 140 master provides the configuration, agents fetch it from the master and
104 apply it locally. The master is also an agent, fetching configuration 141 apply it locally. The master is also an agent, fetching configuration
105 from itself. 142 from itself.
106 143
107 #### Puppet agent 144 ### Puppet agent
108 145
109 1. Attempt an initial provisioning, this will fail 146 1. Attempt an initial provisioning, this will fail
110 147
111 puppet agent --test 148 puppet agent --test
112 149
113 2. On the master: List the certificates to get the name of the new 150 2. On the master: List the certificates to get the name of the new
114 agent's certificate 151 agent's certificate
115 152
116 puppet cert list 153 puppet cert list
117 154
118 3. Still on the master: Sign the certificate, e.g. for serverx: 155 3. Still on the master: Sign the certificate, e.g. for serverx:
119 156
120 puppet cert sign serverx 157 puppet cert sign serverx
121 158
122 4. Back on the agent: Attempt another provisioning, it should work now 159 4. Back on the agent: Attempt another provisioning, it should work now
123 160
124 puppet agent --test 161 puppet agent --test
125 162
126 #### Puppet master 163 ### Puppet master
127 164
128 1. Configure the certificate name (Add the following to the _master_ 165 1. Configure the certificate name (Add the following to the _master_
129 section in _/etc/puppet/puppet.conf_) 166 section in _/etc/puppet/puppet.conf_)
130 167
131 certname = puppetmaster.adblockplus.org 168 certname = puppetmaster.adblockplus.org
132 169
133 2. Install the required packages 170 2. Install the required packages
134 171
135 apt-get install puppetmaster mercurial 172 apt-get install puppetmaster mercurial
136 173
(...skipping 31 matching lines...) Expand 10 before | Expand all | Expand 10 after
168 Monitoring is fully functional in any environment, including development. 205 Monitoring is fully functional in any environment, including development.
169 Here, after bootstrapping the `server4` box, one can access the Nagios GUI 206 Here, after bootstrapping the `server4` box, one can access the Nagios GUI
170 from the host machine via <https://nagiosadmin:nagiosadmin@10.8.0.99/>. 207 from the host machine via <https://nagiosadmin:nagiosadmin@10.8.0.99/>.
171 208
172 The monitoring service of our production environment, however, is accessible 209 The monitoring service of our production environment, however, is accessible
173 via <https://monitoring.adblockplus.org/>. 210 via <https://monitoring.adblockplus.org/>.
174 Add yourself to _files/nagios-htpasswd_ in the _private_ module used on the 211 Add yourself to _files/nagios-htpasswd_ in the _private_ module used on the
175 server, or have someone add you if you don't have access. 212 server, or have someone add you if you don't have access.
176 213
177 214
OLDNEW
« no previous file with comments | « .hgignore ('k') | Vagrantfile » ('j') | Vagrantfile » ('J')

Powered by Google App Engine
This is Rietveld