...
 
Commits (6)
*~
/clouds.yml
*.retry
__pycache__
.cache
pytestdebug.log
docs/_build/
inventories/common/01-hosts.yml
private-key.yml
*.pyc
infrastructure_key*
secret
inventories/common/group_vars/all/domain.yml
inventories/common/group_vars/all/clouds.yml
openrc.sh
inventories/01-hosts.yml
.tox
......
......@@ -5,4 +5,3 @@ source ../virtualenv/bin/activate
sudo apt-get install -y libssl-dev gcc python-dev make
pip install -r requirements.txt
chmod 600 id_rsa # cannot be stored in git
ln -s private-key.yml.example private-key.yml
Ansible
=======
Creation
--------
The `ansible.enough.community` virtual machine was created in the `GRA5` region with:
.. code::
$ openstack keypair create --public-key ~/.ssh/id_rsa.pub loic
$ openstack --quiet server create --image 'Debian 9' --flavor 's1-2' \
--key-name loic --wait ansible
$ scp enough-openrc-production.sh debian@ansible.enough.community:openrc.sh
$ ssh debian@ansible.enough.community
$ sudo apt-get update
$ sudo apt-get install tmux emacs-nox git python-openstackclient rsync virtualenv python-all-dev
$ sudo chown debian /srv
$ rsync -av enough-community/ debian@ansible.enough.community:/srv/enough-community/
$ ( cd /srv/enough-community && git submodule update )
$
$ virtualenv /srv/virtualenv
$ cat >> .bashrc <<EOF
source /srv/virtualenv/bin/activate
source $HOME/openrc.sh
export HISTSIZE=1000000
export PROMPT_COMMAND='history -a' # history -r
EOF
Logout and login again:
.. code::
$ pip install -r /srv/enough-community/requirements.txt
$ ssh-keygen -f infrastructure_key
$ cat > /srv/enough-community/private-key.yml <<EOF
---
ssh_private_keyfile: "{{ lookup('pipe', 'git rev-parse --show-toplevel') }}/infrastructure_key"
EOF
Manually create `/srv/enough-community/clouds.yml` from `~/openrc.sh` and check it works:
.. code::
$ molecule create -s infrastructure
$ molecule destroy -s infrastructure
Set the passwords and other secret credentialis in the file or
directory matching a given host at
`/srv/checkout/inventories/common/host_vars/` (so that the default used during
testing are not used in production).
.. code::
$ echo domain: enough.community | sudo tee /srv/checkout/inventories/common/group_vars/all/domain.yml
Secrets
-------
......@@ -61,41 +8,39 @@ The default credentials (for Weblate, Discourse etc.) are only
suitable for integration testing and must be overriden before
deploying on publicly available hosts. The recommended way of doing this is to:
* fork The `ansible repository <http://lab.enough.community/main/infrastructure/>`_ into a private repository
* add files overriding the secrets in `inventories/common/{host,group}_vars/*/*secrets*.yml`
* create a repository in `~/.enough/enough.community`
* for each files containing secrets in `inventories/common`
(i.e. {host,group}_vars/\*\*/\*.yml`) create a matching file in
`~/.enough/enough.community`
* encrypt those files with `ansible vault <https://docs.ansible.com/ansible/latest/user_guide/vault.html>`_
* share the password to decrypt the files with trusted administrators
* push in a private repository
The encrypted secrets are kept in a private repository to not be
publicly exposed to brute force attacks.
Getting the production repository
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Creation
--------
.. code::
Manually create `~/.enough/enough.community/group_vars/all/clouds.yml` from `~/openrc.sh` and check it works:
$ git clone --recursive \
git@lab.enough.community:main/production-infrastructure.git
$ cd infrastructure
$ git remote add upstream \
git@lab.enough.community:main/infrastructure.git
$ ansible-vault decrypt \
--vault-password-file ~/.vault_pass.txt \
infrastructure_key
.. code::
Rebasing production
~~~~~~~~~~~~~~~~~~~
$ OS_CLIENT_CONFIG_FILE=inventories/common/group_vars/all/clouds.yml openstack --os-cloud ovh server list
.. code::
$ git rebase upstream/master
$ echo domain: enough.community | sudo tee /srv/checkout/inventories/common/group_vars/all/domain.yml
Pushing to production
~~~~~~~~~~~~~~~~~~~~~
Getting the production repository
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. code::
$ git push --force origin master
$ git clone git@lab.enough.community:production/enough.git ~/.enough/enough.community
$ ansible-vault decrypt \
--vault-password-file ~/.enough/enough.community/vault_pass.txt \
~/.enough/enough.community/infrastructure_key
Running
-------
......@@ -103,14 +48,20 @@ Running
Creating new hosts
~~~~~~~~~~~~~~~~~~
From a checkout of the `infrastructure
<https://lab.enough.community/main/infrastructure>`_ repository:
.. code::
ANSIBLE_VAULT_PASSWORD_FILE=$HOME/.vault_pass.txt \
molecule create -s preprod
ansible-playbook --private-key ~/.enough/enough.community/infrastructure_key \
--vault-password-file=~/.enough/enough.community/vault_pass.txt \
-i inventories/common \
-i ~/.enough/enough.community \
molecule/infrastructure/create.yml
It will create the `inventories/01-hosts.yml` file, from which the new
hosts can be copy/pasted into `inventories/common/hosts-definition.yml`
or `inventories/dachary/hosts-definition.yml` etc.
It will create the `inventories/01-hosts.yml` file, which must be
copied to `~/.enough/enough.community/01-hosts.yml` and committed to
the repository.
.. code::
......@@ -126,53 +77,30 @@ Updating
~~~~~~~~
The `ansible repository
<http://lab.enough.community/main/infrastructure/>`_ is run from the
`/srv/checkout` directory of the `ansible.enough.community` virtual
machine as follows:
<http://lab.enough.community/main/infrastructure/>`_ is run as follows:
.. code::
ansible-playbook --private-key infrastructure_key \
--vault-password-file=$HOME/.vault_pass.txt \
ansible-playbook --private-key ~/.enough/enough.community/infrastructure_key \
--vault-password-file=~/.enough/enough.community/vault_pass.txt \
-i inventories/common \
-i ~/.enough/enough.community \
enough-community-playbook.yml
Some hosts contain private information that belong to users who only
trust some administrators of the infrastructure, not all of
them. These hosts only have the ssh public keys of the trusted
administrators and are listed in a dedicated inventory subdirectory.
For instance, the administrator `dachary` owns the the inventory
directory `inventories/dachary`. This administrator can then run the
playbook on all the common infrastructure as well as all the hosts
that can only be accessed by them as follows:
trust some administrators of the infrastructure. These hosts only have
the ssh public keys of the trusted administrators and are listed in a
dedicated inventory subdirectory. For instance, the administrator
`dachary` owns the the inventory directory `inventories/dachary`. This
administrator can then run the playbook on all the common
infrastructure as well as all the hosts that can only be accessed by
them as follows:
.. code::
ansible-playbook --private-key ~/.ssh/id_rsa \
--vault-password-file=$HOME/.vault_pass.txt \
ansible-playbook --private-key ~/.enough/enough.community/infrastructure_key \
--vault-password-file=~/.enough/enough.community/vault_pass.txt \
-i inventories/common \
-i inventories/dachary \
-i ~/.enough/enough.community \
enough-community-playbook.yml
Inventory
---------
The ansible inventory is created by the
``molecule/infrastructure/create.yml`` playbook and stored in the
``inventories/01-hosts.yml`` file every time the ``molecule create``
command runs. The inventory variables (such as the ssh port number)
are read from the ``hosts-base.yml`` file.
It is the responsibility of the system administrator to copy/paste the
content of ``inventories/01-hosts.yml`` in the relevant subdirectory
(`common` etc.).
Updating
--------
The `/srv/checkout` directory is a clone of the `ansible repository
<http://lab.enough.community/main/infrastructure/>`_ and can be updated with:
.. code::
git pull --rebase
......@@ -48,7 +48,7 @@ Getting started
* get OpenStack credentials (ask :doc:`anyone in the <team>`) and store then in `openrc.sh`
* ``source openrc.sh``
* ``openstack server list``: should successfully return nothing on a new tenant
* ``cp clouds.yml.example clouds.yml``
* ``cp clouds.yml.example inventories/common/group_vars/all/clouds.yml`` and edit to match `openrc.sh`
* ``molecule converge -s bind``: create VMs for the scenario `bind` and run ansible playbook defined for this scenario
* ``molecule verify -s bind``: run scenario's tests
* ``molecule login -s bind --host bind-host``: should ssh to the machine
......@@ -68,13 +68,9 @@ consider when working on improving a playbook or a service.
* ``molecule/letsencrypt-nginx``: nginx reverse proxy with letsencrypt integration
* ``molecule/icinga``: resources monitoring
* ``molecule/infrastructure``: VMs creation and firewalling
* ``molecule/misc/roles/commit_etc``: keep track of changes in /etc
* ``molecule/misc/roles/history``: keep track of Ansible runs
* ``molecule/misc/roles/sexy-debian``: non essential Debian specific convenience tweaks
* ``molecule/misc/roles/sshd_config``: /etc/ssh/sshd_config shared by all VMs
* ``molecule/postfix``: outgoing mail relay for all VMs
* ``molecule/preprod``: full preproduction environment. See `Integration testing`_.
* ``molecule/sexy-debian``: optional tools that debian users like to work with
* etc.
The other scenarii found in the `molecule` directory are services such
as `weblate <https://weblate.org/>`_ or `discourse <https://discourse.org/>`_.
......
../../../../../clouds.yml
\ No newline at end of file
---
- include_vars: clouds.yml
- name: mkdir /usr/lib/backup
file:
state: directory
......
......@@ -75,8 +75,6 @@
ns1 1800 IN A {{ ansible_host }}
ansible 1800 IN A 51.68.80.10 ;; remove me
test 1800 IN NS ns1.{{ domain }}.
imap 1800 IN CNAME access.mail.gandi.net.
......
../../../../../clouds.yml
\ No newline at end of file
---
- include_vars: clouds.yml
- name: define os_auth
os_security_group: &os_auth
auth:
......
---
- name: install gitlab runner
hosts: runner-host
vars_files:
- ../../clouds.yml
become: true
roles:
- { role: ansible-role-docker, docker_install_compose: false }
- { role: gitlab-ci }
- role: ansible-role-docker
docker_install_compose: false
become: True
- role: gitlab-ci
......@@ -3,9 +3,6 @@
hosts: gitlab-host
become: true
vars_files:
- ../../clouds.yml
pre_tasks:
- name: set CNAME
nsupdate:
......
......@@ -8,9 +8,6 @@
molecule_instance_config: "{{ lookup('env', 'MOLECULE_INSTANCE_CONFIG') }}"
molecule_yml: "{{ lookup('file', molecule_file) | from_yaml }}"
hosts_orig: "{{ lookup('file', '../../hosts-base.yml') | from_yaml }}"
vars_files:
- ../../clouds.yml
- ../../private-key.yml
pre_tasks:
- name: firewall for ssh ports
include_role:
......
......@@ -6,9 +6,6 @@
molecule_file: "{{ lookup('env', 'MOLECULE_FILE') }}"
molecule_yml: "{{ lookup('file', molecule_file) | from_yaml }}"
hosts_orig: "{{ lookup('file', '../../hosts-base.yml') | from_yaml }}"
vars_files:
- ../../clouds.yml
- ../../private-key.yml
roles:
- role: vm
vars:
......
../../../../../clouds.yml
\ No newline at end of file
---
- include_vars: clouds.yml
- name: Define auth
os_security_group: &os_auth
auth:
......