[xenial] Document upgrade process

[eloquence]

Description

As part of #3204 and 0.12.0, we need to provide detailed, step-by-step instructions for administrators on how to perform the upgrade from Ubuntu 14.04 to Ubuntu 16.04. These can live in the "Upgrade SecureDrop" section of the documentation.

• Recommend sequential mon->app upgrade
• Per [xenial] Investigate/fix apt sources under xenial when upgrading  #4017, ensure we document the need to run playbooks in order to fix apt sources
• Document upgrade prompts we know about

User Story

As a SecureDrop administrator, I want to know how to upgrade the base OS on the servers to a supported LTS series, so that I continue to receive security updates.

[zenmonkeykstop]

Starting to sketch out these docs in a branch on my SD fork.

Ran through do-release-upgrade via Tails admin against Monitor Server on physical hardware (7-series NUC). Prompts were as follows:

• do-release-upgrade:

  • ssh mon
  • edit /etc/update-manager/release-upgrades, change DEFAULT=never to DEFAULT=lts
  • press Enter after the "Some third party entries..." message
  • press y to continue after "Fetching and installing..." message (screen is messed up for me - resizing it helps a little (130x25 seems best) )
  • Choose OK on postfix info dialog
  • Choose No Configuration and OK on General type of mail configuration dialog
  • Choose Yes on Configuring libssl.0.0:amd64 dialog (connection breaks, presumably because sshd restarted)
  • ssh mon again after a few seconds
  • Choose No to overriding local changes, on PAM configuration dialog
  • Choose default action (keep local) on blacklist.conf changes
  • Choose Keep the local version currently installed on Configuring grub-efi-amd64 dialog
  • Choose default (keep current version) for /etc/ssh/moduli
  • Choose default (keep current version) for /etc/ssh/ssh_config
  • Choose default (keep current version) for /etc/pam.d/sshd
  • Choose keep local version currently installed and OK on Configuring unattended-upgrades dialog
  • Choose Y to remove obsolete packages
  • Choose Y to restart system and complete update

• Update completes successfully, can't reconnect after initial reboot (might just be impatience on my part), but after rebooting again I can connect via ssh mon.

• Updated apt repo and key in install_base/ansible_files/roles/install_fpf_repo and ran ./securedrop-admin install (knowing that yes, this will create an unholy monster with a Xenial mon and a Trusty app). It fails at the task common: Install tmux with the error

fatal: [mon] FAILED! => {"changed": false, "msg", "Failed to update apt cache: "}

[eloquence]

WIP branch with some initial content by @zenmonkeykstop and myself, for now mainly focused on the "in-place" story: https://github.com/freedomofpress/securedrop/tree/4057-docs-xenial-update (main file is xenial_upgrade_in_place.rst).