# Immutable Containers

The term container normally refers to a runnable environment that is generated from an existing image; it can run on bare metal, virtual machines or cloud services and can rely on different image formats.

In this document we are going to present a technical solution to orchestrate image (hierarchical) definitions and container executions defining Infrastructure as Code.

We're going to follow the concept of [immutable infrastructure](http://chadfowler.com/blog/2013/06/23/immutable-deployments/) by isolating image creation from orchestration:

Image Creation:
- Packer/Racker to build container images (AMI for AWS, OVF for Virtualbox)
- Chef/Berkshelf to run as Packer provisioner

Orchestration:
- Cloudinit to start services as soon a node is bootstrapped
- Terraform to describe containers and other infrastructure elements as code (and apply)

> Although containers can run any OS of choice, the current solution uses Chef code that only supports RedHat and Debian OSes (potentially Windows); to overcome this limit, an alternative to Chef must be discussed.

## Advantages of Immutable containers

1. Clear separation between provisioning and "state" of the container
2. Simpler and faster bootstrap process, since provisioning is not needed
3. Images can be committed and versioned, allowing easier reuse for development/testing/debugging purposes
4. Images can be hierarchical, avoiding duplication of provisioning efforts across similar containers

## Code structure

We're going to use Github to host our Infrastructure Code.

1. [packer-common](https://github.com/Alfresco/packer-common) repository
  1. Provisioner Templates
  2. Builder Templates
  3. run-packer Command
  4. Terraform templates (to be discussed)
2. Chef Image cookbooks (i.e. [chalf-img-basic](https://github.com/Alfresco/chalf-img-basic))
  1. Image-specific provisioning (Chef cookbook + Berksfile for dependant cookbooks)
  2. Packer Template with Image-specific configurations:
    1. Chef JSON attribute file
    2. AWS and Virtualbox Image sources (AMI and OVF file)
3. Chef Environment cookbooks (i.e. [chalf-env-activiti](https://github.com/Alfresco/chalf-env-activiti))
  1. Environment-specific provisioning (Chef cookbook + Berksfile for dependant cookbooks)
  2. Packer Template with Environment-specific configurations (as above)
  3. Cloudinit scripts (to be done)

## Changes Management

When an Image/Environment cookbook changes, Packer creates a new image with the following (automated) steps:
  1. Identify source container for AWS builder (based on AMI tags) and Vbox (based on file path)
  2. Spin up a container (Vbox and EC2) with the image sources from #1
  3. Apply provisioning steps to the container
  4. Shutdown container
  5. Dump disk content (only for '/' mount point) into a new (target) image
  6. Image is pushed into the Image Repository (AWS for AMIs, HTTP Server for Vagrant boxes)

When containers are bootstrapped from pre-built (Environment) images, Cloudinit kicks in
  1. Mount volumes
  2. Register DNSes
  3. Configure services with state-specific parameters (KISS)
  4. Start services

When something is wrong with a container, it can be simply killed, causing an infrastructure change

When an infrastructure change occurs, Terraform runs, creates an execution plan and makes the changes needed to setup the infrastructure in the desired state.

Bamboo defines the [dashboard](bamboo.alfresco.com/bamboo/browse/DEVOPS) of Image and Environment builds, controlling continuous test/build and release processes.

## Container hierarchy

1. Official CentOS 6.5 latest release
2. Alfresco Base Image (based on #1)
  1. Yum repositories
  2. Yum update (incl. CentOS update to 6.6)
  3. Sudo security patch
  4. Remove unused/unsecure packages
  5. Add Sysadmin users, SSH keys and groups
  6. Install monitoring tools
    1. salt minion
    2. collectd
    3. rsyslogd
    4. lumberjack (now beaver)
    5. graphite provisioning handler
    6. mail provisioning handler
    7. sensu
    8. opsview-agent
  7. Install security tools
    1. snortd
  8 Install Common services
    1. sshd
    2. ntpd
    3. crond
    4. ip6tables
3. Alfresco Service Image (based on #2)
  1. Install Nginx
  2. Install Oracle Java 7.x
4. Alfresco Command (based on #2)
  1. Install some utility scripts
5. my.alfresco.com Stack
  1. Alfresco Cloud App (based on #3)
    1. Installs Alfresco Repository and Share WAR files (2 separate tomcats)
    2. Creates environment-specific configuration (i.e. $TOMCAT_HOME/shared/classes)
  2. Alfresco Cloud Trans (based on #3)
    1. Installs LibreOffice, ImageMagik and font packages
    2. Installs Alfresco Repository WAR file
  3. Alfresco Cloud Trans (based on #3)
    1. Installs LibreOffice, ImageMagik and font packages
    2. Installs Alfresco Repository WAR file
6. activiti.com Stack
  1. Activiti App (based on #3)
    1. Installs Activiti WAR file
7. sparta Stack
  1. ...

## Next Steps

1. Complete Chef code for activiti app node ([env-activiti::node-app](https://github.com/Alfresco/chalf-env-activiti/blob/master/recipes/node-app.rb))
  1. Chef todo: force all services (installed during provisioning), across all Chef code, to be disabled and stopped; it's cloud-init duty to enabling and starting them
2. Write cloudinit file for app node and run it against the node app AMI; check that
  1. All systems are fully functional
  2. Logging and monitoring systems are properly integrated/connected
3. Create Packer templates for [env-activiti::node-es](https://github.com/Alfresco/chalf-env-activiti/blob/master/recipes/node-es.rb) and [env-activiti::node-command](https://github.com/Alfresco/chalf-env-activiti/blob/master/recipes/node-command.rb) activiti nodes
4. Write cloudinit files for node-es and node-command
5. Change `bootstrap.py` to omit Chef run
6. Spin off an Activiti staging stack using Hera and pointing to the new AMIs
