Ensemble user tutorial p1

I've been adding lots of documentation to help a new Ensemble user find her way around Ensemble. I thought it would nice to share this documentation as a series of blog posts, to raise its exposure and help interested users find their way quickly. Here is the first of a series of posts, which I hope you'll enjoy. If you think we can improve the docs in any way, please do let me know

Introduction

This tutorial demonstrates basic features of Ensemble from a user perspective. An Ensemble user would typically be a devops or a sys-admin who is interested in automated deployment and management of servers and services.

Bootstrapping

The first step for deploying an Ensemble system is to perform bootstrapping. Bootstrapping launches a utility instance that is used in all subsequent operations to launch and orchestrate other instances:

$ bin/ensemble bootstrap

Note that while the command should display a message indicating it has finished successfully, that does not mean the bootstrapping instance is immediately ready for usage. Bootstrapping an instance can require a couple of minutes. To check on the status of the Ensemble deployment, we can use the status command:

$ bin/ensemble status

If the bootstrapping node has not yet completed bootstrapping, the status command may either mention the environment is not yet ready, or may display a connection timeout such as:

INFO Connecting to environment.
ERROR Connection refused
ProviderError: Interaction with machine provider failed:
ConnectionTimeoutException('could not connect before timeout after 2
retries',)
ERROR ProviderError: Interaction with machine
provider failed: ConnectionTimeoutException('could not connect before timeout
after 2 retries',)

This is simply an indication the environment needs more time to complete initialization. It is recommended you retry every minute. Once the environment has properly initialized, the status command should display:

machines:
  0: {dns-name: ec2-50-16-61-111.compute-1.amazonaws.com, instance-id: i-2a702745}
  services: {}

Note the following, machine “0” has been started. This is the bootstrapping node and the first node to be started. The dns-name for the node is printed. Also the EC2 instance-id is printed. Since no services are yet deployed to the Ensemble system yet, the list of deployed services is empty

Starting debug-log
While not a requirement, it is beneficial for the understanding of Ensemble to start a debug-log session. Ensemble’s debug-log provides great insight into the execution of various hooks as they are triggered by various events. It is important to understand that debug-log shows events from a distributed environment (multiple-instances). This means that log lines will alternate between output from different instances. To start a debug-log session, from a secondary terminal issue:

$ bin/ensemble debug-log
INFO Connecting to environment.
INFO Enabling distributed debug log.
INFO Tailing logs - Ctrl-C to stop.

This will connect to the environment, and start tailing logs.

Deploying service units
Now that we have bootstrapped the Ensemble environment, and started the debug-log viewer, let’s proceed by deploying a mysql service:

$ bin/ensemble deploy --repository=examples mysql
INFO Connecting to environment.
INFO Formula deployed as service: 'mysql'
INFO 'deploy' command finished successfully

Checking the debug-log window, we can see the mysql service unit being downloaded and started:

Machine:1: ensemble.agents.machine DEBUG: Downloading formula
local:mysql-11...
Machine:1: ensemble.agents.machine INFO: Started service unit mysql/0

It is important to note the different debug levels. DEBUG is used for very detailed logging messages, usually you should not care about reading such messages unless you are trying to debug (hence the name) a specific problem. INFO debugging level is used for slightly more important informational messages. In this case, these messages are generated as the mysql formula’s hooks are being executed. Let’s check the current status:

$ bin/ensemble status
machines:
  0: {dns-name: ec2-50-16-61-111.compute-1.amazonaws.com, instance-id: i-2a702745}
  1: {dns-name: ec2-50-16-117-185.compute-1.amazonaws.com, instance-id: i-227e294d}
services:
  mysql:
    formula: local:mysql-11
    relations: {}
    units:
      mysql/0:
        machine: 1
        relations: {}
        state: null

We can see a new EC2 instance has now been spun up for mysql. Information for this instance is displayed as machine number 1 and mysql is now listed under services. It is apparent the mysql service unit has no relations, since it has not been connected to wordpress yet. Since this is the first mysql service unit, it is being referred to as mysql/0, subsequent service units would be named mysql/1 and so on.

Note

An important distinction to make is the difference between a service and a service unit. A service is a high level concept relating to an end-user visible service such as mysql. The mysql service would be composed of several mysql service units referred to as mysql/0, mysql/1 and so on.

The mysql service state is listed as null since it’s not ready yet. Downloading, installing, configuring and starting mysql can take some time. However we don’t have to wait for it to configure, let’s proceed deploying wordpress:

$ bin/ensemble deploy --repository=examples wordpress

Let’s wait for a minute for all services to complete their configuration cycle and get properly started, then issue a status command:

$ bin/ensemble status
machines:
  0: {dns-name: ec2-50-16-61-111.compute-1.amazonaws.com, instance-id: i-2a702745}
  1: {dns-name: ec2-50-16-117-185.compute-1.amazonaws.com, instance-id: i-227e294d}
  2: {dns-name: ec2-184-72-156-54.compute-1.amazonaws.com, instance-id: i-9c7e29f3}
services:
  mysql:
    formula: local:mysql-11
    relations: {}
    units:
      mysql/0:
        machine: 1
        relations: {}
        state: started
  wordpress:
    formula: local:wordpress-29
    relations: {}
    units:
      wordpress/0:
        machine: 2
        relations: {}
        state: started

mysql/0 as well as wordpress/0 are both now in the started state. Checking the debug-log would reveal wordpress has been started as well

Adding a relation
While mysql and wordpress service units have been started, they are still isolated from each other. An important concept for Ensemble is connecting various service units together to create a bigger ensemble! Adding a relation between service units causes hooks to trigger, in effect causing all service units to collaborate and work together to reach the desired end state. Adding a relation is extremely simple:

$ bin/ensemble add-relation wordpress mysql
INFO Connecting to environment.
INFO Added mysql relation to all service units.
INFO 'add_relation' command finished successfully

Checking the Ensemble status we see that the db relation now exists with state up:

$ bin/ensemble status
machines:
  0: {dns-name: ec2-50-16-61-111.compute-1.amazonaws.com, instance-id: i-2a702745}
  1: {dns-name: ec2-50-16-117-185.compute-1.amazonaws.com, instance-id: i-227e294d}
  2: {dns-name: ec2-184-72-156-54.compute-1.amazonaws.com, instance-id: i-9c7e29f3}
services:
  mysql:
    formula: local:mysql-11
    relations: {db: wordpress}
    units:
      mysql/0:
        machine: 1
        relations:
          db: {state: up}
        state: started
  wordpress:
    formula: local:wordpress-29
    relations: {db: mysql}
    units:
      wordpress/0:
        machine: 2
        relations:
          db: {state: up}
        state: started

You can now point your browser at the public dns-name for instance 2 (running wordpress) to view the wordpress blog

Patience is a virtue, you did read the full thing after all! I would really appreciate some feedback, what you liked, or did not like. And how we can improve the docs. Drop me a comment, shoot what's on your mind