Advanced cloud-init custom handlers

I love cloud-init, the Ubuntu cloud technology that enables a cloud instance to bootstrap itself and customize itself into whatever you want it to be (coming from a generic image). I had previously created a screencast introducing cloud-init, and written an article on running cloud-init locally over KVM. This time however, I demonstrate some advanced cloud-init foo, namely how to write a custom content handler in python. Pass the code and data over the cloud's user-data, and watch your code crunch on the data. The possibilities are endless, using this technique you are basically writing plugins for cloud-init allowing you to do almost anything. Ok, enough babbling, let's do some cool stuff

We need the "write-mime-multipart" script from cloud-init. I would not recommend installing cloud-init on your own machine, since cloud-init is designed to be run on cloud instances (not physical nodes). If you do install it on your own machine, it blocks the boot process waiting for the cloud userdata service to appear (which it never does), so you end up waiting a lot! To get the script, we just check out the code directly

bzr branch lp:cloud-init

You'll find that script in the tools/ directory. Assuming you could run cloud-init on local KVM, we now need to replace the user-data file, which in the previous article was written using cloud-config syntax, with a new file. The new user-data file is a multipart file composed of custom python code and data you want your python code to chew on! Here is how you create the file

./write-mime-multipart --output user-data part-handler.txt one:text/plain two:text/plain

Let's take a look at the contents of those files. Files "one" and "two" are the data, while "part-handler.txt" is the python code adapted from cloud-init. In our case, I chose to let our part handler be a "user-creation" provider, i.e. you supply a list of user names in the data file, the code loops over them creating them. Simple enough for an example I hope. Let's check out the data files

$ cat one
jhonny
cash
$ cat two 
agent
smith

These two files hold the 4 users to be created! I split them into 2 files, just to demo you could have multiple input files. Now let's check out the code living in part-handler.txt

#part-handler
# vi: syntax=python ts=4

def list_types():
    # return a list of mime-types that are handled by this module
    return(["text/plain", "text/go-cubs-go"])

def handle_part(data,ctype,filename,payload):
    # data: the cloudinit object
    # ctype: '__begin__', '__end__', or the specific mime-type of the part
    # filename: the filename for the part, or dynamically generated part if
    #           no filename is given attribute is present
    # payload: the content of the part (empty for begin or end)
    if ctype == "__begin__":
       print "my handler is beginning"
       return
    if ctype == "__end__":
       print "my handler is ending"
       return

    print "==== received ctype=%s filename=%s ====" % (ctype,filename)
    import os
    for user in payload.splitlines():
        print " == Creating user %s" % (user)
        os.system('sudo useradd -p ubuntu -m %s' % (user) )
    print "==== end ctype=%s filename=%s" % (ctype, filename)

Most of the code is just boiler plate. The code needs to implement two functions "list_types()" that returns a list of content types that this code can handle. In our case, we return "text/plain" and "text/go-cubs-go". Note that when we ran "write-mime-multipart" we used the mime type text/plain, and that is the reason cloud-init would invoke this code to handle it. Because the code advertises it can handle text/plain types. The second function the code needs to implement is handle_part. This is called at the very beginning and very end for initialization and tear-down. It is also called on each input file (2 times in our case). A sample run looks like


and sure enough, the 4 users were created


That should be everything you need to know about writing custom mime type handlers as extensions to cloud-init. Indeed that is some pretty amazing stuff! Any questions or comments, leave a comment