Bash script self-updates from github

Recently the systems engineering team @ Cloud9ers ($dayjob) has been busy building the infrastructure for a very ambitious and large scale project for one of our customers. The project involves tons of distributed programming besides lots of systems work as well. I'd like to share with the community a small tidbit of information. It's often some small things the one sometimes gives a moment to admire, and hereby share!

The problem

• We are building a private cloud. Ubuntu server instances running on that are cloned from a master template
• While most clouds provide what I call "Instance Identity" information through some pre-known web service URI, in our case we utilize VMware's "Invoke-VMScript" API to run scripts inside the cloned template, thus customizing it and giving it its identity, then puppet takes it from there. Note that we're using a simple vCenter+ESXi (no vCloud stuff here) no shared storage even!

Ok the real problem

• Editing the template, and publishing it across the cloud (a topic for another post) takes considerable time! I wanted a way to be able to quickly update my identity scripts without having to re-build and re-publish images

Solution

• A script with a trivially simple (thus mostly fixed) "bootstrap" section, which auto-updates itself from github and relaunches its-new-self!

Code

Nothing ground breaking, but still a cool trick eh! Yeah I could have added a broken this into multiple scripts and maybe run-parts them. Of-course this script is quite basic, but it's meant to remain that way since the heavy lifting is puppet's responsibility!
Note: To run this successfully the user this is run under, should have passwordless sudo rights to run "ip" and to run "itself" :) Got cool ideas, thoughts or comments? Leave me a message

LXC lies about /.. inode number making FlexLM unhappy

One of our customers wanted to run a flexlm licensed tool on the Amazon EC2 cloud. He turned to us over here at Cloud Niners and we started doing the work. The approach we chose was to start an ec2 Ubuntu precise machine, host a CentOS LXC container on it and run the license daemon inside that. The reason for the CentOS choice is that the tools and license daemons are only certified on RHEL and CentOS is the closest next thing

Trying to start the license daemon, I hit the following error

 8:59:27 (TOOLNAME) Cannot open daemon lock file

Checking the usual suspicions of permissions and the like led no where. Google immediately led me to similar problems for people wanting to run Flexlm tools under Solaris zones (which are quite similar to Linux LXC containers). I was reading about a guy facing a similar problem, and how Solaris' legend Brendan Gregg wrote dtrace scripts to run-time patch memory structures to resolve the issue.

At first I was dismissive that this was what I was actually see'ing. A quick "ls -lai /" just to confirm /. and /.. actually had the same inode number, and I was sure it was something else

# ls -lai / | grep '\.$'

  256 dr-xr-xr-x   1 root root   212 Dec 26 09:08 .

  256 dr-xr-xr-x   1 root root   212 Dec 26 09:08 ..

but I had one of those mmm moments after stracing the binaries and confirming they are actually failing immediately after calling getdents on "/". This sounds suspiciously close to what the Solaris folks were see'ing. I grabbed gcc and built the sample code in getdents kernel doc page (Thankful!) And much to my surprise, inode numbers for /. and /.. were actually different!

# ./a.out / | grep '\.$'

     256  directory    24            1  .

   42669  directory    24          2  ..

Of course this makes sense since the LXC guest was just another directory on the host, but I didn't suspect ls -i was actually lying inside guests!

At this point, I'm not exactly sure how to resolve my issue apart from reinstalling the LXC guest on a separate block device (like lvm) which I think should resolve the issue. This blog post is simply to confirm the issue, and to gather feedback and potential solutions from smart people reading this. Shoot me a comment

Celebrating Cloud9ers is MEA's leading Amazon Consulting partner

It's been quite a while since I last blogged, I think it was over a year ago! I've been taking a break from blogging and focusing on Cloud Niners my baby startup that I co-founded with a bunch of life long friends.

While many things are interesting over here at Cloud Niners, this blog post is specifically to celebrate the fact that we've just been upgraded to become an Amazon Consulting Partner. I believe this makes us the first active middle-east Amazon AWS cloud consulting partner and certainly the best :) I do hear a lot from Amazon that we are the cloud leader in the middle-east! I am quite happy that we've achieved this milestone. I'm a big believer in cloud computing, and I simply enjoy helping our clients understand how the cloud (and Ubuntu server) can help them achieve more!

My team and I do custom cloud-application development, where we write top notch html5/CSS3/JS web applications that run and make use of the Amazon AWS scalable cloud infrastructure. We also do mobile applications backends and front-end (Android, iOS). We usually use the frankly excellent Ubuntu server, unless the customer forces us otherwise! Ubuntu server is simply the best OS for the cloud IMO. Our DevOps team helps our customers migrate and run their apps on the cloud (and on Ubuntu server) at top quality!

If you've read so far, here's a Gift for you! I'm offering my (and my team's) help (up-to-24hrs) for free in the next 30-days. If you're thinking about coding a custom cloud app, or have questions about any Amazon services, I'll give you free consultation and answer any questions. Shoot me a comment over here, or contact the team. Oh yeah, feel free to share this blog post with friends, to let them have the free goodies!

Ensemble deploy and scale cloud apps

Think deploying and scaling your cloud application is hard ? Think again!



Check out this video demo, where I deploy a multi-tiered cloud application. I'm deploying

• HAproxy load balancer
• Memcached caching server
• MediaWiki application server
• MySQL DB

connecting them together and getting it working. After which, I'm scaling the whole thing from two application servers, to four

How long it takes me ? Well basically 5 minutes for the whole thing!

The secret: Ensemble !

Here's the video

Here is a direct link if you can't see the player

So what do you guys think, like the demo ? Leave me a comment, let me know which demo you'd like to see next. Also if you'd like to see your favorite application deployed with Ensemble, leave me a comment or ping me (kim0) on freenode irc

Cloud Portal one-click launch

Announcing two nice little additions to the cloud portal AMI tool that should make everyone's cloud life a little easier

• Amazon now allows passing parameters to the AWS console to basically be able to choose the region and AMI-ID to launch. This is now integrated in the cloud portal so that you search for the ami, click the link and it's pre-selected, ready to be launched in AWS console. Here's a screenshot

• As that image shows as well, you can search now for "cluster" to get Ubuntu cloud cluster-ready images!

Interested to add any features? Have any thoughts or comments, let me know, leave me a word

Free Ubuntu Server for a year at Amazon

Yes, you can get your very own Free Ubuntu server in the clouds for one full year! The folks at Amazon have just announced

"Beginning November 1, new AWS customers will be able to run a free Amazon EC2 Micro Instance for a year, while also leveraging a new free usage tier for Amazon S3, Amazon Elastic Block Store, Amazon Elastic Load Balancing, and AWS data transfer. AWS’s free usage tier can be used for anything you want to run in the cloud: launch new applications, test existing applications in the cloud, or simply gain hands-on experience with AWS"

You can find the details of the offer at: http://aws.amazon.com/free/

Ubuntu being arguably the most popular Amazon guest image is also available to you for free today! Get the list of Official Ubuntu images created by the Ubuntu's very own server team at

Maverick http://uec-images.ubuntu.com/server/maverick/current/
Lucid http://uec-images.ubuntu.com/server/lucid/current/

For the free offer, you will want to launch a t1.micro instance, and it seems you will want to wait till Nov 1st for registering your account (credit card needed). This is great news! If you ever wanted to play with Ubuntu server on the cloud, now is the best time to get started!

Update: The terms page mentions "Only accounts created after October 20, 2010 are eligible for the Offer"

Update2: Please read Scott's comment below, on why you will be charged 0.5$/mo if you run the standard Ubuntu image (or Amazon's own AMIs)