Friday, October 29, 2010

Ubuntu Cloud Community needs You

"I'm interested in Ubuntu and the cloud, how do I get involved" is a question I got a few times already. I thought it would be a good idea to answer this as a blog post. I believe one of the very first things you'd want to do, is to make sure you're on the main communication channels, talking to the community, asking questions, seeing other questions being answered, trying to answer some yourself, sharing opinions and generally "connecting" with the rest of the community. That is a great first step. So I'll highlight the main communication venues for the Ubuntu cloud community, as well as way to get kick-started.

Places to be
  • Ubuntu Cloud Forums, while pretty young, there has been some pretty good stir in the forums. While IRC and mailing lists may be more focused on "asking questions", the Forums are a great way to get in touch with other community members. To share your experience building your private clouds, the hardware used, software configuration, tuning and optimization, challenges faced ...etc. Come join in, if you would like to ask questions, or if you would like to share opinions, tips or tricks, get on the forums and make some splash :)
  • The Ubuntu-Cloud mailing list is a great technical resource where most of the experts and developers are subscribed. For very technical discussions, questions, feature suggestions, RFEs, development discussions the mailing list is a great resource.
  • The EC2Ubuntu mailing list is a great resource that focuses on running Ubuntu in the Amazon EC2 public cloud. This list is active with a wealth of info on the topic
  • IRC chat has long always been a primary real-time communication tool used by free software enthusiasts. The Ubuntu cloud IRC room is (surprise, surprise) #ubuntu-cloud on Freenode. Jump in, and engage
Once connected, things you can do include playing with the latest technology such as creating yourself a private UEC cloud, verifying latest features work as advertised, report and fix bugs, suggest features, design and implement new projects to advance the state of Ubuntu on the cloud. While the community is very welcoming, I definitely understand we need to create better new-comer friendly engagement paths, more hand-holding if you will. A better mentoring program from senior members as well as low hanging fruit are things the Ubuntu cloud and server communities need to identify and improve to make it easier to attract and engage fresh talent

Wednesday, October 27, 2010

PointnClick guide to running Ubuntu in the cloud

My previous post about running uec on ec2, drew some comments as being a fairly complex process. That may be true because essentially you're attempting to hack together a configuration which is not entirely supported. Anyway, that led me to want to demo a "point-n-clink" guide to running your first Ubuntu server in the EC2 cloud. i.e. no command line allowed, just point and click :) It doesn't get any easier than this, so let's hit it

Assuming you have setup an account with Amazon and could login to the Amazon console, you should see
pointnclick-guide-ubuntinthecloud-1

Before we begin, let's set ourselves a "key-pair". Which is basically a ssh public/private key-pair that enables you to login to your instance once launched. Click on "Key Pairs" and click "Create key pair", I'm gonna name my key "ubuntu" and click create

pointnclick-guide-ubuntinthecloud-2

The key is promptly created and pushed for download through your browser. Let's proceed with another "preparatory" setup. By default, EC2's firewall denies all inbound traffic to the instance, which means you cannot even ssh into your instance. Let's open ports 22 for ssh, and 80 just in case we wanna test by running an apache2 server. So, add your from-to ports and source IPs as per the next screenshot and "Save" it

pointnclick-guide-ubuntinthecloud-3

Great, now we're ready to actually launch an instance. Click "EC2 Dashboard" link, click the button "Launch Instance", a wizard starts asking for which AMI we would like to use. An AMI is a template that will be copied to your instance and used to start it. Click "Community AMIs", it may take a moment for the AMIs to load. Now here's a trick, to quickly "zoom-in" on the official ubuntu AMIs, use the following search string "ubuntu-images/" in order to locate EBS based images, and "ubuntu-images-us/" in order to locate instance store based images. This is in no way a "supported" feature of neither Amazon nor Ubuntu, it's just a convenient hack that works today and may not work tomorrow. We choose an EBS based AMI because we plan on launching a micro instance and those require EBS AMIs

pointnclick-guide-ubuntinthecloud-4

click the "Select" button beside it. Click continue and choose a t1.micro instance and click continue a couple of times

pointnclick-guide-ubuntinthecloud-5

pointnclick-guide-ubuntinthecloud-6

pointnclick-guide-ubuntinthecloud-7

You're now on the key-pair page, we simply choose our previously generated "Ubuntu" key pair and click continue

pointnclick-guide-ubuntinthecloud-8

On the firewall configuration page, we choose the "default" security group, since this is what we had configured to open ports 22 and 80 previously

pointnclick-guide-ubuntinthecloud-9

Voila, we're ready. Review and confirm the settings on the page and click "Launch"

pointnclick-guide-ubuntinthecloud-10

The cloud starts deploying your virtual server and you get the following message

pointnclick-guide-ubuntinthecloud-11

In a minute your instance should be up and running. Let's locate it and login to it. Clicking the "Instances" link, then clicking our only instance so far and locating its "Public DNS" entry setting allows us to ssh to it

pointnclick-guide-ubuntinthecloud-12

Now we're all set, let's jump to our terminal to ssh into the instance. It seems the "ubuntu.pem" key-pair the browser downloaded gets by default permissions that are too open for ssh's taste. Thus we need to "chmod" it to 700, this is the part where I lied about not using any CLI, but hey chmod doesn't really count ;) let's then ssh straight into our instance using the ubuntu.pem key-pair

pointnclick-guide-ubuntinthecloud-13

Awesome! We're in! let's do what the greeting message tells us to "sudo tasksel --section server" and choose to install a LAMP Server. We get asked for a MySQL password twice, the system installs and configures a full LAMP stack for us. Let's visit the apache web server

pointnclick-guide-ubuntinthecloud-14

That concludes this graphical guide. As you can see launching your first Ubuntu server instance in the cloud can't really get much easier than this :) Before you go for a cup of coffee, do not forget to "Terminate" your instance. If you don't, you keep on getting billed by the hour for it.

pointnclick-guide-ubuntinthecloud-15

Let me know in the comments if a step was unclear. Also, let me know if there are other topics you'd want me to demo. If you're interested in running Ubuntu in a cloud context and have any doubts, drop by on IRC channel #ubuntu-cloud on freenode, and grab me "kim0"

Friday, October 22, 2010

Cloud on Cloud, UEC on EC2

So you wanted to play with Ubuntu Enterprise Cloud (UEC), but didn't have a couple of machines to play with ? Want to start a UEC instance right now, no problem. You can use an Amazon EC2 server instance as your base server to install and run UEC on! Of course the EC2 instance is itself a virtual machine, thus running a VM inside that would require nested virtualization which AFAIK wouldn't work over EC2. The trick here is that we switch UEC's hypervisor temporarily to be qemu. Of course this won't win any performance competitions, in fact it'd be quite slow in production, but for playing, it fits the bill just fine.

If you're thinking doing all that is gonna be complex, you have a point, however it won't! In fact it'll be very easy due to efforts of Ubuntu's always awesome Scott Moser. Scott has written a script that automates all needed steps. But wait, it gets better, we're not even going to run this script ourselves, we're passing it as a parameter to the EC2 instance invocation and due to Ubuntu's cloud-init technology, that script is going to be run upon instance boot-up, doing its work automagically. Now let's get started

On your local machine, let's install bzr and get the needed script

cd /tmp
sudo apt-get install bzr -y
bzr branch lp:~smoser/+junk/uec-on-ec2
cd uec-on-ec2/

The file "maverick-commands.txt" contains the script needed to turn the generic Ubuntu image on ec2 into a fully operational single-node UEC install. If you don't have "ssh keys" (seriously?) generate some

ssh-keygen -t rsa

Now let's do a neat trick to import the keys into EC2, marking them with the name "default"

for r in us-east-1 us-west-1 ap-southeast-1 eu-west-1; do ec2-import-keypair --region $r default --public-key-file ~/.ssh/id_rsa.pub ; done

Let's open a few needed ports in EC2's default security group

for port in 22 80 8443 8773; do ec2-authorize default -p $port ; done

Very well .. We now launch our EC2 instance, passing in the "maverick commands" file. What happens is, the server instance is created, booted, Ubuntu's cloud-init reads up the maverick commands script we passed to it, and executes it, it starts downloading, installing and configuring UEC in the background while you ssh into your new EC2 instance

ec2-run-instances ami-688c7801 --instance-type m1.large -k default --user-data-file=maverick-commands.txt

Give it a minute to boot, then try to ssh in
ec2-describe-instances
ssh ubuntu@ec2-a-b-c-d.compute-1.amazonaws.com

Replace the DNS name for the EC2 instance, with the proper one you get from ec2-describe-instances. Once logged into the EC2 instance, I would start byobu and tail the log file to monitor progress. The whole thing takes less than 5 minutes

byobu
tailf uec-setup.log

The script once finished configuring UEC, actually downloads a tiny linux distro and registers its image in UEC, so that you can start your own instances! You know the script has finished when you see a line that looks like
emi="emi-FDC21818"; eri="eri-53721963"; eki="eki-740D19EC";

UEC is now up and running, let's check the web interface! You login with the default credentials admin/admin

uec-on-ec2-1

Let's navigate to the Images tab, to get the EMI ID (Equivalent of an AMI ID)

uec-on-ec2-2

Is that cool or what, Hell Yes! Now let's start our own VM inside UEC that's inside EC2. Remeber to replace emi-FE03181A with the EMI ID you got from the web interface

euca-run-instances --key mykey --addressing private emi-FE03181A

You can use "euca-describe-instances" to get the new instance internal IP address and ssh to that

ubuntu@domU-12-31-38-01-85-91:~$ ssh -i euca/mykey.pem ubuntu@172.19.1.2
The authenticity of host '172.19.1.2 (172.19.1.2)' can't be established.
RSA key fingerprint is db:9b:47:a4:06:81:26:d7:cf:38:a4:0e:6c:05:54:0d.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '172.19.1.2' (RSA) to the list of known hosts.

Chop wood, carry water.

$ uname -r
2.6.35-16-virtual
$ df -h                                                                                                                                                                                                                                     
Filesystem                Size      Used Available Use% Mounted on
/dev/sda1                23.2M     14.1M      7.9M  64% /
tmpfs                    24.0K         0     24.0K   0% /dev/shm

et voila, you're ssh'ed into a ttylinux instance running inside qemu managed by UEC running over EC2 :) If you do find that cool, what about contributing back. How about you start hacking on that script to make it even more awesome, such as maybe installing over multiple nodes or whatever crazy idea you can think of! If you're interested to start hacking, drop me a hi in #ubuntu-cloud at Freenode

Free Ubuntu Server for a year at Amazon

Yes, you can get your very own Free Ubuntu server in the clouds for one full year! The folks at Amazon have just announced

"Beginning November 1, new AWS customers will be able to run a free Amazon EC2 Micro Instance for a year, while also leveraging a new free usage tier for Amazon S3, Amazon Elastic Block Store, Amazon Elastic Load Balancing, and AWS data transfer. AWS’s free usage tier can be used for anything you want to run in the cloud: launch new applications, test existing applications in the cloud, or simply gain hands-on experience with AWS"

You can find the details of the offer at: http://aws.amazon.com/free/

Ubuntu being arguably the most popular Amazon guest image is also available to you for free today! Get the list of Official Ubuntu images created by the Ubuntu's very own server team at

Maverick http://uec-images.ubuntu.com/server/maverick/current/
Lucid http://uec-images.ubuntu.com/server/lucid/current/

For the free offer, you will want to launch a t1.micro instance, and it seems you will want to wait till Nov 1st for registering your account (credit card needed). This is great news! If you ever wanted to play with Ubuntu server on the cloud, now is the best time to get started!

Update: The terms page mentions "Only accounts created after October 20, 2010 are eligible for the Offer"

Update2: Please read Scott's comment below, on why you will be charged 0.5$/mo if you run the standard Ubuntu image (or Amazon's own AMIs)

Hello Planet-Ubuntu

Hello World! I just became an Ubuntu member yesterday, which means I will no longer be on mute ;) I'm working as the Ubuntu Cloud community liaison, which means I finally bring some cloud and server content to this space instead of all being around fluffy desktops :) If you're interested in using or contributing to Ubuntu in a "cloud" context, feel free to grab me (kim0 on Freenode #ubuntu-cloud). Feel free to drop in, say hi, ask questions or whatever you need. Besides this blog, you may also be interested in my tweets. I'm very proud being part of this great community. Rock on!

Friday, October 15, 2010

OpenWeek Introduction to Cloud IRC session logs

If you couldn't attend the Ubuntu Open Week session live on IRC, don't despair! You can read the logs at
http://irclogs.ubuntu.com/2010/10/13/%23ubuntu-classroom.html#t17:01

Monday, September 20, 2010

what I do

I started with Linux and Open source software around 1999, since then and till today I was amazed about the helpful community that form around different open source projects. In the early days, whenever I'd hit a bug or don't know how to do something, I'd jump over to IRC and someone was always there to help debug the problem! This just felt amazingly empowering, I remember thinking to myself, hell this is way much better than what you get with commercial software! In a few years, I transformed from being a passive user, into someone who tries to help IRC users every time someone has a problem. The culture of open source got me I guess. I started giving presentations at Universities, writing for local IT magazines for free to help spread what FOSS is all about. I always wanted to contribute much more to the free software world, however limited free time available was always an issue. Now that my day job is to work with and help grow the community around Ubuntu, I feel extremely excited and thankful for Canonical for giving me this opportunity.

Being the newest member of the "horsemen" team having joined a little over a month now, I feel like I haven't done enough yet. Nevertheless, I'd like to mention a few of the things that have been keeping me busy the past few weeks

  • The very first thing I had done with Canonical was to write the Ubuntu Server Map application. The idea behind it is to encourage and spur the community behind Ubuntu server. Basically the aim is to make Ubuntu server users feel part of a huge user community all over the world. What the application does is to detect the visitor user's city using his IP address after the user accepts (anonymous process) and then marks that city with a cute little orange Ubuntu logo. So far the Map is full of orange Ubuntu logos. It really feels great to be part of such a world wide community of Ubuntu server and cloud users and contributors

  • As part of helping the Ubuntu cloud community grow around open source cloud technologies, I have focused on consolidating any fragmented communication paths available. Thus far, the Ubuntu cloud community has the following #ubuntu-cloud as the official IRC channel for everything Ubuntu cloud related. And the recently created Ubuntu Cloud Forum as the official Ubuntu cloud forum. The Ubuntu Cloud mailing list is an alternate community communication venue as well

  • Recently I've been putting a lot of focus on creating a Ubuntu cloud portal, which aims to be a central hub for the Ubuntu cloud community. You can read all about the portal specs and give me feedback over IRC (kim0 in #ubuntu-cloud). The portal should provide a list of rolling news that relate to Ubuntu cloud, helping interested community members always be on top of all the new happenings. It also helps community new comers by becoming their one stop shop with links to all documentation and support channels. As well as guide new contributers on how to get involved

  • Something which I am thinking about and which will definitely take a lot of focus soon is studying potential hurdles in the way of new contributers to Ubuntu server and cloud. Basically how to make it easy and more fun for newcomers to join in, find all the information they need in place and start contributing and engaging with the community. Part of that is giving tutorials over IRC or other mediums as well as sponsorship and guidance along the way

It is definitely great being part of such a great community such as the Ubuntu one, and I hope the next period is going to be very exciting for the open source Cloud communities in general