PointnClick guide to running Ubuntu in the cloud

My previous post about running uec on ec2, drew some comments as being a fairly complex process. That may be true because essentially you're attempting to hack together a configuration which is not entirely supported. Anyway, that led me to want to demo a "point-n-clink" guide to running your first Ubuntu server in the EC2 cloud. i.e. no command line allowed, just point and click :) It doesn't get any easier than this, so let's hit it

Assuming you have setup an account with Amazon and could login to the Amazon console, you should see


Before we begin, let's set ourselves a "key-pair". Which is basically a ssh public/private key-pair that enables you to login to your instance once launched. Click on "Key Pairs" and click "Create key pair", I'm gonna name my key "ubuntu" and click create



The key is promptly created and pushed for download through your browser. Let's proceed with another "preparatory" setup. By default, EC2's firewall denies all inbound traffic to the instance, which means you cannot even ssh into your instance. Let's open ports 22 for ssh, and 80 just in case we wanna test by running an apache2 server. So, add your from-to ports and source IPs as per the next screenshot and "Save" it



Great, now we're ready to actually launch an instance. Click "EC2 Dashboard" link, click the button "Launch Instance", a wizard starts asking for which AMI we would like to use. An AMI is a template that will be copied to your instance and used to start it. Click "Community AMIs", it may take a moment for the AMIs to load. Now here's a trick, to quickly "zoom-in" on the official ubuntu AMIs, use the following search string "ubuntu-images/" in order to locate EBS based images, and "ubuntu-images-us/" in order to locate instance store based images. This is in no way a "supported" feature of neither Amazon nor Ubuntu, it's just a convenient hack that works today and may not work tomorrow. We choose an EBS based AMI because we plan on launching a micro instance and those require EBS AMIs



click the "Select" button beside it. Click continue and choose a t1.micro instance and click continue a couple of times







You're now on the key-pair page, we simply choose our previously generated "Ubuntu" key pair and click continue



On the firewall configuration page, we choose the "default" security group, since this is what we had configured to open ports 22 and 80 previously



Voila, we're ready. Review and confirm the settings on the page and click "Launch"



The cloud starts deploying your virtual server and you get the following message



In a minute your instance should be up and running. Let's locate it and login to it. Clicking the "Instances" link, then clicking our only instance so far and locating its "Public DNS" entry setting allows us to ssh to it



Now we're all set, let's jump to our terminal to ssh into the instance. It seems the "ubuntu.pem" key-pair the browser downloaded gets by default permissions that are too open for ssh's taste. Thus we need to "chmod" it to 700, this is the part where I lied about not using any CLI, but hey chmod doesn't really count ;) let's then ssh straight into our instance using the ubuntu.pem key-pair



Awesome! We're in! let's do what the greeting message tells us to "sudo tasksel --section server" and choose to install a LAMP Server. We get asked for a MySQL password twice, the system installs and configures a full LAMP stack for us. Let's visit the apache web server



That concludes this graphical guide. As you can see launching your first Ubuntu server instance in the cloud can't really get much easier than this :) Before you go for a cup of coffee, do not forget to "Terminate" your instance. If you don't, you keep on getting billed by the hour for it.



Let me know in the comments if a step was unclear. Also, let me know if there are other topics you'd want me to demo. If you're interested in running Ubuntu in a cloud context and have any doubts, drop by on IRC channel #ubuntu-cloud on freenode, and grab me "kim0"

Cloud on Cloud, UEC on EC2

So you wanted to play with Ubuntu Enterprise Cloud (UEC), but didn't have a couple of machines to play with ? Want to start a UEC instance right now, no problem. You can use an Amazon EC2 server instance as your base server to install and run UEC on! Of course the EC2 instance is itself a virtual machine, thus running a VM inside that would require nested virtualization which AFAIK wouldn't work over EC2. The trick here is that we switch UEC's hypervisor temporarily to be qemu. Of course this won't win any performance competitions, in fact it'd be quite slow in production, but for playing, it fits the bill just fine.

If you're thinking doing all that is gonna be complex, you have a point, however it won't! In fact it'll be very easy due to efforts of Ubuntu's always awesome Scott Moser. Scott has written a script that automates all needed steps. But wait, it gets better, we're not even going to run this script ourselves, we're passing it as a parameter to the EC2 instance invocation and due to Ubuntu's cloud-init technology, that script is going to be run upon instance boot-up, doing its work automagically. Now let's get started

On your local machine, let's install bzr and get the needed script

cd /tmp
sudo apt-get install bzr -y
bzr branch lp:~smoser/+junk/uec-on-ec2
cd uec-on-ec2/

The file "maverick-commands.txt" contains the script needed to turn the generic Ubuntu image on ec2 into a fully operational single-node UEC install. If you don't have "ssh keys" (seriously?) generate some

ssh-keygen -t rsa

Now let's do a neat trick to import the keys into EC2, marking them with the name "default"

for r in us-east-1 us-west-1 ap-southeast-1 eu-west-1; do ec2-import-keypair --region $r default --public-key-file ~/.ssh/id_rsa.pub ; done

Let's open a few needed ports in EC2's default security group

for port in 22 80 8443 8773; do ec2-authorize default -p $port ; done

Very well .. We now launch our EC2 instance, passing in the "maverick commands" file. What happens is, the server instance is created, booted, Ubuntu's cloud-init reads up the maverick commands script we passed to it, and executes it, it starts downloading, installing and configuring UEC in the background while you ssh into your new EC2 instance

ec2-run-instances ami-688c7801 --instance-type m1.large -k default --user-data-file=maverick-commands.txt

Give it a minute to boot, then try to ssh in

ec2-describe-instances
ssh ubuntu@ec2-a-b-c-d.compute-1.amazonaws.com

Replace the DNS name for the EC2 instance, with the proper one you get from ec2-describe-instances. Once logged into the EC2 instance, I would start byobu and tail the log file to monitor progress. The whole thing takes less than 5 minutes

byobu
tailf uec-setup.log

The script once finished configuring UEC, actually downloads a tiny linux distro and registers its image in UEC, so that you can start your own instances! You know the script has finished when you see a line that looks like

emi="emi-FDC21818"; eri="eri-53721963"; eki="eki-740D19EC";

UEC is now up and running, let's check the web interface! You login with the default credentials admin/admin



Let's navigate to the Images tab, to get the EMI ID (Equivalent of an AMI ID)



Is that cool or what, Hell Yes! Now let's start our own VM inside UEC that's inside EC2. Remeber to replace emi-FE03181A with the EMI ID you got from the web interface

euca-run-instances --key mykey --addressing private emi-FE03181A

You can use "euca-describe-instances" to get the new instance internal IP address and ssh to that

ubuntu@domU-12-31-38-01-85-91:~$ ssh -i euca/mykey.pem ubuntu@172.19.1.2
The authenticity of host '172.19.1.2 (172.19.1.2)' can't be established.
RSA key fingerprint is db:9b:47:a4:06:81:26:d7:cf:38:a4:0e:6c:05:54:0d.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '172.19.1.2' (RSA) to the list of known hosts.

Chop wood, carry water.

$ uname -r
2.6.35-16-virtual
$ df -h                                                                                                                                                                                                                                     
Filesystem                Size      Used Available Use% Mounted on
/dev/sda1                23.2M     14.1M      7.9M  64% /
tmpfs                    24.0K         0     24.0K   0% /dev/shm

et voila, you're ssh'ed into a ttylinux instance running inside qemu managed by UEC running over EC2 :) If you do find that cool, what about contributing back. How about you start hacking on that script to make it even more awesome, such as maybe installing over multiple nodes or whatever crazy idea you can think of! If you're interested to start hacking, drop me a hi in #ubuntu-cloud at Freenode

Free Ubuntu Server for a year at Amazon

Yes, you can get your very own Free Ubuntu server in the clouds for one full year! The folks at Amazon have just announced

"Beginning November 1, new AWS customers will be able to run a free Amazon EC2 Micro Instance for a year, while also leveraging a new free usage tier for Amazon S3, Amazon Elastic Block Store, Amazon Elastic Load Balancing, and AWS data transfer. AWS’s free usage tier can be used for anything you want to run in the cloud: launch new applications, test existing applications in the cloud, or simply gain hands-on experience with AWS"

You can find the details of the offer at: http://aws.amazon.com/free/

Ubuntu being arguably the most popular Amazon guest image is also available to you for free today! Get the list of Official Ubuntu images created by the Ubuntu's very own server team at

Maverick http://uec-images.ubuntu.com/server/maverick/current/
Lucid http://uec-images.ubuntu.com/server/lucid/current/

For the free offer, you will want to launch a t1.micro instance, and it seems you will want to wait till Nov 1st for registering your account (credit card needed). This is great news! If you ever wanted to play with Ubuntu server on the cloud, now is the best time to get started!

Update: The terms page mentions "Only accounts created after October 20, 2010 are eligible for the Offer"

Update2: Please read Scott's comment below, on why you will be charged 0.5$/mo if you run the standard Ubuntu image (or Amazon's own AMIs)

Hello Planet-Ubuntu

Hello World! I just became an Ubuntu member yesterday, which means I will no longer be on mute ;) I'm working as the Ubuntu Cloud community liaison, which means I finally bring some cloud and server content to this space instead of all being around fluffy desktops :) If you're interested in using or contributing to Ubuntu in a "cloud" context, feel free to grab me (kim0 on Freenode #ubuntu-cloud). Feel free to drop in, say hi, ask questions or whatever you need. Besides this blog, you may also be interested in my tweets. I'm very proud being part of this great community. Rock on!

OpenWeek Introduction to Cloud IRC session logs

If you couldn't attend the Ubuntu Open Week session live on IRC, don't despair! You can read the logs at
http://irclogs.ubuntu.com/2010/10/13/%23ubuntu-classroom.html#t17:01

what I do

I started with Linux and Open source software around 1999, since then and till today I was amazed about the helpful community that form around different open source projects. In the early days, whenever I'd hit a bug or don't know how to do something, I'd jump over to IRC and someone was always there to help debug the problem! This just felt amazingly empowering, I remember thinking to myself, hell this is way much better than what you get with commercial software! In a few years, I transformed from being a passive user, into someone who tries to help IRC users every time someone has a problem. The culture of open source got me I guess. I started giving presentations at Universities, writing for local IT magazines for free to help spread what FOSS is all about. I always wanted to contribute much more to the free software world, however limited free time available was always an issue. Now that my day job is to work with and help grow the community around Ubuntu, I feel extremely excited and thankful for Canonical for giving me this opportunity.

Being the newest member of the "horsemen" team having joined a little over a month now, I feel like I haven't done enough yet. Nevertheless, I'd like to mention a few of the things that have been keeping me busy the past few weeks

• The very first thing I had done with Canonical was to write the Ubuntu Server Map application. The idea behind it is to encourage and spur the community behind Ubuntu server. Basically the aim is to make Ubuntu server users feel part of a huge user community all over the world. What the application does is to detect the visitor user's city using his IP address after the user accepts (anonymous process) and then marks that city with a cute little orange Ubuntu logo. So far the Map is full of orange Ubuntu logos. It really feels great to be part of such a world wide community of Ubuntu server and cloud users and contributors

• As part of helping the Ubuntu cloud community grow around open source cloud technologies, I have focused on consolidating any fragmented communication paths available. Thus far, the Ubuntu cloud community has the following #ubuntu-cloud as the official IRC channel for everything Ubuntu cloud related. And the recently created Ubuntu Cloud Forum as the official Ubuntu cloud forum. The Ubuntu Cloud mailing list is an alternate community communication venue as well

• Recently I've been putting a lot of focus on creating a Ubuntu cloud portal, which aims to be a central hub for the Ubuntu cloud community. You can read all about the portal specs and give me feedback over IRC (kim0 in #ubuntu-cloud). The portal should provide a list of rolling news that relate to Ubuntu cloud, helping interested community members always be on top of all the new happenings. It also helps community new comers by becoming their one stop shop with links to all documentation and support channels. As well as guide new contributers on how to get involved

• Something which I am thinking about and which will definitely take a lot of focus soon is studying potential hurdles in the way of new contributers to Ubuntu server and cloud. Basically how to make it easy and more fun for newcomers to join in, find all the information they need in place and start contributing and engaging with the community. Part of that is giving tutorials over IRC or other mediums as well as sponsorship and guidance along the way

It is definitely great being part of such a great community such as the Ubuntu one, and I hope the next period is going to be very exciting for the open source Cloud communities in general

Why our Internet2.0 is broken

The modern Internet which I'll refer to as Internet2.0 is being seen as a new applications platform. It is no longer a series of "pages" that you click through, it is rather a collection of applications. I just needed this intro in case you still thought of the Internet as pages. So the Internet is now the Operating System, and different web sites (Gmail, Facebook, Twitter...etc) are the new applications if you will. I've got news for you, this Internet2.0 thing, is horribly broken! Here is why

To begin understanding the kind of problems we have to go through using the online systems of today, let us apply the same online mechanisms, to standard old and boring desktop apps. Let's imagine the following workflow, You are 3 friends working on an important report, and you all share your progress online, each person via his blog

• You login to your computer, you start your email application. You find your friend has edited the report, emailed you the new copy and blogged about his progress


• You check your email, find the attachment, you download that


• You must copy the attachment over a USB stick in order to be able to open it in any other application. You copy it to the USB stick


• You open your word processor. You authenticate to it!


• You plugin your USB stick, open the document, edit it, save it


• Re-copy it to the USB stick


• Re-open your email application, upload the attachment, send it


• You visit your first friend's blog, you add a comment that you updated the report.


• Your third friend is not notified of your comment

You get the idea. If that user experience sounds horrible, it is pretty similar to what we face today with web applications especially if you want to use different applications from different providers in concert. It is just plain broken. Here is my criticizm spot on

• Why do I have to login separately to each and every web application (Google, Facebook, Zoho, Twitter, MS...)? On my Linux PC I don't have to go through that


• Why do I have to teach each web application my social graph. Reconnect to all my friends on facebook, then twitter, then Google Buzz...? The connectivity between me and my friend belongs to us, it does not belong to Facebook. Any other app on the Internet which I allow to access this data, should be able to. It should not be held captive by the likes of Facebook


• Assuming I love to use a live.com email address (which I don't :) and like to use Google docs to edit my email attachments. Why do I have to download the attachment to my PC first, reupload to Google to edit, save, download, upload, email, yikes! The Unix architecture designed 10s of years ago, was all about sharing data between apps (pipes), why in this modern age are we unable to easily connect different webapps especially from different providers


• When I write a comment on my friend's blog, the comment is MINE. I created it, I own it. My friend's blog maybe displaying it currently, but it should not own it! If my friends are interested in seeing my comments on every website I visit, and if I allow them to, they should be able to do so very easily. The 20 comments I've written today, should not die if the websites I have written them onto decide to die


• The same actually goes for using online editors a la Google docs. Google should not "own" the document. The document is mine. It should be stored in a place I control. I may "Allow" Google web based editor to read/write it now, because I "choose" to. Because it maybe the best editor around. Not because I have to, and not because I need to "migrate" my data off of Google should I want to use something else. If I decide to use Zoho editor tomorrow, I should be able to allow it to access my data in-place. Just like how on the desktop one can use MS Office, OpenOffice and Mac Pages to open a presentation on one's desktop

As a new application pops up, say like Apple's new Ping service, you need to (again) rebuild your social graph teaching it all your friends. Afterwards any music you purchase will show on Ping, but won't really show up on Facebook because Apple and Facebook couldn't agree on that. Can you say that again, "I" purchased a music track, and I want to tell "MY" friends about it, and I can't because Apple and Facebook couldn't agree! Can you see how horribly broken our Internet2.0 is

How I see things could improve is as follows. Each user needs to own a certain web exposed storage space somewhere. All of my online trails (documents, friendship, Likes, comments, blogs...) should live there. I should "allow" Google docs to read/write to those document if I like it. Tomorrow I could revoke that access and allow Zoho for example or any other online editor. My connections to my friends should be stored in that storage space. Any application that I allow can see who all or some of my friends are. For example I can allow Linkedin to access my work friends, while allowing Ping to access my music friends. If I make a comment somewhere on the Internet, this is content I created, If I Facebook style "Like" something, it is again content that revolves around me. It should be stored in my storage area, and Facebook should be notified of it and allowed to display it should I want to. This architecture makes it quite easy for a new young startup to create a Facebook or Gmail killer tomorrow. Since the application from day one will have access to all my data, emails, friends, comments, blogs...etc i.e. as much content as I want to give it. We become no longer locked into different online service providers. We can switch at will. This is the way how it can and should be done. The diaspora and FreedomBox folks are doing some awesome work and designing their systems along similar lines of thought. They are however more ambitious, and are after user data confidentiality, while all I'm asking for is open-data access and portability. They want to replace all of today's web applications with distributed clones that you can run inside your home. That would be fantastic, however I still it as advantageous and easier if a user can use "closed" cloud apps like Gmail or Facebook should she want to, and switch to a different provider at will, while still owning her online digital trail and every piece of content she ever created