Magical apt-clone, broken system recovery

I have received some very nice feedback regarding my last blog on D-Bus, thanks guys. This week I'm going to play once more with my Nexenta installation that we completed in part1. This time around I am going to basically demo the integration the Nexenta folks have pulled mixing apt-get with ZFS file-system capabilities, resulting in transactional system upgrades, with multi-snapshots that makes system recovery a snap!

We start off booting our nexenta NCP2-alpha installation, which we'll be upgrading to beta-1 as we speak. Grub shows the default boot kernel



Once booted, and we log in, let's update apt's cache



I have also used the command "apt-clone" to list the currently available boot environments. Basically, using ZFS snapshots the whole machine state is snapshot'ed resulting in a instantly bootable environment. Currently we only have the default environment "rootfs-nmu-000". Now I am going to use apt-clone to "dist-upgrade" the whole system. If this is new to you (who has never seen Ubuntu :) it basically means you're upgrading the whole system to the latest tasty bits



Note that I have used verbose mode "-v" in order to get more information. After a lot of downloading and dpkg grinding, I now have the latest packages installed. apt-clone has created a new system snapshot "001" and the process ends like so



I have simply hit 'n' to discard any automatic changes, then I used apt-clone to activate the upgraded snapshot as instructed by the on-screen output



Now let's reboot and see what kind of coolness we have



Oh cool, Grub now displays two bootable environments! One before the upgrades and one after. The dates besides each entry make that clear. Great, now let's reboot into the new envrionment and list the available clones (cloned file systems)



Note that the "A" and "C" in the output mean "Active" and "Current". Great, now creating instant bootable system snapshots like that is so useful that it does not relate to software installations only. Say you're running some third party closed source installation application and you're not really sure what's it going to do to your system. Or if the new clueless administrator needs to install something to the server but you think he's probably going to destroy it! In such situations, it's very helpful to manually create a system snapshot. So, let's do just that to create the environment 002 and let's activate it



Now we have our own little safety net. Let's now boot into that new environment. Use apt-clone to make sure "002" is the active and current environment. Then's let's simulate complete system failure. I will not be too creative and will simply delete the kernel and reboot!



Once in Grub, boot the new and destroyed system. As expected Grub errors out and we're staring at



So right now we have a broken system. Now do we run to tape, dump | restore, and use some black magic, nope, we simply reboot and choose our safe snapshot 001, boot into that and destroy the corrupt environment 002 and activate the safe one 001 as so



Perfect! Once rebooted, our system is back fully functional. Mind you that taking system snapshots, destroying corrupt ones all take a couple of seconds, and of course does not consume disk space except for the changed disk blocks. Now that we have our server back, let's do something useful with it. Let's "apt-clone install apache2".



Once done, the SMF could not start apache because I had chosen nwam network auto-configuration (think NetworkManager), which apache2 does not like. Anyway, I simply started the SysV script manually, and verified "it works" with a direct telnet connection. You can basically start installing and configuring various server software mostly just like an Ubuntu system. All in all, the Nexenta-2 system is a very promising concept. The integration of advanced Solaris kernel features with GNU userland and attempting to port Ubuntu repos, would basically be paradise. It is however a lot of work. The system is not perfect yet, for example in this beta-1, you cannot install X yet (broken deps). That will surely be resolved by release time. However, rebuilding the huge debian/Ubuntu repos, needs a lot of work. If you have free time to push this interesting FOSS project forward, please do so. Visit nexenta and join their development team. Meanwhile and separately from nexenta, I have been interacting with various open source developers working with opensolaris projects. Unfortunately it seems that Sun is not really making it easy for "outsiders" to work on opensolaris projects. I have heard such comments multiple times, from different people. One would think that after so much time, Sun would start to understand what it takes to build a community around any piece of software! Let me use the moment to shout out: Sun, please cut the community some slack! :)

Linus explains how Linux started

Nice vid

Ride the D-Bus, Control your Linux desktop from the shell

What is the D-Bus?

From the D-Bus documentation: "D-Bus is an inter-process communication mechanism—a medium for local communication between processes running on the same host. (Inter-host connects may be added in the future, but that is not what D-Bus is meant for). D-Bus is meant to be fast and lightweight, and is designed for use as a unified middleware layer underneath the main free desktop environments"

So for the non technically inclined (why are you reading this anyway) ;) basically a simple way to think of D-Bus, is that it's a way for processes on an operating system to communicate with each other. That's probably a simple and non accurate enough wording, but it should help grasp the concept. Many utilities nowadays are implementing D-Bus connectivity. It is especially interesting (well for me anyway) to script some GUI elements like for example my KDE-4 desktop from the command line. This would help automate some tasks and is cool anyways. Let's see how to begin talking on the D-Bus

In order to communicate with running applications on the D-Bus, we need a front end CLI application, or we could use a language binding (say python bindings). However, we will choose a CLI application. It is possible to use "dbus-launch", or the "qdbus" command part of Qt4. I will be using qdbus since it demonstrates the point and is faster to use. So, let's launch qdbus, and list the running communications buses.

# qdbus
:1.1
org.kde.klauncher
:1.10
org.freedesktop.ScreenSaver
org.kde.krunner
org.kde.screensaver
:1.12
org.kde.plasma
:1.1249
... List continues

So basically, qdbus now contacts the running session D-Bus and lists available buses. The buses are named in a reverse DNS style names as you can see. Some buses do not have names, but rather numbers, as far as I can see those represent specific sessions of some applications. Now we need to pick an interesting bus to talk on. I will run the following command to list only freedesktop dbuses, as I believe those should work whether you're running Gnome or KDE. So, it should help any reader follow along

#qdbus | grep freedesktop
org.freedesktop.ScreenSaver
org.freedesktop.Notifications
org.freedesktop.PowerManagement
org.freedesktop.DBus

so, let's pick the ScreenSaver bus. In order to list the available objects on any specific bus, we call it like so:

#qdbus org.freedesktop.ScreenSaver
/
/App
/Interface
/KBookmarkManager
/KBookmarkManager/konqueror
/KDebug
/MainApplication
/ManagerIface_contact
/ScreenSaver

I mostly find only one "useful" object, in this case "/ScreenSaver". Let's contact that object, and list its avialable "methods". The methods of an object for those who have not done any object oriented coding before, is basically the list of actions or "things" this object can "do". So listing the methods:

# qdbus org.freedesktop.ScreenSaver /ScreenSaver
signal void org.freedesktop.ScreenSaver.ActiveChanged(bool)
method bool org.freedesktop.ScreenSaver.GetActive()
method uint org.freedesktop.ScreenSaver.GetActiveTime()
method uint org.freedesktop.ScreenSaver.GetSessionIdleTime()
method uint org.freedesktop.ScreenSaver.Inhibit(QString application_name, QString reason_for_inhibit)
method void org.freedesktop.ScreenSaver.Lock()
method bool org.freedesktop.ScreenSaver.SetActive(bool e)
method void org.freedesktop.ScreenSaver.SimulateUserActivity()
method uint org.freedesktop.ScreenSaver.Throttle(QString application_name, QString reason_for_inhibit)
method void org.freedesktop.ScreenSaver.UnInhibit(uint cookie)
method void org.freedesktop.ScreenSaver.UnThrottle(uint cookie)
method void org.kde.screensaver.configure()
method void org.kde.screensaver.saverLockReady()
method QDBusVariant org.freedesktop.DBus.Properties.Get(QString interface_name, QString property_name)
method void org.freedesktop.DBus.Properties.Set(QString interface_name, QString property_name, QDBusVariant value)
method QString org.freedesktop.DBus.Introspectable.Introspect()

Hmm, interesting stuff. Can you see the following method:
org.freedesktop.ScreenSaver.GetSessionIdleTime
Obviously it gives you the session idle time, i.e. how long has the interactive user not touched his keyboard or mouse. This piece of information is interesting if you want your shell script to only do certain things when the user is not interactively using his machine. Without D-Bus, getting this kind of information would be almost impossible, or too tricky. Now how do we call this method you ask? easy! just append it to the command line:

#qdbus org.freedesktop.ScreenSaver /ScreenSaver org.freedesktop.ScreenSaver.GetSessionIdleTime
0

So, the output from this command is a "0". Meaning the session idle time is zero! Everytime you execute this command, you get the same answer. Starting to guess why ? :) Basically, as you hit "enter" to run this command, you reset the session idle time back to zero! So, what do we do to test the functionality, try this:

# sleep 5 ; qdbus org.freedesktop.ScreenSaver /ScreenSaver org.freedesktop.ScreenSaver.GetSessionIdleTime
4

So, that previous combo command sleeps for 5 seconds .. then executes the D-Bus query without touching any further keyboard keys, so this time we get an answer of "4" seconds. Don't know where that extra second slipped though :)

Ok, as a second example let's focus on the following method:
method bool org.freedesktop.ScreenSaver.SetActive(bool e)

This method sets the screen saver active, i.e. it launches the screen saver from your shell command or script. Let's test it out:

# qdbus org.freedesktop.ScreenSaver /ScreenSaver org.freedesktop.ScreenSaver.SetActive
Error: org.freedesktop.DBus.Error.UnknownMethod
No such method 'SetActive' in interface 'org.freedesktop.ScreenSaver' at object path '/ScreenSaver' (signature '')

Oops, this time we got an error. Why? Basically the error is telling us that it cannot find that method we're calling with that "signature". A method signature usually means the arguments you pass to it and the return type. This simply means we're not calling the method the way it's meant to be called. The sharp shooters are going to instantly know this is because we missed the (bool e) at the end of the method. This means the method call is expecting a boolean (true/false) argument. So, let's call the screensaver method correctly this time:

# qdbus org.freedesktop.ScreenSaver /ScreenSaver org.freedesktop.ScreenSaver.SetActive True
true

Cool! We're now launching the screensaver from the command line. Let's see what else we can do:

# qdbus org.freedesktop.ScreenSaver /ScreenSaver org.freedesktop.ScreenSaver.Lock

Yep, that locks the screen on demand, cool! I'm hungry for more. Let's see. Let's call that inhibit method. This will basically inhibit the screensaver from kicking in, even if your computer is idle. Why would that be useful ? Say you're watching a You-tube longish video, and AFAIK, Flash doesn't yet communicate that to the system, and thus the screensaver will kick in to interrupt the video playback. So, let's do it:

# qdbus org.freedesktop.ScreenSaver /ScreenSaver org.freedesktop.ScreenSaver.Inhibit "$$" "Testing D-Bus Interface"
5822

Let's explain those arguments we passed, the first one should be the application name, in my case, the special shell variable $$ is the PID for the currently executing shell. The second string is the "reason for inhibit"! The method call returns a cookie, i.e. a magic number that identifies my request. This is useful when you want to turn off the inhibit, i.e. return the system to its normal state. You would need to pass back that magic cookie. This is used by the system to identify the different inhibit requests. So finally let's uninhibit our screensaver using that cookie number

# qdbus org.freedesktop.ScreenSaver /ScreenSaver org.freedesktop.ScreenSaver.UnInhibit 5822

Cool! Are you starting to see the possibilities just yet? Hope you had fun on this D-Bus tour

ZFS a smash hit

Pretty freaking amazing

Nexenta, Can you say SolaBuntu (part1)

This time around I will be playing around with nexenta OS. Nexenta is a marriage between OpenSolaris and Ubuntu, or what I like to call SolaBuntu :) Solaris has a pretty decent record in the data-center. It is a solid and widely trusted paltform, however, it was showing its age pretty badly. Many of the most commonly used tools were outdated. And most users begin their *nixy knowledge with Linux, and when faced with Solaris they don't find the GNU tools they have grown accustomed to, and thus don't like the platform. OpenSolaris has been started to fix some of those issues, however, nexneta takes this concept to an extreme. It basically takes the ubuntu userland and plugs underneath it an opensolaris kernel. Nexenta also integrates unique solaris features such as zfs with ubuntu tools like apt-get to provide system wide transactional safe upgrades. I really really like this feature. It basically means you can dist-upgrade your entire system, and once up on the new version, if something is badly broken, you can very easily boot back an earlier snapshot (pre-upgrade) image of your system. We will explore those features later, however, for now, let's get started installing NCP2 alpha. As usual I am choosing to go with the latest and greatest and least stable version ;)

Download the iso, burn it, boot it and you're faced with grub



As you can see, this is based on Ubuntu Hardy. Simply hit enter and the solaris kernel boots on your system. After the boot, you get a splash screen welcoming you to nexenta



Now I'm gonna throw a lot of images, just to give you a feeling of the installer, but I'm also gonna skip some pretty obvious screens that would be mostly obvious (a la are you sure ? Yes/No). Next you choose your country



Now here is an interesting tidbit, the whole installer is running inside a screen session. So, a CTRL-A followed by a 2, takes you to screen window 2, as you can see you can use the format command to view and partition your disks as needed. Window 3 is for logging the installer actions. The log is not very verbose however!



Afterwards, you get to choose your disk, then confirm it



Afterwards, the disk is zfs formatted, and package installation begins .. Go make the usual cup of nescafe



After this step completes (around 20 minutes), you get to specify the root password. "root" can be used for direct login (no need for sudo or pfexec here).



You also create a normal user, set its password. You specify the machine name "nexy" in my case, and domain name. Afterwards, boom the installation is complete. You're ready to boot your new environment



After the reboot, grub starts, load the opensolaris kernel, and a few seconds later, you get the default login screen, which is not a pretty GUI! The default installation is a minimal installation without X.



Once logged in, you can start enjoying apt which is IMO the best main stream package manager. Let update the repos



Let's see how good the repos are. apt-cache show the system has access to around 4700 packages. This is of course not comparable to debian or ubuntu's repos, however, it still is fairly impressive considering how young the project is. If you're feeling excited, one of the best ways you can help is by becoming a packager for nexenta, so go ahead and join their hackathon.



One of the things that make nexenta an attractive opensolaris based system, is that it provides a solid array of open-source software tools that are up2date! This is pretty important at least to me. Let's check a couple of common packages



Your system is now ready, you can apt-get install any software package that's available and start running your nexenta based server. I have to say that my overall feeling is that nexenta core platform is more oritented towards being a server OS more so than a desktop one. Not that you can't install desktop apps, but rather that's what the default install offers, and that's what the core dev team seems to be focused on. NCP-2 alpha is definitely promising, and I really like the idea of an ubuntu/debian/GNU system on a solaris kernel. However, the project is very young, and I did face some bugs along the way, the is to be expected however as this release is still alpha quality. Hopefully along the upcoming posts, I will take that system for a ride and blog about it. Let me know about your experiences as well.

Update: Read part2

OpenSolaris 2008.11 pre-release, The year of the laptop?!

I've always been a Unix junkie, of course mostly running Linux for a decade now, but Solaris is starting to look more and more like a real modern option. We use a lot of opensolaris at $dayjob mostly just because of its wonderful filesystem zfs. I have to say that I am excited about opensolaris much more than I ever was with Solaris proper! With older releases, the non GNU userland was just simply painful and by my "sense" pretty stupid in many ways. Most of the pain is now gone with the new opensolaris releases. Since the release of opensolaris 2008.11 is pretty close (well it's November now already), I decided to snatch the pre-release iso and take it for a spin on my HP Pavilion 6700 laptop!

Visiting http://www.genunix.org/ to download the iso file, I snatched the file which was significantly more than 700MB, so I had to burn that to DVD. The opensolaros 2008.11 iso is both a Live DVD that is also installable. That is sweet, as I can run opensolaris now directly from the live DVD and install latter if I want. Starting the DVD was easy, just pop it in, BIOS boots chooses DVD. Afterwards you stare at a GRUB screen, I simply chose the default option and let it boot the GUI environment. You get asked for keyboard layout and language, I just hit "enter" for the defaults (US/English). More than a minute later (booting is slow since it's using an optical drive), you get the GUI desktop. It's basically a Gnome envrionment.

Let's make sure we're really on opensolaris :)

jack@opensolaris:~$ cat /etc/release
OpenSolaris 2008.11 snv_100a X86
Copyright 2008 Sun Microsystems, Inc. All Rights Reserved.
Use is subject to license terms.
Assembled 24 October 2008
jack@opensolaris:~$ uname -a
SunOS opensolaris 5.11 snv_100 i86pc i386 i86pc Solaris

As you can see the default desktop looks clean and is fairly attractive. After that I started the DDU (Device Driver Utility), a nice utility made by Sun to check your hardware compatibility level with opensolaris. I was of course expecting issues here as this is a recentish laptop bought only a couple of months ago and opensolaris' hardware support on generic x86 hardware not mention laptops is well, less than stellar to say the least. Or at least that was the impression I was under. I was however very positively impressed when the DDU said "Driver Problems: 0" and showed that there were drivers for every single piece of hardware in that laptop. That was actually nice!


Other nice touches as well, is the laptop's screen automatically started in the native 1280x800 mode, and was using the native (closed) nvidia driver



As you can see, life is not full of roses, the volume while works fine and audio generally is ok, however, it seems the volume bar can only control the "master" volume mixer! My Fedora10 rawhide shows a zillion volume bars for all of my hardware features (line in/out, headphones, pcm, ... etc). In general I felt the audio stack needed some work on opensolaris. I wish they'd put in an alsa compatibility layer, that should probably help porting applications. Another issue, was connecting to my WPA2 home network. At first the network applet was very stubborn and would not connect to my network. After a lot of clicking here and there and displaying dialogs which were asking for (username?!) and password for the wireless network, it somehow got connected. After that the connection seemed pretty stable.

Being on the Internet, I started Firefox 3.0.1, pretty solid and fast as usual. Gotta love open-source portable apps ;) Now is the time to put this OS to the test. Let's see if it's gonna let me on youtube.com! It becomes quickly apparent that flash player is not installed by default. A quick visit to adobe, I download a tar file, uncompress it. There were no READMEs of any sort, just a couple of binary files. However creating a "plugins" directory under ~/.mozilla and copying those 2 files under it, seemed to have done the trick. I can now play youtube videos with no problems. Actually the whole video playback experience, and generally firefox seems faster and more responsive than under fedora. All in all, I'm very happy with the browsing experience.


Now I decided to go ahead and take a shot at crashing suspending my laptop! In order to test that, you will need to enable it by adding a line in /etc/power.conf:

S3-support enable

And then enter:

pfexec /usr/sbin/pmconfig

Afterwards, I could not find any GUI elements or commands to initiate a suspend! I hit the power button, and Gnome popped up a dialog asking what I wanted to do, I chose "Suspend" and a couple of seconds later, the laptop was suspended, with the power led blinking. I thought "sweet" .. It was pretty late, so I went to sleep, and thought I'll see if it wakes up (usually much harder) tomorrow. The next day, I open the lid, and boom, it starts back where it were! A fully successful (Hurraaay!) suspend/resume cycle, well I really did not expect that from opensolaris! Here is the relevant log part from /var/adm/messages

Nov 6 19:25:42 opensolaris genunix: [ID 535284 kern.notice] System is being suspended
Nov 6 19:25:44 opensolaris nvidia: [ID 702911 kern.notice] NOTICE: NVRM: RmPowerManagement: 4
Nov 6 19:25:44 opensolaris acpica: [ID 530733 kern.notice] NOTICE: acpica_ddi_setwake: could not evaluate _PRW
Nov 7 15:13:34 opensolaris last message repeated 9 times
Nov 7 15:13:34 opensolaris pcihp: [ID 709949 kern.notice] NOTICE: pcihp (pcie_pci0): card is inserted in the slot pcie1 (pci dev 0)
Nov 7 15:13:34 opensolaris genunix: [ID 358043 kern.warning] WARNING: ieee80211_crypto_enmic failed.
Nov 7 15:13:34 opensolaris mac: [ID 486395 kern.info] NOTICE: iwk0 link down
Nov 7 15:13:34 opensolaris acpica: [ID 530733 kern.notice] NOTICE: acpica_ddi_setwake: could not evaluate _PRW
Nov 7 15:13:34 opensolaris last message repeated 1 time
Nov 7 15:13:36 opensolaris nvidia: [ID 702911 kern.notice] NOTICE: NVRM: RmPowerManagement: 5
Nov 7 15:13:36 opensolaris gnome-session[756]: [ID 702911 daemon.warning] WARNING: Could not ask power manager to suspend: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.
Nov 7 15:13:36 opensolaris in.routed[474]: [ID 238047 daemon.warning] interface iwk0 to 192.168.11.3 turned off
Nov 7 15:13:38 opensolaris /sbin/dhcpagent[2275]: [ID 967406 daemon.warning] refreshing state on iwk0
Nov 7 15:13:39 opensolaris genunix: [ID 583038 kern.notice] System has been resumed.

My conclusion at this point, is that opensolaris is now much more user friendly than it once was, it had no problems detecting my hardware and using it, most of the same applications I use on Linux are there for opensolaris as well. Support for laptops and suspend/resume is not that bad it seems (or I'm extremely lucky!). However, more work needs to be put into improving the audio stack, and other niceties like bluetooth. I suspect more hardware support is also necessary. All in all, using opensolaris as a desktop seems viable if you're coming from a Linux background at least. I still prefer Linux of course for the convenience and for knowing I can almost get anything done with it. However, I will keep a close eye on opensolaris. Overall, I am impressed not only with the current state of the system, but more so with the rate of progress that has been made!

Joining the blogging crowd

So, finally I decided to join the blogging crowd. I am starting this as a way of storing information that is important to me. I am mostly focused on Linux and open source news. That's my job, that's what excites me, and that's what brings joy to my life. Too many times I find myself looking up the same things over and over again, so I will put them on this blog to try and store them both for myself and for other souls