Saturday, May 30, 2009

Traffic accounting with SNMP and Python on Windows

If you're not on a flat-rate ISP data-plan, and your ISP made you pay extra last month such as myself :) You might be interested in monitoring your home bandwidth. A nifty little features of most routers or ADSL modems, is support for SNMP. Since both me and my old folks use the same ADSL line, any host based monitoring solution wouldn't really work. I had to monitor the usage using SNMP. Since my folks are using Windows, the solution had to work on that. The solution overview is:

1- Enable SNMP on ADSL modem
2- Use a windows scheduled task to run a windows Batch file, that uses SNMP-tools for windows to pull data from the router
3- Use a windows scheduled task to run a python script that processes the batch file output and computes the total traffic

Data from the batch files are written in a file signifying the current month (example 05.txt for May). Here is the code for the batch file

cd e:\traffic



FOR /F "TOKENS=1,2 eol=/ DELIMS=/ " %%A IN ('DATE/T') DO SET mm=%%B

echo %mm%

snmpget.exe -c public -O v -v 1 . >> %mm%.txt

Note that the IP should be replaced with the internal IP of your ADSL router. Also, the SNMP OID . (symbolically: IF-MIB::ifInOctets.12) is the incoming octets on network interface "12". You might need to change that last "12" to represent your network topology. You can use the "snmpwalk" command to list your interface names like:

snmpwalk -c public -v1
this yields
IF-MIB::ifDescr.12 = STRING: ppp0

I know that my ppp0 is the link to my ISP, so that's the interface I used

And here is the code for the python script

# -*- coding: utf-8 -*-

import glob

import locale

import os

locale.setlocale(locale.LC_ALL, '')

reports = sorted(glob.glob('??.txt'))

totalsReport = open('totals.txt','w')

totalsReport.write(' Month      Traffic \n')


for report in reports:

        datastring = open(report,'r').readlines()

        data = [ int(datastring[i].strip().split()[1]) for i in range(len(datastring)) ]

        diff = [ data[i+1] - data[i] for i in range(len(data) - 1) ]

        trafficDeltas = [ (data[i+1],diff[i])[diff[i] > 0] for i in range(len(diff)) ]

        totalTraffic = sum(trafficDeltas)

        totalsReport.write(' %s => %s \n'% (report.replace('.txt','') , locale.format('%d', totalTraffic, True)) )


Glancing over the Python code, you can already see I'm a list comprehension addict :) The code reads the snmp bytes values from ??.txt (example: 05.txt), computes bandwidth deltas, handles counter resets (32bit overflow, or router loosing power) and computes totals. Then writes the total to a file named "totals.txt"

This solution is resistant to the router loosing power and resetting its internal traffic counter back to zero. This is because the data is continuously being saved to non-volatile medium (PC's hard-disk). Also, the Windows PC running those two scripts, does NOT need to be running 24x7 .. Just most of the time. If however the PC is powered off, AND the modem power cycles or hangs, you might loose some accuracy, the results will still be fairly valid though. Hope that's helpful to anyone out there

Google Wave, me likes

Check this out! Email+IM+Wiki+RealTimeCollaboration+Documents+Workflows+ServerSideBots+GoogleMagic+OpenStandards = Google Wave

This is the "Wave" of the Future ;)

Wednesday, May 27, 2009

Waiting for Fedora 11

Monday, May 25, 2009

Nexenta 2.0 is out

Check out the googley style toons at

VMs over ZFS rock

when configuring some complex task over a server, sometimes things go bad (corrupt files, wrong versions, bad data in DB ... etc) and sometimes it's really not easy to undo everything such that you're sure the machine is in a known-good-n-clean state. This is where VMs over ZFS rock!

while self.isPatient():
ssh vm poweroff;
zfs rollback vm@someTimeWhenThingsWhereGood
VBoxManage startvm vm

Hyper cool! Snapshots and ease/reliability of backups/restores are about one of the most important things why I love virtulization.

Wednesday, May 20, 2009

Killing Virii with Gentoo and Kaspersky

Sorry but if you pronounce kaspersky to rhyme with whisky, the title isn't going to sound as jiggly as it should ;) I used to pronounce it to rhyme with "sky" but apparently that's incorrect. Anyway, I've been a long time admirer of the Kaspersky AV engine. And today it has saved someone's a$$ one more time ;)

A friend of mine, who is pretty technical, well he maintains windows drivers for a living, so that sure makes him a hot-shot techy in anyone's book. That friend's laptop caught a nasty virus. In his own words, he was only downloading some power point presentations (ugh), when the miserable closed source proprietary OS he's running (euhm Vista) became infected. He was using Google's chrome browser, so the possibility of having been infected through a browser exploit remains pretty low in my opinion. Especially that Chrome auto-updates itself, thus fixing any potential security holes. My first impression was that he got infected through an exploit in MS Office 2007 (yuck). Anyway, with me trying to help him clean up the laptop, we tried the following

- Tried installing Symantec's AV suite. That totally fails to even install. What a piece of crap. Symantec's ware is highly over-rated IMO. I used to really like Norton stuff, back in the days of Norton's DiskDoctor .. those were the days :D

- Tried installing the tried and true MalwareBytes, which did detect and clean a whole bunch of malware, however, much to my surprise, the problem persisted. MalwareBytes is a cool piece of anti-malware, it has worked fantastically for me multiple times, but this time it wasn't enough!

- Having wasted a couple of hours on this already, I wanted to fire some Kaspersky power on the problem. I visited and downloaded a Live CD image Burned the iso, and booted!

Now this Live CD is absolutely cool, it's a customized build of Gentoo linux (w00t!) that automagically detected the hardware, connected to the network, started an X server, launched a customized icewm environment with Kaspersky's "K" logo as the "start" button down below. I was impressed, and through that GUI I could launch Kaspersky's AntiVirus tool.

The first thing it did was to auto-update itself over the internet. Most definitely needed. Afterwards it located and mounted all Windows NTFS partitions, and I was presented with options to scan them. I chose to scan the c: drive. Scan has begun, the scan tool sports a nice looking GUI, although it can be a bit confusing. Anyway scanning has started churning on the hard-disk. It was a bit slow, took around 3 hours for a 100G c: drive!

But I'll sure take slow and reliable over anything else every time! At the end, Kaspersky has located hundreds of infected executable files. I chose to disinfect them. It started disinfection one by one. This took around 20 minutes or so as well. Rebooting after that, windows came up finally clean . The system is working normally again, sigh!

All in all, Kaspersky proved to be a reliable tool. Kudos to their team for providing a top notch Linux based Live CD for free, that updates itself and provides adequate disinfection for free. Thank you Kaspersky. I will surely recommend you guys in the future. This is one AntiVirus tool I will be sure to remember, when a friend comes knocking on my door. Note however, that they're not the only game in town, others like Avira, and BitDefender and others as well offer Live CD "rescue-disks" as they are called. Hope this post helps anyone out there.