Debugging notes-to-self

Stuff I wish I’d known for years (and that the rest of the world probably always knew, just not me):

1. after your buggy program was slowly eating your memory and swap, and you escaped to a console to kill it using Alt+SysRq+R and then Ctrl+Alt+F1, when you get back to your X-session you can make it grab the keyboard again by typing “kbd_mode -s” in an xterm (found here after trying all sorts of incantations in DuckDuckGo)
2. if that was on a laptop keyboard with “AltGr dead keys”, the SysRq combo might need to be Fn+SysRq+AltGr+R (yes, try that with one hand!), where AltGr is the right Alt key (found after lots of failed attempts – very frustrating if your swap runs out before you get to the console)
3. in the end, you wouldn’t typically have needed any of the above if only you’d known that “ulimit -v 1000000″ limits any process in your bash shell to 1GB of memory, so that your buggy program will die superquick without locking up the system for 10 minutes (found in no time through a most obvious query – it just took a long time before I thought of looking for it at all…)

Ksplice Uptrack: a quick-test on Ubuntu 9.04 Live

I’ve been using Ubuntu 8.04 on my laptop for ages, and never had any reason to upgrade from there – “it just works, I’m done upgrading” is what I’d smugly tell people… Now, I’ve found a big reason to upgrade: Ksplice, which I mentioned the other day, put a new service up:

Ksplice Uptrack is a new service that lets you effortlessly keep your systems up to date and secure, without rebooting.

Once you’ve completed the easy installation process, your system will be set up to receive rebootless updates instead of traditional, disruptive updates.  [...]

Ksplice, Inc. is proud to make this service freely available for the latest version of the world’s most popular desktop Linux distribution: Ubuntu 9.04 Jaunty Jackalope.

No more reboots, and still applying security patches as soon as they become available. That’s worth the dist-upgrade hassle.

For now, all I did was running a quick test. I had a USB stick with Ubuntu Netbook Remix 9.04 lying around, so I booted from that, hooked up the wifi (man, connecting is fast with NetworkManager 0.7-something – another reason to upgrade…), downloaded ksplice-uptrack.deb, and installed it on the Live system (you also need network connectivity to fetch some dependencies from the Ubuntu repository). This is what you get:

There’s a little tray-icon (the one resembling a “K”…) informing you that kernel updates are available, and clicking it opens an update window. Nothing exciting to see here, actually.

Still not very exciting. The whole thing is very understated, almost disappointingly so – I mean, something this cool should look cool, shouldn’t it?

…. and everything still works after this. In fact, I’m typing this post from the Live system with the (supposedly) updated kernel. I tried shutting the lid on my D630, and it nicely went into ACPI suspend. And came back up.

Wicked.

(Small disappointment: it seems Firefox crashed between suspend and resume. Did it a second time, and again Firefox died. Third time: no problems. Not sure if this has anything to do with anything, so for now pretend I didn’t mention it.)

Cool stuff, seriously. This will be in 10.04 by default, I’ve no doubt. In case you’re looking, here’s one guy eager to work on that!

One more thing: in their FAQ they suggest a little test to demonstrate that the thing actually does something. I tried their suggestion and ran their test-thing a couple of times. But I’m off to bed now, so here’s the output, and I’ll leave calculating whether the difference before/after updates is statistically significant to you…

ubuntu@ubuntu:~$ wget -O demo.c http://www.ksplice.com/uptrack/2009-06-demo.c
ubuntu@ubuntu:~$ gcc demo.c -o demo
ubuntu@ubuntu:~$ sudo cpufreq-selector -c 0 -g performance
ubuntu@ubuntu:~$ cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
performance
ubuntu@ubuntu:~$ sudo cpufreq-selector -c 1 -g performance
ubuntu@ubuntu:~$ cat /proc/cpuinfo
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 23
model name      : Intel(R) Core(TM)2 Duo CPU     T8100  @ 2.10GHz
stepping        : 6
cpu MHz         : 2101.000
cache size      : 3072 KB
physical id     : 0
siblings        : 2
core id         : 0
cpu cores       : 2
apicid          : 0
initial apicid  : 0
fdiv_bug        : no
hlt_bug         : no
f00f_bug        : no
coma_bug        : no
fpu             : yes
fpu_exception   : yes
cpuid level     : 10
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx lm constant_tsc arch_perfmon pebs bts pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 lahf_lm ida tpr_shadow vnmi flexpriority
bogomips        : 4189.64
clflush size    : 64
power management:

processor       : 1
vendor_id       : GenuineIntel
cpu family      : 6
model           : 23
model name      : Intel(R) Core(TM)2 Duo CPU     T8100  @ 2.10GHz
stepping        : 6
cpu MHz         : 2101.000
cache size      : 3072 KB
physical id     : 0
siblings        : 2
core id         : 1
cpu cores       : 2
apicid          : 1
initial apicid  : 1
fdiv_bug        : no
hlt_bug         : no
f00f_bug        : no
coma_bug        : no
fpu             : yes
fpu_exception   : yes
cpuid level     : 10
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx lm constant_tsc arch_perfmon pebs bts pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 lahf_lm ida tpr_shadow vnmi flexpriority
bogomips        : 4189.57
clflush size    : 64
power management:

ubuntu@ubuntu:~$ ./demo
time to write 100 lines is 6(msec)
# ...hmmm, wait, this is a Live system...
ubuntu@ubuntu:~$ sudo mount /dev/sda3 /mnt
ubuntu@ubuntu:~$ cd /mnt/
ubuntu@ubuntu:/mnt$ sudo mkdir test
ubuntu@ubuntu:/mnt$ sudo chmod a+rwx test
ubuntu@ubuntu:/mnt$ cd test/
ubuntu@ubuntu:/mnt/test$ cp /home/ubuntu/demo .
ubuntu@ubuntu:/mnt/test$ ./demo
time to write 100 lines is 49(msec)
ubuntu@ubuntu:/mnt/test$ ./demo
time to write 100 lines is 54(msec)
ubuntu@ubuntu:/mnt/test$ ./demo
time to write 100 lines is 64(msec)
ubuntu@ubuntu:/mnt/test$ ./demo
time to write 100 lines is 60(msec)
ubuntu@ubuntu:/mnt/test$ ./demo
time to write 100 lines is 75(msec)
ubuntu@ubuntu:/mnt/test$ ./demo
time to write 100 lines is 72(msec)
ubuntu@ubuntu:/mnt/test$ ./demo
time to write 100 lines is 62(msec)
ubuntu@ubuntu:/mnt/test$ ./demo
time to write 100 lines is 65(msec)
ubuntu@ubuntu:/mnt/test$ ./demo
time to write 100 lines is 80(msec)
ubuntu@ubuntu:/mnt/test$ ./demo
time to write 100 lines is 52(msec)
ubuntu@ubuntu:/mnt/test$ sudo uptrack-remove --all -y
The following steps will be taken:
Remove [cdoprpi1] Performance regression in filesystem buffer code.
Remove [9xoc5qmo] Possible erroneous memory overcommit in program start.
Remove [ll9q1ymc] Multiple bugs in filesystem core.
Remove [ovniqwxh] CVE-2009-1192: Information leak in the agp subsystem.
Remove [hrxbvh0e] CVE-2009-1265: Integer overflow in the af_rose maximum user frame size.
Remove [uzolzfa2] CVE-2009-1337: kill the wrong capable(CAP_KILL) check.
Remove [xgqc9vy4] VGA console corrupts non-ASCII characters.
Remove [pdfrn6qa] Denial of service by evading CPU time limits.
Remove [c8ueseae] Symbolic link filenames under eCryptfs can produce alarming warnings in dmesg.
Removing [cdoprpi1] Performance regression in filesystem buffer code.
Removing [9xoc5qmo] Possible erroneous memory overcommit in program start.
Removing [ll9q1ymc] Multiple bugs in filesystem core.
Removing [ovniqwxh] CVE-2009-1192: Information leak in the agp subsystem.
Removing [hrxbvh0e] CVE-2009-1265: Integer overflow in the af_rose maximum user frame size.
Removing [uzolzfa2] CVE-2009-1337: kill the wrong capable(CAP_KILL) check.
Removing [xgqc9vy4] VGA console corrupts non-ASCII characters.
Removing [pdfrn6qa] Denial of service by evading CPU time limits.
Removing [c8ueseae] Symbolic link filenames under eCryptfs can produce alarming warnings in dmesg.
ubuntu@ubuntu:/mnt/test$ ./demo
time to write 100 lines is 816(msec)
ubuntu@ubuntu:/mnt/test$ ./demo
time to write 100 lines is 805(msec)
ubuntu@ubuntu:/mnt/test$ ./demo
time to write 100 lines is 793(msec)
ubuntu@ubuntu:/mnt/test$ ./demo
time to write 100 lines is 786(msec)
ubuntu@ubuntu:/mnt/test$ ./demo
time to write 100 lines is 785(msec)
ubuntu@ubuntu:/mnt/test$ ./demo
time to write 100 lines is 787(msec)
ubuntu@ubuntu:/mnt/test$ ./demo
time to write 100 lines is 791(msec)
ubuntu@ubuntu:/mnt/test$ ./demo
time to write 100 lines is 787(msec)
ubuntu@ubuntu:/mnt/test$ ./demo
time to write 100 lines is 786(msec)
ubuntu@ubuntu:/mnt/test$ ./demo
time to write 100 lines is 785(msec)
ubuntu@ubuntu:/mnt/test$ sudo uptrack-upgrade -y
The following steps will be taken:
Install [c8ueseae] Symbolic link filenames under eCryptfs can produce alarming warnings in dmesg.
Install [pdfrn6qa] Denial of service by evading CPU time limits.
Install [xgqc9vy4] VGA console corrupts non-ASCII characters.
Install [uzolzfa2] CVE-2009-1337: kill the wrong capable(CAP_KILL) check.
Install [hrxbvh0e] CVE-2009-1265: Integer overflow in the af_rose maximum user frame size.
Install [ovniqwxh] CVE-2009-1192: Information leak in the agp subsystem.
Install [ll9q1ymc] Multiple bugs in filesystem core.
Install [9xoc5qmo] Possible erroneous memory overcommit in program start.
Install [cdoprpi1] Performance regression in filesystem buffer code.
Installing [c8ueseae] Symbolic link filenames under eCryptfs can produce alarming warnings in dmesg.
Installing [pdfrn6qa] Denial of service by evading CPU time limits.
Installing [xgqc9vy4] VGA console corrupts non-ASCII characters.
Installing [uzolzfa2] CVE-2009-1337: kill the wrong capable(CAP_KILL) check.
Installing [hrxbvh0e] CVE-2009-1265: Integer overflow in the af_rose maximum user frame size.
Installing [ovniqwxh] CVE-2009-1192: Information leak in the agp subsystem.
Installing [ll9q1ymc] Multiple bugs in filesystem core.
Installing [9xoc5qmo] Possible erroneous memory overcommit in program start.
Installing [cdoprpi1] Performance regression in filesystem buffer code.
ubuntu@ubuntu:/mnt/test$ ./demo
time to write 100 lines is 61(msec)
ubuntu@ubuntu:/mnt/test$ ./demo
time to write 100 lines is 56(msec)
ubuntu@ubuntu:/mnt/test$ ./demo
time to write 100 lines is 47(msec)
ubuntu@ubuntu:/mnt/test$

Ksplice Trophée du Libre

I’ve repeatedly been whining here about how kernel-update reboots kill productivity, but I also think that delaying security updates is the worse alternative.  So I was very excited to learn about Ksplice, through the LWN announcement of the “Trophées du Libre”. Ksplice is the 2009 winner in the Security category.

A quick snippet from the project page:

Ksplice enables running systems to stay secure without the disruption of rebooting.  Specifically, Ksplice creates rebootless updates that are based on traditional source code patches.  These updates are as effective as traditional updates, but they can be applied seamlessly, with no downtime.

Ksplice currently supports updating the Linux kernel, but the core technology applies to any operating system or to user space applications.

A quick search tells me even ZDNet had already heard of this project over a year ago, so I’m half ashamed that it’s news to me, but I’m too excited to keep it to myself :)

MiserWare MicroMiser: Miserable Licensing

Edit dd. 25 May: Miserware are offering the software under a more permissive license now. In short: they ask that you run your benchmarking procedures by them before publishing results. This seems a reasonable compromise, that protects them from poorly conducted testing taking the limelight. I will be playing with it…

—

Through the Ubuntu Weblogs feed I read a post at The Open Sourcerer about a piece of software by a company called MiserWare (yeah, the name inspired the pun even before I concluded as much…). They’ve released a private beta of MicroMiser, a closed-source package that promises a substantial power-consumption reduction on x86 Linux systems. Quoting from their website:

MicroMiser typically lowers total system energy use by 10-35% even when a system is 100% utilized.

That’s interesting. Even better, Alan wrote a follow-up post at The Open Sourcerer which shows that on his Ubuntu Server system the software estimates its own power savings to be close to 65% (!). Now, assuming for the moment that it’s not a bug in the calculations in the beta (hey, I’m in a mild mood tonight), I’d say those 65% would be really worthwile.

Alan was also kind enough to send me an invitation for the beta, and so – always the curious type – I was starting to make a plan on how to test this thing: use a dummy installation (we don’t trust closed-source betas, do we?), current-monitoring at the wall-socket, that sort of thing.Then, I figured I’d download the package first to see what was in it. A kernel module? Lots of ugly scripts? Who knows…

I don’t, anyway. I never got to the download step. In the step before that I was presented with a rather restrictive license. I quote verbatim (only the change to bold-type is mine, all text is original):

5.    Confidentiality

The Software and Documentation, all related Intellectual Property, and any information learned or discovered by You about the performance of the Software in the course of use under this License constitutes proprietary trade secret information owned solely by Licensor (collectively, “Confidential Information”).  You agree that You will not, without the express prior written consent of Licensor, (1) use the Confidential Information other than to use the Software as authorized by this License Agreement; (2) disclose any Confidential Information to any third party; or (3) fail to use best efforts to safeguard the Confidential Information from unauthorized use, copying, or disclosure. You acknowledge that a breach of this Section 5 may cause the Licensor irreparable harm and damages that are difficult to ascertain.  Therefore, the Licensor, upon a disclosure or threatened disclosure of any Confidential Information, will be entitled to injunctive relief (without the requirement of posting bond), without limiting its other remedies under this License Agreement, in equity or at law.  The obligations of this Section shall survive this License Agreement without limitation in duration.  By clicking the “I Agree” button below, You consent to having any information that You provide to the Licensor processed and stored in the United States.

Well, I’m still not a lawyer, nor am I a native speaker, but I think that this basically means I would not be allowed to tell you what I would read on my current monitor. What the….

I also think that this means that the post reporting the 65% savings is violating the license.

After a long half second of thinking about it, I did not agree, so I can’t tell you anything about the software – by the way, even if I had agreed, I still wouldn’t be able to tell you anything! My advice? Don’t waste your time with bloody legalese and read some real information instead: Intel has a very informative website on the topic at LessWatts.org.

The superiority of the distro

Shawn wrote a nice piece pointing out why developers (in particular for embedded systems) are better off running Linux than Windows. In summary, it’s all about the tools that ship with it. If we’re nitpicking then, what we’re actually talking about here is not Linux proper (the kernel), and not GNU/Linux (the operating system), but rather the distribution (whichever one that is). It’s the software stack as a whole that’s making the difference.

This is exactly what I usually tell people who ask why I’m running “this Linux thing”. It’s better suited to my needs on every level of the software stack. Of course, that’s usually way too vague to compel someone who had to ask to begin with. Examples please? My new favourite example is a security thing.

The other week I was catching up with a few long-overdue admin tasks on my parents’ PCs, mostly security updates. They’re (still) on MS Windows, and you should have seen the number of pop-ups when I logged on as admin: I don’t know how many apps, all reporting they’d like permission to fetch and install updates. Compare that to the elegance of the little warning star in the Gnome menu, or the output of a quick “aptitude upgrade”.

Big deal? I think so. I’m quite sure that that single interface to all software updates is not just more elegant, but that it’s also directly keeping systems more secure. Quoting from an old eWeek item:

For example, for the nine highest-profile Windows malicious code incidents as of March 2003, Microsofts patches predated major outbreaks by an average of 305 days, yet most firms hadnt applied the patches.

That is not a statement about Windows, or Linux. It’s a statement about human nature.

Here’s another item, from Ars:

Secunia cited data from Microsoft showing that third-party software vulnerabilities are the ones that are most frequently exploited, and said that its own data showed that users simply don’t update as frequently as they should.

Having a clear, simple, and non-crappy upgrade manager vastly diminishes these problems, because all people are lame, and the number of people that will not apply updates promptly will go up at least quadratically with the number of steps the update takes (and that’s a conservative estimate). That’s why distros will win from bare operating systems with apps dropped on top of them. It’s also why I’m going to press mom and dad to please let me replace their systems…

Is this benefit intrinsically tied to free software? In theory, no. I actually tried to pitch this idea to the Ideastorm crowd at some point. But maybe it would not work so well in practice: if MS would try to turn Windows into a distro, or would try to press other vendors into using their update manager, the anti-monopolistic regulators would be all over them in no time. So in practice, one might say this is a benefit of free software. Yay.