July 2012 Archives

clang is finally debuggable with gdb

It was a pain to debug clang -faddress-sanitizer instrumented executables, because variable values were not visible, though they should be. It was caused by wrong stack realignment, triggered by -faddress-sanitizer, but it really was a clang bug.

This is now fixed in clang svn, since r160248. See http://llvm.org/bugs/show_bug.cgi?id=11468. http://llvm.org/bugs/show_bug.cgi?id=11818 is also fixed now.

With a llvm from svn go into build and do a make update. I'm running with r160959 successfully now.

Update: Sorry, it is not (yet) Nov 2012.

But I got it somehow working with the LLVM debugger, lldb.

E.g. a current problem in blead, debugged in darwin, built with perlall build bleadd-nt-asan and a recent llvm 3.2.svn

$ lldb /usr/local/bin/perl5.17.6d-nt-asan@5ca74088  cpan/Socket/t/sockaddr.t
Current executable set to '/usr/local/bin/perl5.17.6d-nt-asan@5ca74088' (x86_64).
(lldb) b __asan_report_error
breakpoint set --name '__asan_report_error'
Breakpoint created: 1: name = '__asan_report_error', locations = 1
(lldb) r
Process 89832 launched: '/usr/local/bin/perl5.17.6d-nt-asan@5ca74088' (x86_64)
ok 1 - inet_aton returns packed bytes
ok 2 - inet_ntoa from v-string
ok 3 - inet_aton->inet_ntoa roundtrip
ok 4 - inet_ntoa warns about wide characters
ok 5 - inet_pton AF_INET returns packed bytes
Process 89832 stopped
* thread #1: tid = 0x1c03, 0x000000010000b210 perl5.17.6d-nt-asan@5ca74088`__asan_report_error, stop reason = breakpoint 1.1
    frame #0: 0x000000010000b210 perl5.17.6d-nt-asan@5ca74088`__asan_report_error
-> 0x10000b210:  pushq  %rbp
   0x10000b211:  pushq  %r15
   0x10000b213:  pushq  %r14
   0x10000b215:  pushq  %r13
(lldb) up 2
frame select -r 2
frame #2: 0x0000000102f8d8ce Socket.bundle`XS_Socket_inet_ntop + 2462 at Socket.xs:938
   935                "Socket::inet_ntop", af);
   936      }
-> 938      Copy(ip_address, &addr, sizeof addr, char);
   939      inet_ntop(af, &addr, str, sizeof str);
   941      ST(0) = sv_2mortal(newSVpvn(str, strlen(str)));

(lldb) frame variable
(CV *) cv = <variable not available>
(SV **) sp = <variable not available>
(SV **) mark = <variable not available>
(I32) ax = <variable not available>
(SV *) ip_address_sv = <variable not available>
(int) af = 2

(lldb) register read
General Purpose Registers:
   rbx = 0x0000000102844060  libperl.dylib`PL_stack_base
   rbp = 0x00007fff5fbff370
   rsp = 0x00007fff5fbff200
   r12 = 0x0000000000000001
   r13 = 0x00001000207a7911
   r14 = 0x0000000102844060  libperl.dylib`PL_stack_base
   r15 = 0x00001fffebf7fe44
   rip = 0x0000000102f8d8ce  Socket.bundle`XS_Socket_inet_ntop + 2462 at Socket.xs:941
13 registers were unavailable.

See http://lldb.llvm.org/tutorial.html

$ clang --version
clang version 3.2 (trunk 167678)
Target: x86_64-apple-darwin12.2.0
Thread model: posix

New coding style?

Some of my cPanel colleagues came up with a new coding style. They are standing.


First it was Phil King standing in his office, after first attempts with a big ball failed and his back still hurts sometimes. Now Nick Jackson (on the picture) joined him, and others announced to follow.

This desk is called geek desk and has a motor and remote control attached.

Back to Windows

Hi mst! I joined the club.

On Saturday I gave up trying to fix WiFI on debian for my ASUS Zenbook laptop. WiFi works fine on all accesspoints. All but one, my home WiFi. It worked for months in my previous home. Something happened.

First I got around with USB tethering via my phone and the phone wifi. Last week even this failed. Because I have such a new laptop I need a new linux kernel. To support closing and re-opening the lid. I hacked wakeup, but suspend still gets the battery hot, I guess the GPU is still running. linux kernel 3.3 needed some patches, 3.4 works fine out of the box. But NetworkManager not. I suspect it is the new avahi demon, which was added to support link-local connections if everything else fails. But my link-local route has metric 1000. I cannot change nor delete it.

Also, the ELANTech touchpad driver sucks. It gets out of sync every once in a while. This does not happen with the Windows 7 touchpad driver. If you want to do a live presentation with the Zenbook and Elantech touchpad do not use Linux. Your cursor will jump around, you will accidently delete paragraphs in the front of the crowd. Suspend and wakeup on Windows 7 also just works.

So I'm stuck with an awfully slow filesystem again. Local testing time will double. At least cygwin got better and better lately. And the new mosh experimental cxxwrapper makes life easy, even with my slow WiFi.

325547-asus-zenbook-ux31-rsl8.jpg The next step will be to try to make a hackintosh out if this. The Zenbook looks like an Air, it should behave like an Air. There was no success story for this HW so far, but I'll give it a try.

I had to remove perl from mosh

Today is a very sad day for perl: https://twitter.com/Reini_Urban/status/225999360058617856

I had to remove the perl frontend for mosh, being replaced by a C++ script. perl IO::Pty on cygwin and various platforms did not work good enough. mosh was one of the most prominent users of perl.

Accidently some tweet from today says: ": But for some things, ‪#Perl‬ just isn't the optimal choice. (yet) :-) -Larry Wall in 199702221943.LAA20388@wall.org ‪#iFollowBack‬"

A note to folks reading reddit:

This is not a fork, it's a sanctioned pre-release of 1.3 with a bad version number. It was decided for the next major mosh release 1.3 to use the C++ wrapper by Peter. The new c++ wrapper is also used in the android mosh opkg version. mosh is awsome, 1.3 will be even more awsome.

PS: If you don't know that: I am the cygwin maintainer of perl, and toddr the IO::Pty maintainer sits next to me. It's not my personal decision to remove perl, but I just had to do that also for my port.

Comparing two major releases

I did two major releases over this weekend. The big cygwin switch from perl 5.10 to 5.14 (~1 year work) and the next supported ("stable") parrot release 4.6.0 (exactly 1 month work). This deserves a blog post to compare the two efforts.

The cygwin perl packages are using self-written bash scripts to automate the build and release process. They support also self-compiled variants with different features (debugging, non-threaded, different cflags, Policy.sh, ...) and those perl's can be used in parallel to the officially supported one.

It should also be noted that perl in cygwin does not package every single perl module out there, it rather leaves the installation and dependency game to CPAN or cpanm. Official cygwin packages depending on certain perl modules can just use a cygport three-liner to create this package and install it into vendor_perl. cpan installation go into site_perl. debian and fedora e.g. package every single module into their own package format.

The next important information is that cygwin is awfully slow compared to a unix. A typical build needs a few hours to complete. On a unix system it needs ~20 minutes. I can compare numbers because I ran it on the same machine in different VMs. And my windows VM is even faster than my native Windows machines.

The cygwin release process is pretty easy for me. I maintain a lot of cygwin packages, so I have direct ssh access to the master machine on sourceware.org. Others have to write an email to the dev list and request an upload from a public url and some instructions for the release process. (test or normal release, which packages to delete or keep, ...).

You prepare at minimum two tar.bz2 files for binary and source, and a setup.hint file for the central installer database, for each package. perl itself includes perl, perl_vendor, perl_manpages and perl_debuginfo, and for a major release you also have to coordinate updates for all XS modules. About 120 cygwin packages depend on perl, so it would be good to test all of them with your new major release, and about 10 packages needs rebuilding because they are using XS modules in vendor_perl. That's much less work for a release manager than on debian or fedora or macports, because cygwin lets CPAN and the user do most of the work.

So for the 5.14 switch there were 5 people involved who used my test build I prepared in April after about one year work to get it right (and several months waiting time because of my cross-atlantic move and several mainboards burned in texan storms - worse infrastructure here than in India), uploaded their test versions, and than I complete the rest and switched all the packages on one go from old to new. This was a pretty intense release process.

Compare that to parrot


My release was a three-month cycled so-called "supported" release, which most packagers take. A minor major release. There were two potential difficult branches to be merged, io_cleanup1 and threads. Both seemed pretty stable, but small hairy details and lots of testing prevented both of them being merged, so the diff from my release to the previous was super short. You can call it stable because almost nothing changed. Good for me.

The next big improvement for parrot over cygwin is the amount of written release docs and automated release tools. But testing needs much more time on parrot. I tested it with all three branches (master, io_cleanup1 and threads) on debian, cygwin, solaris, openbsd and because threads were supposed to be broken on netbsd, solaris cc64 and darwin/ppc I installed a qemu instance of debian/powerpc which could not reproduce the blocking darwin/ppc errors, so threads could not be merged. Very sad.

But I least I wrote a blog post how easy it is to install various architectures with qemu. I'll make the ready images available for others to test. It will have various perls, parrot, and rakudo installed, so you can test your XS modules on uncommon architectures like powerpc, sparc and sparc64 pretty easily.

The actual parrot release needs two days. On the first day you cut the release, which is pretty much automated by several tools and test it on your various architectures. MSWin32 and cygwin had some issues, but none of them were regressions. And you test against the various users (=languages) of parrot, mainly rakudo and report any problems to them. This is very easy and smooth via irc. Be sure to test against installed versions of parrot also, because that what the pooor user will do. Developers live in their git repo and rarely install, but outsiders who might want to try it out use the official locations.

It should also be noted that the parrot build process is easily disturbed by an installed parrot. And working with git can lead to unexpected side-effects, like unclean repos.

On the second day, the release will be committed, the docs need to be updated on myriads of sites and the various announcements have to be done. Wait with the announcement until everything else is ready. Your fellow devs need to wait from the "bugday" (Saturday) until the release (Thursday) not to disturb the timeline of master. In theory no problem to let committers continue their work, because you have your own clean repo, and can cp (=cherry-pick) commits to master in this period. But then you'll have to rebase origin/master which is always a bad idea.

cygwin perl updated from 5.10 to 5.14

I switched perl and all its dependencies on cygwin from 5.10 to 5.14 today.

This was my announcement mail:

perl has now been updated from 5.10.1-5 to 5.14.2-3.
Most of the dependant official cygwin perl packages containing XS code
have also been updated.
All other packages containing or referencing perl code should just
work, except ming and postgresql.
See below for updating your self-compiled XS modules.

I've got practical and conceptual problems with perl 5.16,
so 5.14 it will …

Architecture testing

Yes, I was architect for more than 10 years, but here I'm writing about testing computer architectures.

The most heard complaints on perl or parrot testing is about certain unknown architectures and platforms. If it's Windows, 64bit, little-endian, sparc, hpux, AIX, solaris, arm and so on.

So why not test out all those OS and architectures by our own?

On Windows you only got VMWare, VirtualBox and Hyper-V, but on debian you can simulate everything easily with qemu-kvm. On MacOSX I was not so happy with virtualization.

I suppose you have an amd64/x86_64 system and linux, with more than 4GB RAM. 16GB was good enough for me to run 8 images and compile bigger programs. With kvm you got fast native support for i386 and x86_64, and the rest can be simulated with qemu.

Images can be downloaded at various places. I usually use http://people.debian.org/~aurel32/qemu/ with debian images for amd64, arm, armel, i386, kfreebsd-amd64, kfreebsd-i386, mips, mipsel, powerpc, sh4, sparc. With this arsenal you can easily test debian and freebsd on most architectures.

kvm images for other OS (amd64/x86) are also available everywhere. I usually test all centos versions, windows xp, windows 8, win64, freebsd, openbsd, solaris. If you cannot find an image, find a OS installer CDROM, and use this as source for the fresh install. This is easiest with the Virtual Machine Manager. E.g. for solaris or Windows. I couldn't find the right HW simulation for OSX, but I heard it should also be possible. Just get a license.

E.g. for ppc I download http://people.debian.org/~aurel32/qemu/powerpc/debian_squeeze_powerpc_standard.qcow2, read the README.txt for the root password, and start it with:

qemu-system-ppc -boot c -m 512 \
    -hda /var/lib/libvirt/images/debian_squeeze_powerpc_standard.qcow2 \
    -net nic,vlan=1 -net user,vlan=1 -localtime

You can also add -nographics to get a console only, without vga graphics. This way it is much easier to cut & paste with your mouse. If your machine does not shutdown -H -h now properly, remember the qemu hotkey Ctrl-a h. Ctrl-a x exits qemu.

If I am crazy enough I use virsh and the new xml format together with the Virtual Machine Manager. Mainly for the kvm images.


If not I study man qemu and use the old qemu-launcher or cmdline.


Recently I tried poky and bitbake, to test perl and parrot on ARM processors.

$ sudo apt-get install gawk wget git-core diffstat unzip texinfo build-essential 
   chrpath libsdl1.2-dev xterm
$ wget http://downloads.yoctoproject.org/releases/yocto/yocto-1.3/poky-danny-8.0.tar.bz2
 $ tar xjf poky-danny-8.0.tar.bz2
 $ cd poky-danny-8.0
 $ source oe-init-build-env
 $ MACHINE=qemuarm bitbake core-image-minimal 

 $ ../scripts/runqemu qemuarm core-image-minimal serial nographic ... => error, no tap
 $ qemu-system-arm -kernel build/tmp/deploy/images/zImage-qemuarm.bin -net nic,vlan=0  -M versatilepb -hda build/tmp/deploy/images/core-image-minimal-qemuarm.ext3 -no-reboot -show-cursor -usb -usbdevice wacom-tablet -no-reboot -nographic -m 128 --append "root=/dev/sda rw console=ttyAMA0,115200 console=tty ip= mem=128M highres=off"

 qemuarm login: root <enter>

root@qemuarm:~# uname -a
Linux qemuarm 3.4.20-yocto-standard #1 PREEMPT Fri Dec 21 09:18:03 
   CST 2012 armv5tejl GNU/Linux
# perl -v
This is perl 5, version 14, subversion 2 (v5.14.2) built for arm-linux-gnueabi

But better adjust your packages in your conf/local.conf. core-image-minimal has no developer tools and no package tools.

EXTRA_IMAGE_FEATURES = "debug-tweaks tools-sdk tools-debug"

Starting an image is easy, get networking working is much harder.


With -net user I have access from the image to my host and the network, but I cannot ssh into the guest. With bridge (same subnet) or tap (different subnet) I can ssh into my guests.

For most of my images I use a host bridge and our company dhcp server. So everybody has access, and vice versa.

See e.g. http://qemu-buch.de/cgi-bin/moin.cgi/QemuAndTuntap Theoretically Virtualbricks could be used to setup easily and tests various network topologies, but it is too buggy.

My host bridge setup looks like this (manual ip for the bridge. in ifconfig br0 has no IP, eth1 does)

$ cat /etc/network/interfaces

auto eth1
iface eth1 inet manual

auto br0
iface br0 inet manual
    bridge_ports eth1
    bridge_hello 2
    bridge_maxage 12
    bridge_stp off

$ sudo brctl show
bridge name bridge id       STP enabled interfaces
br0     8000.4e0b13b861ad   no      eth1

For most of the time my images with qemu on my fast workmachine are faster then any old HW, such as e.g. a SparcStation or a Mac Powerboook. My kvm windows xp image boots in less than 10 seconds.

To setup and test my perls and perl modules I use perlall. For parrot I use rsync and git.


To get best performance buy RAM. The more the better. kvm behaves badly under memory pressure, esp. with hogs like Solaris or Windows. Esp. since linux kernel 3.0 and with cgroups and low memory I could wait 30 minutes until kvm comes back from swapping back and forth with solaris or windows. (not fixed yet)

Use raw partitions - without a filesystem - for the image and use virtio. Redhat has fantastic virtio drivers. This will create esp. on Windows a faster OS than native.

Use LVM to manage your raw partitions, so you can later increase the size of your vm's. You'll need typically 16GB for Windows and Solaris (esp. Solaris ZFS is a hog), and 4-8 GB for small shell-only unices. Name your lvm partitions to your image names, not only numbers.


On Windows 32bit turn off swap (no virtual memory) and give it 3GB RAM. On Win64 even more.


A made some images available to download at http://perl514.cpanel.net/qemu/. With debian squeezy and some perls and parrot. For now only powerpc, mips and an instable sparc.

Each tar comes with a shell script to start, a README, a qcow2 image and sometimes a bios file. It's best put into /root and started as root.