Yes, I was architect for more than 10 years, but here I'm writing about testing computer architectures.
The most heard complaints on perl or parrot testing is about certain unknown architectures and platforms. If it's Windows, 64bit, little-endian, sparc, hpux, AIX, solaris, arm and so on.
So why not test out all those OS and architectures by our own?
On Windows you only got VMWare, VirtualBox and Hyper-V, but on debian you can simulate everything easily with qemu-kvm. On MacOSX I was not so happy with virtualization.
I suppose you have an amd64/x86_64 system and linux, with more than 4GB RAM. 16GB was good enough for me to run 8 images and compile bigger programs. With kvm you got fast native support for i386 and x86_64, and the rest can be simulated with qemu.
Images can be downloaded at various places. I usually use http://people.debian.org/~aurel32/qemu/ with debian images for amd64, arm, armel, i386, kfreebsd-amd64, kfreebsd-i386, mips, mipsel, powerpc, sh4, sparc. With this arsenal you can easily test debian and freebsd on most architectures.
kvm images for other OS (amd64/x86) are also available everywhere. I usually test all centos versions, windows xp, windows 8, win64, freebsd, openbsd, solaris. If you cannot find an image, find a OS installer CDROM, and use this as source for the fresh install. This is easiest with the Virtual Machine Manager. E.g. for solaris or Windows. I couldn't find the right HW simulation for OSX, but I heard it should also be possible. Just get a license.
E.g. for ppc I download http://people.debian.org/~aurel32/qemu/powerpc/debian_squeeze_powerpc_standard.qcow2, read the README.txt for the root password, and start it with:
qemu-system-ppc -boot c -m 512 \ -hda /var/lib/libvirt/images/debian_squeeze_powerpc_standard.qcow2 \ -net nic,vlan=1 -net user,vlan=1 -localtime
You can also add
-nographics to get a console only, without vga graphics. This way it is much easier to cut & paste with your mouse. If your machine does not
shutdown -H -h now properly, remember the qemu hotkey
Ctrl-a x exits qemu.
If I am crazy enough I use
virsh and the new xml format together with the Virtual Machine Manager. Mainly for the kvm images.
If not I study
man qemu and use the old qemu-launcher or cmdline.
Recently I tried poky and
bitbake, to test perl and parrot on ARM processors.
$ sudo apt-get install gawk wget git-core diffstat unzip texinfo build-essential chrpath libsdl1.2-dev xterm $ wget http://downloads.yoctoproject.org/releases/yocto/yocto-1.3/poky-danny-8.0.tar.bz2 $ tar xjf poky-danny-8.0.tar.bz2 $ cd poky-danny-8.0 $ source oe-init-build-env $ MACHINE=qemuarm bitbake core-image-minimal $ ../scripts/runqemu qemuarm core-image-minimal serial nographic ... => error, no tap $ qemu-system-arm -kernel build/tmp/deploy/images/zImage-qemuarm.bin -net nic,vlan=0 -M versatilepb -hda build/tmp/deploy/images/core-image-minimal-qemuarm.ext3 -no-reboot -show-cursor -usb -usbdevice wacom-tablet -no-reboot -nographic -m 128 --append "root=/dev/sda rw console=ttyAMA0,115200 console=tty ip=192.168.7.2::192.168.7.1:255.255.255.0 mem=128M highres=off" qemuarm login: root <enter> root@qemuarm:~# uname -a Linux qemuarm 3.4.20-yocto-standard #1 PREEMPT Fri Dec 21 09:18:03 CST 2012 armv5tejl GNU/Linux # perl -v This is perl 5, version 14, subversion 2 (v5.14.2) built for arm-linux-gnueabi
But better adjust your packages in your
conf/local.conf. core-image-minimal has no developer tools and no package tools.
EXTRA_IMAGE_FEATURES = "debug-tweaks tools-sdk tools-debug"
Starting an image is easy, get networking working is much harder.
With -net user I have access from the image to my host and the network, but I cannot ssh into the guest. With bridge (same subnet) or tap (different subnet) I can ssh into my guests.
For most of my images I use a host bridge and our company dhcp server. So everybody has access, and vice versa.
See e.g. http://qemu-buch.de/cgi-bin/moin.cgi/QemuAndTuntap Theoretically Virtualbricks could be used to setup easily and tests various network topologies, but it is too buggy.
My host bridge setup looks like this (manual ip for the bridge. in ifconfig br0 has no IP, eth1 does)
$ cat /etc/network/interfaces auto eth1 iface eth1 inet manual auto br0 iface br0 inet manual address 10.1.4.44 network 10.1.4.0 netmask 255.255.255.0 broadcast 10.1.4.255 gateway 10.1.4.1 bridge_ports eth1 bridge_hello 2 bridge_maxage 12 bridge_stp off $ sudo brctl show bridge name bridge id STP enabled interfaces br0 8000.4e0b13b861ad no eth1
For most of the time my images with qemu on my fast workmachine are faster then any old HW, such as e.g. a SparcStation or a Mac Powerboook. My kvm windows xp image boots in less than 10 seconds.
To setup and test my perls and perl modules I use perlall. For parrot I use rsync and git.
To get best performance buy RAM. The more the better. kvm behaves badly under memory pressure, esp. with hogs like Solaris or Windows. Esp. since linux kernel 3.0 and with cgroups and low memory I could wait 30 minutes until kvm comes back from swapping back and forth with solaris or windows. (not fixed yet)
Use raw partitions - without a filesystem - for the image and use virtio. Redhat has fantastic virtio drivers. This will create esp. on Windows a faster OS than native.
Use LVM to manage your raw partitions, so you can later increase the size of your vm's. You'll need typically 16GB for Windows and Solaris (esp. Solaris ZFS is a hog), and 4-8 GB for small shell-only unices. Name your lvm partitions to your image names, not only numbers.
On Windows 32bit turn off swap (no virtual memory) and give it 3GB RAM. On Win64 even more.
A made some images available to download at http://perl514.cpanel.net/qemu/. With debian squeezy and some perls and parrot. For now only powerpc, mips and an instable sparc.
Each tar comes with a shell script to start, a README, a qcow2 image and sometimes a bios file. It's best put into /root and started as root.
For networking with qemu I use -redir:
qemu-system-i386 -hda debian.qcow2 -redir tcp:5555:10.0.2.1:22
That will forward local 5555 port to remote 22 port. That's it.
Helpful notes, thanks for them. Are you doing any kind of snapshot and restore work yet?
@Nate: These are just very short notes to get you an idea and started. Other blog posts go more into depth.
Basically you have two use cases:
1. qcow2 files for fast'n easy setup of new vm's, copy, test and destroy,
and 2. raw images for super-fast and fixed-size test vm's, mostly running.
I don't have snapshot and restore automated yet. I have perl testing, setup and vm management automated, but not as nice as your stuff.
And thanks for your windows help and bug reports with the IO memory pressure and cgroups on newer linux kernels!
And while you are here: Nate is using ZFS and qcow2 as host system, which seems to be better than my btfrs for files and LVM for raw. There snapshots are easily supported on the fs level. But his QA machine is huge, I only have a normal dev box.