BuildBot:vm-setup

From Askmonty.org

MariaDB:Development >

More information about BuildBot is available in the Knowledgebase.

This page has been moved to: https://kb.askmonty.org/en/buildbot-setup-for-virtual-machines

1 Exact steps used to setup virtual machine images for the package testing in Buildbot
2 General principles
3 Detail of each vm image
4 Additional steps

Exact steps used to setup virtual machine images for the package testing in Buildbot

This page documents the exact steps used to set up each virtual machine used in Buildbot for package build and test. Lots of boring details, but should be useful for reference, and as a help when installing new virtual images for additional distros and versions.

This documents how the virtual machine images were actually set up, not the most efficient way to recreate all of them. Thus, some historical artifacts remain, reflecting the order they were initially added and lessons learned on the way.

For example, the steps for the earliest vms are somewhat overly entusiastic about using the `qemu-img create -b` option to make one vm a copy-on-write copy on top of another image. This makes it harder to modify the images afterwards, since if the base image is modified, any copy-on-write vm on top of it needs to be re-created.

The descriptions do not go to painful detail about the installation procedure. Generally the .iso for the distro was downloaded, and the default, non-graphical installer started. Installing Linux these days is quite easy, and generally just defaults were picked. With respect to harddisk partitioning, generally an 8Gb image was configured, of which 1/2 Gb was reserved for swap and the rest for a single root partition "/".

General principles

The installations are kept minimal, picking mostly default options. This helps ensure our stuff works on default installs, and also saves effort during the creation of the virtual machines.

Since most of the logic happens in the Buildbot slave on host machine and inside the Buildbot master, there is no need to install Buildbot or other similar complex stuff inside the vms.

ssh

The virtual machines are configured with ssh access. An account called 'buildbot' is created, with passwordless login (using ssh public key authentication), and with passwordless sudo access (needed for automated scripting). The public key for user buildbot is installed in every vm, and the private key is added to the account running the Buildbot master on the host machine.

To generate the ssh keys:

   ssh-keygen -t dsa

Leave the passphrase empty. Run this as the user running the Buildbot slave on the KVM host machine. The resulting ~/.ssh/id_dsa.pub will be needed below for each virtual machine install.

Serial port

The vms are configured to use the (emulated) serial port as the console. When running KVM on the host, the console then maps to the stdin/stdout. This is useful to get kernel log messages available to easier debug any problems. The bootloader Grub is similarly configured to use the serial port.

The vms are also set up to give a login prompt on the serial port (running getty). In retrospect this has proven not really needed, as if needing to manually log in and investigate things, it is often as easy to just run KVM in graphics mode. It may be useful in cases where the host machine is remote though (in this case graphics mode does still work but can be a bit slow).

To configure for serial console, the following two files are referenced below, and should be prepared in advance:

ttyS0

    cat > ttyS0 <<END
# ttyS0 - getty
#
# This service maintains a getty on ttyS0 from the point the system is
# started until it is shut down again.

start on stopped rc2
start on stopped rc3
start on stopped rc4
start on stopped rc5

stop on runlevel 0
stop on runlevel 1
stop on runlevel 6

respawn
exec /sbin/getty 115200 ttyS0
END

ttyS0.conf

# ttyS0 - getty
#
# This service maintains a getty on ttyS0 from the point the system is
# started until it is shut down again.

start on stopped rc RUNLEVEL=[2345]
stop on runlevel [!2345]

respawn
exec /sbin/getty -L 115200 ttyS0 vt102
END

Unattended MariaDB install on Debian/Ubuntu

The MariaDB (and MySQL) package on Debian and Ubuntu prompts the user about the root password to set. We want to test this important step, but we do not want the prompt of course. To do this we use the following configuration file for defaults for debconf. This file is needed in the steps below under the name "my.seed" (be careful to preserve it exactly as here, including trailing space and tab characters!)

   mariadb-server-5.1	mysql-server/root_password_again	password	rootpass
   mariadb-server-5.1	mysql-server/root_password	password	rootpass
   mariadb-server-5.1	mysql-server/error_setting_password	error	
   mariadb-server-5.1	mysql-server-5.1/nis_warning	note	
   mariadb-server-5.1	mysql-server-5.1/really_downgrade	boolean	false
   mariadb-server-5.1	mysql-server-5.1/start_on_boot	boolean	true
   mariadb-server-5.1	mysql-server-5.1/postrm_remove_databases	boolean	false
   mariadb-server-5.1	mysql-server/password_mismatch	error	
   mariadb-server-5.1	mysql-server/no_upgrade_when_using_ndb	error

For some background see here. The file my.seed can be generated from an existing install using debconf-get-selections.

For Debian/Ubuntu, we add a local repository to the list of apt source to be able to test `apt-get install`. For this, the following file "sources.append" must be prepared:

    cat >sources.append <<END
deb file:///home/buildbot/buildbot/debs binary/
deb-src file:///home/buildbot/buildbot/debs source/
END

Emulation options

The default network card emulated by KVM has poor performance. To solve this we instead use the "virtio" network device, using the KVM options "-net nic,model=virtio -net user". Except on Debian 4, which has an old kernel without support for virtio. Background info here.

A detail is that some 32-bit vms crashed during boot with default options. This was fixed by using the kvm "-cpu qemu32,-nx" option. Some background information here.

Detail of each vm image

Ubuntu 9.04 i386

This vm is used to build source tarball and for Ubuntu 9.04 32-bit .deb.

First create and install image:

   qemu-img create -f qcow2 vm-jaunty-i386-base.qcow2 8G
   kvm -m 2047 -hda vm-jaunty-i386-base.qcow2 -cdrom ubuntu-9.04-server-i386.iso -redir tcp:2222::22 -boot d -cpu qemu32,-nx
   # Install

Create a vm, based on the first one, which is configured for serial port.

   qemu-img create -b vm-jaunty-i386-base.qcow2 -f qcow2 vm-jaunty-i386-serial.qcow2
   kvm -m 2047 -hda vm-jaunty-i386-base.qcow2 -redir tcp:2222::22 -boot c -cpu qemu32,-nx

To configure kernel and grub for serial console, add the following to /boot/grub/menu.lst:

   serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1
   terminal --timeout=3 serial console

also add in menu.lst to kernel line (after removing `quiet splash'):

   console=tty0 console=ttyS0,115200n8

Create /etc/event.d/ttyS0:

   # ttyS0 - getty
   #
   # This service maintains a getty on ttyS0 from the point the system is
   # started until it is shut down again.

   start on stopped rc2
   start on stopped rc3
   start on stopped rc4
   start on stopped rc5

   stop on runlevel 0
   stop on runlevel 1
   stop on runlevel 6

   respawn
   exec /sbin/getty 115200 ttyS0

Add an account:

   sudo adduser --disabled-password buildbot

Copy public ssh key into ~/.ssh/authorized_keys

Enable passwordless sudo:

   sudo adduser buildbot sudo
   # uncomment `%sudo ALL=NOPASSWD: ALL' line in `visudo`, and move to end.

Create a new vm for building source tarballs:

   qemu-img create -b vm-jaunty-i386-serial.qcow2 -f qcow2 vm-jaunty-i386-deb-tarbake.qcow2

Install compilers:

   sudo apt-get build-dep mysql-5.1-server
   sudo apt-get install devscripts hardening-wrapper bzr

Copy in an existing bzr shared repository to ~buildbot/.bzr (or run bzr init-repo and bzr branch --no-tree lp:maria).

Create a new vm for building .debs:

   qemu-img create -b vm-jaunty-i386-serial.qcow2 -f qcow2 vm-jaunty-i386-deb-build.qcow2

Install compilers:

   sudo apt-get build-dep mysql-5.1-server
   sudo apt-get install devscripts hardening-wrapper

Create a new VM for testing .deb installs:

   qemu-img create -b vm-jaunty-i386-serial.qcow2 -f qcow2 vm-jaunty-i386-deb-install.qcow2

Install tools and local apt repository.

    sudo apt-get install debconf-utils
    cat >>/etc/apt/sources.list <<END
deb file:///home/buildbot/buildbot/debs binary/
deb-src file:///home/buildbot/buildbot/debs source/
END

Setup default package config for debconf to enable unattended install. Copy in my.seed (see above) to vm.

   sudo debconf-set-selections /tmp/my.seed

Create a new VM for testing upgrades:

   qemu-img create -b vm-jaunty-i386-deb-install.qcow2 -f qcow2 vm-jaunty-i386-deb-upgrade.qcow2

Prepare initial MySQL install with some test data.

   sudo apt-get install mysql-server-5.1
   mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"

Centos 5 i386

Base install

   cd /kvm
   qemu-img create -f qcow2 vms/vm-centos5-i386-base.qcow2 8G
   # ISO (dvd) install:
   kvm -m 2047 -hda /kvm/vms/vm-centos5-i386-base.qcow2 -cdrom CentOS-5.3-i386-bin-DVD.iso -redir 'tcp:2225::22' -boot d -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user

Configure for serial console

   (cd vms && qemu-img create -b vm-centos5-i386-base.qcow2 -f qcow2 vm-centos5-i386-serial.qcow2)
   kvm -m 2047 -hda /kvm/vms/vm-centos5-i386-serial.qcow2 -cdrom CentOS-5.3-i386-bin-DVD.iso -redir 'tcp:2225::22' -boot c -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user -nographic

Add to /boot/grub/menu.lst:

   serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1
   terminal --timeout=3 serial console

also add in menu.lst to kernel line (after removing `quiet splash'):

   console=tty0 console=ttyS0,115200n8

Add login prompt on serial console:

   cat >>/etc/inittab <<END

# Serial console.
S0:2345:respawn:/sbin/agetty -h -L ttyS0 19200 vt100
END

Create account.

   useradd buildbot
   # Password is disabled by default in Centos5.
   usermod -a -G wheel buildbot
   visudo
   # Uncomment the line "%wheel        ALL=(ALL)       NOPASSWD: ALL"
   # Comment out this line:
   # Defaults    requiretty

   # Put in public ssh key for own account and host buildbot account.
   # Note that Centos5 seems to require .ssh/authorized_keys chmod go-rwx.
   su - buildbot
   mkdir .ssh
   chmod go-rwx .ssh
   cat >.ssh/authorized_keys
   # Paste the id_dsa.pub key, see above.
   chmod go-rwx .ssh/authorized_keys

Image for rpm build

   qemu-img create -b vm-centos5-i386-serial.qcow2 -f qcow2 vm-centos5-i386-build.qcow2
   kvm -m 2047 -hda /kvm/vms/vm-centos5-i386-build.qcow2 -cdrom /kvm/CentOS-5.3-i386-bin-DVD.iso -redir 'tcp:2225::22' -boot c -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user -nographic

Install compilers etc:

   sudo yum groupinstall "Development Tools"
   sudo yum install gperf readline-devel ncurses-devel zlib-devel perl perl\(DBI\)

Download 5.0 rpm for shared-compat:

   sudo mkdir -p /srv/shared/yum/CentOS/5/i386/RPMS/
   cd /srv/shared/yum/CentOS/5/i386/RPMS/
   sudo wget http://mirror.ourdelta.org/yum/CentOS/5/i386/RPMS/MySQL-OurDelta-shared-5.0.87.d10-65.el5.i386.rpm

Image for install/test

   qemu-img create -b vm-centos5-i386-serial.qcow2 -f qcow2 vm-centos5-i386-install.qcow2
   kvm -m 2047 -hda /kvm/vms/vm-centos5-i386-install.qcow2 -cdrom /kvm/CentOS-5.3-i386-bin-DVD.iso -redir 'tcp:2225::22' -boot c -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user -nographic

Install extra dependencies:

   sudo yum install perl perl\(DBI\)

Debian 5 i386

Base install

Download netinst CD image debian-503-i386-netinst.iso and install:

   cd /kvm
   qemu-img create -f qcow2 vms/vm-debian5-i386-base.qcow2 8G
   kvm -m 2047 -hda /kvm/vms/vm-debian5-i386-base.qcow2 -cdrom /kvm/debian-503-i386-netinst.iso -redir 'tcp:2226::22' -boot d -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user

Serial port console

From base install, setup for serial port, and setup accounts for passwordless ssh login and sudo:

   qemu-img create -b vm-debian5-i386-base.qcow2 -f qcow2 vm-debian5-i386-serial.qcow2
   kvm -m 2047 -hda /kvm/vms/vm-debian5-i386-serial.qcow2 -redir 'tcp:2226::22' -boot c -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user -nographic
   su
   apt-get install sudo openssh-server emacs22-nox
   visudo
   # uncomment %sudo ALL=NOPASSWD: ALL
   # add user account to group sudo.
   # Copy in public ssh key.

   # Add in /etc/inittab:
   S0:2345:respawn:/sbin/agetty -h -L ttyS0 19200 vt100

Add to /boot/grub/menu.lst:

   serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1
   terminal --timeout=3 serial console

also add in menu.lst to kernel line (after removing `quiet splash'):

   console=tty0 console=ttyS0,115200n8

Add user buildbot, with disabled password. Add as sudo, and add ssh key.

   sudo adduser --disabled-password buildbot
   sudo adduser buildbot sudo
   sudo su - buildbot
   mkdir .ssh
   # Paste all necessary keys.
   cat >.ssh/authorized_keys
   chmod -R go-rwx .ssh

Image for .deb build

   qemu-img create -b vm-debian5-i386-serial.qcow2 -f qcow2 vm-debian5-i386-build.qcow2
   kvm -m 2047 -hda /kvm/vms/vm-debian5-i386-build.qcow2 -redir 'tcp:2226::22' -boot c -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user -nographic

   sudo apt-get build-dep mysql-server-5.0
   sudo apt-get install devscripts hardening-wrapper doxygen texlive-latex-base ghostscript libevent-dev libssl-dev zlib1g-dev libreadline5-dev

Image for install testing

   qemu-img create -b vm-debian5-i386-serial.qcow2 -f qcow2 vm-debian5-i386-install.qcow2
   kvm -m 2047 -hda /kvm/vms/vm-debian5-i386-install.qcow2 -redir 'tcp:2226::22' -boot c -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user -nographic

   # No packages mostly!
   sudo apt-get install debconf-utils
   cat >>/etc/apt/sources.list <<END
deb file:///home/buildbot/buildbot/debs binary/
deb-src file:///home/buildbot/buildbot/debs source/
END
   sudo debconf-set-selections /tmp/my.seed

See above for how to obtain my.seed.

Image for upgrade testing

   qemu-img create -b vm-debian5-i386-install.qcow2 -f qcow2 vm-debian5-i386-upgrade.qcow2
   kvm -m 2047 -hda /kvm/vms/vm-debian5-i386-upgrade.qcow2 -redir 'tcp:2226::22' -boot c -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user -nographic

   sudo apt-get install mysql-server-5.0
   mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"

Ubuntu 9.04 amd64

Base install

   qemu-img create -f qcow2 vm-jaunty-amd64-serial.qcow2 8G
   kvm -m 1024 -hda vm-jaunty-amd64-serial.qcow2 -cdrom /kvm/ubuntu-9.04-server-amd64.iso -redir tcp:2227::22 -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net user

Serial port and passwordless login/sudo

   kvm -m 1024 -hda vm-jaunty-amd64-serial.qcow2 -cdrom /kvm/ubuntu-9.04-server-amd64.iso -redir tcp:2227::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user

   sudo apt-get install emacs22-nox

Add to /boot/grub/menu.lst:

   serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1
   terminal --timeout=3 serial console

also add in menu.lst to kernel line (after removing `quiet splash'):

   console=tty0 console=ttyS0,115200n8

Create /etc/event.d/ttyS0:

   # ttyS0 - getty
   #
   # This service maintains a getty on ttyS0 from the point the system is
   # started until it is shut down again.

   start on stopped rc2
   start on stopped rc3
   start on stopped rc4
   start on stopped rc5

   stop on runlevel 0
   stop on runlevel 1
   stop on runlevel 6

   respawn
   exec /sbin/getty 115200 ttyS0

Add account and allow passwordless sudo:

   sudo adduser --disabled-password buildbot
   sudo adduser buildbot sudo
   sudo EDITOR=emacs visudo
   # uncomment `%sudo ALL=NOPASSWD: ALL' line in `visudo`, and move to end.

   sudo su -s /bin/bash - buildbot
   mkdir .ssh
   cat >.ssh/authorized_keys
   # Paste public key for buildbot user on host system.
   chmod -R go-rwx .ssh
   # Do manual login from host to guest once, to make sure .ssh/known_hosts is updated.

VM for .deb building

   qemu-img create -b vm-jaunty-amd64-serial.qcow2 -f qcow2 vm-jaunty-amd64-build.qcow2
   kvm -m 1024 -hda vm-jaunty-amd64-build.qcow2 -cdrom /kvm/ubuntu-9.04-server-amd64.iso -redir tcp:2227::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic

   sudo apt-get build-dep mysql-server-5.1

VM for install testing

   qemu-img create -b vm-jaunty-amd64-serial.qcow2 -f qcow2 vm-jaunty-amd64-install.qcow2
   kvm -m 1024 -hda vm-jaunty-amd64-install.qcow2 -cdrom /kvm/ubuntu-9.04-server-amd64.iso -redir tcp:2227::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic

   sudo adduser --system --group mysql

Ubuntu 8.04, 8.10, and 9.10; i386 and amd64

These 6 platforms were installed together.

Base installs

   qemu-img create -f qcow2 vm-hardy-amd64-serial.qcow2 8G
   kvm -m 1024 -hda vm-hardy-amd64-serial.qcow2 -cdrom /kvm/ubuntu-8.04.3-server-amd64.iso -redir tcp:2228::22 -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net user
   # Install, picking default options mostly, only adding openssh server.
   kvm -m 1024 -hda vm-hardy-amd64-serial.qcow2 -cdrom /kvm/ubuntu-8.04.3-server-amd64.iso -redir tcp:2228::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic
   ssh -p 2228 localhost
   # edit /boot/grub/menu.lst and visudo, see below
   sudo /sbin/shutdown -h now
   kvm -m 1024 -hda vm-hardy-amd64-serial.qcow2 -cdrom /kvm/ubuntu-8.04.3-server-amd64.iso -redir tcp:2228::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic
   ssh -t -p 2228 localhost "mkdir .ssh; sudo addgroup $USER sudo"
   scp -P 2228 authorized_keys localhost:.ssh/
   echo $'Buildbot\n\n\n\n\ny' | ssh -p 2228 localhost 'chmod -R go-rwx .ssh; sudo adduser --disabled-password buildbot; sudo addgroup buildbot sudo; sudo mkdir ~buildbot/.ssh; sudo cp .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -R buildbot:buildbot ~buildbot/.ssh; sudo chmod -R go-rwx ~buildbot/.ssh'
   scp -P 2228 ttyS0 buildbot@localhost:
   ssh -p 2228 buildbot@localhost 'sudo cp ttyS0 /etc/event.d; rm ttyS0; sudo shutdown -h now'

   qemu-img create -f qcow2 vm-hardy-i386-serial.qcow2 8G
   kvm -m 1024 -hda vm-hardy-i386-serial.qcow2 -cdrom /kvm/ubuntu-8.04.3-server-i386.iso -redir tcp:2229::22 -boot d -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user
   # Install, picking default options mostly, only adding openssh server.
   kvm -m 1024 -hda vm-hardy-i386-serial.qcow2 -cdrom /kvm/ubuntu-8.04.3-server-i386.iso -redir tcp:2229::22 -boot c -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user -nographic
   ssh -p 2229 localhost
   # edit /boot/grub/menu.lst and visudo, see below
   ssh -t -p 2229 localhost "mkdir .ssh; sudo addgroup $USER sudo"
   scp -P 2229 authorized_keys localhost:.ssh/
   echo $'Buildbot\n\n\n\n\ny' | ssh -p 2229 localhost 'chmod -R go-rwx .ssh; sudo adduser --disabled-password buildbot; sudo addgroup buildbot sudo; sudo mkdir ~buildbot/.ssh; sudo cp .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -R buildbot:buildbot ~buildbot/.ssh; sudo chmod -R go-rwx ~buildbot/.ssh'
   scp -P 2229 ttyS0 buildbot@localhost:
   ssh -p 2229 buildbot@localhost 'sudo cp ttyS0 /etc/event.d; rm ttyS0; sudo shutdown -h now'

   qemu-img create -f qcow2 vm-intrepid-amd64-serial.qcow2 8G
   kvm -m 1024 -hda vm-intrepid-amd64-serial.qcow2 -cdrom /kvm/ubuntu-8.10-server-amd64.iso -redir tcp:2230::22 -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net user
   # Install, picking default options mostly, only adding openssh server.
   kvm -m 1024 -hda vm-intrepid-amd64-serial.qcow2 -cdrom /kvm/ubuntu-8.10-server-amd64.iso -redir tcp:2230::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic
   ssh -p 2230 localhost
   # edit /boot/grub/menu.lst and visudo, see below
   ssh -t -p 2230 localhost "mkdir .ssh; sudo addgroup $USER sudo"
   scp -P 2230 authorized_keys localhost:.ssh/
   echo $'Buildbot\n\n\n\n\ny' | ssh -p 2230 localhost 'chmod -R go-rwx .ssh; sudo adduser --disabled-password buildbot; sudo addgroup buildbot sudo; sudo mkdir ~buildbot/.ssh; sudo cp .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -R buildbot:buildbot ~buildbot/.ssh; sudo chmod -R go-rwx ~buildbot/.ssh'
   scp -P 2230 ttyS0 buildbot@localhost:
   ssh -p 2230 buildbot@localhost 'sudo cp ttyS0 /etc/event.d; rm ttyS0; sudo shutdown -h now'

   qemu-img create -f qcow2 vm-intrepid-i386-serial.qcow2 8G
   kvm -m 1024 -hda vm-intrepid-i386-serial.qcow2 -cdrom /kvm/ubuntu-8.10-server-i386.iso -redir tcp:2231::22 -boot d -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user
   # Install, picking default options mostly, only adding openssh server.
   kvm -m 1024 -hda vm-intrepid-i386-serial.qcow2 -cdrom /kvm/ubuntu-8.10-server-i386.iso -redir tcp:2231::22 -boot c -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user -nographic
   ssh -p 2231 localhost
   # edit /boot/grub/menu.lst and visudo, see below
   ssh -t -p 2231 localhost "mkdir .ssh; sudo addgroup $USER sudo"
   scp -P 2231 authorized_keys localhost:.ssh/
   echo $'Buildbot\n\n\n\n\ny' | ssh -p 2231 localhost 'chmod -R go-rwx .ssh; sudo adduser --disabled-password buildbot; sudo addgroup buildbot sudo; sudo mkdir ~buildbot/.ssh; sudo cp .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -R buildbot:buildbot ~buildbot/.ssh; sudo chmod -R go-rwx ~buildbot/.ssh'
   scp -P 2231 ttyS0 buildbot@localhost:
   ssh -p 2231 buildbot@localhost 'sudo cp ttyS0 /etc/event.d; rm ttyS0; sudo shutdown -h now'

   qemu-img create -f qcow2 vm-karmic-amd64-serial.qcow2 8G
   kvm -m 1024 -hda vm-karmic-amd64-serial.qcow2 -cdrom /kvm/ubuntu-9.10-server-amd64.iso -redir tcp:2232::22 -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net user
   # Install, picking default options mostly, only adding openssh server.
   kvm -m 1024 -hda vm-karmic-amd64-serial.qcow2 -cdrom /kvm/ubuntu-9.10-server-amd64.iso -redir tcp:2232::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic
   ssh -p 2232 localhost
   # edit /boot/grub/menu.lst and visudo, see below
   ssh -t -p 2232 localhost "mkdir .ssh; sudo addgroup $USER sudo"
   scp -P 2232 authorized_keys localhost:.ssh/
   echo $'Buildbot\n\n\n\n\ny' | ssh -p 2232 localhost 'chmod -R go-rwx .ssh; sudo adduser --disabled-password buildbot; sudo addgroup buildbot sudo; sudo mkdir ~buildbot/.ssh; sudo cp .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -R buildbot:buildbot ~buildbot/.ssh; sudo chmod -R go-rwx ~buildbot/.ssh'
   scp -P 2232 ttyS0.conf buildbot@localhost:
   ssh -p 2232 buildbot@localhost 'sudo cp ttyS0.conf /etc/init/; rm ttyS0.conf; sudo shutdown -h now'

   qemu-img create -f qcow2 vm-karmic-i386-serial.qcow2 8G
   kvm -m 1024 -hda vm-karmic-i386-serial.qcow2 -cdrom /kvm/ubuntu-9.10-server-i386.iso -redir tcp:2233::22 -boot d -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user
   # Install, picking default options mostly, only adding openssh server.
   kvm -m 1024 -hda vm-karmic-i386-serial.qcow2 -cdrom /kvm/ubuntu-9.10-server-i386.iso -redir tcp:2233::22 -boot c -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user -nographic
   ssh -p 2233 localhost
   # edit /boot/grub/menu.lst and visudo, see below
   ssh -t -p 2233 localhost "mkdir .ssh; sudo addgroup $USER sudo"
   scp -P 2233 authorized_keys localhost:.ssh/
   echo $'Buildbot\n\n\n\n\ny' | ssh -p 2233 localhost 'chmod -R go-rwx .ssh; sudo adduser --disabled-password buildbot; sudo addgroup buildbot sudo; sudo mkdir ~buildbot/.ssh; sudo cp .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -R buildbot:buildbot ~buildbot/.ssh; sudo chmod -R go-rwx ~buildbot/.ssh'
   scp -P 2233 ttyS0.conf buildbot@localhost:
   ssh -p 2233 buildbot@localhost 'sudo cp ttyS0.conf /etc/init/; rm ttyS0.conf; sudo shutdown -h now'

For visudo to enable passwordless sudo:

   sudo VISUAL=vi visudo
   # uncomment `%sudo ALL=NOPASSWD: ALL' line in `visudo`, and move to end.

For 8.04/8.10/9.04, /boot/grub/menu.lst is edited as follows:

    sudo vi /boot/grub/menu.lst
    # Add this:
serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1
terminal --timeout=3 serial console
    # Also add in menu.lst to kernel line (after removing `quiet splash'):
console=tty0 console=ttyS0,115200n8

For 9.10, /boot/grub/menu.lst is edited as follows:

    sudo vi /etc/default/grub
    # Add/edit these entries:
GRUB_CMDLINE_LINUX_DEFAULT="console=tty0 console=ttyS0,115200n8"
GRUB_TERMINAL="serial"
GRUB_SERIAL_COMMAND="serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1"
    sudo update-grub

VMs for building .debs

   for i in 'vm-hardy-amd64-serial.qcow2 2228 qemu64' 'vm-hardy-i386-serial.qcow2 2229 qemu32,-nx' 'vm-intrepid-amd64-serial.qcow2 2230 qemu64' 'vm-intrepid-i386-serial.qcow2 2231 qemu32,-nx' ; do \
     set $i; \
     runvm --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/build/')" \
       "sudo DEBIAN_FRONTEND=noninteractive apt-get build-dep -y mysql-server-5.0" \
       "sudo DEBIAN_FRONTEND=noninteractive apt-get install -y devscripts hardening-wrapper fakeroot doxygen texlive-latex-base ghostscript libevent-dev libssl-dev zlib1g-dev libreadline5-dev" ; \
   done

   for i in 'vm-karmic-amd64-serial.qcow2 2232 qemu64' 'vm-karmic-i386-serial.qcow2 2233 qemu32,-nx' ; do \
     set $i; \
     runvm --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/build/')" \
       "sudo DEBIAN_FRONTEND=noninteractive apt-get update" \
       "sudo DEBIAN_FRONTEND=noninteractive apt-get -y build-dep mysql-server-5.1" \
       "sudo DEBIAN_FRONTEND=noninteractive apt-get install -y devscripts hardening-wrapper fakeroot doxygen texlive-latex-base ghostscript libevent-dev libssl-dev zlib1g-dev libreadline5-dev" ; \
   done

VMs for install testing

See above for how to make file 'my.seed' and 'sources.append'.

   for i in 'vm-hardy-amd64-serial.qcow2 2228 qemu64' 'vm-hardy-i386-serial.qcow2 2229 qemu32,-nx' 'vm-intrepid-amd64-serial.qcow2 2230 qemu64' 'vm-intrepid-i386-serial.qcow2 2231 qemu32,-nx' 'vm-karmic-amd64-serial.qcow2 2232 qemu64' 'vm-karmic-i386-serial.qcow2 2233 qemu32,-nx' ; do \
     set $i; \
     runvm --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/install/')" \

"sudo DEBIAN_FRONTEND=noninteractive apt-get update" \ "sudo DEBIAN_FRONTEND=noninteractive apt-get install -y debconf-utils" \ "= scp -P $2 my.seed sources.append buildbot@localhost:/tmp/" \ "sudo debconf-set-selections /tmp/my.seed" \ "sudo sh -c 'cat /tmp/sources.append >> /etc/apt/sources.list'"; \

   done

VMs for upgrade testing

   for i in 'vm-hardy-amd64-install.qcow2 2228 qemu64' 'vm-hardy-i386-install.qcow2 2229 qemu32,-nx' 'vm-intrepid-amd64-install.qcow2 2230 qemu64' 'vm-intrepid-i386-install.qcow2 2231 qemu32,-nx' ; do \
     set $i; \
     runvm --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/install/upgrade/')" \
     'sudo DEBIAN_FRONTEND=noninteractive apt-get install -y mysql-server-5.0'
     'mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"' ; \
   done

   for i in 'vm-karmic-amd64-install.qcow2 2232 qemu64' 'vm-karmic-i386-install.qcow2 2233 qemu32,-nx' ; do \
     set $i; \
     runvm --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/install/upgrade/')" \
       'sudo DEBIAN_FRONTEND=noninteractive apt-get install -y mysql-server-5.1' \
       'mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"' ; \
   done

Debian 5 amd64

Download netinst CD image debian-503-amd64-netinst.iso

   cd /kvm/vms
   qemu-img create -f qcow2 vm-debian5-amd64-serial.qcow2 8G
   kvm -m 2047 -hda /kvm/vms/vm-debian5-amd64-serial.qcow2 -cdrom /kvm/debian-503-amd64-netinst.iso -redir 'tcp:2234::22' -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net user

Serial console and account setup

From base install, setup for serial port, and setup accounts for passwordless ssh login and sudo:

   kvm -m 2047 -hda /kvm/vms/vm-debian5-amd64-serial.qcow2 -redir 'tcp:2234::22' -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user

   su
   apt-get install sudo openssh-server
   visudo
   # uncomment %sudo ALL=NOPASSWD: ALL
   # add user account to group sudo `addgroup <USER> sudo`.
   # Copy in public ssh key.

   # Add in /etc/inittab:
   S0:2345:respawn:/sbin/getty -L ttyS0 19200 vt100

Add to /boot/grub/menu.lst:

   serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1
   terminal --timeout=3 serial console

also add in menu.lst to kernel line (after removing `quiet splash'):

   console=tty0 console=ttyS0,115200n8

Add user buildbot, with disabled password. Add as sudo, and add ssh key.

   sudo adduser --disabled-password buildbot
   sudo adduser buildbot sudo
   sudo su - buildbot
   mkdir .ssh
   # Add all necessary keys.
   cat >.ssh/authorized_keys
   chmod -R go-rwx .ssh

VM for building .deb

   qemu-img create -b vm-debian5-amd64-serial.qcow2 -f qcow2 vm-debian5-amd64-build.qcow2
   kvm -m 2047 -hda /kvm/vms/vm-debian5-amd64-build.qcow2 -redir 'tcp:2234::22' -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic

   sudo apt-get build-dep mysql-server-5.0
   sudo apt-get install devscripts hardening-wrapper doxygen texlive-latex-base ghostscript libevent-dev libssl-dev zlib1g-dev libreadline5-dev

VM for install testing

   qemu-img create -b vm-debian5-amd64-serial.qcow2 -f qcow2 vm-debian5-amd64-install.qcow2
   kvm -m 2047 -hda /kvm/vms/vm-debian5-amd64-install.qcow2 -redir 'tcp:2234::22' -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic

   # No packages mostly!
   sudo apt-get install debconf-utils
   cat >>/etc/apt/sources.list <<END
deb file:///home/buildbot/buildbot/debs binary/
deb-src file:///home/buildbot/buildbot/debs source/
END
   sudo debconf-set-selections /tmp/my.seed

See above for how to obtain my.seed

VM for upgrade testing

   qemu-img create -b vm-debian5-amd64-install.qcow2 -f qcow2 vm-debian5-amd64-upgrade.qcow2
   kvm -m 2047 -hda /kvm/vms/vm-debian5-amd64-upgrade.qcow2 -redir 'tcp:2234::22' -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic

   sudo apt-get install mysql-server-5.0
   mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"

Ubuntu 10.04 (alpha), i386 and amd64

Base install

   qemu-img create -f qcow2 vm-lucid-amd64-serial.qcow2 8G
   kvm -m 1024 -hda vm-lucid-amd64-serial.qcow2 -cdrom /kvm/lucid-server-amd64.iso -redir tcp:2238::22 -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net user
   # Install, picking default options mostly, only adding openssh server.
   kvm -m 1024 -hda vm-lucid-amd64-serial.qcow2 -cdrom /kvm/lucid-server-amd64.iso -redir tcp:2238::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic
   ssh -p 2238 localhost
   # edit /boot/grub/menu.lst and visudo, see below
   ssh -t -p 2238 localhost "mkdir .ssh; sudo addgroup $USER sudo"
   scp -P 2238 authorized_keys localhost:.ssh/
   echo $'Buildbot\n\n\n\n\ny' | ssh -p 2238 localhost 'chmod -R go-rwx .ssh; sudo adduser --disabled-password buildbot; sudo addgroup buildbot sudo; sudo mkdir ~buildbot/.ssh; sudo cp .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -R buildbot:buildbot ~buildbot/.ssh; sudo chmod -R go-rwx ~buildbot/.ssh'
   scp -P 2238 ttyS0.conf buildbot@localhost:
   ssh -p 2238 buildbot@localhost 'sudo cp ttyS0.conf /etc/init/; rm ttyS0.conf; sudo shutdown -h now'

   qemu-img create -f qcow2 vm-lucid-i386-serial.qcow2 8G
   kvm -m 1024 -hda vm-lucid-i386-serial.qcow2 -cdrom /kvm/lucid-server-i386.iso -redir tcp:2239::22 -boot d -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user
   # Install, picking default options mostly, only adding openssh server.
   kvm -m 1024 -hda vm-lucid-i386-serial.qcow2 -cdrom /kvm/lucid-server-i386.iso -redir tcp:2239::22 -boot c -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user -nographic
   ssh -p 2239 localhost
   # edit /boot/grub/menu.lst and visudo, see below
   ssh -t -p 2239 localhost "mkdir .ssh; sudo addgroup $USER sudo"
   scp -P 2239 authorized_keys localhost:.ssh/
   echo $'Buildbot\n\n\n\n\ny' | ssh -p 2239 localhost 'chmod -R go-rwx .ssh; sudo adduser --disabled-password buildbot; sudo addgroup buildbot sudo; sudo mkdir ~buildbot/.ssh; sudo cp .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -R buildbot:buildbot ~buildbot/.ssh; sudo chmod -R go-rwx ~buildbot/.ssh'
   scp -P 2239 ttyS0.conf buildbot@localhost:
   ssh -p 2239 buildbot@localhost 'sudo cp ttyS0.conf /etc/init/; rm ttyS0.conf; sudo shutdown -h now'

Enabling passwordless sudo:

   sudo VISUAL=vi visudo
   # uncomment `%sudo ALL=NOPASSWD: ALL' line in `visudo`, and move to end.

Editing /boot/grub/menu.lst:

   sudo vi /etc/default/grub
   # Add/edit these entries:
GRUB_CMDLINE_LINUX_DEFAULT="console=tty0 console=ttyS0,115200n8"
GRUB_TERMINAL="serial"
GRUB_SERIAL_COMMAND="serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1"
   sudo update-grub

VMs for building .debs

   for i in 'vm-lucid-amd64-serial.qcow2 2238 qemu64' 'vm-lucid-i386-serial.qcow2 2239 qemu32,-nx' ; do \
     set $i; \
     runvm --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/build/')" \
       "sudo DEBIAN_FRONTEND=noninteractive apt-get update" \
       "sudo DEBIAN_FRONTEND=noninteractive apt-get -y build-dep mysql-server-5.1" \
       "sudo DEBIAN_FRONTEND=noninteractive apt-get install -y devscripts hardening-wrapper fakeroot doxygen texlive-latex-base ghostscript libevent-dev libssl-dev zlib1g-dev libreadline5-dev" ; \
   done

VMs for install testing.

See above for how to obtain my.seed and sources.append.

   cat >sources.append <<END
deb file:///home/buildbot/buildbot/debs binary/
deb-src file:///home/buildbot/buildbot/debs source/
END

   for i in 'vm-lucid-amd64-serial.qcow2 2238 qemu64' 'vm-lucid-i386-serial.qcow2 2239 qemu32,-nx' ; do \
     set $i; \
     runvm --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/install/')" \
       "sudo DEBIAN_FRONTEND=noninteractive apt-get update" \
       "sudo DEBIAN_FRONTEND=noninteractive apt-get install -y debconf-utils" \
       "= scp -P $2 my.seed sources.append buildbot@localhost:/tmp/" \
       "sudo debconf-set-selections /tmp/my.seed" \
       "sudo sh -c 'cat /tmp/sources.append >> /etc/apt/sources.list'"; \
   done

VMs for upgrade testing

   for i in 'vm-lucid-amd64-install.qcow2 2238 qemu64' 'vm-lucid-i386-install.qcow2 2239 qemu32,-nx' ; do \
     set $i; \
     runvm --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/install/upgrade/')" \
       'sudo DEBIAN_FRONTEND=noninteractive apt-get install -y mysql-server-5.1' \
       'mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"' ;\
   done

Ubuntu 10.10 "maverick", i386 and amd64

Base install

   qemu-img create -f qcow2 vm-maverick-amd64-serial.qcow2 8G
   kvm -m 1024 -hda vm-maverick-amd64-serial.qcow2 -cdrom /kvm/ubuntu-10.10-server-amd64.iso -redir tcp:2246::22 -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net user
   # Install, picking default options mostly, only adding openssh server.
   kvm -m 1024 -hda vm-maverick-amd64-serial.qcow2 -cdrom /kvm/ubuntu-10.10-server-amd64.iso -redir tcp:2246::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic
   ssh -p 2246 localhost
   # edit /boot/grub/menu.lst and visudo, see below
   ssh -t -p 2246 localhost "mkdir .ssh; sudo addgroup $USER sudo"
   scp -P 2246 authorized_keys localhost:.ssh/
   echo $'Buildbot\n\n\n\n\ny' | ssh -p 2246 localhost 'chmod -R go-rwx .ssh; sudo adduser --disabled-password buildbot; sudo addgroup buildbot sudo; sudo mkdir ~buildbot/.ssh; sudo cp .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -R buildbot:buildbot ~buildbot/.ssh; sudo chmod -R go-rwx ~buildbot/.ssh'
   scp -P 2246 ttyS0.conf buildbot@localhost:
   ssh -p 2246 buildbot@localhost 'sudo cp ttyS0.conf /etc/init/; rm ttyS0.conf; sudo shutdown -h now'

   qemu-img create -f qcow2 vm-maverick-i386-serial.qcow2 8G
   kvm -m 1024 -hda vm-maverick-i386-serial.qcow2 -cdrom /kvm/ubuntu-10.10-server-i386.iso -redir tcp:2247::22 -boot d -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user
   # Install, picking default options mostly, only adding openssh server.
   kvm -m 1024 -hda vm-maverick-i386-serial.qcow2 -cdrom /kvm/ubuntu-10.10-server-i386.iso -redir tcp:2247::22 -boot c -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user -nographic
   ssh -p 2247 localhost
   # edit /boot/grub/menu.lst and visudo, see below
   ssh -t -p 2247 localhost "mkdir .ssh; sudo addgroup $USER sudo"
   scp -P 2247 authorized_keys localhost:.ssh/
   echo $'Buildbot\n\n\n\n\ny' | ssh -p 2247 localhost 'chmod -R go-rwx .ssh; sudo adduser --disabled-password buildbot; sudo addgroup buildbot sudo; sudo mkdir ~buildbot/.ssh; sudo cp .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -R buildbot:buildbot ~buildbot/.ssh; sudo chmod -R go-rwx ~buildbot/.ssh'
   scp -P 2247 ttyS0.conf buildbot@localhost:
   ssh -p 2247 buildbot@localhost 'sudo cp ttyS0.conf /etc/init/; rm ttyS0.conf; sudo shutdown -h now'

Enabling passwordless sudo:

   sudo VISUAL=vi visudo
   # Add line at end: `%sudo ALL=NOPASSWD: ALL'

Editing /boot/grub/menu.lst:

   sudo vi /etc/default/grub
   # Add/edit these entries:
GRUB_CMDLINE_LINUX_DEFAULT="console=tty0 console=ttyS0,115200n8"
GRUB_TERMINAL="serial"
GRUB_SERIAL_COMMAND="serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1"
   sudo update-grub

VMs for building .debs

   for i in 'vm-maverick-amd64-serial.qcow2 2246 qemu64' 'vm-maverick-i386-serial.qcow2 2247 qemu32,-nx' ; do \
     set $i; \
     runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/build/')" \
       "sudo DEBIAN_FRONTEND=noninteractive apt-get update" \
       "sudo DEBIAN_FRONTEND=noninteractive apt-get -y build-dep mysql-server-5.1" \
       "sudo DEBIAN_FRONTEND=noninteractive apt-get install -y devscripts hardening-wrapper fakeroot doxygen texlive-latex-base ghostscript libevent-dev libssl-dev zlib1g-dev libreadline6-dev" ; \
   done

VMs for install testing.

See above for how to obtain my.seed and sources.append.

   cat >sources.append <<END
deb file:///home/buildbot/buildbot/debs binary/
deb-src file:///home/buildbot/buildbot/debs source/
END

   for i in 'vm-maverick-amd64-serial.qcow2 2246 qemu64' 'vm-maverick-i386-serial.qcow2 2247 qemu32,-nx' ; do \
     set $i; \
     runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/install/')" \
       "sudo DEBIAN_FRONTEND=noninteractive apt-get update" \
       "sudo DEBIAN_FRONTEND=noninteractive apt-get install -y debconf-utils" \
       "= scp -P $2 my.seed sources.append buildbot@localhost:/tmp/" \
       "sudo debconf-set-selections /tmp/my.seed" \
       "sudo sh -c 'cat /tmp/sources.append >> /etc/apt/sources.list'"; \
   done

VMs for upgrade testing

   for i in 'vm-maverick-amd64-install.qcow2 2246 qemu64' 'vm-maverick-i386-install.qcow2 2247 qemu32,-nx' ; do \
     set $i; \
     runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/install/upgrade/')" \
       'sudo DEBIAN_FRONTEND=noninteractive apt-get install -y mysql-server-5.1' \
       'mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"' ;\
   done

Ubuntu 11.04 "natty", i386 and amd64

Base install

   qemu-img create -f qcow2 vm-natty-amd64-serial.qcow2 8G
   kvm -m 1024 -hda vm-natty-amd64-serial.qcow2 -cdrom /kvm/iso/ubuntu-11.04-server-amd64.iso -redir tcp:2255::22 -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net user
   # Install, picking default options mostly, only adding openssh server.
   kvm -m 1024 -hda vm-natty-amd64-serial.qcow2 -cdrom /kvm/iso/ubuntu-11.04-server-amd64.iso -redir tcp:2255::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic
   ssh -p 2255 localhost
   # edit /boot/grub/menu.lst and visudo, see below
   ssh -t -p 2255 localhost "mkdir .ssh; sudo addgroup $USER sudo"
   scp -P 2255 authorized_keys localhost:.ssh/
   echo $'Buildbot\n\n\n\n\ny' | ssh -p 2255 localhost 'chmod -R go-rwx .ssh; sudo adduser --disabled-password buildbot; sudo addgroup buildbot sudo; sudo mkdir ~buildbot/.ssh; sudo cp .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -R buildbot:buildbot ~buildbot/.ssh; sudo chmod -R go-rwx ~buildbot/.ssh'
   scp -P 2255 ttyS0.conf buildbot@localhost:
   ssh -p 2255 buildbot@localhost 'sudo cp ttyS0.conf /etc/init/; rm ttyS0.conf; sudo shutdown -h now'

   qemu-img create -f qcow2 vm-natty-i386-serial.qcow2 8G
   kvm -m 1024 -hda vm-natty-i386-serial.qcow2 -cdrom /kvm/iso/ubuntu-11.04-server-i386.iso -redir tcp:2256::22 -boot d -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user
   # Install, picking default options mostly, only adding openssh server.
   kvm -m 1024 -hda vm-natty-i386-serial.qcow2 -cdrom /kvm/iso/ubuntu-11.04-server-i386.iso -redir tcp:2256::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic
   ssh -p 2256 localhost
   # edit /boot/grub/menu.lst and visudo, see below
   ssh -t -p 2256 localhost "mkdir .ssh; sudo addgroup $USER sudo"
   scp -P 2256 authorized_keys localhost:.ssh/
   echo $'Buildbot\n\n\n\n\ny' | ssh -p 2256 localhost 'chmod -R go-rwx .ssh; sudo adduser --disabled-password buildbot; sudo addgroup buildbot sudo; sudo mkdir ~buildbot/.ssh; sudo cp .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -R buildbot:buildbot ~buildbot/.ssh; sudo chmod -R go-rwx ~buildbot/.ssh'
   scp -P 2256 ttyS0.conf buildbot@localhost:
   ssh -p 2256 buildbot@localhost 'sudo cp ttyS0.conf /etc/init/; rm ttyS0.conf; sudo shutdown -h now'

Enabling passwordless sudo:

   sudo VISUAL=vi visudo
   # Add line at end: `%sudo ALL=NOPASSWD: ALL'

Editing /boot/grub/menu.lst:

   sudo vi /etc/default/grub
   # Add/edit these entries:
GRUB_CMDLINE_LINUX_DEFAULT="console=tty0 console=ttyS0,115200n8"
GRUB_TERMINAL="serial"
GRUB_SERIAL_COMMAND="serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1"
   sudo update-grub

VMs for building .debs

   for i in 'vm-natty-amd64-serial.qcow2 2255 qemu64' 'vm-natty-i386-serial.qcow2 2256 qemu64' ; do \
     set $i; \
     runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/build/')" \
       "sudo DEBIAN_FRONTEND=noninteractive apt-get update" \
       "sudo DEBIAN_FRONTEND=noninteractive apt-get -y build-dep mysql-server-5.1" \
       "sudo DEBIAN_FRONTEND=noninteractive apt-get install -y devscripts hardening-wrapper fakeroot doxygen texlive-latex-base ghostscript libevent-dev libssl-dev zlib1g-dev libreadline6-dev" ; \
   done

VMs for install testing.

See above for how to obtain my.seed and sources.append.

   for i in 'vm-natty-amd64-serial.qcow2 2255 qemu64' 'vm-natty-i386-serial.qcow2 2256 qemu64' ; do \
     set $i; \
     runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/install/')" \
       "sudo DEBIAN_FRONTEND=noninteractive apt-get update" \
       "sudo DEBIAN_FRONTEND=noninteractive apt-get install -y debconf-utils" \
       "= scp -P $2 my.seed sources.append buildbot@localhost:/tmp/" \
       "sudo debconf-set-selections /tmp/my.seed" \
       "sudo sh -c 'cat /tmp/sources.append >> /etc/apt/sources.list'"; \
   done

VMs for upgrade testing

   for i in 'vm-natty-amd64-install.qcow2 2255 qemu64' 'vm-natty-i386-install.qcow2 2256 qemu64' ; do \
     set $i; \
     runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/install/upgrade/')" \
       'sudo DEBIAN_FRONTEND=noninteractive apt-get install -y mysql-server-5.1' \
       'mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"' ;\
   done

Centos 5 amd64

Base install

   cd /kvm
   wget http://ftp.klid.dk/ftp/centos/5.3/isos/x86_64/CentOS-5.3-x86_64-bin-DVD.iso
   qemu-img create -f qcow2 vms/vm-centos5-amd64-serial.qcow2 8G
   kvm -m 2047 -hda /kvm/vms/vm-centos5-amd64-serial.qcow2 -cdrom CentOS-5.3-x86_64-bin-DVD.iso -redir 'tcp:2237::22' -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net user

Serial port and account setup

   kvm -m 2047 -hda /kvm/vms/vm-centos5-amd64-serial.qcow2 -cdrom CentOS-5.3-i386-bin-DVD.iso -redir 'tcp:2237::22' -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic

Add to /boot/grub/menu.lst:

   serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1
   terminal --timeout=3 serial console

also add in menu.lst to kernel line (after removing `quiet splash'):

   console=tty0 console=ttyS0,115200n8

Run these commands:

   cat >>/etc/inittab <<END

# Serial console.
S0:2345:respawn:/sbin/agetty -h -L ttyS0 19200 vt100
END
   useradd buildbot
   # Password is disabled by default in Centos5.
   usermod -a -G wheel buildbot
   visudo
   # Uncomment the line "%wheel        ALL=(ALL)       NOPASSWD: ALL"
   # Comment out this line:
   # Defaults    requiretty

   # Put in public ssh key for own account and host buildbot account.
   # Note that Centos5 seems to require .ssh/authorized_keys chmod go-rwx.
   su - buildbot
   mkdir .ssh
   chmod go-rwx .ssh
   cat >.ssh/authorized_keys
   chmod go-rwx .ssh/authorized_keys

Image for rpm build

   qemu-img create -b vm-centos5-amd64-serial.qcow2 -f qcow2 vm-centos5-amd64-build.qcow2
   kvm -m 2047 -hda /kvm/vms/vm-centos5-amd64-build.qcow2 -cdrom /kvm/CentOS-5.3-x86_64-bin-DVD.iso -redir 'tcp:2237::22' -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic

Install compilers etc:

   sudo yum groupinstall "Development Tools"
   sudo yum install gperf readline-devel ncurses-devel zlib-devel perl perl\(DBI\)

Download 5.0 rpm for shared-compat:

   sudo mkdir -p /srv/shared/yum/CentOS/5/x86_64/RPMS/
   cd /srv/shared/yum/CentOS/5/x86_64/RPMS/
   sudo wget http://mirror.ourdelta.org/yum/CentOS/5/x86_64/RPMS/MySQL-OurDelta-shared-5.0.87.d10-65.el5.x86_64.rpm

Image for install/test

   qemu-img create -b vm-centos5-amd64-serial.qcow2 -f qcow2 vm-centos5-amd64-install.qcow2
   kvm -m 2047 -hda /kvm/vms/vm-centos5-amd64-install.qcow2 -cdrom /kvm/CentOS-5.3-x86_64-bin-DVD.iso -redir 'tcp:2237::22' -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic

Install extra dependencies:

   sudo yum install perl perl\(DBI\)

Debian 4 amd64

   cd /kvm/vms
   qemu-img create -f qcow2 vm-debian4-amd64-serial.qcow2 8G
   kvm -m 2047 -hda /kvm/vms/vm-debian4-amd64-serial.qcow2 -cdrom /kvm/debian-40r8-amd64-netinst.iso -redir 'tcp:2240::22' -boot d -smp 2 -cpu qemu64 -net nic,model=e1000 -net user

Serial console and account setup

From base install, setup for serial port, and setup accounts for passwordless ssh login and sudo:

   kvm -m 2047 -hda /kvm/vms/vm-debian4-amd64-serial.qcow2 -cdrom /kvm/debian-40r8-amd64-netinst.iso -redir 'tcp:2240::22' -boot c -smp 2 -cpu qemu64 -net nic,model=e1000 -net user

   su
   apt-get install sudo openssh-server
   VISUAL=vi visudo
   # Add at the end:t %sudo ALL=NOPASSWD: ALL

   # Add account <USER> to group sudo
   # Copy in public ssh key.

   # Add in /etc/inittab:
   S0:2345:respawn:/sbin/getty -L ttyS0 19200 vt100

Add to /boot/grub/menu.lst:

   serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1
   terminal --timeout=3 serial console

also add in menu.lst to kernel line:

   console=tty0 console=ttyS0,115200n8

   # Add user buildbot, with disabled password. Add as sudo, and add ssh key.
   sudo /usr/sbin/adduser --disabled-password buildbot
   sudo /usr/sbin/adduser buildbot sudo
   sudo su - buildbot
   mkdir .ssh
   # Add all necessary keys.
   cat >.ssh/authorized_keys
   chmod -R go-rwx .ssh

VM for build

   qemu-img create -b vm-debian4-amd64-serial.qcow2 -f qcow2 vm-debian4-amd64-build.qcow2
   kvm -m 2047 -hda /kvm/vms/vm-debian4-amd64-build.qcow2 -cdrom /kvm/debian-40r8-amd64-netinst.iso -redir 'tcp:2240::22' -boot c -smp 2 -cpu qemu64 -net nic,model=e1000 -net user -nographic

   sudo apt-get build-dep mysql-server-5.0
   # Some latex packages fail to install because they complain that the
   # source is more than 5 years old! I solved by setting back the clock a
   # couple of years temporarily ...
   sudo apt-get install devscripts doxygen texlive-latex-base gs lsb-release fakeroot libevent-dev libssl-dev zlib1g-dev libreadline5-dev

VM for install testing

   qemu-img create -b vm-debian4-amd64-serial.qcow2 -f qcow2 vm-debian4-amd64-install.qcow2
   kvm -m 2047 -hda /kvm/vms/vm-debian4-amd64-install.qcow2 -cdrom /kvm/debian-40r8-amd64-netinst.iso -redir 'tcp:2240::22' -boot c -smp 2 -cpu qemu64 -net nic,model=e1000 -net user -nographic

See above for how to prepare file "my.seed".

   # No packages mostly!
   sudo apt-get install debconf-utils
   cat >>/etc/apt/sources.list <<END
deb file:///home/buildbot/buildbot/debs binary/
deb-src file:///home/buildbot/buildbot/debs source/
END
   sudo debconf-set-selections /tmp/my.seed

VM for upgrade testing

   qemu-img create -b vm-debian4-amd64-install.qcow2 -f qcow2 vm-debian4-amd64-upgrade.qcow2
   kvm -m 2047 -hda /kvm/vms/vm-debian4-amd64-upgrade.qcow2 -cdrom /kvm/debian-40r8-amd64-netinst.iso -redir 'tcp:2240::22' -boot c -smp 2 -cpu qemu64 -net nic,model=e1000 -net user -nographic

   sudo apt-get install mysql-server-5.0
   mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"

Debian 4 i386

   cd /kvm/vms
   qemu-img create -f qcow2 vm-debian4-i386-serial.qcow2 8G
   kvm -m 2047 -hda /kvm/vms/vm-debian4-i386-serial.qcow2 -cdrom /kvm/debian-40r8-i386-netinst.iso -redir 'tcp:2241::22' -boot d -smp 2 -cpu qemu32,-nx -net nic,model=e1000 -net user

Serial console and account setup

From base install, setup for serial port, and setup accounts for passwordless ssh login and sudo:

   kvm -m 2047 -hda /kvm/vms/vm-debian4-i386-serial.qcow2 -cdrom /kvm/debian-40r8-i386-netinst.iso -redir 'tcp:2241::22' -boot c -smp 2 -cpu qemu32,-nx -net nic,model=e1000 -net user

   su
   apt-get install sudo openssh-server
   VISUAL=vi visudo
   # Add at the end:t %sudo ALL=NOPASSWD: ALL

   # Add account <USER> to group sudo
   # Copy in public ssh key.

   # Add in /etc/inittab:
   S0:2345:respawn:/sbin/getty -L ttyS0 19200 vt100

Add to /boot/grub/menu.lst:

   serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1
   terminal --timeout=3 serial console

also add in menu.lst to kernel line:

   console=tty0 console=ttyS0,115200n8

Do these steps:

   # Add user buildbot, with disabled password. Add as sudo, and add ssh key.
   sudo /usr/sbin/adduser --disabled-password buildbot
   sudo /usr/sbin/adduser buildbot sudo
   sudo su - buildbot
   mkdir .ssh
   # Add all necessary keys.
   cat >.ssh/authorized_keys
   chmod -R go-rwx .ssh

VM for build

   qemu-img create -b vm-debian4-i386-serial.qcow2 -f qcow2 vm-debian4-i386-build.qcow2
   kvm -m 2047 -hda /kvm/vms/vm-debian4-i386-build.qcow2 -cdrom /kvm/debian-40r8-i386-netinst.iso -redir 'tcp:2241::22' -boot c -smp 2 -cpu qemu32,-nx -net nic,model=e1000 -net user -nographic

   sudo apt-get build-dep mysql-server-5.0
   # Some latex packages fail to install because they complain that the
   # source is more than 5 years old! I solved by setting back the clock a
   # couple of years temporarily ...
   sudo apt-get install devscripts doxygen texlive-latex-base gs lsb-release fakeroot libevent-dev libssl-dev zlib1g-dev libreadline5-dev

VM for install testing

   qemu-img create -b vm-debian4-i386-serial.qcow2 -f qcow2 vm-debian4-i386-install.qcow2
   kvm -m 2047 -hda /kvm/vms/vm-debian4-i386-install.qcow2 -cdrom /kvm/debian-40r8-i386-netinst.iso -redir 'tcp:2241::22' -boot c -smp 2 -cpu qemu32,-nx -net nic,model=e1000 -net user -nographic

See above for how to obtain file "my.seed".

   # No packages mostly!
   sudo apt-get install debconf-utils
   cat >>/etc/apt/sources.list <<END
deb file:///home/buildbot/buildbot/debs binary/
deb-src file:///home/buildbot/buildbot/debs source/
END
   sudo debconf-set-selections /tmp/my.seed

VM for upgrade testing

   qemu-img create -b vm-debian4-i386-install.qcow2 -f qcow2 vm-debian4-i386-upgrade.qcow2
   kvm -m 2047 -hda /kvm/vms/vm-debian4-i386-upgrade.qcow2 -cdrom /kvm/debian-40r8-i386-netinst.iso -redir 'tcp:2241::22' -boot c -smp 2 -cpu qemu64 -net nic,model=e1000 -net user -nographic

   sudo apt-get install mysql-server-5.0
   mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"

Debian 6 amd64

  cd /kvm/vms
  qemu-img create -f qcow2 vm-debian6-amd64-serial.qcow2 8G
  kvm -m 2047 -hda /kvm/vms/vm-debian6-amd64-serial.qcow2 -cdrom /kvm/debian-6a1-amd64-netinst.iso -redir 'tcp:2244::22' -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net user

Serial console and account setup

From base install, setup for serial port, and setup accounts for passwordless ssh login and sudo:

  kvm -m 2047 -hda /kvm/vms/vm-debian6-amd64-serial.qcow2 -redir 'tcp:2244::22' -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user

  su
  apt-get install sudo openssh-server
  visudo
  # Add: %sudo ALL=NOPASSWD: ALL
  # add user account to group sudo `addgroup <USER> sudo`.
  # Copy in public ssh key.
  
  # Add in /etc/inittab:
  S0:2345:respawn:/sbin/getty -L ttyS0 19200 vt100

Editing /etc/default/grub:

  sudo vi /etc/default/grub
  # Add/edit these entries:
GRUB_CMDLINE_LINUX_DEFAULT="console=tty0 console=ttyS0,115200n8"
GRUB_TERMINAL="serial"
GRUB_SERIAL_COMMAND="serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1"
  sudo update-grub

Add user buildbot, with disabled password. Add as sudo, and add ssh key.

  sudo adduser --disabled-password buildbot
  sudo adduser buildbot sudo
  sudo su - buildbot
  mkdir .ssh
  # Add all necessary keys.
  cat >.ssh/authorized_keys
  chmod -R go-rwx .ssh

VM for building .deb

  qemu-img create -b vm-debian6-amd64-serial.qcow2 -f qcow2 vm-debian6-amd64-build.qcow2
  kvm -m 2047 -hda /kvm/vms/vm-debian6-amd64-build.qcow2 -redir 'tcp:2244::22' -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic

  sudo apt-get build-dep mysql-server-5.1
  sudo apt-get install devscripts hardening-wrapper doxygen texlive-latex-base ghostscript libevent-dev libssl-dev zlib1g-dev libreadline-dev

VM for install testing

  qemu-img create -b vm-debian6-amd64-serial.qcow2 -f qcow2 vm-debian6-amd64-install.qcow2
  kvm -m 2047 -hda /kvm/vms/vm-debian6-amd64-install.qcow2 -redir 'tcp:2244::22' -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic

See above for how to obtain my.seed

  # No packages mostly!
  sudo apt-get install debconf-utils
  cat >>/etc/apt/sources.list <<END
deb file:///home/buildbot/buildbot/debs binary/
deb-src file:///home/buildbot/buildbot/debs source/
END
  sudo debconf-set-selections /tmp/my.seed

VM for upgrade testing

  qemu-img create -b vm-debian6-amd64-install.qcow2 -f qcow2 vm-debian6-amd64-upgrade.qcow2
  kvm -m 2047 -hda /kvm/vms/vm-debian6-amd64-upgrade.qcow2 -redir 'tcp:2244::22' -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic

  sudo apt-get install mysql-server-5.1
  mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"

Debian 6 i386

  cd /kvm/vms
  qemu-img create -f qcow2 vm-debian6-i386-serial.qcow2 8G
  kvm -m 2047 -hda /kvm/vms/vm-debian6-i386-serial.qcow2 -cdrom /kvm/debian-6a1-i386-netinst.iso -redir 'tcp:2245::22' -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net user

Serial console and account setup

From base install, setup for serial port, and setup accounts for passwordless ssh login and sudo:

  kvm -m 2047 -hda /kvm/vms/vm-debian6-i386-serial.qcow2 -redir 'tcp:2245::22' -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user

  su
  apt-get install sudo openssh-server
  visudo
  # Add: %sudo ALL=NOPASSWD: ALL
  # add user account to group sudo `addgroup <USER> sudo`.
  # Copy in public ssh key.

  # Add in /etc/inittab:
  S0:2345:respawn:/sbin/getty -L ttyS0 19200 vt100

Editing /etc/default/grub:

  sudo vi /etc/default/grub
  # Add/edit these entries:
GRUB_CMDLINE_LINUX_DEFAULT="console=tty0 console=ttyS0,115200n8"
GRUB_TERMINAL="serial"
GRUB_SERIAL_COMMAND="serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1"
  sudo update-grub

Add user buildbot, with disabled password. Add as sudo, and add ssh key.

  sudo adduser --disabled-password buildbot
  sudo adduser buildbot sudo
  sudo su - buildbot
  mkdir .ssh
  # Add all necessary keys.
  cat >.ssh/authorized_keys
  chmod -R go-rwx .ssh

VM for building .deb

  qemu-img create -b vm-debian6-i386-serial.qcow2 -f qcow2 vm-debian6-i386-build.qcow2
  kvm -m 2047 -hda /kvm/vms/vm-debian6-i386-build.qcow2 -redir 'tcp:2245::22' -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic

  sudo apt-get build-dep mysql-server-5.1
  sudo apt-get install devscripts hardening-wrapper doxygen texlive-latex-base ghostscript libevent-dev libssl-dev zlib1g-dev libreadline-dev

VM for install testing

  qemu-img create -b vm-debian6-i386-serial.qcow2 -f qcow2 vm-debian6-i386-install.qcow2
  kvm -m 2047 -hda /kvm/vms/vm-debian6-i386-install.qcow2 -redir 'tcp:2245::22' -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic

See above for how to obtain my.seed

  # No packages mostly!
  sudo apt-get install debconf-utils
  cat >>/etc/apt/sources.list <<END
deb file:///home/buildbot/buildbot/debs binary/
deb-src file:///home/buildbot/buildbot/debs source/
END
  sudo debconf-set-selections /tmp/my.seed

VM for upgrade testing

  qemu-img create -b vm-debian6-i386-install.qcow2 -f qcow2 vm-debian6-i386-upgrade.qcow2
  kvm -m 2047 -hda /kvm/vms/vm-debian6-i386-upgrade.qcow2 -redir 'tcp:2245::22' -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic

  sudo apt-get install mysql-server-5.1
  mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"

Additional steps

This section documents some additions that have been made after the initial setup.

Installing the Boost library needed for the OQGraph storage engine

The OQGraph storage engine needs a newer version of Boost that what is available on (most) distributions. The version installed is 1.42.0, available from http://www.boost.org/.

The boost library is installed in all the builder virtual machine images with the following single command:

   for i in "vm-hardy-amd64-build qemu64" "vm-hardy-i386-build qemu32,-nx" \
           "vm-intrepid-amd64-build qemu64" "vm-intrepid-i386-build qemu32,-nx" \
           "vm-karmic-amd64-build qemu64" "vm-karmic-i386-build qemu32,-nx" \
           "vm-jaunty-amd64-build qemu64" "vm-jaunty-i386-deb-build qemu32,-nx" \
           "vm-lucid-amd64-build qemu64" "vm-lucid-i386-build qemu32,-nx" \
           "vm-maverick-amd64-build qemu64" "vm-maverick-i386-build qemu32,-nx" \
           "vm-natty-amd64-build qemu64" "vm-natty-i386-build qemu64" \
           "vm-debian5-amd64-build qemu64" "vm-debian5-i386-build qemu32,-nx" \
           "vm-debian4-amd64-build qemu64 --netdev=e1000" "vm-debian4-i386-build qemu32,-nx --netdev=e1000" \
           "vm-centos5-i386-build qemu32,-nx" "vm-centos5-amd64-build qemu64" \
           "vm-hardy-amd64-build qemu64" "vm-hardy-i386-build qemu32,-nx" \
           "vm-jaunty-i386-deb-tarbake qemu32,-nx" ; do \
    set $i; \
    runvm -m 512 --smp=1 --port=2200 --user=buildbot --cpu=$2 $3 $1.qcow2 \
      "= scp -P 2200 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/boost_1_42_0.tar.gz buildbot@localhost:/dev/shm/" \
      "sudo mkdir -p /usr/local/src /usr/local/include" \
      "cd /usr/local/src && sudo tar zxf /dev/shm/boost_1_42_0.tar.gz" \
      "cd /usr/local/include && sudo ln -s ../src/boost_1_42_0/boost ." ; \
  done

Installing VM images for testing .deb upgrade between versions

This step creates virtual machine images used to do an important additional upgrade test for .debs. Each virtual machine is pre-installed with an older version of MariaDB, and the test tries upgrading this version to the newly build one (to check that dependencies etc. work out correctly).

This step may not be easily possible to replicate exactly for new or updated images. For example, when we add a new platform/distro, we will not have the same old version of MariaDB available for installation on the new planform. Or if needing to re-create an image in the future, the original old .debs used before may no longer be available. However, this is not a big problem, as we can just use whatever version of MariaDB is available. In fact, while we cannot reasonably test every possible upgrade combination between MariaDB versions, it is still useful to test upgrades from different versions on different platforms, to increase coverage a bit.

The bulk of the images were installed with the following two loops, using packages from the OurDelta repository for MariaDB 5.1.42, which was the one available from there at the time of installation.

 for i in "vm-hardy-amd64-install qemu64 hardy" "vm-hardy-i386-install qemu32,-nx hardy" \
          "vm-intrepid-amd64-install qemu64 intrepid" "vm-intrepid-i386-install qemu32,-nx intrepid" \
          "vm-karmic-amd64-install qemu64 karmic" "vm-karmic-i386-install qemu32,-nx karmic" \
          "vm-jaunty-amd64-install qemu64 jaunty" "vm-jaunty-i386-deb-install qemu32,-nx jaunty" \
          "vm-debian5-amd64-install qemu64 lenny" "vm-debian5-i386-install qemu32,-nx lenny" ; do \
   set $i; \
   runvm -b  $1.qcow2 -m 512 --smp=1 --port=2200 --user=buildbot --cpu=$2 $(echo $1 | sed -e 's/-install/-upgrade2/').qcow2 \
     "wget -O- http://ourdelta.org/deb/ourdelta.gpg | sudo apt-key add -" \
     "sudo sh -c \"echo 'deb http://mirror.ourdelta.org/deb $3 mariadb-ourdelta' > /etc/apt/sources.list.d/ourdelta.list\"" \
     "sudo apt-get update || true" \
     "sudo DEBIAN_FRONTEND=noninteractive apt-get install -y mariadb-server mariadb-test libmariadbclient-dev || true" \
     'mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"' \
     "sudo rm /etc/apt/sources.list.d/ourdelta.list" \
     "sudo apt-get update || true" \
     "sudo DEBIAN_FRONTEND=noninteractive apt-get upgrade -y || true" ; \
 done

 for i in "vm-debian4-amd64-install qemu64 etch /kvm/debian-40r8-amd64-netinst.iso" "vm-debian4-i386-install qemu32,-nx etch /kvm/debian-40r8-i386-netinst.iso" ; do \
   set $i; \
   runvm -b  $1.qcow2 -m 512 --smp=1 --port=2200 --user=buildbot --cpu=$2 --netdev=e1000 --kvm=-cdrom --kvm=$4 $(echo $1 | sed -e 's/-install/-upgrade2/').qcow2 \
     "wget -O- http://ourdelta.org/deb/ourdelta.gpg | sudo apt-key add -" \
     "sudo sh -c \"echo 'deb http://mirror.ourdelta.org/deb $3 mariadb-ourdelta' > /etc/apt/sources.list.d/ourdelta.list\"" \
     "sudo apt-get update || true" \
     "sudo sh -c 'DEBIAN_FRONTEND=noninteractive apt-get install -y mariadb-server mariadb-test libmariadbclient-dev' || true" \
     'mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"' \
     "sudo rm /etc/apt/sources.list.d/ourdelta.list" \
     "sudo apt-get update || true" \
     "sudo sh -c 'DEBIAN_FRONTEND=noninteractive apt-get upgrade -y' || true" ; \
 done

The Ubuntu 10.04 "lucid" images were installed manually (as no packages for lucid were available from OurDelta at the time of installation):

Create and boot 64-bit lucid upgrade image:

   qemu-img create -b vm-lucid-amd64-install.qcow2 -f qcow2 vm-lucid-amd64-upgrade2.qcow2 
   kvm -m 512 -hda vm-lucid-amd64-upgrade2.qcow2 -redir 'tcp:2200::22' -boot c -smp 1 -cpu qemu64 -net nic,model=virtio -net user -nographic

Create directory inside the image:

   mkdir buildbot

Copy in packages to be installed from outside:

   scp -P 2200 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -r /archive/pack/mariadb-5.1-knielsen/build-277/kvm-deb-lucid-amd64/debs buildbot@localhost:buildbot/

Install the MariaDB packages, remove package dir, and upgrade to latest security fixes:

   sudo apt-get update
   sudo DEBIAN_FRONTEND=noninteractive apt-get install -y --allow-unauthenticated mariadb-server mariadb-test libmariadbclient-dev
   mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"
   rm -Rf buildbot
   sudo apt-get update
   sudo DEBIAN_FRONTEND=noninteractive apt-get upgrade -y

Now the same, but for 32-bit lucid:

   qemu-img create -b vm-lucid-i386-install.qcow2 -f qcow2 vm-lucid-i386-upgrade2.qcow2 
   kvm -m 512 -hda vm-lucid-i386-upgrade2.qcow2 -redir 'tcp:2200::22' -boot c -smp 1 -cpu qemu32,-nx -net nic,model=virtio -net user -nographic

Create directory inside the image:

   mkdir buildbot

Copy in packages to be installed from outside:

   scp -P 2200 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -r /archive/pack/mariadb-5.1-knielsen/build-277/kvm-deb-lucid-x86/debs buildbot@localhost:buildbot/

Install the MariaDB packages, remove package dir, and upgrade to latest security fixes:

   sudo apt-get update
   sudo DEBIAN_FRONTEND=noninteractive apt-get install -y --allow-unauthenticated mariadb-server mariadb-test libmariadbclient-dev
   mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"
   rm -Rf buildbot
   sudo apt-get update
   sudo DEBIAN_FRONTEND=noninteractive apt-get upgrade -y

The Ubuntu 10.10 "maverick" images were likewise installed manually:

Create and boot 64-bit maverick upgrade image:

   qemu-img create -b vm-maverick-amd64-install.qcow2 -f qcow2 vm-maverick-amd64-upgrade2.qcow2
   kvm -m 512 -hda vm-maverick-amd64-upgrade2.qcow2 -redir 'tcp:2200::22' -boot c -smp 1 -cpu qemu64 -net nic,model=virtio -net user -nographic

   ssh -p 2200 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost mkdir buildbot
   scp -P 2200 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -r /archive/pack/mariadb-5.1-knielsen/build-619/kvm-deb-maverick-amd64/debs buildbot@localhost:buildbot/
   ssh -p 2200 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost

   sudo apt-get update
   sudo DEBIAN_FRONTEND=noninteractive apt-get install -y --allow-unauthenticated mariadb-server mariadb-test libmariadbclient-dev
   mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"
   rm -Rf buildbot
   sudo apt-get update
   sudo DEBIAN_FRONTEND=noninteractive apt-get upgrade -y

Same for 32-bit maverick:

   qemu-img create -b vm-maverick-i386-install.qcow2 -f qcow2 vm-maverick-i386-upgrade2.qcow2
   kvm -m 512 -hda vm-maverick-i386-upgrade2.qcow2 -redir 'tcp:2200::22' -boot c -smp 1 -cpu qemu32,-nx -net nic,model=virtio -net user -nographic

   ssh -p 2200 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost mkdir buildbot
   scp -P 2200 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -r /archive/pack/mariadb-5.1-knielsen/build-619/kvm-deb-maverick-x86/debs buildbot@localhost:buildbot/
   ssh -p 2200 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost

   sudo apt-get update
   sudo DEBIAN_FRONTEND=noninteractive apt-get install -y --allow-unauthenticated mariadb-server mariadb-test libmariadbclient-dev
   mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"
   rm -Rf buildbot
   sudo apt-get update
   sudo DEBIAN_FRONTEND=noninteractive apt-get upgrade -y

For Ubuntu 11.04 "natty", mariadb packages were installed from the repository for the previous version.

64-bit Ubuntu natty:

   qemu-img create -b vm-natty-amd64-install.qcow2 -f qcow2 vm-natty-amd64-upgrade2.qcow2
   kvm -m 512 -hda vm-natty-amd64-upgrade2.qcow2 -redir 'tcp:2200::22' -boot c -smp 1 -cpu qemu64 -net nic,model=virtio -net user -nographic
   ssh -p 2200 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost
   sudo sh -c 'cat > /etc/apt/sources.list.d/tmp.list'
deb http://ftp.osuosl.org/pub/mariadb/repo/5.1/ubuntu maverick main
deb-src http://ftp.osuosl.org/pub/mariadb/repo/5.1/ubuntu maverick main
   sudo apt-get update
   sudo DEBIAN_FRONTEND=noninteractive apt-get install -y --allow-unauthenticated mariadb-server mariadb-test libmariadbclient-dev
   mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"
   sudo rm /etc/apt/sources.list.d/tmp.list
   sudo apt-get update
   sudo DEBIAN_FRONTEND=noninteractive apt-get upgrade -y

32-bit Ubuntu natty:

   qemu-img create -b vm-natty-i386-install.qcow2 -f qcow2 vm-natty-i386-upgrade2.qcow2
   kvm -m 512 -hda vm-natty-i386-upgrade2.qcow2 -redir 'tcp:2200::22' -boot c -smp 1 -cpu qemu64 -net nic,model=virtio -net user -nographic
   ssh -p 2200 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost
   sudo sh -c 'cat > /etc/apt/sources.list.d/tmp.list'
deb http://ftp.osuosl.org/pub/mariadb/repo/5.1/ubuntu maverick main
deb-src http://ftp.osuosl.org/pub/mariadb/repo/5.1/ubuntu maverick main
   sudo apt-get update
   sudo DEBIAN_FRONTEND=noninteractive apt-get install -y --allow-unauthenticated mariadb-server mariadb-test libmariadbclient-dev
   mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"
   sudo rm /etc/apt/sources.list.d/tmp.list
   sudo apt-get update
   sudo DEBIAN_FRONTEND=noninteractive apt-get upgrade -y

Update Debian 4 mirrors

Debian 4 has become so old that the apt repository has been moved out of the main Debian 4 mirror servers, and into the archive of old versions. This needs to be fixed by pointing the Debian 4 images to a different mirror:

64-bit:

   kvm -m 512 -hda vm-debian4-amd64-install.qcow2 -redir 'tcp:2200::22' -boot c -smp 1 -cpu qemu64 -net nic,model=e1000 -net user -nographic

   sudo vi /etc/apt/sources.list
   # replace http://ftp.dk.debian.org/debian/ with http://ftp.de.debian.org/archive/debian/

32-bit:

   kvm -m 512 -hda vm-debian4-i386-install.qcow2 -redir 'tcp:2200::22' -boot c -smp 1 -cpu qemu64 -net nic,model=e1000 -net user -nographic

   sudo vi /etc/apt/sources.list
   # replace http://ftp.dk.debian.org/debian/ with http://ftp.de.debian.org/archive/debian/

After that, it is necessary to re-do from scratch the -update and -update2 debian4 images as per the above instructions (as these are built on top of the -install images).

Retrieved from "http://askmonty.org/wiki/BuildBot:vm-setup"

BuildBot:vm-setup

From Askmonty.org

Contents

Exact steps used to setup virtual machine images for the package testing in Buildbot

General principles

ssh

Serial port

ttyS0

ttyS0.conf

Unattended MariaDB install on Debian/Ubuntu

Emulation options

Detail of each vm image

Ubuntu 9.04 i386

Centos 5 i386

Base install

Configure for serial console

Image for rpm build

Image for install/test

Debian 5 i386

Base install

Serial port console

Image for .deb build

Image for install testing

Image for upgrade testing

Ubuntu 9.04 amd64

Base install

Serial port and passwordless login/sudo

VM for .deb building

VM for install testing

Ubuntu 8.04, 8.10, and 9.10; i386 and amd64

Base installs

VMs for building .debs

VMs for install testing

VMs for upgrade testing

Debian 5 amd64

Serial console and account setup

VM for building .deb

VM for install testing

VM for upgrade testing

Ubuntu 10.04 (alpha), i386 and amd64

Base install

VMs for building .debs

VMs for install testing.

VMs for upgrade testing

Ubuntu 10.10 "maverick", i386 and amd64

Base install

VMs for building .debs

VMs for install testing.

VMs for upgrade testing

Ubuntu 11.04 "natty", i386 and amd64

Base install

VMs for building .debs

VMs for install testing.

VMs for upgrade testing

Centos 5 amd64

Base install

Serial port and account setup

Image for rpm build

Image for install/test

Debian 4 amd64

Serial console and account setup

VM for build

VM for install testing

VM for upgrade testing

Debian 4 i386

Serial console and account setup

VM for build

VM for install testing

VM for upgrade testing

Debian 6 amd64

Serial console and account setup

VM for building .deb

VM for install testing

VM for upgrade testing

Debian 6 i386

Serial console and account setup

VM for building .deb

VM for install testing

VM for upgrade testing

Additional steps