12.07.2009

Building a virtual network with qemu

I do not know much about computer networks, but I try to learn things about them. They say, the best way to do is by building a real local network and experimenting with the settings of the software and the hardware. Unfortunately I just cannot do this. I do not have enough space, money, etc. As an alternative I have considered to play with emulated machines which I can run inside my desktop computer, so they do not require additional space. I am somewhat familiar with qemu, I used it earlier to have a look on different Linux distributions without rebooting my PC. I have chosen Debian Lenny as a base system, and I have downloaded the install CD iso from their project page. I have created a 3G size hard-disk image and I have installed the system in the same way, as if I would have done it on a real computer.

qemu-img create qemu_debian_501.img 3G # creating hdd image
qemu -hda qemu_debian_501.img -cdrom debian_cd.iso -boot d -m 256 # installing debian

I have done a minimal installation, since I do not really need any graphic program or interface at this stage on the system. I have run the installed system with the most basic method:

qemu -hda qemu_debian_501.img -m 256 # running debian

This way I will have an internet connection, so I can access anything on the net from the guest system. For example it is not a problem to use the apt to update the system and use the latest packages from the repositories. The first difference of this virtual computer to a real one is the way how it can be reached from the outside world. I had to learn that it is not as simple as I thought earlier. Since (in this configuration) the virtual machine does not have an own IP, we cannot reach it via the network as a normal physical computer. There are several solutions for this problem depending on our needs. Now I do not need much, I just want to access the virtual machine via the standard ssh from outside, mostly from the host machine itself. I have learnt that it can be done easily by redirecting some of the free ports of the host to any ports of the guest system:

qemu -hda qemu_debian_501.img -m 256 -redir tcp:2222::22 # running debian with 2222 to 22 port redirection

With this command the 2222 port of the host is redirected to the port 22 of our virtual debian, so the ssh access becomes possible. So now this simple command on the host will connect to the virtual system:

ssh -p 2222 localhost

Now I have the basic system and I can boot it up with a proper reach from outside. Now I have made three copies of the hard-disk image as sys1, sys2 and sys3.

cp qemu_debian_501.img qemu_debian_501_sys1.img
cp qemu_debian_501.img qemu_debian_501_sys2.img
cp qemu_debian_501.img qemu_debian_501_sys3.img

I will organize these three virtual system into a local network. There are also several solutions to this problem, but it was really hard (to me) to figure out the proper way from the qemu documentation. But it is there, and my efforts were rewarded with success! So I have dreamed a LAN, where sys1 is a gateway to the internet, while sys2 and sys3 are inside my virtual LAN. I want to reach sys1 with the aforementioned way by ssh, but in other respects the LAN should not be reachable from outside. For this configuration I need two network card in sys1, one connecting to the WAN, and the other is connecting to the LAN. Now we have to mention that the qemu environment have some mostly hidden, but very useful features, which are very handy in basic usage. When we fire up a virtual machine, qemu emulate us a network environment with a DHCP server. Since most operating systems tries to configure the first available network card with some DHCP client, this means that in most cases we have immediately network connection, so we can access the Internet from our guest system without any manual configuration. So in our case with the virtual network, connecting the sys1 with its first network card toward the WAN is easy. The other network card of sys1, and in addition the sys2 and sys3 systems will form a separate vlan, therefore we have to configure them independently from the aforementioned network environment. The simplest solution is to add static IP to the three cards from the 192.168.0.1... range, as we would do in a real physical network. Additionally we have to set up sys1 as a router between the two vlans.
The qemu virtual machines are independent program processes on the host system, so we have to let them know how they can communicate with each other. For that purposes if we create a vlan (as in this case), we have to assign to it a physical port on the host system. One of our qemu virtual machines will listen on that port, and the others can connect there, so they can do their network communication via this solution. The problem is that we have to start the instance of listening qemu first, and all the others just after that. (But we do not have to wait at all till the OS on the listening qemu boot up fully.) In addition if the listening qemu process crash for whatever reason, the vlan, which was administered by that process will fail to function, we have to reboot all the systems in the proper order to restart its functionality. In our scenario I have chosen sys1 as the listening qemu process for our vlan.
I can reach sys1 via ssh. I have tarted the three machines like this:

qemu -hda qemu_debian_501_sys1.img -m 256 -net nic,vlan=1 -net user,vlan=1 -net nic,vlan=2,macaddr=52:54:00:12:34:57 -net socket,vlan=2,listen=localhost:1234 -redir tcp:2222::22
qemu -hda qemu_debian_501_sys2.img -m 256 -net nic,vlan=2,macaddr=52:54:00:12:34:01 -net socket,vlan=2,connect=localhost:1234
qemu -hda qemu_debian_501_sys3.img -m 256 -net nic,vlan=2,macaddr=52:54:00:12:34:02 -net socket,vlan=2,connect=localhost:1234

I set up the network cards on the systems as follows:

On sys1 the eth0 card which connects the virtual network to the outside. It gets IP by DHCP client. I put this into the /etc/network/interfaces file:

# The primary network interface
allow-hotplug eth0
iface eth0 inet dhcp

The second card connects to the LAN, so the virtual network itself. I cannot set it up properly with the /etc/network/interfaces settings (maybe my fault), so I just put this line into /etc/rc.local:

ifconfig eth1 192.168.10.2

On the sys2 and sys3 I specified static IP for the network cards. On sys2 I put this into the /etc/network/interfaces :

# The primary network interface
allow-hotplug eth0
auto eth0
iface eth0 inet static
address 192.168.10.3
gateway 192.168.10.2
netmask 255.255.255.0
network 192.168.10.0
broadcast 192.168.10.255

Sys3 was set up in a similar way, with a different IP (192.168.10.4).

After these settings, my virtual network is ready to work.

Additionally I wanted to boot up the virtual machines in “headless mode”, so without the graphical window of qemu. This way I can set up the whole network with three computers, and I can log out from my account on the desktop, and the system is still up and running.

qemu -hda qemu_debian_501_sys1.img -m 256 -net nic,vlan=1 -net user,vlan=1 -net nic,vlan=2,macaddr=52:54:00:12:34:57 -net socket,vlan=2,listen=localhost:1234 -redir tcp:2222::22 -nographic
qemu -hda qemu_debian_501_sys2.img -m 256 -net nic,vlan=2,macaddr=52:54:00:12:34:01 -net socket,vlan=2,connect=localhost:1234 -nographic
qemu -hda qemu_debian_501_sys3.img -m 256 -net nic,vlan=2,macaddr=52:54:00:12:34:02 -net socket,vlan=2,connect=localhost:1234 -nographic

Now everything is perfect! I have a nice system, where I can experiment with the network. I can go in via ssh to localhost port 2222. The systems can see the internet, so I can update them etc.

I have used this page as reference.