Using Proxy ARP with Xen hypervisor on Linux

This article can be helpful for dedicated/virtual hosting users, who want to route several IP addresses via a single network card.

With certain network cards hardware Xen does not work in Bridged mode, or it would be more correct to say, that briding does not work. In this situation the network card will only be able to receive traffic for the main IP address, but not for the secondary IP, assigned to the Xen virtual machine.

For such situation, the solution is to use Proxy ARP functionality of Linux TCP/IP implementation and turn the non-virtualized distribution into a router. There is a standard way of doing so via vif-route script, however for some reason this failed to work for me. So the below solution is especially useful, when one uses GUI tools to configure Xen machines for bridging, since it's a drop-in replacement for bridge configuration.

The solution proposed below was tested with OpenSuSE distribution, but should equally work with another Linux distribution.

The steps are such:

  1. Please configure Xen regularly, with networking set up as a bridge.
  2. Download vif-proxyarp.tar.gz and unpack it to /etc/xen/scripts (or another Xen scripts directory according to the installed Linux distribution)
  3. Open Xen virtual machine configuration file. In OpenSuSE the file is located in /etc/xen/vm, and the necessary files to edit are those without xml extension. In CentOS, these files are located directly in /etc/xen. There will be a line present similar to this:
    vif=[ 'mac=00:16:3e:11:22:33,bridge=br0' ]
  4. From this line please note a bridge name and a MAC address. If the line does not contain the MAC address, it might be necessary to generate a new one. Xen virtual MAC address should be generated in this form: 00:16:3e:xx:xx:xx, please choose xx values to be random, or click here to generate a random Xen MAC address.
  5. Rewrite the line to look like this:
    vif=[ 'script=vif-proxyarp,mac=00:16:3e:11:22:33,ip=123.45.67.8,bridge=br0' ]
     
    It is important that the external IP address of the interface is also specified on this line. If a dedicated hosting is used, this address is allocated and given by the hosting company.  
  6. If it is required, notify the hosting company of a new MAC address of the virtual interface. Certain dedicated hosting providers need to know this MAC address in order to enable traffic routing.
  7. If there are more than one interfaces set up with Xen (vif=[ 'XXX', 'YYY', 'ZZZ' ]), then please only change the one, which has to be routed to the outside network (Internet). All local addresses (e.g. for backup or file exchange between virtual and host installations of Linux) do not require Proxy ARP.

After this is finished, doing traceroute to the Xen IP address should show the main non-virtualized IP address just one hop before.

Please note, if an automatic functionality is used to generate Xen configuration and Xen vm configuration files are this rewritten at some time - it might be necessary to apply this change once again.