set_trace # timfreund's blog on dev, ops, and other work stuff

Get a Routed KVM Network Talking to the Internet with DD-WRT and NAT

Posted: Thu, 08 Aug 2013

Today we configure a DD-WRT router to allow servers on a KVM virtual routed network access to the local network and the internet.

If you’ve found entry #3 in this libvirt networking FAQ and you’re now searching for the solution, you may be close.

First, we need to let the DD-WRT router know about the new network. This is under Setup -> Advanced Routing on my Buffalo device, and hopefully not far from there on your device.

Under “Static Routing” fill in appropriate values:

Route Name: your choice
Destination LAN NET: the network address of your KVM virtual routed network.
Subnet Mask: Probably 255.255.255.0, but you’ll know better if it isn’t.
Gateway: The LAN address of the host that runs your KVM virtual routed network.
Interface: LAN & WLAN

Save and apply those settings, then try to ping the router from a virtual machine on the virtual network and vice versa. Assuming both pings work, we’re ready for phase two.

Anything on the LAN can now talk to anything else on the LAN, but the virtual machines on the routed virtual network can’t talk to the Internet yet. Any attempts will time out. This assumes your DD-WRT router does NAT (Network Address Translation) for hosts on your LAN when they access the internet, but you’ll know better if it doesn’t and you probably wouldn’t be reading this right now.

The DD-WRT router knows to apply NAT to any traffic heading to the internet originating from the local network configured in the “Setup” -> “Basic Setup” -> “Network Setup” -> “Router IP” configuration area.
It does not apply NAT rules to any traffic originating on any other network that happens to come its way.

That configuration happens in the “Administration” -> “Commands” area of the administrative interface.

iptables -I FORWARD -s YOUR.KVM.NETWORK.ADDRESS/24 -j ACCEPT
iptables -t nat -A POSTROUTING -j SNAT --source YOUR.KVM.NETWORK.ADDRESS/24 --to-source YOUR.WAN.IP.ADDRESS

Bash

Click “Save Firewall” so the commands will run after each reboot, and click “Run Commands” to execute the commands immediately. Once complete, log in to one of your virtual machines and try to access any site on the Internet. With any luck it now works. I’m perhaps a bit lucky because I have a static IP address at home. I’m not sure how to dynamically update that command to include the correct WAN IP if you use DHCP. I’ll offer up the following bit of shell scripting in case it helps:

netstat -nr | grep ^0.0.0.0 | sed -e 's/0.0.0.0         //' -e 's/ .*//'

Bash

That dumps the routing table and extracts the default gateway for the device. I’ll update this post if you can tell me a better way to get that information.

Disable DHCP on a QEMU/libvirt/KVM Network

Posted: Sat, 27 Apr 2013

Libvirt + QEMU + KVM allows easy virtualization on Linux. Virtual machines are placed on a virtual network that comes complete with a DHCP server and DNS forwarder. There’s no reason to give up these default conveniences until your work involves building and configuring DNS and DHCP servers. Running two DHCP servers on the same virtual network is a recipe for frustration.

The configuration files for libvirt are stored in /etc/libvirt by default, but pretend like they aren’t even there. Read any of the files and you’ll find a warning:

<!--
WARNING: THIS IS AN AUTO-GENERATED FILE. CHANGES TO IT ARE LIKELY TO BE 
OVERWRITTEN AND LOST. Changes to this xml configuration should be made using:
  virsh net-edit devservers
or other application using the libvirt API.
-->

XML

They aren’t kidding. Don’t change these files. There’s one other trick:
Running the net-edit command alone won’t really do anything either if the network is already running. The following operations will do the trick:

root@host # virsh
virsh # net-destroy yournetwork
virsh # net-edit yournetwork
[remove the <dhcp></dhcp> element, save, exit]
virsh # net-start yournetwork

Bash

Rumor has it that later versions of the virt-manager GUI support these operations, but this info will come in handy if you’re still on a LTS system.

Reconnect any hosts that were on the virtual network, because they aren’t anymore. Configure your own DHCP server or configure all hosts with static interfaces as well. Remember that you’re on your own for DNS resolution in addition to DHCP at this point. An easy solution is to set domain-name-servers in your dhcpd.conf file to your router or access point.

Now that we’ve wrapped up our business here, I’d like to take a minute to tell you how happy all this stuff makes me. I’m running a small colony of machines on my middle of the line desktop computer, and I can practice the exact same work that happens in real data centers. Back when I was a newbie every machine was a real, physical machine, and PXE booting required careful hardware selection.

We’re living in the future. We just need the Hoverboard design team to catch up.