Building a better internet exchange with OpenFlow

Internet exchanges

The internet is a collection of cables and routers - cables join the routers together, and routers decide where your data should go. Internet exchanges perform a crucial job in the middle of this - they provide a way for different (often competing) providers to exchange internet traffic. An internet exchange is typically implemented using some number of switches that are federated together somehow so that every port can exchange traffic with every other port as fast as they need to.

Problems with internet exchanges

Many internet exchanges are implemented as a layer-2 broadcast domain, and often include a route server to make it easier for participants to exchange routes with everyone over a single peering. Layer-2 broadcast domains need to be well policed though, because people can easily mess with other peoples' traffic, either accidentally, or maliciously.

Things that you need to watch out for on a layer-2 broadcast domain:

• Broadcast traffic - ARP requests are necessary, others probably aren't
• MAC spoofing
• ARP spoofing/proxy ARP
• STP & other lovely protocols

Things that you need at an IPv4/IPv6 peering exchange

• Unicast IPv4 and IPv6 traffic
• ARP requests (not necessarily broadcast)
• Legitimate ARP replies

Where's the button to turn this all on?

You'll often build an internet exchange with cheap and big switches, so you're not likely to have a huge amount of functionality to do clever stuff like this - and I've never seen a switch config before that lets me check the validity of ARP replies. To do this in software, however, is pretty easy.

Software Peering Exchange

Here's something I put together last night: https://github.com/samrussell/ryu/blob/master/ryu/app/spe.py

It's an extension of my OF1.2 learning switch, with an extra table to handle ARP entries. Here's how the tables are laid out:

The controller takes some simple JSON config of which IP is expected on each port, and loads these into the switch. Table 0 forwards all ARP packets to table 1, and table 1 checks the packets - if they're ARP requests, then it forwards them out the appropriate port (so ARP requests are effectively unicast across the internet exchange), and for ARP replies, it only allows each port to reply for its own IP address. Any packets that don't get forwarded out a port get sent to the controller (including valid ARP replies before we learn any MAC addresses, but the controller deals with this smartly & learns the MAC address properly).

This is combined with the learning switch that I posted earlier in the week, with one alteration - our source-MAC matches also have to match the ethertype, and this can be constrained in the controller to limit it to IPv4 or IPV6 (or either or - this could be done on a per port basis if you wanted to). Packets with known source-MACs are send to table 2, and then forwarded out the right port if we've learned their MAC - otherwise, it goes to the controller for processing.

What else can we do?

We can lock down the controller a bit to only allow IPv4, IPv6 and ARP - this would stop CDP, STP and the like leaking into the exchange - only valid traffic, and no spoofing.

We could go a step further though - we could build in ACLs on each port based on the routes that they announce for basic RPF, so that a participant can only route traffic from an IP range if they advertise that route to the exchange already - this would prevent asymmetric routing.

So go, have a play - see if you can break it or find something else to add!