ARP and ping in POX - Building a POX-based OpenFlow router

What are we doing?

Today, we're going to look at how to handle ARP and ICMP ping messages in the OpenFlow controller POX. The results aren't amazing - latency is between 5 and 50 milliseconds (using pypy makes no difference) - but it's an important feature for any layer 3 device.

Why is it important?

If we want to make native router modules in OpenFlow, we need to be able to assign IP addresses to interfaces on our device. This means the router can talk IP to other devices on the network, a vital step towards building an OpenFlow router.

What about RouteFlow?

RouteFlow is a fully-functional OpenFlow router that you can use today, that translates the physical ports on your OpenFlow device to interfaces on a virtual machine. This virtual machine runs a software router daemon like Quagga or BIRD, meaning you can leverage a mature software router instead of making your own.

RouteFlow represents an important step in OpenFlow routing, but I think we can do better. RouteFlow polls the RIB on a virtual machine and translates that to OpenFlow, which means the router daemons don't know that they're talking to the controller.

If we build a clean interface, we can write POX modules for OSPF, IS-IS, BGP and the like, and let them talk directly to the controller.

How to make packets in POX

I love the packet library in POX, it's clean and easy to use. To make your own packets, just do what your network stack normally does - create the payload, wrap that in the layer below, then the layer below that, and once you're at Ethernet you're finished.

Step 1: ARP replies

I've started with the forwarding.l2_learning module from POX, and added some code to the _handle_PacketIn function, just under self.macToPort[packet.src] = event.port (so that MAC addresses are still stored for each new port).

match = of.ofp_match.from_packet(packet)

if ( match.dl_type == packet.ARP_TYPE and

match.nw_proto == arp.REQUEST and

match.nw_dst == IPAddr("10.1.1.253")):

  self.RespondToARP(packet, match, event)

  return

This checks for ARP requests for our hardcoded IP 10.1.1.253, and responds. The code to respond is as follows:

  def RespondToARP(self, packet, match, event):

    # reply to ARP request

    r = arp()

    r.opcode = arp.REPLY

    r.hwdst = match.dl_src

    r.protosrc = IPAddr("10.1.1.253")

    r.protodst = match.nw_src

    r.hwsrc = EthAddr("00:12:34:56:78:90")

    e = ethernet(type=packet.ARP_TYPE, src=r.hwsrc, dst=r.hwdst)

    e.set_payload(r)

    log.debug("%i %i answering ARP for %s" %

     ( event.dpid, event.port,

       str(r.protosrc)))

    msg = of.ofp_packet_out()

    msg.data = e.pack()

    msg.actions.append(of.ofp_action_output(port =

                                          of.OFPP_IN_PORT))

    msg.in_port = event.port

    event.connection.send(msg)

We build an ARP packet by calling the arp() function from pox.lib.packet, and it initialises the packet as follows:

def __init__(self, raw=None, prev=None, **kw):

        packet_base.__init__(self)

        self.prev = prev

        self.hwtype     = arp.HW_TYPE_ETHERNET

        self.prototype  = arp.PROTO_TYPE_IP

        self.hwsrc      = ETHER_ANY

        self.hwdst      = ETHER_ANY

        self.hwlen      = 6

        self.opcode     = 0

        self.protolen   = 4

        self.protosrc   = IP_ANY 

        self.protodst   = IP_ANY

        self.next       = b''

        if raw is not None:

            self.parse(raw)

        self._init(kw)

We just need to set the OPCODE, HWSRC, HWDST, PROTOSRC and PROTODST fields of this. I've done this in the body of the code, but we can simplify it by passing extra arguments as follows:

r = arp( opcode=arp.REPLY, 

         hwsrc=EthAddr("00:12:34:56:78:90"),

         hwdst=match.dl_src,

         protosrc = IPAddr("10.1.1.253"),

         protodst = match.nw_src)

Once we've created the ARP packet, we need to create an Ethernet packet to put this into. This isn't perfect (we should check for VLAN tags and add them, or steal the body of the original packet and modify that), but it works if we're just dealing with a straight Ethernet network.

e = ethernet(type=packet.ARP_TYPE, src=r.hwsrc, dst=r.hwdst)

e.set_payload(r)

Then we send this off to the controller, which sends it out the port it came through. Now we have an IP address that people can find, let's make it respond to something.

Step 2: Ping replies

If we can reply to ARP requests, we can reply to pings. This has a few more layers, but that just makes the code a little longer, not any more complicated.

The ARP reply is easy - we make an ARP packet, then put that in an Ethernet packet. For ping reply, this is what we do:

1. Get the payload from the echo request (ping)
2. Create an echo reply packet, insert the old payload
3. Create an ICMP packet, insert the echo reply
4. Create an IPv4 packet, insert the ICMP
5. Create an Ethernet packet, insert the IPv4

Here's what the code looks like:

  def RespondToPing(self, ping, match, event):

    p = ping

    # we know this is an ICMP Echo packet, so loop through

    # maybe this needs a try... except?

    while not isinstance(p, echo):

      p = p.next

    

    r = echo(id=p.id, seq=p.seq)

    r.set_payload(p.next)

    i = icmp(type=0, code=0)

    i.set_payload(r)

    ip = ipv4(protocol=ipv4.ICMP_PROTOCOL,

              srcip=IPAddr("10.1.1.253"),

              dstip=match.nw_src)

    ip.set_payload(i)

    e = ethernet(type=ping.IP_TYPE,

                 src=match.dl_dst,

                 dst=match.dl_src)

    e.set_payload(ip)

    log.debug("%i %i answering PING for %s" % (

              event.dpid, event.port,

              str(match.nw_src)))

    msg = of.ofp_packet_out()

    msg.data = e.pack()

    msg.actions.append(of.ofp_action_output(port =

                                          of.OFPP_IN_PORT))

    msg.in_port = event.port

    event.connection.send(msg)

Simple, just slightly longer than the ARP code.

Pictures

Here's a look at the controller output, and the view from Wireshark.

The OpenFlow dissector for Wireshark is part of the OpenFlow reference switch. It's a few years old, and uses an obselete API call - I can put up a patch if anyone gets stuck.

Next steps

• ARP tables - if we're going to route traffic, we need to find the MAC addresses of destination IPs so that we send traffic to them
• Routing protocol - RIP and OSPF will be fairly easy, BGP will be a bit harder due to relying on TCP. These can all be added to POX as modules
• TUN/TAP support - we can create TUN/TAP interfaces and let the linux TCP stack do the hard work for us. This means a BGP module would create the TUN/TAP interface and handle OpenFlow encapsulation/decapsulation, but could offload the TCP to the Linux stack.

DjangoFlow part two: Quick and simple UI

In the last episode...

My previous blog post showed how to integrate POX with Django, and didn't have too much colour. It took a bit of playing to get it to integrate for the first time, but now it's done, it is incredibly easy to do cool stuff with this software combo, and make new apps for easy OpenFlow access.

Part 2: An actual User Interface

I put in about 10 minutes extra just so I could give you all some lovely pictures, and here's what it all looks like:

This is all Django - I've just turned on the admin interface. Here we can add and remove flows from the database.

Here's a list of the two flows that I put in for testing

And here's the output from my OpenFlow switch.

It doesn't update in real time, but you can manipulate the database via the interweb and push those flows directly out to your switch. Making this update in real time is going to be another half-hour's work, tops.

What's new?

I changed the model a bit - now we can choose ports as well. Remember to delete your old database before syncing again or else it'll get upset

class Flow(models.Model):

internalip = models.CharField(max_length=200)

externalip = models.CharField(max_length=200)

internalport = models.IntegerField()

externalport = models.IntegerField()

idletime = models.IntegerField()

hardtime = models.IntegerField()

def __unicode__(self):

return "Internal: IP=" + self.internalip + " port=" + str(self.internalport) +", External: IP=" + self.externalip + " port=" + str(self.externalport)

The code in l2_learning.py got a bit of a birthday too:

class LearningSwitch (EventMixin):

  def __init__ (self, connection, transparent):

    # Switch we'll be adding L2 learning switch capabilities to

    self.connection = connection

    self.transparent = transparent

    # Our table

    self.macToPort = {}

    # We want to hear PacketIn messages, so we listen

    self.listenTo(connection)

    #log.debug("Initializing LearningSwitch, transparent=%s",

    #          str(self.transparent))

    

    # add new flows by default

    for flow in Flow.objects.all():

self.AddFlowFromModel(flow)

    

  

  def AddFlowFromModel(self, flow):

    # add outgoing flow

    msg = of.ofp_flow_mod()

    msg.match = of.ofp_match()

    msg.match.dl_type = ethernet.IP_TYPE

    msg.match.nw_src = str(flow.internalip)

    msg.match.nw_dst = str(flow.externalip)

    msg.match.in_port = flow.internalport

    msg.idle_timeout = flow.idletime

    msg.hard_timeout = flow.hardtime

    msg.actions.append(of.ofp_action_output(port = flow.externalport))

    self.connection.send(msg)

    

    # add incoming flow

    msg = of.ofp_flow_mod()

    msg.match = of.ofp_match()

    msg.match.dl_type = ethernet.IP_TYPE

    msg.match.nw_src = str(flow.externalip)

    msg.match.nw_dst = str(flow.internalip)

    msg.match.in_port = flow.externalport

    msg.idle_timeout = flow.idletime

    msg.hard_timeout = flow.hardtime

    msg.actions.append(of.ofp_action_output(port = flow.internalport))

    self.connection.send(msg)

After this, it was just a case of enabling the admin interface as per https://docs.djangoproject.com/en/dev/intro/tutorial02/ - this is our admin.py:

from django.contrib import admin

from flew.models import Flow

admin.site.register(Flow)

And for the sake of completeness, here's how to start them - the web interface is

python manage.py runserver

and the controller is

python pox.py forwarding.l2_learning

What next?

I don't know... I thought this would be much harder? Authentication will be fun, some way to dynamically check the database and update flows in real time (and remove them maybe) - this is left as an exercise for the reader though

DjangoFlow - Web UI for the POX OpenFlow controller

The Plan

OpenFlow is a simple concept - an open interface to switch and router hardware. Can we tie this into an open web framework to create a foundation for a really simple Web UI?

The tools

I've been learning how to use Django recently, and it seems like a perfect choice for this task. It's written in Python, so it integrates easily with the POX controller. I've also used virtualenv to make it more portable - this means the project is largely self-contained and can be copied to a new system by simply copying the folder.

Implementation

To start, I created a virtualenv called "djangoflow" on an Ubuntu machine and installed django in it. There are lots of ways to do this - do a google for "django setup virtualenv ubuntu" and you'll get tons of hits.

I made a project called mysite, and an app called flew (NZ english for "flow"), and left that bit for the time being.

The folder structure then looks something like this:

djangoflow

- bin

- include

- lib

- local

- mysite

--- flew

--- mysite

I then did a git clone of the latest build of POX from the NOXREPO github, into the djangoflow folder, and then copied everything from the base mysite folder into the pox folder. The folder structure then looks like:

djangoflow

- bin

- include

- lib

- local

- pox

--- .git

--- flew

--- mysite

--- pox

This is great - now back to Django.

The model that I used was really simple, here's what I've got so far:

from django.db import models

# Create your models here.

class Flow(models.Model):

internalip = models.CharField(max_length=200)

externalip = models.CharField(max_length=200)

idletime = models.IntegerField()

hardtime = models.IntegerField()

def __unicode__(self):

return "Internal: " + self.internalip + ", External: " + self.externalip

class User(models.Model):

name = models.CharField(max_length=200)

class Device(models.Model):

dpid = models.CharField(max_length=200)

We'll only use the Flow model today, the others are for expansion later. To make this work, we'll need to edit the settings in mysite.settings - set up the databases and installed_apps sections as follows:

DATABASES = {

    'default': {

        'ENGINE': 'django.db.backends.sqlite3', # Add 'postgresql_psycopg2', 'mysql', 'sqlite3' or 'oracle'.

        'NAME': 'flew.db',                      # Or path to database file if using sqlite3.

        'USER': '',                      # Not used with sqlite3.

        'PASSWORD': '',                  # Not used with sqlite3.

        'HOST': '',                      # Set to empty string for localhost. Not used with sqlite3.

        'PORT': '',                      # Set to empty string for default. Not used with sqlite3.

    }

}

INSTALLED_APPS = (

    'django.contrib.auth',

    'django.contrib.contenttypes',

    'django.contrib.sessions',

    # 'django.contrib.sites',

    'django.contrib.messages',

    'django.contrib.staticfiles',

    # Uncomment the next line to enable the admin:

    'django.contrib.admin',

    # Uncomment the next line to enable admin documentation:

    'django.contrib.admindocs',

    'flew',

)

Awesome, now run syncdb (check against whatever tutorial you use to see how this is done) and populate the database. Now, fire up the shell and create your first flow:

python manage.py shell

from flew.models import Flow

f = Flow(internalip = "10.1.10.2", externalip = "10.1.20.2", idletime = 300, hardtime = 3600)

f.save()

If that all worked, then you'll have a new entry in your database. We are never going to access the database directly though - we can access all the Django goodness from POX.

Open up pox/forwarding/l2_learning.py and have a look through - if you've used this before, then good, if not, then see what it all does.

I've hijacked this just for this example, but it sets a starting point for any POX-Django integrated tools. Make sure your imports section looks like this:

# import django stuff

from django.core.management import setup_environ

from mysite import settings

setup_environ(settings)

from flew.models import Flow

from pox.core import core

import pox.openflow.libopenflow_01 as of

from pox.lib.revent import *

from pox.lib.util import dpidToStr

from pox.lib.util import str_to_bool

from pox.lib.packet import ethernet

from pox.lib.packet import ipv4

import time

Then go down to the __init__ function for LearningSwitch and add this to the end:

    # add new flow by default

    flow1 = Flow.objects.all()[0]

    msg = of.ofp_flow_mod()

    msg.match = of.ofp_match()

    msg.match.dl_type = ethernet.IP_TYPE

    msg.match.nw_src = str(flow1.internalip)

    msg.match.nw_dst = str(flow1.externalip)

    msg.idle_timeout = flow1.idletime

    msg.hard_timeout = flow1.hardtime

    msg.actions.append(of.ofp_action_output(port = 1))

    #msg.buffer_id = event.ofp.buffer_id # 6a

    self.connection.send(msg)

What does this do? It takes the first Flow out of our database, creates a flow to allow traffic from internalip, going to externalip, to go out port 1, which should be pointing at the outside world. This flow will stay in the switch for an hour, or 5 minutes without being triggered - whichever happens first.

Question time

Q: Is it really this easy?

A: Yes. Python is easy, Django is easy, Virtualenv is easy, POX is easy. It's just a little tricky making them all work together - that's what this guide is for.

Q: Why add flows this way when we already have a CLI?

A: Django has a clean admin interface that I haven't covered here (check the setup tutorial on the Django website) - you can set flows in there, and every time a switch connects, it will use those flows.

Q: Could I start using this right now?

A: Totally. If you want your switches to retrieve their configuration from the controller automatically when they start up, you can set a bunch of super specific flows here and roll it out now. If you want something a bit more clever, then you can jump into the Django and POX API and make it happen yourself.

Q: What next?

A: There are two extra models that we didn't use - one for user authentication, and one for managing devices. If your OpenFlow devices connect by SSL (which they should in the real world) then you can create models for them that hold particular flows - this can all be managed via a web interface. As for user authentication, there are millions of ways to do this - you can set privileges for who can set certain flows, make requests that admins can approve - the possibilities are endless!