Ouroboros

Example Multi-Layer Ouroboros network (visualized using Rumba)

Rebuilding packet networks from the ground up.

Summary

Ouroboros (abbreviated as O7s) is a prototype packet-switching technology that can function as an alternative to TCP/IP. It is aimed at substantially simplifying configuration and operation of packet-switched networks. It is based on a redesign of the Internet model – from the programming API almost to the wire. If we had to describe Ouroboros in a single sentence, it could be micro-services architecture applied to the network itself.

From an end-user application perspective, an Ouroboros network is a black box with a simple application programming interface to request communication services. Ouroboros can provision unicast flows - (bidirectional) channels that deliver message streams or byte streams with some requested operational (QoS) parameters such as maximum delay and bandwidth, protection against packet loss and authentication of peers and encryption of in-flight data; or it can provide broadcast flows to sets of processes.

From an administrative perspective, an Ouroboros network is a collection of daemons that can be thought of as software routers (unicast) or software hubs (broadcast) that can be connected to each other; again through a simple management API. The tutorials highlight the administrative perspective of Ouroboros.

The prototype is developed for POSIX operating systems (Linux, *BSD, OS X). It is not directly compatible with TCP/IP (it uses different protocols) nor with POSIX sockets (it has a different API), but it has interfaces and tools to run over Ethernet or UDP, or to create IP/Ethernet tunnels over Ouroboros by exposing tap or tun devices for injecting packets. Check out the implementation status for more information.

Motivation and Objectives

Setting up a service over TCP/IP usually involves many different technologies. By the time the service is up and running, it will likely have involved configuring (switchport-based and trunk) VLANs, enabling some Spanning Tree Protocol variant in parts of the network, setting up link aggregation between ports on stacked switches, defining IP subnetworks, configuring a DHCP server to assign addresses to the subnets, setting up gateways, DNS servers, possibly configuring OSPF, IS-IS or iBGP/eBGP, selecting TCP and UDP ports for the applications, configuring reverse proxies, setting firewall and Network Address Translation (NAT) rules, adding some servers to a demilitarized zone, configuring a Virtual Private Network server, establishing a few SSH tunnels here and there... the list is almost endless.

A lot of this configuration is mostly static and done manually, once the service is in place, everything needs to be painstakingly documented because the configuration can be very brittle: introducing small changes can inadvertently bring the service down, and tracking down bugs, configuration errors or faults can take hours or even days. News stories about some DNS or BGP misconfiguration taking down a global service pop up regularly in the media.

The configuration is also spread out as IP address and port configurations need to be applied consistently between different devices (application servers, DHCP and DNS servers, NAT firewalls, application client hardware) and software packages (server software, reverse proxies, client software), whose configuration files often have different formats. Storing, maintaining and automating network and service configuration has become so elaborate and daunting that it now has its own buzzword: infrastructure as code.

The service configuration is not very scalable or portable. The IP network topology is defined by manual subnet assignments, and IP addresses often double as server identifiers in the minds of network admins. If an IP subnet has been over- or under-dimensioned, changing it can cause the need for redesigning many parts of the network. More than often, design compromises are made leaving as much of the network as-is because changing it would take weeks and involve downtime for many critical services. Moving infrastructure within or between datacenters or reintegrating it in a different parts can cause many headaches, some can be mitigated using virtualization, but the configuration of virtual machines and containers is still much more complicated than necessary.

Core Internet technology itself has become ossified; the core protocols haven't changed much in 30 years because making changes that are easy in theory, such as adding a new L3 protocol (IPv6) has become a decades-long slog. There are many layer violations creating unnecessary and unwanted ties between different layers in the model: L2 (Ethertype), L3 (protocol field), L4 (well-known ports) and L5 (struct sockaddr_in/_in6). A change in one of the layers permeates all the way through the network stack. To make things worse, standardization and research bodies have been established along the (blurry) demarcation lines of the 5-Layer Internet model. Conway's Law prevents any significant architectural or structural innovations from taking root.

The Internet is not a friendly place. Deploying a service over the Internet is more akin to venturing out into a lawless warzone than taking a nice stroll in the friendly neighborhood park. The first thing on any dotcom entrepreneur's mind is cybersecurity (quickly followed by how much it's going to cost him or her when the inevitable security breach comes). We would argue that the reason that the Internet has become ossified is also the reason for many network-level exploits: its bloated protocols rolled out the red carpet for cybercrime, cyberterrorism and cyberwarfare. TCP/IP has enabled an Internet that is robust to physical attacks on its infrastructure but left vulnerable to cyberattacks on its services. Unless we revise its architecture and core protocols, mitigating cyberthreats will remain a perpetual arms race between service providers and malicious hackers.

The Internet model is also rife with internal contradictions and inconsistencies. Examples abound of protocols that are jammed into the model without fitting - if IP is L3, how is ICMP also L3 as it runs on top of IP? Such contradictions hint at an incomplete understanding of packet networks.

In a nutshell, our objectives are to simplify and reduce network and service configuration and prevent ossification, and to do this by cleaning up and polishing the architecture and its protocols.

Ouroboros characteristics

O7s has no well-known ports, ephemeral endpoint identifiers (EIDs) are defined at the model's network end-to-end layer and are (randomly) assigned at runtime - the sole exception is the flow allocator at EID 0. This implies:

• no direct tie between the network protocols and certain application, and thus no need to 'standardize' certain values for certain identifiers (Ethertype/protocol/port).
• no need to define, track and/or de-conflict port values when setting up networked applications
• no hacks such as port-scanning a server for (vulnerable) applications

The flow allocator is a single point of contact:

• endpoint identifiers assigned when a flow is allocated
• sets and stores peer information (remote address and endpoint identifier) at the start of communication, so
  • no need for source address and source endpoint in every packet
  • even if an attacker would find a valid address/endpoint, fabricated packets sent there will not result in any knowledge gained by the attacker as return traffic is sent to the stored peer, not the attacker.
  • no smurf/amplification attacks leveraging faked source addresses
• backpressure may be an option to quench DDoS attempts
• authentication/encryption before first application byte

The application library and IRMd work together to create a single point of configuration:

• Instead of having network configuration per application, single network configuration file per system
• All O7s applications in the system inherit the network capabilities of the installed O7s version.

All traffic is congestion controlled, the congestion avoidance algorithm sits in the network end-to-end layer.

The project

Ouroboros leverages about 4 decades of hindsight and 2 decades worth of research experience in academic projects on future networks, more specifically in EU-funded (FP6, FP7 and Horizon 2020) projects on IP/WDM, Carrier Ethernet, Software Defined Networks (SDNs) and the Recursive InterNetwork Architecture (RINA). After gaining development experience on the IRATI kernel RINA prototype for OS/Linux, in early 2016 the decision was made to restart from scratch and try to avoid some issues we identified with the RINA model as it stood at that time. We aimed at POSIX user-space to make the prototype more accessible as compiling a kernel or kernel modules is a steep barrier to entry. Development was done in parallel with ongoing funded research to keep things as independent as possible from outside interference, but we used Ouroboros as one of the prototypes for the development of the Rumba orchestration framework for recursive networks. After leaving university we decided to keep taking the prototype forward in our spare time. It's a lot of work, but we still have a pretty clear view of the path forward, and there's only one way to eat an elephant: one bite at a time.

What to expect from this right now

In its current state, the prototype can demonstrate the validity of many (but definitely not all) aspects of the model and the potential of a full implementation of this model. It is far from a production piece of software, but rather a proof-of-concept that you can toy around with. See our tutorials section to get an overview of what the prototype can currently do. Our implementation overview tries to paint a picture of the implementation status of various components and functions. We are aware that parts of the implementation are quite buggy and are working towards getting this fixed.

Initially, a lot of the information on this website may sound a bit awkward, especially if configuring VLANs, defining IP subnets and configuring BGP routers are your daily bread and butter. We are the first to admit some of the terminology could use an overhaul. It will require a bit of effort to plow through the concepts of flow allocation and binding / registering names but once the penny drops, understanding O7s should prove rewarding.

If you find this interesting, just ask around on our matrix channel (or other preferred means of contact).