Ouroboros
Rebuilding packet networks from the ground up.
Summary
Computer networking is a lot more complicated than it needs to be.
Configuration and management of TCP/IP networks is an endeavor that can only be performed by highly trained experts. When integrating new appliances or re-racking and restructuring existing hardware in a data center, it is invariably the networking side of things that causes the most headaches as a lot of it needs to be manually re-configured, even in cases where a lot of the services are virtualized. The objective of this project is to simplify the network protocol stack so that it doesn't require an army of prized experts to understand and configure the infrastructure to run tomorrow's applications.
Ouroboros (abbreviated as O7s) is a prototype packet-switching technology that can function as an alternative to TCP/IP[1]. Its aim is to substantially simplify the configuration and operation of packet-switched networks.
The solution is based on a redesign of the Internet model – from the programming API almost to the wire. If we had to describe Ouroboros in a single sentence, it could be micro-services architecture applied to the network itself.
From an end-user application perspective, an Ouroboros network is a black box with a simple application programming interface to request communication services. Ouroboros can provision unicast flows - (bidirectional) channels that deliver message streams or byte streams with some requested operational (QoS) parameters such as maximum delay and bandwidth, protection against packet loss and authentication of peers and encryption of in-flight data; or it can provide broadcast flows to sets of processes.
From an administrative perspective, an Ouroboros network is a collection of daemons that can be thought of as software routers (unicast) or software hubs (broadcast) that can be connected to each other; again through a simple management API. The tutorials highlight the administrative perspective of Ouroboros.
The prototype is developed for POSIX operating systems (Linux, *BSD, OS X). It is not directly compatible with TCP/IP (it uses different protocols) nor with POSIX sockets (it has a different API), but it has interfaces and tools to run over Ethernet or UDP, or to create IP/Ethernet tunnels over Ouroboros by exposing tap or tun devices for injecting packets. Check out the implementation status for more information.
Motivation and Objectives
Setting up a service over TCP/IP usually involves many different technologies. By the time the service is up and running, it will likely have involved configuring (switchport-based and trunk) VLANs, enabling some Spanning Tree Protocol variant in parts of the network, setting up link aggregation between ports on stacked switches, defining IP subnetworks, configuring a DHCP server to assign addresses to the subnets, setting up gateways, DNS servers, possibly configuring OSPF, IS-IS or iBGP/eBGP, selecting TCP and UDP ports for the applications, configuring reverse proxies, setting firewall and Network Address Translation (NAT) rules, adding some servers to a demilitarized zone, configuring a Virtual Private Network server, establishing a few SSH tunnels here and there... the list is almost endless.
A lot of this configuration is mostly static and done manually, once the service is in place, everything needs to be painstakingly documented because the configuration can be very brittle: introducing small changes can inadvertently bring the service down, and tracking down bugs, configuration errors or faults can take hours or even days. News stories about some DNS or BGP misconfiguration taking down a global service pop up regularly in the media.
The configuration is also spread out as IP address and port configurations need to be applied consistently between different devices (application servers, DHCP and DNS servers, NAT firewalls, application client hardware) and software packages (server software, reverse proxies, client software), whose configuration files often have different formats. Storing, maintaining and automating network and service configuration has become so elaborate and daunting that it now has its own buzzword: infrastructure as code.
The service configuration is not very scalable or portable. The IP network topology is defined by manual subnet assignments, and IP addresses often double as server identifiers in the minds of network admins. If an IP subnet has been over- or under-dimensioned, changing it can cause the need for redesigning many parts of the network. More than often, design compromises are made leaving as much of the network as-is because changing it would take weeks and involve downtime for many critical services. Moving infrastructure within or between datacenters or reintegrating it in a different parts can cause many headaches, some can be mitigated using virtualization, but the configuration of virtual machines and containers is still much more complicated than necessary.
Core Internet technology itself has become ossified; the core protocols haven't changed much in 30 years because making changes that are easy in theory, such as adding a new L3 protocol (IPv6) has become a decades-long slog. There are many layer violations creating unnecessary and unwanted ties between different layers in the model: L2 (Ethertype), L3 (protocol field), L4 (well-known ports) and L5 (struct sockaddr_in/_in6). A change in one of the layers permeates all the way through the network stack. To make things worse, standardization and research bodies have been established along the (blurry) demarcation lines of the 5-Layer Internet model. Conway's Law prevents any significant architectural or structural innovations from taking root.
The Internet is not a friendly place. Deploying a service over the Internet is more akin to venturing out into a lawless warzone than taking a nice stroll in the friendly neighborhood park. The first thing on any dotcom entrepreneur's mind is cybersecurity (quickly followed by how much it's going to cost him or her when the inevitable security breach comes). We would argue that the reason that the Internet has become ossified is also the reason for many network-level exploits: its bloated protocols rolled out the red carpet for cybercrime, cyberterrorism and cyberwarfare. TCP/IP has enabled an Internet that is robust to physical attacks on its infrastructure but left vulnerable to cyberattacks on its services. Unless we revise its architecture and core protocols, mitigating cyberthreats will remain a perpetual arms race between application developers, service providers and malicious hackers[2]. The Internet ecosystem is so expertly divided so that any responsibility for security flaws can always be passed onto the next link down the supply chain. It shouldn't be.
On an abstract level, the Internet model is rife with internal contradictions and inconsistencies. Examples abound of protocols that are jammed into the model without fitting - if IP is L3, how is ICMP also L3 as it runs on top of IP? Such contradictions hint at an incomplete understanding of fundamental rules that underly all packet networks.
There is no incremental path to fix the Internet starting from TCP/IP.
In a nutshell, our objectives are to simplify and reduce network and service configuration and prevent ossification, and to do this by cleaning up and polishing the architecture and its protocols.
How is this supposedly going to work?
Everything starts with service names. They are the equivalent of Domain Names, but unlike their DNS counterpart, they are bound to processes, not addresses. Also unlike DNS names, service names are not optional. To deploy a service in an Ouroboros network, first and foremost you need to choose an available service name. That name comes with a (signed) certificate[3], just like certificates for Domain Names. For private networks, that can be a self-signed certificate, or no certificate at all.
Once in possession of a freshly minted service name and its certificate, it can be registered in one or more Ouroboros network at certain locations. There's a lot more to it, but just assume that these locations are at certain network addresses[4]. The naming system of the O7s network (called the directory) stores the address(es) together with the service name certificate.
To communicate with a service, the client application needs to request a flow to that service from the Ouroboros system. The system will figure out in which networks that service is registered (if any) and where in the network it is by quering network directories. If one responds, the client side O7s system will know 3 things:
- the O7s network(s) in which the service is available
- the address at which the service is available in each network
- the certificate for the service in each network (if any).
Now it can contact the server Ouroboros system and submit a flow request. That request contains the client system local address in the network and a (randomly generated) client flow endpoint id (EID). That request can be authenticated (i.e. encrypted using the public key in the service certificate received from the directory). It can also be 2-way authenticated using a client certificate (previously issued by the service owner). When the destination receives the flow request, it finds the source address and flow EID inside the request message. It then (randomly) generates its local flow EID and sends that back in a flow response. The response should be authenticated (i.e. encrypted using the service private key associated with the service certificate), else the source system will reject the flow response. If encryption of data over the flow is required, an (ephemeral) Diffie-Hellman exchange can be performed as part of this 2-way handshake. Only once this is completed, the client application receives a descriptor to read and write over this flow, possibly with a symmetric key for data encryption.
That is in a nutshell how O7s establishes (secure) communication between processes.
If reliable transmission is required (protection against packet loss and/or corruption), the flow is augmented with an Automated Repeat-reQuest (ARQ) protocol, adding packet integrity checks (e.g. CRC), numbering, ordering and fragmentation (as part of the O7s application library).
This framework allows for many small (and not-so-small) simplifications to protocols and APIs used in the current TCP/IP network stack.
Ouroboros characteristics
O7s has no well-known ports, ephemeral endpoint identifiers (EIDs) are defined at the model's network end-to-end layer and are (randomly) assigned at runtime - the sole exception is the flow allocator at EID 0. This implies:
- no direct tie between the network protocols and certain application, and thus no need to 'standardize' certain values for certain identifiers (Ethertype/protocol/port).
- no need to define, track and/or de-conflict port values when setting up networked applications
- no hacks such as port-scanning a server for (vulnerable) applications
The flow allocator is a single point of contact:
- endpoint identifiers assigned when a flow is allocated
- sets and stores peer information (remote address and endpoint identifier) at the start of communication, so
- no need for source address and source endpoint in every packet
- even if an attacker would find a valid address/endpoint, fabricated packets sent there will not result in any knowledge gained by the attacker as return traffic is sent to the stored peer, not the attacker.
- no smurf/amplification attacks leveraging faked source addresses
- backpressure may be an option to quench DDoS attempts
- authentication/encryption completed before first foreign byte arrives in application virtual memory space. This is interesting for the design of secure distributed microkernels.
The application library and IRMd work together to create a single point of configuration:
- Instead of having network configuration per application, single network configuration file per system
- All O7s applications in the system inherit the network capabilities of the installed O7s version.
All traffic is congestion controlled, the congestion avoidance algorithm sits in the network end-to-end layer.
The project
Ouroboros leverages about 4 decades of hindsight and 2 decades worth of research experience in academic projects on future networks, more specifically in EU-funded (FP6, FP7 and Horizon 2020) projects on IP/WDM, Carrier Ethernet, Software Defined Networks (SDNs) and the Recursive InterNetwork Architecture (RINA). After gaining development experience on the IRATI kernel RINA prototype for OS/Linux, in early 2016 the decision was made to restart from scratch and try to avoid some issues we identified with the RINA model as it stood at that time. We aimed at POSIX user-space to make the prototype more accessible as compiling a kernel or kernel modules is a steep barrier to entry. Development was done in parallel with ongoing funded research to keep things as independent as possible from outside interference, but we used Ouroboros as one of the prototypes for the development of the Rumba orchestration framework for recursive networks. After leaving university we decided to keep taking the prototype forward in our spare time. It's a lot of work, but we still have a pretty clear view of the path forward.
There's only one way to eat an elephant: one bite at a time.
What to expect from this right now
In its current state, the prototype can demonstrate the validity of many (but definitely not all) aspects of the model and the potential of a full implementation of this model. It is far from a production piece of software, but rather a proof-of-concept that you can toy around with. See our tutorials section to get an overview of what the prototype can currently do. Our implementation overview tries to paint a picture of the implementation status of various components and functions. We are aware that parts of the implementation are quite buggy and are working towards getting this fixed.
Initially, a lot of the information on this website may sound a bit awkward, especially if configuring VLANs, defining IP subnets and configuring BGP routers are your daily bread and butter. We are the first to admit some of the terminology could use an overhaul. It will require a bit of effort to plow through the concepts of flow allocation and binding / registering names but once the penny drops, understanding O7s should prove rewarding.
If you find this interesting, just ask around on our matrix channel (or other preferred means of contact).
References
- ↑ Don't worry, it works as an overlay network as well, there's no need to chuck out your NICs and switches.
- ↑ The DDoS That Knocked Spamhaus Offline (And How We Mitigated It)
- ↑ In essence, the registrar is the certificate authority.
- ↑ It is important to note, however, that the location is not specified by an address while registering.