diff options
Diffstat (limited to 'content/en/docs/Concepts')
| -rw-r--r-- | content/en/docs/Concepts/_index.md | 12 | ||||
| -rw-r--r-- | content/en/docs/Concepts/aschenbrenner.png | bin | 0 -> 53954 bytes | |||
| -rw-r--r-- | content/en/docs/Concepts/creating_layers.jpg | bin | 0 -> 54903 bytes | |||
| -rw-r--r-- | content/en/docs/Concepts/dependencies.jpg | bin | 0 -> 12970 bytes | |||
| -rw-r--r-- | content/en/docs/Concepts/elements.md | 96 | ||||
| -rw-r--r-- | content/en/docs/Concepts/layers.jpg | bin | 0 -> 104947 bytes | |||
| -rw-r--r-- | content/en/docs/Concepts/layers.md | 90 | ||||
| -rw-r--r-- | content/en/docs/Concepts/protocols.md | 130 | ||||
| -rw-r--r-- | content/en/docs/Concepts/rec_netw.jpg | bin | 0 -> 63370 bytes | |||
| -rw-r--r-- | content/en/docs/Concepts/what.md | 146 |
10 files changed, 474 insertions, 0 deletions
diff --git a/content/en/docs/Concepts/_index.md b/content/en/docs/Concepts/_index.md new file mode 100644 index 0000000..ea9b33c --- /dev/null +++ b/content/en/docs/Concepts/_index.md @@ -0,0 +1,12 @@ +--- +title: "Concepts" +linkTitle: "Concepts" +weight: 40 +description: > + The concepts underpinning recursive networks in general + and Ouroboros in particular. +--- + +{{% pageinfo %}} +Under construction, subpages are accessible. +{{% /pageinfo %}} diff --git a/content/en/docs/Concepts/aschenbrenner.png b/content/en/docs/Concepts/aschenbrenner.png Binary files differnew file mode 100644 index 0000000..b9b11f6 --- /dev/null +++ b/content/en/docs/Concepts/aschenbrenner.png diff --git a/content/en/docs/Concepts/creating_layers.jpg b/content/en/docs/Concepts/creating_layers.jpg Binary files differnew file mode 100644 index 0000000..fe60019 --- /dev/null +++ b/content/en/docs/Concepts/creating_layers.jpg diff --git a/content/en/docs/Concepts/dependencies.jpg b/content/en/docs/Concepts/dependencies.jpg Binary files differnew file mode 100644 index 0000000..eaa9e79 --- /dev/null +++ b/content/en/docs/Concepts/dependencies.jpg diff --git a/content/en/docs/Concepts/elements.md b/content/en/docs/Concepts/elements.md new file mode 100644 index 0000000..bfc7fab --- /dev/null +++ b/content/en/docs/Concepts/elements.md @@ -0,0 +1,96 @@ +--- +title: "Elements of a recursive network" +author: "Dimitri Staessens" +date: 2019-07-11 +weight: 2 +description: > + The building blocks for recursive networks. +--- + +This section describes the high-level concepts and building blocks are +used to construct a decentralized [recursive network](/docs/what): +layers and flows. (Ouroboros has two different kinds of layers, but +we will dig into all the fine details in later posts). + +A __layer__ in a recursive network embodies all of the functionalities +that are currently in layers 3 and 4 of the OSI model (along with some +other functions). The difference is subtle and takes a while to get +used to (not unlike the differences in the term *variable* in +imperative versus functional programming languages). A recursive +network layer handles requests for communication to some remote +process and, as a result, it either provides a handle to a +communication channel -- a __flow__ endpoint --, or it raises some +error that no such flow could be provided. + +A layer in Ouroboros is built up from a bunch of (identical) programs +that work together, called Inter-Process Communication (IPC) Processes +(__IPCPs__). The name "IPCP" was first coined for a component of the +[LINCS] +(https://www.osti.gov/biblio/5542785-delta-protocol-specification-working-draft) +hierarchical network architecture built at Lawrence Livermore National +Laboratories and was taken over in the RINA architecture. These IPCPs +implement the core functionalities (such as routing, a dictionary) and +can be seen as small virtual routers for the recursive network. + +{{<figure width="60%" src="/docs/concepts/rec_netw.jpg">}} + +In the illustration, a small 5-node recursive network is shown. It +consists of two hosts that connect via edge routers to a small core. +There are 6 layers in this network, labelled __A__ to __F__. + +On the right-hand end-host, a server program __Y__ is running (think a +mail server program), and the (mail) client __X__ establishes a flow +to __Y__ over layer __F__ (only the endpoints are drawn to avoid +cluttering the image). + +Now, how does the layer __F__ get the messages from __X__ to __Y__? +There are 4 IPCPs (__F1__ to __F4__) in layer __F__, that work +together to provide the flow between the applications __X__ and +__Y__. And how does __F3__ get the info to __F4__? That is where the +recursion comes in. A layer at some level (its __rank__), will use +flows from another layer at a lower level. The rank of a layer is a +local value. In the hosts, layer __F__ is at rank 1, just above layer +__C__ or layer __E_. In the edge router, layer __F__ is at rank 2, +because there is also layer __D__ in that router. So the flow between +__X__ and __Y__ is supported by flows in layer __C__, __D__ and __E__, +and the flows in layer __D__ are supported by flows in layers __A__ +and __B__. + +Of course these dependencies can't go on forever. At the lowest level, +layers __A__, __B__, __C__ and __E__ don't depend on a lower layer +anymore, and are sometimes called 0-layers. They only implement the +functions to provide flows, but internally, they are specifically +tailored to a transmission technology or a legacy network +technology. Ouroboros supports such layers over (local) shared memory, +over the User Datagram Protocol, over Ethernet and a prototype that +supports flows over an Ethernet FPGA device. This allows Ouroboros to +integrate with existing networks at OSI layers 4, 2 and 1. + +If we then complete the picture above, when __X__ sends a packet to +__Y__, it passes it to __F3__, which uses a flow to __F1__ that is +implemented as a direct flow between __C2__ and __C1__. __F1__ then +forwards the packet to __F2__ over a flow that is supported by layer +__D__. This flow is implemented by two flows, one from __D2__ to +__D1__, which is supported by layer A, and one from __D1__ to __D3__, +which is supported by layer __B__. __F2__ will forward the packet to +__F4__, using a flow provided by layer __E__, and __F4__ then delivers +the packet to __Y__. So the packet moves along the following chain of +IPCPs: __F3__ --> __C2__ --> __C1__ --> __F1__ --> __D2__ --> __A1__ +--> __A2__ --> __D1__ --> __B1__ --> __B2__ --> __D3__ --> __F2__ --> +__E1__ --> __E2__ --> __F4__. + +{{<figure width="40%" src="/docs/concepts/dependencies.jpg">}} + +A recursive network has __dependencies__ between layers in the +network, and between IPCPs in a __system__. These dependencies can be +represented as a directed acyclic graph (DAG). To avoid problems, +these dependencies should never contain cycles (so a layer I should +not directly or indirectly depend on itself). The rank of a layer is +defined (either locally or globally) as the maximum depth of this +layer in the DAG. + +--- +Changelog: + +2019 07 11: Initial version.<br> +2019 07 23: Added dependency graph figure diff --git a/content/en/docs/Concepts/layers.jpg b/content/en/docs/Concepts/layers.jpg Binary files differnew file mode 100644 index 0000000..5d3020c --- /dev/null +++ b/content/en/docs/Concepts/layers.jpg diff --git a/content/en/docs/Concepts/layers.md b/content/en/docs/Concepts/layers.md new file mode 100644 index 0000000..e7c1441 --- /dev/null +++ b/content/en/docs/Concepts/layers.md @@ -0,0 +1,90 @@ +--- +title: "Recursive layers" +author: "Dimitri Staessens" +#description: "IRMd" +date: 2019-07-23 +weight: 20 +#type: page +draft: false +description: > + How to build a recursive network. +--- + +The most important structure in recursive networks are the layers, +that are built op from [elements](/docs/elements/) called +Inter-Process Communication Processes (IPCPs). (Note again that the +layers in recursive networks are not the same as layers in the OSI +model). + +{{<figure width="50%" src="/docs/concepts/creating_layers.jpg">}} + +Now, the question is, how do we build these up these layers? IPCPs are +small programs (think of small virtual routers) that need to be +started, configured and managed. This functionality is usually +implemented in some sort of management daemon. Current RINA +implementations call it the *IPC manager*, Ouroboros calls it the +__IPC Resource Management daemon__ or __IRMd__ for short. The IRMd +lies at the heart of each system that is participating in an Ouroboros +network, implementing the core function primitives. It serves as the +entry point for the system/network administrator to manage the network +resources. + +We will describe the functions of the Ouroboros IRMd with the +Ouroboros commands for illustration and to make things a bit more +tangible. + +The first set of primitives, __create__ (and __destroy__), allow +creating IPCPs of a given *type*. This just runs the process without +any further configuration. At this point, that process is not part of +any layer. + +``` +$ irm ipcp create type unicast name my_ipcp +$ irm ipcp list ++---------+----------------------+------------+----------------------+ +| pid | name | type | layer | ++---------+----------------------+------------+----------------------+ +| 7224 | my_ipcp | unicast | Not enrolled | ++---------+----------------------+------------+----------------------+ +``` + +The example above creates a unicast IPCP and gives that IPCP a name +(we called it "my_ipcp"). A listing of the IPCPs in the system shows +that the IPCP is running as process 7224, and it is not part of a +layer ("*Not enrolled*"). + +To create a new functioning network layer, we need to configure the +IPCP, using a primitive called __bootstrapping__. Bootstrapping sets a +number of configuration optionss for the layer (such as the routing +algorithm to use) and activates the IPCP to allow it to start +providing flows. The Ouroboros command line allows creating an IPCP +with some default values, that are a bit like a vanilla IPv4 network: +32-bit addresses and shortest-path link-state routing. + +``` +$ irm ipcp bootstrap name my_ipcp layer my_layer +$ irm ipcp list ++---------+----------------------+------------+----------------------+ +| pid | name | type | layer | ++---------+----------------------+------------+----------------------+ +| 7224 | my_ipcp | unicast | my_layer | ++---------+----------------------+------------+----------------------+ +``` + +Now we have a single node-network. In order to create a larger +network, we connect and configure new IPCPs using a third primitive +called __enrollment__. When enrolling an IPCP in a network, it will +create a flow (using a lower layer) to an existing member of the +layer, download the bootstrapping information, and use it to configure +itself as part of this layer. + +The final primitive is the __connect__ (and __disconnect__) +primitive. This allows to create *adjacencies* between network nodes. + +An example of how to create a small two-node network is given in +[tutorial 2](/docs/tutorials/tutorial-2/) + +--- +Changelog: + +2019-07-23: Initial version diff --git a/content/en/docs/Concepts/protocols.md b/content/en/docs/Concepts/protocols.md new file mode 100644 index 0000000..16c72cb --- /dev/null +++ b/content/en/docs/Concepts/protocols.md @@ -0,0 +1,130 @@ +--- +title: "The protocols" +author: "Dimitri Staessens" +#description: protocols +date: 2019-09-06 +#type: page +draft: false +description: > + A brief introduction to the main protocols. +--- + +# Network protocol + +As Ouroboros tries to preserve privacy as much as possible, it has an +*absolutely minimal network protocol*: + +``` + 0 1 2 3 + 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | | + + Destination Address + + | | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | Time-to-Live | QoS | ECN | EID | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | EID | + +-+-+-+-+-+-+-+-+ +``` + +The 5 fields in the Ouroboros network protocol are: + +* **Destination address**: This specifies the address to forward the + packet to. The width of this field is configurable based on various + preferences and the size of the envisioned network. The Ouroboros + default is 64 bits. Note that there is _no source address_, this is + agreed upon during _flow allocation_. + +* **Time-to-Live**: Similar to IPv4 and IPv6 (where this field is called + Hop Limit), this is decremented at each hop to ensures that packets + don't get forwarded forever in the network, for instance due to + (transient) loops in the forwarding path. The Ouroboros default for + the width is one octet (byte). + +* **QoS**: Ouroboros supports Quality of Service via a number of methods + (out of scope for this page), and this field is used to prioritize + scheduling of the packets when forwarding. For instance, if the + network gets congested and queues start filling up, higher priority + packets (e.g. a voice call) get scheduled more often than lower + priority packets (e.g. a file download). By default this field takes + one octet. + +* **ECN**: This field specifies Explicit Congestion Notification (ECN), + with similar intent as the ECN bits in the Type-of-Service field in + IPv4 / Traffic Class field in IPv6. The Ouroboros ECN field is by + default one octet wide, and its value is set to an increasing value + as packets are queued deeper and deeper in a congested routers' + forwarding queues. Ouroboros enforces Forward ECN (FECN). + +* **EID**: The Endpoint Identifier (EID) field specified the endpoint for + which to deliver the packet. The width of this field is configurable + (the figure shows 16 bits). The values of this field is chosen by + the endpoints, usually at _flow allocation_. It can be thought of as + similar to an ephemeral port. However, in Ouroboros there is no + hardcoded or standardized mapping of an EID to an application. + +# Transport protocol + +Packet switched networks use transport protocols on top of their +network protocol in order to deal with lost or corrupted packets. + +The Ouroboros Transport protocol (called the _Flow and Retransmission +Control Protocol_, FRCP) has only 4 fields: + +``` + 0 1 2 3 + 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | Flags | Window | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | Sequence Number | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | Acknowledgment Number | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + +``` + +* **Flags**: There are 7 flags defined for FRCP. + + - **DATA**: Indicates that the packet is carrying data (this allows + for 0 length data). + + - **DRF** : Data Run Flag, indicates that there are no unacknowledged + packets in flight for this connection. + + - **ACK** : Indicates that this packet carries an acknowledgment. + - **FC** : Indicates that this packet updates the flow control window. + - **RDVZ**: Rendez-vous, this is used to break a zero-window deadlock + that can arise when an update to the flow control window + gets lost. RDVZ packets must be ACK'd. + - **FFGM**: First Fragment, this packet contains the first fragment of + a fragmented payload. + - **MFGM**: More Fragments, this packet is not the last fragment of a + fragmented payload. + +* **Window**: This updates the flow control window. + +* **Sequence Number**: This is a monotonically increasing sequence number + used to (re)order the packets at the receiver. + +* **Acknowledgment Number**: This is set by the receiver to indicate the + highest sequence number that was received in + order. + +# Operation + +The operation of the transport protocol is based on the [Delta-t +protocol]((https://www.osti.gov/biblio/5542785-delta-protocol-specification-working-draft)), +which is a timer-based protocol that is a bit simpler in operation +than the equivalent functionalities in TCP. In contrast with TCP/IP, +Ouroboros does congestion control purely in the network protocol, and +fragmentation and flow control purely in the transport protocol. + + + + +--- Changelog: + +2019 09 05: Initial version.<br> +2019 09 06: Added section on transport protocol.
\ No newline at end of file diff --git a/content/en/docs/Concepts/rec_netw.jpg b/content/en/docs/Concepts/rec_netw.jpg Binary files differnew file mode 100644 index 0000000..bddaca5 --- /dev/null +++ b/content/en/docs/Concepts/rec_netw.jpg diff --git a/content/en/docs/Concepts/what.md b/content/en/docs/Concepts/what.md new file mode 100644 index 0000000..bacd4d6 --- /dev/null +++ b/content/en/docs/Concepts/what.md @@ -0,0 +1,146 @@ +--- +title: "What is a recursive network?" +author: "Dimitri Staessens" + +date: 2019-07-06 +weight: 1 +description: > + Introduction to recursive networks. +--- + +# The current networking paradigm +{{<figure width="40%" src="/docs/concepts/aschenbrenner.png">}} + +Every computer science class that deals with networks explains the +[7-layer OSI model](https://www.bmc.com/blogs/osi-model-7-layers/). +Open Systems Interconnect (OSI) defines 7 layers, each providing an +abstraction for a certain *function* that a network application may +need. + +From top to bottom, the layers provide (roughly) the following +functions: + +The __application layer__ implements the details of the application +protocol (such as HTTP), which specifies the operations and data that +the application understands (requesting a web page). + +The __presentation layer__ provides independence of data representation, +and may also perform encryption. + +The __session layer__ sets up and manages sessions (think of a session +as a conversation or dialogue) between the applications. + +The __transport layer__ handles individual chunks of data (think of them +as words in the conversation), and can ensure that there is end-to-end +reliability (no words or phrases get lost). + +The __network layer__ forwards the packets across the network, it +provides such things as addressing and congestion control. + +The __datalink layer__ encodes data into bits and moves them between +hosts. It handles errors in the physical layer. It has two sub-layers: +Media access control layer (MAC), which says when hosts can transmit +on the medium, and logical link control (LLC) that deals with error +handling and control of transmission rates. + +Finally, the __physical layer__ is responsible for translating the +bits into a signal (e.g. laser pulses in a fibre) that is carried +between endpoints. + +This functional layering provides a logical order for the steps that +data passes through between applications. Indeed, every existing +(packet) network goes through these steps in roughly this order +(however, some may be skipped). There can be some small variations on +where the functions are implemented. For instance, TCP does congestion +control, but is a Layer 4 protocol. + +However, when looking at current networking solutions, things are not +as simple as these 7 layers seem to indicate. Just consider this +realistic scenario for a software developer working remotely. Usually +it goes something like this: You connect to the company __Virtual +Private Network__ (VPN), setup an SSH __tunnel__ over the development +server to your virtual machine and then SSH into that virtual machine. + +The use of VPNs and various tunneling technologies draw a picture +where the __functions__ of Layers 2, 3, and 4, and often layers 5 and +6 (encryption) are __repeated__ a number of times in the actual +functional path that data follows through the network stack. + +Enter __*recursive networks*__. + +# The recursive network paradigm + +The functional repetition in the network stack is discussed in +detail in the book __*"Patterns in Network Architecture: A Return to +Fundamentals"*__. From the observations in the book, a new architecture +was proposed, called the "__R__ecursive __I__nter__N__etwork +__A__rchitecture", or [__RINA__](http://www.pouzinsociety.org). + +__Ouroboros__ follows the recursive principles of RINA, but deviates +quit a bit from its internal design. There are resources on the +Internet explaining RINA, but here we will focus +on its high level design and what is relevant for Ouroboros. + +Let's look at a simple scenario of an employee contacting an internet +corporate server over a Layer 3 VPN from home. Let's assume for +simplicity that the corporate LAN is not behind a NAT firewall. All +three networks perform (among some other things): + +__Addressing__: The VPN hosts receive an IP address in the VPN, let's +say some 10.11.12.0/24 address. The host will also have a public IP +address, for instance in the 20.128.0.0/16 range . Finally that host +will have an Ethernet MAC address. Now the addresses __differ in +syntax and semantics__, but for the purpose of moving data packets, +they have the same function: __identifying a node in a network__. + +__Forwarding__: Forwarding is the process of moving packets to a +destination __with intent__: each forwarding action moves the data +packet __closer__ to its destination node with respect to some +__metric__ (distance function). + +__Network discovery__: Ethernet switches learn where the endpoints are +through MAC learning, remembering the incoming interface when it sees +a new SRC address; IP routers learn the network by exchanging +informational packets about adjacency in a process called *routing*; +and a VPN proxy server relays packets as the central hub of a network +connected as a star between the VPN clients and the local area +network (LAN) that is provides access to. + +__Congestion management__: When there is a prolonged period where a +node receives more traffic than can forward forward, for instance +because there are incoming links with higher speeds than some outgoing +link, or there is a lot of traffic between different endpoints towards +the same destination, the endpoints experience congestion. Each +network could handle this situation (but not all do: TCP does +congestion control for IP networks, but Ethernet just drops traffic +and lets the IP network deal with it. Congestion management for +Ethernet never really took off). + +__Name resolution__: In order not having to remember addresses of the +hosts (which are in a format that make it easier for a machine to deal +with), each network keeps a mapping of a name to an address. For IP +networks (which includes the VPN in our example), this is done by the +Domain Name System (DNS) service (or, alternatively, other services +such as *open root* or *namecoin*). For Ethernet, the Address +Resolution Protocol maps a higher layer name to a MAC (hardware) +address. + +{{<figure width="50%" src="/docs/concepts/layers.jpg">}} + +Recursive networks take all these functions to be part of a network +layer, and layers are mostly defined by their __scope__. The lowest +layers span a link or the reach of some wireless technology. Higher +layers span a LAN or the network of a corporation e.g. a subnetwork or +an Autonomous System (AS). An even higher layer would be a global +network, followed by a Virtual Private Network and on top a tunnel +that supports the application. Each layer being the same in terms of +functionality, but different in its choice of algorithm or +implementation. Sometimes the function is just not implemented +(there's no need for routing in a tunnel!), but logically it could be +there. + + +--- +Changelog: + +2019 07 09: Initial version.
\ No newline at end of file |
