Ouroboros Prototype: Difference between revisions

From Ouroboros
Jump to navigation Jump to search
No edit summary
 
(32 intermediate revisions by 2 users not shown)
Line 1: Line 1:
{{Under construction}}
{{Under construction}}


The Ouroboros prototype is a user-space packet network implementation based on the [[Ouroboros | networking model]] of the same name. It is being developed in C for POSIX operating systems.
The Ouroboros prototype is a user-space decentralized packet network implementation based on an overhauled [[Ouroboros Model| networking model]]. It is being developed in C for POSIX operating systems.
This prototype is still a work in progress.


== Prototype summary ==
This prototype is very much a work in progress.
Main Page: [[Ouroboros Implementation Overview]]


The prototype consists of the '''library''', which provides the core Application Programming Interfaces (APIs) and functions for the other components in the prototype; the IPC Resource Management daemon '''IRMd''', which is the core component for administering Ouroboros; the '''IPCPs''' that actually forward packets and make up the (peer-to-peer) packet network; and a collection of '''tools''', of which the most important is the IPC Resource Management (IRM) Command Line Interface (CLI) for management and some basic tools for demo purposes. In addition, there is an [[Rumba | orchestration framework]] for setting up experiments and an [[Ouroboros metrics | InfluxDB exporter]] for observability. The prototype is written in C, but we provide some APIs for other popular languages, currently [[pyOuroboros | Python]] and rust.
On this page are instructions for installing the prototype. Once done, head to the [[Ouroboros Tutorials | tutorials]] section. The [https://ouroboros.rocks/docs old documentation website] is still available for reference, but only this wiki is maintained.
 
= Install the Prototype =
 
The prototype consists of the '''library''', which provides the core Application Programming Interfaces (Unicast API and Broadcast API), the functions of the [[Ouroboros Functional Layering | application end-to-end layer]] and various functions used by the other components in the prototype; the IPC Resource Management daemon '''IRMd''', which is the core component for administering Ouroboros; the '''IPCPs''' that actually forward packets and make up the (peer-to-peer) packet network; and a collection of '''tools''', of which the most important is the IPC Resource Management (IRM) Command Line Interface (CLI) for management and some basic tools for demo purposes. In addition, there is an [[Rumba | orchestration framework]] for setting up experiments and an [[Ouroboros metrics | InfluxDB exporter]] for observability. The prototype is written in C, but we provide some APIs for other popular languages, currently [[pyOuroboros | Python]] and [[OuroboRs | Rust]].


== Requirements ==
== Requirements ==


Ouroboros builds on most POSIX compliant systems. Below you will find instructions for GNU/Linux, FreeBSD and OS X. On Windows 10, you can build Ouroboros using the Linux Subsystem for Windows .
Ouroboros builds on most POSIX compliant systems. Below you will find instructions for GNU/Linux, FreeBSD and OS X. On Windows 10, you can build Ouroboros using the [https://learn.microsoft.com/en-us/windows/wsl/install Linux Subsystem for Windows].


== Get Ouroboros ==
== Get Ouroboros ==
Line 17: Line 19:
You can clone Ouroboros using the command line from its website or, if you prefer you can clone and/or fork from our [https://github.com/dstaesse/ouroboros Github] or [https://bitbucket.org/dstaesse/ouroboros BitBucket] mirrors.
You can clone Ouroboros using the command line from its website or, if you prefer you can clone and/or fork from our [https://github.com/dstaesse/ouroboros Github] or [https://bitbucket.org/dstaesse/ouroboros BitBucket] mirrors.


<syntaxhighlight lang="bash">
<syntaxhighlight>
$ git clone https://ouroboros.rocks/git/ouroboros
$ git clone https://ouroboros.rocks/git/ouroboros
$ git clone git://ouroboros.rocks/ouroboros
$ git clone git://ouroboros.rocks/ouroboros
Line 23: Line 25:
</syntaxhighlight>
</syntaxhighlight>


Optional repositories are [[pyOuroboros]], a Python wrapper to write Ouroboros programs in Python, and [[ouroboros-metrics]] that contains an [https://influxdb.com InfluxDB] exporter.
Another option is downloading a [https://{{SERVERNAME}}/cgit/ouroboros snapshot] tarball and extracting it.
Optional repositories are [[pyOuroboros]], a Python wrapper to write Ouroboros programs in Python, and [[Ouroboros metrics | ouroboros-metrics]] that contains an [https://influxdb.com InfluxDB] exporter.


== Dependencies ==
== Dependencies ==
Line 37: Line 40:
=== Debian/Ubuntu Linux ===
=== Debian/Ubuntu Linux ===


<syntaxhighlight lang="bash">
$ apt-get install git protobuf-c-compiler cmake
$ apt-get install git protobuf-c-compiler cmake
$ apt-get install libgcrypt20-dev libssl-dev libfuse-dev dnsutils cmake-curses-gui
$ apt-get install libgcrypt20-dev libssl-dev libfuse-dev dnsutils cmake-curses-gui
</syntaxhighlight>


If during the build process cmake complains that the Protobuf C compiler is required but not found, and you installed the protobuf-c-compiler package, you will also need this:
If during the build process cmake complains that the Protobuf C compiler is required but not found, and you installed the protobuf-c-compiler package, you will also need this:


<syntaxhighlight lang="bash">
$ apt-get install libprotobuf-c-dev
$ apt-get install libprotobuf-c-dev
</syntaxhighlight>


=== Arch Linux ===
=== Arch Linux ===


<syntaxhighlight lang="bash">
$ pacman -S git protobuf-c cmake
$ pacman -S git protobuf-c cmake
$ pacman -S libgcrypt openssl fuse dnsutils
$ pacman -S libgcrypt openssl fuse dnsutils
 
</syntaxhighlight>
<div class="mw-collapsible mw-collapsed" style="width:400px; overflow:auto;">
Note AUR:
<div class="mw-collapsible-content">
There is an unmaintained [https://aur.archlinux.org/packages/ouroboros-git AUR] package which installs also the dependencies. It clones directly from the git repository and builds. Please report any issues since it's not actively supported at the moment.
</div></div>


=== FreeBSD 11/12/13 ===
=== FreeBSD 11/12/13 ===


<syntaxhighlight lang="bash">
$ pkg install git protobuf-c cmake
$ pkg install git protobuf-c cmake
$ pkg install libgcrypt openssl fusefs-libs bind-tools
$ pkg install libgcrypt openssl fusefs-libs bind-tools
</syntaxhighlight>


=== Mac OS X 10/11/12 ===
=== Mac OS X 10/11/12 ===


<syntaxhighlight lang="bash">
$ brew install git protobuf-c cmake
$ brew install git protobuf-c cmake
$ brew install libgcrypt openssl
$ brew install libgcrypt openssl
 
</syntaxhighlight>
== Configuration file support ==
 
It is highly recommended you install [https://github.com/cktan/tomlc99 libtomlc99] ''before'' building O7s to enable configuration file support.
 
To install (on GNU/Linux):
 
git clone https://github.com/cktan/tomlc99
cd tomlc99
make && sudo make install


== Installation instructions ==
== Installation instructions ==
Line 74: Line 83:
Run cmake providing the path to where you cloned the Ouroboros repository. Assuming you created the build directory inside the repository directory, the following commands build and install O7s:
Run cmake providing the path to where you cloned the Ouroboros repository. Assuming you created the build directory inside the repository directory, the following commands build and install O7s:


<syntaxhighlight lang="bash">
$ cd <path/to/ouroboros/repo>
$ mkdir build && cd build
$ mkdir build && cd build
$ cmake ..
$ cmake ..   # optional: configure build parameters
$ sudo make install
$ make && sudo make install
</syntaxhighlight>
 
== Running the unit tests ==


== Remove Ouroboros ==
To compile and run the unit tests, do


To uninstall Ouroboros, simply execute the following command from your build directory:
$ make check


<syntaxhighlight lang="bash">
Note that "make test" will only try to run the tests, not compile the necessary sources and may result in errors.
$ sudo make uninstall
</syntaxhighlight>


== Build configuration parameters ==
== Build configuration parameters ==


At compile time, there are a number of parameters that can be tweaked, for instance to increase performance or decrease memory consumption.
At compile time, there are a number of parameters that can be tweaked, for instance to enable debug logging, enable various sanitizers, to increase performance or decrease memory consumption. You can find a full list on the [[Ouroboros Build Parameters]] page.


These parameters can be enabled by specifying them to cmake
== Remove Ouroboros ==


<syntaxhighlight lang=bash>
To uninstall Ouroboros, simply execute the following command from your build directory:
$ cmake -D<option>=<value> ..
</syntaxhighlight>
 
or interactively by running
 
<syntaxhighlight lang=bash>
$ ccmake .
</syntaxhighlight>
 
in the build directory.


{|class="wikitable" style="margin:auto"
  $ sudo make uninstall
|+ Ouroboros build configuration
! Parameter !! Description !! Values || Default
|-
| ACK_WHEEL_RESOLUTION
| Minimum acknowledgment delay (ns), as a power to 2
|
| 18
|-
| ACK_WHEEL_SLOTS
| Number of slots in the acknowledgment wheel, must be a power of 2
|
| 256
|-
| BOOTSTRAP_TIMEOUT
| Timeout for an IPCP to bootstrap (ms)
|
| 5000
|-
| CMAKE_BUILD_TYPE
| Build type
|
* Release: A release build, less debug logs, compiled with performance flags
* Debug: A debug build with extra logging and debug symbols (gdb)
* DebugASan: Enables Address Sanitizer
* DebugTSan: Enables Thread Sanitizer
* DebugLSan: Enables Leak Sanitizer
* DebugUSan: Enables Undefined behaviour Sanitizer
* DebugAnalyzer: Enables static analysis during compilation
| Release
|-
| CMAKE_INSTALL_PREFIX
| Prefix to install Ouroboros
|
|
/usr (Linux, *BSD)
/usr/local (OS X)
|-
| CONNECT_TIMEOUT 
| Timeout to connect an IPCP to another IPCP (ms)
|
| 60000
|-
| DELTA_T_ACK
| Maximum time to acknowledge a packet (s)
|
| 10
|-
| DELTA_T_MPL
| Maximum packet lifetime (s)
|
| 60
|-
| DELTA_T_RTX
| Maximum time to retransmit a packet (s)
|
| 120
|-
| DHT_ENROLL_SLACK
| DHT enrollment waiting time (ms)
| 0-999
| 50
|-
| DISABLE_CORE_LOCK
| Disable locking performance threads to a core
| Bool
| ON
|-
| DISABLE_DDNS
| Disable DDNS support for ipcpd-udp
| Bool
| OFF
|-
| DISABLE_FUSE  
| Disable FUSE support for exporting metrics (observability)
| Bool
| OFF
|-
| DISABLE_LIBGCRYPT
| Disable libgcrypt support
|
| OFF
|-
| DISABLE_OPENSSL
| Disable OpenSSL support
|
| OFF
|-
| DISABLE_RAW_SOCKETS
| Disable raw socket support for Ethernet IPCPs
|
| OFF
|-
| DISABLE_ROBUST_MUTEXES
| Disable robust mutex support
|
| OFF
|-
| DU_BUFF_HEADSPACE
| Bytes of headspace to reserve for future headers
|
| 256
|-
| DU_BUFF_TAILSPACE
| Bytes of tailspace to reserve for future tails
|
| 32
|-
| ENROLL_TIMEOUT
| Timeout the IRMd waits for an IPCP to complete enrollment (ms)
|
| 60000
|-
| FRCT_LINUX_RTT_ESTIMATOR
| Use Linux RTT estimator formula instead of the TCP RFC formula
|
| ON
|-
| FRCT_REORDER_QUEUE_SIZE
| Size of the reordering queue
| Must be a power of 2
| 256
|-
| FRCT_RTO_INC_FACTOR
| Divisor for RTO increase after ACK timeout: RTO += RTX >> X
|
| 0: Karn/Partridge
|-
| FRCT_RTO_MDEV_MULTIPLIER
| Multiplier for deviation term in the RTO: RTO = sRTT + (mdev << X)
|
| 2
|-
| FRCT_RTO_MIN
| Minimum Retransmission Timeout (RTO) for FRCT (us)
|
| 250
|-
| FRCT_START_WINDOW
| Start window
| Must be a power of 2
| 64
|-
| FRCT_TICK_TIME
| Tick time for FRCT activity (retransmission, acknowledgments) (us)
|
| 5000
|-
| FUSE_PREFIX
| Mountpoint for FUSE filesystem (observability)
|
| /tmp/ouroboros
|-
| IPCP_ADD_THREADS
| Number of extra threads to start when an IPCP faces thread starvation
|
| 4
|-
| IPCP_BROADCAST_MPL
| Default maximum packet lifetime for the broadcast IPCP, in seconds
|
| 60
|-
| IPCP_CONN_WAIT_DIR
| Check the running state of the directory (DHT) when adding a data transfer connection
|
| ON
|-
| IPCP_ETH_LO_MTU
| Restrict Ethernet MTU over loopback interface
|
| 1500
|-
| IPCP_ETH_MPL
| Default maximum packet lifetime for the Ethernet IPCPs, in seconds
|
| 5
|-
| IPCP_ETH_QDISC_BYPASS
| Bypass the Qdisc in the kernel when using raw sockets
|
| OFF
|-
| IPCP_ETH_RD_THR
| Number of reader threads in Ethernet IPCP
|
| 1
|-
| IPCP_ETH_WR_THR
| Number of writer threads in Ethernet IPCP
|
| 1
|-
| IPCP_FLOW_STATS
| Enable flow statistics tracking in IPCP (observability)
| Bool
| ON
|-
| IPCP_LINUX_TIMERSLACK_NS
| Slack value for high resolution timers on Linux systems
|
| 1000
|-
| IPCP_LOCAL_MPL
| Default maximum packet lifetime for the Ethernet IPCPs, in seconds
|
| 2
|-
| IPCP_MIN_THREADS
| Minimum number of worker threads in the IPCP
|
| 4
|-
| IPCP_QOS_CUBE_BE_PRIO
| Priority for best effort QoS cube (0-99)
|
| 50
|-
| IPCP_QOS_CUBE_VIDEO_PRIO
| Priority for video QoS cube (0-99)
|
| 90
|-
| IPCP_QOS_CUBE_VOICE_PRIO
| Priority for voice QoS cube (0-99)
|
| 99
|-
| IPCP_SCHED_THR_MUL
| Number of scheduler threads per QoS cube
|
| 2
|-
| IPCP_UDP_MPL
| Default maximum packet lifetime for the UDP IPCP, in seconds
|
| 60
|-
| IPCP_UDP_RD_THR
| Number of reader threads in UDP IPCP
|
| 3
|-
| IPCP_UDP_WR_THR
| Number of writer threads in UDP IPCP
|
| 3
|-
| IPCP_UNICAST_MPL
| Default maximum packet lifetime for the unicast IPCP, in seconds
|
| 60 
|-
| IRMD_ADD_THREADS
| Number of extra threads to start when the IRMD faces thread starvation
|
| 8     
|-
| IRMD_FLOW_TIMEOUT
| Timeout for a flow allocation response (ms)
|
| 5000
|-
| IRMD_MIN_THREADS
| Minimum number of worker threads in the IRMd
|
| 8   
|-
| IRMD_REQ_ARR_TIMEOUT
| Timeout for an application to respond to arrival of a new flow (ms)
|
| 500
|-
| PFT_SIZE
| Size of the PDU forwarding (hash) table
|
| 4096
|-
| PROC_FLOW_STATS
| Enable flow statistics tracking for application flows
| Bool
| ON
|-
| PROG_MAX_FLOWS
| Maximum number of flows in an application
|
| 4096
|-
| PROG_MAX_FQUEUES
| Maximum number of flow sets per application
|
| 32
|-
| PROG_RES_FDS
| Number of reserved flow descriptors per application
|
| 64
|-
| PTHREAD_COND_CLOCK
| Clock to use for condition variable timing
|
| CLOCK_MONOTONIC
|-
| QOS_DISABLE_CRC
| Ignores bit-error rate (BER) setting on all QoS cubes to avoid calculating CRCs
|
| ON
|-
| QUERY_TIMEOUT
| Timeout to query a name with an IPCP (ms)
|
| 3000
|-
| REG_TIMEOUT
| Timeout for registering a name (ms)
|
| 10000
|-
| RXM_BLOCKING
| Use blocking writes for retransmission
|
| ON
|-
| RXM_BUFFER_ON_HEAP
| Store packets for retransmission on the heap instead of in packet buffer
|
| OFF
|-
| RXM_MIN_RESOLUTION
| Minimum retransmission delay (ns), as a power to 2
|
| 20
|-
| RXM_WHEEL_LEVELS
| Number of levels in the retransmission wheel
|
| 3   
|-
| RXM_WHEEL_MULTIPLIER
| Factor for retransmission wheel levels as a power to 2
|
| 4
|-
| RXM_WHEEL_SLOTS_PER_LEVEL
| Number of slots per level in the retransmission wheel
| Must be a power of 2
| 256
|-
| SHM_BUFFER_SIZE
| Number of blocks in packet buffer,
| Must be a power of 2
| 4096
|-
| SHM_PREFIX
| String to prepend to POSIX shared memory filenames
|
| ouroboros
|-
| SHM_RBUFF_LOCKLESS
| Enable shared memory lockless rbuff support (experimental, unstable)
|
| OFF
|-
| SHM_RBUFF_SIZE
| Number of blocks in rbuff buffer
| Must be a power of 2
| 1024
|-
| SHM_RDRB_BLOCK_SIZE
| Packet buffer block size
| Defaults to pagesize for performance
| sysconf(_SC_PAGESIZE)
|-
| SHM_RDRB_MULTI_BLOCK
| Packet buffer multiblock packet support (allows packets larger than block size)
|
| ON
|-
| SOCKET_TIMEOUT
| Default timeout for internal responses from IPCPs (ms)
|
| 1000
|-
| SOCK_BUF_SIZE
| Size of the buffer used by the UNIX sockets for local IPC
|
| 10240
|-
| SYS_MAX_FLOWS
| Maximum number of total flows for this system
|
| 10240
|}

Latest revision as of 10:15, 17 January 2024

Under contruction This page is under construction  

The Ouroboros prototype is a user-space decentralized packet network implementation based on an overhauled networking model. It is being developed in C for POSIX operating systems.

This prototype is very much a work in progress.

On this page are instructions for installing the prototype. Once done, head to the tutorials section. The old documentation website is still available for reference, but only this wiki is maintained.

Install the Prototype

The prototype consists of the library, which provides the core Application Programming Interfaces (Unicast API and Broadcast API), the functions of the application end-to-end layer and various functions used by the other components in the prototype; the IPC Resource Management daemon IRMd, which is the core component for administering Ouroboros; the IPCPs that actually forward packets and make up the (peer-to-peer) packet network; and a collection of tools, of which the most important is the IPC Resource Management (IRM) Command Line Interface (CLI) for management and some basic tools for demo purposes. In addition, there is an orchestration framework for setting up experiments and an InfluxDB exporter for observability. The prototype is written in C, but we provide some APIs for other popular languages, currently Python and Rust.

Requirements

Ouroboros builds on most POSIX compliant systems. Below you will find instructions for GNU/Linux, FreeBSD and OS X. On Windows 10, you can build Ouroboros using the Linux Subsystem for Windows.

Get Ouroboros

You can clone Ouroboros using the command line from its website or, if you prefer you can clone and/or fork from our Github or BitBucket mirrors.

$ git clone https://ouroboros.rocks/git/ouroboros
$ git clone git://ouroboros.rocks/ouroboros
$ git clone https://github.com/dstaesse/ouroboros

Another option is downloading a snapshot tarball and extracting it. Optional repositories are pyOuroboros, a Python wrapper to write Ouroboros programs in Python, and ouroboros-metrics that contains an InfluxDB exporter.

Dependencies

To build Ouroboros, you need cmake, google protocol buffers installed in addition to a C compiler (gcc or clang) and make.

On GNU/Linux you will need either libgcrypt (≥ 1.7.0) or libssl if your glibc is older than version 2.25.

On OS X, you will need homebrew. Disable System Integrity Protection during the installation and/or removal of Ouroboros.

Optionally, you can also install libgcrypt, libssl (to enable encryption), fuse (to enable exporting internal metrics), and dnsutils (to enable use of a DynDNS server to be used as a naming directory for a UDP layer). Setting up a DNS server (e.g. bind) is out of scope of this installation guide.

Debian/Ubuntu Linux

$ apt-get install git protobuf-c-compiler cmake
$ apt-get install libgcrypt20-dev libssl-dev libfuse-dev dnsutils cmake-curses-gui

If during the build process cmake complains that the Protobuf C compiler is required but not found, and you installed the protobuf-c-compiler package, you will also need this:

$ apt-get install libprotobuf-c-dev

Arch Linux

$ pacman -S git protobuf-c cmake
$ pacman -S libgcrypt openssl fuse dnsutils

Note AUR:

There is an unmaintained AUR package which installs also the dependencies. It clones directly from the git repository and builds. Please report any issues since it's not actively supported at the moment.

FreeBSD 11/12/13

$ pkg install git protobuf-c cmake
$ pkg install libgcrypt openssl fusefs-libs bind-tools

Mac OS X 10/11/12

$ brew install git protobuf-c cmake
$ brew install libgcrypt openssl

Configuration file support

It is highly recommended you install libtomlc99 before building O7s to enable configuration file support.

To install (on GNU/Linux):

git clone https://github.com/cktan/tomlc99
cd tomlc99
make && sudo make install

Installation instructions

When installing from source, go to the cloned git repository or extract the tarball and enter the main directory. We recommend creating a build directory inside this directory. Run cmake providing the path to where you cloned the Ouroboros repository. Assuming you created the build directory inside the repository directory, the following commands build and install O7s:

$ cd <path/to/ouroboros/repo>
$ mkdir build && cd build
$ cmake ..    # optional: configure build parameters
$ make && sudo make install

Running the unit tests

To compile and run the unit tests, do

$ make check

Note that "make test" will only try to run the tests, not compile the necessary sources and may result in errors.

Build configuration parameters

At compile time, there are a number of parameters that can be tweaked, for instance to enable debug logging, enable various sanitizers, to increase performance or decrease memory consumption. You can find a full list on the Ouroboros Build Parameters page.

Remove Ouroboros

To uninstall Ouroboros, simply execute the following command from your build directory:

$ sudo make uninstall