Ouroboros Tutorial 02

From Ouroboros
Jump to navigation Jump to search
Ouroboros Tutorial 02

The description of this tutorial uses a debug build with default build options.

The objective for this tutorial is to create a simple 2-node unicast Layer and to ping an oping server from an oping client over this network. We will bootstrap a first unicast IPC process (IPCP), enroll a second IPCP with the first, and connect them. This tutorial builds on Tutorial 01, which explains flow allocation, direct flows and the security configuration used below.[1]

We recommend opening a couple of terminal windows for this tutorial. In the first one, start the IRMd: 

$ sudo irmd --stdout

Security configuration

An IPCP is a process like any other: to enroll and connect, it allocates flows to other IPCPs, and these flows run the same Ouroboros Allocation Protocol (OAP) as the flows between our oping applications. So, just like in Tutorial 01, a client IPCP will require the server IPCP to present a certificate. Tutorial 06 shows how to use certificates to control which IPCPs can join a Layer. For this tutorial, we tell the clients that the names we will allocate flows to, the Layer name unicast-layer, the IPCP name uni-1 and our service oping-server, do not need to authenticate: 

$ for n in unicast-layer uni-1 oping-server; do
>   sudo mkdir -p /etc/ouroboros/security/client/$n
>   printf 'auth=optional\nencryption=none\n' | sudo tee /etc/ouroboros/security/client/$n/sec.conf > /dev/null
> done

Bootstrapping the first IPCP

To create the unicast network, we bootstrap the first IPCP in a new Layer that we call unicast-layer. Just like for the local IPCP in Tutorial 01, the bootstrap command creates the IPCP if it does not exist yet: 

$ irm ipcp bootstrap type unicast name uni-1 layer unicast-layer

The IRMd log shows the following output: 

==02519== irmd(II): Instantiated /usr/sbin/ipcpd-unicast as process 2551.
==02519== irmd(II): Process added: 2551 (ipcpd-unicast).
==02551== ipcpd/ipcp(II): IPCP unicast 2551 initialized.
==02519== irmd(II): Created IPCP 2551.
==02551== ipcpd/ipcp(II): Bootstrapping...
==02551== flat-addr-auth(DB): Flat address initialized to f7.02.b6.16.
==02551== unicast-ipcp(II): IPCP got address 381027063.
==02551== ca(DB): Using multi-bit ECN.
==02551== link-state-routing(DB): Using Link State Routing policy.
==02551== link-state-routing(DB): LS update interval: 15 seconds.
==02551== link-state-routing(DB): LS link timeout   : 60 seconds.
==02551== link-state-routing(DB): LS recalc interval: 4 seconds.
==02551== pff(DB): Using simple PFF policy.
==02551== pff(DB): Using simple PFF policy.
==02551== pff(DB): Using simple PFF policy.
==02551== directory(II): Using DHT policy.
==02551== dht(II): DHT initialized.
==02551== dht(DB):   ID: 420371d5bd94165a [32 bytes].
==02551== dht(DB):   address: f7.02.b6.16.
==02551== dht(DB):   peer: 00.00.00.00.
==02551== dht(DB):   magic cookie: 20f88c5e1c9e4b52.
==02551== dht(II):   parameters: alpha=3, k=8, t_exp=86400, t_refr=900, t_replicate=900.
==02551== ipcpd/ipcp(II): Finished bootstrapping in unicast-layer.
==02551== ipcpd/ipcp(II):   type: unicast
==02551== ipcpd/ipcp(II):   hash: SHA3-256 [32 bytes]
==02551== dht(DB): No-one to contact.
==02551== dht(DB): DHT worker starting 450 seconds interval.
==02551== dht(DB): No-one to contact.
==02519== irmd(II): Bootstrapped IPCP 2551.

The IRMd has created our first IPCP, in our case it has pid 2551. It is using the default policies:

  • flat addressing: the IPCP picks a random 32-bit address. The log prints it as a number (381027063), other log lines print its bytes (f7.02.b6.16).
  • multi-bit ECN (mb-ecn) congestion avoidance.
  • link-state routing, with its update, timeout and recalculation intervals, and a simple packet forwarding function (PFF). Other routing policies are loop-free alternates (lfa) and equal-cost multipath (ecmp). There are 3 PFFs, one for each QoS level the Layer supports.
  • a Distributed Hash Table (DHT) as the directory, which maps names to addresses. With only one member in the Layer, the DHT has no-one to contact yet.

The default hash for names in a unicast Layer is SHA3-256, the same as for the local Layer.

Enrolling the second IPCP

To be able to enroll, the second IPCP will need to allocate a flow to the first one. For that, the first IPCP needs to be bound to a name. We will leverage Ouroboros' anycast: instead of binding the IPCP to a single name, we will bind it to two names. The first name will be unique for each IPCP and can be used to create flows to that specific IPCP. The second name will be common for all IPCPs in the Layer. This is useful for enrollment, as a new IPCP can enroll with any IPCP in the network. 

$ irm bind ipcp uni-1 name uni-1
$ irm bind ipcp uni-1 name unicast-layer
==02519== irmd(II): Created new name: uni-1.
==02519== reg/name(DB): Process 2551 accepting flows for uni-1.
==02519== irmd(II): Bound process 2551 to name uni-1.
==02519== irmd(II): Created new name: unicast-layer.
==02519== reg/name(DB): Process 2551 accepting flows for unicast-layer.
==02519== irmd(II): Bound process 2551 to name unicast-layer.

Note that we do not register these names in any Layer. Our two IPCPs run on the same machine, as processes of the superuser, so the IRMd will connect them with direct flows, as we saw in Tutorial 01. IPCPs on different machines need a lower Layer to reach each other, which is what Tutorial 04 and Tutorial 05 are about.

With the first IPCP reachable, let's create and enroll the second one. Just like the bootstrap command, the enroll command will create an IPCP if there is no IPCP with that name yet in the system: 

$ irm ipcp enroll name uni-2 layer unicast-layer autobind

This command will do a number of things:

  1. create a unicast IPCP with name uni-2, as there is no IPCP with that name yet.
  2. allocate a flow from uni-2 to the name "unicast-layer". IPCP uni-1 is the only process that accepts flows for that name.
  3. over this flow, create a connection with the enrollment component.
  4. enroll uni-2 with uni-1: uni-1 will send its configuration (the Layer's policies), and uni-2 will configure itself with these policies.
  5. deallocate the flow used for enrollment.
  6. bind the process to the unique name uni-2 and the Layer name unicast-layer (autobind).
Enrollment of uni-2 with uni-1. The enrollment flow is a direct flow to the Layer name. Over it, OCEP selects the enrollment component and OEP carries the Layer configuration.

Let's break it down using the IRMd log output. We left out the lines that look up the certificates and security configuration, which are explained in Tutorial 01: 

==02519== irmd(II): Instantiated /usr/sbin/ipcpd-unicast as process 2602.
==02519== irmd(II): Process added: 2602 (ipcpd-unicast).
==02602== ipcpd/ipcp(II): IPCP unicast 2602 initialized.
==02519== irmd(II): Created IPCP 2602.
==02602== ipcpd/ipcp(II): Enrolling with unicast-layer...
==02602== unicast-ipcp(II): [36600d69e324762e] Requesting enrollment.
==02519== irmd(II): Allocating flow for 2602 to unicast-layer (uid 0).
==02519== irmd/oap(DB): [28ae174e102860b6] Preparing OAP request for unicast-layer.
==02519== irmd/oap(DB): [28ae174e102860b6] OAP request prepared for unicast-layer.
==02519== irmd(DB): IPCP 2551 accepting flow 1 for unicast-layer.
==02519== irmd/oap(DB): Processing OAP request for unicast-layer.
==02519== irmd/oap(DB): [28ae174e102860b6] No crt provided.
==02519== irmd/oap(II): [28ae174e102860b6] OAP request processed for unicast-layer.
==02519== irmd(II): Flow 1 accepted (direct) by 2551 for unicast-layer.
==02519== irmd/oap(DB): [28ae174e102860b6] Completing OAP for unicast-layer.
==02519== irmd/oap(DB): [28ae174e102860b6] No crt provided.
==02519== irmd/oap(II): [28ae174e102860b6] OAP completed for unicast-layer.
==02519== irmd(II): Flow 1 allocated (direct) for 2602 to unicast-layer.

The first part shows the creation of the IPCP and the (direct) flow allocation from uni-2 to uni-1. Enrollment requests get their own random 64-bit ID (here 36600d69e324762e), separate from the OAP ID of the flow, so they can be traced in the logs.

With this flow established, uni-2 connects to the enrollment component of uni-1 to get its configuration using the Ouroboros Enrollment Protocol (OEP). This connection establishment is taken care of by the connection manager in each IPCP, which sends and receives Ouroboros Connection Establishment Protocol (OCEP) information: 

==02551== connection-manager(II): Handling incoming flow 64.
==02602== connection-manager(DB): Sending OCEP info for protocol OEP to fd 64.
==02551== connection-manager(II): Request to connect to Enrollment.
==02551== connection-manager(II): Finished handling incoming flow 64 for Enrollment.
==02551== enrollment(II): Incoming enrollment connection on flow 64.
==02551== enrollment(II): [36600d69e324762e] Handling incoming enrollment.
==02551== enrollment(DB): [36600d69e324762e] Sending enrollment info (103 bytes).
==02519== reg/name(DB): Process 2551 accepting flows for unicast-layer.
==02519== reg/name(DB): Process 2551 accepting flows for uni-1.
==02602== enrollment(DB): [36600d69e324762e] Received configuration info (103 bytes).

uni-2 sends the connection establishment information for protocol OEP. uni-1 notices the request is for Enrollment, and hands the flow over to its enrollment component, which sends the configuration to uni-2. uni-2 can now start its internals using this configuration. This is the same as the bootstrap procedure, except that it uses the information received from an existing member of the Layer: 

==02602== flat-addr-auth(DB): Flat address initialized to c9.8b.1d.e3.
==02602== unicast-ipcp(II): IPCP got address 3810364361.
==02602== ca(DB): Using multi-bit ECN.
==02602== link-state-routing(DB): Using Link State Routing policy.
==02602== link-state-routing(DB): LS update interval: 15 seconds.
==02602== link-state-routing(DB): LS link timeout   : 60 seconds.
==02602== link-state-routing(DB): LS recalc interval: 4 seconds.
==02602== pff(DB): Using simple PFF policy.
==02602== pff(DB): Using simple PFF policy.
==02602== pff(DB): Using simple PFF policy.
==02602== directory(II): Using DHT policy.
==02602== dht(II): DHT initialized.
==02602== dht(DB):   ID: ef4422cc3b194fef [32 bytes].
==02602== dht(DB):   address: c9.8b.1d.e3.
==02602== dht(DB):   peer: f7.02.b6.16.
==02602== dht(DB):   magic cookie: c149382517ff35e8.
==02602== dht(II):   parameters: alpha=3, k=8, t_exp=86400, t_refr=900, t_replicate=900.

uni-2 got (random) address 3810364361. Its DHT knows uni-1 (f7.02.b6.16) as its peer, and immediately tries to contact it. That doesn't work yet, as there is no data transfer flow between the two IPCPs, so you will see these messages repeat until we connect the IPCPs below: 

==02602== dht(DB): Pinging emergency peer f7.02.b6.16.
==02602== dt(DB): Could not get nhop for f7.02.b6.16.
==02602== dht(WW): DHT_FIND_NODE_REQ write failed
==02602== dht(WW): K<ef4422cc3b194fef> Failed to send any DHT_FIND_NODE_REQ msg.
==02602== dht(WW): Failed to bootstrap peer.

Enrollment then completes, the enrollment flow is deallocated, and the autobind option binds uni-2 to its own name and the Layer name. This is shorthand for the 2 bind operations that we performed explicitly for uni-1: 

==02551== enrollment(II): [36600d69e324762e] Enrollment completed.
==02519== irmd(II): Deallocating flow 1 for process 2551 (timeout: 34 s).
==02551== enrollment(II): Enrollment flow 64 closed.
==02519== irmd(II): Deallocating flow 1 for process 2602 (timeout: 34 s).
==02602== unicast-ipcp(II): [36600d69e324762e] Enrolled with unicast-layer.
==02602== ipcpd/ipcp(II): Finished enrolling with unicast-layer in layer unicast-layer.
==02602== ipcpd/ipcp(II):   type: unicast
==02602== ipcpd/ipcp(II):   hash: SHA3-256 [32 bytes]
==02519== irmd(II): Enrolled IPCP 2602 in layer unicast-layer.
==02519== irmd(II): Created new name: uni-2.
==02519== reg/name(DB): Process 2602 accepting flows for uni-2.
==02519== irmd(II): Bound process 2602 to name uni-2.
==02519== reg/name(DB): Process 2602 accepting flows for unicast-layer.
==02519== irmd(II): Bound process 2602 to name unicast-layer.

The enrollment flow is a reliable flow, so each side lingers a bit (34 seconds here) after deallocation to finish any retransmissions.

Connecting the IPCPs

Now we have two IPCPs that are enrolled with each other. In order to transfer packets, they need a forwarding adjacency, which we call a data transfer flow for the Ouroboros Data Transfer Protocol. The routing components also need to exchange routing information, which they do over management flows. The link-state routing uses these management flows to send link-state updates about the data transfer flows in the network. The connect command creates both: 

$ irm ipcp connect name uni-2 dst uni-1

(irm ipcp connect name uni-2 dst uni-1 component mgmt or component dt creates only one of them.) 

==02519== irmd(DB): Connecting Management to uni-1.
==02602== ipcpd/ipcp(II): Connecting Management to uni-1...
==02519== irmd(II): Allocating flow for 2602 to uni-1 (uid 0).
==02519== irmd/oap(DB): [0cb0ece6399cced1] Preparing OAP request for uni-1.
==02519== irmd/oap(DB): [0cb0ece6399cced1] OAP request prepared for uni-1.
==02519== irmd(DB): IPCP 2551 accepting flow 2 for uni-1.
==02519== irmd/oap(DB): Processing OAP request for uni-1.
==02519== irmd/oap(DB): [0cb0ece6399cced1] No crt provided.
==02519== irmd/oap(II): [0cb0ece6399cced1] OAP request processed for uni-1.
==02519== irmd(II): Flow 2 accepted (direct) by 2551 for uni-1.
==02519== irmd/oap(DB): [0cb0ece6399cced1] Completing OAP for uni-1.
==02519== irmd/oap(DB): [0cb0ece6399cced1] No crt provided.
==02519== irmd/oap(II): [0cb0ece6399cced1] OAP completed for uni-1.
==02519== irmd(II): Flow 2 allocated (direct) for 2602 to uni-1.
==02602== connection-manager(DB): Sending OCEP info for protocol LSP to fd 64.
==02551== connection-manager(II): Handling incoming flow 64.
==02551== connection-manager(II): Request to connect to Management.
==02551== connection-manager(II): Finished handling incoming flow 64 for Management.
==02602== link-state-routing(DB): Type mgmt neighbor f7.02.b6.16 added.
==02602== ipcpd/ipcp(II): Finished connecting.
==02551== link-state-routing(DB): Type mgmt neighbor c9.8b.1d.e3 added.
==02519== irmd(II): Established Management connection between IPCP 2602 and uni-1.
==02519== reg/name(DB): Process 2551 accepting flows for unicast-layer.
==02519== reg/name(DB): Process 2551 accepting flows for uni-1.
==02519== irmd(DB): Connecting Data Transfer to uni-1.
==02602== ipcpd/ipcp(II): Connecting Data Transfer to uni-1...
==02519== irmd(II): Allocating flow for 2602 to uni-1 (uid 0).
==02519== irmd/oap(DB): [794d7fa680fe9cbd] Preparing OAP request for uni-1.
==02519== irmd/oap(DB): [794d7fa680fe9cbd] OAP request prepared for uni-1.
==02519== irmd(DB): IPCP 2551 accepting flow 1 for uni-1.
==02519== irmd/oap(DB): Processing OAP request for uni-1.
==02519== irmd/oap(DB): [794d7fa680fe9cbd] No crt provided.
==02519== irmd/oap(II): [794d7fa680fe9cbd] OAP request processed for uni-1.
==02519== irmd(II): Flow 1 accepted (direct) by 2551 for uni-1.
==02519== irmd/oap(DB): [794d7fa680fe9cbd] Completing OAP for uni-1.
==02519== irmd/oap(DB): [794d7fa680fe9cbd] No crt provided.
==02519== irmd/oap(II): [794d7fa680fe9cbd] OAP completed for uni-1.
==02519== irmd(II): Flow 1 allocated (direct) for 2602 to uni-1.
==02551== connection-manager(II): Handling incoming flow 65.
==02602== connection-manager(DB): Sending OCEP info for protocol dtp to fd 65.
==02551== connection-manager(II): Request to connect to Data Transfer.
==02551== connection-manager(II): Finished handling incoming flow 65 for Data Transfer.
==02602== dt(DB): Added fd 65 to packet scheduler.
==02602== link-state-routing(DB): Type dt neighbor f7.02.b6.16 added.
==02602== ipcpd/ipcp(II): Finished connecting.
==02519== reg/name(DB): Process 2551 accepting flows for unicast-layer.
==02519== reg/name(DB): Process 2551 accepting flows for uni-1.
==02551== dt(DB): Added fd 65 to packet scheduler.
==02519== irmd(II): Established Data Transfer connection between IPCP 2602 and uni-1.
==02551== link-state-routing(DB): Type dt neighbor c9.8b.1d.e3 added.
==02551== dht(II): K<ef4422cc3b194fef> Responding with 1 contacts
==02602== dht(DB): DHT worker starting 450 seconds interval.

The establishment of the management and data transfer flows follows the same procedure as the enrollment flow, handled by the connection manager. The link-state routing components now know each other as neighbours over both flows. Once the data transfer flow is up, uni-1's DHT answers uni-2, and uni-2's DHT is up and running.

The unicast Layer after the connect command: a management flow and a data transfer flow between uni-1 and uni-2.

Registering and binding oping

Now that the network is up-and-running, we can configure the oping service. We will use the name oping-server, so let's register it in our unicast Layer. We use the CLI shorthand feature, which will match the commands with the characters provided. So, instead of typing irm name register, we can use irm name reg or even irm n r: 

$ irm n r oping-server layer unicast-layer

This registers the name oping-server with both IPCPs in our network. In a real network scenario using multiple servers, this command would only register with the IPCPs on that server. Registering a name in a unicast Layer stores it, with the address of the registering IPCP, in the DHT: 

==02519== irmd(II): Created new name: oping-server.
==02551== ipcpd/ipcp(II): Registering f6c93ff2...
==02551== dht(DB): <f6c93ff23bd91094, 0000000016b602f7> Adding entry (publish).
==02551== dht(DB): <f6c93ff23bd91094, 0000000016b602f7> Adding lval.
==02551== ipcpd/ipcp(II): Finished registering f6c93ff2.
==02602== dht(DB): <f6c93ff23bd91094, 0000000016b602f7> Adding entry (store).
==02519== irmd(II): Registered oping-server with IPCP 2551 as f6c93ff2.
==02602== dht(DB): <f6c93ff23bd91094, 0000000016b602f7> Stored value until 2026-09-27 12:52:14 (UTC).
==02602== ipcpd/ipcp(II): Registering f6c93ff2...
==02602== dht(DB): <f6c93ff23bd91094, 00000000e31d8bc9> Adding lval.
==02602== ipcpd/ipcp(II): Finished registering f6c93ff2.
==02519== irmd(II): Registered oping-server with IPCP 2602 as f6c93ff2.
==02551== dht(DB): <f6c93ff23bd91094, 00000000e31d8bc9> Stored value until 2026-09-27 12:52:14 (UTC).

In the first tutorial, we started the server and then bound that process to the oping-server name. Here, we will do it a bit differently, and bind the program to that name: 

$ irm b prog oping n oping-server auto -- --listen

The command irm b prog is CLI shorthand for irm bind program. It will look for the specified binary in your PATH. The auto option, with the arguments after the --, allows the IRMd to start an oping server itself if a flow request arrives and no server is running. We will use that in Tutorial 03. 

==02519== reg/name(DB): Add prog /usr/bin/oping to name oping-server.
==02519== irmd(II): Bound program /usr/bin/oping to name oping-server.

When we now start the oping server as a regular user with oping --listen, we will see the following in the IRMd log: 

==02519== reg/pool(DB): Created PUP for uid 967 gid 966.
==02519== irmd(II): Process added: 2684 (oping).
==02519== reg/name(DB): Process 2684 accepting flows for oping-server.

Whenever we start this binary, the IRMd will automatically bind the service name for the program to that new process.

Pinging over the unicast Layer

Finally, we start the oping client to send a few pings. As in Tutorial 01, we run the client as the superuser, so that it does not share a packet pool with the server and the IRMd will not create a direct flow: 

$ sudo oping -n oping-server -c 4
Pinging oping-server with 64 bytes of data (4 packets):

64 bytes from oping-server: seq=0 time=0.441 ms
64 bytes from oping-server: seq=1 time=0.434 ms
64 bytes from oping-server: seq=2 time=0.298 ms
64 bytes from oping-server: seq=3 time=0.303 ms

--- oping-server ping statistics ---
4 packets transmitted, 4 received, 0 out-of-order, 0% packet loss, time: 4001.032 ms
rtt min/avg/max/mdev = 0.298/0.369/0.441/0.079 ms

The IRMd logs the flow allocation between the two unicast IPCPs: 

==02519== irmd(II): Process added: 2696 (oping).
==02519== irmd(II): Allocating flow for 2696 to oping-server (uid 0).
==02519== irmd/oap(DB): [835cffc634dfa2ba] Preparing OAP request for oping-server.
==02519== irmd/oap(DB): [835cffc634dfa2ba] OAP request prepared for oping-server.
==02519== irmd(DB): UID mismatch, falling back.
==02519== irmd/oap(DB): [96582d295811cc19] Preparing OAP request for oping-server.
==02519== irmd/oap(DB): [96582d295811cc19] OAP request prepared for oping-server.
==02551== ipcpd/ipcp(II): Allocating flow 6 for 2696 to f6c93ff2.
==02551== ipcpd/ipcp(II): Finished allocating flow 6 to f6c93ff2.
==02519== irmd(DB): Flow req arrived from IPCP 2602 for f6c93ff2.
==02519== irmd(II): Flow request arrived for oping-server.
==02519== irmd(DB): IPCP 2684 accepting flow 5 for oping-server.
==02519== irmd/oap(DB): Processing OAP request for oping-server.
==02519== irmd/oap(DB): [96582d295811cc19] No crt provided.
==02519== irmd/oap(II): [96582d295811cc19] OAP request processed for oping-server.
==02602== ipcpd/ipcp(II): Responding 0 to alloc on flow_id 5.
==02602== ipcpd/ipcp(II): Finished responding 0 to allocation request.
==02551== flow-allocator(DB): IPCP received msg response 0 for flow on fd 66.
==02519== irmd(II): Flow 5 accepted by 2684 for oping-server (uid 967).
==02519== irmd(DB): Response for flow 6 to oping-server.
==02519== irmd/oap(DB): [96582d295811cc19] Completing OAP for oping-server.
==02519== irmd/oap(DB): [96582d295811cc19] No crt provided.
==02519== irmd/oap(II): [96582d295811cc19] OAP completed for oping-server.
==02519== reg/name(DB): Process 2684 accepting flows for oping-server.
Flow allocation over the unicast Layer. The flow allocation request and response travel between the IPCPs over the data transfer flow.

The IRMd asks uni-1 (process 2551) to allocate the flow. uni-1 looks up the hash of oping-server in the DHT, and sends a flow allocation request over the data transfer flow to the address of uni-2 (process 2602). uni-2 passes the request to the IRMd, which hands it to our oping server. Our client process 2696 (which, because it is /usr/bin/oping, is also bound to the name oping-server, but since it doesn't call flow_accept() it is not reachable) now has a flow to the server, and the ping packets travel over the data transfer flow between uni-1 and uni-2.

The oping flow over the unicast Layer. Each end has its own flow_id, and the packets cross the Layer on the data transfer flow.

After the pings, both ends of the flow are deallocated, each through its own IPCP: 

==02519== irmd(II): Deallocating flow 6 for process 2696 (timeout: 0 s).
==02551== ipcpd/ipcp(II): Deallocating flow 6.
==02519== irmd(II): Completed deallocation of flow_id 6 by process 2551.
==02551== ipcpd/ipcp(II): Finished deallocating flow 6.
==02519== irmd(II): Process removed: 2696.
==02519== irmd(II): Deallocating flow 5 for process 2684 (timeout: 0 s).
==02602== ipcpd/ipcp(II): Deallocating flow 5.
==02519== irmd(II): Completed deallocation of flow_id 5 by process 2602.
==02602== ipcpd/ipcp(II): Finished deallocating flow 5.

If you run the metrics exporter during this tutorial, the Grafana dashboard shows each step in the state of the IPCPs. These panels come from a separate run of the commands above, with the exporter sampling every second:

From left to right, the dotted lines mark the enroll, connect, register and ping steps. The IPCPs become link-state neighbours when they connect, the DHT of each IPCP holds a value for oping-server once it is registered, and each IPCP provides one flow while the pings run. The server end stays until it times out.

Adding a local Layer

When a name is registered in multiple Layers, the IRMd tries the local Layer first. To demonstrate this, let's create the local Layer from Tutorial 01 and register the name oping-server in it: 

$ irm ipcp bootstrap type local name local-ipcp layer local-layer
$ irm n r oping-server layer local-layer
==02519== irmd(II): Instantiated /usr/sbin/ipcpd-local as process 2766.
==02519== irmd(II): Process added: 2766 (ipcpd-local).
==02766== ipcpd/ipcp(II): IPCP local 2766 initialized.
==02519== irmd(II): Created IPCP 2766.
==02766== ipcpd/ipcp(II): Bootstrapping...
==02766== ipcpd/ipcp(II): Finished bootstrapping in local-layer.
==02766== ipcpd/ipcp(II):   type: local
==02766== ipcpd/ipcp(II):   hash: SHA3-256 [32 bytes]
==02519== irmd(II): Bootstrapped IPCP 2766.
==02766== ipcpd/ipcp(II): Registering f6c93ff2...
==02766== ipcpd/ipcp(II): Finished registering f6c93ff2.
==02519== irmd(II): Registered oping-server with IPCP 2766 as f6c93ff2.

and then run the oping client again: 

$ sudo oping -n oping-server -c 4
Pinging oping-server with 64 bytes of data (4 packets):

64 bytes from oping-server: seq=0 time=0.215 ms
64 bytes from oping-server: seq=1 time=0.104 ms
64 bytes from oping-server: seq=2 time=0.162 ms
64 bytes from oping-server: seq=3 time=0.139 ms

--- oping-server ping statistics ---
4 packets transmitted, 4 received, 0 out-of-order, 0% packet loss, time: 4000.973 ms
rtt min/avg/max/mdev = 0.104/0.155/0.215/0.047 ms

We see that the ipcpd-local is now used instead of the unicast Layer: 

==02519== irmd(II): Process added: 2783 (oping).
==02519== irmd(II): Allocating flow for 2783 to oping-server (uid 0).
==02519== irmd/oap(DB): [c1950f7fdc04aaa6] Preparing OAP request for oping-server.
==02519== irmd/oap(DB): [c1950f7fdc04aaa6] OAP request prepared for oping-server.
==02519== irmd(DB): UID mismatch, falling back.
==02519== irmd/oap(DB): [53ff427500041776] Preparing OAP request for oping-server.
==02519== irmd/oap(DB): [53ff427500041776] OAP request prepared for oping-server.
==02766== ipcpd/ipcp(II): Allocating flow 5 for 2783 to f6c93ff2.
==02766== ipcpd-local(DB): Allocating flow to f6c93ff2 on fd 64.
==02519== irmd(DB): Flow req arrived from IPCP 2766 for f6c93ff2.
==02519== irmd(II): Flow request arrived for oping-server.
==02519== irmd(DB): IPCP 2684 accepting flow 7 for oping-server.
==02766== ipcpd-local(II): Pending local allocation request on fd 64.

A small note on performance: don't worry too much about the latencies being hundreds of microseconds. That's because of CPU scaling and the system not doing much (and a little bit because we are using a debug build). Latencies drop to a few microseconds if the system is under load. To see this for yourself, you can run: 

$ sudo oping -n oping-server -c 10000 --interval 0 --quiet

--- oping-server ping statistics ---
10000 packets transmitted, 10000 received, 0 out-of-order, 0% packet loss, time: 49.688 ms
rtt min/avg/max/mdev = 4.000/69.030/896.000/87.774 µs

There are a lot of things we can do to get these latencies to the lowest possible number, but those numbers usually have little relevance to real-world performance.

Notes

  1. ↑ As in Tutorial 01, this tutorial assumes you run the oping server as a regular user, not as root or as a member of the ouroboros group.
Retrieved from "https://ouroboros.rocks/mediawiki/index.php?title=Ouroboros_Tutorial_02&oldid=2045"