Ouroboros Tutorial 01
The description of this tutorial uses a debug build with default build options.
We will start the IRMd, start a ping server and connect a client to send a few pings. This involves binding the server application to a name, after which a client can allocate a flow to that name. On the way, we will run into Ouroboros' flow allocation security, and we will see that two processes on the same machine do not need an IPC process at all. In the second part, we create a local Layer and look at when the IRMd uses it.[1]
We recommend opening multiple terminal windows for this tutorial. In the first window, start the IRMd (as a superuser) in stdout mode. After a banner, the output shows the process id (pid) of the IRMd, which will be different on your machine.
$ sudo irmd --stdout
==02067== irmd(II): Loaded certificates from /etc/ouroboros/security/cacert.
==02067== irmd(II): Loaded certificates from /etc/ouroboros/security/untrusted.
==02067== irmd(II): Ouroboros IPC Resource Manager daemon started...At startup, the IRMd loads the trusted certificates for authenticating flows. We will use those in Tutorial 06; for now, these directories are empty.
Part 1: Direct IPC
From a second terminal window, start our oping application in server mode as a regular user ("oping --help" shows all oping command line parameters):
$ oping --listen
Ouroboros ping server started.The IRMd notices that a new process has started:
==02067== reg/pool(DB): Created PUP for uid 967 gid 966.
==02067== irmd(II): Process added: 2084 (oping).Processes exchange packets through shared memory packet pools. The IRMd creates a per-user pool (PUP) for each user, which is only accessible to that user's processes. Processes of the superuser use a global pool.
The server application is not yet reachable by clients, as there are no names bound to it. Next we will bind the server to a name. The name for the server can be chosen at will, let's take “oping-server”. In a third terminal window, execute:
$ irm bind process 2084 name oping-serverThe IRMd creates the name, and acknowledges that the process is bound to it and accepting flows for it:
==02067== irmd(II): Created new name: oping-server.
==02067== reg/name(DB): Process 2084 accepting flows for oping-server.
==02067== irmd(II): Bound process 2084 to name oping-server.Now that we have bound our server, we can connect from the client. In the third terminal window, start an oping client with destination oping-server:
$ oping --server-name oping-server --count 5 #short form: oping -n oping-server -c 5
Failed to allocate flow: -1008.That did not work. The IRMd log tells us why:
==02067== irmd(II): Process added: 2109 (oping).
==02067== irmd(II): Allocating flow for 2109 to oping-server (uid 967).
==02067== irmd/oap(DB): [79582569030b4d18] Preparing OAP request for oping-server.
==02067== irmd/oap(DB): File /etc/ouroboros/security/client/oping-server/crt.pem does not exist.
==02067== irmd/oap(II): No authentication certificates for oping-server.
==02067== irmd/oap(DB): File /etc/ouroboros/security/client/oping-server/sec.conf does not exist.
==02067== irmd/oap(DB): No encryption /etc/ouroboros/security/client/oping-server/sec.conf for oping-server.
==02067== irmd/oap(II): Key exchange not configured for oping-server.
==02067== irmd/oap(DB): [79582569030b4d18] OAP request prepared for oping-server.
==02067== irmd(DB): IPCP 2084 accepting flow 1 for oping-server.
==02067== irmd/oap(DB): Processing OAP request for oping-server.
==02067== irmd/oap(DB): File /etc/ouroboros/security/server/oping-server/crt.pem does not exist.
==02067== irmd/oap(II): No authentication certificates for oping-server.
==02067== irmd/oap(DB): File /etc/ouroboros/security/server/oping-server/sec.conf does not exist.
==02067== irmd/oap(DB): No encryption /etc/ouroboros/security/server/oping-server/sec.conf for oping-server.
==02067== irmd/oap(II): Key exchange not configured for oping-server.
==02067== irmd/oap(DB): [79582569030b4d18] No crt provided.
==02067== irmd/oap(II): [79582569030b4d18] OAP request processed for oping-server.
==02067== irmd(II): Flow 1 accepted (direct) by 2084 for oping-server.
==02067== irmd/oap(DB): [79582569030b4d18] Completing OAP for oping-server.
==02067== irmd/oap(EE): [79582569030b4d18] Peer did not provide a certificate.
==02067== irmd/oap(EE): [79582569030b4d18] Failed to authenticate server.
==02067== irmd(EE): OAP completion failed for oping-server.
==02067== reg/name(DB): Process 2084 accepting flows for oping-server.
==02067== irmd(II): Process removed: 2109.Every flow allocation runs the Ouroboros Allocation Protocol (OAP) between the IRMd on the client side and the IRMd on the server side (here, both are the same IRMd). OAP carries the key exchange for encrypted flows and the certificates for authenticated flows. Each allocation gets a random 128-bit ID, so it can be traced through the logs; the logs show its first 8 bytes between square brackets.
The client side prepares an OAP request. It looks for a certificate (crt.pem) and a security configuration (sec.conf) for oping-server in the client security directory, finds neither, and so requests no encryption. The server side processes the request, looking in the server security directory, and responds. The client then checks the response, and rejects it: the server did not present a certificate.
Just like a web browser that insists on https, an Ouroboros client by default requires the server to prove its identity with a certificate. Tutorial 06 shows how to set up certificates. For now, we will tell the client that oping-server does not need to authenticate, and that we do not want encryption, by creating a client security configuration for that name:
$ sudo mkdir -p /etc/ouroboros/security/client/oping-server
$ printf 'auth=optional\nencryption=none\n' | sudo tee /etc/ouroboros/security/client/oping-server/sec.conf
auth=optional
encryption=noneThe IRMd reads this file on every flow allocation, so there is no need to restart it. Let's try again:
$ oping -n oping-server -c 5
Pinging oping-server with 64 bytes of data (5 packets):
64 bytes from oping-server: seq=0 time=55.000 µs
64 bytes from oping-server: seq=1 time=0.100 ms
64 bytes from oping-server: seq=2 time=0.206 ms
64 bytes from oping-server: seq=3 time=0.273 ms
64 bytes from oping-server: seq=4 time=26.000 µs
--- oping-server ping statistics ---
5 packets transmitted, 5 received, 0 out-of-order, 0% packet loss, time: 5001.070 ms
rtt min/avg/max/mdev = 0.026/0.132/0.273/0.104 msThe server output shows two flows:
New flow 64.
New flow 65.
Received 64 bytes on fd 65.
Received 64 bytes on fd 65.
Received 64 bytes on fd 65.
Received 64 bytes on fd 65.
Received 64 bytes on fd 65.
Flow 64 timed out.
Flow 65 timed out.Flow descriptor 64 is the flow from our first attempt. The server accepted that flow when it responded to the OAP request, before the client rejected the response, and the client does not send its verdict back. Like any flow whose peer has gone, it times out at the server 10 seconds after the last packet, in this case without having received any. Flow descriptor 65 carries our pings.
The IRMd log shows the flow allocation. We left out the lines that look up the certificates and security configuration, which are the same as before:
==02067== irmd(II): Process added: 2117 (oping).
==02067== irmd(II): Allocating flow for 2117 to oping-server (uid 967).
==02067== irmd/oap(DB): [8bda96eaecaf505d] Preparing OAP request for oping-server.
==02067== irmd/oap(DB): [8bda96eaecaf505d] OAP request prepared for oping-server.
==02067== irmd(DB): IPCP 2084 accepting flow 2 for oping-server.
==02067== irmd/oap(DB): Processing OAP request for oping-server.
==02067== irmd/oap(DB): [8bda96eaecaf505d] No crt provided.
==02067== irmd/oap(II): [8bda96eaecaf505d] OAP request processed for oping-server.
==02067== irmd(II): Flow 2 accepted (direct) by 2084 for oping-server.
==02067== irmd/oap(DB): [8bda96eaecaf505d] Completing OAP for oping-server.
==02067== irmd/oap(DB): [8bda96eaecaf505d] No crt provided.
==02067== irmd/oap(II): [8bda96eaecaf505d] OAP completed for oping-server.
==02067== irmd(II): Flow 2 allocated (direct) for 2117 to oping-server.
==02067== reg/name(DB): Process 2084 accepting flows for oping-server.This time, the client accepts the server without a certificate, and OAP completes. Note the word direct: there is no IPC process (IPCP) involved in this flow at all. Both oping processes are on the same machine and share a packet pool, so the IRMd connects them directly over shared memory. We did not have to create any Layer or register any name. The flow has flow_id 2, and both ends of a direct flow share the same flow_id. After it has accepted the flow, the oping server calls flow_accept() again, and the IRMd reports that it accepts flows for “oping-server”.
After all packets have been sent, the oping client (process 2117) deallocates the flow and exits. Ten seconds later, both flows time out at the server, and the server deallocates its end of each flow:
==02067== irmd(II): Deallocating flow 2 for process 2117 (timeout: 0 s).
==02067== irmd(II): Process removed: 2117.
==02067== irmd(II): Deallocating flow 1 for process 2084 (timeout: 0 s).
==02067== irmd(II): Deallocating flow 2 for process 2084 (timeout: 0 s).The timeout shown is how long the flow should linger to finish retransmissions. Our flow is a raw flow, without retransmission, so it is 0.
Part 2: The local Layer
A direct flow is only possible between processes that share a packet pool. In all other cases, the IRMd needs an IPC process (IPCP) to carry the flow. The simplest type is the local IPCP. It is a kind of loopback interface: it implements all the functions that the O7s API provides, but only for communication between processes on the same (local) machine.
The ipcp create CLI command instantiates a new IPCP, but it will not be a part of a Layer. We also need to bootstrap this IPCP in a Layer, we will name it “local-layer”. As a shortcut, the ipcp bootstrap command automatically creates an IPCP if no IPCP by that name exists, so the ipcp create command is usually optional. In the third terminal, enter the command:
$ irm ipcp bootstrap type local name local-ipcp layer local-layerThe IRMd and IPCP output in the first terminal reads:
==02067== irmd(II): Instantiated /usr/sbin/ipcpd-local as process 2178.
==02067== irmd(II): Process added: 2178 (ipcpd-local).
==02178== ipcpd/ipcp(II): IPCP local 2178 initialized.
==02067== irmd(II): Created IPCP 2178.
==02178== ipcpd/ipcp(II): Bootstrapping...
==02178== ipcpd/ipcp(II): Finished bootstrapping in local-layer.
==02178== ipcpd/ipcp(II): type: local
==02178== ipcpd/ipcp(II): hash: SHA3-256 [32 bytes]
==02067== irmd(II): Bootstrapped IPCP 2178.To be reachable over this Layer, the name “oping-server” has to be registered in it:
$ irm name register oping-server layer local-layerThe IPCP will acknowledge that the name is registered:
==02178== ipcpd/ipcp(II): Registering f6c93ff2...
==02178== ipcpd/ipcp(II): Finished registering f6c93ff2.
==02067== irmd(II): Registered oping-server with IPCP 2178 as f6c93ff2.Ouroboros registers names not in plaintext but using a (configurable) hashing algorithm. The default hashing algorithm for a local Layer is a 256-bit SHA3 hash. The output in the logs is truncated to the first 4 bytes in a HEX notation.
To get the IRMd to use this Layer, we need a client that does not share a packet pool with the server. Our server runs as a regular user in its own per-user pool, so let's run the client as the superuser:
$ sudo oping -n oping-server -c 5
Pinging oping-server with 64 bytes of data (5 packets):
64 bytes from oping-server: seq=0 time=0.315 ms
64 bytes from oping-server: seq=1 time=0.141 ms
64 bytes from oping-server: seq=2 time=0.274 ms
64 bytes from oping-server: seq=3 time=0.377 ms
64 bytes from oping-server: seq=4 time=0.222 ms
--- oping-server ping statistics ---
5 packets transmitted, 5 received, 0 out-of-order, 0% packet loss, time: 5001.616 ms
rtt min/avg/max/mdev = 0.141/0.266/0.377/0.090 msThe IRMd and IPCP logs show the flow allocation, again without the certificate and configuration lookups:
==02067== irmd(II): Process added: 2202 (oping).
==02067== irmd(II): Allocating flow for 2202 to oping-server (uid 0).
==02067== irmd/oap(DB): [44e049673dcff8e4] Preparing OAP request for oping-server.
==02067== irmd/oap(DB): [44e049673dcff8e4] OAP request prepared for oping-server.
==02067== irmd(DB): UID mismatch, falling back.
==02067== irmd/oap(DB): [4cd232bda0cd0011] Preparing OAP request for oping-server.
==02067== irmd/oap(DB): [4cd232bda0cd0011] OAP request prepared for oping-server.
==02178== ipcpd/ipcp(II): Allocating flow 1 for 2202 to f6c93ff2.
==02178== ipcpd-local(DB): Allocating flow to f6c93ff2 on fd 64.
==02067== irmd(DB): Flow req arrived from IPCP 2178 for f6c93ff2.
==02067== irmd(II): Flow request arrived for oping-server.
==02067== irmd(DB): IPCP 2084 accepting flow 3 for oping-server.
==02178== ipcpd-local(II): Pending local allocation request on fd 64.
==02178== ipcpd/ipcp(II): Finished allocating flow 1 to f6c93ff2.
==02067== irmd/oap(DB): Processing OAP request for oping-server.
==02067== irmd/oap(DB): [4cd232bda0cd0011] No crt provided.
==02067== irmd/oap(II): [4cd232bda0cd0011] OAP request processed for oping-server.
==02178== ipcpd/ipcp(II): Responding 0 to alloc on flow_id 3.
==02067== irmd(DB): Response for flow 1 to oping-server.
==02067== irmd/oap(DB): [4cd232bda0cd0011] Completing OAP for oping-server.
==02067== irmd/oap(DB): [4cd232bda0cd0011] No crt provided.
==02067== irmd/oap(II): [4cd232bda0cd0011] OAP completed for oping-server.
==02178== ipcpd-local(II): Flow allocation completed, fds (64, 65).
==02178== ipcpd/ipcp(II): Finished responding 0 to allocation request.
==02067== irmd(II): Flow 3 accepted by 2084 for oping-server (uid 967).
==02067== reg/name(DB): Process 2084 accepting flows for oping-server.
The IRMd first tries a direct flow, but the client (uid 0) and the server (uid 967) are in different pools, so it falls back to an IPCP. It resolves the name to its hash and asks the local IPCP (2178) to allocate a flow towards “f6c93ff2”, on behalf of the client with flow_id 1 (the rejected flow from Part 1 has released it). The IPCP creates a flow endpoint with flow descriptor (fd) 64, and passes the flow request, with the OAP request, back to the IRMd. The IRMd resolves the hash to “oping-server”, a name that process 2084 accepts flows for, and creates the server end of the flow with flow_id 3. After the server end has processed the OAP request, the IPCP responds, and the IRMd completes OAP on the client side with the server's response. The IPCP completes the flow internally between its endpoints with flow descriptors 64 and 65, which map to flow_id 1 and flow_id 3 respectively. The IPCP cannot directly access flow_ids, they are assigned and managed by the IRMd.
After all packets have been sent, the oping client (process 2202) deallocates the flow and exits, and 10 seconds later, the flow times out at the server side:
==02067== irmd(II): Deallocating flow 1 for process 2202 (timeout: 0 s).
==02178== ipcpd/ipcp(II): Deallocating flow 1.
==02067== irmd(II): Completed deallocation of flow_id 1 by process 2178.
==02178== ipcpd-local(II): Flow with fd 64 deallocated.
==02178== ipcpd/ipcp(II): Finished deallocating flow 1.
==02067== irmd(II): Process removed: 2202.
==02067== irmd(II): Deallocating flow 3 for process 2084 (timeout: 0 s).
==02178== ipcpd/ipcp(II): Deallocating flow 3.
==02067== irmd(II): Completed deallocation of flow_id 3 by process 2178.
==02178== ipcpd-local(II): Flow with fd 65 deallocated.
==02178== ipcpd/ipcp(II): Finished deallocating flow 3.Deallocation happens independently at each side of the flow. It is initiated by the process that owns the flow endpoint, and finalized by the IPCP. The first 5 lines show the deallocation by the oping client and the IPCP, and line 6 shows the client process has exited. The last 5 lines show the same for the server end of the flow.
This concludes this first short tutorial. All running processes can be terminated by issuing a Ctrl-C command in their respective terminals.
Notes
- ↑ This tutorial assumes you run the oping server as a regular user, not as root or as a member of the ouroboros group. If you do, the flow in Part 2 will be direct, just like in Part 1.