Ouroboros Tutorial 04
In this tutorial, we show how to run oping between two machines connected to the same Ethernet LAN.
For the second machine, we use a Raspberry Pi Model 4B+ (ARM64, 32-bit Raspbian 11) connected to the same WiFi LAN as my desktop (Intel 13900KF, ArchLinux).
We chose the Ethernet DIX option, because our LLC packets sometimes get dropped on wireless LANs (protocol ossification at work). To use the LLC Ethernet 0-Layer, all that is needed is to change eth-dix to eth-llc in (both) the configuration files in this tutorial (left to the reader).
We run the server on the raspberry pi. The O7s config file for the Raspberry Pi connects to the wlan0 device and registers the "oping-server" application with the Ethernet 0-Layer:
[name."oping-server"] prog=["/usr/bin/oping"] args=["-l"]
[eth-dix.wifi1] bootstrap="wifi-network" dev="wlan0" reg=["oping-server"]
The configuration for the desktop machine (O7s client) is quite simple:
[eth-dix.wifi1] bootstrap="wifi-network" dev="wlan0"
Save these configurations as a file on each machine - we chose to save them at /etc/ouroboros/tut04.conf - and start an IRMd on each machine with that config file. The IRMd will log the use of the configuration file, on the raspberry pi it will create the service name for oping-server:
dstaesse@raspberrypi:~ $ sudo irmd --stdout --config /etc/ouroboros/tut04.conf ==28267== irmd(II): Ouroboros IPC Resource Manager daemon started... ==28267== irmd/configuration(II): Reading configuration from file /etc/ouroboros/tut04.conf ==28267== irmd/configuration(DB): Found service name oping-server in configuration file. ==28267== irmd(II): Created new name: oping-server. ==28267== irmd(II): Bound program /usr/bin/oping to name oping-server.
The IRMd then bootstraps the IPCP (this happens on both machines):
==28267== irmd/configuration(DB): Found IPCP wifi1 in configuration file. ==28267== irmd(II): Created IPCP 28279. ==28267== irmd/configuration(DB): No hash specified, using default. ==28279== ipcpd/ipcp(II): Bootstrapping... ==28279== ipcpd/eth-dix(DB): Device MTU is 1500. ==28279== ipcpd/eth-dix(DB): Layer MTU is 1500. ==28279== ipcpd/eth-dix(II): Using raw socket device. ==28279== ipcpd/eth-dix(DB): Bootstrapped IPCP over DIX Ethernet with pid 28279 and Ethertype 0xA000. ==28279== ipcpd/ipcp(II): Finished bootstrapping: 0. ==28267== irmd(II): Bootstrapped IPCP 28279 in layer wifi-network.
The ipcpd-eth-* detects the device MTU (which is usually 1500 bytes, but can be lower on virtual machine guests, or higher, for instance the loopback interface). The ipcpd-eth-* Layer MTU will be set to the device MTU, or to IPCP_ETH_LO_MTU when using the loopback adapter. We will probably make the Layer MTU a runtime configuration parameter in the future, so it can be configured in the configuration file. The Raspberry Pi is using Linux, which defaults to a raw socket device for interfacing with Ethernet devices. Other options for interfacing with Ethernet are Berkeley Packet Filter (BPF) (for *BSD and OS X) or netmap (if installed). Our IPCP will process all packets on a specified Ethertype. The default value is 0xA000.
The IRMd on the raspberry Pi will make the oping service available on the Ethernet 0-Layer:
==28267== irmd/configuration(DB): Registering oping-server in 28279 ==28267== irmd(WW): Name oping-server already exists. ==28279== ipcpd/ipcp(II): Registering f6c93ff2... ==28279== ipcpd/ipcp(II): Finished registering f6c93ff2 : 0. ==28267== irmd(II): Registered oping-server with IPCP 28279 as f6c93ff2.
With the IRMd started on both machines, we can start the oping server. We use the --quiet (-Q) option to reduce output in the example (oping -l -Q).
In addition to demonstrating the use of the ipcpd-eth-dix, we will also show the use of encrypted flows. To make this more visible, we dump the traffic on the Raspberry Pi's wlan0 device for ethertype 0xA000 using tcpdump.
dstaesse@raspberrypi:~ $ sudo tcpdump -i wlan0 ether proto 0xa000 tcpdump: verbose output suppressed, use -v[v]... for full protocol decode listening on wlan0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
Our desktop machine has MAC address c8:cb:9e:d0:04:08, the Raspberry Pi has MAC address d8:3a:dd:39:0a:b5.
Our ipcpd-eth-dix has the following packet header, note that the dst and src hardware (MAC) addresses and the ethertype are shown on the first line of the tcpdump output. The hex output starts at eid.
uint8_t dst_hwaddr[6]; uint8_t src_hwaddr[6]; uint16_t ethertype;
uint16_t eid; uint16_t length;
The endpoint identifier (EID, eid field in the header above) is used for multiplexing flows, in the ipcpd-eth-* it is 16 bits wide and set to the flow descriptor used by the application (in the ipcpd-unicast the EID is 64 bits and assigned in a random way). The length field is needed to deal with Ethernet frames that would be shorter than 64 bytes, which are padded with zero bytes, and we need to know if those zeroes are padding or payload. These are known as runt frames.
For readers also trying the ipcpd-eth-llc, the header format is a little bit different: no eid instead we use LLC SAPs as endpoint identifiers and no length (Ethernet LLC has its own length field instead of the Ethertype to deal with runt frames). The control field cf is fixed and set to LLC type 1 (0x03).
When we ping (one ping only) from the desktop (oping -n oping-server -c 1), we see the traffic exchange on the wlan. Let's break it down:
The client initiates the communications using the flow_alloc() function call. This will send a Flow Allocation Request message to the server. We have not requested anything special, so this will be a request for a vanilla best-effort raw flow:
09:57:08.998005 c8:cb:9e:d0:04:08 (oui Unknown) > d8:3a:dd:39:0a:b5 (oui Unknown), ethertype Unknown (0xa000), length 88:
0x0000: 0000 0046 0040 0000 0000 0001 0000 0000 ...F.@..........
0x0010: 0000 0000 0000 0001 ffff ffff ffff ffff ................
0x0020: 0001 d4c0 0000 0000 0000 f6c9 3ff2 3bd9 ............?.;.
0x0030: 1094 3a77 8e7b 3faf 7bff ec03 796d b676 ..:w.{?.{...ym.v
0x0040: 1bfa 5f70 758f 9ee2 e6fa .._pu.....
The first two bytes are the endpoint id, which is 0 (0000): this identifies the flow allocator. The next two bytes is the length of the payload, which is 70 bytes (0046).
The packet is a flow allocation message (fixed packet format for the flow allocator)
uint16_t src eid: 0040 = 64 (EID assigned to the oping client) uint16_t dst eid: 0000 = 0 (not used in allocation request) uint32_t loss: 0000 0001 = 1 (disables ARQ) uint64_t bandwidth: 0000 0000 0000 0000 = 0 (no reservation) uint32_t ber: 0000 0001 = 1 (disables CRC) uint32_t max_gap: ffff ffff = MAX_INT (any inter-packet gap is fine) uint32_t delay: ffff ffff = MAX_INT (any delay is fine) uint32_t timeout: 0001 d4c0 = 120000 (2 minutes) uint16_t cypher_s: 0000 = 0 (no encryption) uint8_t in_order: 00 = 0 (no in-order delivery) uint8_t code: 00 = 0 (flow allocation request) uint8_t availability: 00 = 0 (9's, any availability will do) int8_t response 00 = 0 (not used in flow allocation request)
The final 32 bytes of the message (f6c9 3f.. ..e2 e6fa) is the SHA3-256 hash of oping-server. When the flow is accepted at the server side (on the Raspberry Pi), the ipcpd-eth-dix responds with a flow allocation response.
The format is the same as above, without the destination has, and the QoS is set to 0 (QoS negotiation is not yet implemented). The main difference is the
09:57:09.000685 d8:3a:dd:39:0a:b5 (oui Unknown) > c8:cb:9e:d0:04:08 (oui Unknown), ethertype Unknown (0xa000), length 56:
0x0000: 0000 0026 0040 0040 0000 0000 0000 0000 ...&.@.@........
0x0010: 0000 0000 0000 0000 0000 0000 0000 0000 ................
0x0020: 0000 0000 0000 0001 0000 ..........
The full explanation of the response is:
uint16_t src eid: 0040 = 64 (EID assigned to the oping server) uint16_t dst eid: 0040 = 64 (EID assigned to the oping client) /* QoS all zero, we explain last 32 bits: */ uint8_t in_order: 00 = 0 (part of QoS, zero) uint8_t code: 01 = 1 (flow allocation response) uint8_t availability: 00 = 0 (part of QoS, zero) int8_t response 00 = 0 (flow allocation request was successful)
The next message is
09:57:10.020082 c8:cb:9e:d0:04:08 (oui Unknown) > d8:3a:dd:39:0a:b5 (oui Unknown), ethertype Unknown (0xa000), length 82:
0x0000: 0040 0040 0000 0000 0000 0000 e307 0000 .@.@............
0x0010: 0000 0000 aa02 f70d 0000 0000 0000 0000 ................
0x0020: 0000 0000 0000 0000 0000 0000 0000 0000 ................
0x0030: 0000 0000 0000 0000 0000 0000 0000 0000 ................
0x0040: 0000 0000 ....
09:57:10.020890 d8:3a:dd:39:0a:b5 (oui Unknown) > c8:cb:9e:d0:04:08 (oui Unknown), ethertype Unknown (0xa000), length 82:
0x0000: 0040 0040 0000 0001 0000 0000 e307 0000 .@.@............
0x0010: 0000 0000 aa02 f70d 0000 0000 0000 0000 ................
0x0020: 0000 0000 0000 0000 0000 0000 0000 0000 ................
0x0030: 0000 0000 0000 0000 0000 0000 0000 0000 ................
0x0040: 0000 0000