info_tfgrid/collections/developers/internals/zos/internals/network/topology

On boot

this is setup by internet daemon, which is part of the bootstrap process.

the first basic network setup is done, the point of this setup is to connect the node to the internet, to be able to continue the rest of the boot process.

• Go over all PLUGGED, and PHYSICAL interfaces
• For each matching interface, the interface is tested if it can get both IPv4 and IPv6
• If multiple interfaces have been found to receive ipv4 from dhcp, we find the smallest ip, with the private gateway IP, otherwise if no private gateway ip found, we only find the one with the smallest IP.
• Once the interface is found we do the following: (we will call this interface eth)
  • Create a bridge named zos
  • Disable IPv6 on this bridge, and ipv6 forwarding
• Run udhcpc on zos bridge

Once this setup complete, the node now has access to the internet which allows it to download and run networkd which takes over the network stack and continue the process as follows.

Network Daemon

• Validate zos setup created by the internet on boot daemon
• Send information about all local nics to the explorer (?)

Setting up ndmz

First we need to find the master interface for ndmz, we have the following cases:

• master of public_config if set. Public Config is an external configuration that is set by the farmer on the node object. that information is retrieved by the node from the public explorer.
• otherwise (if public_config is not set) check if the public namespace is set (i think that's a dead branch because if this exist (or can exist) it means the master is always set. which means it will get used always.
• otherwise find first interface with ipv6
• otherwise check if zos has global unicast ipv6
• otherwise hidden node (still uses zos but in hidden node setup)

Hidden node ndmz

Dualstack ndmz

Setting up Public Config

this is an external configuration step that is configured by the farmer on the node object. The node then must have setup in the explorer.

Setting up Yggdrasil

• Get a list of all public peers with status up
• If hidden node:
  • Find peers with IPv4 addresses
• If dual stack node:
  • Filter out all peers with same prefix as the node, to avoid connecting locally only
• write down yggdrasil config, and start yggdrasil daemon via zinit
• yggdrasil runs inside the ndmz namespace
• add an ipv6 address to npub in the same prefix as yggdrasil. this way when npub6 is used as a gateway for this prefix, traffic will be routed through yggdrasil.

Creating a network resource

A network resource (NR for short) as a user private network that lives on the node and can span multiple nodes over wireguard. When a network is deployed the node builds a user namespace as follows:

• A unique network id is generated by md5sum(user_id + network_name) then only take first 13 bytes. We will call this net-id.

Create the wireguard interface

if the node has public_config so the public namespace exists. then the wireguard device is first created inside the public namespace then moved to the network-resource namespace.

Otherwise, the port is created on the host namespace and then moved to the network-resource namespace. The final result is

Finally the wireguard peer list is applied and configured, routing rules is also configured to route traffic to the wireguard interface

Member joining a user network (network resource)