This will result in a routable mesh network that can survive any one node failure or any one cable failure. Alls the steps in this section must be performed on each node

Key Information Used Note my IPv6 addresses ad redacted using x for the network portion. I used addresses from my actual IPv6 range. You will need to determine the IP addresses for your environement. The NET bellow can be reused with no modification

Node l:

• lo:0 = 10.0.0.81/32
• en05 = 10.0.0.5/30
• en06 = 10.0.0.9/30
• ospf router-id = 0.0.0.1

Node 2:

• lo:0 = 10.0.0.82/32
• en05 = 10.0.0.10/30
• en06 = 10.0.0.13/30
• ospf router-id = 0.0.0.2

Node 3:

• lo:0 = 10.0.0.82/32
• en05 = 10.0.0.14/30
• en06 = 10.0.0.6/30
• ospf router-id = 0.0.0.3

Using IPv4 to take advantage of not needing to use addresses - does make things simpler

• uncomment #net.ipv4.ip_forward=1 using nano /etc/sysctl.conf (remove the # symbol and save the file)

doing this means we don't have to give each thunderbolt a manual IPv6 addrees and that these addresses stay constant no matter what Add the following to each node using nano /etc/network/interfaces

This should go uder the auto lo section and for each node the X should be 1, 2 or depending on the node

auto lo:0
iface lo:0 inet static
        address 10.0.0.8X/32

so on the first node it would look comething like this:

...
auto lo
iface lo inet loopback
 
auto lo:0
iface lo:0 inet static
        address 10.0.0.81/32
...

Save file.

1. use the table further up and assign addresses
2. after appliying both addresss remeber to hit apply configuration button

1. Install Free Range Routing (FRR) apt install frr
2. Edit the FRR config file: nano /etc/frr/daemons
3. Adjust ospfd=no to ospfd=yes
4. save the file
5. restart the service with systemctl restart frr

1. enter the FRR shell with vtysh
2. optionally show the current config with show running-config
3. enter the configure mode with configure
4. Apply the bellow configuration (it is possible to cut and paste this into the shell instead of typing it manually, you may need to press return to set the last !. Also check there were no errors in repsonse to the paste text.). Note: the X should be the number of the node you are working on, so for my stetup this would 0.0.0.1, 0.0.0.2 or 0.0.0.3.

ip forwarding
!
router ospf
 ospf router-id 0.0.0.X
 log-adjacency-changes
 exit
!
interface lo
 ip ospf area 0
 exit
!
interface en05
 ip ospf area 0
 ip ospf network point-to-multipoint
 exit
!
interface en06
 ip ospf area 0
 ip ospf network point-to-multipoint
 exit
!

5. you may need to pres return after the last ! to get to a new line - if so do this
6. exit the configure mode with the command end
7. save the configu with write memory
8. show the configure applied correctly with show running-config - note the order of the items will be different to how you entered them and thats ok. (If you made a mistake i found the easiest way was to edt /etc/frr/frr.conf - but be careful if you do that.)
9. use the command exit to leave setup
10. rpeat steps 1 to 10 on the other 3 nodes
11. once you have configured all 3 nodes issue the command vtysh -c "show ip ospf neighbor" you will see:

pve2# show ipv ospf neighbor
Neighbor ID     Pri    DeadTime    State/IfState         Duration I/F[State]

10. now issue the command show ip route and you will see:

pve2# show ip route
<I GET ONLY NORMAL ROUTES - NO OSPF BASED ONES >

10. Exit the shell with Exit

Check networking with lldpctl you should see something like this, where you will see the two other nodes (note you may also see other devices on your network that are over the 2.5gbe proxmox management interface).

I HAVE NO VALID EXAMPLE AT THIS POINT AS OSPF NOT WORKING
-------------------------------------------------------------------------------

You can now test the network by pinging FC00:: addresses of the other nodes (don't ping the node your on, and by pulling cables and seeing if it works). Note routing changes can take 15s to take effect, I am not sure how to speed up that detection - but it will be an FRR conf setting or config setting I assume. This is for future research task.