This supplement is if you want dual stack IPv4 abd IPv6

Note id you are doing CEPH it should be either IPv4 or IPv6 for all the monitors, MDS and daemons - do not try and dual stack it, depite the docs implying it is ok my findsing on quincy are is it is funky....  so stick yp IPv4 or IPv6 - it is possible to switch ceph back and forth - but be very careful.... it will be scary (tl;dr pick one)

## Create an IPv6 loopback
In /etc/network/intefaces you will want an IPv6 loopback for IPv6 seperate from IPv4.
My best pactice is to use the same number in the last hextet as the last octet from IPv4 - makes things easy to remember)

so PVE1 would look like this, increment the last digit of the IP for each subsequent node.
```
...
auto lo:6
iface lo:6 inet static
        address fc00::81/128

...
````
## Enable IPv4 and IPv6 forwarding
1. use `nano /etc/sysctl.conf` to open the file
2. uncomment `#net.ipv6.conf.all.forwarding=1` (remove the # symbol)
3. uncomment `#net.ipv4.ip_forward=1` (remove the # symbol)
4. save the file

## FRR Setup
This is the content for FRR - rememvber to increment the router-ids on each node you use this on where you see X

edit the frr daemons file to change `ospf6d=no` to `ospf6d=yes` and  `ospfd=no` to `ospfd=yes`

This is the config to issue in vtysh - note if you are moving from a pure IPv4 configu youy might want to stop the serice and delete the frr config file before doing this to reset the config. 

```
ip forwarding
ipv6 forwarding
!
router ospf
 ospf router-id 0.0.0.X
 log-adjacency-changes
 exit
!
router ospf6
 ospf6 router-id 0.0.0.1
 log-adjacency-changes
 timers throttle spf 100 200 5000
 exit
!
interface lo
 ip ospf area 0
 ipv6 ospf6 area 0
 exit
!
interface en05
 ip ospf area 0
 ip ospf network broadcast
 ipv6 ospf6 area 0
 ipv6 ospf6 network broadcast
 exit
!
interface en06
 ip ospf area 0
 ip ospf network broadcast
 ipv6 ospf6 area 0
 ipv6 ospf6 network broadcast
 exit
!
```
to do: speed up faiover by plaing with deadtime, hello time etc.