From BCCD 3.0

Revision as of 01:48, 10 September 2020

Stages

Stage descriptions

BCCD goes through the following network configuration stages:

1. Live mode
   1. Initial boot:
      1. Try to obtain an IP address from a BCCD DHCP server
      2. If no BCCD DHCP servers are available, obtain an IP address from any DHCP server
   2. Head node: Upon user request, start DHCP server that will answer requests only from other BCCD systems
2. Liberated mode
   1. Initial boot:
      1. Try to obtain an IP address from a BCCD DHCP server
      2. If no BCCD DHCP servers are available, obtain an IP address from any DHCP server
   2. Diskless boot: Upon user request, start DHCP server that will answer requests from diskless BCCD systems on only a given interface

Open questions:

1. Can bccd-reset-network be rewritten to produce systemd.network configuration files via templates for each stage?
2. How do we detect that a BCCD DHCP server has not responded to a client, and obtain an address from any DHCP server?

Answered questions:

1. Should we continue using bccd-identifier or switch to using the standard vendor-class-identifier that can be provided via systemd.network? vendor-class-identifier confirmed to work. dhcpd.conf snippet:

# dhclient on live BCCD systems will set this identifier
class "bccd-nodes" {
  match if option vendor-class-identifier = "bccd-client";
}

# Anything else, including PXE boot, will not
class "pxelinux-nodes" {
  match if option vendor-class-identifier != "bccd-client";
}

1. How do we have both a static (i.e. 192.168.3.1/24) and DHCP address on one interface, with a DHCP server listening on the static (192.168.3.1/24) address? Supply an Address stanza in addition to a Network stanza with DHCP set

Stage configuration

Current testing files:

• Head node
  • eth0
    • /etc/systemd/network/10-dhcp-eth0.network

[Match]
Name=eth0

[Network]
DHCP=yes

• • • or having a DHCP and static address:

[Match]
Name=eth0

[Network]
DHCP=yes

[Address]
Address=192.168.3.1/24

• • /etc/systemd/network/20-static-eth1.network

[Match]
Name=eth1

[Network]
Address=192.168.3.1/24
DHCP=no

• Client node
  • /etc/systemd/network/10-dhcp-eth0.network

[Match]
Name=eth0

[Network]
DHCP=yes

[DHCP]
VendorClassIdentifier=bccd-client

DHCP configs:

• Head node should use vendor-class-identifier to identify clients but should still send bccd-identifier = "bccd-server" to clients
• Head node can also match both vendor-class-identifier and bccd-identifier:

class "bccd-nodes" {
  match if (option vendor-class-identifier = "bccd-client") or
    (option bccd-identifier = "bccd-client");
}

• BCCD clients should require bccd-identifier (from server) but send vendor-class-identifier = "bccd-client", set in dhclient.conf (for stage detection) and systemd config file for actual network configuration
• Add LinkLocalAddresing=yes to turn on IPv4LL (aka zeroconf) addressing

systemd notes

systemd commands

Restart networking 

systemctl restart systemd-networkd

Networking status 

networkctl

systemd tricks

• Move /etc/network/interfaces out of the way to ensure that regular network setup doesn't occur
• Interface files need to have a numeric prefix and a .network suffix (i.e. /etc/systemd/network/10-static-eth0.network

Troubleshooting

RTNETLINK answers: File exists

If you get an error like this:

RTNETLINK answers: File exists
Failed to bring up eth1.

Try flushing that interface and replumbing it:

$ sudo ip addr flush dev eth1

Getting a previous IP address

If you run dhclient and it just decides to pick a previous IP address (it might even do a DHCPREQUEST, but not a DHCPDISCOVER):

1. Release the IP address: dhclient -r
2. Shutdown the interface: ifconfig nic down
3. Try doing DHCP again

mdns notes

Bookmark: https://learning.oreilly.com/library/view/zero-configuration-networking/0596101007/ch06.html

• Subscribers use multicast address 224.0.0.251, port 5353 (not 53)
  • Uses .local domain
• Hostnames
  • BCCD node could see if anyone else has registered bccd0.local and become head node if not
  • _bccd.local service could be registered with a list of all active BCCD clients (registration link)
• Uses back-off protocol for conflict resolution
• Sub-types allow for further qualification of the service being advertised
• TXT records can be used to store key/value information
• dns-sd and avahi-browse CLI utilities
  • avahi-browse -p produces parseable (rather than ini-style) output
• Interfaces
  • libavahi-compat-libdnssd-dev provides dns_sd.h header file
  • Python bindings
• avahi-discover is a graphical tool that will bring up all services/addresses/names published in the local mDNS
• Commands
  • Publish service: avahi-publish-service node000 _bccd._tcp 22
  • Publish name: avahi-publish -a -R node000.bccd.local 192.168.11.17
  • Lookup service: avahi-browse -rt _bccd._tcp
  • Lookup name: avahi-resolve -a -n node000.bccd.local
• Services
  • Avahi will automatically start any service described by an XML document named /etc/avahi/service/*.service:

<?xml version="1.0" standalone="no"?>
<!DOCTYPE service-group SYSTEM "avahi-service.dtd">
<service-group>
   <name>Bootable Cluster CD</name>
   <service>
       <type>_bccd._tcp</type>
       <port>22</port>
   </service>
</service-group>

• • Can limit which IP addresses are published via allow-interfaces in the server stanza in /etc/avahi/avahi-daemon.conf
  • Better/more-sustainable option might be to just parse out the interface in bccd-neighbors, and filter on BCCD_NIC variable set in ~/bccd.conf