From: "emulti@disroot.org" Subject: Re: 7.6 /etc/rc blocks NFS-mounting /usr for diskless clients on boot To: tech@openbsd.org Cc: Zé Loff Date: Tue, 1 Apr 2025 15:55:36 +0800 On Tue, 1 Apr 2025 08:46:50 +0100 Zé Loff wrote: > On Tue, Apr 01, 2025 at 02:31:50PM +0800, emulti@disroot.org wrote: > > I am doing a project with net-booted diskless OpenBSD/amd64 clients and an NFS shared read-only /usr directory, as described in the diskless(8) manpage. > > > > However, I found that diskless clients are unable to mount /usr during boot, due to pf rules implemented in the standard /etc/rc. > > > > /etc/rc contains a section (starting l466) with pf rules followed by initial mounts with comment "don't kill NFS": > > > > RULES="$RULES > > pass in proto carp keep state (no-sync) > > pass out proto carp !received-on any keep state (no-sync)" > > > > if (($(sysctl -n vfs.mounts.nfs 2>/dev/null)+0 > 0)); then > > # Don't kill NFS. > > RULES="set reassemble yes no-df > > $RULES > > pass in proto { tcp, udp } from any port { sunrpc, nfsd } to any > > pass out proto { tcp, udp } from any to any port { sunrpc, nfsd } !received-on any" > > fi > > ... > > ... > > mount -s /var >/dev/null 2>&1 # cannot be on NFS > > mount -s /var/log >/dev/null 2>&1 # cannot be on NFS > > mount -s /usr >/dev/null 2>&1 # if NFS, fstab must use IP address > > > > However, the /usr/ mount doesn't make it through pf, I think because portmap is exposing dynamic reserved ports for mountd that are not in the ruleset. > > rpcinfo: > > program vers proto port > > 100000 2 tcp 111 portmapper > > 100000 2 udp 111 portmapper > > 100004 2 udp 838 ypserv > > 100004 2 tcp 669 ypserv > > 100007 2 udp 926 ypbind > > 100007 2 tcp 1007 ypbind > > 100005 1 udp 648 mountd > > 100005 3 udp 648 mountd > > 100005 1 tcp 965 mountd > > 100005 3 tcp 965 mountd > > 100003 2 udp 2049 nfs > > 100003 3 udp 2049 nfs > > 100003 2 tcp 2049 nfs > > 100003 3 tcp 2049 nfs > > 100026 1 udp 710 bootparam > > > > I couldn't work out a way to get the portmap ports simply, so made the following small change to bring the /usr mount before the pf rules are activated, which allows booting to continue: > > FWIW, I do something similar to this by adding an anchor to pf.conf: > > anchor "pxe" in on $pxe_if to $filesvr > > and then have script (well, just a long one-liner, that probably can be > improved but I don't care): > > rpcinfo -p 10.17.18.10 | awk 'NR>1 { print "pass inet proto " $3 " to port " $4 " flags any" }' | uniq | pfctl -f - -a pxe > > which is called periodically from crontab. > > > > > --- /etc/rc Mon Sep 30 22:33:10 2024 > > +++ ./rc Tue Apr 1 14:01:16 2025 > > @@ -463,6 +463,8 @@ > > pass in inet6 proto udp from any port dhcpv6-server to any port dhcpv6-client" > > fi > > > > +mount -s /usr >/dev/null 2>&1 # if NFS, fstab must use IP address > > + > > RULES="$RULES > > pass in proto carp keep state (no-sync) > > pass out proto carp !received-on any keep state (no-sync)" > > @@ -486,7 +488,6 @@ > > > > mount -s /var >/dev/null 2>&1 # cannot be on NFS > > mount -s /var/log >/dev/null 2>&1 # cannot be on NFS > > -mount -s /usr >/dev/null 2>&1 # if NFS, fstab must use IP address > > > > reorder_libs 2>&1 |& > > > > It's still not quite right- I occasionally get boot failures on clients until mountd is reloaded. I suspect this is because /var and /var/log are also on an NFS exported rootfs, in contravention of the comments above. > > Is there a better or more elegant way of doing this, or avoiding the issue, or is it worth the probably minor and transient risk of mounting /usr without pf rules running, to restore the functionality of NFS-mounting /usr on boot? > > > > -- > > Chris Billington > > > > -- >   thanks- that script would be a useful thing to have if I was running a restrictive pf set on the server and blocking RPC. But here I am talking about /etc/rc on the diskless client, early in boot, when /usr is being mounted for the first time. The default pf ruleset is applied a few lines later. Perhaps I should have made that more clear. -- Chris Billington