On Tue, Apr 01, 2025 at 02:31:50PM +0800, emulti@disroot.org wrote:
> I am doing a project with net-booted diskless OpenBSD/amd64 clients and an NFS shared read-only /usr directory, as described in the diskless(8) manpage.
> 
> However, I found that diskless clients are unable to mount /usr during boot, due to pf rules implemented in the standard /etc/rc.
> 
> /etc/rc contains a section (starting l466) with pf rules followed by initial mounts with comment "don't kill NFS":
> 
>         RULES="$RULES
>         pass in proto carp keep state (no-sync)
>         pass out proto carp !received-on any keep state (no-sync)"
> 
>         if (($(sysctl -n vfs.mounts.nfs 2>/dev/null)+0 > 0)); then
>                 # Don't kill NFS.
>                 RULES="set reassemble yes no-df
>                 $RULES
>                 pass in proto { tcp, udp } from any port { sunrpc, nfsd } to any
>                 pass out proto { tcp, udp } from any to any port { sunrpc, nfsd } !received-on any"
>         fi
> ...
> ...
> mount -s /var >/dev/null 2>&1           # cannot be on NFS
> mount -s /var/log >/dev/null 2>&1       # cannot be on NFS
> mount -s /usr >/dev/null 2>&1           # if NFS, fstab must use IP address
> 
> However, the /usr/ mount doesn't make it through pf, I think because portmap is exposing dynamic reserved ports for mountd that are not in the ruleset.
> rpcinfo: 
>    program vers proto   port
>     100000    2   tcp    111  portmapper
>     100000    2   udp    111  portmapper
>     100004    2   udp    838  ypserv
>     100004    2   tcp    669  ypserv
>     100007    2   udp    926  ypbind
>     100007    2   tcp   1007  ypbind
>     100005    1   udp    648  mountd
>     100005    3   udp    648  mountd
>     100005    1   tcp    965  mountd
>     100005    3   tcp    965  mountd
>     100003    2   udp   2049  nfs
>     100003    3   udp   2049  nfs
>     100003    2   tcp   2049  nfs
>     100003    3   tcp   2049  nfs
>     100026    1   udp    710  bootparam
> 
> I couldn't work out a way to get the portmap ports simply, so made the following small change to bring the /usr mount before the pf rules are activated, which allows booting to continue:

FWIW, I do something similar to this by adding an anchor to pf.conf:

    anchor "pxe"   in on $pxe_if to $filesvr

and then have script (well, just a long one-liner, that probably can be
improved but I don't care):

    rpcinfo -p 10.17.18.10 | awk 'NR>1 { print "pass inet proto " $3 " to port "  $4 " flags any" }' | uniq | pfctl -f - -a pxe

which is called periodically from crontab.

> 
> --- /etc/rc     Mon Sep 30 22:33:10 2024
> +++ ./rc        Tue Apr  1 14:01:16 2025
> @@ -463,6 +463,8 @@
>                 pass in inet6 proto udp from any port dhcpv6-server to any port dhcpv6-client"
>         fi
>  
> +mount -s /usr >/dev/null 2>&1          # if NFS, fstab must use IP address
> +
>         RULES="$RULES
>         pass in proto carp keep state (no-sync)
>         pass out proto carp !received-on any keep state (no-sync)"
> @@ -486,7 +488,6 @@
>  
>  mount -s /var >/dev/null 2>&1          # cannot be on NFS
>  mount -s /var/log >/dev/null 2>&1      # cannot be on NFS
> -mount -s /usr >/dev/null 2>&1          # if NFS, fstab must use IP address
>  
>  reorder_libs 2>&1 |&
> 
> It's still not quite right- I occasionally get boot failures on clients until mountd is reloaded. I suspect this is because /var and /var/log are also on an NFS exported rootfs, in contravention of the comments above.
> Is there a better or more elegant way of doing this, or avoiding the issue, or is it worth the probably minor and transient risk of mounting /usr without pf rules running, to restore the functionality of NFS-mounting /usr on boot?
> 
> -- 
> Chris Billington
> 

--