From: patrick keshishian <sidster@boxsoft.com>
Subject: Re: Earlier /tmp cleanup in /etc/rc
To: Theo de Raadt <deraadt@openbsd.org>
Cc: Andrew Hewus Fresh <andrew@afresh1.com>, tech@openbsd.org, kn@openbsd.org
Date: Sat, 29 Aug 2026 00:09:01 -0700

On Wed, Aug 26, 2026 at 09:14:25PM -0600, Theo de Raadt wrote:
> I think that email failed to explain the various parts of the problem
> and the solution we've arrived at.
> 
> It needs to be explained, in case we still have something wrong after
> the 50th iteration...
> 
> > Index: rc
> > ===================================================================
> > RCS file: /cvs/src/etc/rc,v
> > diff -u -p -r1.590 rc
> > --- rc	17 Aug 2026 22:41:38 -0000	1.590
> > +++ rc	27 Aug 2026 01:25:03 -0000
> > @@ -502,6 +502,24 @@ sysctl_conf
> >  mount -s /var >/dev/null 2>&1		# cannot be on NFS
> >  mount -s /var/log >/dev/null 2>&1	# cannot be on NFS
> >  mount -s /usr >/dev/null 2>&1		# if NFS, fstab must use IP address
> > +mount -s /tmp >/dev/null 2>&1		# if NFS, fstab must use IP address
> 
> We mount /tmp very early, so that we can prune it early.  Look below and
> you see reorder_libs &.  That runs asyncronously.  We don't know if a /tmp
> pruner running in parallel to that will delete files.  The same applies to
> other daemons that get started.  Previous sersion of the /tmp pruning code
> could still be running while sshd is starting, or possibly later.
> 
> So the goals now are:
> 
> 1) free up at least one inode, because mkstemp needs at least one
> 2) free up a few more, just in case.  We settled on 50.
> 3) With an inode freed up for mktemp, create a directory and move all the
>    deleteable files into that directory and remove it asyncronously.

I don't have a patch, and it is late, so I shouldn't be making
any noise, but, just perchance this does help I had to chime in.

Reading the clearly stated description above, I had to wonder if
all this making a temp dir, to which to move deletable files, so
that the pruner does not delete files created by reorder_libs and
friends, can be avoided if one hoisted up the patching of /etc/motd,
appended "! -anewer /etc/motd" to $skip to be used in one simple
find and delete command.

(sorry about the run-on sentence)
--patrick


> > +
> > +# Prune /tmp before users can authenticate.  A few files are removed
> > +# immediately, but most of the deletion happens asynchronously while
> > +# daemons are being started
> > +( cd /tmp && {
> > +    echo clearing /tmp
> > +    skip="-mindepth 1 -maxdepth 1 ! -name lost+found ! -name quota.user 
> > +	! -name quota.group ! -name vi.recover"
> > +    find . $skip ! -perm -1000 -print0 |
> > +	xargs -r0J % find -d % -exec rm -rf -- {} \; -print |
> > +	    head -50 >/dev/null 2>&1
> 
> This is the step which prunes 50 inodes.  It is very complicated.
> 
> The first find runs without -d, so that it can generate a list of top-level
> objects except for a) +t directories, b) lost+found, quota.user, quota.group,
> vi.recover.
> 
> Anything else at the top-level is a target for removing inodes.
> 
> The next clever put is that we are going to print every (single file
> or directory) as we delete it, and pipe that to head -50.  head -50
> will stop file deletions when we reach the limit.  This printing may
> include files with newlines in it, which means we may delete less than
> 50.  In reality we definately only need 1, ok maybe 3, inodes
> 
> The nested find now processes the permitted files and directories, finding
> one file at a time, bottom up, because of -d.  The -print is located
> after the rm -rf -- {} so that the failure to remove a file (for whatever
> reason does not) advance the counter towards 50 files.
> 
> Of course, the failure to remove files will be noisy, so the entire
> contraption redirects errors to /dev/null.
> 
> Now that we have enough inodes, we to hunt the other files:
> 
> > +    tmpdir=$(mktemp -d .delXXXXXXXXXX) && {
> > +	find . $skip ! -name "$tmpdir" -print0 |
> > +	    xargs -r0J % mv -f -- % "$tmpdir"
> > +	rm -rf -- "$tmpdir" &
> > +    }
> > +} )
> 
> We create a mktemp directory, move all the permitted files into that
> directory, and then asyncronously delete that directory.  This can
> take as long as it needs to, while the system boots, because those
> files cannot be reached because they are inside a mktemp directory.
> 
> I think there is still reason to redirect the final rm -rf (or maybe
> even the find + mv to /dev/null, becuase noone wants to see any noise
> from this.
>