From: "Theo de Raadt" <deraadt@openbsd.org>
Subject: Re: Earlier /tmp cleanup in /etc/rc
To: Andrew Hewus Fresh <andrew@afresh1.com>
Cc: tech@openbsd.org, kn@openbsd.org
Date: Wed, 26 Aug 2026 21:14:25 -0600

I think that email failed to explain the various parts of the problem
and the solution we've arrived at.

It needs to be explained, in case we still have something wrong after
the 50th iteration...

> Index: rc
> ===================================================================
> RCS file: /cvs/src/etc/rc,v
> diff -u -p -r1.590 rc
> --- rc	17 Aug 2026 22:41:38 -0000	1.590
> +++ rc	27 Aug 2026 01:25:03 -0000
> @@ -502,6 +502,24 @@ sysctl_conf
>  mount -s /var >/dev/null 2>&1		# cannot be on NFS
>  mount -s /var/log >/dev/null 2>&1	# cannot be on NFS
>  mount -s /usr >/dev/null 2>&1		# if NFS, fstab must use IP address
> +mount -s /tmp >/dev/null 2>&1		# if NFS, fstab must use IP address

We mount /tmp very early, so that we can prune it early.  Look below and
you see reorder_libs &.  That runs asyncronously.  We don't know if a /tmp
pruner running in parallel to that will delete files.  The same applies to
other daemons that get started.  Previous sersion of the /tmp pruning code
could still be running while sshd is starting, or possibly later.

So the goals now are:

1) free up at least one inode, because mkstemp needs at least one
2) free up a few more, just in case.  We settled on 50.
3) With an inode freed up for mktemp, create a directory and move all the
   deleteable files into that directory and remove it asyncronously.

> +
> +# Prune /tmp before users can authenticate.  A few files are removed
> +# immediately, but most of the deletion happens asynchronously while
> +# daemons are being started
> +( cd /tmp && {
> +    echo clearing /tmp
> +    skip="-mindepth 1 -maxdepth 1 ! -name lost+found ! -name quota.user 
> +	! -name quota.group ! -name vi.recover"
> +    find . $skip ! -perm -1000 -print0 |
> +	xargs -r0J % find -d % -exec rm -rf -- {} \; -print |
> +	    head -50 >/dev/null 2>&1

This is the step which prunes 50 inodes.  It is very complicated.

The first find runs without -d, so that it can generate a list of top-level
objects except for a) +t directories, b) lost+found, quota.user, quota.group,
vi.recover.

Anything else at the top-level is a target for removing inodes.

The next clever put is that we are going to print every (single file
or directory) as we delete it, and pipe that to head -50.  head -50
will stop file deletions when we reach the limit.  This printing may
include files with newlines in it, which means we may delete less than
50.  In reality we definately only need 1, ok maybe 3, inodes

The nested find now processes the permitted files and directories, finding
one file at a time, bottom up, because of -d.  The -print is located
after the rm -rf -- {} so that the failure to remove a file (for whatever
reason does not) advance the counter towards 50 files.

Of course, the failure to remove files will be noisy, so the entire
contraption redirects errors to /dev/null.

Now that we have enough inodes, we to hunt the other files:

> +    tmpdir=$(mktemp -d .delXXXXXXXXXX) && {
> +	find . $skip ! -name "$tmpdir" -print0 |
> +	    xargs -r0J % mv -f -- % "$tmpdir"
> +	rm -rf -- "$tmpdir" &
> +    }
> +} )

We create a mktemp directory, move all the permitted files into that
directory, and then asyncronously delete that directory.  This can
take as long as it needs to, while the system boots, because those
files cannot be reached because they are inside a mktemp directory.

I think there is still reason to redirect the final rm -rf (or maybe
even the find + mv to /dev/null, becuase noone wants to see any noise
from this.