From: Philip Guenther <guenther@gmail.com>
Subject: Re: amd64/fpu: Avoid multiple FPU resets
To: Theo de Raadt <deraadt@openbsd.org>
Cc: Christian Ludwig <cludwig@genua.de>, tech@openbsd.org
Date: Fri, 13 Jun 2025 13:30:50 -0700

On Fri, Jun 13, 2025 at 12:08 PM Theo de Raadt <deraadt@openbsd.org> wrote:
>
> Philip Guenther <guenther@gmail.com> wrote:
>
> > On Fri, Jun 13, 2025 at 5:32 AM Christian Ludwig <cludwig@genua.de> wrote:
> > > All kernel crypto code follows the scheme:
> > >
> > >         for (objects) {
> > >                 fpu_kernel_enter();
> > >                 ...
> > >                 fpu_kernel_exit();
> > >         }
> > >
> > > In every iteration, fpu_kernel_exit() resets the FPU state and
> > > fpu_kernel_enter() resets it, again. FPU resets are expensive on some
> > > platforms. Doing the operation twice per loop iteration is clearly not
> > > necessary.
> > >
> > > The FPU is always in one of two states when we reach fpu_kernel_enter().
> > > It either holds user state when CPUPF_USERXSTATE is set or it is in
> > > reset state already. The context switching code and signal code follow
> > > this assumption, too. So we can simply drop resetting the FPU in
> > > fpu_kernel_enter() when it does not hold user state.
> >
> > Hmm, yes, all the places that clear CPUF_USERXSTATE reset the state.
> >
> > Does fpu_kernel_enter() get used from interrupt context?  Do we have
> > to worry about an interrupt occurring between the clearing of the flag
> > and the resetting of the state?
>
> Is that a remaining artifact of the old lazy code?

No, it's real: we got rid of lazy _loading_ of FPU state--on return to
userspace the thread's FPU state is loaded and ready to go--but we
still do lazy _saving_ of FPU state, saving it on context switch and
when the kernel wants to use the FPU so it doesn't need to be
saved+restored if the thread returns to userspace without a context
switch.  The USERXSTATE flag is the "nope, haven't saved it yet" flag


Philip