From: j@bitminer.ca
Subject: Re: exact floating point calculations in roff(7)
To: Tech <tech@openbsd.org>
Cc: schwarze@usta.de
Date: Wed, 02 Apr 2025 14:52:39 -0400

Hello, I think you left out the best part:


         switch (unit) {
         case 'f':
                 myres *= 65536.0;
                 break;
     ...
     <snip>
     ...
         case 'M':
                 myres *= 24.0 / 100.0;
                 break;
         default:
                 break;
         }
         if (res != NULL)
                 *res = myres;    <--- ***here***

The whole point seems to be to calculate an integer,
returned in *res.

Some programmer who might have been less inclined to write scaled
integer arithmetic code decided, instead, to adopt doubles.

OK, fine, but I think an implied goal here is to generate an exact
*rounded* result.  Which is necessary because as often seen doubles
don't calculate integer values very reliably.

While your fix is obviously valid I think the fix could also have
been


     *res = trunc(myres + 0.5);

which eliminates the 23.999999 problem in the usual floating-point
way.


J