Download raw body.
exact floating point calculations in roff(7)
Hello, I think you left out the best part:
switch (unit) {
case 'f':
myres *= 65536.0;
break;
...
<snip>
...
case 'M':
myres *= 24.0 / 100.0;
break;
default:
break;
}
if (res != NULL)
*res = myres; <--- ***here***
The whole point seems to be to calculate an integer,
returned in *res.
Some programmer who might have been less inclined to write scaled
integer arithmetic code decided, instead, to adopt doubles.
OK, fine, but I think an implied goal here is to generate an exact
*rounded* result. Which is necessary because as often seen doubles
don't calculate integer values very reliably.
While your fix is obviously valid I think the fix could also have
been
*res = trunc(myres + 0.5);
which eliminates the 23.999999 problem in the usual floating-point
way.
J
exact floating point calculations in roff(7)