From: Thomas de Grivel <billitch@gmail.com>
Subject: Re: [REPOST] ksh: utf8 full width character support for emacs.c
Cc: tech@openbsd.org
Date: Tue, 25 Mar 2025 08:59:34 +0100

I did implement UTF-8 in my runtime : you need Unicode codepoints
(UTF32) which is uint32_t and you need UTF-8 because it is a little
easier on the memory bus. And a conversion function back and forth.

Please see https://git.kmx.io/kc3-lang/kc3/_tree/master/libkc3/str.c#1263
for a tested implementation.

On Sat, Mar 22, 2025 at 3:04 PM Ingo Schwarze <schwarze@usta.de> wrote:
>
> Hello Anthony,
>
> Anthony J. Bentley wrote on Fri, Mar 21, 2025 at 04:38:27PM -0600:
>
> > For what it's worth I strongly suspect that the existing double-width
> > handling in ksh causes this crash:
>
> Are you sure there is any code for handling double-width characters
> in ksh(1)?  I'm not aware of any such code.  I just briefly looked through
> the source code of vi.c and emacs.c again and the changelogs of both
> files and see no indication.
>
> For all i know, our shell simply assumes that every Unicode code point
> other than U+0000 takes up one display column.
>
> > https://marc.info/?l=openbsd-bugs&m=172993414312117&w=2
>
> From that report, i suspect the machine is a workstation used for
> production work, but not a sensitive Internet server.  In that case,
> would it be feasible for you to compile the shell with
>
>   $ make DEBUG='-g -O0'
>
> and install the resulting binary as /bin/sh and /bin/ksh ?
> I admit that might slow your shell down a bit, but unless i'm quite
> mistaken, when it then happens again, we would likely have a useful
> backtrace that can be used to track down the problem.
>
> Ideally, take a note which version of the code you are using exactly,
> for example using a CVS tag or date+time.  That way, it should even be
> possible for others to check out that exact version and analyze your
> core dump on their machine.
>
> What you describe does sound like a serious bug (use after free).
> It would be very good to get that fixed.
>
> However, without a backtrace, finding such a bug is very hard and
> might require an audit of the complete file vi.c - and even then risk
> not finding the bug because it might hide in a different file and
> merely be triggered from vi.c.
>
> Yours,
>   Ingo
>


-- 
 Thomas de Grivel
 https://www.kmx.io