Re: [PATCH] fix mulibyte input/mbstate_t problem

[Peter Stephenson <pws@xxxxxxx>]

Andrey Borzenkov wrote:
> Attached patch fixes multibyte input (verified with UTF-8). As it turns out, 
> mbstate_t works quite differently from expectation :)
> 
> The patch makes it static (with implicit initialization). It is fundamentally
> wrong to reinitialize it every time. mbstate_t is a function of all preceding
> input; for shift state encoding it will also keep current shift state among 
> other things. It also means that in the long run every input must have own 
> mbstate_t which is initialized when stream is first opened. We need one 
> mbstate_t for zle.

It's not quite so clear in our case.  We're not just inputting a stream
of characters, we've got editing characters in between.  However,
there's not a lot we can do apart from making consecutively typed
characters work properly, so what you say is the best bet.

We do probably need to reset the shift state each time zleread() is
called to get a complete line, however, so the static in getrestchar()
probably needs to be a global somewhere.

Presumably also leaving the mbrtowc in stringaszleline separate, as it
is at the moment, is also the right thing to do.

> Editing Russian is funny; "echo xxxx" outputs correct text but during line 
> editing display is wrong (it counts every UTF-8 as 2 screen characters).

If it works otherwise, e.g. the cursor moves over the right number of
characters (but not screen columns), I guess that's something in
zle_refresh.c.  We're not using wcwidth, yet, but it doesn't sound like
that's the problem since it should assume a single column per character.

> BTW calling getbyte from getsrestchar resets lastchar_wide_valid.

Aha, that's probably what Bart meant by "are we setting
lastchar_wide_valid too early", which I misinterpreted.  I'll fix it in
the patch I'm doing.

> PS am I the only one to have problems with SourceForge ssh CVS? It does not 
> hang completely but it is painfully slow.

I've been having that for a day or two, too.

-- 
Peter Stephenson <pws@xxxxxxx>                  Software Engineer
CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
Cambridge, CB4 0WZ, UK                          Tel: +44 (0)1223 692070


**********************************************************************
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
the system manager.

**********************************************************************