Re: Problem with unicode string length zsh segfaults

[Peter Stephenson <pws@xxxxxxx>]

On Thu, 30 Oct 2008 12:03:22 -0200
"Kazuo Teramoto" <kaz.rag@xxxxxxxxx> wrote:
> Hello!
> (My first email to list, I subscribed to workers, but I send this type
> of msg to workers or users?)

Yes, zsh-workers is correct.

> I'm having a problem with string size and unicode chars. I'm using
> string size in my prompt theme, I'm using the following code to
> calculate the string size of pwd:
> 
> ${#${(%):-%~}}
> 
> But sometimes this got the number wrong e.g. I have a dir named
> '~/Music/ææçè' (Yuki Kajiura composer of Noir anime soundtrack) the
> code above give 13 chars but it have 12 chars (count or using "print
> -Pn '%~'|wc -m" and wc -c give 20 bytes).
> 
> And if I cd on this dir my prompt get wrong by 3 chars (e.g. on a
> prompt like adams2 the first line is bigger by 3 chars)
> 
> The problem is greater for a dir named 'ÐÑÑÑ ÐÐÑÐÑ ÐÐÐÐÐÐÑÐÐÐ' (Pyotr
> Il'yich Tchaikovsky, Russian composer) if I cd on it and execute the
> code above I get
> 
> zsh: unmatched '
> zsh: parse error in command substitution
> 
> and if I try this several times (repeat this command) my zsh segfaults:
> 
> 1: subst.c:222: BUG: parse error in command substitution

Thanks for the report.

Same bug in each case; it's taken surprisingly long to find and I can't
understand how we thought the code should look like that.

I found a similar bug in new code at the same time.

Index: Src/subst.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/subst.c,v
retrieving revision 1.89
diff -u -r1.89 subst.c
--- Src/subst.c	27 Sep 2008 20:58:28 -0000	1.89
+++ Src/subst.c	30 Oct 2008 15:22:34 -0000
@@ -2738,8 +2738,8 @@
 		char *tmps;
 		unmetafy(*ap, &len);
 		untokenize(*ap);
-		tmps = unmetafy(promptexpand(metafy(*ap, len, META_NOALLOC),
-					     0, NULL, NULL, NULL), &len);
+		tmps = promptexpand(metafy(*ap, len, META_NOALLOC),
+				    0, NULL, NULL, NULL);
 		*ap = dupstring(tmps);
 		free(tmps);
 	    }
@@ -2749,8 +2749,8 @@
 		val = dupstring(val), copied = 1;
 	    unmetafy(val, &len);
 	    untokenize(val);
-	    tmps = unmetafy(promptexpand(metafy(val, len, META_NOALLOC),
-					0, NULL, NULL, NULL), &len);
+	    tmps = promptexpand(metafy(val, len, META_NOALLOC),
+					0, NULL, NULL, NULL);
 	    val = dupstring(tmps);
 	    free(tmps);
 	}
Index: Src/utils.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/utils.c,v
retrieving revision 1.205
diff -u -r1.205 utils.c
--- Src/utils.c	30 Oct 2008 14:04:31 -0000	1.205
+++ Src/utils.c	30 Oct 2008 15:22:34 -0000
@@ -857,7 +857,9 @@
     scanhashtable(nameddirtab, 0, 0, 0, finddir_scan, 0);
 
     if (func) {
-	char **ares = subst_string_by_func(func, "d", finddir_full);
+	char *dir_meta = metafy(finddir_full, strlen(finddir_full),
+				META_ALLOC);
+	char **ares = subst_string_by_func(func, "d", dir_meta);
 	int len;
 	if (ares && arrlen(ares) >= 2 &&
 	    (len = (int)zstrtol(ares[1], NULL, 10)) > finddir_best) {
@@ -868,6 +870,8 @@
 	    finddir_last->diff = len - strlen(finddir_last->node.nam);
 	    finddir_best = len;
 	}
+	if (dir_meta != finddir_full)
+	    zsfree(dir_meta);
     }
 
     return finddir_last;
Index: Test/D07multibyte.ztst
===================================================================
RCS file: /cvsroot/zsh/zsh/Test/D07multibyte.ztst,v
retrieving revision 1.27
diff -u -r1.27 D07multibyte.ztst
--- Test/D07multibyte.ztst	18 Oct 2008 16:03:18 -0000	1.27
+++ Test/D07multibyte.ztst	30 Oct 2008 15:22:34 -0000
@@ -403,3 +403,10 @@
 0:Backslashes and metafied characters in patterns
 >Matched against glob/()Ä/*
 >Matched against glob/()Ä/*
+
+  mkdir ææçè 'ÐÑÑÑ ÐÐÑÐÑ ÐÐÐÐÐÐÑÐÐÐ'
+  (cd ææçè; print ${${(%):-%~}:t})
+  (cd 'ÐÑÑÑ ÐÐÑÐÑ ÐÐÐÐÐÐÑÐÐÐ'; print ${${(%):-%~}:t})
+0:Metafied characters in prompt expansion
+>ææçè
+>ÐÑÑÑ ÐÐÑÐÑ ÐÐÐÐÐÐÑÐÐÐ


-- 
Peter Stephenson <pws@xxxxxxx>                  Software Engineer
CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
Cambridge, CB4 0WZ, UK                          Tel: +44 (0)1223 692070