Re: 4.3.2/20061219 -> 4.3.2/20070126 very broken

[Peter Stephenson <p.w.stephenson@xxxxxxxxxxxx>]

Here's a further improvement... the previous patch didn't pass on the
Bnull, which I added specially so that the completion code could see it
when trying to complete words inside $'...', and I missed the admittedly
rather special case of an escaped backslash that had \M- or \C- before
it.


Index: Src/utils.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/utils.c,v
retrieving revision 1.151
diff -u -r1.151 utils.c
--- Src/utils.c	27 Jan 2007 19:01:11 -0000	1.151
+++ Src/utils.c	27 Jan 2007 23:45:36 -0000
@@ -4915,27 +4915,45 @@
 	    *t++ = *++s ^ 32;
 	else {
 	    if (itok(*s)) {
+		/*
+		 * We need to be quite careful here.  We haven't
+		 * necessarily got an input stream with all tokens
+		 * removed, so the majority of tokens need passing
+		 * through untouched and without Meta handling.
+		 * However, me may need to handle tokenized
+		 * backslashes.
+		 */
 		if (meta || control) {
 		    /*
 		     * Presumably we should be using meta or control
 		     * on the character representing the token.
+		     *
+		     * Special case: $'\M-\\' where the token is a Bnull.
+		     * This time we dump the Bnull since we're
+		     * replacing the whole thing.  The lexer
+		     * doesn't know about the meta or control modifiers.
 		     */
-		    *t++ = ztokens[*s - Pound];
+		    if ((how & GETKEY_DOLLAR_QUOTE) && *s == Bnull)
+			*t++ = *++s;
+		    else
+			*t++ = ztokens[*s - Pound];
 		} else if (how & GETKEY_DOLLAR_QUOTE) {
+		    /*
+		     * We don't want to metafy this, it's a real
+		     * token.
+		     */
+		    *tdest++ = *s;
 		    if (*s == Bnull) {
 			/*
 			 * Bnull is a backslash which quotes a couple
 			 * of special characters that always appear
 			 * literally next.  See strquote handling
-			 * in gettokstr() in lex.c.
+			 * in gettokstr() in lex.c.  We need
+			 * to retain the Bnull (as above) so that quote
+			 * handling in completion can tell where the
+			 * backslash was.
 			 */
 			*tdest++ = *++s;
-		    } else {
-			/*
-			 * We don't want to metafy this, it's a real
-			 * token.
-			 */
-			*tdest++ = *s;
 		    }
 		    continue;
 		} else
Index: Test/A03quoting.ztst
===================================================================
RCS file: /cvsroot/zsh/zsh/Test/A03quoting.ztst,v
retrieving revision 1.2
diff -u -r1.2 A03quoting.ztst
--- Test/A03quoting.ztst	27 Jan 2007 19:01:11 -0000	1.2
+++ Test/A03quoting.ztst	27 Jan 2007 23:45:36 -0000
@@ -17,6 +17,20 @@
 0:$'-style quotes with backslashed backslashes
 >'a \' is 'a backslash' is 'a \'
 
+  chars=$(print -r $'BS\\MBS\M-\\')
+  for (( i = 1; i <= $#chars; i++ )); do
+    char=$chars[$i]
+    print $(( [#16] #char ))
+  done
+0:$'-style quote with metafied backslash
+>16#42
+>16#53
+>16#5C
+>16#4D
+>16#42
+>16#53
+>16#DC
+
   print -r ''''
   setopt rcquotes
 # We need to set rcquotes here for the next example since it is

-- 
Peter Stephenson <p.w.stephenson@xxxxxxxxxxxx>
Web page now at http://homepage.ntlworld.com/p.w.stephenson/