PATCH: allow baseline and custom cursor forms to be set

[Oliver Kiddle <opk@xxxxxxx>]

This revisits the support for changing the cursor from zle. Mikael had
remarked about how temporary use of vi mode would leave the cursor
blinking. The source of this problem is two fundamental problems with
the well-supported DECSCUSR sequence. Firstly you can't change the shape
independently of whether the cursor blinks. And secondly, while many
terminals reset the cursor to the default status when passing 0 to the
sequence, there are also many that treat it the same as 1, i.e. blinking
block. That is in line with the earliest documentation for the sequence
but that was for a physical hardware terminal where the default was
probably hard-wired. The approach I had taken was to use 0 and hope for
the best.

My initial thought was to add support for an additional default: or
reset: key to zle_cursorform and use that when zle ends. But when
testing this, I was unhappy with the result. For one thing it extended
the problems to emacs mode because if you set it to "steady" that change
would be forced even if the cursor was otherwise unchanged.

Given that the aim is to avoid unwanted cursor changes, I concluded
that it is better to have a key to set a baseline form for the cursor.
The baseline is used as the starting point for all cursor contexts in
addition to the reset to default context. But values coming from the
baseline are only applied when undoing a previous cursor change. So
despite it being set by default to "steady", on a terminal where the
cursor blinks by default it should remain blinking for unconfigured
emacs mode because it never gets corrupted by setting a shape.

The default is now:

  zle_cursorform=( insert:bar pending:underline base:steady )

It was already the case that insert:bar would force a steady cursor so
this is now explicit. base:none is largely a return to how things have
worked so far.

In general, you should use base: to indicate what your terminal's
default cursor is where it can't otherwise reset it properly.

I had previously avoided OSC 112 or resetting cursor colour having
concluded that the sequence was problematic. I must have hit what
appears to be an rxvt-unicode specific bug. Adding an extra (harmless)
semi-colon to the sequence works around that bug. The code now uses two
approaches to resetting the cursor for maximum compatibility with
different terminals even if this will be superfluous for many.

Furthermore, I discovered that $termcap[vs] for many terminals contains
two sequences, setting also the blink status of the cursor. So this
allows for that having been corrupted.

Support is also added for zle_cursorform=(none) mirroring zle_highlight.
This should be simpler than .term.extensions=( -cursor ) and won't cause
errors if used with an older zsh (one of Mikael's earlier comments).
I've added a note under incompatibilities in the README just under
the existing note that is relevant to vi mode users (only vi mode is
affected by the defaults).

The vi f F t and T commands will now also use the pending context when
waiting for the character. Unlike c, d etc they don't wait for a
movement command but it is similar enough.

The patch also adds support for a CURSOR_FORM variable for use from zle
widgets to specify a custom cursor context. This needs special handling
to allow it to be modified from a completion widget. Viewed from the
perspective of the code internals, it should be named to indicicate that
it specifies a "context" but the contexts are just an extra layer of
abstraction and for most users, the closer association with
zle_cursorform may be simpler. An addition to _main_complete sets it to
"completion" for the duration of completion which supports one of the
use cases Mikael had mentioned - and I quite like the feature now that
I've tried it. Mikael also mentioned changing the cursor for various
commands but this doesn't fit in well with what is a ZLE feature and
trying to integrate properly with job control would be too difficult to
contemplate.

The patch also adds test cases.

Oliver

diff --git a/Completion/Base/Core/_main_complete b/Completion/Base/Core/_main_complete
index b0c2db5b7..74e6f736a 100644
--- a/Completion/Base/Core/_main_complete
+++ b/Completion/Base/Core/_main_complete
@@ -54,6 +54,7 @@ local -ar builtin_precommands=(- builtin eval exec nocorrect noglob time)
 typeset -U _lastdescr _comp_ignore _comp_colors
 
 {
+CURSOR_FORM=completion
 
 [[ -z "$curcontext" ]] && curcontext=:::
 
@@ -382,6 +383,8 @@ fi
     compstate[list]="${compstate[list]//messages} force"
 
 } always {
+  CURSOR_FORM=
+
   # Stuff we always do to clean up.
   if [[ "$compstate[old_list]" = keep ]]; then
     if [[ $_saved_colors_set = 1 ]]; then
diff --git a/Doc/Zsh/zle.yo b/Doc/Zsh/zle.yo
index 761bd5bc4..e28986bc7 100644
--- a/Doc/Zsh/zle.yo
+++ b/Doc/Zsh/zle.yo
@@ -822,6 +822,13 @@ The offset of the cursor, within the edit buffer.  This is in the range
 Attempts to move the cursor outside the buffer will result in the
 cursor being moved to the appropriate end of the buffer.
 )
+vindex(CURSOR_FORM)
+item(tt(CURSOR_FORM) (scalar))(
+The name of a custom context to allow the shape and color of the cursor to be
+changed. The context is looked up in tt(zle_cursorform) and applies for as long
+as the line editor is active or until the variable is changed or unset from a ZLE
+Widget.
+)
 vindex(CUTBUFFER)
 item(tt(CUTBUFFER) (scalar))(
 The last item cut using one of the `tt(kill-)' commands; the string
@@ -2987,14 +2994,16 @@ vindex(zle_cursorform, setting)
 Some terminals support the ability to change the shape and color of the cursor.
 On such terminals, the line editor will use cursor styles appropriate to
 different contexts. This is controlled via the array parameter
-tt(zle_cursorform). To disable all cursor changes, see the tt(.term.extensions)
-parameter.
+tt(zle_cursorform).
 
 Each element of the array should consist of a word indicating a context
 followed by a colon, then a comma-separated list of properties describing the
-shape and color to apply to the cursor.
+shape and color to apply to the cursor. All cursor changes may be disabled
+with the following setting:
 
-The available contexts follow with the default value shown in
+example(zle_cursorform=(none))
+
+The available predefined contexts follow with the default value shown in
 parentheses. Where no default is given, the terminal's default is applied:
 
 startitem()
@@ -3033,8 +3042,23 @@ item(tt(visual))(
 Used when vi visual mode is active. The visual selection always includes the
 cursor position so the same advice as for tt(regionstart) applies.
 )
+item(tt(base) (tt(steady)))(
+Used as a baseline for all other contexts and when resetting the cursor after
+the line editor ceases to be active. Only those cursor attributes that have
+been otherwise altered will be set according to a value given for this context.
+
+This exists because many terminals don't provide a way to reset the cursor to
+its default state nor to set the shape independently of the blinking status. It
+may be useful if your cursor is not correctly restored when the line editor
+becomes inactive. It is best that the value reflects the default state of the
+terminal's cursor but can be set to tt(none) where the terminal does support
+resetting the cursor.
+)
 enditem()
 
+User-defined ZLE widgets can implement additional custom contexts by setting
+the tt(CURSOR_FORM) parameter.
+
 The available cursor shapes are tt(none), tt(bar), tt(block), tt(underline) and
 tt(hidden). Additionally, you can specify either tt(blink) or tt(steady) to
 indicate whether the cursor should flash and specify a color as an RGB triplet
diff --git a/README b/README
index f03fd78e2..8becbaf21 100644
--- a/README
+++ b/README
@@ -52,6 +52,11 @@ or, if your $VISUAL and $EDITOR environment variables vary,
 to your .zshrc file.  These snippets are compatible with previous
 versions of the shell.
 
+If you use "vi" mode, the line editor's new support for changing the shape
+of the terminal cursor is enabled by default. To disable this use:
+
+    zle_cursorform=(none)
+
 Restricted mode has been removed.  This was associated with the option
 RESTRICTED (-r).  This was an outdated way to restrict what users may do and
 was very difficult to apply safely.  Furthermore, modern systems have better,
diff --git a/Src/Zle/termquery.c b/Src/Zle/termquery.c
index 4ea084be7..f73072a7c 100644
--- a/Src/Zle/termquery.c
+++ b/Src/Zle/termquery.c
@@ -455,7 +455,8 @@ handle_color(int bg, int red, int green, int blue)
 		    "dark" : "light"), 0);
 	    break;
         case 2:  /* cursor color */
-	    memo_cursor = (red << 24) | (green << 16) | (blue << 8);
+	    memo_cursor = (red << CURF_RED_SHIFT) | (green << CURF_GREEN_SHIFT) |
+		    (blue << CURF_BLUE_SHIFT) | CURF_COLOR;
 	    break;
 	default: /* invalid color */
 	    return;
@@ -811,6 +812,9 @@ notify_pwd(void)
     write_loop(SHTTY, "\033\\", 2);
 }
 
+/**/
+char *custom_cursor_form;
+
 static unsigned int *cursor_forms;
 static unsigned int cursor_enabled_mask;
 
@@ -886,21 +890,36 @@ zle_set_cursorform(void)
 	"pending:",
 	"regionstart:",
 	"regionend:",
-        "visual:"
+	"visual:",
+	"base:"
     };
 
     if (!cursor_forms)
-	cursor_forms = zalloc(CURC_DEFAULT * sizeof(*cursor_forms));
-    memset(cursor_forms, 0, CURC_DEFAULT * sizeof(*cursor_forms));
+	cursor_forms = zalloc((CURC_DEFAULT + 2) * sizeof(*cursor_forms));
+    memset(cursor_forms, 0, (CURC_DEFAULT + 2) * sizeof(*cursor_forms));
     cursor_forms[CURC_INSERT] = CURF_BAR;
     cursor_forms[CURC_PENDING] = CURF_UNDERLINE;
+    cursor_forms[CURC_DEFAULT] = CURF_STEADY;
+    cursor_forms[CURC_CUSTOM] = CURF_INVALID;
 
     for (; atrs && *atrs; atrs++) {
+	if (!strcmp(*atrs, "none")) {
+	    memset(cursor_forms, 0, (CURC_DEFAULT + 2) * sizeof(*cursor_forms));
+	    continue;
+	}
 	if (strpfx("region:", *atrs)) {
 	    match_cursorform(*atrs + 7, &cursor_forms[CURC_REGION_END]);
 	    cursor_forms[CURC_REGION_START] = cursor_forms[CURC_REGION_END];
 	    continue;
 	}
+	if (*custom_cursor_form && strpfx(custom_cursor_form, *atrs)) {
+	    int len = strlen(custom_cursor_form);
+	    if ((*atrs)[len] == ':') {
+		match_cursorform(*atrs + len + 1, &cursor_forms[CURC_CUSTOM]);
+		continue;
+	    }
+	}
+
 	for (i = 0; i < sizeof(contexts) / sizeof(*contexts); i++) {
 	    if (strpfx(contexts[i], *atrs)) {
 		match_cursorform(*atrs + strlen(contexts[i]), &cursor_forms[i]);
@@ -912,7 +931,7 @@ zle_set_cursorform(void)
     if (trashedzle)
 	setup = 0;
     else if (!setup) {
-	cursor_enabled_mask = 0;
+	cursor_enabled_mask = CURF_HIDDEN;
 	setup = 1;
 	if (extension_enabled("cursor", "shape", 6, 1))
 	    cursor_enabled_mask |= CURF_SHAPE_MASK | CURF_BLINK | CURF_STEADY;
@@ -926,65 +945,140 @@ void
 free_cursor_forms(void)
 {
     if (cursor_forms)
-	zfree(cursor_forms, CURC_DEFAULT * sizeof(*cursor_forms));
+	zfree(cursor_forms, (CURC_DEFAULT + 1) * sizeof(*cursor_forms));
     cursor_forms = 0;
 }
 
+/* Need to write TCCURINV/TCCURVIS termcap sequencies to SHTTY */
+static int
+putshtty(int c)
+{
+    const char x = c;
+    write_loop(SHTTY, &x,1);
+    return 0;
+}
+
 /**/
 void
 cursor_form(void)
 {
-    char seq[32];
+    /* representation of maximum length sequence to size array */
+    char seq[sizeof("E[0 qE[?12xE]12;rgb:xx00/xx00/xx00STE]112;ST")];
     char *s = seq;
-    unsigned int want, changed;
-    static unsigned int state = CURF_DEFAULT;
     enum cursorcontext context = CURC_DEFAULT;
+    unsigned int
+	reset = CURF_INVALID, /* bits being reset to the defaults */
+	want = 0, /* the desired new cursor form */
+	changed; /* bit mask for changes */
+    int resetcolor = 0; /* really reset, not just applying base:color */
+    static unsigned int
+	altered = 0, /* bits that have changed from the defaults */
+	state = 0; /* current cursor form */
+    unsigned int exclusive[] = {
+	CURF_HIDDEN,
+	CURF_SHAPE_MASK,
+	CURF_BLINK | CURF_STEADY,
+	CURF_COLOR_MASK
+    };
+    int ex;
 
     if (!cursor_forms)
 	return;
 
     if (trashedzle) {
 	;
+    } else if (cursor_forms[CURC_CUSTOM] != CURF_INVALID) {
+	context = CURC_CUSTOM;
     } else if (!insmode) {
 	context = CURC_OVERWRITE;
     } else if (vichgflag == 2) {
 	context = CURC_PENDING;
     } else if (region_active) {
-        if (invicmdmode()) {
+	if (invicmdmode()) {
 	    context = CURC_VISUAL;
 	} else {
 	    context = mark > zlecs ? CURC_REGION_START : CURC_REGION_END;
 	}
     } else
-	context = invicmdmode() ? CURC_COMMAND : (vichgflag ? CURC_INSERT : CURC_EDIT);
-    want = (context == CURC_DEFAULT) ? CURF_DEFAULT : cursor_forms[context];
-    if (!(changed = (want ^ state) & cursor_enabled_mask))
+	context = invicmdmode() ? CURC_COMMAND :
+		(vichgflag ? CURC_INSERT : CURC_EDIT);
+
+    if (context != CURC_DEFAULT)
+	want = cursor_forms[context];
+    for (ex = 0; ex < sizeof(exclusive) / sizeof(*exclusive); ex++) {
+	if (!(want & exclusive[ex])) {
+	    want |= cursor_forms[CURC_DEFAULT] & exclusive[ex];
+	    reset |= exclusive[ex];
+	}
+    }
+    if (!(want & CURF_COLOR) && (reset & CURF_COLOR)) {
+	want |= memo_cursor;
+	resetcolor = 1;
+    }
+
+    /* get which bits have changed minus disabled changes and
+     * no need to reset bits that are unaltered from initial defaults */
+    if (!(changed = (want ^ state) & cursor_enabled_mask & ~(reset & ~altered)))
 	return;
 
-    if (changed & CURF_HIDDEN)
-	 tcout(want & CURF_HIDDEN ? TCCURINV : TCCURVIS);
+    /* setting shape or visible also affects flashing status */
+    if ((changed & ((state & CURF_HIDDEN) | CURF_SHAPE_MASK))) {
+	changed |= CURF_BLINK | CURF_STEADY;
+	if (!(want & (CURF_BLINK | CURF_STEADY))) {
+	    want |= cursor_forms[CURC_DEFAULT] & (CURF_BLINK | CURF_STEADY);
+	    reset |= CURF_BLINK | CURF_STEADY;
+	}
+    }
+
+    if (changed & CURF_HIDDEN) {
+	/* A terminal might not support hidden so it is intentional
+	 * that we still set shape/color when hiding the cursor
+	 * even though that change may be invisible. */
+	int cap = want & CURF_HIDDEN ? TCCURINV : TCCURVIS;
+	if (tcout_func_name) {
+	    tcout_via_func(cap, -1, putshtty);
+	} else {
+	    tputs(tcstr[cap], 1, putshtty);
+	}
+	state &= ~CURF_HIDDEN;
+	state |= want & CURF_HIDDEN;
+    }
     if (changed & CURF_SHAPE_MASK) {
 	char c = '0';
 	switch (want & CURF_SHAPE_MASK) {
-	    case CURF_BAR: c += 2;
-	    case CURF_UNDERLINE: c += 2;
+	    case CURF_BAR: c += 2; /* fall-through */
+	    case CURF_UNDERLINE: c += 2; /* fall-through */
 	    case CURF_BLOCK:
 		c += 2 - !!(want & CURF_BLINK);
-		changed &= ~(CURF_BLINK | CURF_STEADY);
+		changed &= ~(CURF_BLINK | CURF_STEADY); /* mark flash as done */
+		break;
 	}
 	s += sprintf(s, "\033[%c q", c);
+	state &= ~(CURF_SHAPE_MASK | CURF_BLINK | CURF_STEADY);
+	state |= want & (CURF_SHAPE_MASK | CURF_BLINK | CURF_STEADY);
     }
-    if (changed & (CURF_BLINK | CURF_STEADY)) {
+    if (changed & want & (CURF_BLINK | CURF_STEADY)) {
 	s += sprintf(s, "\033[?12%c", (want & CURF_BLINK) ? 'h' : 'l');
+	state &= ~(CURF_BLINK | CURF_STEADY);
+	state |= want & (CURF_BLINK | CURF_STEADY);
     }
     if (changed & CURF_COLOR_MASK) {
-	if (!(want & CURF_COLOR_MASK))
-	    want = memo_cursor | (want & 0xff);
-	s += sprintf(s, "\033]12;rgb:%02x00/%02x00/%02x00\033\\",
-		want >> CURF_RED_SHIFT, (want >> CURF_GREEN_SHIFT) & 0xff,
-		(want >> CURF_BLUE_SHIFT) & 0xff);
+	if (want & CURF_COLOR)
+	    s += sprintf(s, "\033]12;rgb:%02x00/%02x00/%02x00\033\\",
+		    want >> CURF_RED_SHIFT, (want >> CURF_GREEN_SHIFT) & 0xff,
+		    (want >> CURF_BLUE_SHIFT) & 0xff);
+
+	state &= ~CURF_COLOR_MASK;
+	/* Semi-colon included in this sequence because urxvt fails to consume
+	 * the final backslash without it. Otherwise it should be harmless. */
+	if (resetcolor)
+	    s += sprintf(s, "\033]112;\033\\");
+	else
+	    state |= want & CURF_COLOR_MASK;
     }
     if (s - seq)
 	write_loop(SHTTY, seq, s - seq);
-    state = want;
+
+    /* for the next call reset gives us the unaltered bits */
+    altered = ~reset;
 }
diff --git a/Src/Zle/zle.h b/Src/Zle/zle.h
index bf479d4cc..75d6f33ee 100644
--- a/Src/Zle/zle.h
+++ b/Src/Zle/zle.h
@@ -482,22 +482,23 @@ enum cursorcontext {
     CURC_REGION_START,
     CURC_REGION_END,
     CURC_VISUAL,
-    CURC_DEFAULT
+    CURC_DEFAULT,
+    CURC_CUSTOM
 };
 
-#define CURF_DEFAULT 0
 #define CURF_UNDERLINE 1
 #define CURF_BAR 2
 #define CURF_BLOCK 3
 #define CURF_SHAPE_MASK 3
 #define CURF_BLINK (1 << 2)
 #define CURF_STEADY (1 << 3)
-#define CURF_HIDDEN (1 << 4)
+#define CURF_HIDDEN (1 << 4) /* no flag for visible: assuming it's default */
 #define CURF_COLOR (1 << 5)
 #define CURF_COLOR_MASK ((0xffffffu << 8) | CURF_COLOR)
 #define CURF_RED_SHIFT   24
 #define CURF_GREEN_SHIFT 16
 #define CURF_BLUE_SHIFT  8
+#define CURF_INVALID (3 << 6)
 
 #ifdef MULTIBYTE_SUPPORT
 /*
diff --git a/Src/Zle/zle_main.c b/Src/Zle/zle_main.c
index 921ebb9ec..772d16947 100644
--- a/Src/Zle/zle_main.c
+++ b/Src/Zle/zle_main.c
@@ -2077,6 +2077,7 @@ trashzle(void)
 	int sl = showinglist;
 	showinglist = 0;
 	trashedzle = 1;
+	custom_cursor_form = ztrdup("");
 	zrefresh();
 	showinglist = sl;
 	treplaceattrs(prompt_attr);
@@ -2266,6 +2267,7 @@ setup_(UNUSED(Module m))
     kungetbuf = (char *) zalloc(kungetsz = 32);
     comprecursive = 0;
     rdstrs = NULL;
+    custom_cursor_form = ztrdup("");
 
     /* detect terminal color and features */
     if (shout)
@@ -2342,6 +2344,7 @@ finish_(UNUSED(Module m))
     cleanup_keymaps();
     deletehashtable(thingytab);
 
+    zsfree(custom_cursor_form);
     zfree(lastvichg.buf, lastvichg.bufsz);
     zfree(kungetbuf, kungetsz);
     free_isrch_spots();
diff --git a/Src/Zle/zle_params.c b/Src/Zle/zle_params.c
index b828e145c..bf1e90470 100644
--- a/Src/Zle/zle_params.c
+++ b/Src/Zle/zle_params.c
@@ -48,6 +48,8 @@ static const struct gsu_scalar buffer_gsu =
 { get_buffer, set_buffer, zleunsetfn };
 static const struct gsu_scalar context_gsu =
 { get_context, nullstrsetfn, zleunsetfn };
+static const struct gsu_scalar cursorform_gsu =
+{ strvargetfn, set_cursorform, zleunsetfn };
 static const struct gsu_scalar cutbuffer_gsu =
 { get_cutbuffer, set_cutbuffer, unset_cutbuffer };
 static const struct gsu_scalar keymap_gsu =
@@ -146,6 +148,7 @@ static struct zleparam {
 	NULL },
     { "CURSOR",  PM_INTEGER, GSU(cursor_gsu),
 	NULL },
+    { "CURSOR_FORM", PM_SCALAR, GSU(cursorform_gsu), (void *) &custom_cursor_form },
     { "CUTBUFFER", PM_SCALAR, GSU(cutbuffer_gsu), NULL },
     { "HISTNO", PM_INTEGER, GSU(histno_gsu), NULL },
     { "KEYMAP", PM_SCALAR | PM_READONLY, GSU(keymap_gsu), NULL },
@@ -198,7 +201,7 @@ makezleparams(int ro)
 
     for(zp = zleparams; zp->name; zp++) {
 	Param pm = createparam(zp->name, (zp->type |PM_SPECIAL|PM_REMOVABLE|
-					  PM_LOCAL|(ro ? PM_READONLY : 0)));
+		PM_LOCAL|((ro && zp->data != &custom_cursor_form) ? PM_READONLY : 0)));
 	if (!pm)
 	    pm = (Param) realparamtab->getnode2(realparamtab, zp->name);
 	DPUTS1(!pm, "param %s not set in makezleparams", zp->name);
@@ -617,6 +620,15 @@ get_suffixactive(UNUSED(Param pm))
     return suffixlen;
 }
 
+/**/
+static void
+set_cursorform(UNUSED(Param pm), char *x)
+{
+    strvarsetfn(pm, x);
+    zle_set_cursorform();
+    cursor_form();
+}
+
 /**/
 static char *
 get_cutbuffer(UNUSED(Param pm))
diff --git a/Src/Zle/zle_refresh.c b/Src/Zle/zle_refresh.c
index b3a648056..d58645a8b 100644
--- a/Src/Zle/zle_refresh.c
+++ b/Src/Zle/zle_refresh.c
@@ -2345,7 +2345,7 @@ tc_downcurs(int ct)
  */
 
 /**/
-static void
+void
 tcout_via_func(int cap, int arg, int (*outc)(int))
 {
     Shfunc tcout_func;
diff --git a/Src/Zle/zle_vi.c b/Src/Zle/zle_vi.c
index 783b49fb2..052681661 100644
--- a/Src/Zle/zle_vi.c
+++ b/Src/Zle/zle_vi.c
@@ -130,6 +130,11 @@ vigetkey(void)
     Keymap mn = openkeymap("main");
     char m[3], *str;
     Thingy cmd;
+    int nochg = vichgflag;
+
+    vichgflag = 2;
+    cursor_form();
+    vichgflag = nochg;
 
     if (getbyte(0L, NULL, 1) == EOF)
 	return ZLEEOF;
diff --git a/Test/X07termcontrol.ztst b/Test/X07termcontrol.ztst
new file mode 100644
index 000000000..018be63dc
--- /dev/null
+++ b/Test/X07termcontrol.ztst
@@ -0,0 +1,289 @@
+# Tests for terminal control sequences emitted by zsh
+%prep
+  if ( zmodload zsh/zpty 2>/dev/null ); then
+    . $ZTST_srcdir/comptest
+    comptestinit -v -z $ZTST_testdir/../Src/zsh &&
+    {
+      # Not testing termcap based sequences here so the zle transformation
+      # elides those while zle is active. Except for vi/ve which it transforms
+      # to a canonical form for predictable test results.
+      # The unsetopt minimises extra cruft which is not required but does make it
+      # easier when debugging. Similarly for the use of TERM=xterm for which
+      # many sequences are shorter while vt100 sequences include time
+      # delays/NUL padding to accomodate requirements of hardware terminals.
+      comptesteval \
+        'unsetopt promptsp promptcr' \
+        'TERM=xterm' \
+        "tcfunc() {
+          case \$1 in
+            vi) REPLY=$'\e[?25l' ;;
+            ve) REPLY=$'\e[?25h' ;;
+            *) REPLY='';;
+          esac
+        }" \
+        'reset-cursor() {
+          local -a zle_cursorform=( all:hidden,blink,bar,color=\#ffffff )
+          local CURSOR_FORM=all
+          zle_cursorform=( base:none reset:none )
+          CURSOR_FORM=reset
+        }' \
+        'zle -N reset-cursor' \
+        'bindkey "^R" reset-cursor' \
+        'zle -T tc tcfunc' \
+        'POSTEDIT="<END>"'
+    }
+  else
+    ZTST_unimplemented="the zsh/zpty module is not available"
+  fi
+
+%test
+# Note that this uses a single ZLE instance so leftover state from one test
+# affects the next. The reset-cursor widget reverts the cursor where needed.
+
+# In some cases, vi command mode seems to be skipped because the escape and
+# following command come faster than KEYTIMEOUT. Intervening h or l commands
+# are used where this is unwanted
+
+# Cursor shape and colour
+
+  zpty_run '.term.extensions=( -bracketed-paste -integration )'
+  esctest
+0:Default vi-mode cursor and reset
+>CSI 6 q
+>---
+>CSI 0 q
+>CSI ?12l
+
+  zpty_run 'zle_cursorform=( base:blink )'
+  esctest 'text'
+0:With base set to blink, use blink for insert and reset
+>CSI 5 q
+>---
+>CSI 0 q
+>CSI ?12h
+
+  zpty_run 'zle_cursorform=( base:none )'
+  esctest
+0:With base set to none, trust 0q sequence to reset flashing to default
+# Still uses steady sequence when selecting a bar shape
+>CSI 6 q
+>---
+>CSI 0 q
+
+  zpty_run 'zle_cursorform=( none )'
+  esctest $'word\ebce'
+0:With none, make no cursor changes
+>---
+
+  zpty_run 'zle_cursorform=( insert:blink )'
+  esctest
+0:Set flashing cursor with default shape, revert to steady default
+>CSI ?12h
+>---
+>CSI ?12l
+
+  zpty_run 'zle_cursorform=( insert:blink base:none )'
+  esctest
+0:Set flashing cursor with default shape, no default to revert to
+>CSI ?12h
+>---
+# a final CSI 0 q would be reasonable too
+
+  zpty_run 'zle_cursorform=( insert:underline command:blink )'
+  esctest $'\el'
+0:Switch from defined shape to flashing cursor with default shape
+>CSI 4 q
+>CSI 0 q
+>CSI ?12h
+>---
+>CSI ?12l
+
+  zpty_run 'zle_cursorform=( insert:underline command:none base:block )'
+  esctest $'\el'
+0:Switch to default shape
+>CSI 4 q
+>CSI 2 q
+>---
+
+  zpty_run 'zle_cursorform=( overwrite:underline )'
+  esctest $'\elR'
+0:Switch to overwrite mode via command mode
+>CSI 6 q
+>CSI 0 q
+>CSI ?12l
+>CSI 4 q
+>---
+>CSI 0 q
+>CSI ?12l
+
+  zpty_run 'zle_cursorform=( insert:blink overwrite:hidden base:steady )'
+  esctest $'\eR'
+0:Reset blink when new form doesn't affect use of it
+>CSI ?12h
+>CSI ?25l
+>CSI ?12l
+>---
+>CSI ?25h
+>CSI ?12l
+
+  zpty_run 'zle_cursorform=( insert:none )'
+  esctest
+0:Nothing changed, no reset
+>---
+
+  zpty_run 'zle_cursorform=( insert:color=#000055 base:steady,color=#803030 )'
+  esctest
+0:Colour reset when changed
+>OSC 12;rgb:0000/0000/5500 ST
+>---
+>OSC 12;rgb:8000/3000/3000 ST
+
+  zpty_run 'unset zle_cursorform'
+  esctest $'word\eFw'
+0:Use pending context for vi-find- widget
+>CSI 6 q
+>CSI 4 q
+>CSI 0 q
+>CSI ?12l
+>---
+
+  zpty_run 'zle_cursorform=( insert:color=#000055 overwrite:underline )'
+  esctest $'\eR'
+0:Colour should be reset when new form doesn't affect colour
+>OSC 12;rgb:0000/0000/5500 ST
+>CSI 4 q
+>OSC 112; ST
+>---
+>CSI 0 q
+>CSI ?12l
+
+  zpty_run 'zle_cursorform=( insert:color=#000055 overwrite:color=#000055,underline )'
+  esctest $'\eR'
+0:Change of context with unchanged colour
+>OSC 12;rgb:0000/0000/5500 ST
+>CSI 4 q
+>---
+>CSI 0 q
+>CSI ?12l
+>OSC 112; ST
+
+  zpty_run 'zle_cursorform=( command:block base:color=#803030 )'
+  esctest $'\el'
+0:Don't use base color when it started at the reset state
+>CSI 6 q
+>CSI 2 q
+>---
+>CSI 0 q
+
+  zpty_run 'zle_cursorform=( insert:none base:block )'
+  esctest $'\el'
+0:Don't use base shape when it started at the reset state
+>---
+
+  zpty_run 'zle_cursorform=( insert:bar,color=#332211 base:color=#112233 )'
+  esctest
+0:Use base color instead of doing a reset
+>CSI 6 q
+>OSC 12;rgb:3300/2200/1100 ST
+>---
+>CSI 0 q
+>OSC 12;rgb:1100/2200/3300 ST
+
+  zpty_run 'zle_cursorform=( insert:color=#000055 overwrite:underline base:color=#803030 )'
+  esctest $'\eR'
+0:Use base color for other forms
+>OSC 12;rgb:0000/0000/5500 ST
+>CSI 4 q
+>OSC 12;rgb:8000/3000/3000 ST
+>---
+>CSI 0 q
+
+  zpty_run 'zle_cursorform=( insert:hidden,underline )'
+  esctest
+0:Combining shape and hidden still selects the shape
+# May seem pointless because if hidden you won't see the shape but do as the
+# user says. Could be a fallback if their terminal doesn't support hidden.
+>CSI ?25l
+>CSI 4 q
+>---
+>CSI ?25h
+>CSI 0 q
+>CSI ?12l
+
+  zpty_run 'zle_cursorform=( insert:hidden,blink command:blink )'
+  esctest $'\el'
+0:Compensate for disabling hidden also turning off blink in many terminals
+# Note that $termcap[vs] often contains \e[?12l\e[?25h
+>CSI ?25l
+>CSI ?12h
+>CSI ?25h
+>CSI ?12h
+>---
+>CSI ?12l
+
+  zpty_run 'zle_cursorform=( base:none )'
+  esctest $'_\ehcltest\eFt'
+0:Pending context used for change and next-character movements
+>CSI 6 q
+>CSI 0 q
+>CSI 4 q
+>CSI 6 q
+>CSI 4 q
+>CSI 0 q
+>---
+
+  zpty_run 'custom() { () { local CURSOR_FORM=custom } }; zle -N custom'
+  zpty_run 'bindkey -a gc custom'
+  zpty_run 'zle_cursorform=( custom:underline command:block )'
+  esctest $'\ehgc'
+0:Setting custom cursor context
+>CSI 6 q
+>CSI 2 q
+>CSI 4 q
+>CSI 2 q
+>---
+>CSI 0 q
+>CSI ?12l
+
+  zpty_run 'zle_cursorform=( custom:none command:underline )'
+  esctest $'\egc'
+0:Setting custom cursor context to none uses default
+>CSI 6 q
+>CSI 0 q
+>CSI ?12l
+>CSI 4 q
+>---
+>CSI 0 q
+>CSI ?12l
+
+  zpty_run 'zle_cursorform=( command:underline )'
+  esctest $'\ehgca'
+0:Apply custom cursor context for which no form is defined
+>CSI 6 q
+>CSI 4 q
+>CSI 6 q
+>---
+>CSI 0 q
+>CSI ?12l
+
+  zpty_run $'\C-R.term.extensions=( -bracketed-paste -integration -cursor-shape )'
+  zpty_run 'zle_cursorform=( insert:hidden,underline,blink,color=#803030 )'
+  esctest
+0:Setting cursor shape disabled
+>CSI ?25l
+>OSC 12;rgb:8000/3000/3000 ST
+>---
+>CSI ?25h
+>CSI ?12l
+>OSC 112; ST
+
+  zpty_run '.term.extensions=( -bracketed-paste -integration -cursor-color )'
+  zpty_run 'zle_cursorform=( insert:hidden,underline,blink,color=#803030 )'
+  esctest
+0:Setting cursor color disabled
+>CSI ?25l
+>CSI 3 q
+>---
+>CSI ?25h
+>CSI 0 q
+>CSI ?12l
diff --git a/Test/comptest b/Test/comptest
index 13bbce3c8..377b1468e 100644
--- a/Test/comptest
+++ b/Test/comptest
@@ -87,15 +87,23 @@ zle-finish () {
   (( $+mark )) && print -lr "MARK: $mark"
   zle accept-line
 }
+esc-finish () {
+  print "<FINISH>"
+  BUFFER=""
+  zle accept-line
+}
 zle -N complete-word-with-report
 zle -N list-choices-with-report
 zle -N comp-finish
 zle -N zle-finish
+zle -N esc-finish
 bindkey "^I" complete-word-with-report
 bindkey "^D" list-choices-with-report
 bindkey "^Z" comp-finish
 bindkey "^X" zle-finish
 bindkey -a "^X" zle-finish
+bindkey "^Y" esc-finish
+bindkey -a "^Y" esc-finish
 '
 }
 
@@ -218,3 +226,23 @@ zletest () {
   # zpty_flush After zletest
   print -lr "${(@)${(@ps:\r\n:)log##*<WIDGET><finish>}[2,-2]}"
 }
+
+csi=$'\x1b\\[[0-9:;<=>?]#[ "#$%&\')(*+,./!-]#[@A-Z[\\\]_a-z{|}\`~^]'
+dcs=$'\x1bP[0-9;]#[+,./-]#[@A-Z[\\\]_a-z{|}\`~^]\x1b\\'
+osc=$'\x1b\\][ -~]#\x1b\\'
+
+esctest() {
+  local log
+  local -a out
+  zpty -n -w zsh "$*"$'\C-Y'
+  zpty_read log "*<FINISH>*<PROMPT>*" || {
+    print "failed to invoke finish widget."
+    return 1
+  }
+  log=${log%<END>*}
+  while [[ -n $log ]]; do
+    out+=( ${${${${${${(M)log##(<FINISH>|$~csi|$~osc|$~_dcs)}/<FINISH>/---}/$'\x1b\['/CSI }/$'\x1b\]'/OSC }/$'\x1bP'/DCS }/$'\x1b\\'/ ST} )
+    log=${log##(<FINISH>|$~csi|$~osc|$~dcs|?)}
+  done
+  print -rl - "$out[@]"
+}