Re: Help Request/Bug Report: comparguments causes _arguments to fail in certain cases

[Daniel Shahaf <d.s@xxxxxxxxxxxxxxxxxx>]

> > > > After _python returned, my script was invoked again and printed
> > > > `BUFFER=python ps.py` and `words=python*3* ps.py` (note the 3).  
> >
> > I'm not sure offhand where this comes from.  $context, $funcstack, et al
> > may have clues.  
> 
> 
> I may have neglected to say that I aliased `python` to `python3`, so it
> makes sense since you said that words is the result of alias expansion.

Yes, that'd be it.

> > You might try looking up the place that originally populates $words and
> > making it stash a copy of $words in another, read-only array; then you'll
> > be able to use «compset -P» and access that array.  (Compare vcs_info's
> > use of *_orig keys in ${hook_com}.)  
> 
> 
> I tried looking at things related to `vcs_info`, and I saw they were
> outside of the Completion directory.
> I was able to find the place where the `*_orig` keys are set, but couldn't
> find where they are used.
> It would be nice to get some context (or reference) about how this part of
> zsh connects to compsys, and how variables like `hook_com` become available
> to completion functions.

vcs_info has nothing to do with compsys.  Sorry for the unclarity.

*_orig variables would be used by hooks in people's dotfiles.  There
are examples of hooks in Misc/vcs_info-examples, though they don't
access the *_orig keys.

I was trying to explain by way of analogy: just like vcs_info sets
${hook_com[revision_orig]}, in order that if two hooks are installed
the later one be able to access the original value even if the
earlier one had modified ${hook_com[revision]}, so I imagined
a ${words_orig} array, to let your completion function access the
value of $words before the _arguments call in _python shifted some
elements off it.

> > Something along these lines, I think:
> >
> >     local -a words2=( ${(z):-"$PREBUFFER${LBUFFER}x"} )
> >     words2[-1]=${words2[-1]%x}  
> >     words2[1,${words2[(I)(;|&&|…)]}]=()  
> 
> 
> A lot for me to unpack here and in the paragraphs that follow.
> I'll try to understand as much as I can, but will probably ask about it
> later.

Sure, feel free.

«${foo[(I)bar]}» expands to an index into the array $foo; which index is
determined by the pattern «bar».  (You can get quick help by tab
completing after the opening round parenthesis; the full description of
${[(I)]} is in the manual.)

Do consider the other two alternatives, though.  Just because it's
_possible_ to reimplement the $BUFFER-to-$words transformation doesn't
imply that's the best way forward.

Cheers,

Daniel

> Thanks!
> Dan
> 
> On Tue, May 5, 2020 at 7:49 PM Daniel Shahaf <d.s@xxxxxxxxxxxxxxxxxx> wrote:
> 
> > > > > Well, for starters, if you «shift words», you should decrement  
> > CURRENT  
> > > > > as well.
> > > > >
> > > > > However, _python uses the «*::…» form of an _arguments optspec,
> > > > > which should take care of $words/$NUMERIC for you.  I assume the  
> > reason  
> > > > > it didn't is that you used «compdef -p».  Try -P instead?  If I'm not
> > > > > mistaken, that would also handle «python … script.py <TAB>» for you
> > > > > (where the ellipsis stands for python's --options).  
> > > >
> > > >
> > > > I tested the 4 possible options, and found that switching -p with -P
> > > > doesn't change anything, but shifting words and decrementing CURRENT
> > > > works!  But... I feel like it's a hack I'm not supposed to do
> > > > (correct me if I'm wrong).  
> >
> > Those parameters are documented, and manipulated by a number of
> > functions (including _python, as it happens), so you _are_ using the API.
> > See also the 'compset' builtin.
> >
> > However, as mentioned, I suspect that hardcoding a skip/decrement of one
> > element won't DTRT if python is invoked with options (e.g., «python
> > --foo script.py <TAB>») — not unless you reimplement _python's parsing
> > of those options and any arguments to them.
> >  
> > > > I now tried using the BUFFER variable instead of words (since it  
> > remains  
> > > > unchanged), but found that it also has caveats: The BUFFER variable
> > > > contains the raw value of the line, whereas the words array contains  
> > the  
> > > > words after expansions (I think that's the right term).  
> >
> > Partly.  $words has undergone alias expansion, but not further
> > expansions.  In particular, the words are still quoted.
> >  
> > > > To demonstrate this I tried running again completion for `python ps,py`
> > > > with `#compdef -P *`, and printing out BUFFER and words. The debug  
> > log  
> > > > (and my terminal) showed the following:
> > > > First of all _python was called and dispatched my script which printed
> > > > `BUFFER=python ps.py` and `words=ps.py`.  
> >
> > That's what I'd expect.
> >  
> > > > After _python returned, my script was invoked again and printed
> > > > `BUFFER=python ps.py` and `words=python*3* ps.py` (note the 3).  
> >
> > I'm not sure offhand where this comes from.  $context, $funcstack, et al
> > may have clues.
> >  
> > > > I do think the right solution is to be dispatched by _python with
> > > > `#compdef -P *.py`, but to be able to run I need the words array as
> > > > seen by _python  
> >
> > Once an element is shifted off $words, it's lost forever (free() is
> > called on it), so there'll be no way to access a "previous" value of $words
> > unless a copy had been made somewhere.
> >
> > Moreover, consider that python may be itself wrapped by some precommand:
> > for example, «env X=Y python», «sudo -u foo python», or «ssh foo python».
> >
> > You might try looking up the place that originally populates $words and
> > making it stash a copy of $words in another, read-only array; then you'll
> > be able to use «compset -P» and access that array.  (Compare vcs_info's
> > use of *_orig keys in ${hook_com}.)
> >  
> > > > or a to understand how BUFFER can be expanded/converted to its  
> > correct  
> > > > form.  If you have any input on this, I would very much appreciate
> > > > it.  
> >
> > Something along these lines, I think:
> >
> >     local -a words2=( ${(z):-"$PREBUFFER${LBUFFER}x"} )
> >     words2[-1]=${words2[-1]%x}  
> >     words2[1,${words2[(I)(;|&&|…)]}]=()  
> >
> > To understand the second line, see addx() in Src/Zle/zle_tricky.c.  With
> > that out of the way, the first line's a textbook use of the ${(z)}
> > tokenizer.  The third line just throws out everything until the start of
> > the current _simple command_ (see zshmisc(1)) — or, at least, it will
> > once the ellipsis is filled out.
> >
> > Caveats: that snippet doesn't do alias expansion, doesn't try to deal
> > with ${RBUFFER}, doesn't try to deal with SHORT_LOOPS, doesn't try to
> > deal with precommands and precommand assignments, should be changed to
> > use zZ+c+ if the INTERACTIVE_COMMENTS option is set…
> >
> > Although it's a plugin that I worked on myself, I should probably
> > mention that zsh-syntax-highlighting has faced the same problem and you
> > and implemented a solution to it, which handles a number of the above
> > caveats.
> >  
> > >  
> >
> > There's also a third approach: instead of trying to run the binary, make
> > it stash the output in a well-known location.  For example, /usr/bin/foo
> > could store the output in /usr/share/argcomplete/foo.  (I've always
> > wanted to standardize _some_ solution to this problem; I'm tired of
> > writing completion functions by hand…)
> >
> > Cheers,
> >
> > Daniel
> >  
> 
>