Regression with completion cache (was: Re: Performance of _store_cache and _retrieve_cache)

[Daniel Hahler <genml+zsh-workers@xxxxxxxxxx>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

the patch from 34476 (ae7dcab) breaks the caching for the _docker completion.

- From https://github.com/docker/docker/blob/master/contrib/completion/zsh/_docker#L177:

  __docker_commands () {
      # local -a  _docker_subcommands
      local cache_policy
  
      zstyle -s ":completion:${curcontext}:" cache-policy cache_policy
      if [[ -z "$cache_policy" ]]; then
          zstyle ":completion:${curcontext}:" cache-policy __docker_caching_policy
      fi
  
      if ( [[ ${+_docker_subcommands} -eq 0 ]] || _cache_invalid docker_subcommands) \
          && ! _retrieve_cache docker_subcommands;
      then
          local -a lines
          lines=(${(f)"$(_call_program commands docker 2>&1)"})
          _docker_subcommands=(${${${lines[$((${lines[(i)Commands:]} + 1)),${lines[(I)    *]}]}## #}/ ##/:})
          _docker_subcommands=($_docker_subcommands 'help:Show help for a command')
          _store_cache docker_subcommands _docker_subcommands
      fi
      _describe -t docker-commands "docker command" _docker_subcommands
  }

The cache being used is not the same as when storing it.

"docker ps" completes to "docker stops," then: "ps" is not in the cache,
and "stops," should be a description.

(backing out this commit made it work again)

The cache file has a line which is 1827 bytes long, which appears to be
an issue: removing some of the first entries makes "docker ps" complete
again.  Is there some 1024 bytes limit or something similar?

The cache file looks like this:

% cat .zcompcache/docker_subcommands
_docker_subcommands=( "${(zQ)$(<<\EO:_docker_subcommands
'attach:Attach to a running container' 'build:Build an image from a Dockerfile' … 'help:Show help for a command'
EO:_docker_subcommands
)}" )


Regards,
Daniel.

On 08.02.2015 21:27, Bart Schaefer wrote:
> On Feb 8,  5:19pm, Daniel Hahler wrote:
> }
> } _store_cache saves the array like this:
> } 
> }     _zsh_all_pkgs=( '02exercicio' '0x10c-asm'  ... )
> } 
> } The problem is that `source ./pip_allpkgs.slow` takes about 8 seconds,
> } and is slower than generating the list anew!
> 
> The slowdown here appears to be with compiling the source'd file into
> the internal wordcode format before executing it.  Even dumping the
> whole assignment as a single string and then using "eval" on that, is
> faster than allowing "source" to parse the words directly.  We may
> want to dig further into why that is the case.
> 
> The following is a LOT faster than either "source" or "eval", but may
> require recent changes that fix bugs in the parsing of $(...) :
> 
> _zsh_all_pkgs=( "${(zQ)$(<<\EO:_zsh_all_pkgs
> '02exercicio' '0x10c-asm' ...
> EO:_zsh_all_pkgs
> )}" )
> 
> 
> diff --git a/Completion/Base/Utility/_store_cache b/Completion/Base/Utility/_store_cache
> index 86e72e9..8feaee6 100644
> --- a/Completion/Base/Utility/_store_cache
> +++ b/Completion/Base/Utility/_store_cache
> @@ -46,8 +46,15 @@ if zstyle -t ":completion:${curcontext}:" use-cache; then
>    for var; do
>      case ${(Pt)var} in
>      (*readonly*) ;;
> -    (*(association|array)*) print -r "$var=( ${(kv@Pqq)^^var} )";;
> -    (*)                     print -r "$var=${(Pqq)^^var}";;
> +    (*(association|array)*)
> +	# Dump the array as a here-document to reduce parsing overhead
> +	# when reloading the cache with "source" from _retrieve_cache
> +	print -r "$var=( "'"${(zQ)$(<<\EO:'"$var"
> +	print -r "${(kv@Pqq)^^var}"
> +	print -r "EO:$var"
> +	print -r ')}" )'
> +	;;
> +    (*) print -r "$var=${(Pqq)^^var}";;
>      esac
>    done >! "$_cache_dir/$_cache_ident"
>  else
> 
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iD8DBQFVX9OifAK/hT/mPgARAgxeAKDFsc0cg+pzpQ5xmF99D78nq3EqrQCfdpU9
r5vXtuoDJrVpTtuZ6lOcdNs=
=xe31
-----END PGP SIGNATURE-----