Re: zsh generates invalid UTF-8 encoding in the history

[Bart Schaefer <schaefer@xxxxxxxxxxxxxxxx>]

On Oct 7, 10:57am, Vincent Lefevre wrote:
} Subject: Re: zsh generates invalid UTF-8 encoding in the history
}
} Thanks. Note that it is not clear in the man page whether the
} least recent starts at 0 or at 1. And when -r is used, whether
} one should use <most recent> <least recent> or the reverse.

Indeed, the use of "first last" in the doc is a bit confusing, and
so is the interpretation of the numbers by the command.  It's always
most then least recent; but since negative numbers count backwards
from the most recent (largest number) and the default behavior is
described in terms of using negative offsets, it can be confusing.
 
} And how can one cleanly append history lines to a history file?
} I was using "cat some_file >> ~/.zhistory", which seems to work,

It'll work as long as there are no 0x83 bytes in some_file.

To be completely safe, you need to do something like this:

  # Pass input file as $1, output as $2
  append_plain_file_to_history_file() {
      emulate -LR zsh
      local -a entries

      # Implementation issue:  read -r ignores backslash-newline
      # folding, but without -r embedded backslashes are stripped,
      # which seems a bigger problem.  Fix up $entries later.

      IFS=$'\n' read -r -d '' -A entries <$1
      (( $#entries )) || return

      # Must supply a file name here to set HISTSIZE and SAVEHIST
      fc -pa /dev/null $#entries $(( SAVEHIST + $#entries ))

      while (( $#entries )); do
          if [[ "$entries[1]" == *\\ ]]; then
              entries[1,2]=( ${entries[1]%\\}$'\n'${entries[2]} )
          else
              print -S $entries[1]
              shift 1 entries
          fi
      done
      fc -A ${2:-$HISTFILE}

      # Reset SAVEHIST to avoid attempting to lock /dev/null
      SAVEHIST=0	# fc -p makes this implicitly local
  }