Re: BUG: Zsh loses history entries since 2015

[Oliver Kiddle <opk@xxxxxxx>]

Michael Stapelberg wrote:
> As I said, I have run into this issue many times over the years,
> it just took a lot of time to properly track it down to this bug.

I've occasionally had a shell start with a truncated history. I include
the history number in the shell prompt so it's generally fairly obvious
and I tend to immediately kill any shell that comes up with a history
number reset to 1. I think I remember seeing it along with an error
indicating a failure to lock the history file so that's perhaps linked
to using NFS.

> I move my history files out of the way every couple of years to keep them lean.
> Currently, I have 60k lines in .zsh_history (3.7 MB), and >>350k lines
> in historical ~/.zsh_history_<date> files (15 MB), which zsh ignores,
> but I grep on occasion when I need to go back very far in time.
>
> I’m not sure if this qualifies as “very large” :)

It's probably not very large these days but would have been back when
much of the history code was written.

> Why? What does it even mean to interrupt the loading of history at
> shell startup?
> Why would I ever want a shell process to only partially load my history?

If shell startup seems to be especially slow, being able to interrupt it
with Ctrl-C and get a basic shell prompt is useful. NFS and temporary
network issues can be the cause. If I just want to get in to fix a
problem a degraded but working shell is much better than having it hang
indefinitely.

> > 1) If reading history is interrupted, disable saving history entirely,
> > perhaps by unset/reset of HISTFILE and/or SAVEHIST.  This would
> > preserve file contents but lose current shell history.

This seems like a reasonable option.

> Do you mean just when exiting (that seems okay) or also on startup?
> Applying this behavior at startup would mean that users could run a shell
> that just silently does not save to history, right? That does not sound good.

Not good indeed but losing one session's new commands may be preferable
to losing part of the existing history file. Perhaps the shell can print
something to warn that history saving is disabled along with the command
to ignore and reenable it.

> > 2) Change behavior to append current shell history to the file as if
> > APPEND_HISTORY, without attempting to enforce SAVEHIST size or
> > duplicate expiration.  This is probably the option most likely to
>
> This approach would no longer rewrite history when exiting,
> so at which points would zsh rewrite history then? Or would history
> files only ever grow and effectively SAVEHIST size has no more effect?

I imagine that once you have a different shell that manages to startup
without interruption. it would then apply the SAVEHIST limit. So it'd
only be a problem if something causes the interruption to occur for all
shells or if someone is in the habit of doing everything from a single
terminal that they leave logged in for an extended time.

Oliver