Zsh Mailing List Archive
Messages sorted by: Reverse Date, Date, Thread, Author

Re: The HIST_EXPIRE_DUPS_FIRST might corrupt and wipe partially history file if many shells exit at the same time


On 19/03/2023 17.53, Bart Schaefer wrote:
 > On Sun, Mar 19, 2023 at 4:56 AM Piotr Karbowski
 > <piotr.karbowski@xxxxxxxxxxxxx> wrote:
 >> They way I can reproduce it sometimes is if multiple zsh shells exit at
 >> the very same time. It happens when I terminate tmux session that have
 >> 10+ zsh instances,or when I just reboot my system while I have dozens of
 >> urxvt instances open.
 >>       +SAVEHIST='96000'
 >>       +setopt hist_expire_dups_first
 >> I have disabled this feature since due those corruptions. Would love to
 >> get back to it though, perhaps adding some locking mechanism would help
 >> here?
 > There is a locking mechanism.  A couple of things may be happening:
 > 1) Your home directory (or ZDOTDIR) is on a remote filesystem with
 > asynchronous mounting, so the locking mechanism doesn't always work.
 > You haven't described your system configuration in any detail, but
 > from what you have said this one seems unlikely.
 > 2) The locking mechanism doesn't work for some other reason.  Make
 > sure the HIST_SAVE_BY_COPY option is set (it's the default but check
 > anyway).
 > 3) When exiting a session (or particularly when rebooting), there may
 > be a limited amount of time for the processes to clean up and exit
 > before they are killed.  With the combination of options you have,
 > each shell must re-read the history file to merge it with its local
 > history before writing it back.  With multiple shells and tens of
 > thousands of lines of history, this might take longer than all the
 > shells are allowed to stay alive.
 > Try putting something in your .zlogout file that writes a timestamp to
 > a file (different for each shell, e.g., use $$ in the filename).
 > Since .zlogout runs after history is saved, if you find any of those
 > files to be missing, that's a clue that #3 is happening.  You could
 > also look to see if a ".zsh_history.new" file is left behind.
Thanks for the response and applogies for missing details.

I am runnin XFS (on the top of lvm, on the top of dmcrypt).

The current setopt is (without the expire dups first)


The HIST_SAVE_BY_COPY does not seems to be enabled and I am unable to
enable it. Running

     setopt HIST_SAVE_BY_COPY

Does not bring this option into list I can see with running setopt. This
is zsh-5.9. Running unsetopt without params does show nohistsavebycopy
there, however running

     setopt histsavebycopy

Does nothing, it still remains as nohistsavebycopy in unsetopt output.

The question then is -- why this feature remains disabled and cannot be
toggled on? Even running zsh -dfi to get a shell without zshrc behave
the same way, this feature remains disabled and cannot be toggled on.

-- Piotr.

Messages sorted by: Reverse Date, Date, Thread, Author