Zsh Mailing List Archive
Messages sorted by: Reverse Date, Date, Thread, Author

Re: compsys maps anonymous memory and never frees it





On Wed, Sep 3, 2008 at 5:06 AM, Bart Schaefer <schaefer@xxxxxxxxxxxxxxxx> wrote:
On Sep 3,  4:09am, xRaich[o]²x wrote:
}
} I made an interesting observation while testing compsys in zsh on
} OpenSolaris. Everytime completion kicks in zsh maps a segment of
} anonymous memory and it looks like that it won't unmap it.

Version number of zsh involved?

There are only three places where zsh maps memory.  One is when it
loads a .zwc file (zcompiled functions); one is whan it references
the $mapfile special parameter (which the standard set of compsys
functions never does); and one is the heap allocator in Src/mem.c:

    There are two ways to allocate memory in zsh.  The first way is
    to call zalloc/zshcalloc, which call malloc/calloc directly.  It
    is legal to call realloc() or free() on memory allocated this way.
    The second way is to call zhalloc/hcalloc, which allocates memory
    from one of the memory pools on the heap stack.  Such memory pools
    will automatically created when the heap allocation routines are
    called.  [...]

    If possible, the heaps are allocated using mmap() so that the
    (*real*) heap isn't filled up with empty zsh heaps. If mmap()
    is not available and zsh's own allocator is used, we use a simple trick
    to avoid that: we allocate a large block of memory before allocating
    a heap pool, this memory is freed again immediately after the pool
    is allocated. If there are only small blocks on the free list this
    guarantees that the memory for the pool is at the end of the memory
    which means that we can give it back to the system when the pool is
    freed.

It's possible that there's a popheap() or the equivalent missing from
somewhere.  It's also possible that the OpenSolaris munmap() does not
really (or not immediately) release memory back to the system, because
if I repeat your same example on my CentOS 4 box, the number of mapped
blocks remains constant.

Zsh version is 4.3.6 and 4.3.4. Both show the same behavior.

I looked at the problem a little closer. Zsh does not call mmap to allocate them and they dont get allocated when completion happens but when the next command gets issued. So in my example the new maps got added to the process' address space when i executed pmap, but the same happens with any other programm. Builtins however are an exception. So things start to go wrong when it comes to forking.

I hope this helps to pinpoint the problem. Here are the backtraces i got from dtrace when one of those "sticky" segments get added to the zsh  address space. This might also help:

Userspace stacktrace:

  0  -> as_addseg                            
              libc.so.1`__forkx+0xb
              libc.so.1`fork+0x1a
              zsh`zfork+0x56
              zsh`execcmd+0xe85
              zsh`execpline2+0xe4
              zsh`execpline+0x195
              zsh`execlist+0x390
              zsh`execode+0x38
              zsh`loop+0x26e
              zsh`zsh_main+0x1ea
              zsh`main+0x11
              zsh`_start+0x7a

Kernelspace backtrace:
              genunix`seg_attach+0x21
              genunix`seg_alloc+0xb4
              genunix`as_dup+0xba
              genunix`cfork+0xe1
              genunix`forksys+0x19
              unix`sys_call+0x10c

Regards,
Björn



Messages sorted by: Reverse Date, Date, Thread, Author