Re: Slowdown around 5.0.5-dev-0

[Sebastian Gniazdowski <sgniazdowski@xxxxxxxxx>]

The patch (alone) much helped, zprof time is now 3027 and there is no
after-while slow down. Htop shows RES of 152 MB. For the first patch
RES is 166 MB. For the clean version it's 152 MB. I didn't combine the
patches (BTW. the problems with applying patches were caused by expand
tabs that occured on copy-pasting).


On 11 October 2015 at 19:31, Bart Schaefer <schaefer@xxxxxxxxxxxxxxxx> wrote:
> On Oct 11,  6:48pm, Sebastian Gniazdowski wrote:
> }
> } It's now much faster, zprof time 11851 vs 3433. That's still strongly
> } lagging behavior. One other trait is that it slows down after a while.
>
> The whole algorithm is designed to maximally fill the allocated space
> and thereby minimize the memory footprint, not to maximize speed.  So
> the more often you grow/free the heap, the more time it spends looking
> for unused space in the existing arenas.
>
> Besides zprof, it would be interesting to see the maximum memory usage
> of the patched and unpatched shells.
>
> Does this change in zhalloc() make any difference?  This goes farther
> in the direction of abandoning small bits of earlier arenas until the
> popheap() comes around.
>
> diff --git a/Src/mem.c b/Src/mem.c
> index b9569ea..63a5afe 100644
> --- a/Src/mem.c
> +++ b/Src/mem.c
> @@ -543,9 +543,14 @@ zhalloc(size_t size)
>
>      /* find a heap with enough free space */
>
> -    for (h = ((fheap && ARENA_SIZEOF(fheap) >= (size + fheap->used))
> -             ? fheap : heaps);
> -        h; h = h->next) {
> +    /*
> +     * This previously assigned:
> +     *   h = ((fheap && ARENA_SIZEOF(fheap) >= (size + fheap->used))
> +     *       ? fheap : heaps);
> +     * but we think that nothing upstream of fheap has more free space,
> +     * so why start over at heaps just because fheap has too little?
> +     */
> +    for (h = (fheap ? fheap : heaps); h; h = h->next) {
>         if (ARENA_SIZEOF(h) >= (n = size + h->used)) {
>             void *ret;
>

patched (zhalloc):
num  calls                time                       self            name
-----------------------------------------------------------------------------------
 1)    1        3027,41  3027,41   99,53%   2025,50  2025,50   66,59%  n-list
 2)  161         674,02     4,19   22,16%    674,02     4,19   22,16%  _nlist_print_with_ansi
 3)    7         962,52   137,50   31,64%    288,50    41,21    9,48%  n-list-draw
 4)    7          24,50     3,50    0,81%     24,31     3,47    0,80%  n-list-input
 5)    2          13,09     6,54    0,43%     13,09     6,54    0,43%  _nlist_cursor_visibility
 6)    7           9,18     1,31    0,30%      8,99     1,28    0,30%  _nlist_setup_user_vars
 7)    1          14,33    14,33    0,47%      6,23     6,23    0,20%  _nlist_exit
 8)    7           0,71     0,10    0,02%      0,71     0,10    0,02%  _nlist_status_msg
( n-panelize < ~/lsoflsof; )  4,62s user 0,18s system 88% cpu 5,432 total

patched (the first patch):
num  calls                time                       self            name
-----------------------------------------------------------------------------------
 1)    1        3547,43  3547,43   99,63%   2560,58  2560,58   71,92%  n-list
 2)  161         636,56     3,95   17,88%    636,56     3,95   17,88%  _nlist_print_with_ansi
 3)    7         921,28   131,61   25,88%    284,71    40,67    8,00%  n-list-draw
 4)    7          49,42     7,06    1,39%     49,33     7,05    1,39%  n-list-input
 5)    2          11,96     5,98    0,34%     11,96     5,98    0,34%  _nlist_cursor_visibility
 6)    7          10,80     1,54    0,30%     10,64     1,52    0,30%  _nlist_setup_user_vars
 7)    1          13,05    13,05    0,37%      6,17     6,17    0,17%  _nlist_exit
 8)    7           0,28     0,04    0,01%      0,28     0,04    0,01%  _nlist_status_msg
( n-panelize < ~/lsoflsof; )  5,17s user 0,18s system 88% cpu 6,032 total

clean:
num  calls                time                       self            name
-----------------------------------------------------------------------------------
 1)  161        5176,77    32,15   43,84%   5176,77    32,15   43,84%  _nlist_print_with_ansi
 2)    1       11785,16 11785,16   99,79%   3755,23  3755,23   31,80%  n-list
 3)    7        7640,70  1091,53   64,70%   2463,93   351,99   20,86%  n-list-draw
 4)    7         316,45    45,21    2,68%    310,97    44,42    2,63%  n-list-input
 5)    7          37,72     5,39    0,32%     30,00     4,29    0,25%  _nlist_setup_user_vars
 6)    7          26,52     3,79    0,22%     26,52     3,79    0,22%  _nlist_status_msg
 7)    2          23,68    11,84    0,20%     23,68    11,84    0,20%  _nlist_cursor_visibility
 8)    1          24,45    24,45    0,21%      9,32     9,32    0,08%  _nlist_exit
( n-panelize < ~/lsoflsof; )  13,64s user 0,20s system 95% cpu 14,548 total