Re: One more heap optimization trick for zsh < 5.2

[Sebastian Gniazdowski <sgniazdowski@xxxxxxxxx>]

On 11 June 2016 at 05:30, Bart Schaefer <schaefer@xxxxxxxxxxxxxxxx> wrote:
> This means that it has to be something that's happening in the loop
> inside the n-list function, affecting every iteration.

Agree

> Consequently it must be one of n-list-input or n-list-draw, both of
> which are declared with [the shadowed] autoload inside n-list itself.
> Something about nesting calls to --zplg-reload-and-run is causing
> the issue.

Hmm didn't thought about that, about nesting

> It might have something to do with retaining the old copy
> of the function body until the call stack unwinds and it can safely
> be freed; a large value for $argv would therefore also hang around
> (two copies of it) for all of that time.

That's apparently a hit. I attach two test files. First doesn't pass
anything to sub-function:

# ./aload.zsh2.txt
Normal autoload:
( aload_fun_main; )  10,91s user 0,23s system 99% cpu 11,156 total
Special autoload:
( aload_fun_main; )  9,54s user 0,23s system 99% cpu 9,792 total
More special autoload:
( aload_fun_main; )  9,15s user 0,22s system 99% cpu 9,393 total

# ./aload.zsh2.txt
Normal autoload:
( aload_fun_main; )  8,85s user 0,22s system 99% cpu 9,103 total
Special autoload:
( aload_fun_main; )  8,15s user 0,23s system 99% cpu 8,405 total
More special autoload:
( aload_fun_main; )  9,36s user 0,22s system 99% cpu 9,596 total

# ./aload.zsh2.txt
Normal autoload:
( aload_fun_main; )  10,22s user 0,22s system 99% cpu 10,454 total
Special autoload:
( aload_fun_main; )  9,73s user 0,22s system 99% cpu 9,971 total
More special autoload:
( aload_fun_main; )  8,02s user 0,22s system 99% cpu 8,262 total

(I wonder why the varying). Second one passes long array to sub-function:

# ./aload.zsh3.txt
Normal autoload:
( aload_fun_main; )  11,58s user 0,25s system 99% cpu 11,847 total
Special autoload:
( aload_fun_main; )  10,77s user 0,25s system 99% cpu 11,042 total
More special autoload:
( aload_fun_main; )  18,38s user 0,27s system 99% cpu 18,679 total

# ./aload.zsh3.txt
Normal autoload:
( aload_fun_main; )  10,99s user 0,25s system 99% cpu 11,270 total
Special autoload:
( aload_fun_main; )  10,71s user 0,26s system 99% cpu 10,984 total
More special autoload:
( aload_fun_main; )  19,02s user 0,28s system 99% cpu 19,329 total

> Incidentally, the way n-list is currently written, it will re-autoload
> those functions and re-create all the _nlist_* helpers EVERY TIME IT
> IS RUN.  Mostly that's all of no net effect, but it must be slowing
> down each call to n-list at least a little bit.

I have a check if the function exists and don't do autoload, like the
real autoload function does afaik.

-- 
Best regards,
Sebastian Gniazdowski

#!/bin/zsh

emulate -LR zsh

--reload-and-run () {
    local fpath_prefix="$1" autoload_opts="$2" func="$3"
    shift 3
    unfunction "$func"
    local FPATH="$fpath_prefix":"${FPATH}"
    builtin autoload $=autoload_opts "$func"
    "$func" "$@"
}

fbody1='#!/bin/zsh
str=""
str=${(r:100000::_:)str};
arr=( "${(@s::)str}" )
repeat 10; do
    aload_fun "${arr[@]}"
done
'

fbody2='#!/bin/zsh
str=""
str=${(r:100000::_:)str};
arr=( "${(@s::)str}" )
repeat 10; do
    arr2=( "${(@M)arr:#(#a1)_}" )
done
'

echo "$fbody1" > aload_fun_main
echo "$fbody2" > aload_fun

# Normal autoload
FPATH+=:`pwd`
autoload aload_fun_main
autoload aload_fun
echo "Normal autoload:"
time ( aload_fun_main )
unfunction aload_fun_main
unfunction aload_fun

# Special autoload
eval "
function aload_fun_main {
    local FPATH=$FPATH:`pwd`
    autoload -X
}
"
eval "
function aload_fun {
    local FPATH=$FPATH:`pwd`
    autoload -X
}
"
echo "Special autoload:"
time ( aload_fun_main )
unfunction aload_fun_main
unfunction aload_fun

# More special autoload
PLUGIN_DIR=`pwd`
eval "
function aload_fun_main {
    --reload-and-run ${(q)PLUGIN_DIR} '' aload_fun_main
}
"
eval "
function aload_fun {
    --reload-and-run ${(q)PLUGIN_DIR} '' aload_fun
}
"
echo "More special autoload:"
time ( aload_fun_main )
unfunction aload_fun_main
unfunction aload_fun

#!/bin/zsh

emulate -LR zsh

--reload-and-run () {
    local fpath_prefix="$1" autoload_opts="$2" func="$3"
    shift 3
    unfunction "$func"
    local FPATH="$fpath_prefix":"${FPATH}"
    builtin autoload $=autoload_opts "$func"
    "$func" "$@"
}

fbody1='#!/bin/zsh
str=""
str=${(r:100000::_:)str};
arr=( "${(@s::)str}" )
repeat 10; do
    aload_fun
done

#print -rl "${arr2[@]}"
'

fbody2='#!/bin/zsh
str=""
str=${(r:100000::_:)str};
arr=( "${(@s::)str}" )
repeat 10; do
    arr2=( "${(@M)arr:#(#a1)_}" )
done

#print -rl "${arr2[@]}"
'

echo "$fbody1" > aload_fun_main
echo "$fbody2" > aload_fun

# Normal autoload
FPATH+=:`pwd`
autoload aload_fun_main
autoload aload_fun
echo "Normal autoload:"
time ( aload_fun_main )
unfunction aload_fun_main
unfunction aload_fun

# Special autoload
eval "
function aload_fun_main {
    local FPATH=$FPATH:`pwd`
    autoload -X
}
"
eval "
function aload_fun {
    local FPATH=$FPATH:`pwd`
    autoload -X
}
"
echo "Special autoload:"
time ( aload_fun_main )
unfunction aload_fun_main
unfunction aload_fun

# More special autoload
PLUGIN_DIR=`pwd`
eval "
function aload_fun_main {
    --reload-and-run ${(q)PLUGIN_DIR} '' aload_fun_main
}
"
eval "
function aload_fun {
    --reload-and-run ${(q)PLUGIN_DIR} '' aload_fun
}
"
echo "More special autoload:"
time ( aload_fun_main )
unfunction aload_fun_main
unfunction aload_fun