On 19 October 2015 at 19:21, Bart Schaefer <schaefer@xxxxxxxxxxxxxxxx> wrote: > We leave that empty arena at the end, but then on the next zhalloc() > it's still not big enough, so another one gets allocated. > > You could try removing that "else" to let the trailing block be freed > again, but then we're back to needing to compare speed as well as space > across shell versions. I removed the else, code works longer and memory is still allocated. This is seen in mem-test3.txt, where there are many samples and large numbers, and in perf-test.txt, where there noted is long running time (numbers are milliseconds) I wrote perf-test.zsh to automatically perform performance tests: https://github.com/psprint/zsh-tools/ With it and with mem-test.zsh I tested following zsh variants: - 5.0.2 osx - newheaps patch (zsh-newheaps-only) - newheaps and zhalloc patch 36836 (zsh-newheaps-zhalloc) - zhalloc and the previous patch 36834 (zsh-head-both-patches) - all three patches (zsh-newheaps-three-patches) - a 5.1.1-dev-0 clean zsh - the empty arena else removed My conclusions: - 36834 introduces high memory usage; I would suggest to use only two optimizations – newheaps and zhalloc; my znavtools are instant fast with them, what's slower is searching (wrote search_test function, results attached); basically half of pattern search speedup is lost with 36834, however what's gained is beautiful memory usage, without firefox-like values for RES (~700 MB); newheaps is responsible for instant-responsivity of my script, 36836 for much faster searching – that's a good compromise - searching with patterns through very large arrays (700k) allocates 264 MB of memory regardless of zsh version; this probably tells something - still not sure why 36834 allocates much memory for the string test - newheaps makes function calls longer, but it's not a substantial difference – 21196 ms vs. 18549 ms Best regards, Sebastian Gniazdowski
# Tests for zsh-empty-arena-else # string_test 51, 152, 206, 247, 280, 305, 327, 345, 363, 380, 394, 407, 421, 432, 444, 455, 464, 474, 483, 493, 502, 509, 518, 525, 534, 541, 549, 555, 560, 569, 575, 581, last: 586 string_test 421.3 # array_test 1, 2.7, 2.9, 3.8, 4.3, 5.6, 6.5, 7.6, 8, 8.4, 8.7, 8.9, last: 9.1 array_test 5.4 # function_test 2.6, 2.9, 2.7, 2.9, 2.9, 2.8, 2.9, 2.7, last: 2.6 function_test 2.8 # search_test 110, 215, last: 264 search_test 110.0 # Tests for zsh-newheaps-zhalloc # string_test 2.9, 7.7, 9.3, 11, 11, 11, 12, 11, 12, 11, 11, 11, 12, last: 11 string_test 10.1 # array_test 1, 2, 3.1, 3.3, 4.5, 5.3, 6.3, 7.5, 7.8, 8.2, 8.5, 8.7, last: 8.9 array_test 5.2 # function_test 2.7, 2.7, 2.8, 2.6, 2.6, 2.9, 2.9, 3, last: 2.6 function_test 2.7 # search_test 86, 158, 200, 237, last: 265 search_test 148.0 # Tests for zsh-newheaps-only # string_test 3, 7.7, 9.1, 11, 11, 11, 12, 11, 11, 11, 12, 11, 11, last: 10 string_test 10.1 # array_test 1.4, 2, 3, 3.8, 4.4, 5.3, 6.3, 7, 7.7, 8, 8.3, 8.5, last: 8.8 array_test 5.2 # function_test 2.7, 2.6, 2.8, 2.7, 2.9, 2.7, 2.7, 2.7, last: 2.6 function_test 2.7 # search_test 62, 122, 144, 171, 196, 218, 238, 259, last: 266 search_test 164.4 # Tests for zsh-5.0.2-osx # string_test 2.5, 7.4, 8.9, 11, 11, 12, 11, 11, 11, 11, 12, 12, last: 10 string_test 9.9 # array_test 1.4, 2.2, 3, 4, 4.6, 5.3, 6.3, 7.4, 7.8, 8, 8.4, 8.7, last: 8.8 array_test 5.3 # function_test 2.6, 2.6, 2.6, 2.6, 2.6, 2.6, 2.6, last: 2.6 function_test 2.6 # search_test 59, 122, 152, 179, 204, 226, 245, last: 264 search_test 157.0 # Tests for zsh-head-both-patches # string_test 40, 182, 318, 456, 589, 725, last: 9.7 string_test 317.0 # array_test 1.1, 2.5, 3, 3.8, 4.3, 5, 6, 7, 7.5, 7.8, 8.2, 8.3, last: 8.5 array_test 5.1 # function_test 2.6, 2.6, 2.6, 2.6, 2.5, 2.6, 2.6, last: 2.5 function_test 2.6 # search_test 122, 217, last: 266 search_test 122.0 # Tests for zsh-newheaps-three-patches # string_test 44, 188, 331, 471, 612, last: 8.9 string_test 258.5 # array_test 1.1, 2, 3.1, 3.8, 4.4, 5.1, 6.2, 7.2, 7.7, 8, 8.2, 8.6, last: 8.7 array_test 5.2 # function_test 2.7, 2.7, 2.9, 2.8, 2.9, 2.7, 2.9, 2.6, last: 2.6 function_test 2.8 # search_test 116, 216, last: 265 search_test 116.0 # Tests for zsh-5.1.1-dev-0-clean # string_test 2.6, 7.3, 8.8, 11, 11, 11, 10, 11, 12, 11, 12, 11, last: 10 string_test 9.8 # array_test 1.3, 2.2, 2.3, 3.4, 4.2, 5, 6.1, 6.9, 7.6, 7.8, 8.1, 8.4, last: 8.6 array_test 5.0 # function_test 2.6, 2.6, 2.6, 2.6, 2.6, 2.6, last: 2.6 function_test 2.6 # search_test 62, 121, 151, 177, 201, 223, 243, 264, last: 265 search_test 168.3
Running [zsh-empty-arena-else]: string_test 13749,47 Running [zsh-empty-arena-else]: array_test 12182,06 Running [zsh-empty-arena-else]: function_test 21345,66 Running [zsh-empty-arena-else]: search_test 3849,80 Running [zsh-newheaps-zhalloc]: string_test 9990,75 Running [zsh-newheaps-zhalloc]: array_test 12109,69 Running [zsh-newheaps-zhalloc]: function_test 21318,52 Running [zsh-newheaps-zhalloc]: search_test 6842,58 Running [zsh-newheaps-only]: string_test 9999,97 Running [zsh-newheaps-only]: array_test 11925,05 Running [zsh-newheaps-only]: function_test 21196,27 Running [zsh-newheaps-only]: search_test 11670,71 Running [zsh-5.0.2-osx]: string_test 9628,87 Running [zsh-5.0.2-osx]: array_test 12294,85 Running [zsh-5.0.2-osx]: function_test 18668,42 Running [zsh-5.0.2-osx]: search_test 11518,38 Running [zsh-head-both-patches]: string_test 4385,93 Running [zsh-head-both-patches]: array_test 11933,38 Running [zsh-head-both-patches]: function_test 18549,57 Running [zsh-head-both-patches]: search_test 3949,40 Running [zsh-newheaps-three-patches]: string_test 4351,58 Running [zsh-newheaps-three-patches]: array_test 12111,25 Running [zsh-newheaps-three-patches]: function_test 21356,30 Running [zsh-newheaps-three-patches]: search_test 4063,73 Running [zsh-5.1.1-dev-0-clean]: string_test 9651,08 Running [zsh-5.1.1-dev-0-clean]: array_test 12749,21 Running [zsh-5.1.1-dev-0-clean]: function_test 18667,25 Running [zsh-5.1.1-dev-0-clean]: search_test 11386,36
Attachment:
mem-test.zsh
Description: Binary data
Attachment:
perf-test.zsh
Description: Binary data