Zsh Mailing List Archive
Messages sorted by: Reverse Date, Date, Thread, Author

Re: How to add string to end of each array element without loops



On Mar 26,  4:39am, nix@xxxxxxxxxxxxxxxx wrote:
}
} Line 143: print -l ${(n)PROXIES} >> $Proxy_List causes segfault. I can't
} see where the problem is, if you try IP range up to one million, it will
} work good.

Aha.  I'm able to reproduce this with

typeset -A a
a=({5000000..1} {1..5000000})
print -l $a >> /dev/null

What's actually failing is builtin.c line 283, the declaration

	VARARR(char *, argarr, argc + 1);

This is attempting to allocate the entire argument list of "print"
(which in my example is 5 million words) on the C function call
stack.  Even with "unlimit stacksize" this is likely to overflow.
The program merely doesn't discover that this has failed until a
few lines later when it tries to use the first element of the array.

There's really nothing that can be done about this.  The whole shell
paradigm of passing arguments as an argv[] array means that $a has
to be expanded and then "print" called with the words by value.

So what you have to do to work around this is NOT rely on "print -l"
to insert the newlines, and instead insert them yourself:

print ${(F)PROXIES} >> $Proxy_List

This passes a single giant word with embedded newlines to "print" and
thus avoids allocating all that space on the C stack.

You're really past the design limits here of what a language with the
semantic rules of a command shell is meant to deal with.  If "print"
were not a builtin you'd have blown out the limits of argument passing
before even getting as far as you did; even with the (F) trick, using
/bin/echo will fail with "argument list too long".

I noticed you loaded the mapfile module even though you don't use it.
It should work to do

mapfile[$Proxy_list]=${(F)PROXIES}

but whether that's actually faster than "print >>" I haven't checked.



Messages sorted by: Reverse Date, Date, Thread, Author