Re: Parallel processing

[Philippe Troin <phil@xxxxxxxx>]

On Fri, 2022-03-25 at 11:27 -0700, Bart Schaefer wrote:
> On Thu, Mar 24, 2022 at 9:34 PM Perry Smith <pedz@xxxxxxxxxxxxxxxx> wrote:
> 
> 
> This isn't exactly what you want because it waits for all four jobs
> before starting the next batch, but keeping a specific number of
> children running is not straightforward with the job-management
> operations available to a shell.

There may be a way to achieve keeping a set number of children around,
by trapping SIGCHLD, but we would completely lose the exit status of
the command.  There does not seem to be a way to retrieve the exit
status of a command as soon as SIGCHLD is trapped:

   % zsh -f
   % echo $ZSH_VERSION
   5.8.1
   
   % setopt monitor
   % trap 'x=$?; echo "CHLD args=$* exit=$x"; wait $PID ' CHLD; (sleep 1;  exit 1) & PID=$!; wait; echo "wait: $?"
   [1] 1192215
   [1]  + exit 1     ( sleep 1; exit 1; )
   CHLD args= exit=0
   wait: pid 1192215 is not a child of this shell
   wait: 0
   
   % setopt nomonitor
   % trap 'x=$?; echo "CHLD args=$* exit=$x"; wait $PID ' CHLD; (sleep 1; exit 1) & PID=$!; wait; echo "wait: $?"
   CHLD args= exit=0
   wait: pid 1192528 is not a child of this shell
   wait: 0
   
   % trap - SIGCHLD
   % TRAPCHLD()  { echo "CHLD args=$*"; wait $PID }; (sleep 1; exit 1) & PID=$!; wait; echo "wait: $?"
   CHLD args=17
   TRAPCHLD:wait: pid 1192701 is not a child of this shell
   wait: 0

Collecting background jobs' exit status is discussed in the manual,
under the POSIX_JOBS option:

   In  previous  versions  of the shell, it was necessary to enable
   POSIX_JOBS in order for the builtin command wait to  return  the
   status of  background jobs that had already exited.  This is no
   longer the case.

Setting/unsetting POSIX_JOBS does not make any difference.

Anyways, zargs is not doing a stellar job currently with collecting
exit statuses from commands ran in parallel:

   % zsh -f
   % autoload zargs
   % zargs -n 4 -P 2 -- 1 0 -- zsh -c 'sleep $1 ; exit $1 ' -; echo $?
   0
   % zargs -n 4 -P 2 -- 0 1 -- zsh -c 'sleep $1 ; exit $1 ' -; echo $?
   123
   % zargs -n 2 -P 2 -- 1 0 -- eval '(){ sleep $1 ; return $1 }' ; echo $?
   0
   % zargs -n 2 -P 2 -- 0 1 -- eval '(){ sleep $1 ; return $1 }' ; echo $? 
   123

Phil.