Re: Running N jobs from M all the time

["Bart Schaefer" <schaefer@xxxxxxxxxxxxxxxxxxxxxxx>]

On Oct 12,  6:38am, Szabolcs Szakacsits wrote:
} Subject: Running N jobs from M all the time
}
} My question would be there is an easy way for $SUBJECT?

If I'm understanding you correctly, you want to do something like

repeat 1000 do
    if (( number_of_jobs >= number_of_processors )); then
	wait $any_job
    fi
    command &
done

And the complaint is that "wait" can only wait for one specific job or
for all of them, not for "the next one that exits."

} I know this can be done with ps, grep, wait, etc but a bit painful and
} not elegant/modern I'd like to use a cleaner way.

You've just discovered another use for the coprocess.  The rest of this
is just standard parallel programming stuff.

function run_parallel {
    # Runs N copies of command at once until M copies have run.

    emulate -L zsh
    if (( $# < 3 )); then
	print "usage: run_parallel N M command [args ...]" >&2
	return 1
    fi

    integer N=$1 M=$2
    shift 2

    if ((M <= N)); then
	repeat $M do $==* & done
	wait
    else
	# Set up a loop to read and write one byte at a time.  This
	# semaphores between the parallel children and the parent zsh.  
	# It's important to pass exactly one byte here and not rely on
	# reading complete lines, lest a race condition develop.

	# Unfortunately, a race still can occur, it's just less likely.
	# Maybe SMP systems are better about multiple writers on one FD?

	coproc while ((--M > 0)) && read -u0k; do print -n .; done

	# Start the first N children

	repeat $N do
	    ((--M))
	    { $==* ; print -np . } &
	done

	# Wait for the semaphore from each child and start another as
	# soon as we get it, up to M children.

	while ((--M > 0)) && read -pk; do
	    { $==* ; print -np . } &
	done

	# Wait for the last N semaphores.

	repeat $N do read -pk; done
    fi

    return $?
}

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com