Re: "make check" suggestion

[Felipe Contreras <felipe.contreras@xxxxxxxxx>]

On Sat, Nov 28, 2020 at 12:53 AM Bart Schaefer
<schaefer@xxxxxxxxxxxxxxxx> wrote:
>
> I haven't messed around with ztst.zsh for some years, so I'm not
> familiar with how the newer features like intentionally failed tests
> are handled.  A suggestion perhaps for someone who has worked on it
> more recently:
>
> It might be nice to have a mode in which ALL the tests in each file
> are run, rather than abort the whole test file when one of them fails
> (or unexpectedly succeeds).

Coming from Git, I have to say their testing framework is the best
I've ever seen.

It is developed in shell. There may be some bashisms, but they try to
avoid them. Last time I tried it in zsh, it didn't work correctly, but
shouldn't be too hard to fix (and maybe it was a problem in zsh), and
I'm sure they would accept the patches.

The project that tried to extirpate the test system from git.git is
called sharness [1]. I've used it in all my projects, and it works
like a charm.

In this test system all you have to do is pass -i to the test, to stop
at a failure.

For example:

  ./completion-zsh.t -i

Additionally, the system supports Perl's Test Anything Protocol (TAP) [2].

So you can do:

  prove completion-zsh.t :: -i

That's how I developed the git-completion zsh specific tests, which
work great, and are integrated into Travis-CI [3].

If anyone is interested, I could give it a try to port some zsh test
to sharness.

In my personal opinion the testing system of zsh is far from ideal.
But that's just my opinion.

Cheers.

[1] https://github.com/chriscool/sharness
[2] https://en.wikipedia.org/wiki/Test_Anything_Protocol
[3] https://travis-ci.org/github/felipec/git-completion

-- 
Felipe Contreras