Skip to content

Crashing in subtests does not fail the subtest #1075

Description

@mediaalpha-blackmore

Given the following, to simulate a subroutine or external module call that may die:

use Test::More tests=>1;
subtest 'B'=>sub {ok(1);...;ok(1)};

The output from prove -v is

1..1
# Subtest: B
    ok 1
    1..1
ok 1 - B
Unimplemented at ./crashdemo.t line 10.
# Looks like your test exited with 255 just after 1.

The desired behavior would be for subtest B to not report okay:

1..1
# Subtest: B
    ok 1
    # Subtest died: Unimplemented at ./crashdemo.t line 10.
    1..1
    # Looks like you failed 1 test of 1.
not ok 1 - B

#   Failed test 'B'
#   at ./crashdemo.t line 10.
Unimplemented at ./crashdemo.t line 10.
# Looks like your test exited with 255 just after 1.

This may be possible around https://github.com/Test-More/test-more/blob/master/lib/Test/Builder.pm#L407 by setting $st_hub->{_passing} = 0; and/or $st_hub->{failed} = 1; in the crashing case, but I don't know enough about the ordering or the steps to submit a PR.

Note that proper test plans can produce the desired output, in cases where an accurate plan count is possible:

subtest 'B'=>sub {plan tests=>2; ok(1);...;ok(1)};

# Subtest: B
    1..2
    ok 1
    # Looks like you planned 2 tests but ran 1.
not ok 1 - B

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions