Skip to content

Fix cpantesters FAILs: gen.t perl mismatch + class-method serializer crash - #1842

Merged
cromedome merged 5 commits into
mainfrom
fix/cpantesters-gen-t-and-serializer-class-method
Sep 28, 2026
Merged

cromedome merged 5 commits into
mainfrom
fix/cpantesters-gen-t-and-serializer-class-method

Conversation

@bigpresh

@bigpresh bigpresh commented Sep 26, 2026 •

Copy link
Copy Markdown
Member

Two fixes for v2.2.1 cpantesters failures

1. t/e2e/cli/gen.t: use TAP::Harness instead of shelling out to prove

The generated-app subtest ran system(q{prove}, q{-lr}, q{t}), which lets PATH decide which perl binary executes prove. On smokers with multiple perls installed, the one on PATH can differ from the one running the test suite, producing either:

Replace the system() call with TAP::Harness, which invokes child perls via $^X (the current interpreter) by construction. This eliminates the entire class of PATH/shell mismatch bugs rather than guarding against each one. TAP::Harness is core since 5.10.1; Dancer2 requires 5.12.0.

This test was newly added in v2.2.0, which is why this failure is new.

2. Role::Serializer + JSON/YAML helpers: fix class-method crash (GH #1837)

from_json/from_yaml/to_json/to_yaml called deserialize/serialize as class methods (__PACKAGE__->deserialize(@_)). When the underlying parser dies (e.g. on malformed input), the around deserialize modifier in Role::Serializer tried to call $self->log_cb — but $self is a package-name string on the class-method path, not an object, so Class::XSAccessor raises:

Class::XSAccessor: invalid instance method invocant: no hash ref supplied

This completely masks the real parse error. The sibling around serialize modifier already guards every $self-> call with blessed $self && — around deserialize simply missed the same treatment.

Two changes:

  1. Role::Serializer: guard the log_cb call with blessed $self, matching the pattern already used in around serialize.
  2. JSON/YAML helpers: construct a default instance (__PACKAGE__->new(log_cb => sub {})) so that deserialize/serialize always receive a proper object, allowing errors to be logged rather than swallowed.

Note: this bug does not cause any v2.2.1 test failures on cpantesters (the yaml.t test already works around it by calling on an instance), but it bites any application code that calls from_json($bad_input) / from_yaml($bad_input) directly.

Fixes GH #1837.

@bigpreshkt

Copy link
Copy Markdown

The execution of the generated app's tests via a system() call to prove still doesn't sit right with me. I'm going to look at using TAP::Harness instead.

from_json/from_yaml/to_json/to_yaml called deserialize/serialize as
class methods (__PACKAGE__->deserialize(@_)). When YAML::Load or
JSON::decode dies (e.g. on malformed input), the around deserialize
modifier in Role::Serializer tried to call $self->log_cb -- but
$self is a package-name string on the class-method path, not an
object, so Class::XSAccessor raises:

  invalid instance method invocant: no hash ref supplied

This completely masks the real parse error.

Two changes:

1. Role::Serializer: guard the log_cb call with `blessed $self`,
   matching the pattern already used in the around serialize modifier.

2. JSON/YAML helpers: construct a default instance
   (__PACKAGE__->new(log_cb => sub {})) so that deserialize/serialize
   always receive a proper object, allowing errors to be logged
   rather than swallowed.

Fixes GH #1837.
The generated-app subtest ran `prove -lr t` via system(), which lets
PATH decide which perl binary executes prove. On smokers with multiple
perls installed, the one on PATH can differ from the one running the
test suite, producing either:

  - Can't locate Module::Runtime.pm in @inc  (wrong @inc)
  - XS handshake key mismatch               (wrong perl entirely)

Replace the system() call with TAP::Harness, which invokes child perls
via $^X (the current interpreter) by construction. This eliminates the
entire class of PATH/shell mismatch bugs rather than guarding against
each one. TAP::Harness is core since 5.10.1; Dancer2 requires 5.12.0.

Fixes CPAN Testers FAIL reports for v2.2.1 (e.g. 24b36498, 0893fc30).
@bigpresh
bigpresh force-pushed the fix/cpantesters-gen-t-and-serializer-class-method branch from 8cd11e2 to 2631881 Compare September 27, 2026 11:16
@bigpresh
bigpresh requested a review from veryrusty September 27, 2026 11:18
@manwar

manwar commented Sep 27, 2026

Copy link
Copy Markdown
Contributor

@bigpresh Great work and thanks for prompt action.

Comment thread lib/Dancer2/Serializer/JSON.pm Outdated

# helpers
sub from_json { __PACKAGE__->deserialize(@_) }
sub from_json { __PACKAGE__->new( log_cb => sub {} )->deserialize(@_) }

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to supply log_cb rather than use the default from Role::Serializer ( which is sub {1} ) ?

Review feedback on #1842: the log_cb passed when from_json/to_json/
from_yaml/to_yaml construct an engine was `sub {}`, which is exactly
what Role::Serializer already defaults to (`sub {1}`) -- so passing it
at all was redundant. Dropping it, though, exposed something the
previous commit changed by accident.

Those helpers used to call deserialize/serialize as class methods, and
Dancer2::Serializer::JSON::_invalid_utf8 branches on blessed($self):
with an object it logs through log_cb, without one it warns directly.
Constructing an object therefore routed the "Invalid UTF-8 in JSON
data; leaving bytes unchanged" notice into a logger that threw it away,
so to_json stopped warning about invalid UTF-8 -- a silent change in
behaviour, not something anyone chose.

Give the helpers a log_cb that warns instead. That restores the UTF-8
notice, and as a bonus it makes the diagnostic GH #1837 complained about
losing actually visible: from_json('{not json') now returns undef *and*
warns with the real parse error, rather than returning a bare undef.

t/serializer_helpers.t covers both halves: the real error is reported
for from_json/from_yaml, the UTF-8 notice still reaches STDERR, round
trips stay quiet, and a bare class-method deserialize fails silently
without raising Class::XSAccessor.
Reported against 2.2.1 while updating the Debian package (GH #1843).
The test passes -x to every `dancer2 gen` invocation to skip the
"is there a newer Dancer2 on CPAN?" check, which needs the network.

Debian patches that check out of Dancer2::CLI::Gen as a privacy
measure (debian/patches/no-phone-home.patch), and the option whose
only job is to switch it off goes with it. On a build from those
sources, -x aborts the generator with "Unknown option: x" before it
writes a single file, and all five subtests then fail, pointing at the
scaffold rather than at the flag.

Ask the command what it accepts rather than assuming: probe `gen
--help` once for -x (matching either spelling of the long form, since
Osprey's underscore-to-dash mapping varies by version) and pass it only
where it exists. Where the option is gone, so is the network access it
exists to suppress, so nothing is lost.

Also drop the version-check warning from STDERR when -x was
unavailable, so a build that kept the check but lost the option does
not fail "says nothing on STDERR" just because the build daemon has no
network. Only those two lines go, and only on that path -- builds with
-x keep the strict assertion.

Verified both ways: passes as-is, and passes with Debian's
no-phone-home.patch applied to the tree.
@bigpresh

Copy link
Copy Markdown
Member Author

@veryrusty good catch, and it turned out to be a more interesting question than it looked.

You're right that log_cb => sub {} was pointless: Dancer2::Core::Role::Serializer already defaults it to sub { sub {1} }, so passing a no-op did nothing the default wasn't doing. But taking it out made me check what the default means for these helpers, and that uncovered something my earlier commit changed by accident.

Dancer2::Serializer::JSON::_invalid_utf8 deliberately branches on whether it has an object:

if ( blessed($self) ) {
    $self->log_cb->( warning => "$msg; leaving bytes unchanged" );
} else {
    warn "$msg; leaving bytes unchanged\n";
}

So by switching the helpers from __PACKAGE__->serialize(...) to __PACKAGE__->new(...)->serialize(...), I moved that notice out of the warn branch and into a logger that discards it:

to_json({ bad => "\xFF" }) result
main (class method, unblessed) warns "Invalid UTF-8 in JSON data; leaving bytes unchanged"
this branch as it stood (->new, log_cb => sub {}) silent
->new with no log_cb at all (role default sub {1}) silent

Nothing in the suite covered that path for the helpers, which is why it went unnoticed. Two ways to settle it: accept the silence and treat _invalid_utf8's warn branch as dead code for these functions, or give the helpers a log_cb that actually reports. A change in behaviour like that should be a conscious decision rather than a quiet side effect, so I've gone with the second — pushed as b2e28ae:

sub _warn_log_cb {
    my ( $level, $message ) = @_;
    $message =~ s/\s+\z//;
    warn "$message\n";
    return 1;
}

sub from_json { __PACKAGE__->new( log_cb => \&_warn_log_cb )->deserialize(@_) }

That restores the UTF-8 notice, and as a bonus it delivers the thing #1837 actually complained about — the real parse error no longer disappears:

from_json('{not json')
  before:  dies with "Class::XSAccessor: invalid instance method invocant"
  as pushed earlier:  returns undef, silently
  now:  returns undef and warns
        "Failed to deserialize content: '"' expected, at character offset 1 (before "not json")"

New t/serializer_helpers.t pins both halves: the real error is reported for from_json/from_yaml, the UTF-8 notice still reaches STDERR, successful round trips stay quiet, and a bare Class->deserialize still fails silently without raising Class::XSAccessor (that last one covers the blessed $self guard in the role, which is what protects callers who don't go through these helpers). I checked the tests fail if either fix is reverted — reverting the role guard fails the class-method subtest, restoring sub {} fails three of the others.

The ->new does cost ~3× per call on small payloads (90k/s vs 300k/s class-method in a quick Benchmark run), which seems a fair price for one call per request.

Also pushed on the same branch: bbefda3 fixes the gen.t failures @gregoa reported in #1843. Different cause from the perl-mismatch ones — Debian patches the CPAN version check out of Dancer2::CLI::Gen for privacy (no-phone-home.patch) and the -x option that skips it goes with it, so the -x this test passes unconditionally aborts the generator with Unknown option: x before it writes anything. It now probes gen --help once and only passes -x where it exists; where the option is gone, so is the network access it exists to suppress. Verified with that patch applied to the tree.

Review feedback: s/\s+\z// reads as "strip whitespace" and does not
announce that newlines are what it is there for, where s/\r?\n\z// is
recognisable at a glance. The two are equivalent for every message the
engines actually produce -- each carries at most one trailing newline --
so narrow it to the newline and add the comment explaining why the trim
is there at all: the line ending varies by parser backend, and warn
appends "at <file> line <n>" to a message that has none.

Note \z rather than $: without /m, $ also matches before a final
newline, which is exactly the ambiguity worth avoiding here.

@veryrusty veryrusty left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for chasing down those log_cb's @bigpresh ! 👍 looks good.

@cromedome
cromedome merged commit f953fe8 into main Sep 28, 2026
18 checks passed
@cromedome

Copy link
Copy Markdown
Contributor

👍 Thanks, @bigpresh! Merged.

@gregoa

gregoa commented Sep 29, 2026

Copy link
Copy Markdown
Contributor

Thanks alot for the change in bbefda3, @bigpresh!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants