Skip to content

Stop adding a trailing line feed to clip-chomped block scalars when the stream has no final break #52

Description

Context and request

Observed behavior: When a YAML stream ends without a final line break, a clip-chomped block scalar (the default, no - or + indicator) gains a trailing line feed that the source never contained.

"k: |`n  a`n  b" | ConvertFrom-Yaml   # -> "a`nb`n"
"k: >`n  a`n  b" | ConvertFrom-Yaml   # -> "a b`n"

Expected behavior: No trailing line feed, because there is no final break to clip.

"k: |`n  a`n  b" | ConvertFrom-Yaml   # -> "a`nb"
"k: >`n  a`n  b" | ConvertFrom-Yaml   # -> "a b"

YAML 1.2.2 defines b-chomped-last(clip) ::= b-as-line-feed | <end-of-input>. At end of input that production yields nothing, so clip chomping has no break to keep. Strip (-) and keep (+) chomping are already correct for the same inputs, and every case where the stream does end with a line break is correct.

Reproduction:

Import-Module Yaml
("k: |`n  a`n  b" | ConvertFrom-Yaml).k -replace "`n", '\n'   # actual: a\nb\n   expected: a\nb
("k: >`n  a`n  b" | ConvertFrom-Yaml).k -replace "`n", '\n'   # actual: a b\n   expected: a b
("k: |2`n  a`n  b" | ConvertFrom-Yaml).k -replace "`n", '\n'  # actual: a\nb\n   expected: a\nb
("k: |-`n  a`n  b" | ConvertFrom-Yaml).k -replace "`n", '\n'  # correct: a\nb
("k: |+`n  a`n  b" | ConvertFrom-Yaml).k -replace "`n", '\n'  # correct: a\nb

Environment: PowerShell 7.6 Core, Yaml at release/v1.0.0. Platform independent — the divergence is in Read-YamlBlockScalar.ps1, not in file or encoding handling.

Regression: No. This has been present for the whole life of the block-scalar reader and is not introduced by any recent change.

Workaround: Ensure the YAML stream ends with a line break before parsing. Import-Yaml reads whole files, which normally end with a break, so this mostly affects strings built in memory.

Acceptance criteria:

  • Clip-chomped block scalars produce no trailing line feed when the stream ends without a final break, for both literal and folded styles and with or without an explicit indentation indicator.
  • Strip and keep chomping behavior is unchanged.
  • All streams that do end with a line break are unchanged.
  • The 402-input yaml-test-suite conformance accounting in tests/Conformance.Tests.ps1 stays at 400 syntax passes, 0 failures, and the 2 documented duplicate-key policy differences.

Technical decisions

The root cause is bounded to the chomping tail of src/functions/private/Read-YamlBlockScalar.ps1. The reader records a per-line hasLineBreak flag, and the clip branch appends "n"whenever any content line exists, without consulting whether the final content line actually had a break. The correction is to make the clip branch honorhasLineBreak` for the last content line rather than assuming one.

This is a value-level correctness fix only; no public parameter, output type, or error contract changes.

Implementation plan

  1. Add a failing regression test in tests/ConvertFrom-Yaml.Tests.ps1 covering literal, folded, and explicit-indent clip cases with no final break, plus strip and keep controls.
  2. Make the clip branch respect the last content line's hasLineBreak.
  3. Run tests/ConvertFrom-Yaml.Tests.ps1, tests/Format-Yaml.Tests.ps1, tests/Specification.Tests.ps1, and tests/Conformance.Tests.ps1.

Notes

Found while validating #47 with a differential harness against a reference parser. Two other block/flow scalar defects found in the same sweep were corrected inside that pull request; this one is deliberately deferred because it is pre-existing and outside that release's scope.

Activity

  1. MariusStorhaug commented on Aug 2, 2026

    @MariusStorhaug
    MemberAuthor

    Closing as not a defect. The premise was wrong, and the current behavior is the conformant one.

    What I got wrong

    I raised this from a differential run against PyYAML's BaseLoader. PyYAML returns "a\nb" for k: | + a + b with no final line break, while this module returns "a\nb\n". I treated PyYAML as authoritative. It is not — on this specific point it disagrees with the official yaml-test-suite, and the suite is the conformance source this module is validated against.

    Evidence

    Three cases in the pinned yaml-test-suite data release have an in.yaml whose bytes do not end with a line break, and two of them pin the behavior directly:

    Case Input (escaped) Official expected value PyYAML
    L24T/01 "Trailing line of spaces" foo: |\n x\n "x\n \n" (in.json, and test.event records =VAL |x\n \n) "x\n "
    JEF9/02 - |+\n "\n" "\n"
    96NN/01 foo: |-\n \tbar "\tbar" "\tbar"

    L24T/01 is decisive: the reference implementation emits the b-chomped-last line feed at end of input rather than suppressing it. Prototyping the proposed change made the module match PyYAML and fail L24T/01 across three separate accounting assertions in tests/Conformance.Tests.ps1 — parser/event comparisons dropped 308 to 307, JSON construction 277 to 276, and official emit.yaml fixtures 55 to 54.

    YAML 1.2.2 production 165 writes the alternation as b-chomped-last(clip) ::= b-as-line-feed | <end-of-input> without stating what <end-of-input> produces. The suite resolves that ambiguity in favor of still producing the feed, which is also what this module does.

    Outcome

    No change. tests/Conformance.Tests.ps1 already pins L24T/01 and JEF9/02 through the corpus accounting assertions, so a future regression in this area is caught.

    The two genuine scalar defects found in the same sweep were real and were fixed in #47 — folded keep chomping (>+) dropping the b-chomped-last feed, and empty lines after an escaped break folding to a space instead of line feeds. Both were confirmed against the YAML 1.2.2 grammar as well as PyYAML, and neither is contradicted by the official corpus.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions