Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 63 additions & 0 deletions docs/archive/2026-10-18-item-first-line-break/plan.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
# An item whose first line reads as a break with its bullet

## Why

The serializer wrote a bullet item's content on its marker's line even when
the two together read as one thematic break: three nested empty items as
`- - - `, and an item holding a paragraph `--` as `- --`. Both were written
with no diagnostic and read back as a thematic break. Settle's random key
test found them.

- Scope: the canonical writer and the source writer's edited items; the
validation rule that already depends on where the serializer starts an
item's content.

## What changes

### Validation
- `validation` › Shapes that cannot be written: modified — a list that the
serializer starts on a line after its bullet now includes one whose first
line reads as a thematic break with a `-` or `*` bullet, such as a
paragraph `--` or a nested empty item two levels down

### Outside the specs
- The serializer starts such an item's content on the line after its
bullet (`-\n --`), as it already did for a thematic break of the
bullet's char; parsed documents such as `-\n --` and `*\n **` now read
back exactly.

Specs:
- `validation` (modified)

## Out of scope
- A paragraph whose text reads as syntax wherever it is written (`___`,
`-`, `* *`) — text the builder answers for.
- Validating against the caller's `SerializeOptions.bullet` — validation
judges default options, as before this change.

## Design

`starts_below_bullet` decides from the item's first written line: padding,
or, after a `-` or `*` bullet, a line of only that char, spaces, and tabs
with two or more of the char. `write_item` applies it to the written
content; validation's `first_item_starts_below_its_bullet` applies it to
`item_content_first_line`, which follows the first-child chain of nested
lists and writes at most one leaf, so validation and the writer agree and
validation stays linear. The source writer renders an edited item
canonically when its kept bullet and new first line read as a break. The
first line ends at `\n` or `\r`.

### Decisions
- Move the content below the bullet — because that is the writer's existing
answer for a thematic break of the bullet's char. Turned down: refusing
the shape, which `-\n --` writes faithfully.

## Tasks

### 1. Writer
- [x] 1.1 The canonical writer starts such items below the bullet; verified by `serialize_regressions` › `an_item_whose_first_line_reads_as_a_break_with_its_bullet_starts_below_it`.
- [x] 1.2 An edited item written from the source does the same; verified by `serialize_source` › `an_edited_item_that_would_read_as_a_break_starts_below_its_bullet`.
- [x] 1.3 A tight item refuses such a list after a paragraph; verified by `validate_regressions` › `a_block_after_a_paragraph_in_a_tight_item_interrupts_it`.

### 2. Integration
- [x] 2.1 Every CI command in `CLAUDE.md` › Commands passes locally.
6 changes: 4 additions & 2 deletions docs/specs/validation.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,8 +42,10 @@ label:
`Frontmatter`, a `Heading` written as a setext heading, an ordered
`List` whose first item's number is not 1, a `List` whose first item has
no blocks or that the serializer starts on a line after its bullet (a
thematic break of the bullet's char, or content opening with one to
three spaces or a tab), or an `HtmlBlock` whose first line opens no HTML
first line that with a `-` or `*` bullet reads as a thematic break, such
as a thematic break of the bullet's char, a paragraph `--`, or a nested
empty item two levels down, or content opening with one to three spaces
or a tab), or an `HtmlBlock` whose first line opens no HTML
block or opens one that only a blank line ends (a lone tag such as
`<span>`);
- after a `Definition`, those blocks other than a `Paragraph`, a
Expand Down
134 changes: 89 additions & 45 deletions src/serialize.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1194,40 +1194,94 @@ fn write_item(
format!("{marker_char} ")
};
let inner = write_item_blocks(&item.children, options, list.tight, item.checked)?;
let break_after_bullet = breaks_after_bullet(list, item, marker_char);
let opens_with_whitespace = opens_with_padding(&inner);
Ok(if break_after_bullet || opens_with_whitespace {
format!(
"{}\n{}",
marker.trim_end(),
prefix_container_lines(&inner, &" ".repeat(marker.len()))
)
Ok(
if starts_below_bullet(list, item, marker_char, first_line(&inner)) {
format!(
"{}\n{}",
marker.trim_end(),
prefix_container_lines(&inner, &" ".repeat(marker.len()))
)
} else {
format!("{marker}{}", indent_after_first_line(&inner, marker.len()))
},
)
}

/// Whether `item` of `list`, written with `marker_char`, starts its content
/// on the line after its marker, `first_line` being the first line its
/// blocks are written as: where that line would read as padding after the
/// marker, or, after a bullet, as one thematic break with it (`- --`, or
/// the `- - -` of three nested empty items).
fn starts_below_bullet(list: &List, item: &ListItem, marker_char: char, first_line: &str) -> bool {
opens_with_padding(first_line)
|| (!list.ordered
&& item.checked.is_none()
&& reads_as_thematic_break(marker_char, first_line))
}

/// Whether `marker_char`, a space, and `rest` read as a thematic break of
/// that char: three or more `-` or `*`, with only spaces and tabs between.
pub(super) fn reads_as_thematic_break(marker_char: char, rest: &str) -> bool {
matches!(marker_char, '-' | '*')
&& rest
.chars()
.all(|char| matches!(char, ' ' | '\t') || char == marker_char)
&& rest.chars().filter(|char| *char == marker_char).count() >= 2
}

/// The first line the serializer writes `item` of `list` with, its marker
/// `marker_char` and `options`' text included.
fn item_first_line(
list: &List,
item: &ListItem,
marker_char: char,
options: &SerializeOptions,
) -> String {
let marker = if list.ordered {
format!("{}{marker_char}", item.number.unwrap_or(1))
} else {
format!("{marker}{}", indent_after_first_line(&inner, marker.len()))
})
String::from(marker_char)
};
let content = item_content_first_line(item, options);
if starts_below_bullet(list, item, marker_char, &content) {
marker
} else {
format!("{marker} {content}")
}
}

fn thematic_break_char(marker: ThematicBreakMarker) -> char {
match marker {
ThematicBreakMarker::Dash => '-',
ThematicBreakMarker::Asterisk => '*',
ThematicBreakMarker::Underscore => '_',
/// The first line of `item`'s blocks as the serializer writes them with
/// `options`: a nested list's first item is followed into, and a leaf block
/// is written to tell.
fn item_content_first_line(item: &ListItem, options: &SerializeOptions) -> String {
if item.checked.is_some() {
return String::from("[");
}
match item.children.first() {
None => String::new(),
Some(Block::List(nested)) => match nested.children.first() {
Some(first) => {
item_first_line(nested, first, list_marker(nested, options, None), options)
}
None => String::new(),
},
Some(
block @ (Block::Paragraph(_)
| Block::Heading(_)
| Block::HtmlBlock(_)
| Block::CodeBlock(_)
| Block::ThematicBreak(_)),
) => write_block(block, options)
.map(|written| String::from(first_line(&written)))
.unwrap_or_default(),
// Every other block opens with syntax of its own.
Some(_) => String::from(">"),
}
}

/// An item's blocks; a task item's, `task` holding whether it is checked,
/// open with its checkbox: at the start of the first paragraph after the
/// definitions the item starts with, or else of the item.
/// A thematic break of the bullet's own char right after the bullet would
/// read as one longer break, so it starts on the next line.
fn breaks_after_bullet(list: &List, item: &ListItem, marker_char: char) -> bool {
!list.ordered
&& item.checked.is_none()
&& matches!(
item.children.first(),
Some(Block::ThematicBreak(ThematicBreak { marker, .. }))
if thematic_break_char(*marker) == marker_char
)
/// `text` up to its first line ending.
fn first_line(text: &str) -> &str {
text.split(['\n', '\r']).next().unwrap_or_default()
}

/// Spaces or a tab opening an item's written content would read as padding
Expand Down Expand Up @@ -1255,30 +1309,20 @@ pub(crate) fn written_after_paragraph_interrupts(block: &Block) -> bool {

/// Whether, with default options, the serializer starts `list`'s first item
/// on a line after its bullet, so that the list, written after a paragraph's
/// line, cannot interrupt the paragraph. Only a leaf block can open with
/// whitespace, so only a leading leaf block is written to tell.
/// line, cannot interrupt the paragraph. Only the item's first line is
/// written to tell.
pub(crate) fn first_item_starts_below_its_bullet(list: &List) -> bool {
let Some(item) = list.children.first() else {
return false;
};
let options = SerializeOptions::default();
if breaks_after_bullet(list, item, list_marker(list, &options, None)) {
return true;
}
if item.checked.is_some() {
return false;
}
match item.children.first() {
Some(
block @ (Block::Paragraph(_)
| Block::Heading(_)
| Block::HtmlBlock(_)
| Block::CodeBlock(_)),
) => write_block(block, &options).is_ok_and(|written| opens_with_padding(&written)),
_ => false,
}
let content = item_content_first_line(item, &options);
starts_below_bullet(list, item, list_marker(list, &options, None), &content)
}

/// An item's blocks; a task item's, `task` holding whether it is checked,
/// open with its checkbox: at the start of the first paragraph after the
/// definitions the item starts with, or else of the item.
fn write_item_blocks(
blocks: &[Block],
options: &SerializeOptions,
Expand Down
14 changes: 12 additions & 2 deletions src/serialize/reuse.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1704,8 +1704,18 @@ impl Reuse<'_, '_> {
Children::Render => return Ok((Out::Render, false)),
};
let indent = " ".repeat(column);
// A break's own `-` or `*` after the marker would open a list.
let out = if matches!(first_block(&item.children), Some(Block::ThematicBreak(_))) {
// A break's own `-` or `*` after the marker would open a list, and a
// first line of only the bullet's char after it would read as a
// break with the bullet.
let bullet = marker.trim_start().chars().next().unwrap_or_default();
let out = if matches!(first_block(&item.children), Some(Block::ThematicBreak(_)))
|| (same_line
&& !list.ordered
&& item.checked.is_none()
&& lines
.first()
.is_some_and(|first| super::reads_as_thematic_break(bullet, &first.body)))
{
Out::Render
} else if same_line {
self.marked(&marker, lines, &indent)
Expand Down
62 changes: 62 additions & 0 deletions tests/serialize_regressions.rs
Original file line number Diff line number Diff line change
Expand Up @@ -699,6 +699,68 @@ mod recorded_spellings {
assert_eq!(rendered(&document(vec![list])), "*\n ***\n");
}

/// A first line of only the bullet's char after the bullet reads as one
/// thematic break with it, so the item starts on the line after its
/// marker: a paragraph `--`, or three nested empty items.
#[test]
fn an_item_whose_first_line_reads_as_a_break_with_its_bullet_starts_below_it() {
let list = |delimiter, children: Vec<Block>| {
Block::List(List {
meta: NodeMeta::default(),
ordered: false,
delimiter,
tight: true,
children: vec![ListItem {
meta: NodeMeta::default(),
checked: None,
number: None,
children,
}],
})
};
let dash = |children| list(ListDelimiter::Dash, children);
assert_eq!(
rendered(&document(vec![dash(vec![paragraph(vec![text("--")])])])),
"-\n --\n"
);
assert_eq!(
rendered(&document(vec![list(
ListDelimiter::Asterisk,
vec![paragraph(vec![text("* *")])]
)])),
"*\n * *\n"
);
let empty_items = dash(vec![dash(vec![dash(vec![])])]);
assert_eq!(rendered(&document(vec![empty_items])), "-\n - - \n");
// Two nested empty items and a `+` bullet read as items.
assert_eq!(
rendered(&document(vec![dash(vec![dash(vec![])])])),
"- - \n"
);
assert_eq!(
rendered(&document(vec![list(
ListDelimiter::Plus,
vec![paragraph(vec![text("++")])]
)])),
"+ ++\n"
);
// A first line ending in a carriage return reads the same.
let crlf_code = Block::CodeBlock(CodeBlock {
meta: NodeMeta::default(),
kind: CodeBlockKind::Indented,
info: None,
value: "--\r\nx\r\n".into(),
});
assert_eq!(
rendered(&document(vec![dash(vec![crlf_code])])),
"-\n --\r\n x\r\n"
);
for written in ["-\n --\n", "-\n - - \n"] {
let reparsed = parse(written).document.without_source();
assert_eq!(rendered(&reparsed), written);
}
}

#[test]
fn a_nested_bullet_is_written_as_a_list() {
let list = |children| {
Expand Down
24 changes: 24 additions & 0 deletions tests/serialize_source.rs
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,30 @@ mod unchanged_source_is_kept {
assert_eq!(document.to_markdown().unwrap(), "* x\n\n# T #");
}

/// An edited item whose kept bullet and new first line would read as one
/// thematic break starts its content on the line after the bullet.
// serialization: Unchanged source is kept
#[test]
fn an_edited_item_that_would_read_as_a_break_starts_below_its_bullet() {
let mut document = parse("- a\n").document;
let Block::List(list) = &mut document.children[0] else {
panic!("{:?}", document.children);
};
list.children[0].children[0] = paragraph("--");
assert_eq!(document.to_markdown().unwrap(), "-\n --\n");

let mut document = parse("- - a\n").document;
let Block::List(list) = &mut document.children[0] else {
panic!("{:?}", document.children);
};
let Block::List(inner) = &mut list.children[0].children[0] else {
panic!("{list:?}");
};
inner.children[0].children =
vec![Block::List(List::new([ListItem::new(Vec::<Block>::new())]))];
assert_eq!(document.to_markdown().unwrap(), "-\n - - \n");
}

#[test]
fn removed_blocks_leave_canonical_gaps() {
let mut document = parse("a\n\n\nb\n\n\nc").document;
Expand Down
2 changes: 2 additions & 0 deletions tests/validate_regressions.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1281,6 +1281,8 @@ mod written_shapes {
)],
),
list(true, vec![item(None, vec![html_block(" <v>")])]),
// A first line that reads as a break with the bullet: `- --`.
list(true, vec![item(None, vec![paragraph(vec![text("--")])])]),
] {
assert!(invalid(&after(true, bad.clone())), "{bad:?}");
assert!(valid(&after(false, bad)));
Expand Down
Loading