Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion shared/yeast-macros/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -9,4 +9,4 @@ proc-macro = true
[dependencies]
proc-macro2 = "1.0"
quote = "1.0"
syn = "3.0"
syn = { version = "3.0", features = ["full"] }
134 changes: 134 additions & 0 deletions shared/yeast-macros/src/ast.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,134 @@
use proc_macro2::{Ident, Literal};

/// A complete rewrite rule such as `(input) where guard => (output)`.
pub(crate) struct Rule {
/// Input pattern before `where` or `=>`.
pub(crate) pattern: Pattern,
/// Optional Rust guard expression after `where`.
pub(crate) guard: Option<syn::Expr>,
/// Template or annotated Rust replacement after `=>`.
pub(crate) replacement: Replacement,
}

pub(crate) enum Replacement {
/// One or more output templates such as `(call ...) {extra_nodes}`.
Templates(Vec<Template>),
/// Annotated Rust body such as `expr? { ... }`.
Rust {
/// Retained in the rule AST for validation and analysis; runtime
/// lowering only needs the Rust body.
#[allow(dead_code)]
annotation: ReturnAnnotation,
body: syn::Block,
},
}

#[allow(dead_code)]
pub(crate) struct ReturnAnnotation {
/// Declared output kind before the Rust body.
pub(crate) kind: Ident,
/// Declared output cardinality (`kind`, `kind?`, or `kind*`).
pub(crate) cardinality: Cardinality,
}

#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub(crate) struct Cardinality {
pub(crate) multiple: bool,
pub(crate) required: bool,
}

impl Cardinality {
pub(crate) const SINGLE: Self = Self {
multiple: false,
required: true,
};
pub(crate) const OPTIONAL: Self = Self {
multiple: false,
required: false,
};
pub(crate) const ZERO_OR_MORE: Self = Self {
multiple: true,
required: false,
};
pub(crate) const ONE_OR_MORE: Self = Self {
multiple: true,
required: true,
};
}

/// An input pattern such as `(call method: (identifier) @name)`.
pub(crate) enum Pattern {
/// `_` or `(_)`, distinguished by whether unnamed nodes may match.
Any { match_unnamed: bool },
/// A named node and the pattern matched against each field.
Node {
kind: String,
fields: Vec<(String, Pattern)>,
},
/// An unnamed token pattern such as `"+"`.
Unnamed(String),
/// A captured pattern such as `(identifier) @name` or `_ @@raw`.
Capture {
/// Capture identifier after `@` or `@@`.
name: Ident,
/// Whether `@@` keeps the captured input node untranslated.
raw: bool,
pattern: Box<Pattern>,
cardinality: Cardinality,

@asgerf asgerf Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A bit surprising to find cardinality baked into Capture. Could it not be represented as a nested Repeated node?

@tausbn tausbn Oct 19, 2026 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that it's a slightly curious choice. Ultimately, it all goes back to how captures currently work. If you write something like

(foo bar: (item) @items)*

then we currently treat items as a capture that can contain multiple values, even though it locally only captures a single value, because this capture happens inside a repeated context. Getting the cardinality right is important, because it determines the type of the lowered variable -- Id or Vec<Id>. So at some point we need to look at the context of the @items capture, and determine that it is inside a Repeated node and therefore a Vec. It made lowering a bit cleaner to have this captured already in the AST (otherwise we would need to pass this information around in a context during lowering). Under these semantics, using a nested Repeated node would imply that we're looking for internal repetition, which is not the intent.

Honestly, I'm beginning to wonder if we even need to support this weird pattern. My impression is that almost all "list-like" captures are of the form (foo)* @foos. I would have to double-check swift.rs (and my Python experiment) to see if this is the case.

I think you're to call this out. I will look into whether we actually use this, and see how things look if we clean up the AST so we don't do this kind of unintuitive capturing.

(In the short term, I'll go ahead with the merge.)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, having looked into it, it does seem to be a pattern we use quite often. For instance, something like

(genericArgument argument: @args)*

or

(importPathComponent name: @@parts)*

We also have a lot of uses with optional captures, like

(initializerClause value: @@val)?

or

(inheritanceClause inheritedTypes: (inheritedType type: @bound))?

So I think this behaviour is here to stay. At the very least, I don't see any other ways of expressing these captures that are as convenient as these.

},
/// An ordered sequence of patterns within a field.
Sequence(Vec<Pattern>),
/// A repeated or optional pattern.
Repeated {
pattern: Box<Pattern>,
cardinality: Cardinality,
},
}

impl Pattern {
pub(crate) fn matches_single_node(&self) -> bool {
match self {
Pattern::Any { .. } | Pattern::Node { .. } | Pattern::Unnamed(_) => true,
Pattern::Capture { cardinality, .. } => *cardinality == Cardinality::SINGLE,
Pattern::Sequence(_) | Pattern::Repeated { .. } => false,
}
}
}

pub(crate) enum Template {
/// A parenthesized node template such as `(call method: {name})`.
Node(Node),
/// An embedded Rust block such as `{args}`.
Splice(syn::Block),
}

pub(crate) struct Node {
/// The node kind at the start of the template, such as `call` in `(call)`.
pub(crate) kind: Ident,
/// Optional leaf content such as `"foo"` or `#{name}` in
/// `(identifier "foo")` or `(identifier #{name})`.
pub(crate) content: Option<Content>,
/// Named fields such as `method: {name}` in `(call method: {name})`.
pub(crate) fields: Vec<Field>,
}

pub(crate) enum Content {
/// Static leaf content such as `"foo"` in `(identifier "foo")`.
Static(Literal),
/// Computed leaf content such as `#{name}` in `(identifier #{name})`.
Computed(syn::Block),
}

pub(crate) enum Field {
/// A named field such as `argument: {args}` or
/// `label: (identifier #{name})?`.
Named {
/// The field name before `:`, such as `argument`.
name: Ident,
/// The node or Rust splice after `:`, such as `(identifier "x")` or
/// `{args}`.
value: Template,
/// Whether the field value has a trailing `?`.
optional: bool,
},
}
23 changes: 20 additions & 3 deletions shared/yeast-macros/src/lib.rs
Original file line number Diff line number Diff line change
@@ -1,9 +1,13 @@
use proc_macro::TokenStream;
use proc_macro2::TokenStream as TokenStream2;

mod ast;
mod lower;
mod parse;
mod rule_parse;
mod template_parse;

/// Proc macro for constructing a `QueryNode` from a tree-sitter-inspired pattern.
/// Proc macro for constructing a tree-sitter-inspired `yeast::query::QueryNode`.
///
/// # Syntax
///
Expand All @@ -19,11 +23,24 @@ mod parse;
/// (pattern) @capture - capture the matched node
/// "literal" @capture - capture an unnamed token
/// _ @capture - capture any node
/// (pattern)* @capture - capture each repeated match
/// (pattern)? - zero or one
/// (kind)* @capture - capture each repeated named-node match
/// "literal"* @capture - capture each repeated unnamed-token match
/// (kind)? - zero or one named node
/// ```
///
/// Named fields and bare child patterns may be intermixed in any order.
///
/// A parenthesized query whose first element is a node kind is always parsed as
/// that node pattern. Otherwise, two or more sibling patterns form a sequence.
/// A sequence may be repeated, but cannot itself be captured because it does
/// not represent one node:
///
/// ```text
/// ((identifier) @items (integer) @items)* // valid: explicit node captures
/// ((identifier) (integer))* @items // invalid: sequence capture
/// ("+" ",")* // valid: repeated token sequence
/// (_ ",")* // valid: repeated wildcard/token sequence
/// ```
#[proc_macro]
pub fn query(input: TokenStream) -> TokenStream {
let input2: TokenStream2 = input.into();
Expand Down
Loading
Loading