Skip to content

Detect missing buy token accounts for solvers - #4982

Merged
tilacog merged 17 commits into
mainfrom
tiago/be-305-detect-missing-buy-token-accounts-for-solvers
Sep 30, 2026
Merged

tilacog merged 17 commits into
mainfrom
tiago/be-305-detect-missing-buy-token-accounts-for-solvers

Conversation

@tilacog

@tilacog tilacog commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

Description

The driver now checks chain state at solve time to determine which user's buy_token_account needs to be created for an order to be filled, inform solvers about those and creates them during settlement.

Mind that only the owner's ATA for the buy token can possibly be created, so any other invalid destinations (that aren't already filtered by the autopilot's cut) will cause the pre-submission simulation to fail, same as before.

Changes

  • autopilot-svm: if a receivable account isn't found on chain, it fills w/ the user's ATA for the buy token
  • solana-driver:
  • driver -> solver DTO: new optional missingBuyTokenAccount boolean per order on the engine /solve request, omitted when false. The jupiter engine parses it and ignores it for now.
  • openapi: buyTokenAccount no longer says the settlement never creates it.
  • solana-testlib: token_account_json and multiple_accounts_json mock helpers, also used by the autopilot-svm tests.

Follow Up PRs

  • Try to convey order creation to /quote (I don't believe we have a Linear ticket for this yet)

@linear-code

linear-code Bot commented Sep 25, 2026

Copy link
Copy Markdown

BE-305

@github-actions

Copy link
Copy Markdown

Reminder: Please consider backward compatibility when modifying the API specification.
If breaking changes are unavoidable, ensure:

  • You explicitly pointed out breaking changes.
  • You communicate the changes to affected teams (at least Frontend team and SAFE team).
  • You provide proper versioning and migration mechanisms.

Caused by:

@tilacog

tilacog commented Sep 28, 2026

Copy link
Copy Markdown
Contributor Author

@claude review

@claude

claude Bot commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

Claude finished @tilacog's task in 3m 51s —— View job


PR Review

  • Read PR diff and gather context
  • Review autopilot-svm changes
  • Review solana-driver changes (auction, settlement, competition, DTOs)
  • Review tests and openapi
  • Post inline comments and summary

Reviewed the full diff, tracing the flow end-to-end: autopilot's receivable_orders cut → driver's solve-time flag_missing_buy_token_accounts → the missingBuyTokenAccount wire flag → settle-time re-check in accounts_to_create.

This is a clean, well-tested change. The re-check at settle time (rather than trusting the solve-time flag) correctly guards against an account closing in between, the Ata { owner, mint } refactor is a nice generalization of the payer-ATA logic, the on-the-wire skip_serializing_if + #[serde(default)] pairing keeps the DTO backward compatible, and the begin_ix_index/finalize_ix_index computation already accounts for the extra ATA-create instructions. The token-2022 boundaries are consistently documented. Test coverage is strong across all layers.

I left one low-severity inline note: autopilot drops orders whose buy ATA is a pre-funded system account, whereas the driver would treat that same account as creatable — a small asymmetry in the exact alignment this PR is trying to establish. Conservative and rare, so non-blocking.

The rent-not-priced-in behavior (jupiter ignoring the flag for now) is already acknowledged in the PR description and follow-ups, so I didn't flag it.
· tiago/be-305-detect-missing-buy-token-accounts-for-solvers

Comment on lines +68 to +71
let receivable = match accounts.get(&account) {
Some(found) => receivable_token_account(found, order.buy_token.0),
None => account == associated_token_address(order),
};

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Low severity / edge case — autopilot is slightly stricter than the driver here.

This keeps an order only when the buy account is fully absent (None) and equals the owner's ATA. But the driver's buy_token_account_missing (via TokenAccountState::NeedsCreation) also treats a pre-funded system account (system-owned, zero data) at the ATA address as creatable — create_associated_token_account_idempotent allocates over it.

In that case the account is present, so this branch takes Some(found) => receivable_token_account(...), which returns false (owner is the system program, not the SPL token program), and the order is dropped — even though the driver could have created the ATA and settled it.

It's the conservative direction (a dropped order, never a failed settlement), and the scenario is rare, so this isn't blocking. But since the whole point of the change is to align autopilot's cut with what the driver can create, it may be worth either handling the Some(system-owned, empty) case symmetrically or leaving a short note explaining the intentional gap.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can consider unifying this as a shared module in the future.

@tilacog
tilacog marked this pull request as ready for review September 28, 2026 15:31
@tilacog
tilacog requested a review from a team as a code owner September 28, 2026 15:31
@claude

claude Bot commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

Claude finished @tilacog's task in 2m 9s —— View job


PR Review

  • Read PR diff and gather context
  • Review autopilot-svm changes
  • Review solana-driver changes (auction, settlement, competition, DTOs)
  • Review tests and openapi
  • Post inline comments and summary

Re-reviewed the full diff end-to-end: autopilot's receivable_orders cut (keep an absent buy account when it's the owner's ATA) → driver's solve-time flag_missing_buy_token_accounts → the missingBuyTokenAccount wire flag → the settle-time re-check in accounts_to_create.

This is a clean, well-scoped change and I found no new blocking issues. Highlights that hold up under scrutiny:

  • Solve-time flag vs. settle-time truth is correctly separated. The wire flag exists only to let the engine price in rent; accounts_to_create re-derives from a fresh snapshot, so an account created or closed between solve and settle can't produce a wrong create/skip. create_associated_token_account_idempotent makes the create a safe no-op if it already exists.
  • buy_token_account_missing is symmetric with what can actually be created — it flags only when the account is missing and equals associated_token_address(owner, buy_token) under the classic SPL program, matching the program id used by both the address derivation and the create instruction.
  • begin_ix_index/finalize_ix_index are computed from instructions.len() after the ATA creates are pushed, so folding user buy ATAs into missing_atas shifts the reciprocal indices correctly. The Ata { owner, mint } refactor cleanly generalizes the old payer-only path, and the combined list is sorted + deduped.
  • DTO stays backward compatible — skip_serializing_if on the driver side + #[serde(default)] on the solver side, verified by the on-the-wire test.
  • solve() taking auction by value is safe — auction_id is read before the move.
  • token-2022 boundaries are consistently documented across the autopilot helper, the driver token helpers, and both DTOs.

Test coverage is strong across every layer (autopilot cut, driver flag, settlement re-check, DTO wire shape, and the api.rs integration path).

The one prior finding — autopilot dropping an order whose buy ATA is a pre-funded system account while the driver would treat it as creatable — was already discussed and acknowledged (unify as a shared module later), and the rent-not-priced-in / jupiter-ignores-the-flag behavior is covered in the PR description and follow-ups, so I'm not re-flagging either.

LGTM. 👍
· tiago/be-305-detect-missing-buy-token-accounts-for-solvers

@tilacog tilacog changed the title Tiago/be 305 detect missing buy token accounts for solvers Detect missing buy token accounts for solvers Sep 28, 2026
.is_some_and(|found| receivable_token_account(found, order.buy_token.0));
let receivable = match accounts.get(&account) {
Some(found) => receivable_token_account(found, order.buy_token.0),
None => account == associated_token_address(order),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the line that turns the feature on for on-chain orders, so it's the one I'd hold until the rent is priced. Every order it keeps costs the solver keypair an ATA rent that the owner can reclaim right after the fill.

Comment on lines +57 to +65
/// True when the order's buy token account does not exist on chain
/// yet. The settlement creates it and the solver keypair pays its rent,
/// so the solution should price that rent in.
///
/// TODO(token-2022): a token-2022 account rents more bytes, so once
/// those mints are supported this boolean becomes the missing account's
/// token program.
#[serde(skip_serializing_if = "std::ops::Not::not")]
pub missing_buy_token_account: bool,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand the purpose of this field if we decided that driver creates missing buy token ATAs using the solver's private key. The solver engine doesn't even see the user's account, butDestination is a buffer.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idea for that field is to just signal to solvers that the driver will create the buy token ATA, so that they can adjust their solutions/prices to it.

IIUC Haris asked for this to be communicated w/ solvers somehow.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright, I see. If that is still a useful info for solvers, let's keep it, but I am not sure about the overall shape mentioned below.

Comment on lines +61 to +63
/// TODO(token-2022): a token-2022 account rents more bytes, so once
/// those mints are supported this boolean becomes the missing account's
/// token program.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it stays, I'd send the rent in lamports instead of a bool, like the setupCostLamports idea from the relevant Slack thread. Token-2022 then only changes the number, and engines don't have to know the rent math.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes sense. I'll update the comment to include that.

Comment on lines +76 to +78
auction
.flag_missing_buy_token_accounts(&self.blockchain)
.await;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This puts a getMultipleAccounts round trip in front of the engine call on every /solve, so it eats into the deadline. Nobody reads the flag yet, so for now it only adds latency.

@tilacog tilacog Sep 28, 2026 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's correct, but it's required for telling solvers about the need to create the missing buy token accounts (ctx here) before they submit their solutions.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we could put this behind a cache, se we only call it once per auction? (instead of once per auction, per solver)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, a shared cache is a good idea 👍

Comment thread crates/solana-driver/openapi.yml Outdated
Comment on lines +195 to +197
The token account the settlement pays out to. When it does not
exist yet and is the owner's associated token account, the
settlement creates it and the solver pays its rent.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This turns into a requirement for colocated drivers as well. Once the cut keeps these orders, every driver gets orders whose payout account doesn't exist, and it has to create it or fail. In the thread I suggested the autopilot forwards this since it already runs the check, and BE-312 describes the cut's filter turning into a flag for drivers. Here each driver re-derives it instead. Not a blocker, but we should pick one and tell the colocated teams.

@tilacog tilacog Sep 28, 2026 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the idea of the autopilot taking care of this, so both colocated and non-colocated drivers can benefit from these findings.

Comment on lines +126 to +130
pub fn buy_token_account_missing(&self, order: &Order) -> bool {
matches!(
self.token_account_state(&order.buy_token_account),
TokenAccountState::NeedsCreation
) && order.buy_token_account == associated_token_address(&order.owner, &order.buy_token)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: this pulls crate::domain::Order into the blockchain adapter, while the module doc says the adapter only hands out classified account states. "Only the owner's ATA can be created" feels like a domain rule to me, so I'd keep token_account_state here and move this check next to accounts_to_create.

Comment on lines +122 to +125
/// Whether the order's buy token account, the settlement's payout
/// destination, is missing on chain and is the owner's associated token
/// account: the one destination an idempotent create can produce. Any
/// other absent destination stays the owner's to create.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BE-305 says a missing destination we can't create drops the order instead of failing the settlement. The first version did that at /solve, and 47b4494 took it out. The cut drops those orders too, but it fails open when its lookup fails, and then a single such order fails the whole settlement with every other order in it. Was there a reason to remove it? If we keep it like this, let's update BE-305.

@tilacog tilacog Sep 28, 2026 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was a move for the driver to not be too restrictive on the orders it can't create, but that might be created from account creation instructions from the sponsored orders transaction.

I'm in favor of filtering the orders again now that we have more clarity on other component's responsibilities, and we likely won't have fancy account creation logic being supported by sponsored orders.

I'll add it back.

Comment on lines -54 to -63
/// A settlement with its on-chain accounts resolved.
///
/// The chain facts (lookup tables, missing setup accounts) are fetched in
/// [`Settlement::resolve_accounts`].
///
/// The transaction optionally sets a compute-unit limit and creates the
/// missing setup accounts (buy-mint buffer PDAs, the payer's sell-mint ATAs),
/// then runs `BeginSettle` (pulls sell tokens into the payer's sell ATAs),
/// the solver interactions, and `FinalizeSettle` (pushes buy tokens out of
/// the buy-mint buffer PDAs).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why drop the transaction layout paragraph? It was the only place describing the whole instruction order. I'd keep it and add the buy ATAs to the setup accounts.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was part of an effort to reduce verbosity of code comments. I will place it back 👍

Comment on lines +285 to +288
/// The setup accounts the settlement must create before `BeginSettle`, each
/// list sorted and deduplicated: the mints whose buffer PDA is missing on
/// chain, and the missing ATAs, the payer's sell ATAs and the orders' buy
/// ATAs.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: "the missing ATAs, the payer's sell ATAs and the orders' buy ATAs" reads like three lists. Maybe "and the missing ATAs, both the payer's sell ATAs and the orders' buy ATAs".

Comment on lines +306 to +312
let missing_user_atas = orders
.iter()
.filter(|order| snapshot.buy_token_account_missing(order))
.map(|order| Ata {
owner: order.owner,
mint: order.buy_token,
});

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we count these? A counter, or an info log with the owner and mint. It's the one place where the solver keypair pays rent for someone else, and I'd like to see how much it adds up to.

Comment on lines +103 to +106
/// Whether `buy_token_account` was missing on chain at solve time and
/// the settlement will create it, see
/// [`Auction::flag_missing_buy_token_accounts`].
pub missing_buy_token_account: bool,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: this is a per-solve annotation on the domain order, so every constructor has to set missing_buy_token_account: false now (the quote auction, the solve DTO, plus a bunch of tests). If the flag stays, a HashSet<OrderUid> built in Competition::solve and passed to the DTO builder would leave Order alone. Moot if we drop the flag, tho.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like that suggestion, and I'd prefer not to touch the Order datatype. I'll implement this way in this PR.

@squadgazzz

Copy link
Copy Markdown
Contributor

A few things in the description. The title is BE-305's old name, the ticket is "Create missing buy token accounts in the driver at settle" now, which matches the code better. The follow-up says there's no ticket for /quote, but that's BE-310, and it's assigned to you, isn't it? The solana-driver: bullet is empty. I'd also mention that the rent is unpriced and paid by our solver keypair, and add the How to test section from the template.

@squadgazzz

Copy link
Copy Markdown
Contributor

IIUC the thread converged on the driver creating a missing buy ATA at settle, paid by the solver keypair. That part is here. Creating it inside the settlement tx is also nice, since a failed settlement reverts the create and nobody donates rent.

What we didn't agree on is turning it on before the rent is priced. The BE-305's note says launch keeps the user creating the account at placement, the cut keeps dropping unreceivable on-chain orders, and driver-side creation and pricing land together with BE-310 and BE-312. With this PR the cut keeps those orders right away, and nothing prices the rent yet: quotes can't see the account, and the Jupiter engine ignores the flag. So our solver keypair pays per created ATA, and the owner can close the account after the fill and take the rent back. It's the farming loop I described at the start of the thread.

On pricing, the last messages leaned towards the EVM model (the orderbook adjusts the quote with native prices, and quoters ignore the ATA cost) rather than a per-order flag for solvers. It isn't closed yet, so I'd rather not add the solver-facing part before it is.

My suggestion is to merge the driver half now, since it does next to nothing while the cut drops these orders and sponsored placement still creates the ATA. Then hold the cut change until the pricing and BE-312 are in, and drop missingBuyTokenAccount with its lookup at /solve for now.

@squadgazzz squadgazzz left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving to unblock the merge. The remaining comments are mainly nits.

Every order the branch would keep costs the winning solver's keypair the
rent for an account its owner can close right after the fill, and no
engine prices that rent in yet. The expression stays next to the TODO so
turning it on is one edit.
"Only the owner's ATA can be created" is a domain rule, but it sat in the
blockchain adapter, which had to pull in `domain::Order` to apply it. The
adapter is back to handing out classified account states only.
Reinstates the filter 47b4494 removed. The autopilot's cut drops these
orders too, but it fails open when its own lookup fails, and then one
such order takes down the whole settlement it lands in.
The flag is a per-solve annotation, so carrying it on the domain order
forced every constructor and test fixture to set it. The resolution now
returns the uids and the DTO builder looks them up.
Every solver engine this driver hosts receives the same auction, so each
of them paid for its own getMultipleAccounts round trip in front of the
engine call. One shared slot now serves them all, and the engines that
arrive while the lookup is in flight wait for its result.
The paragraph on ResolvedSettlement was the only place describing the
whole instruction order; it comes back with the buy ATAs among the setup
accounts. The DTO TODOs now point at setupCostLamports, which keeps the
rent math out of the engines, and the openapi says what happens to an
order whose destination cannot be created.
This is the one place the solver keypair pays rent for someone else, and
the owner can close the account for the lamports right after the fill, so
the cost needs to be countable.
@tilacog

tilacog commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor Author

I'll wait for #4986 to reach main then adjust this PR on top of that.

…issing-buy-token-accounts-for-solvers

Main's native SOL buys (#4985, #4986) pay out to a wallet, not a token
account, so the buy token account resolution at /solve and the account
creation at /settle skip them. The payer's wSOL ATA that main always
creates joins the branch's list of ATAs to create. Main's size-limit
test drops its mint override: the branch's fixture already gives the
solve request the intent's mints.
@tilacog

tilacog commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor Author

@claude review the changes introduced by the merge commit cdba0c9

(for visibility of what I needed to change)

@claude

claude Bot commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

Claude finished @tilacog's task in 4m 18s —— View job


Review — changes introduced by merge commit cdba0c9

  • Identify what the merge commit changed
  • Review the integration for correctness
  • Post findings

cdba0c9 merges origin/main into the PR branch. The substantive thing it reconciles is PR #4986 (settle native SOL buys from the state PDA) against this PR's missing buy-token-account creation. Both features add setup accounts and instructions to the same settlement path, so the merge is effectively an integration commit, not a trivial fast-forward. I traced it end-to-end.

No conflict markers, no dropped hunks, and the two features compose correctly. The reconciliation is centered on one idea: a native-SOL buy pays out to a wallet, not a token account, so it must be excluded from every buy-token-account code path and instead routed through the wSOL-unwrap flow.

Key integration points I verified:

  • token_buys() filter (settlement.rs) — native-SOL buys are excluded from both buffer creation and user-ATA creation. Correct: their payout destination is a wallet the transfer itself creates, so there's no ATA to pre-create. classify_buy_token_accounts (auction.rs) applies the same exclusion at solve time, so a native-SOL order is never in missing/unreceivable, and therefore never gets missingBuyTokenAccount: true on the wire. Symmetric across all three layers.
  • finalize_ix_index now folds in funding.len() alongside the existing interaction count, and the ATA-create instructions are still pushed before begin_ix_index is computed — so both features' extra instructions shift the reciprocal BeginSettle/FinalizeSettle indices correctly. The native_sol_payouts_are_funded_from_the_unwrapped_wsol test's len() == 6 assertion holds because resolve_for_test bypasses buffer/ATA creation.
  • accounts_to_create chains missing_user_atas (token buys' buy ATAs, this PR) with missing_payer_atas (sell ATAs + the always-created wSOL ATA, from main) and then sorts + dedups the combined list — the wSOL push-regardless is idempotent against dedup. The rent-paying tracing::info! for user-ATA creation that was requested in earlier review is present.
  • fitting_solutions / transaction_size (from main) build the real settlement via resolve_accounts + encode, so the size estimate now automatically includes both the ATA-create and native-funding instructions — a nice synergy, not a conflict.
  • DTO buy_destination correctly diverges: buffer PDA for token buys, taker's wSOL ATA for native-SOL buys, and the taker param is threaded through Order::new at every call site (incl. tests).
  • solve() still takes auction by value, auction_id is read before any move, and the compute_solutions(&auction, &missing) → fitting_solutions(...) ordering is coherent.

No blocking issues found — nothing to post inline. The integration is correct.

One caveat: I could not compile or run the test suite in this environment (cargo invocations are blocked by the sandbox here), so the above is a static trace of the merge, not a green build. Worth a local cargo nextest run -p solana-driver -p autopilot-svm -p solana-solvers to confirm, since the merge changed a fair number of shared test helpers and instruction-index assertions.
· tiago/be-305-detect-missing-buy-token-accounts-for-solvers

@tilacog
tilacog added this pull request to the merge queue Sep 30, 2026
Merged via the queue into main with commit e3ac7ac Sep 30, 2026
40 of 42 checks passed
@tilacog
tilacog deleted the tiago/be-305-detect-missing-buy-token-accounts-for-solvers branch September 30, 2026 15:50
@github-actions github-actions Bot locked and limited conversation to collaborators Sep 30, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants