Detect missing buy token accounts for solvers - #4982
Conversation
|
Reminder: Please consider backward compatibility when modifying the API specification.
Caused by: |
|
@claude review |
|
Claude finished @tilacog's task in 3m 51s —— View job PR Review
Reviewed the full diff, tracing the flow end-to-end: autopilot's This is a clean, well-tested change. The re-check at settle time (rather than trusting the solve-time flag) correctly guards against an account closing in between, the I left one low-severity inline note: autopilot drops orders whose buy ATA is a pre-funded system account, whereas the driver would treat that same account as creatable — a small asymmetry in the exact alignment this PR is trying to establish. Conservative and rare, so non-blocking. The rent-not-priced-in behavior (jupiter ignoring the flag for now) is already acknowledged in the PR description and follow-ups, so I didn't flag it. |
| let receivable = match accounts.get(&account) { | ||
| Some(found) => receivable_token_account(found, order.buy_token.0), | ||
| None => account == associated_token_address(order), | ||
| }; |
There was a problem hiding this comment.
Low severity / edge case — autopilot is slightly stricter than the driver here.
This keeps an order only when the buy account is fully absent (None) and equals the owner's ATA. But the driver's buy_token_account_missing (via TokenAccountState::NeedsCreation) also treats a pre-funded system account (system-owned, zero data) at the ATA address as creatable — create_associated_token_account_idempotent allocates over it.
In that case the account is present, so this branch takes Some(found) => receivable_token_account(...), which returns false (owner is the system program, not the SPL token program), and the order is dropped — even though the driver could have created the ATA and settled it.
It's the conservative direction (a dropped order, never a failed settlement), and the scenario is rare, so this isn't blocking. But since the whole point of the change is to align autopilot's cut with what the driver can create, it may be worth either handling the Some(system-owned, empty) case symmetrically or leaving a short note explaining the intentional gap.
There was a problem hiding this comment.
We can consider unifying this as a shared module in the future.
…issing-buy-token-accounts-for-solvers
|
Claude finished @tilacog's task in 2m 9s —— View job PR Review
Re-reviewed the full diff end-to-end: autopilot's This is a clean, well-scoped change and I found no new blocking issues. Highlights that hold up under scrutiny:
Test coverage is strong across every layer (autopilot cut, driver flag, settlement re-check, DTO wire shape, and the api.rs integration path). The one prior finding — autopilot dropping an order whose buy ATA is a pre-funded system account while the driver would treat it as creatable — was already discussed and acknowledged (unify as a shared module later), and the rent-not-priced-in / jupiter-ignores-the-flag behavior is covered in the PR description and follow-ups, so I'm not re-flagging either. LGTM. 👍 |
| .is_some_and(|found| receivable_token_account(found, order.buy_token.0)); | ||
| let receivable = match accounts.get(&account) { | ||
| Some(found) => receivable_token_account(found, order.buy_token.0), | ||
| None => account == associated_token_address(order), |
There was a problem hiding this comment.
This is the line that turns the feature on for on-chain orders, so it's the one I'd hold until the rent is priced. Every order it keeps costs the solver keypair an ATA rent that the owner can reclaim right after the fill.
| /// True when the order's buy token account does not exist on chain | ||
| /// yet. The settlement creates it and the solver keypair pays its rent, | ||
| /// so the solution should price that rent in. | ||
| /// | ||
| /// TODO(token-2022): a token-2022 account rents more bytes, so once | ||
| /// those mints are supported this boolean becomes the missing account's | ||
| /// token program. | ||
| #[serde(skip_serializing_if = "std::ops::Not::not")] | ||
| pub missing_buy_token_account: bool, |
There was a problem hiding this comment.
I don't understand the purpose of this field if we decided that driver creates missing buy token ATAs using the solver's private key. The solver engine doesn't even see the user's account, butDestination is a buffer.
There was a problem hiding this comment.
The idea for that field is to just signal to solvers that the driver will create the buy token ATA, so that they can adjust their solutions/prices to it.
IIUC Haris asked for this to be communicated w/ solvers somehow.
There was a problem hiding this comment.
Alright, I see. If that is still a useful info for solvers, let's keep it, but I am not sure about the overall shape mentioned below.
| /// TODO(token-2022): a token-2022 account rents more bytes, so once | ||
| /// those mints are supported this boolean becomes the missing account's | ||
| /// token program. |
There was a problem hiding this comment.
If it stays, I'd send the rent in lamports instead of a bool, like the setupCostLamports idea from the relevant Slack thread. Token-2022 then only changes the number, and engines don't have to know the rent math.
There was a problem hiding this comment.
This makes sense. I'll update the comment to include that.
| auction | ||
| .flag_missing_buy_token_accounts(&self.blockchain) | ||
| .await; |
There was a problem hiding this comment.
This puts a getMultipleAccounts round trip in front of the engine call on every /solve, so it eats into the deadline. Nobody reads the flag yet, so for now it only adds latency.
There was a problem hiding this comment.
That's correct, but it's required for telling solvers about the need to create the missing buy token accounts (ctx here) before they submit their solutions.
There was a problem hiding this comment.
Maybe we could put this behind a cache, se we only call it once per auction? (instead of once per auction, per solver)
There was a problem hiding this comment.
Yeah, a shared cache is a good idea 👍
| The token account the settlement pays out to. When it does not | ||
| exist yet and is the owner's associated token account, the | ||
| settlement creates it and the solver pays its rent. |
There was a problem hiding this comment.
This turns into a requirement for colocated drivers as well. Once the cut keeps these orders, every driver gets orders whose payout account doesn't exist, and it has to create it or fail. In the thread I suggested the autopilot forwards this since it already runs the check, and BE-312 describes the cut's filter turning into a flag for drivers. Here each driver re-derives it instead. Not a blocker, but we should pick one and tell the colocated teams.
There was a problem hiding this comment.
I like the idea of the autopilot taking care of this, so both colocated and non-colocated drivers can benefit from these findings.
| pub fn buy_token_account_missing(&self, order: &Order) -> bool { | ||
| matches!( | ||
| self.token_account_state(&order.buy_token_account), | ||
| TokenAccountState::NeedsCreation | ||
| ) && order.buy_token_account == associated_token_address(&order.owner, &order.buy_token) |
There was a problem hiding this comment.
Nit: this pulls crate::domain::Order into the blockchain adapter, while the module doc says the adapter only hands out classified account states. "Only the owner's ATA can be created" feels like a domain rule to me, so I'd keep token_account_state here and move this check next to accounts_to_create.
| /// Whether the order's buy token account, the settlement's payout | ||
| /// destination, is missing on chain and is the owner's associated token | ||
| /// account: the one destination an idempotent create can produce. Any | ||
| /// other absent destination stays the owner's to create. |
There was a problem hiding this comment.
BE-305 says a missing destination we can't create drops the order instead of failing the settlement. The first version did that at /solve, and 47b4494 took it out. The cut drops those orders too, but it fails open when its lookup fails, and then a single such order fails the whole settlement with every other order in it. Was there a reason to remove it? If we keep it like this, let's update BE-305.
There was a problem hiding this comment.
This was a move for the driver to not be too restrictive on the orders it can't create, but that might be created from account creation instructions from the sponsored orders transaction.
I'm in favor of filtering the orders again now that we have more clarity on other component's responsibilities, and we likely won't have fancy account creation logic being supported by sponsored orders.
I'll add it back.
| /// A settlement with its on-chain accounts resolved. | ||
| /// | ||
| /// The chain facts (lookup tables, missing setup accounts) are fetched in | ||
| /// [`Settlement::resolve_accounts`]. | ||
| /// | ||
| /// The transaction optionally sets a compute-unit limit and creates the | ||
| /// missing setup accounts (buy-mint buffer PDAs, the payer's sell-mint ATAs), | ||
| /// then runs `BeginSettle` (pulls sell tokens into the payer's sell ATAs), | ||
| /// the solver interactions, and `FinalizeSettle` (pushes buy tokens out of | ||
| /// the buy-mint buffer PDAs). |
There was a problem hiding this comment.
Why drop the transaction layout paragraph? It was the only place describing the whole instruction order. I'd keep it and add the buy ATAs to the setup accounts.
There was a problem hiding this comment.
It was part of an effort to reduce verbosity of code comments. I will place it back 👍
| /// The setup accounts the settlement must create before `BeginSettle`, each | ||
| /// list sorted and deduplicated: the mints whose buffer PDA is missing on | ||
| /// chain, and the missing ATAs, the payer's sell ATAs and the orders' buy | ||
| /// ATAs. |
There was a problem hiding this comment.
Nit: "the missing ATAs, the payer's sell ATAs and the orders' buy ATAs" reads like three lists. Maybe "and the missing ATAs, both the payer's sell ATAs and the orders' buy ATAs".
| let missing_user_atas = orders | ||
| .iter() | ||
| .filter(|order| snapshot.buy_token_account_missing(order)) | ||
| .map(|order| Ata { | ||
| owner: order.owner, | ||
| mint: order.buy_token, | ||
| }); |
There was a problem hiding this comment.
Could we count these? A counter, or an info log with the owner and mint. It's the one place where the solver keypair pays rent for someone else, and I'd like to see how much it adds up to.
| /// Whether `buy_token_account` was missing on chain at solve time and | ||
| /// the settlement will create it, see | ||
| /// [`Auction::flag_missing_buy_token_accounts`]. | ||
| pub missing_buy_token_account: bool, |
There was a problem hiding this comment.
Nit: this is a per-solve annotation on the domain order, so every constructor has to set missing_buy_token_account: false now (the quote auction, the solve DTO, plus a bunch of tests). If the flag stays, a HashSet<OrderUid> built in Competition::solve and passed to the DTO builder would leave Order alone. Moot if we drop the flag, tho.
There was a problem hiding this comment.
I like that suggestion, and I'd prefer not to touch the Order datatype. I'll implement this way in this PR.
|
A few things in the description. The title is BE-305's old name, the ticket is "Create missing buy token accounts in the driver at settle" now, which matches the code better. The follow-up says there's no ticket for |
|
IIUC the thread converged on the driver creating a missing buy ATA at settle, paid by the solver keypair. That part is here. Creating it inside the settlement tx is also nice, since a failed settlement reverts the create and nobody donates rent. What we didn't agree on is turning it on before the rent is priced. The BE-305's note says launch keeps the user creating the account at placement, the cut keeps dropping unreceivable on-chain orders, and driver-side creation and pricing land together with BE-310 and BE-312. With this PR the cut keeps those orders right away, and nothing prices the rent yet: quotes can't see the account, and the Jupiter engine ignores the flag. So our solver keypair pays per created ATA, and the owner can close the account after the fill and take the rent back. It's the farming loop I described at the start of the thread. On pricing, the last messages leaned towards the EVM model (the orderbook adjusts the quote with native prices, and quoters ignore the ATA cost) rather than a per-order flag for solvers. It isn't closed yet, so I'd rather not add the solver-facing part before it is. My suggestion is to merge the driver half now, since it does next to nothing while the cut drops these orders and sponsored placement still creates the ATA. Then hold the cut change until the pricing and BE-312 are in, and drop |
squadgazzz
left a comment
There was a problem hiding this comment.
Approving to unblock the merge. The remaining comments are mainly nits.
Every order the branch would keep costs the winning solver's keypair the rent for an account its owner can close right after the fill, and no engine prices that rent in yet. The expression stays next to the TODO so turning it on is one edit.
"Only the owner's ATA can be created" is a domain rule, but it sat in the blockchain adapter, which had to pull in `domain::Order` to apply it. The adapter is back to handing out classified account states only.
Reinstates the filter 47b4494 removed. The autopilot's cut drops these orders too, but it fails open when its own lookup fails, and then one such order takes down the whole settlement it lands in.
The flag is a per-solve annotation, so carrying it on the domain order forced every constructor and test fixture to set it. The resolution now returns the uids and the DTO builder looks them up.
Every solver engine this driver hosts receives the same auction, so each of them paid for its own getMultipleAccounts round trip in front of the engine call. One shared slot now serves them all, and the engines that arrive while the lookup is in flight wait for its result.
The paragraph on ResolvedSettlement was the only place describing the whole instruction order; it comes back with the buy ATAs among the setup accounts. The DTO TODOs now point at setupCostLamports, which keeps the rent math out of the engines, and the openapi says what happens to an order whose destination cannot be created.
This is the one place the solver keypair pays rent for someone else, and the owner can close the account for the lamports right after the fill, so the cost needs to be countable.
|
I'll wait for #4986 to reach |
…issing-buy-token-accounts-for-solvers Main's native SOL buys (#4985, #4986) pay out to a wallet, not a token account, so the buy token account resolution at /solve and the account creation at /settle skip them. The payer's wSOL ATA that main always creates joins the branch's list of ATAs to create. Main's size-limit test drops its mint override: the branch's fixture already gives the solve request the intent's mints.
|
Claude finished @tilacog's task in 4m 18s —— View job Review — changes introduced by merge commit
|
Description
The driver now checks chain state at solve time to determine which user's
buy_token_accountneeds to be created for an order to be filled, inform solvers about those and creates them during settlement.Mind that only the owner's ATA for the buy token can possibly be created, so any other invalid destinations (that aren't already filtered by the autopilot's cut) will cause the pre-submission simulation to fail, same as before.
Changes
autopilot-svm: if a receivable account isn't found on chain, it fills w/ the user's ATA for the buy tokensolana-driver:missingBuyTokenAccountboolean per order on the engine/solverequest, omitted whenfalse. The jupiter engine parses it and ignores it for now.buyTokenAccountno longer says the settlement never creates it.token_account_jsonandmultiple_accounts_jsonmock helpers, also used by the autopilot-svm tests.Follow Up PRs
/quote(I don't believe we have a Linear ticket for this yet)