testbench improvements - #31
Draft
rcgoodfellow wants to merge 14 commits into
Draft
Conversation
rcgoodfellow
force-pushed
the
ry/bench-init
branch
from
August 31, 2026 23:59
167c9af to
16ce457
Compare
- anodizer support for subtypes - initializers for vhdl/cosim/mist benches w/ option for DUT plumbing - standalone `vw cosim anodize` function for static checks breaking cli change: vw bench is now vw bench run
rcgoodfellow
force-pushed
the
ry/bench-init
branch
from
September 4, 2026 07:30
16ce457 to
9a22ee4
Compare
--beta was a service running beside production on one host, told apart by an
object prefix, a second set of ports and a second set of files. The premise
does not hold: vw-svc reaches agents on their private VPC addresses and a VPC
does not span projects, so a service outside the project it provisions into
cannot talk to anything it creates. A deployment has to be a whole project.
So --beta becomes --deployment <name>, required and with no default. Objects
are `vwsvc-{deployment}-...`, which keeps two deployments off each other's
silo ssh keys -- the one thing a project cannot separate, since that list
belongs to the token's user and is scoped by nothing. The name may not contain
a hyphen: `prod` would otherwise claim, and then delete, everything belonging
to `prod-west`.
What that subsumes, all deleted: vw-svc-beta.service, vw-svc-beta.env.example,
install.sh --beta, and $VW_BETA/BETA_PORTS/announce_beta in the client, which
now picks a deployment by URL and nothing else. `is_beta()` in visible_images
goes too -- every deployment boots its own project's images and never the
silo's, which was always the general argument and only ever applied to one.
Ports move to 443 for the user API and 2053 for the admin API, in vw-svc and
vw-cli together, so a service is reached as `https://vw-cloud.dev` with no port
to quote.
The deployment segment also eats into the 63 characters an Oxide name gets, so
environment creation now checks the assembled instance name and says which
field to shorten, rather than leaving it to fail as a 400 on the reconciler's
next pass against a name nobody typed.
`vw cloud admin` was hardcoded to https://vw-cloud.dev:2053 and deliberately did not follow --url or $VW_SVC_URL, on the reasoning that a separate listener is separately reachable. That reasoning was about the service and got applied to the client, where it is wrong. What it looks like in practice: point $VW_SVC_URL at the beta, `vw cloud list` answers from the beta, and `vw cloud admin list` hangs -- because it is talking to production's domain, whose floating IP currently has no instance behind it. The hang is the lucky version. Once production is up, the same command reaches it and answers, and `vw cloud admin delete` deletes somebody's environment on the deployment the caller thought they had stopped talking to. These are the only commands that can touch another person's work, and they were the ones pointed somewhere the caller did not choose. So derive the admin URL from the service URL, replacing only the port. --admin-url and $VW_SVC_ADMIN_URL still name it outright, which is what a deployment that puts it somewhere else needs. The trailing slash matters and is trimmed: Url serializes an empty path as `/`, the generated client builds `{base}/v1/...`, and `//v1/...` is a 404 that would have been diagnosed as anything but this.
`$HOME/.vw/deps` is shared by every workspace on a machine and keyed `<name>-<sha>`, so two processes wanting one commit want one directory. Nothing coordinated that. A fetch wrote straight into the final path and readers decided it was cached by asking whether the directory had anything in it -- true of a download that has written its first file and has minutes left to run. So one build could link half a dependency while another was still extracting it, and the loser of the other interleaving called remove_dir_all on a tree the winner was writing into. Today that needs a crash to provoke, because one instance serves one workspace. It stops needing one as soon as several workspaces share an instance, which is where this is going: `fetch_dependencies` runs on every session start, and two `vw cloud sync --watch` on common deps at the same sha land on the same path. A download now assembles in a private directory under `.staging` and is published with a rename, which is atomic within a filesystem -- hence staging beside the cache rather than in /tmp. The private half is what the random suffix buys: a shared `<name>-<sha>.tmp` would have moved the collision one directory down and fixed nothing. Completeness is stated rather than inferred. A `.vw-complete` marker is written last, inside, so it can only appear on a finished tree, and it replaces the non-empty heuristic in all four places that had a copy of it -- the dependency graph, `dependencies_present`, and both halves of the VHDL stdlib fetch -- plus `materialized_cache_entry`, the one reader that lives in vw-analyzer. That also settles what to do about a partial left by an older vw: nothing considers it usable, so nobody can be reading it, so publishing over it is safe. Which leaves the published path never written again, and that is what lets any number of readers use the cache with no coordination at all. `clear_cache` retracts the marker before it removes a tree so that it cannot be the exception. Orphans are only reachable by killing a process outright -- every other ending drops the staging directory and takes it with it -- so the sweep that reclaims them is a backstop, and its deadline is a day. Generous on purpose: a staging directory being written to belongs to a live download, and sweeping one would reintroduce from the other side exactly what this removes.
The name was free text. `docs/snippets/vw.toml` carried `name = "."`, and nothing would have stopped `name = "../../etc"` either -- which mattered little while the name only ever labelled things, and starts mattering a great deal now that it is about to become a directory on a build instance and a bucket in an object store. Checked in `load_workspace_config`, so a bad name stops every command rather than only the ones that reach a cloud environment. The name is already load-bearing well before then: it is the module a workspace's own imports resolve through (`src @foo/bar`), and the answer `vw::project_name` hands a design. Somewhere in that list is a rule every candidate has to satisfy, and there is no version of this where finding out at a first sync is better than finding out at `vw init` -- which is why init checks it too, before writing anything, rather than leaving behind a workspace its own loader will not open. Lowercase letters, digits and `-`, starting with a letter and not ending with one, at most 40 characters. Hyphens are allowed here even though an environment name may not have them: that ban exists because Oxide instance names are parsed by splitting on `-`, and a workspace name never becomes part of one. It is the tail of a bucket name instead, which is what rules out the trailing hyphen and sets the length -- a cap that cannot be complete on its own, since how much of S3's 63 characters is left depends on an environment name this side never sees, so the assembled name gets checked again where both halves are known. One consequence worth naming: `load_workspace_config` is also how a dependency's own vw.toml is read, and the callers that do so skip a config they cannot parse rather than failing. A dependency whose name does not pass is therefore ignored rather than reported. Every vw dependency is first-party today, so nothing hits this now.
An environment was one workspace, and it costs 36 vCPUs, 80 GiB of memory and
1.8 TiB of disk across three instances. Anybody with a design, a driver and a
scratch copy of each was paying that three times for machines that sit idle
between builds. An environment now holds a tree per workspace and every request
that means a tree says which one.
The key namespaces exactly three things: the source tree on each instance, the
content store behind it, and the bucket its artifacts go to. Nothing else about
an environment divides -- one owner, one ssh key, three instances, as before.
No Oxide object name gains a segment, so the 63-character budget the deployment
name already eats into is untouched.
vw-agent's `--root` and `--store` become the directories those sit under, and
what was fixed at startup is now made when a workspace is first synchronized:
its tree, its store, its own artifact uploader, and the lock a sync takes --
per workspace now, so two trees no longer wait on each other, which under
`--watch` is most of the time. Buckets follow: a store cannot know at boot what
buckets it needs, because that answer is a list of workspaces that did not
exist yet, so `{kind}-{environment}-{workspace}` is made when somebody first
asks for it. Nothing records which workspaces an environment has -- one exists
because it was synchronized -- so listing them asks the instance, and so does
the startup pass that wires each one to its bucket.
On the client, `vw-cloud.toml` beside `vw.toml`, gitignored on first write:
vw cloud set workspace redhawk-feat
vw cloud set environment darmok
The workspace there is a slot, and emphatically not `[workspace] name` -- which
is what a design's own imports resolve through (`src @redhawk/...`) and what
`vw::project_name` reports. Override that instead and a feature checkout stops
resolving its own imports: the two things being compared would differ in a way
that has nothing to do with the change under test. Which is also why the file
is ignored rather than tracked. A slot name committed to a branch travels to
everyone who checks that branch out, and lands every one of them in the slot it
was meant to keep them out of.
The environment resolves the same way everywhere now -- argument, `$VW_ENV`,
`vw-cloud.toml`, then your only one -- which `vw cloud` did not participate in
at all before: `--env` and `$VW_ENV` reached `vw run` and never `vw cloud sync`.
Every command that takes an environment now takes it optionally. `create` and
`delete` still do not: inferring what to name a new thing is meaningless, and
inferring which three instances to destroy is worse than meaningless.
Every sync prints which environment and slot it is pushing to and where each
name came from. Two checkouts resolving to one slot overwrite each other on
every sync, and that line is the only place it would ever be visible.
`vw cloud workspaces` lists what is on an environment with how long ago each
was pushed to, and `vw cloud forget` removes one -- tree, build output and
artifacts. Not a nicety: renaming a workspace otherwise strands its tree
somewhere nothing will ever come back for. The bucket is left standing, empty,
so a workspace of the same name returning finds its own.
Both APIs go to a single version -- user 5.0.0, agent 4.0.0 -- with every
earlier one dropped from the supported set rather than retired endpoint by
endpoint. That is a real break, stated plainly: moving the tree into a path
segment leaves no spelling of the old routes that means anything, and serving
them would mean picking a workspace on the caller's behalf. The count keeps
climbing anyway, because a version nobody may send is still one somebody once
did.
One thing found on the way: `build_dependency_graph` skipped a manifest it
could not parse, including the entry's own, so `vw update` on a workspace whose
name the new validator rejects reported success and did nothing. The entry's
manifest is now strict. A dependency's is still skipped -- that costs an
unresolved import later, where failing would let somebody else's manifest stop
this build outright.
The workspaces change moved the tree a request acts on into a path segment by
editing the endpoints in place, dropped versions 1-4 from the supported set,
and deleted their OpenAPI documents. `cargo xtask openapi check` said so, eight
times: "formerly blessed version has been removed. This version will no longer
be supported! This will break upgrade from software that still uses this
version."
It was right, and the reasoning behind ignoring it was wrong twice over. I had
it that the guide's forwarding pattern could not express this change, because
there is no conversion from "no workspace" to "a workspace" -- but dropshot
publishes `Path::map`, `Query::map` and `TypedBody::map` for exactly this, with
a doc comment saying so, and provided trait methods are supported. And I had it
that any forwarding would have to guess which tree an old client meant, which
is only true if old and new clients are made to share one. They are not: an old
client synchronizes into a reserved slot and then builds out of it, so it sees
what it always saw -- one environment, one tree, entirely its own.
So every endpoint that changed is now two. The prior one keeps its path, its
`operation_id`, its signature and its doc comment, gains an upper version
bound, and is a provided method forwarding to the new one with
`LEGACY_WORKSPACE`. Neither vw-svc nor vw-agent needed a line for any of it,
which is the point of the pattern.
All eleven prior documents regenerate byte-identical to what is in git, which
is the check that matters. Getting there turned up one thing worth naming: a
sentence I had appended to `get_environment_keys`'s doc comment changed the v4
document, because a description is part of the document. It is reverted. The
guide's rule about preserving comments verbatim is not about taste.
Two further comments have gone back to their original wording where the code
they belong to moved -- how a bucket is named, and why an existing one is
adopted rather than replaced -- and the reasoning the deleted `garage::KINDS`
carried, that helios gets a bucket before it has anything to put in it, is
recorded on the loop that now provides that guarantee.
Types follow the guide's naming: `TargetPathParam`, `TargetBlobPathParam` and
`ArtifactPathParam` gained a workspace, so v5 supersedes them under the same
names and `latest` re-exports the new ones. `WorkspacePathParam` is a genuinely
new type, for the endpoints that used to take an environment and nothing else.
The agent's own path parameters stay in `vw-sync-api` rather than moving to the
versions crate as the guide asks. They cannot move: `EnvironmentPathParam`
there is `{environment}` and in the versions crate it is `{name}`, so sharing
one crate means renaming one of them, and a renamed type is a renamed schema,
and a renamed schema changes a blessed document. The stronger rule wins.
Tested rather than assumed: an agent answers a 1.0.0 client on the old routes,
lands its tree in the reserved slot, and provably does not write into a current
client's tree; the service routes an old sync request as far as the relay; and
a version the service does not have is refused.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
vw cosim anodizefunction for static checksbreaking cli change: vw bench is now vw bench run.
Dependencies