Repository navigation
feat(kubernetes): configure workload and supervisor placement - #5629
TaylorMutch wants to merge 9 commits into
Conversation
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
|
🌿 Preview your docs: https://nvidia-preview-pr-5629.docs.buildwithfern.com/openshell |
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
…tart Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
Compatibility review (advisory)New or updated breaking changes; see the full report for all findings. 1. helm — No explicit risk acceptance foundHelm previously rendered arbitrary gateway podLabels, but now rejects podLabels["openshell.ai/runtime-role"] unless its value is "gateway".
2. configuration — No explicit risk acceptance foundruntime_bootstrap_grace_seconds previously triggered reclamation of stale bootstraps, but released bootstraps with Unschedulable pods now bypass that deadline indefinitely.
3. cli — No explicit risk acceptance foundHELM_K3S_AGENT_COUNT=2 mise run helm:k3s:create previously created three nodes, but now creates one unless callers switch to HELM_K3S_NODES=3 mise run helm:k3s:create.
Review details
Advisory review; publication gates are unchanged. |
Summary
Prefer separate nodes for sandbox workloads and supervisors, including independently deployed shared supervisors. Operators can require separation or opt out, and can optionally select distinct node pools for gateways, supervisors, and workloads.
Related Issue
Closes #5619
Changes
openshell.ai/runtime-rolelabel to workload, supervisor, and gateway pods without changing Service or controller selectors.Release Notes
Workloads and supervisors prefer separate nodes by default. Node selectors and tolerations remain opt-in. Existing pods acquire role labels and placement rules when recreated.
Testing
The focused live test is
mise run e2e:kubernetes:scheduling. It also passes withoperatoromitted from an operator-configured toleration. Its dedicated Cargo feature excludes the test from the general Kubernetes suite. Test fixtures use matching gateway, sandbox, and supervisor images built from this branch. The isolated validation cluster was removed after testing.Checklist