Skip to content

bug(sandbox): SSH exec ends with exit 74 when the sandbox process reads stdin slower than the client sends (breaks git push to a sandbox) #5727

Open

Description

@averi

User Story

I use OpenShell to run coding agents in per-repository sandboxes. My wrapper syncs code between the host and the sandbox with plain git over the sandbox SSH endpoint (openshell sandbox ssh-config plus git push/git fetch against ssh://<host>/<path>).

I directly hit this on the first push of a medium-sized repository (about 38 MiB of history). git push fails every time, so the sandbox never gets the code.

Problem Statement

If a command run over sandbox SSH reads its stdin more slowly than the client sends, the supervisor ends the exec with exit status 74 once about 4 MiB of input is waiting. The client gets no flow control. OpenSSH keeps sending at full speed, and the session is dropped instead of slowed down.

A fast reader such as cat or wc -c takes 64 MiB without trouble. A reader that pauses briefly fails on anything over about 4 MiB. git receive-pack → git index-pack is a slow reader, so any push that produces a pack larger than roughly 4–8 MiB fails:

send-pack: unexpected disconnect while reading sideband packet
fatal: the remote end hung up unexpectedly

#4430 reported the same failure for openshell sandbox upload. #4448 fixed that by adding a credit protocol just for uploads, and its description says "The general 4 MiB stdin guard, ordinary exec stdin and SFTP behavior remain unchanged." So every other stdin-heavy SSH workload still has the bug.

Impact / Why This Matters

  • git push to a sandbox fails for any repository whose push produces a pack larger than a few MiB. That covers most real first pushes. The same applies to rsync, tar c | ssh … tar x, ssh … 'docker load', database restores, and any other pipe into a process that does work as it reads.
  • Whether it fails depends on timing, not on a fixed size. The same push can work on a fast moment and fail on a slow one, so it looks flaky rather than broken.
  • The client sees a generic "remote end hung up". The only explanation is a WARN line in openshell logs.
  • Workarounds today: split the push into several small incremental pushes, or copy a git bundle with a fast reader (ssh … 'cat > f') and then fetch from it inside the sandbox. Both mean every tool that talks to a sandbox needs to know about this limit.
  • Raising or removing the bound isn't the answer, because the bound protects supervisor memory (fix(supervisor): bound pending exec stdin and cancel stalled writers #3846). The client needs backpressure.

Acceptance Criteria

  • head -c 64M /dev/urandom | ssh <sandbox> 'sleep 5; wc -c' prints 67108864 and exits 0.
  • git push of a repository with a 40+ MiB pack into an empty repository in the sandbox succeeds.
  • Supervisor memory per exec channel stays bounded while a slow reader is consuming a large input. The client is paused instead of the exec being terminated.
  • openshell logs doesn't show process stdin buffer is full; terminating exec for a reader that is slow but still making progress.

Reproduction Steps

  1. openshell sandbox create --name stdin-repro
  2. openshell sandbox ssh-config stdin-repro > /tmp/stdin-repro.ssh
  3. H=$(awk '/^Host /{print $2; exit}' /tmp/stdin-repro.ssh)
  4. Fast reader, works:
    head -c 64M /dev/urandom | ssh -F /tmp/stdin-repro.ssh "$H" 'wc -c'; echo "exit=$?"
    → 67108864, exit=0
  5. Slow reader, fails:
    for mb in 1 4 8 16; do
      head -c ${mb}M /dev/urandom | ssh -F /tmp/stdin-repro.ssh "$H" 'sleep 3; wc -c'
      echo "${mb}MB exit=$?"
    done
    
    → 1 MB and 4 MB print the byte count and exit 0. 8 MB and 16 MB print nothing and exit 74.
  6. openshell logs stdin-repro | grep 'stdin buffer' shows one terminating exec line per failure.

Same failure with git (any repository with more than about 10 MiB of history):

ssh -F /tmp/stdin-repro.ssh "$H" 'git init -q /tmp/r'
GIT_SSH_COMMAND="ssh -F /tmp/stdin-repro.ssh" git push "ssh://$H/tmp/r" HEAD:refs/heads/main

Suggested UX (if applicable)

No response

Environment

  • OpenShell CLI and gateway: 0.1.5 (the bound and the process stdin buffer is full path are unchanged in v0.1.9 and on main, crates/openshell-supervisor-process/src/ssh.rs)
  • OS: Fedora 44, kernel 7.3.3, x86_64
  • Deployment: local gateway (mTLS), Podman 5.8.9, isolation backend openshell-sandbox
  • Client: OpenSSH 10.2p1, git 2.56.1 on host and in the sandbox

Logs

$ head -c 8M /dev/urandom | ssh -F /tmp/stdin-repro.ssh "$H" 'sleep 3; wc -c'; echo $?
74

$ openshell logs devdocsgjs-6ee5 | grep 'stdin buffer'
[sandbox] [WARN ] [openshell_supervisor_process::ssh] process stdin buffer is full; terminating exec channel=ChannelId(2)

# git push --receive-pack='GIT_TRACE=1 git-receive-pack' ... (remote side)
remote: trace: built-in: git index-pack --stdin --pack_header=2,10852 ... --fix-thin
send-pack: unexpected disconnect while reading sideband packet
fatal: the remote end hung up unexpectedly

# ssh -v
debug1: Exit status 74
Transferred: sent 16622844, received 2884 bytes, in 0.7 seconds

Activity

  1. added theissue type on Nov 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    state:triage-neededOpened without agent diagnostics and needs triage

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions