Async Serial Console Writer

Async Serial Console Writer

AsyncSerialConsoleWriter decouples console output from the UART: callers append to a bounded queue and a dedicated drain thread performs the potentially unbounded serial writes, so a stalled port can no longer hold the shared console-writer monitor.

Overview

core/src/driver/org/jnode/driver/serial/console/AsyncSerialConsoleWriter.java is the ConsoleWriter used by the serial agent console. Before it existed, every write() eventually called SerialPortAPI.writeSingle(), which busy-waits on the UART line-status register. A stalled or flow-controlled port therefore parked the calling thread — often a shell or log writer — while it held the console writer's monitor. All other console users queued behind it, and Ctrl-C recovery over serial could not make progress because the console itself was the blocked resource.

The async writer turns that unbounded blocking write into a bounded queue operation. write() only copies characters into a StringBuilder, drops the excess when the queue is full, and wakes a daemon drain thread. The drain thread owns the serial port writes. If the port never drains, the queue stays full and new output is dropped with accounting; the caller still returns immediately.

The class extends org.jnode.driver.console.spi.ConsoleWriter, so it keeps the normal foreground-color handling and console association. It was added with the serial-console hardening that also introduced AsyncSerialConsoleWriterTest (7 tests) into CoreTestSuite.

Key Components

Member	Value / Role
MAX_PENDING_CHARS	8192 — hard upper bound on queued characters
CLOSE_JOIN_TIMEOUT_MS	1000 — maximum time close() waits for the drain thread
pending	StringBuilder holding characters not yet handed to the UART
lock	Guards pending, closed, failed, droppedChars and unreportedDrops; also the wait()/notifyAll() monitor
drainThread	Daemon thread named serial-console-output-drain
droppedChars	Lifetime count of characters discarded, including output written after a fatal port failure
unreportedDrops	Drops accumulated since the last truncation marker was emitted
closed	Set by close(); the drain loop exits once the queue and drop counter are empty
failed	Set when a runtime exception escapes the port write; all later writes are dropped

How It Works

The fast path — write(char[], int, int)

write(cbuf, off, len)
  1. Validate off/len; return for len == 0
  2. synchronized(lock)
       - if failed: droppedChars += len, return
       - space = MAX_PENDING_CHARS - pending.length()
       - if len > space: drop (len - space) and remember the count
       - append the surviving prefix
       - notifyAll()

The caller never invokes the serial port and never blocks on it. The only synchronization is the short critical section around the queue. A full queue does not stall the caller: it drops the tail of the new write and records the loss.

The write(int) overload funnels through the array form with a one-character array.

The drain loop

drainLoop()
  synchronized(lock):
      wait while pending is empty, no unreported drops, and not closed
      exit when closed and fully drained
      snapshot pending into chunk; clear pending
      take and clear unreportedDrops
  if dropped > 0:
      chunk += truncationMarker(dropped)
  writeChunk(chunk)              // '\n' expanded to "\r\n"
  on RuntimeException:
      failed = true
      droppedChars += chunk length + remaining pending
      clear pending and unreportedDrops
      log and return
  exit if closed and fully drained

writeChunk() converts every \n into \r + \n before writing the original character, so serial terminals receive CRLF line endings without changing the queue contents.

Truncation marker

When a write is larger than the remaining queue space, the dropped count is carried until the next drain iteration and appended to the chunk:

return "\n[... truncated " + dropped + " chars ...]\n";

Because writeChunk() expands newlines, the last bytes on the wire are chars ...]\r\n. The marker is deliberately part of the same chunk: it cannot itself be dropped behind the data it describes, and the test suite asserts that the marker is retained at the tail of the received output.

A write that fits entirely produces no marker and increments no drop counter.

close()

close() sets closed and notifies the drain thread, then interrupts it and joins for at most CLOSE_JOIN_TIMEOUT_MS unless close() is being called by the drain thread itself. It is bounded even when the port is permanently stalled: the drain thread survives the stalled writeChunk(), and close() returns after the join timeout. Once the port is released, the drain thread finishes the pending chunk and exits.

flush() is intentionally a no-op. There is no synchronous flush contract for a best-effort console; blocking a caller until a stalled UART drains would reintroduce the bug this class removes.

Failure latch

If the port throws a RuntimeException, the drain thread marks the writer failed, accounts both the unsent chunk and anything still pending as dropped, clears the queue, and exits. Later write() calls increment droppedChars and return. This gives one clear failure point instead of a thread that keeps retrying a dead port.

Threading Model

There are exactly two threads that touch the writer:

Thread	Enters	Owns
Any command/log/shell thread	write(int), write(char[], int, int)	Nothing after it leaves the synchronized block
serial-console-output-drain	drainLoop(), writeChunk()	The SerialPortAPI write path

The monitor is held only for queue manipulation. The drain thread deliberately releases it before calling writeChunk(), because a stalled UART would otherwise pin lock and block every producer — the original failure mode in a new place. The trade-off is that close() can race with a chunk that is already being written: the writer cannot cancel a writeSingle() call that the port has already entered.

close() is safe to call from multiple threads. It is idempotent: setting closed twice and interrupting an already-exited thread are both harmless, and the drain thread's join timeout bounds every caller. If close() is invoked by the drain thread itself (for example from a console teardown path), the self-join is skipped.

Queue arithmetic

The buffer is measured in char units, not bytes. That is the correct unit for ConsoleWriter, but it means the memory footprint depends on the JVM's char[]/StringBuilder representation:

• A full queue holds at most 8192 characters of payload.
• The pending StringBuilder grows to fit the accepted prefix; it is not preallocated at 8192.
• A single write larger than the free space is truncated to the free space; the remainder is never copied.
• getPendingCount() reports the current character count, not including a marker that has not been emitted yet.
• droppedChars is cumulative and never decreases; it is a diagnostic, not a queue length.

Interaction with the serial appender

SerialAppender (the log4j sink) buffers its own pending output while the device is unavailable. The async console writer is a different layer: it sits in front of the agent console's ConsoleWriter and applies to interactive shell output, not to log4j's lkd path. A boot-time log burst can therefore still be synchronous through SerialAppender even while shell output uses the async writer; do not assume the two queues share a bound.

Failure and shutdown scenarios

Scenario	Caller behavior	Drain thread	Output
Port healthy	Returns after queue append	Writes and waits	Full output, CRLF converted
Port briefly stalled	Returns; queue may fill	Blocks in writeSingle()	Delivered when the port recovers
Port stalled indefinitely	Returns; excess dropped	Stays blocked in one write	Prefix delivered after release, marker for drops
Port throws RuntimeException	Later writes return and count as dropped	Marks failed, logs, exits	Everything before the failure only
close() with empty queue	Returns quickly	Sees closed and exits	No pending output
close() with pending output	Waits up to 1 s	Finishes current chunk, drains, exits	Best effort

The distinction between "stalled" and "failed" matters: a stall is recoverable and the drain thread remains alive; a failed writer will never write again and should be replaced.

Design Constraints

The writer is intentionally not a general-purpose logging framework. Its constraints follow from the console's role:

1. Console output must be best-effort. A shell prompt that cannot reach the UART is annoying; a shell prompt that blocks the whole system because the UART is dead is a failure. Dropping the tail of a large cat is acceptable, blocking the command that produced it is not.
2. No unbounded buffering. An unbounded queue would merely move the failure from "blocked thread" to "out of memory" under a stalled port. 8192 characters is large enough for interactive output and small enough to bound the damage.
3. No synchronous flush. flush() is a no-op because a flush that waits for a stalled port reintroduces the original bug. Callers that need ordering must not assume flush() waits.
4. Truncation must be visible. Silent drops make a boot log look complete when it is not. The marker names the exact number of characters lost and stays attached to the chunk that lost them.
5. One failure latch. After a port exception, retrying every write would burn CPU and produce a stream of identical log errors. The latch converts one fault into one diagnostic and cheap no-ops afterwards.
6. Shutdown must not hang. A console teardown path runs while the system is already under stress. A 1 s join bound is a deliberate compromise: enough time to drain a short prompt, short enough not to stall shutdown on a dead socket.

These constraints also explain why MAX_PENDING_CHARS and CLOSE_JOIN_TIMEOUT_MS are package-private constants rather than configuration: the test fixture reads them directly, and making them configurable would allow a caller to reintroduce an unbounded wait.

Observing the Writer

There is no shell command that dumps the writer's counters; the package-private accessors exist for tests and for a debugger:

writer.getPendingCount();  // characters still queued
writer.getDroppedChars();  // cumulative characters discarded
writer.isFailed();         // true after an unrecoverable port exception
writer.getDrainThread();   // the daemon thread, for liveness checks

A useful diagnostic sequence on a live system:

1. Check getDrainThread().isAlive(). If false, the writer is either closed or latched failed; inspect isFailed().
2. If the drain thread is alive but getPendingCount() stays at MAX_PENDING_CHARS, the port is stalled rather than failing.
3. If getDroppedChars() grows while pending stays full, producers are outrunning the port and output is being truncated by design.
4. If getPendingCount() is zero and getDrainThread() is alive, the writer is idle and healthy.

Because the fields are guarded by lock, these reads are consistent snapshots, but they are not atomic with respect to each other: a concurrent drain can clear the queue between two calls. Treat them as indicators, not as a transactional view.

Test Coverage

AsyncSerialConsoleWriterTest contributes 7 host-runnable tests:

Test	Pins
testWriteDoesNotBlockOnStalledPort	write() returns in under 2 s while the drain thread is stuck in a port write
testQueueIsBoundedAndDropsExcess	getPendingCount() <= MAX_PENDING_CHARS and getDroppedChars() > 0 after overflow
testOversizedOutputRetainsTruncationMarker	exact marker count and that the marker survives at the tail as chars ...]\r\n
testSmallOutputIsDeliveredWithoutTruncationMarker	full delivery, zero drops, no marker
testCloseIsBoundedAndDrainExitsAfterPortRelease	bounded close(), drain thread alive while stalled, exits after release
testDrainResumesWhenPortRecovers	queued hello\nworld arrives as !hello\r\nworld
testThrowingPortDisablesWriterWithAccounting	failed latch, post-failure drops, empty pending queue

The fixture classes StubSerialPort, BlockingSerialPort and ThrowingSerialPort live inside the test file and implement SerialPortAPI directly, so the tests need no UART hardware and run on the build host.

Gotchas

• Dropping is the contract, not an error path. A stalled port cannot apply backpressure to the console without blocking the shell, so the queue is bounded and the excess is discarded. getDroppedChars() is the only accounting.
• The truncation marker is generated at drain time, not at write time. Several small writes can accumulate drops before one marker is emitted; the marker reports the total carried by that chunk.
• close() does not guarantee the queue is empty when it returns. It waits at most one second for the drain thread. Output that cannot be written because the port is stalled is abandoned rather than allowed to block shutdown.
• A write racing with close() may be lost. write() does not check closed: it appends and notifies unconditionally. If the drain thread has already seen closed and the queue was empty, it exits and the late characters are never written. Stop producers before calling close().
• write(int) allocates a one-character array. This is the console API's convenience path, not a hot bulk path.
• The writer depends on a non-null SerialPortAPI. The constructor throws IllegalArgumentException("null serial port") immediately.
• CRLF conversion belongs to the writer. Callers can and do queue plain \n; do not pre-convert or the wire will receive \r\r\n.
• A failed writer never recovers. failed is one-way and there is no reopen; construct a new writer around a replacement port instead.

Terminology

• Producer — any thread calling write(). Producers never touch the UART.
• Drain thread — the single daemon that owns writeChunk().
• Chunk — the snapshot of the pending buffer taken at one drain iteration, optionally with the marker appended; writeChunk() writes it character by character.
• Marker — the synthetic truncation line, emitted with the chunk that follows the loss.
• Latch — the sticky failed flag that converts repeated errors into a single diagnostic.

Related Pages

• Serial-Port-Driver — Parent hub: UART registers, SerialPortAPI, agent console plugin
• JNode-Serial-Skill — Driving the agent console and KDB over serial
• Core-Test-Suite — The suite that runs AsyncSerialConsoleWriterTest
• Agent-Issue-Pipeline — Why a stalled agent console can otherwise wedge a ticket run
• Testing — Host-JVM unit-test conventions