Skip to content

MovingBoundary: allow local (quick) runs - #2155

Merged
jcschaff merged 8 commits into
masterfrom
solvers/movingboundary-local-run
Oct 2, 2026
Merged

jcschaff merged 8 commits into
masterfrom
solvers/movingboundary-local-run

Conversation

@jcschaff

@jcschaff jcschaff commented Oct 1, 2026 •

Copy link
Copy Markdown
Member

Lets MovingBoundary simulations run locally on the desktop (quick run), and makes MovingBoundary results work in the field viewer ("View in 3D"), for local and server runs alike. Before this PR the field viewer had never displayed a MovingBoundary result anywhere.

Deploy note: server-run MovingBoundary results display in the field viewer only after this is merged and a dev release deploys the data server. The dev data container has no vcell.vtk.pythonDir, so today every server-run MovingBoundary /grid or probe fails after about 37 s with DataRequestQueue:getEmptyVtuMeshFiles() … required System property "vcell.vtk.pythonDir" not defined.

1. Allow local runs

The desktop's local-run button was always disabled for MovingBoundary simulations.

  • Cause: SolverDescription.MovingBoundary was declared with Feature_ServerOnly in 2018 (34a571a, "Temporarily disable blue button running for Moving Boundary solver"). SimulationListPanel.canQuickRun() rejects such solvers before looking for the executable.
  • Why it's safe to remove now: since 8.2.0.08 (B2: desktop build downloads each solver from its own vcell-* repo release #2142), the desktop ships MovingBoundary_x64 from virtualcell/vcell-mbsolver v1.0.5 for macOS (universal), Linux and Windows. MovingBoundary was the only solver with this flag, and nothing else uses it.

2. A pure-Java MovingBoundary VTU writer, used everywhere

The field viewer reads MovingBoundary meshes through the VTU seam (getEmptyVtuMeshFiles / getVtuMeshData). The .vtu and .movingboundaryindex behind that seam were written only by the Python VTK service, and neither the desktop nor the dev data server has it.

  • org.vcell.vis.vtk.VtuWriter writes a VisMesh volume grid (polygons, voxels, tetrahedra) as a VTK XML UnstructuredGrid.
    • The form is the restricted one the existing readers take: one piece, LittleEndian, UInt32 header, inline uncompressed base64, empty PointData/CellData.
    • Cells come in the order the Python getVolumeVtkGrid() inserts them. Points are Float64; Python rounds them to Float32.
  • VtkService.writeMovingBoundaryVtkGridAndIndexData is now a concrete base-class method. It uses that writer plus the same thrift MovingBoundaryIndexData. The Python service no longer overrides it, so server and desktop both use Java, with no fallback to Python. The grid is just points and polygons, so VTK has nothing to compute, and the output is equivalent (see below).
  • Local data directory: a LocalVCDataIdentifier (quick run) uses its own directory, not the server-only vcell.primarySimdatadir.internal. Every other identifier takes exactly the path it took before.

Still on Python (follow-ups, unchanged here):

  • Chombo volume and membrane. Stored Chombo results reach the field viewer through the same seam, so they will also fail on a data server without Python. Moving them to Java needs VTK_POLYHEDRON face streams and a Chombo equivalence fixture.
  • Comsol.
  • Finite-volume "smoothed" (VTK windowed-sinc smoothing). The field viewer does not use it; it reads FV through the raw Cartesian path.

3. MovingBoundary reads: about 1.1 s → about 15 ms per saved time

Every MovingBoundary VTU request called MovingBoundaryReader.getMeshInfo(). That reads elements/startX, hx, numX, … (and the same for Y), which are attributes of the compound [time][x][y] elements dataset. MovingBoundaryVH5Path.walk() tried compound members first, and reading a member decodes the whole dataset across every saved time. So each call did 8 full decodes before finding the attributes. walk() now decodes the compound only for a name its type lists as a member. The results are the same.

Measured on a local run of Solver Suite 6.2 "2D kinematics analytic" Simulation33 (386 saved times, 31×31, 49 MB .h5), through DataSetControllerImpl:

before after
per saved time (mesh + data) 1103 ms 23 ms (12–18 ms steady; about 15 ms cold, including writing .vtu and index)
probe series, 386 times 496 s (meshes already written) 9.0 s cold
kymograph, 386 times (20 times took 20.5 s) about 4.9 s

This is also where the 37 s before each server-side 500 went: the whole result was decoded before the pythonDir check failed.

4. Field viewer: no more spurious "busy"

Heavy jobs (a kymograph, or several probes over a Chombo or MovingBoundary run) run one at a time, and a second one got an immediate 503. The viewer itself sends two at once: a variable switch asks for the probes' series and the kymograph together. So with probes and a line open, one of them always failed (probe time series failed: … busy), and the probe series had no retry.

  • Now one more heavy request waits for the running one, up to vcell.fieldViewer.heavyWaitMillis (default 30 s). Anything beyond that single waiter still gets an immediate 503, so a long job still can't pile work up behind it.
  • The viewer retries a 503 probe series once after a pause, as it already did for a kymograph.

Equivalence against the Python service

MovingBoundaryVtuPythonEquivalenceTest runs python_vtk (mesh type movingboundary) and the Java writer on every saved time of a real mbsolver result. For every time:

  • identical point and cell counts, cell types and connectivity;
  • points equal to Float32 precision (max |Δ| 3.9e-7);
  • byte-identical index files.

It runs in the Fast CI job, which installs pythonVtk.

Tests

  • vcell-core
    • MovingBoundaryVtuWriterTest runs the writer seam end to end on a real result, with no Python. At every saved time it checks that the cells are exactly the solver's inside and boundary elements, the index maps each cell back to its element, the data lands on the right cells, and writeCellDataToVtu accepts the file.
    • MovingBoundaryVtuPythonEquivalenceTest (above).
    • MovingBoundaryServerVtuPathTest drives DataServerImpl → DataSetControllerImpl the way the data server does:
      • a non-local identifier, vcell.primarySimdatadir.internal set, vcell.vtk.pythonDir unset;
      • meshes and data at every saved time, under 5 s each;
      • on the PR's base it fails with the dev server's exact error.
    • MovingBoundaryVH5PathTest: reading the eight mesh attributes decodes elements 0 times (8 before the fix), and a real member is still read from the data.
  • vcell-client
    • FieldViewerServerMovingBoundaryLocalRunTest uses a real locally run result (biomodel_165181964.vcml Simulation0, MovingBoundary_x64 1.0.5, coarsened 12×12, 107 KB). It reads it the way the desktop reads a quick run, with neither server property set, and checks /info, /grid at two times, /timeseries (inside and outside), /kymograph, and a probe series plus kymograph sent together, which both return 200. Without the fixes it fails with the same 500 / 503 the end-to-end run hit.
    • FieldViewerServerMovingBoundaryTest: a waiter is served when the running job ends, a third request is turned away at once, and a waiter gives up after the wait. The other busy tests now hold both slots.
  • webapp-viewer (Playwright, run by hand per its README): a new probe busy-retry test, and the whole suite passes on Chromium, WebKit and Firefox (216 tests).

End to end in the desktop client

Built with mvn clean install dependency:copy-dependencies -DskipTests and launched through tools/debug-bridge/launch-client.sh against the dev server. Both models were opened from local files and quick-run; nothing was saved.

  • biomodel_165181964.vcml Simulation0 (expanding disc):
    • View in 3D loads and /grid returns 200 at every time;
    • the disc grows from r = 1 to about 1.2 as the time slider steps;
    • the probe trace and the kymograph render.
  • Solver_Suite_6_2.vcml Simulation33 (moving disc, 386 saved times):
    • renders at t = 0, 0.49 and 1;
    • a probe placed while a kymograph is open, then a variable switch that refetches both: every request returned 200 and no error appeared.
  • The run directories hold .vtu and .movingboundaryindex files and no .visMesh files.

Known limits

  • The MovingBoundary membrane domain is still not written. It wasn't before either (DomainType.MEMBRANE is unimplemented).
  • A request the browser abandons (a superseded probe or line) still runs to the end on the server while it holds the heavy-job slot. That is now seconds rather than minutes.

🤖 Generated with Claude Code

MovingBoundary was marked Feature_ServerOnly in 2018 (34a571a, "Temporarily
disable blue button running for Moving Boundary solver"), so the desktop's
local-run button was always disabled for it. Since 8.2.0.08 the desktop ships
MovingBoundary_x64 from virtualcell/vcell-mbsolver for macOS (universal),
Linux and Windows, so the flag is stale. It was the only server-only solver.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@jcschaff

jcschaff commented Oct 1, 2026

Copy link
Copy Markdown
Member Author

End-to-end test on macOS (arm64), desktop client built from this branch

Step Result
Quick-run button enabled for MovingBoundary PASS (log: canQuickRun(): solver MovingBoundary supported for local computation)
Local run completes PASS: biomodel_165181964 Simulation0 (40×40) and Solver Suite 6.2 Simulation33 (31×31, 386 time points), about 2 s each, using localsolvers/mac64/vcell-mbsolver/MovingBoundary_x64 (v1.0.5)
Desktop results viewer PASS: the moving front and the species display at every time point checked, with no exceptions
Browser field viewer ("View in 3D") FAIL: /info returns 200, but /grid returns 500
Negative control (before this change) PASS: the button was disabled; log server-only feature required

Known gap, not caused by this change: the field viewer's MovingBoundary mesh path has never worked for a local run.

  • FieldViewerServer.mbGrid → DataSetControllerImpl.getEmptyVtuMeshFiles / getVtuMeshData require the server-only property vcell.primarySimdatadir.internal.
  • After that, MovingBoundaryVtkFileWriter → VtkServicePython needs vcell.vtk.pythonDir and the Python python_vtk module, which the desktop doesn't ship.

The field viewer's MovingBoundary tests stub this out, which is why it was never caught.

Options:

  • (a) Disable "View in 3D" for local MovingBoundary results with a clear message.
  • (b) Write the MovingBoundary mesh files in Java.

🤖 Generated with Claude Code

jcschaff and others added 7 commits October 1, 2026 19:22
The MovingBoundary .vtu (and its .movingboundaryindex) was the one VTU the
field viewer needs that only the Python VTK service could write, so a
desktop quick run - which has no vcell.vtk.pythonDir and no python_vtk -
could not be opened in "View in 3D".

The grid is just the mesh's points and polygons: nothing for VTK to
compute. VtuWriter writes it as a VTK XML UnstructuredGrid in the restricted
form the existing readers take (one piece, LittleEndian, UInt32 header,
inline uncompressed base64 arrays, empty PointData/CellData), with cells in
the order the Python getVolumeVtkGrid() inserts them. VtkService now writes
MovingBoundary grids with it for every service, server included: the
output is equivalent, and Python adds nothing there but a subprocess.
Chombo, finite-volume (surface smoothing) and Comsol stay on Python.

Tests, on the real mbsolver fixture moving-boundary-2d.h5, every saved time:
- MovingBoundaryVtuWriterTest: the writer seam end to end with no Python -
  one cell per inside/boundary element with exactly its boundary points,
  the index maps cells to elements, getVtuMeshData lands each value on
  its cell, VisMeshUtils.writeCellDataToVtu accepts the file.
- MovingBoundaryVtuPythonEquivalenceTest: against python_vtk itself (the
  Fast CI job installs pythonVtk; skipped where it is missing) - identical
  cell counts, types and connectivity, points equal to Float32 precision
  (Python stores Float32; max |dx| 3.9e-7), byte-identical index files.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…operties

getEmptyVtuMeshFiles(MovingBoundarySimFiles, ...) and the MovingBoundary
getVtuMeshData read vcell.primarySimdatadir.internal, which exists only on
the server, so on the desktop the field viewer's /grid failed with
"required System property ... not defined". A LocalVCDataIdentifier (a
quick run) now uses its own local directory; any other identifier takes
the same server path as before.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
FieldViewerServerMovingBoundaryLocalRunTest reads a real MovingBoundary
result the way the desktop reads a quick run (DataSetControllerImpl behind a
LocalDataSetController, a LocalVCSimulationDataIdentifier), so every mesh is
built by MovingBoundaryVtkFileWriter and the Java VTU writer, with neither
the server data directory nor the Python VTK service configured: /info,
/grid at two times (different geometry), /timeseries inside and outside
the disc, and a kymograph. Without the two preceding commits it fails with
the same HTTP 500 the end-to-end run hit.

The fixture is biomodel_165181964.vcml Simulation0 (an expanding disc) run
locally by MovingBoundary_x64 1.0.5 on a coarsened 12x12 grid to t = 0.2,
HDF5 repacked with gzip (107 KB); its mb.xml input is kept beside it.

Also drops the comments that called MovingBoundary server-only and its .vtu
Python-written.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… past *.log ignore)

The log is the run's data index (MBSData: one line per saved time); the
other fixture runs' logs are force-added the same way.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…alars

MovingBoundaryReader.getMeshInfo() reads elements/startX, endX, hx, numX
(and Y): attributes of the compound [time][x][y] "elements" dataset.
MovingBoundaryVH5Path.walk() tried the compound members first, and reading
a member means ds.getData() - decoding every saved time's cells - before
falling through to the attribute. Eight full decodes per getMeshInfo(), and
getPlane() calls it, so every MovingBoundary VTU request (getVtuMeshData,
and a cold getEmptyVtuMeshFiles) paid them.

walk() now decodes the compound only for a name its type actually lists
as a member; anything else goes straight to the attributes. Same results.

On a local run of Solver Suite 6.2 "2D kinematics analytic" Simulation33
(386 saved times, 49 MB .h5), per saved time through
DataSetControllerImpl (mesh + data): 1103 ms before, 23 ms after (12-18 ms
steady; ~15 ms cold, including writing the .vtu and index). A probe over
all 386 times went from ~8 min to a few seconds.

MovingBoundaryVH5PathTest pins it: the eight mesh attributes are found
with zero full decodes of "elements" (eight before this change), and a
real member is still read from the data.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…iling busy

Heavy jobs (a kymograph; several probes over a Chombo or MovingBoundary
run) run one at a time, and a second one got an immediate 503. The viewer
itself sends two together - a variable switch asks for the probes' series
and the kymograph at the same moment - so with probes and a line open one
of them always failed ("probe time series failed: ... busy"), and the
probe series had no retry at all. Slow MovingBoundary reads (fixed in the
previous commit) made the window minutes long.

Now one more heavy request may wait for the running one, up to
vcell.fieldViewer.heavyWaitMillis (30 s by default); anything beyond that
single waiter still gets a 503 at once, so a long job still cannot pile
work up behind it. In the viewer, a probe series answered 503 is retried
once after a pause, as a kymograph already was.

Tests:
- FieldViewerServerMovingBoundaryTest: the waiter is served once the
  running job ends while a third request is turned away at once; a waiter
  gives up with 503 after the wait.
- FieldViewerServerMovingBoundaryLocalRunTest: a probe series and a
  kymograph sent together on the real local run both answer 200 (one of
  them was 503 before this change).
- The busy tests now hold both the running and the waiting slot.
- webapp-viewer test_probes: a busy /timeseries is retried once.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The dev data server (the "data" container) has no vcell.vtk.pythonDir, so
DataRequestQueue:getEmptyVtuMeshFiles failed for every server-run
MovingBoundary result - after decoding the whole .h5 first, which is where
the ~37 s before each 500 went (fixed two commits back).

MovingBoundaryServerVtuPathTest drives DataServerImpl over a
DataSetControllerImpl the way the data server does: a non-local
VCSimulationDataIdentifier, the run under
vcell.primarySimdatadir.internal/<owner>/, vcell.vtk.pythonDir unset.
isMovingBoundary, getVtuVarInfos, getDataSetTimes, and
getEmptyVtuMeshFiles + getVtuMeshData at every saved time: one value per
served cell, under 5 s a time, meshes and indices written where the server
always wrote them, no .visMesh. On the PR's base it fails with the same
"required System property vcell.vtk.pythonDir not defined" the dev server
returns.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@jcschaff
jcschaff merged commit 4079674 into master Oct 2, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant