Repository navigation
Add experimental per-trace and batched mode refinement - #4
jimsweeney wants to merge 16 commits into
Conversation
f052bdf to
647fa99
Compare
|
Important Review skippedAuto reviews are limited based on label configuration. 🏷️ Required labels (at least one): 1
Please check the settings in the CodeRabbit UI or the ⚙️ Run configuration
You can disable this status message by setting the Use the checkbox below for a quick retry:
Comment |
|
@coderabbitai full review |
|
647fa99 to
67887d4
Compare
7d9c016 to
4150197
Compare
|
/ok to test 4150197 |
a873a86 to
f01f5a7
Compare
0a8ca02 to
824edd0
Compare
Adds cuphoton.xray.synthetic_validation: a fixture of known damped modes (the synthetic_trace modes plus white Gaussian noise), Cramer-Rao bounds from the exact Fisher information of the same model, a Monte Carlo sweep that reports per-mode bias, standard deviation and rmse of every parameter against the bound together with the mode-loss rate, optional model-mismatch traces with a residual-to-noise ratio, and a schema_version 1 summary (config, runtime via runtime_metadata, results per condition with attempted/successful/failed trials, artifacts). The linear-prediction-validate (lpv) command runs the sweep and, with --output-dir, writes summary.json and a Bokeh figure outside the checkout. Documented in docs/xray/LINEAR-PREDICTION-VALIDATION.md. The sweep is the reproducer for the mode loss discussed in NVIDIA#1: a lightly damped mode whose fitted decay crosses zero under noise is dropped by the root filter. No default behaviour changes. Refs NVIDIA#1 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014wyj5iPSHmtR3vSTvJWKfw Signed-off-by: James Sweeney <jmsweene@gmail.com> Signed-off-by: Trent Nelson <trentn@nvidia.com>
Keep small-trial summaries valid JSON, integrate the requested chirp, and retain floating parameters for Fisher derivatives. Match nearby modes globally and reject incompatible combined sweep settings. Signed-off-by: Trent Nelson <trentn@nvidia.com>
Use the same amplitude and phase convention for truth, fitted modes, and summary metadata so equivalent modes have zero parameter error. Signed-off-by: Trent Nelson <trentn@nvidia.com>
Keep text formatting separate from command setup and execution. Signed-off-by: Trent Nelson <trentn@nvidia.com>
Reject invalid numerical inputs before trials and distinguish estimator exceptions from mode loss in the text report. Record revisions only for tracked source checkouts, together with tracked- file dirtiness. Embed plot resources for offline use without changing Bokeh output state. Clarify numerical Fisher bounds and conditional scatter, and anchor the Gaussian distortion envelope at the first sample. Signed-off-by: Trent Nelson <trentn@nvidia.com>
Report unidentifiable modes with the validation error context. Clarify that the frequency standard-deviation bound uses the square root of the closed-form variance. Signed-off-by: Trent Nelson <trentn@nvidia.com>
Signed-off-by: Trent Nelson <trentn@nvidia.com>
Give probe, validation, and benchmark results their own text formatting. Keep batch aggregation and validation orchestration in the owning modules, with commands routing reports through their injected output. Signed-off-by: Trent Nelson <trentn@nvidia.com>
Use analytic derivatives and scaled singular values to reject unidentifiable model parameters before reporting Cramer-Rao bounds. Normalize frequency signs with phases so equivalent traces report the same recovery and parameter statistics. Signed-off-by: Trent Nelson <trentn@nvidia.com>
Keep recovered-mode count and frequency error ahead of decay error. Compare assignment costs exactly so decay only resolves frequency ties, including reversed modes and extra fitted candidates. Signed-off-by: Trent Nelson <trentn@nvidia.com>
cuphoton.xray.mode_refinement refines the damped-mode model by nonlinear least squares from the linear-prediction result (analytic Jacobian, scipy.optimize.least_squares), seeding a mode the root filter dropped from the residual spectrum; RefinedModes carries Jacobian uncertainties and the residual rms. cuphoton.xray.mode_refinement_batched runs the same Levenberg-Marquardt for a batch of traces on NumPy or CuPy and matches the per-trace solver to about 1e-8. linear-prediction-validate gains --refine, which adds the refined estimator to the sweep and the summary; linear-prediction-refine-benchmark (lprb) times the SciPy loop against the batched solver on NumPy and CuPy (best of --repeat, transfers excluded, warm-up call). examples/xray_lp_modes_review.py writes a Bokeh review page of traces, reconstructions and modes. Documented in docs/xray/LINEAR-PREDICTION-REFINEMENT.md with RTX 4050 numbers. Overlaps the fit-diagnostics, iterative-fitting and GPU-batching work the maintainers describe in NVIDIA#1; posted so the overlap can be sorted out. Refs NVIDIA#1 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014wyj5iPSHmtR3vSTvJWKfw Signed-off-by: James Sweeney <jmsweene@gmail.com> Signed-off-by: Trent Nelson <trentn@nvidia.com>
Keep LP modal parameters relative to the first sample during refinement. Recognize stationary starts and use positive diagonal damping so a zero-amplitude mode does not make the batch solve singular. Signed-off-by: Trent Nelson <trentn@nvidia.com>
Normalize Jacobian columns for batched solves and require a small gradient to converge. Estimate local uncertainty with a rank-aware SVD and report unavailable uncertainties for rank-deficient fits. Signed-off-by: Trent Nelson <trentn@nvidia.com>
Keep the reproducible harness and explain the experimental fit scope, local uncertainty assumptions, and convergence limits. Remove fixed-run accuracy statistics and laptop timing tables. Signed-off-by: Trent Nelson <trentn@nvidia.com>
Signed-off-by: Trent Nelson <trentn@nvidia.com>
Reject malformed fit inputs, seed from complete residual traces and keep reported models consistent with canonical phases. Report convergence alongside fit comparisons. Expose offline mode review through the CLI. Exercise XRay CPU/CuPy parity in the existing GPU job. Signed-off-by: Trent Nelson <trentn@nvidia.com>
824edd0 to
f342338
Compare
| jac = xp.where(accept[:, None, None], jac_t, jac) | ||
| cost = xp.where(accept, cost_t, cost) | ||
| lam = xp.where(accept, lam / 3.0, lam * 5.0) | ||
| _sync(xp) |
There was a problem hiding this comment.
Please check convergence after the final accepted step and add a regression at the iteration limit. The returned parameters and residual include that step, but converged still describes the previous iterate. The example below returns identical parameters for max_iter=5 and 6, yet reports False and True despite both satisfying the stopping criterion. Recompute the final stationarity flag before returning.
Minimal reproducer
Run against this revision with NumPy 2.5.2 and SciPy 1.18.0.
import numpy as np
from cuphoton.xray.mode_refinement_batched import (
refine_modes_batched, model_and_jacobian_batched,
)
from cuphoton.xray.synthetic_validation import synthetic_modes_trace, modes_to_theta
f = synthetic_modes_trace(96)
x = modes_to_theta(f.modes, f.constant)[None, :]
x[:, [2, 6]] *= 1.05
fits = []
for limit in (5, 6):
fit = refine_modes_batched(np, f.time, f.clean[None, :], x, 2, max_iter=limit)
model, jac = model_and_jacobian_batched(np, fit.theta, f.time, 2)
r = model - f.clean
scale = np.linalg.norm(jac, axis=1)
j = jac / np.where(scale > 0, scale, 1)[:, None, :]
g = np.max(np.abs(np.einsum('bnp,bn->bp', j, r)))
expected = g <= 1e-9 * max(np.linalg.norm(r), 1)
print(limit, bool(fit.converged[0]), bool(expected), f'{g:.3e}')
fits.append(fit)
print('same parameters:', np.array_equal(fits[0].theta, fits[1].theta))Observed (limit, returned flag, expected flag, gradient):
5 False True 5.650e-11
6 True True 5.650e-11
same parameters: True
The first returned flag should also be True.
melo-gonzo
left a comment
There was a problem hiding this comment.
Please fix the final-step convergence status before merging. The returned fit can satisfy the documented stopping criterion while reporting converged=False.
|
Fix for the final-step convergence flag is on fix-refine-final-convergence in my fork (one commit on top of f342338): it rechecks stationarity on the returned iterate when max_iter is exhausted, plus a regression at the iteration limit using Carmelo's case. @tpn, feel free to cherry-pick it, or I can push it to this branch if you prefer. |
Adds optional nonlinear refinement for XRay linear-prediction modes, with residual-spectrum initialization for missing modes and batched NumPy/CuPy solves. Fit results include reconstruction, convergence and local uncertainty estimates; benchmark reports retain convergence counts and residual quality. The
lpmrcommand writes an offline comparison page with the actual noise-free input, fitted modes and solver status.The tools use an unconstrained damped-mode model and remain experimental, alongside the existing constrained production fitter. The guide explains initialization, uncertainty assumptions, convergence limits and timing boundaries. Validation supplies a known mode count; production detector acceptance and end-to-end speedup remain separate work. Includes the corrected diagnostics from #2, which should land first.
Validation: 2,970 full CPU tests, 69 focused refinement/visualization tests including CuPy, and all nine CI-selected GPU cases passed. An independent numerical review and a 114-trace comparison covered noise, close/coincident modes, difficult starts, distortions and long records; expected nonconvergence and local minima are retained in the results. Full lint/hooks and an isolated final-wheel CLI/offline-render smoke passed. The existing GPU CI job now includes three XRay time-scale parity cases.
Refs #1, #2.