Skip to content

Harden the orbfit C library: sanitizer CI, precision, leaks, error handling - #26

Open
ijiraq wants to merge 7 commits into
OSSOS:masterfrom
ijiraq:cursor/c-code-hardening
Open

ijiraq wants to merge 7 commits into
OSSOS:masterfrom
ijiraq:cursor/c-code-hardening

Conversation

@ijiraq

@ijiraq ijiraq commented Oct 6, 2026 •

Copy link
Copy Markdown
Member

Summary

This hardens the C orbit-fitting library that mp_ephem.BKOrbit calls through ctypes. Results change slightly because RA/Dec are now parsed in double precision, so this should be released as a major version. All other changes leave the fitted numbers bit-for-bit the same.

  • CI and tests: a new c-checks job compiles every C file with -std=gnu2x -Wall -Wextra -Werror, runs a standalone C driver under AddressSanitizer, UBSan and LeakSanitizer, and runs pytest against a sanitizer build. New tests pin the o3o08 fit, check error reporting and thread safety, and check the exported symbols.
  • Precision: dmsdeg parsed sexagesimal RA/Dec through float, which caused errors up to 0.05 arcsec. It now uses double and agrees with astropy to about 1e-10 arcsec. .abg files are written with %.17g. The o3o08 semi-major axis moves by 0.0018 au and the 2016 RA prediction by 0.22 arcsec, both well inside the fit uncertainties. Pinned values in test_orbit.py were updated.
  • MPC parsing: the second line of satellite observations lost signs and rejected +. Dates, RA/Dec and observatory codes are now read by column.
  • Memory and undefined behaviour: fixes leaks in every entry point (93 leaked blocks per three fits before, none now), an uninitialized observation pointer, missing MAXOBS bounds, unchecked fopen/fread, NULL derivative pointers in kbo2d/kbo3d, wrong partials in ec_to_proj, acos domain errors in orbitElements, and pointer arithmetic before the start of an allocation in nrutil.
  • No more exit(): errors inside the library used to end the whole Python process. Each entry point now sets a setjmp guard. A failure frees all live allocations, resets the integrator and returns NaN, and BKOrbitError is raised with the library's message (from orbfit_last_error()). The ephemeris reader checks its reads and seeks to records by absolute position.
  • Threads: the library keeps its state in static storage and ctypes releases the GIL, so bk_orbit.py now loads the library once and holds a lock around every call.
  • Symbols: the library is built with -fvisibility=hidden. Only the ctypes entry points, the set_* functions and utc_jd_to_tt are exported, so generic names such as gaussj and dvector no longer leak into the process. File-local helpers are static, and dead code (read_options, accel, write_obs, degdms, deghms, ephem_read.h) is removed.
  • C23: nrutil.c is converted from K&R definitions to ANSI. GCC 15 only warns on K&R definitions, but clang in C23 mode rejects them.

Three numerical questions that need a domain decision rather than a code fix are in #25.

Test plan

  • pytest test: 36 passed (macOS, Python 3.12)
  • Strict -Werror compile with clang (macOS) and with GCC 15 (gcc:15 Docker image), including -O2
  • C driver on o3o08 and hst, including the failure paths, under ASan/UBSan (clang) and LeakSanitizer (GCC 15); leaks --atExit reports 0 leaks
  • Snapshot of fit, prediction and heliocentric outputs for four test files is bit-identical across every phase except the precision change
  • GitHub Actions: c-checks and the Python 3.12-3.14 builds

ijiraq and others added 7 commits October 5, 2026 20:08
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
dmsdeg accumulated in float, giving up to 0.05 arcsec of RA error; results
for short arcs move by a fraction of their quoted uncertainty.  The 80-column
MPC reader also dropped the last RA/Dec digit and lost the signs of satellite
positions on two-line records.

Co-authored-by: Cursor <cursoragent@cursor.com>
Free all work matrices in predict, predict_helio, mrqfit and on error paths;
return NaN rather than stale results on failure; check fopen; bound the
observation count; initialise pivots in ludcmp/gaussj; fix the kbo2d/kbo3d
NULL checks and ec_to_proj partials; keep orbitElements finite for circular
and ecliptic orbits.  A degenerate linear fit now falls back to the
energy-constrained fit instead of exiting.

Co-authored-by: Cursor <cursoragent@cursor.com>
The C library no longer calls exit().  Errors deep in a calculation
(out-of-range ephemeris dates, Numerical Recipes failures, bad
observatory files, unbound orbits in elements_to_xv) call orbfit_fail(),
which longjmps back to a guard in each entry point.  The guard frees all
Numerical Recipes allocations (now tracked in a list), resets the orbit
integrator, and returns NaN results.  Input errors are recorded with
orbfit_error() and propagated through return codes.  The message is
available from orbfit_last_error().

The ephemeris reader checks its reads, can be retried after a failed
initialization, and seeks to records by absolute position.

bk_orbit.py loads the library once, raises BKOrbitError with the
library's message, and holds a module lock around every call because
the library keeps its state in static storage.

Remove the unused read_options() command-line parser.

Co-authored-by: Cursor <cursoragent@cursor.com>
Build with -fvisibility=hidden and mark the entry points used through
ctypes (and utc_jd_to_tt, used by the tests) with ORBFIT_API in
orbfit_api.h, so the generic Numerical Recipes names and other internals
cannot collide with other libraries loaded in the Python process.  Make
file-local helpers and state static, remove the unused accel() and
write_obs(), and send the ephemeris debug output to stderr.

Co-authored-by: Cursor <cursoragent@cursor.com>
The .abg file is now written at full precision, so a fit on Linux x86-64
differs from the committed macOS-generated file by up to ~4e-13 relative
instead of being hidden by 8-decimal rounding.

Co-authored-by: Cursor <cursoragent@cursor.com>
@ijiraq

ijiraq commented Oct 6, 2026

Copy link
Copy Markdown
Member Author

@jmpetit this PR on the liborbfit attempts to address a number of issues in the library, likely things you would be concerned about for space based observations. Would you like to review the code changes?

@jmpetit

jmpetit commented Oct 9, 2026 •

Copy link
Copy Markdown

I had a look at the proposed changes, they look good to me.

I tried to install and run the code. All unittests in test/ ran fine, once I understood how to run them.

Since the only "documentation" is the README file, that contains very little indications, I decided to run the srcipt/example.py script. As is, it doesn't work. It was written in python2, and the print statement uses the old "statement" format rather than the new function format.

When I changed the call to print in the code on my computer, I ran it with all test/data/*.mpc files, almost fine.

In the prediction, when changing time from end of 2028 to start of 2029, get warning from ERFA function "taiutc":

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
/home/petit/Research/JPL_ephemerides/src/OSSOS/liborbfit-pr26/.venv-pr26/lib/python3.12/site-packages/erfa/core.py:133: ErfaWarning: ERFA function "taiutc" yielded 1 of "dubious year (Note 4)"
warn(f'ERFA function "{func_name}" yielded {wmsg}', ErfaWarning)
/home/petit/Research/JPL_ephemerides/src/OSSOS/liborbfit-pr26/.venv-pr26/lib/python3.12/site-packages/erfa/core.py:133: ErfaWarning: ERFA function "utctai" yielded 1 of "dubious year (Note 3)"
warn(f'ERFA function "{func_name}" yielded {wmsg}', ErfaWarning)
/home/petit/Research/JPL_ephemerides/src/OSSOS/liborbfit-pr26/.venv-pr26/lib/python3.12/site-packages/erfa/core.py:133: ErfaWarning: ERFA function "d2dtf" yielded 1 of "dubious year (Note 5)"
warn(f'ERFA function "{func_name}" yielded {wmsg}', ErfaWarning)
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

Probably nothing we can do, just annoying. It's because this is too far in the future.

Print(orb.summarize()) has a weird output, besides missing some carriage returns:

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
FbAS2001 distance a e Inc. Node peri. on JD 2022-08-31 08:34:44.544
fit 42.80 AU 44.31 AU 0.02 2.24 deg 14.36 deg 313.69 deg
uncertainty 2.49 AU 22.90 AU 0.56 0.58 deg 12.66 deg 2058.37 deg
arc 2.12 d ephemeris uncertainty: 6746.73 arcsec 127.09 arcsec 68.12 deg

2022 08 31.35746 -0.08 +0.09 # FbAS2001 C2022 08 31.35746 22 24 20.536-11 34 27.54 568
2022 08 31.41996 +0.05 -0.01 # FbAS2001 C2022 08 31.41996 22 24 20.246-11 34 28.99 568
2022 08 31.48246 +0.18 -0.09 # FbAS2001 C2022 08 31.48246 22 24 19.956-11 34 30.45 568
2022 09 01.37455 -0.17 +0.11 # FbAS2001 C2022 09 01.37455 22 24 16.049-11 34 52.70 568
2022 09 01.43705 -0.09 +0.01 # FbAS2001 C2022 09 01.43705 22 24 15.763-11 34 54.14 568
2022 09 01.49955 -0.03 -0.09 # FbAS2001 C2022 09 01.49955 22 24 15.478-11 34 55.58 568
2022 09 02.34810 +0.16 -0.21 # FbAS2001 C2022 09 02.34810 22 24 11.737-11 35 16.36 568
2022 09 02.41060 +0.05 -0.01 # FbAS2001 C2022 09 02.41060 22 24 11.464-11 35 18.10 568
2022 09 02.47310 -0.07 +0.20 # FbAS2001 C2022 09 02.47310 22 24 11.192-11 35 19.84 568

arc length 2.115639999974519 d Expected accuracy on 2026-10-09 14:20:16.256925: 0.11 arcsec 0.11 arcsec moving at 2.15 arcsec / h
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

The ephemeris uncertainty reported on line 4 is actually the one for prediction at the current date, but not mentionned. The uncertainty given on the last line should be that of the current date, but it's not. A "carriage return" before "ephemeris uncertainty" and "Expected accuracy" would help reading output.

More important, I noticed that if one doesn't run orb.summarize, then the following orb.predict will fail:

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Traceback (most recent call last):
File "/home/petit/Research/JPL_ephemerides/src/OSSOS/liborbfit-pr26/.venv-pr26/lib/python3.12/site-packages/astropy/units/quantity_helper/converters.py", line 192, in converters_and_unit
if can_have_arbitrary_unit(args[i]):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/petit/Research/JPL_ephemerides/src/OSSOS/liborbfit-pr26/.venv-pr26/lib/python3.12/site-packages/astropy/units/quantity_helper/converters.py", line 134, in can_have_arbitrary_unit
return np.all(np.logical_or(np.equal(value, 0.0), ~np.isfinite(value)))
^^^^^^^^^^^^^^^^^^
TypeError: ufunc 'isfinite' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/petit/Research/JPL_ephemerides/src/OSSOS/liborbfit-pr26/scripts/example.py", line 12, in
orb.predict(orb.time + 10 * units.day)
~~~~~~~~~^~~~~~~~~~~~~~~~
File "/home/petit/Research/JPL_ephemerides/src/OSSOS/liborbfit-pr26/.venv-pr26/lib/python3.12/site-packages/astropy/units/quantity.py", line 701, in array_ufunc
raise e
File "/home/petit/Research/JPL_ephemerides/src/OSSOS/liborbfit-pr26/.venv-pr26/lib/python3.12/site-packages/astropy/units/quantity.py", line 646, in array_ufunc
converters, unit = converters_and_unit(function, method, *inputs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/petit/Research/JPL_ephemerides/src/OSSOS/liborbfit-pr26/.venv-pr26/lib/python3.12/site-packages/astropy/units/quantity_helper/converters.py", line 203, in converters_and_unit
raise TypeError(
TypeError: Unsupported operand type(s) for ufunc add: 'NoneType,Quantity'
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

That's weird, given that first thing done in orb.summarize is to call self.predict... We need an explanation that summarize or something else is needed before predict, or make sure that this something is either called at start of predict, or when creating the NKOrbit object.

JM

PS: I cannot push the changes I made on my local copy; branch review-pr26 is not visible on github.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants