Skip to content

3325 lo pivot platform ck generation - #3518

Open
tmplummer wants to merge 5 commits into
IMAP-Science-Operations-Center:devfrom
tmplummer:3325-lo-pivot-platform-ck-generation
Open

tmplummer wants to merge 5 commits into
IMAP-Science-Operations-Center:devfrom
tmplummer:3325-lo-pivot-platform-ck-generation

Conversation

@tmplummer

Copy link
Copy Markdown
Contributor

Change Summary

Overview

Closes #3325. Adds a Lo pivot-ckernel job that generates one SPICE CK per pointing for the IMAP_LO pivot-platform frame, from that pointing's Lo L1B NHK. The pivot angle uses the same calculation as goodtimes. Kernels are named imap_lopivot-repoint#####_<start>_<end>_<NNN>.bc, which imap-data-access v1.3.0 supports.

File changes

  • lo/lo_pivot_kernel.py (new): builds and writes the single-segment pivot CK. Errors if the kernel already exists or there are no valid pivot samples.
  • lo/l1b/lo_l1b.py: the goodtimes pivot calculation is now get_median_pivot_angle, shared with the new job. Goodtimes output is unchanged.
  • spice/pointing_frame.py: CK writing and filename helpers extracted for reuse. DPS output is unchanged.
  • cli.py: new Lo l1b / pivot-ckernel job (--repointing, one NHK input). Kernel version lookup is now shared with pointing-attitude.
  • pyproject.toml / poetry.lock: bump to imap-data-access>=1.3.0.

Testing

  • 17 new tests: 12 for the kernel module (pivot angle round trip against the existing pivot-angle geometry, coverage, naming, error cases), 2 for get_median_pivot_angle, 3 for the CLI job.
  • Full suite passes: 2031 passed, 4 skipped.
  • Not yet checked against a real repoint's goodtimes pivot.

Closes: #3325

@tmplummer
tmplummer requested review from laspsandoval and vineetbansal and a balanced review from Copilot October 1, 2026 19:56
@tmplummer tmplummer self-assigned this Oct 1, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

The job omits a required repoint-table dependency and has unhandled empty-input and partial-output failure paths.

Review effort: Balanced
Findings: 1 High severity · 2 Medium severity

Open (3)
What changed in this PR

Adds per-pointing IMAP-Lo pivot-platform CK generation using L1B NHK pivot measurements.

Changes:

  • Adds pivot-angle calculation and CK generation.
  • Integrates the pivot-ckernel CLI job.
  • Extracts reusable CK writing/naming utilities and updates data-access support.
File Description
pyproject.toml Updates data-access dependency.
poetry.lock Locks data-access 1.3.0.
imap_processing/​cli.py Integrates pivot-kernel processing.
imap_processing/​lo/​lo_pivot_kernel.py Generates per-pointing pivot CKs.
imap_processing/​lo/​l1b/​lo_l1b.py Shares median pivot calculation.
imap_processing/​spice/​pointing_frame.py Extracts reusable CK utilities.
imap_processing/​tests/​test_cli.py Tests CLI integration.
imap_processing/​tests/​lo/​test_lo_pivot_kernel.py Tests pivot CK behavior.
imap_processing/​tests/​lo/​test_lo_l1b.py Tests median pivot calculation.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread imap_processing/lo/lo_pivot_kernel.py
Comment thread imap_processing/lo/l1b/lo_l1b.py
Comment thread imap_processing/lo/lo_pivot_kernel.py Outdated

@laspsandoval laspsandoval left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. One question just to make certain timing is correct.

"""
pointing_start_met, pointing_end_met = get_pointing_times_from_id(repoint_id)
# Use the same pivot angle as the goodtimes product.
pivot_angle = get_median_pivot_angle(load_cdf(l1b_nhk_path))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The averaging window starts from the first NHK sample, not the pointing start. Is the repoint NHK file always timed so its first sample matches the pointing start? Is it possible that the window of time could include samples when the pivot is moving?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The get_median_pivot_angle() function actually uses data in the range [hk_start_epoch + 0.5 hours:hk_start_epoch. + 22.5 hours]. This was what the Lo IT wanted in their other L1B processing, so I just extracted that into a function and call it here.

@leowerneck leowerneck left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

with tempfile.TemporaryDirectory(dir=kernel_path.parent) as tmp_dir:
tmp_kernel_path = Path(tmp_dir) / kernel_path.name
write_lo_pivot_ck(tmp_kernel_path, segment, pivot_angle, l1b_nhk_path.name)
os.link(tmp_kernel_path, kernel_path)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think Opus 5.5 might be overly-conservative/paranoid here, but it says os.link() may fail on FAT/exFAT and some network mounts, so it might be worth to consider something like:

  try:
      os.link(tmp_kernel_path, kernel_path)
  except FileExistsError:
      raise
  except OSError:
      # The filesystem does not support hard links. Create the kernel exclusively
      # and copy into it, removing it if the copy fails.
      with open(kernel_path, "xb") as dst:
          try:
              with open(tmp_kernel_path, "rb") as src:
                  shutil.copyfileobj(src, dst)
          except BaseException:
              dst.close()
              kernel_path.unlink()
              raise

I don't see an obvious reason why this would be necessary, but figured it wouldn't hurt to forward the comment to you.

coarse_pot_pri[(hk_epoch_ets >= start_et_hk) & (hk_epoch_ets <= end_et_hk)]
)
pivot = get_median_pivot_angle(cdf_hk)
if np.isnan(pivot):

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm wondering if we should fail loudly here on encountering a nan, since the kernel generation process is going through all the trouble of making sure that the correct angle is spit out. The only reason this is in (I'm sure I put this default 90 in) was to have the code do the same thing as Nathan's pipeline.

On the other hand - the code is calling np.nanmedian. Perhaps this check should go inside the function and it return float | np.nan.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I made it fail loudly in the kernel generation code:

if np.isnan(pivot_angle):
        raise ValueError(f"No valid pivot angle samples in {l1b_nhk_path.name}.")

@vineetbansal

Copy link
Copy Markdown
Collaborator

This makes sense to me reading it at a high-level - using ckw02 with frame IMAP_LO and reference frame IMAP_LO_BASE and the quaternion data corresponding to the median value of pcc_coarse_pot_pri in the time window used for the goodtimes, as the segment data.

This is instructive for me because I was always under the impression that these kernels are generated by code close to the hardware, not code downstream of a pipeline that looks at L1 products (with all the potential buginess of pipeline code affecting the kernel generation).

@vineetbansal

Copy link
Copy Markdown
Collaborator

One part that always confused me was that the l1b de product looks at pcc_cumulative_cnt_pri in the nhk data (and thus the code has a get_pivot_angle_from_nhk function). The code will now also have a get_median_pivot_angle function (also looking at nhk, but its pcc_coarse_pot_pri attribute) so there's a possibility of potential confusion between the two. (the confusion will be made explicit instead of being buried in the code, which is good).

This doesn't affect the correctness of this PR, but unless you know the history and rationale behind this, maybe we should open an issue on this? I'm happy to bring it up with the team when I talk to them next.

@vineetbansal vineetbansal left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

General approach makes sense - I haven't gone through the tests line by line but their intention going by their test names makes sense.

@tmplummer

Copy link
Copy Markdown
Contributor Author

One part that always confused me was that the l1b de product looks at pcc_cumulative_cnt_pri in the nhk data (and thus the code has a get_pivot_angle_from_nhk function). The code will now also have a get_median_pivot_angle function (also looking at nhk, but its pcc_coarse_pot_pri attribute) so there's a possibility of potential confusion between the two. (the confusion will be made explicit instead of being buried in the code, which is good).

This doesn't affect the correctness of this PR, but unless you know the history and rationale behind this, maybe we should open an issue on this? I'm happy to bring it up with the team when I talk to them next.

Good point. The get_pivot_angle_from_nhk function was implemented by myself and Greg (not at LASP anymore) with no input from Nathan or the IT. I went with the method that you implemented based on the IT dropbox code. It is a good thing to bring up... and probably migrate the L1B DE to use the same method as the others.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

Lo pivot platform CK generation

5 participants