Repository navigation
StripEnergyThresholdFinder for per-strip slow and fast threshold extraction with diagnostics - #166
JarredMRoberts wants to merge 44 commits into
Conversation
|
I still need to fix all of the code style issues and work on some optimizations to speed the code up a bit. |
Co-authored-by: Felix Hagemann <hagemann@berkeley.edu>
Co-authored-by: Felix Hagemann <hagemann@berkeley.edu>
…nventions and integrate energy calibration improvements
There was a problem hiding this comment.
Hi Jarred,
I had an extensive look at this app and finally got this to run.
I replied to all comments in this PR, and opened a separate PR onto your fork/branch, addressing some code changes to
- get this app running
- remove the helper classes by replacing them with existing nuclearizer/megalib code
Here is the PR onto your branch with detailed code changes: JarredMRoberts#1
|
I second not to have an additional parser linked. |
fhagemann
left a comment
There was a problem hiding this comment.
One more remark: I got this app to run and got reasonable results for the fast and slow thresholds only when using a dataset taken with NN off.
We might want to always filter out NN events, for this app to also run using datasets taken with NN on.
bd7161b to
20de738
Compare
| if (n1 > n0) { | ||
| crossoverADC = ADC_val; | ||
| break; | ||
| } |
There was a problem hiding this comment.
| if (n1 > n0) { | |
| crossoverADC = ADC_val; | |
| break; | |
| } | |
| if (n1 > n0) { | |
| crossoverADC = ADC_val; | |
| // Make sure that this is an actual crossover and not just the first bin | |
| // with more than FAST_MIN_CROSSOVER_COUNTS where n1 was above n0 | |
| while (n1 > n0 && ADC_val >= first_nonzero) { | |
| crossoverADC = ADC_val; | |
| ADC_val--; | |
| if (ADCMap.find(ADC_val) != ADCMap.end()) { | |
| n0 = ADCMap[ADC_val].first; | |
| n1 = ADCMap[ADC_val].second; | |
| } | |
| } |
Adding this backward search seems to resolve the issue above:

There was a problem hiding this comment.
All major changes I requested are implemented.
| for (auto& a : ADCMap) { | ||
| int ADC_val = a.first; | ||
| int n0 = a.second.first; | ||
| int n1 = a.second.second; | ||
|
|
||
| // Require minimum statistics to avoid noise triggers | ||
| if (n0 + n1 < FAST_MIN_CROSSOVER_COUNTS) { | ||
| continue; | ||
| } | ||
|
|
||
| if (n1 > n0) { | ||
| crossoverADC = ADC_val; | ||
| // Make sure that this is an actual crossover and not just the first bin | ||
| // with more than FAST_MIN_CROSSOVER_COUNTS where n1 was above n0 | ||
| while (n1 > n0 && ADC_val >= first_nonzero) { | ||
| crossoverADC = ADC_val; | ||
| ADC_val--; | ||
| if (ADCMap.find(ADC_val) != ADCMap.end()) { | ||
| n0 = ADCMap[ADC_val].first; | ||
| n1 = ADCMap[ADC_val].second; | ||
| } | ||
| } | ||
| break; | ||
| } | ||
| } | ||
|
|
||
| const int nbins = FAST_CROSSOVER_WINDOW_BINS; | ||
|
|
||
| // Extend search window for stability | ||
| int searchMax = first_nonzero + FAST_SEARCH_RANGE_ADC; | ||
|
|
||
| for (int ADC = first_nonzero; ADC < searchMax; ADC++) { | ||
| int n_dt0 = 0; | ||
| int n_dt1 = 0; | ||
|
|
||
| for (int a = ADC; a < ADC + nbins; a++) { | ||
| if (ADCMap.find(a) == ADCMap.end()) { | ||
| continue; | ||
| } | ||
|
|
||
| n_dt0 += ADCMap[a].first; | ||
| n_dt1 += ADCMap[a].second; | ||
| } |
There was a problem hiding this comment.
When trying this out for Cs-137 data to explore the possibility to use Cs-137 data in payload calibrations for threshold determination, I found that the current implementation would give wrong results if there are a lot of events with fast timing in the noise peak (I guess these are NN events featuring a dominant peak-dip structure, that might give a fast-shaper signal that exceeds the threshold, but in the slow shaper average out to a small energy value).
In this case, the condition n1 .> n0 is reached early:

A fix for this might be to start the crossover search only for ADC bins right of the noise peak, i.e. the slow threshold (plus some tolerance):
| for (auto& a : ADCMap) { | |
| int ADC_val = a.first; | |
| int n0 = a.second.first; | |
| int n1 = a.second.second; | |
| // Require minimum statistics to avoid noise triggers | |
| if (n0 + n1 < FAST_MIN_CROSSOVER_COUNTS) { | |
| continue; | |
| } | |
| if (n1 > n0) { | |
| crossoverADC = ADC_val; | |
| // Make sure that this is an actual crossover and not just the first bin | |
| // with more than FAST_MIN_CROSSOVER_COUNTS where n1 was above n0 | |
| while (n1 > n0 && ADC_val >= first_nonzero) { | |
| crossoverADC = ADC_val; | |
| ADC_val--; | |
| if (ADCMap.find(ADC_val) != ADCMap.end()) { | |
| n0 = ADCMap[ADC_val].first; | |
| n1 = ADCMap[ADC_val].second; | |
| } | |
| } | |
| break; | |
| } | |
| } | |
| const int nbins = FAST_CROSSOVER_WINDOW_BINS; | |
| // Extend search window for stability | |
| int searchMax = first_nonzero + FAST_SEARCH_RANGE_ADC; | |
| for (int ADC = first_nonzero; ADC < searchMax; ADC++) { | |
| int n_dt0 = 0; | |
| int n_dt1 = 0; | |
| for (int a = ADC; a < ADC + nbins; a++) { | |
| if (ADCMap.find(a) == ADCMap.end()) { | |
| continue; | |
| } | |
| n_dt0 += ADCMap[a].first; | |
| n_dt1 += ADCMap[a].second; | |
| } | |
| for (auto& a : ADCMap) { | |
| int ADC_val = a.first; | |
| int n0 = a.second.first; | |
| int n1 = a.second.second; | |
| // Require minimum statistics to avoid noise triggers | |
| if (n0 + n1 < FAST_MIN_CROSSOVER_COUNTS || ADC_val <= m_SlowThresholdsADC[R] + FAST_FIRST_ADC_OFFSET) { | |
| continue; | |
| } | |
| if (n1 > n0) { | |
| crossoverADC = ADC_val; | |
| // Make sure that this is an actual crossover and not just the first bin | |
| // with more than FAST_MIN_CROSSOVER_COUNTS where n1 was above n0 | |
| while (n1 > n0 && ADC_val >= m_SlowThresholdsADC[R] + FAST_FIRST_ADC_OFFSET) { | |
| crossoverADC = ADC_val; | |
| ADC_val--; | |
| if (ADCMap.find(ADC_val) != ADCMap.end()) { | |
| n0 = ADCMap[ADC_val].first; | |
| n1 = ADCMap[ADC_val].second; | |
| } | |
| } | |
| break; | |
| } | |
| } | |
| const int nbins = FAST_CROSSOVER_WINDOW_BINS; | |
| // Extend search window for stability | |
| int searchMax = m_SlowThresholdsADC[R] + FAST_SEARCH_RANGE_ADC; | |
| for (int ADC = m_SlowThresholdsADC[R]; ADC < searchMax; ADC++) { | |
| int n_dt0 = 0; | |
| int n_dt1 = 0; | |
| for (int a = ADC; a < ADC + nbins; a++) { | |
| if (ADCMap.find(a) == ADCMap.end()) { | |
| continue; | |
| } | |
| n_dt0 += ADCMap[a].first; | |
| n_dt1 += ADCMap[a].second; | |
| } |
(and I also had to set FAST_FIRST_ADC_OFFSET to 20 for this specific dataset I was looking at).
This is what I get with the updated code (but note that FAST_FIRST_ADC_OFFSET might need to be optimized):

| // Fill graphs | ||
| for (const auto& kv : m_SlowHardwareThresholds) { | ||
|
|
||
| MReadOutElementDoubleStrip R = kv.first; | ||
| int Strip = R.GetStripID(); | ||
| double Threshold = kv.second; | ||
|
|
||
| // Ignore invalid hardware thresholds | ||
| if (Threshold < 0.0) { | ||
| continue; | ||
| } | ||
|
|
||
| if (R.IsLowVoltageStrip() == true) { | ||
|
|
||
| gSlowHardwareThresh_LV->SetPoint( | ||
| gSlowHardwareThresh_LV->GetN(), | ||
| Strip, | ||
| Threshold); | ||
|
|
||
| } else { | ||
|
|
||
| gSlowHardwareThresh_HV->SetPoint( | ||
| gSlowHardwareThresh_HV->GetN(), | ||
| Strip, | ||
| Threshold); | ||
| } | ||
| } |
|
I had to run this app now several times on the same data, and the part that takes the longest is reading the HDF5 files and applying the strip map / energy calibration. Would it be possible to also support reading ROA files as input (and not just HDF5) to avoid this first step in the threshold app having to be re-run every time I want to change settings or fix bugs in the app. |
|
A lot of the plots and the analysis are hard-coded to the Am-241 energy range, which makes the fast threshold extraction from the FM calibration Cs-137 partyhat measurement challenging. It would be nice to have the option to set some of the constants (e.g. |
…trips and guard rings
…re, and additional configurations for FAST analysis
|
This PR has merge conflicts. Can you resolve them? |
|
And would this be the threshold app in its final form (ready for final review), or are you still working on something? |
83e2912 to
b97f78e
Compare
|
My last push was to fix the merge conflict in the strip map file. |
|
I'm re-running everything with the latest version of this app right now, stay tuned :) <?xml version="1.0" encoding="UTF-8"?>
<StripEnergyThresholdFinder>
<Input>
<DataFiles>
<DataFile>/volumes/metis/COSI/SMEX/fm/config2b/raw/20261014/gse_20261014T124603.hdf5</DataFile>
<DataFile>/volumes/metis/COSI/SMEX/fm/config2b/raw/20261014/gse_20261014T124603_1.hdf5</DataFile>
<DataFile>/volumes/metis/COSI/SMEX/fm/config2b/raw/20261014/gse_20261014T124603_2.hdf5</DataFile>
<DataFile>/volumes/metis/COSI/SMEX/fm/config2b/raw/20261014/gse_20261014T124603_3.hdf5</DataFile>
<DataFile>/volumes/metis/COSI/SMEX/fm/config2b/raw/20261014/gse_20261014T124603_4.hdf5</DataFile>
<DataFile>/volumes/metis/COSI/SMEX/fm/config2b/raw/20261014/gse_20261014T124603_5.hdf5</DataFile>
</DataFiles>
<CalibrationFile>20261013_FMPayload_config2b_prequick_PR235.ecal</CalibrationFile>
<StripMap>strip-map-FM-payload.map</StripMap>
</Input>
<Output>
<Prefix>/volumes/mobius/users/felix/Analysis/FM/PR166_Cs137_partyhat_20261014_124603</Prefix>
<SaveDiagnosticPlots>true</SaveDiagnosticPlots>
</Output>
<Analysis>
<ProcessAllDetectors>true</ProcessAllDetectors>
<MinEntries>50</MinEntries>
<FallbackThresholdKeV>-1</FallbackThresholdKeV>
<NoiseSearchMaxKeV>50</NoiseSearchMaxKeV>
<FastFallbackKeV>-1</FastFallbackKeV>
<NoiseStartMinCounts>5</NoiseStartMinCounts>
<FastSearchRangeADC>2000</FastSearchRangeADC>
</Analysis>
</StripEnergyThresholdFinder> |
fhagemann
left a comment
There was a problem hiding this comment.
Here is my review after running the newest version of the app with the XML config file I posted earlier:
- The new panel overview plots are nice, and make it way easier checking if the thresholds were determined correctly or not :)
- This is the output when running
cSlowSpectraADC_Det1_LV->Draw():

andcSlowSpectraADC_Det1_HV->Draw():

It looks like these histograms are still smoothed (which is also what is fitted in the end), but I think this is OK. - Could these 8x8 panels also be created for the fast threshold determination?
- I can see you added the detector names to the titles of those plots (which would make this an FM threshold app, but show non-accurate detector names, e.g. when running this app on unit-level data or data from the EM - I would favor removing them here to keep the app as generic as possible).
- For the guard rings: they tend to have more counts than the strips, which makes it hard finding a good set of parameters (especially
NoiseStartMinCounts) that works well for both the strips and the GR contacts:

Might be worth having the option to set them separately. For now, I've just been running the threshold app twice, with different settings forNoiseStartMinCountsto extract the strip and the GR thresholds separately. - The summary plots for the thresholds look nice now that they are split by detector ID. However, their display arrangement differ from the arrangement that we use in the other nuclearizer expos, e.g. in #244:

Same applies to the slow hardware plots:

- The individual detector plots look nice, but I would again vote for not having the hard-coded detector names in them:

| const map<unsigned int, pair<string, string>> DetectorInfo = { | ||
| {0, {"Q0L0", "HP52366-2"}}, | ||
| {1, {"Q0L1", "HP52358-2"}}, | ||
| {2, {"Q0L2", "HP52432-1"}}, | ||
| {3, {"Q0L3", "M61617 (FM2)"}}, | ||
| {4, {"Q1L0", "HP52542-1"}}, | ||
| {5, {"Q1L1", "HP52358-3"}}, | ||
| {6, {"Q1L2", "HP52434-1"}}, | ||
| {7, {"Q1L3", "HP52406-1"}}, | ||
| {8, {"Q2L0", "HP52542-2"}}, | ||
| {9, {"Q2L1", "HP52407-3"}}, | ||
| {10, {"Q2L2", "HP52379-1"}}, | ||
| {11, {"Q2L3", "HP52358-1"}}, | ||
| {12, {"Q3L0", "HP52434-2"}}, | ||
| {13, {"Q3L1", "HP52361-1"}}, | ||
| {14, {"Q3L2", "M61618 (FM-3)"}}, | ||
| {15, {"Q3L3", "HP52406-3"}} | ||
| }; |
There was a problem hiding this comment.
While this is nice to include in the title of the plots, this is only true for the FM payload. I can envision the threshold app to also be used for other purposes (unit-level testing data, the EM model, other experiments like GRIPS) - not sure if we want this here. I'd be fine with just using the detector ID in the titles instead of all of this information.
| // ------------------------------------------------------------- | ||
| // Canvas with LV and HV overlaid | ||
| // ------------------------------------------------------------- | ||
|
|
||
| TCanvas* cSlowHardwareThresh = | ||
| new TCanvas( | ||
| "cSlowHardwareThresh", | ||
| "Slow Hardware Threshold per Strip", | ||
| 800, | ||
| 600); | ||
|
|
||
| cSlowHardwareThresh->cd(); | ||
|
|
||
| gSlowHardwareThresh_LV->Draw("AP"); | ||
|
|
| cout << "Paneled raw ADC or Energy spectra with slow thresholds:" << endl; | ||
| cout << " One 8x8 canvas per detector and detector side (strips 0-63)" << endl; | ||
| cout << " Red solid line = software slow threshold" << endl; | ||
| cout << " Blue dashed line = hardware slow threshold" << endl; | ||
| cout << " Examples:" << endl; | ||
| cout << " cSlowSpectraADC_Det1_LV->Draw()" << endl; | ||
| cout << " cSlowSpectraADC_Det1_HV->Draw()" << endl; | ||
| cout << " cSlowSpectraEnergy_Det1_LV->Draw()" << endl; | ||
| cout << " cSlowSpectraEnergy_Det1_HV->Draw()" << endl; |
There was a problem hiding this comment.
Would it be easy to also add these paneled plots for the fast threshold determination?
| int m_HistogramBins = 512; | ||
| double m_HistogramMaxADC = 4096.0; | ||
| double m_NoiseSearchMaxKeV = 40.0; | ||
| double m_NoiseStartMinCounts = 50.0; |
There was a problem hiding this comment.
It might be nice being able to set different numbers for strips and GR contacts, as the latter tend to have way more counts (but not super urgent, as this app can just be run twice with different values for NoiseStartMinCounts)





Adds a standalone application, StripEnergyThresholdFinder, for computing per-strip slow and fast energy thresholds. Slow thresholds are determined from ADC spectra using a noise peak and trough method, while fast thresholds are determined from dt0/dt1 timing crossover. The tool reads calibrated HDF5 data via MModuleLoaderMeasurementsHDF, applies strip mapping and energy calibration from a YAML configuration, and produces ROOT diagnostic outputs (energy spectra with thresholds, dt0 vs dt1 per strip, and threshold distributions) along with CSV export files. The implementation is self-contained under apps/ and does not modify existing modules. Tested on COSI datasets with consistent threshold behavior and expected diagnostic results. Target branch is develop/em.