[li-lizhe]

Summary

EoraProcessor.process() computed its per-module PROCESS_USED_MEMORY value with
torch.cuda.memory_stats(DEVICE_0) and only looked at the second device when
torch.cuda.device_count() > 1. DEVICE_0 / DEVICE_1 come from auto_select_torch_device(), so on a
host whose active accelerator is an Ascend NPU they are npu:0 / npu:1 while
torch.cuda.device_count() is 0: the CUDA allocator reports no statistics for those devices, the log row
silently read 0.00MB, and the second-device branch was unreachable. get_max_memory() in
gptqmodel/looper/loop_processor.py carried the same torch.cuda query.

resolve_memory_stats_callable() / memory_stats_for_device() are added next to the existing
resolve_empty_cache_callable() / empty_cache_for_device() pair, and both call sites now use them. The
second-device condition is DEVICE_1 != DEVICE_0. Distinct devices retain their log slots even when
statistics are empty or unavailable (reported as 0.00MB), preserving the CUDA two-device format.
CPU behaviour is unchanged (telemetry simply stays empty when no allocator applies).

Fixes #3437

What Changed

• Dispatch allocator telemetry to the requested CUDA, XPU, or NPU backend; unsupported or failing queries return an empty mapping.
• Use that helper in get_max_memory() and EoraProcessor.process(), preserving current versus peak reporting.
• Review fix: keep the second device's 0.00MB slot when its query is empty or fails, rather than silently dropping it.
• Expand tests to cover real resolver dispatch, missing/disabled allocators, one/two-device rows, empty/error results, and CPU fallback. Clear the resolver cache between tests.
• Clean up the touched EoRA imports and unused current-memory calculations.

Tests

• Unit tests have been executed and passed.
• Added simple, fast regression tests.

Review validation used mocked allocator APIs; it did not rerun the contributor's hardware validation below.

Check	Result
Expanded tests against original PR head	18 regression failures, 34 passed
Expanded tests after review fix	52 passed in 2.43 seconds
Ruff 0.14.2 on all four PR code/test paths	Passed
Python compilation and git diff --check	Passed
Latest GitHub Actions run	Awaiting maintainer approval (action_required)

cd tests && python -m pytest test_accelerator_memory_stats.py -q --disable-warnings
# From repository root:
ruff check --config format/ruff.toml gptqmodel/utils/torch.py gptqmodel/looper/eora_processor.py gptqmodel/looper/loop_processor.py tests/test_accelerator_memory_stats.py
python -m py_compile gptqmodel/utils/torch.py gptqmodel/looper/eora_processor.py gptqmodel/looper/loop_processor.py tests/test_accelerator_memory_stats.py
git diff --check

Contributor's initial hardware verification

Real hardware: Ascend 910B host (aarch64, 4 NPUs, torch 2.15.0.dev + torch_npu,
torch.cuda.is_available() == False). The real gptqmodel/utils/torch.py was loaded from main and from
this branch on that host, and the log row was computed both ways while allocations were actually held on
two NPUs:

DEVICE_0 / DEVICE_1 : npu:0 / npu:1
allocated           : 64MB on npu:0, 128MB on npu:1
PROCESS_USED_MEMORY, main  -> 0.00MB
PROCESS_USED_MEMORY, patch -> 64.00MB, 128.00MB

torch.cuda.memory_stats(torch.device('npu:0')) -> 0 keys (also after the allocations)
torch.npu.memory_stats(torch.device('npu:0'))  -> 123 keys (tracks the allocation)

The original five unit tests in tests/test_accelerator_memory_stats.py fail on main
(AttributeError: ... has no attribute 'memory_stats_for_device') and pass with this change. They were run
on the same Ascend host with a stand-in for the monkeypatch fixture, and python -m py_compile plus
ruff check (ruff 0.16.8 with format/ruff.toml) report no new findings for the changed files.

Notes

• End-to-end gptqmodel quantization with an EoRA adapter on the NPU host: that machine has
  torch / torch_npu but not the rest of the gptqmodel dependency set, so only the changed module was
  exercised there.
• Real CUDA and XPU devices were not tested. The follow-up review used mocked CUDA/XPU/NPU allocators, not accelerator hardware.

Review Requirements

• Reviewed this PR.
• Checked existing project structure, APIs, and conventions.
• Limited monkeypatching to hardware-independent allocator tests and EoRA setup outside the telemetry under test.
• Kept production changes focused on telemetry and touched-file lint cleanup.
• Added regression coverage for identified formatting and dispatch risks.

Kernel Accuracy Requirements

• Not applicable: no quantization math or inference kernels changed.

AI assistance

This change was prepared with AI assistance. The contributor reviewed the diff and this description and
approved publishing it.

Link to Devin session: https://app.devin.ai/sessions/cc3422193f164d47afc2ba905ebcc77b
Open in Devin Desktop: https://app.devin.ai/desktop/session/cc3422193f164d47afc2ba905ebcc77b?variant=devin
Requested by: @Qubitium

[Qubitium]

@li-lizhe Thank you. Pushed small changes. Ready to merge.