[Xiaoyu Z (xiaoyu-work)]

Describe your changes

Extract the OpenVINO plugin EP context-compilation support from #2717 into a standalone change to EPContextBinaryGenerator. This PR intentionally excludes OpenVINOModelPackage, package manifests, shared assets, and multi-device package assembly.

• Dispatch explicitly to _run_qnn and _run_openvino; preserve QNN's existing session-based compilation, composite/weight-sharing behavior, metadata, and optional GenAI config updates.
• Add OpenVINO plugin registration, accelerator-type/device filtering, virtual cross-compilation targets, AOT via ModelCompiler, and JIT via InferenceSession.
• Correct the extracted OpenVINO path to honor embed_context, disable_cpu_fallback, and typed logging options. Parse JSON boolean/compiler enum values correctly and avoid redundant AOT session entries already controlled by the compile API.
• Preserve registration ownership, keep context sidecars in the output resource, and reject missing EPContext nodes or referenced binaries. Validate protobuf attributes directly so embedded binary data is not incorrectly decoded as UTF-8.
• Document standalone usage, compiler options, and current single-ONNX-model limitations.

Review fixes

• Discover the installed OpenVINO plugin in the existing EP bootstrap before Windows LocalSystem provider validation, preserving explicitly configured library paths. Include a regression through the full workflow entry point with a virtual NPU target and no physical NPU.
• Compare hexadecimal device IDs numerically, accepting case differences, an optional 0x prefix, leading zeros, and integer IDs.
• Clean the failed invocation's context ONNX and matching binaries after compilation or validation failure, allowing retries at the same output path. Keep existing successful outputs and unrelated files protected; no staging/publish framework was added.

Validation

• & .\.venv\Scripts\python.exe -m pytest test\passes\onnx\test_context_binary.py test\common\test_ort_inference.py test\systems\test_local.py -q — passed: 72 tests; 4 QNN hardware tests skipped because QNNExecutionProvider is unavailable.
• lintrunner olive\common\ort_inference.py olive\passes\onnx\context_binary.py test\common\test_ort_inference.py test\passes\onnx\test_context_binary.py docs\source\reference\pass.rst — passed, including PYLINT.
• Additional real integration checks using ONNX Runtime 1.28.0 and OpenVINO EP plugin 1.7.0: ran the complete Olive workflow from a fresh process without manual EP registration, using a generated external-weight MatMul/Relu ONNX model. Virtual NPU platform 5010 and physical CPU both passed AOT/JIT with embedded/external context data (8 combinations). The NPU workflow used device_id: 0x00B03E.
• Injected a transient failure after real context files were generated, then reran the unchanged workflow successfully at the identical cache output path. All four CPU workflow outputs reloaded and produced correct inference results with the original model/weights unavailable.
• Full repository suite and on-device NPU/GPU inference were not run; the plugin exposed only a physical CPU on this machine.

Release note: EPContextBinaryGenerator now supports standalone OpenVINO plugin EP compilation through ORT's AOT/JIT APIs, including automatic Windows LocalSystem plugin discovery, virtual target selection, embedded/external context data, and clean retries after failed compilation.

Checklist before requesting a review

• Add unit tests for this change.
• Make sure all tests can pass.
• Update documents if necessary.
• Lint and apply fixes to your code by running lintrunner -a
• Is this a user-facing change? If yes, give a description of this change to be included in the release notes.

(Optional) Issue link

Extracted compilation support from #2717; the original packaging PR remains separate.

[Copilot]

🟡 Changes recommended

Workflow initialization can reject OpenVINO before compilation, and device matching and failed-output cleanup need correction.

3 open findings

• Plugin registration occurs after provider validation · New
• Normalize hexadecimal hardware IDs before comparison · New
• Failed attempts leave artifacts blocking retries · New

What changed in this PR

Adds standalone OpenVINO EPContext compilation to Olive’s existing context-generation pass, without model packaging.

Changes:

• Adds OpenVINO AOT/JIT compilation, device filtering, and virtual targets.
• Validates generated context artifacts while preserving QNN behavior.
• Documents configuration and adds focused tests.

File	Description
test/​passes/​onnx/​test_context_binary.py	Tests OpenVINO options, registration ownership, and output validation.
olive/​passes/​onnx/​context_binary.py	Adds OpenVINO compilation and explicit provider dispatch.
olive/​olive_config.json	Declares OpenVINO and CPU/GPU/NPU support.
docs/​source/​reference/​pass.rst	Documents standalone compilation and limitations.

🧠 Review effort: Balanced

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Mayuresh Varerkar (MayureshV1)]

Preetha Veeramalai (@preetha-intel).. Can you please review.

[Preetha Veeramalai (preetha-intel)]

This EpctxBinaryGenerator is limited to the single model run for a single run where its not linked to CompositeModelHandler which helps in creating the production level ModelPackage.
Also we require a ORTModelPackage pass that uses the ORT API's for production packages to be uniform across platforms and HW .

Both are updated in the original parent PR-2717

[Mayuresh Varerkar (MayureshV1)]

Thanks Xiaoyu Z (@xiaoyu-work) for extracting this. Single-device OpenVINO compilation looks solid: it honours embed_context/disable_cpu_fallback, rejects weight_sharing, parses booleans and enums properly, and cleans up after failed runs.

Before we go further, could you clarify the intent with respect to #2717? Is the plan for #2732 to replace the EPContextBinaryGenerator changes in #2717, with those changes then dropped from #2717? If so, #2732 would need to merge first and #2717 would be rebased to contain only ORTModelPackage. We're fine with that order; we just want to agree on it so the two PRs don't keep conflicting in context_binary.py.

A few questions on scope:

1. Multi-device compilation isn't covered yet. ORTModelPackage in #2717 consumes a composite model produced by compiling one source model for several devices in one run (target_devices): one component per device, each in its own <device_id> folder, with component names equal to the device IDs (these become variant names). It also relies on these model attributes: source_model_path (where the shared weights come from), device_id per component, and no_flatten. #2732 is single-device and rejects composite input. Do you plan to add this to #2732, or should it come in a follow-up PR on top of it before #2717 is rebased?

2. Weightless compilation by default. The docs in #2732 say weightless compilation isn't enabled by default. For packaging, weightless is the main use case: each device variant references one shared copy of the weights instead of duplicating them. Should the pass keep weightless opt-in (users set ep.enable_weightless) and document it as the recommended setting for model packages, or turn it on by default when compiling for packaging? Either is fine with us; we'd just like it to be explicit. Note that ORT only defines the ep.enable_weightless key from 1.30; earlier releases define ep.enable_weightless_ep_context_nodes, so the docs should say which ORT version the recommended setting applies to.

3. Per-device binary naming (set_ep_context_binary_information). #2717 uses ModelCompiler.set_ep_context_binary_information to control the folder and name of each device's binary. The underlying C API has been available since ORT 1.23, but the Python binding is only being added in microsoft/onnxruntime#33640 (not merged yet, expected in ORT 1.32). Is this something you'd want in this pass? If so, it would need to be optional (used only when the binding is available).

Inline comments below.

[Mayuresh Varerkar (MayureshV1)]

When the plugin is registered both by maybe_register_ep_libraries and under ovep_registration.virtual, a physical and a virtual NPU can report the same device_id (e.g. B03E on PTL), so {"device_id": "B03E"} matches two devices. Could selection prefer the entry with npu_platform metadata when a device_id filter matches more than one, or document that npu_platform is required in this case?

[Mayuresh Varerkar (MayureshV1)]

Is installing onnxruntime-openvino and onnxruntime-ep-openvino together intended? Both provide OpenVINOExecutionProvider, and onnxruntime-openvino is now deprecated. Our preference is to pull in and use onnxruntime + onnxruntime-ep-openvino instead.

[Mayuresh Varerkar (MayureshV1)]

Could we add a minimum ORT version check for the plugin-EP path, so older ORT fails with a clear message instead of an AttributeError? The OpenVINO plugin supports ORT 1.23.0+ (the plugin EP registration, get_ep_devices/add_provider_for_devices and the compile API used here are all in 1.23), so >= 1.23.0 looks like the right floor.