Skip to content

Honor model_external_initializers_file_folder_path for file-path model loads - #29459

Merged
jambayk merged 4 commits into
mainfrom
jambayk/file-external
Jul 6, 2026
Merged

Honor model_external_initializers_file_folder_path for file-path model loads#29459
jambayk merged 4 commits into
mainfrom
jambayk/file-external

Conversation

@jambayk

@jambayk jambayk commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

Description

session.model_external_initializers_file_folder_path now takes effect for every model load path and overrides the model's own directory as the base for resolving external initializers.

  • New Model::Load(file_path, graph_model_path, ...) overload reads the model bytes from file_path while storing a separate graph model path that is used only for external-data resolution.
  • InferenceSession applies the option in all ONNX load paths (file path, memory buffer, ModelProto, istream, and the parsed-proto Load()): the model is read from its real location, but external initializers are resolved from the option's folder. The prior restriction that only honored the option when the model location was empty is removed.

Adds a shared-lib test that loads a model from a path whose directory does not contain the external data and resolves it via the option.

Motivation and Context

The option was originally added in #23557 to let a model loaded from an in-memory buffer point ORT at a folder holding its external initializer files, since a buffer has no directory of its own. That implementation only applied when the model location was empty, so it worked exclusively for buffer/stream loads and required the caller to clear the model directory first.

Since then the option is used for more scenarios: weightless/cache models, models that share a single weights file, and packaged models whose weights live outside the model's own folder. In all of these the caller wants to point ORT at a specific external-initializers folder regardless of how the model itself is loaded. Restricting it to buffer loads made the usage narrow and unexpected.

This change makes the behavior generic: the option is honored for file-path loads too and overrides the model directory. Overriding is safe because the option only changes the base directory used to resolve external initializer files; the model bytes are still read from the real location. Downstream consumers of the graph model path already tolerate a virtual or non-existent model path, since buffer loads (a very common case) have always produced exactly such a path, and they either use only the parent directory or guard file access with existence checks.

A follow-up PR will build on this to simplify the model package flow (fold the external_data variant field into general session-options path resolution) and to remove the legacy directory-based CreateSession path.

jambayk and others added 3 commits July 1, 2026 19:42
…l loads

Make the session.model_external_initializers_file_folder_path option take
effect for all model load paths and let it override the model's own
directory as the base for resolving external initializers.

Core:
- Add a Model::Load(file_path, graph_model_path, ...) overload that reads the
  model bytes from file_path but stores a separate graph model path used for
  external-data resolution.
- InferenceSession: apply the option in every ONNX load path (file path,
  buffer, proto, istream, and the parsed-proto Load()), reading the model from
  its real location while resolving external initializers from the option
  folder. Drop the previous restriction that only honored the option when the
  model location was empty.

Model package:
- Resolve path-valued session options (an allowlist: the external initializers
  folder and ep.context_file_path) against the package at variant-parse time,
  so variants reference shared assets by sha256: URI or relative path directly
  in session_options.
- Remove the dedicated external_data variant field and its special injection;
  the model is now always loaded from the selected variant path.
- On the advanced path, carry over path-valued session options from the
  variant for keys the caller did not set.

Tests:
- Shared-lib test loading a model from a path whose directory lacks the
  external data, resolved via the option.
- Model package test resolving a relative external-initializers folder option
  and loading external data on both the default and advanced paths.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
GetExternalInitializersFolderModelPath is only referenced by the ONNX load
paths, which are compiled out in ORT_MINIMAL_BUILD. Guard its definition with
!defined(ORT_MINIMAL_BUILD) so it is not flagged as an unused function under
-Werror in the minimal build CI leg.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Move the model package improvements (allowlist path resolution, external_data
removal) to a follow-up PR. This PR only enables the external initializers
folder option for file-path model loads.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates ONNX model loading so session.model_external_initializers_file_folder_path is honored for all load paths (including file-path loads), overriding the model’s directory when resolving external initializer data while still reading model bytes from the original location.

Changes:

  • Added a Model::Load(file_path, graph_model_path, ...) overload to decouple “where bytes are read from” vs “which path is used for external-data resolution”.
  • Updated InferenceSession ONNX load paths to apply model_external_initializers_file_folder_path uniformly (file path, memory buffer, ModelProto, istream, parsed-proto Load()).
  • Added a shared-lib test that validates external initializers can be resolved from an override folder even when the model is loaded from a path in a different directory.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
onnxruntime/test/shared_lib/test_model_loading.cc Adds a new test covering path-based loads with external-initializer folder override.
onnxruntime/core/session/inference_session.cc Applies the external-initializers folder override across all ONNX load entrypoints.
onnxruntime/core/graph/model.h Declares a new Model::Load overload that accepts a separate graph model path for external-data resolution.
onnxruntime/core/graph/model.cc Implements the new Model::Load(file_path, graph_model_path, ...) overload via LoadModelHelper.
include/onnxruntime/core/session/onnxruntime_session_options_config_keys.h Updates documentation for session.model_external_initializers_file_folder_path to reflect broader behavior.

Comment thread onnxruntime/test/shared_lib/test_model_loading.cc Outdated
Comment thread onnxruntime/test/shared_lib/test_model_loading.cc Outdated
Address review feedback: use the non-throwing copy_file/remove_all overloads
with error_code and assert on the result, so a filesystem failure produces a
clear assertion message instead of an uncaught throw.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@jambayk
jambayk merged commit a0d9008 into main Jul 6, 2026
87 checks passed
@jambayk
jambayk deleted the jambayk/file-external branch July 6, 2026 20:11
jambayk added a commit that referenced this pull request Jul 7, 2026
…acy directory load path (#29501)

### Description

Two changes to the model package flow, enabled by the file-path external
initializers support:

**1. Fold `external_data` into `session_options` with path resolution.**
- Path-valued session options (an allowlist:
`session.model_external_initializers_file_folder_path` and
`ep.context_file_path`) are resolved against the package
(`sha256:<hex>`, relative, or absolute) at variant-parse time, using the
same rules as `model_file`.
- The dedicated `external_data` variant field and its special session
injection are removed; the model is always loaded from the selected
variant path.
- On the advanced path (caller supplies their own `OrtSessionOptions`),
path-valued options are carried over from the variant for keys the
caller did not set, so a model that needs its external-initializers
folder still loads.

**2. Remove the legacy directory-based `CreateSession(package_dir)`
path.**
- Removes the `is_directory(package_root)` branch in
`CreateSessionAndLoadModelImpl`. Model packages are loaded through the
experimental `OrtModelPackageApi` (`CreateModelPackageContext` ->
`SelectComponent` -> `CreateSession`).
- The three end-to-end tests that exercised the directory path are
migrated to the experimental API via a `CreateSessionFromModelPackage`
test helper, preserving coverage for factory-based selection,
`PREFER_CPU` policy selection, and compiled-model compatibility scoring.

### Motivation and Context

The file-path external initializers folder support (#29459) lets the
model package flow drop its workaround of memory-mapping the model and
forcing a buffer load: it sets the folder option and loads from the
selected variant path directly. Building on that, the dedicated
`external_data` field folds into the general session-options mechanism.
A variant declares
`session.model_external_initializers_file_folder_path` (or other
path-valued options) in its `session_options`, and ORT resolves those
values against the package at parse time. This removes special-case code
and lets other path-valued options such as the EPContext file path be
resolved the same way. An allowlist is used rather than value-syntax
sniffing because session-option values are arbitrary strings; only known
path-valued keys are resolved, so ordinary values pass through
untouched.

The legacy directory-based `CreateSession(package_dir)` path was meant
to be removed when the experimental `OrtModelPackageApi` landed. It only
did variant/EP selection and never merged the variant's
`session_options`/`provider_options`, so a package loaded via
`Ort::Session(env, dir, so)` silently ignored its manifest session
options, inconsistent with the experimental API. Removing it leaves a
single, consistent way to load a model package.

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
tianleiwu pushed a commit that referenced this pull request Jul 7, 2026
…acy directory load path (#29501)

### Description

Two changes to the model package flow, enabled by the file-path external
initializers support:

**1. Fold `external_data` into `session_options` with path resolution.**
- Path-valued session options (an allowlist:
`session.model_external_initializers_file_folder_path` and
`ep.context_file_path`) are resolved against the package
(`sha256:<hex>`, relative, or absolute) at variant-parse time, using the
same rules as `model_file`.
- The dedicated `external_data` variant field and its special session
injection are removed; the model is always loaded from the selected
variant path.
- On the advanced path (caller supplies their own `OrtSessionOptions`),
path-valued options are carried over from the variant for keys the
caller did not set, so a model that needs its external-initializers
folder still loads.

**2. Remove the legacy directory-based `CreateSession(package_dir)`
path.**
- Removes the `is_directory(package_root)` branch in
`CreateSessionAndLoadModelImpl`. Model packages are loaded through the
experimental `OrtModelPackageApi` (`CreateModelPackageContext` ->
`SelectComponent` -> `CreateSession`).
- The three end-to-end tests that exercised the directory path are
migrated to the experimental API via a `CreateSessionFromModelPackage`
test helper, preserving coverage for factory-based selection,
`PREFER_CPU` policy selection, and compiled-model compatibility scoring.

### Motivation and Context

The file-path external initializers folder support (#29459) lets the
model package flow drop its workaround of memory-mapping the model and
forcing a buffer load: it sets the folder option and loads from the
selected variant path directly. Building on that, the dedicated
`external_data` field folds into the general session-options mechanism.
A variant declares
`session.model_external_initializers_file_folder_path` (or other
path-valued options) in its `session_options`, and ORT resolves those
values against the package at parse time. This removes special-case code
and lets other path-valued options such as the EPContext file path be
resolved the same way. An allowlist is used rather than value-syntax
sniffing because session-option values are arbitrary strings; only known
path-valued keys are resolved, so ordinary values pass through
untouched.

The legacy directory-based `CreateSession(package_dir)` path was meant
to be removed when the experimental `OrtModelPackageApi` landed. It only
did variant/EP selection and never merged the variant's
`session_options`/`provider_options`, so a package loaded via
`Ort::Session(env, dir, so)` silently ignored its manifest session
options, inconsistent with the experimental API. Removing it leaves a
single, consistent way to load a model package.

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants