Introduce PTEFile class

[lucylq]

Stack from ghstack (oldest at bottom):

• -> Introduce PTEFile class #15800

PTEFile class holds the components of a PTE file: the program, mutable constants and named data.

Currently, the program definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the program concept.

Now, segment data is included in the PTEFile class.

Differential Revision: D86814175

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/15800

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 6 New Failures, 2 Unrelated Failures

As of commit 6c8e9e4 with merge base b1e3e28 ():

NEW FAILURES - The following jobs have failed:

• Test CUDA Builds / test-model-cuda-e2e (openai, whisper-large-v3-turbo, non-quantized) / linux-job (gh)
  RuntimeError: Command docker exec -t 82630ac29d815c4828976b31cc2c9cf790a0323f1b151774b86c0d2c57339dcb /exec failed with exit code 1
• Test CUDA Builds / test-model-cuda-e2e (openai, whisper-large-v3-turbo, quantized-int4-tile-packed) / linux-job (gh)
  RuntimeError: Command docker exec -t 81d93dc54bc9a2beeeb35b4221befa0184b1d3cbaf26853660e7ebe61a54ea68 /exec failed with exit code 1
• Test CUDA Builds / test-model-cuda-e2e (openai, whisper-large-v3-turbo, quantized-int4-weight-only) / linux-job (gh)
  RuntimeError: Command docker exec -t d85cfe17cc5bee703e1e5cbbfe8f795a6561e111c6ba1f05bae3d1a89cc53f42 /exec failed with exit code 1
• Test CUDA Builds / test-model-cuda-e2e (openai, whisper-small, non-quantized) / linux-job (gh)
  RuntimeError: Command docker exec -t 72910975926e90f72cb469277a7e3801631028b411d6b7866705ceb355f9bf04 /exec failed with exit code 1
• Test CUDA Builds / test-model-cuda-e2e (openai, whisper-small, quantized-int4-tile-packed) / linux-job (gh)
  RuntimeError: Command docker exec -t dea75d85fc51e58e2a7749c0d351ad2040489dd4d6f01fe3b60d9d6e7bb25d6a /exec failed with exit code 1
• Test CUDA Builds / test-model-cuda-e2e (openai, whisper-small, quantized-int4-weight-only) / linux-job (gh)
  RuntimeError: Command docker exec -t 1e49b4e29bb6e68795feb5f467587bc6beb4b5758f015e7a597bc5c7f89a09c8 /exec failed with exit code 1

FLAKY - The following job failed but was likely due to flakiness present on trunk:

• Test Metal Backend / test-model-metal-e2e (openai, whisper-small, non-quantized) / macos-job (gh) (similar failure)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 1

BROKEN TRUNK - The following job failed but was present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• Test Metal Backend / test-model-metal-e2e (openai, whisper-large-v3-turbo, non-quantized) / macos-job (gh) (trunk failure)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 1

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

[zingo]

It might be nice it bundleIO BPTE files could be mixed into this in some way and handled similarly.

[lucylq]

It might be nice it bundleIO BPTE files could be mixed into this in some way and handled similarly.

@zingo thanks for the suggestion - could you comment a bit more on what you're looking for?

[zingo]

It might be nice it bundleIO BPTE files could be mixed into this in some way and handled similarly.

@zingo thanks for the suggestion - could you comment a bit more on what you're looking for?

Im thinking the it might be god to have if you made tools parsing the different files. As Im not writing those kind of tools right now Im not sure its fully applicable but I feel the bpte are sometimes forgotten or need special handling.

[Copilot]

Pull Request Overview

This PR introduces a new PTEFile class to wrap together the components needed for PTE (PyTorch Edge) file serialization and deserialization: the program, mutable data, and named data. This is a breaking API change that updates deserialize_pte_binary() to return a PTEFile instead of a Program.

Key Changes:

• Introduces PTEFile dataclass to encapsulate Program, mutable data segments, and named data
• Refactors segment restoration logic by extracting _restore_constant_segment() function
• Updates deserialize_pte_binary() return type from Program to PTEFile
• Updates all call sites to access .program property from the returned PTEFile object

Reviewed Changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
exir/_serialize/_program.py	Core implementation: adds PTEFile class, refactors _restore_constant_segment(), updates _restore_segments() and deserialize_pte_binary() return types to support mutable and named data extraction
exir/_serialize/init.py	Exports the new PTEFile class as _PTEFile
exir/_serialize/test/test_program.py	Updates tests to handle PTEFile return type, adds helper methods _check_named_data_entries() and _check_named_data_store_output() for validation, adds roundtrip serialization tests
exir/emit/test/test_emit.py	Updates test to access .program from deserialized PTEFile
examples/qualcomm/oss_scripts/llama/decoder_utils.py	Updates to access .program from deserialize_pte_binary() result
codegen/tools/gen_ops_def.py	Updates to access .program from deserialize_pte_binary() result
backends/qualcomm/utils/utils.py	Updates to access .program from deserialize_pte_binary() result
backends/cadence/runtime/runtime.py	Updates to access .program from nested deserialize_pte_binary() result

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.