Qualcomm AI Engine Direct - Meta CI for Mobilebert , W2L, and Llama

[winskuo-quic]

Summary

• Some fix from Llama
• Mobilebert CI enablement
• Wav2Letter CI enablement

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/8616

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure

As of commit 275164d with merge base 24671a9 ():

NEW FAILURE - The following job has failed:

• trunk / test-llama-runner-mac (fp32, coreml) / macos-job (gh)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 1

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[winskuo-quic]

Hi @guangy10, @cccclai,
This PR should be able to fix some of the CI issues: including Mobilebert , W2L, and Llama.
For now, I am using 1 epoch for Mobilebert to expedite training process.
For W2L, I am using empty weights since we are just compiling the model. However, if weights can cause regression during inference, we should probably be using real weights in future.
Also, I notice that most of the example files(e.g., ic3. mv3, etc) are using only 1 random input when setting args.compile_only. This is probably not straight forward to community users, and I think we should either consider creating a new flag for CI or use actual data set when compiling the model.
Please let me know your thoughts. Thanks

[cccclai]

There is an CI error, can you fix it?

[winskuo-quic]

There is an CI error, can you fix it?

Thanks for the catch.
Another thing we noticed is that we are using nightly environment to develop, while it seems like CI is not using nightly libraries. Think this can possibly cause CI to fail sometimes while our environment is working fine. For example: #7634 (comment) could possibly be caused by using different libraries version for torchaudio.

[cccclai]

There is an CI error, can you fix it?

Thanks for the catch. Another thing we noticed is that we are using nightly environment to develop, while it seems like CI is not using nightly libraries. Think this can possibly cause CI to fail sometimes while our environment is working fine. For example: #7634 (comment) could possibly be caused by using different libraries version for torchaudio.

Hmm I think CI is using this version

executorch/install_requirements.py

Line 70 in 65432b1

NIGHTLY_VERSION = "dev20250131"

maybe we need to bump the version more regularly.

[facebook-github-bot]

@cccclai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

[winskuo-quic]

There is an CI error, can you fix it?

Thanks for the catch. Another thing we noticed is that we are using nightly environment to develop, while it seems like CI is not using nightly libraries. Think this can possibly cause CI to fail sometimes while our environment is working fine. For example: #7634 (comment) could possibly be caused by using different libraries version for torchaudio.

Hmm I think CI is using this version

executorch/install_requirements.py

Line 70 in 65432b1

NIGHTLY_VERSION = "dev20250131"

maybe we need to bump the version more regularly.

@cccclai ,
Unsure if I traced this correctly, but seems like the CI starts off from
setup-linux.sh, which calls install_executorch "use-pt-pinned-commit"
and inside install_executorch.py, I saw the following code block that sets use_pytorch_nightly = False if args.use_pt_pinned_commit, as shown below:

Therefore, it will not be using the nightly version but the released version.

[cccclai]

@guangy10 @kirklandsign Do you know which version we're running?

[cccclai]

This PR needs to rebase...

[facebook-github-bot]

@cccclai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

[cccclai]

Still failing..maybe it's related to #8642. Can you rebase again?

[cccclai]

Hmm looks like CI is failing..

[facebook-github-bot]

@cccclai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.