Skip to content

Cache plugin namespace during executor skill discovery - #29831

Merged
jif-oai merged 3 commits into
mainfrom
jif/cache-executor-skill-namespace
Jun 24, 2026
Merged

Cache plugin namespace during executor skill discovery#29831
jif-oai merged 3 commits into
mainfrom
jif/cache-executor-skill-namespace

Conversation

@jif-oai

@jif-oai jif-oai commented Jun 24, 2026

Copy link
Copy Markdown
Contributor

Why

Executor skill discovery runs before the remote skills catalog is available. For a remote environment, each ExecutorFileSystem operation becomes an exec-server RPC.

Previously, every discovered SKILL.md independently resolved its plugin namespace by walking its ancestors and probing both supported manifest locations. In the common plugin/skills/<skill>/SKILL.md layout, that repeats 8 RPCs per skill even though every skill under the plugin root uses the same namespace. These lookups happen while skills are parsed, so their cost grows linearly with the skill count and adds directly to first-turn latency.

A selected capability root can also contain standalone skills, multiple sibling plugins, nested plugins, or symlinked directories. The optimization therefore needs to retain the nearest-ancestor namespace for each skill rather than assuming the selected root represents exactly one plugin.

What changed

  • record plugin-root candidates from directory entries already returned during skill discovery
  • prune candidates that are not ancestors of any discovered SKILL.md before reading manifests
  • resolve each relevant plugin root once, with one fallback lookup per canonical traversal root for symlinked directories
  • select the nearest cached plugin namespace for each discovered skill
  • avoid namespace lookup entirely when the root contains no skills

No additional directory traversal is required. Namespace work now scales with the number of plugin roots that contain discovered skills, rather than the total number of skills or unrelated sibling plugins. Standalone and nested-plugin names keep their previous behavior.

Benchmarks

I used a temporary counting ExecutorFileSystem around the real local filesystem. Each filesystem operation was counted as one remote RPC and given 1 ms of injected latency. Each variant ran three times; times below are medians.

One plugin with 100 skills

Operation Before After Delta
get_metadata 1,002 303 -699
read_file 200 101 -99
read_directory 102 102 0
Total filesystem RPCs 1,304 506 -798 (-61.2%)
Median load time 2.890 s 0.997 s 2.90× faster

The namespace-specific work drops from 800 RPCs to 2 in this layout.

Multiple plugins under one selected root

These runs compare the correct pre-optimization implementation with the final nearest-plugin-root cache. The total plugin skill count stays at 100 while the number of plugin roots changes.

Layout Before RPCs After RPCs Reduction Before After Speedup
2 plugins × 50 skills 1,312 530 59.6% 1,819 ms 711 ms 2.56×
10 plugins × 10 skills 1,344 578 57.0% 1,850 ms 778 ms 2.38×
50 plugins × 2 skills 1,504 818 45.6% 2,094 ms 1,086 ms 1.93×
10 plugins × 10 skills + 10 standalone skills 1,596 630 60.5% 2,209 ms 860 ms 2.57×

The remaining cost grows with the number of relevant plugin manifests. Each relevant manifest is read once instead of once per skill, while sibling plugins with no discovered skills are not read. Absolute latency savings depend on the executor's real RPC latency.

Tests

  • just test -p codex-core-skills (109 passed across the library and integration-test binaries)
  • one integration test covers standalone, outer-plugin, nested-plugin, and unused sibling-plugin layouts, and asserts the exact set of manifests read

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 472b5ece5f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread codex-rs/core-skills/src/loader/environment.rs Outdated
@jif-oai

jif-oai commented Jun 24, 2026

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown
Contributor

Codex Review: Didn't find any major issues. More of your lovely PRs please.

Reviewed commit: 2e2290c2f4

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@jif-oai
jif-oai merged commit 390b731 into main Jun 24, 2026
31 checks passed
@jif-oai
jif-oai deleted the jif/cache-executor-skill-namespace branch June 24, 2026 16:14
@github-actions github-actions Bot locked and limited conversation to collaborators Jun 24, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant