Pipeline bounded AGENTS.md and Git root probes - #29870
Merged
Merged
Conversation
anp-oai
approved these changes
Jun 24, 2026
anp-oai
left a comment
Contributor
There was a problem hiding this comment.
Couple test nits/gaps, lgtm to me otherwise
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
When Codex uses a remote
ExecutorFileSystem, everyget_metadatacall is an exec-server round trip. Upward discovery currently pays those round trips serially in two latency-sensitive places:AGENTS.md; andThe goal is to remove the serial ancestor dependency without adding a new filesystem RPC, JSON-RPC batch method, Git executable dependency, or cache.
Example
Assume this layout, with
.gitas the configured project-root marker:The marker probes have this required precedence:
Previously, probe 2 was not sent until probe 1 returned, and probe 3 was not sent until probe 2 returned. With this change, the client lazily keeps up to eight ordinary
fs/getMetadatarequests in flight, but consumes their results in the order above. Codex must still learn that probes 1 and 2 are absent before accepting probe 3, so the nearest root always wins. Once probe 3 succeeds, the client has its answer and stops awaiting probes 4 and 5. Requests that were already sent may still finish on the worker.For the marker phase alone, with a 50 ms client-to-worker round trip and fast local metadata calls, finding the root at probe 3 changes from roughly three serialized round trips (150 ms) to one round trip plus worker processing. The later
AGENTS.mdcandidate phase remains separate and ordered.Only after
/workspace/repois selected doesAGENTS.mddiscovery check instruction candidates, in root-to-cwd order:The first configured candidate found in each directory wins. These checks remain ordered and no instruction candidate above
/workspace/repois issued. Git-root discovery uses the same bounded lookup with only.gitas the marker.What changed
ancestor x markerprobes lazily, nearest directory first and configured marker order within each directory.AbsolutePathBuf, converting only each complete.gitprobe toPathUri. This preserves native paths that require an opaque URI fallback, such as Windows namespace paths.AGENTS.mddiscovery propagates non-NotFoundmetadata errors, while Git discovery treats a failed marker probe as absent and continues upward.AGENTS.mddirectly instead of statting it a second time.No filesystem trait or exec-server protocol method is added. An empty
project_root_markerslist performs no ancestor-marker I/O and checks instruction candidates only incwd. This change also deliberately does not cache roots across turns.Symlinks
Upward traversal remains lexical. The helper does not canonicalize
cwd; it appends marker names to the supplied path and walks that path's textual parents. The filesystem performs the actual metadata/read operation, and the current local and exec-server implementations follow live symlink targets.For example:
The lexical probes are
/tmp/pkg/src/.git,/tmp/pkg/.git,/tmp/.git, and/.git. They do not jump from/tmp/pkgto the target's parent/workspace/repo, so this spelling ofcwddoes not discover/workspace/repo/.git. That is the existing behavior and is unchanged by this PR.Conversely, if
/tmp/repo -> /workspace/repo, then probing/tmp/repo/.gitfollows the directory symlink and finds/workspace/repo/.git; the reported root remains the lexical path/tmp/repo. A live symlink used directly as.git, another configured marker, orAGENTS.mdis also followed. A symlinkedAGENTS.mdis loaded when its target is a regular file, while a broken symlink behaves asNotFound.