Skip to content

Simplify list query walk and local filesystem pairing#106

Merged
woutervanranst merged 21 commits into
masterfrom
listquery-rewrite2
Jun 12, 2026
Merged

Simplify list query walk and local filesystem pairing#106
woutervanranst merged 21 commits into
masterfrom
listquery-rewrite2

Conversation

@woutervanranst

@woutervanranst woutervanranst commented Jun 12, 2026

Copy link
Copy Markdown
Owner

Summary

  • rewrite ListQueryHandler as a single-stage breadth-first remote/local directory walk
  • replace the local snapshot builder with LocalDirectoryReader and symmetric list-query models
  • simplify local filesystem enumeration to return RelativePath directly and update affected callers/tests

Testing

  • Not run in this session

Summary by CodeRabbit

Release Notes

  • Documentation

    • Added comprehensive design specifications for dynamic, self-describing shard-prefix architecture and list-query operation enhancements.
  • Specification Updates

    • Updated list-query specification to define breadth-first traversal semantics for recursive directory listing. All entries within a directory are now enumerated completely before descending into subdirectories, improving consistency and predictability of listing results.
  • Tests

    • Updated list-query test suite to validate breadth-first traversal behavior.

chore: add specs of recent fable implementations
…eSystem.EnumerateFiles and

  EnumerateDirectories now return IEnumerable<RelativePath> directly, and every caller lost its .Path hop
  (archive's LocalFileEnumerator, ListQuery's LocalDirectoryReader.PairFiles, the three E2E sites, and the
  filesystem tests). The architecture test's internal-type list dropped the two deleted entries. LocalFile stays
  untouched in ListQuery as the paired logical-file type.
@coderabbitai

coderabbitai Bot commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Warning

Review limit reached

@woutervanranst, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 32 minutes and 57 seconds. Learn how PR review limits work.

Your organization has run out of usage credits. Purchase more credits in the billing tab to continue.

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: f6b209b4-e901-4c23-b1ba-f0b42517ea85

📥 Commits

Reviewing files that changed from the base of the PR and between 0dc62bb and 6c8f7d1.

📒 Files selected for processing (6)
  • .agents/skills/dotnet-coverage/SKILL.md
  • .github/workflows/ci.yml
  • src/Arius.Cli.Tests/Commands/Ls/ListQueryParsingTests.cs
  • src/Arius.Core.Tests/Features/ListQuery/ListQueryHandlerTests.cs
  • src/Arius.Core/Features/ListQuery/LocalDirectoryReader.cs
  • src/Arius.Core/Shared/FileSystem/PathSegmentComparers.cs
📝 Walkthrough

Walkthrough

This PR executes the list-query handler simplification design plan by refactoring the handler from a multi-stage channel-based pipeline into a breadth-first async iterator, introducing symmetric local/remote directory listing models, and simplifying filesystem enumeration APIs to return paths directly instead of wrapper objects.

Changes

List-query handler simplification and filesystem refactor

Layer / File(s) Summary
Design plans and spec update
openspec/changes/2026-06-10-3-dynamic-shard-length/PLAN.MD, openspec/changes/2026-06-12-simplify-list/PLAN.MD, openspec/specs/list-query/spec.md
Documents rationale for handler simplification, dynamic shard-length future work, and updates specification to require breadth-first traversal when Recursive=true.
FileSystem enumeration API refactor
src/Arius.Core/Shared/FileSystem/RelativeFileSystem.cs
EnumerateDirectories() and EnumerateFiles() now return IEnumerable<RelativePath> instead of entry wrapper objects, removing the Local*Entry wrapper construction overhead.
Centralized path segment comparers
src/Arius.Core/Shared/FileSystem/PathSegmentComparers.cs, src/Arius.Core/Shared/FileTree/FileTreeSerializer.cs, src/Arius.Core/Shared/FileTree/FileTreeService.cs
Introduces centralized PathSegmentOrdinalIgnoreCaseComparer and PathSegmentOrdinalComparer singletons; removes duplicate comparer implementations from serializer and service.
Directory listing model types
src/Arius.Core/Features/ListQuery/Models.cs
Adds RemoteDirectoryListing, LocalDirectoryListing, and LocalFile records representing the symmetric vocabulary for merging persisted and on-disk directory contents.
Local directory reader and file pairing
src/Arius.Core/Features/ListQuery/LocalDirectoryReader.cs
New utility that enumerates a single local directory, pairs files with .pointer.arius sidecars via PairFiles, and returns a LocalDirectoryListing with pointer/binary existence flags and optional file metadata.
ListQueryHandler breadth-first rewrite
src/Arius.Core/Features/ListQuery/ListQueryHandler.cs, src/Arius.Core/Features/ListQuery/ListQuery.cs
Refactors handler from multi-stage channel pipeline to breadth-first async iterator that directly reads/merges remote and local directories per work item; updates RepositoryEntryState flag semantics to use independent bits for hydrated/archived/rehydrating states rather than auto-composing with Repository bit; removes channel/batch-resolve dependencies.
Update filesystem API consumers
src/Arius.Core/Features/ArchiveCommand/LocalFileEnumerator.cs, src/Arius.E2E.Tests/Datasets/SyntheticRepositoryMaterializer.cs, src/Arius.E2E.Tests/Datasets/SyntheticRepositoryStateAssertions.cs, src/Arius.E2E.Tests/Workflows/Steps/ArchiveTierLifecycleStep.cs
Updates all code iterating filesystem enumerations to use RelativePath results directly instead of entry wrapper objects with .Path properties.
Update list-query handler tests
src/Arius.Core.Tests/Features/ListQuery/ListQueryHandlerTests.cs, src/Arius.Core.Tests/Shared/FileSystem/RelativeFileSystemTests.cs
Retargets unit tests to use new LocalDirectoryReader.PairFiles API; removes intermediate .Select(entry => entry.Path) projections; updates assertions to validate pointer/binary existence and stat-based metadata.
Architecture and dependency adjustments
src/Arius.Architecture.Tests/DependencyTests.cs
Updates internal-types allowlist: removes LocalDirectoryEntry and LocalFileEntry from visibility checks; adds RelativeFileSystem to non-public type assertions.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

  • woutervanranst/Arius7#48: Earlier list-query naming refactor (Ls*ListQuery* across tests/specs) touches the same test/spec surface area.
  • woutervanranst/Arius7#103: Directly precedes this PR with foundational list-query pipeline changes and RepositoryEntryState flag updates.
  • woutervanranst/Arius7#81: Introduces the RelativeFileSystem domain types that this PR simplifies (removing Local*Entry-based enumeration in favor of direct path yields).
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 38.71% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The pull request title directly and accurately summarizes the main objective: simplifying the list query walk implementation and local filesystem pairing logic, which are the core refactoring themes across the changeset.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch listquery-rewrite2

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@codecov

codecov Bot commented Jun 12, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 97.14286% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 80.39%. Comparing base (b435cc9) to head (6c8f7d1).

Files with missing lines Patch % Lines
...us.Core/Features/ListQuery/LocalDirectoryReader.cs 94.00% 3 Missing ⚠️
.../Arius.Core/Features/ListQuery/ListQueryHandler.cs 97.87% 0 Missing and 2 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #106      +/-   ##
==========================================
- Coverage   80.47%   80.39%   -0.08%     
==========================================
  Files          91       93       +2     
  Lines        6166     6081      -85     
  Branches      824      815       -9     
==========================================
- Hits         4962     4889      -73     
+ Misses        976      964      -12     
  Partials      228      228              
Flag Coverage Δ
linux 83.32% <97.14%> (-0.04%) ⬇️
windows 79.18% <97.14%> (-0.13%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (3)
openspec/changes/2026-06-12-simplify-list/PLAN.MD (1)

22-47: 💤 Low value

Add language specifiers to code blocks for syntax highlighting.

The code blocks containing C# code should declare the language to enable syntax highlighting and improve readability on documentation platforms.

Proposed fix for code block language specifiers
-```csharp
+```csharp
 /// <summary>What the repository knows about one directory: the persisted filetree node,

(Apply the same fix to all code blocks: add csharp after the opening ```.)

Also applies to: 59-96

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@openspec/changes/2026-06-12-simplify-list/PLAN.MD` around lines 22 - 47, Add
the C# language specifier to the fenced code blocks in PLAN.MD so syntax
highlighting is enabled for the shown types (e.g., the blocks containing
RemoteDirectoryListing, LocalDirectoryListing and LocalFile). Edit the opening
fence for each affected block from ``` to ```csharp (apply to the block starting
around the snippet that defines RemoteDirectoryListing, the block with
LocalDirectoryListing.Empty, and the block with LocalFile) and leave the
contents unchanged.
src/Arius.Core/Features/ListQuery/LocalDirectoryReader.cs (1)

80-86: 💤 Low value

Edge case: pointer enumerated but binary not enumerated (yet exists on disk) results in no entry.

When fileExists(binaryPath) returns true but the binary wasn't enumerated (a race where binary was created mid-enumeration), line 83 skips the pointer assuming the binary pass will handle it. However, the binary won't be iterated since it's not in filePaths, resulting in no LocalFile entry for this pair.

This is a narrow race window and the degradation is acceptable (the file will appear on next list), but consider whether the intent is to create a BinaryExists: true entry here.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/Arius.Core/Features/ListQuery/LocalDirectoryReader.cs` around lines 80 -
86, The pointer-only branch currently skips creating an entry when
fileExists(binaryPath) is true, which loses the file if the binary wasn't
enumerated; in LocalDirectoryReader.cs adjust the logic so the pointer entry is
created with BinaryExists set based on fileExists(binaryPath) rather than
skipping — e.g., only skip when enumeratedPaths.Contains(binaryPath) but when
not enumerated create files[binaryName] = new LocalFile(binaryName,
BinaryExists: fileExists(binaryPath), PointerExists: true, ...); ensure you
reference variables/methods binaryPath, binaryName, fileExists, enumeratedPaths,
files and the LocalFile constructor when making the change.
src/Arius.Core/Shared/FileSystem/PathSegmentComparers.cs (1)

3-16: ⚡ Quick win

Split these comparers into separate files.

Lines 3-16 introduce two top-level types in PathSegmentComparers.cs. This breaks the repo’s C# file-layout rule and makes symbol-to-file lookup less predictable. Please move PathSegmentOrdinalIgnoreCaseComparer and PathSegmentOrdinalComparer into files that match their type names. As per coding guidelines, "Prefer one top-level class per file with the filename matching the class name."

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/Arius.Core/Shared/FileSystem/PathSegmentComparers.cs` around lines 3 -
16, The file contains two top-level comparer types; split them so each top-level
type lives in its own file: create PathSegmentOrdinalIgnoreCaseComparer.cs
containing the internal sealed class PathSegmentOrdinalIgnoreCaseComparer with
its static Instance and Compare implementation, and create
PathSegmentOrdinalComparer.cs containing the internal sealed class
PathSegmentOrdinalComparer with its static Instance and Compare implementation;
remove the duplicate type declarations from PathSegmentComparers.cs and ensure
both new files use the same namespace as the original file and keep the existing
signatures (Compare(PathSegment x, PathSegment y) and Instance property)
unchanged.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/Arius.Core/Features/ListQuery/LocalDirectoryReader.cs`:
- Around line 40-47: The stat lambda passed to PairFiles in
LocalDirectoryReader.cs can throw if files are removed or inaccessible; wrap the
logic that calls fileSystem.GetTimestamps(path) and fileSystem.GetFileSize(path)
in a try/catch that returns null (or a nullable metadata tuple) on failure and
logs a warning (mirroring the graceful degradation used for enumeration), or
alternatively update PairFiles to accept a nullable-returning stat delegate so
stat failures do not abort the entire PairFiles operation.

---

Nitpick comments:
In `@openspec/changes/2026-06-12-simplify-list/PLAN.MD`:
- Around line 22-47: Add the C# language specifier to the fenced code blocks in
PLAN.MD so syntax highlighting is enabled for the shown types (e.g., the blocks
containing RemoteDirectoryListing, LocalDirectoryListing and LocalFile). Edit
the opening fence for each affected block from ``` to ```csharp (apply to the
block starting around the snippet that defines RemoteDirectoryListing, the block
with LocalDirectoryListing.Empty, and the block with LocalFile) and leave the
contents unchanged.

In `@src/Arius.Core/Features/ListQuery/LocalDirectoryReader.cs`:
- Around line 80-86: The pointer-only branch currently skips creating an entry
when fileExists(binaryPath) is true, which loses the file if the binary wasn't
enumerated; in LocalDirectoryReader.cs adjust the logic so the pointer entry is
created with BinaryExists set based on fileExists(binaryPath) rather than
skipping — e.g., only skip when enumeratedPaths.Contains(binaryPath) but when
not enumerated create files[binaryName] = new LocalFile(binaryName,
BinaryExists: fileExists(binaryPath), PointerExists: true, ...); ensure you
reference variables/methods binaryPath, binaryName, fileExists, enumeratedPaths,
files and the LocalFile constructor when making the change.

In `@src/Arius.Core/Shared/FileSystem/PathSegmentComparers.cs`:
- Around line 3-16: The file contains two top-level comparer types; split them
so each top-level type lives in its own file: create
PathSegmentOrdinalIgnoreCaseComparer.cs containing the internal sealed class
PathSegmentOrdinalIgnoreCaseComparer with its static Instance and Compare
implementation, and create PathSegmentOrdinalComparer.cs containing the internal
sealed class PathSegmentOrdinalComparer with its static Instance and Compare
implementation; remove the duplicate type declarations from
PathSegmentComparers.cs and ensure both new files use the same namespace as the
original file and keep the existing signatures (Compare(PathSegment x,
PathSegment y) and Instance property) unchanged.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 50882d21-ca39-4657-81e0-089ea4875f28

📥 Commits

Reviewing files that changed from the base of the PR and between b435cc9 and 0dc62bb.

📒 Files selected for processing (26)
  • openspec/changes/2026-06-10-1-rewrite-list/2026-06-10-234022-local-command-caveatcaveat-the-messages-below.txt
  • openspec/changes/2026-06-10-1-rewrite-list/2026-06-11-010913-this-session-is-being-continued-from-a-previous-c.txt
  • openspec/changes/2026-06-10-1-rewrite-list/PLAN.MD
  • openspec/changes/2026-06-10-1-rewrite-list/PLAN2-Coderabbit fixes.MD
  • openspec/changes/2026-06-10-2-rewrite-restore/PLAN.MD
  • openspec/changes/2026-06-10-3-dynamic-shard-length/PLAN.MD
  • openspec/changes/2026-06-12-simplify-list/PLAN.MD
  • openspec/specs/list-query/spec.md
  • src/Arius.Architecture.Tests/DependencyTests.cs
  • src/Arius.Core.Tests/Features/ListQuery/ListQueryHandlerTests.cs
  • src/Arius.Core.Tests/Shared/FileSystem/RelativeFileSystemTests.cs
  • src/Arius.Core/Features/ArchiveCommand/LocalFileEnumerator.cs
  • src/Arius.Core/Features/ListQuery/ListQuery.cs
  • src/Arius.Core/Features/ListQuery/ListQueryHandler.cs
  • src/Arius.Core/Features/ListQuery/LocalDirectoryReader.cs
  • src/Arius.Core/Features/ListQuery/LocalFileSnapshotBuilder.cs
  • src/Arius.Core/Features/ListQuery/Models.cs
  • src/Arius.Core/Shared/FileSystem/LocalDirectoryEntry.cs
  • src/Arius.Core/Shared/FileSystem/LocalFileEntry.cs
  • src/Arius.Core/Shared/FileSystem/PathSegmentComparers.cs
  • src/Arius.Core/Shared/FileSystem/RelativeFileSystem.cs
  • src/Arius.Core/Shared/FileTree/FileTreeSerializer.cs
  • src/Arius.Core/Shared/FileTree/FileTreeService.cs
  • src/Arius.E2E.Tests/Datasets/SyntheticRepositoryMaterializer.cs
  • src/Arius.E2E.Tests/Datasets/SyntheticRepositoryStateAssertions.cs
  • src/Arius.E2E.Tests/Workflows/Steps/ArchiveTierLifecycleStep.cs
💤 Files with no reviewable changes (6)
  • src/Arius.Core/Shared/FileSystem/LocalFileEntry.cs
  • src/Arius.Core/Features/ListQuery/LocalFileSnapshotBuilder.cs
  • src/Arius.Core/Shared/FileSystem/LocalDirectoryEntry.cs
  • src/Arius.Architecture.Tests/DependencyTests.cs
  • src/Arius.Core/Shared/FileTree/FileTreeSerializer.cs
  • src/Arius.Core/Shared/FileTree/FileTreeService.cs

Comment thread src/Arius.Core/Features/ListQuery/LocalDirectoryReader.cs
@woutervanranst woutervanranst merged commit 6b14487 into master Jun 12, 2026
12 checks passed
@woutervanranst woutervanranst deleted the listquery-rewrite2 branch June 12, 2026 16:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant