Skip to content

Speed up filetree archive staging#76

Closed
woutervanranst wants to merge 89 commits into
masterfrom
speedup-filetree-upload
Closed

Speed up filetree archive staging#76
woutervanranst wants to merge 89 commits into
masterfrom
speedup-filetree-upload

Conversation

@woutervanranst

@woutervanranst woutervanranst commented Apr 29, 2026

Copy link
Copy Markdown
Owner

Summary

  • replace manifest-based filetree assembly with hashed .staging directory nodes, staging sessions/writers, and a staging-driven FileTreeBuilder
  • wire archive to the new staging flow, add FileTreeService.EnsureStoredAsync, and remove obsolete manifest code
  • add recovery and validation coverage for staging lock contention, cancellation, cleanup failures, and record the ArchiveStep benchmark result

Test Plan

  • dotnet test --project \"src/Arius.Core.Tests/Arius.Core.Tests.csproj\"
  • dotnet test --project \"src/Arius.Integration.Tests/Arius.Integration.Tests.csproj\" --treenode-filter \"/*/*/FileTreeBuilderIntegrationTests/*|/*/*/RoundtripTests/*\"
  • slopwatch analyze --fail-on warning
  • dotnet run --project \"src/Arius.Benchmarks/Arius.Benchmarks.csproj\" -c Release

Benchmark

  • Archive_Step_V1_Representative_Azurite: mean 34.24 s, stddev 0.292 s, allocated 455.67 MB

Summary by CodeRabbit

  • New Features

    • Implemented file-tree staging system for archive building
    • Added performance benchmarks for archive operations
  • Documentation

    • Added architecture decision record and design specification for file-tree staging
  • Tests

    • Expanded archive recovery test coverage
    • Added file-tree staging writer tests
    • Updated file-tree builder tests
  • Refactor

    • Replaced manifest-based building with disk-backed file-tree staging
    • Improved file-tree serializer code reuse
    • Enhanced concurrency handling in file-tree operations

woutervanranst and others added 30 commits April 29, 2026 15:15
…woutervanranst/Arius7 into speedup-filetree-upload"

This reverts commit 353e702, reversing
changes made to 46ab8fd.
Preserve leading and trailing spaces in filetree path segments while keeping canonical path validation for rooted paths, dot segments, repeated separators, and backslashes.
@woutervanranst

Copy link
Copy Markdown
Owner Author

@CodeRabbit please review

@coderabbitai

coderabbitai Bot commented May 2, 2026

Copy link
Copy Markdown
Contributor

Tip

For best results, initiate chat on the files or code changes.

@woutervanranst Sure! I'll do a full review of PR #76 now.

[full_review]

@woutervanranst

Copy link
Copy Markdown
Owner Author

Superseded by a replacement PR with ADR-aligned title and description.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant