Skip to content

maintenance: add new cache-local-objects maintenance task#720

Merged
mjcheetham merged 4 commits intomicrosoft:vfs-2.47.2from
mjcheetham:scalar-cache-task
Jan 31, 2025
Merged

maintenance: add new cache-local-objects maintenance task#720
mjcheetham merged 4 commits intomicrosoft:vfs-2.47.2from
mjcheetham:scalar-cache-task

Conversation

@mjcheetham
Copy link
Member

@mjcheetham mjcheetham commented Jan 21, 2025 •
edited
Loading

Introduce a new maintenance task, cache-local-objects, that operates on Scalar or VFS for Git repositories with a per-volume, shared object cache (specified by gvfs.sharedCache) to migrate packfiles and loose objects from the repository object directory to the shared cache.

Older versions of microsoft/git incorrectly placed packfiles in the repository object directory instead of the shared cache; this task will help clean up existing clones impacted by that issue.

Fixes #716

@mjcheetham mjcheetham force-pushed the scalar-cache-task branch 2 times, most recently from 1c29b72 to 4a5255d Compare January 21, 2025 09:41
dscho
dscho reviewed Jan 21, 2025
View reviewed changes
@mjcheetham mjcheetham force-pushed the scalar-cache-task branch 2 times, most recently from cb383ec to 904f61a Compare January 21, 2025 13:47
derrickstolee
derrickstolee reviewed Jan 21, 2025
View reviewed changes
derrickstolee
derrickstolee reviewed Jan 21, 2025
View reviewed changes
@mjcheetham mjcheetham changed the base branch from vfs-2.47.1 to vfs-2.47.2 January 22, 2025 14:14
@mjcheetham mjcheetham force-pushed the scalar-cache-task branch 2 times, most recently from 9c30bd3 to e3d64ab Compare January 22, 2025 15:07
@mjcheetham mjcheetham marked this pull request as ready for review January 22, 2025 15:08
@mjcheetham mjcheetham force-pushed the scalar-cache-task branch 2 times, most recently from 66a83df to 6981c37 Compare January 23, 2025 11:06
@mjcheetham
Copy link
Member Author

mjcheetham commented Jan 23, 2025

@derrickstolee @dscho I've addressed the issues raised in the comments so far. Please could I have another look over? Thanks! :)

dscho
dscho reviewed Jan 23, 2025
View reviewed changes
builtin/gc.c Outdated
Comment on lines +1434 to +1435
static void move_pack_to_vfs_cache(const char *full_path, size_t full_path_len,
const char *file_name, UNUSED void *data)
Copy link
Member

@dscho dscho Jan 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might make sense to merge migrate_pack() directly into this function, after the .pack suffix check.

@mjcheetham mjcheetham force-pushed the scalar-cache-task branch 2 times, most recently from 097250d to dd3cb57 Compare January 28, 2025 16:46
derrickstolee
derrickstolee reviewed Jan 29, 2025
View reviewed changes
dscho
dscho previously approved these changes Jan 30, 2025
View reviewed changes
Copy link
Member

@dscho dscho left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me!

@mjcheetham
fixup! maintenance: care about gvfs.sharedCache config
b2aaf4f
@mjcheetham
t7900-maintenance.sh: reset config between tests
385e620 
Tests in t7900 assume the state of the `maintenance.strategy`
config setting; set/unset by previous tests. Correct this by
explictly unsetting and re-setting the config at the start of the
tests.

Signed-off-by: Matthew John Cheetham <mjcheetham@outlook.com>
@mjcheetham
maintenance: add cache-local-objects maintenance task
784e82b 
Introduce a new maintenance task, `cache-local-objects`, that operates
on Scalar or VFS for Git repositories with a per-volume, shared object
cache (specified by `gvfs.sharedCache`) to migrate packfiles and loose
objects from the repository object directory to the shared cache.

Older versions of `microsoft/git` incorrectly placed packfiles in the
repository object directory instead of the shared cache; this task will
help clean up existing clones impacted by that issue.

Migration of packfiles involves the following steps for each pack:

1. Hardlink (or copy):
   a. the .pack file
   b. the .keep file
   c. the .rev file
2. Move (or copy + delete) the .idx file
3. Delete/unlink:
   a. the .pack file
   b. the .keep file
   c. the .rev file

Moving the index file after the others ensures the pack is not read
from the new cache directory until all associated files (rev, keep)
exist in the cache directory also.

Moving loose objects operates as a move, or copy + delete.

Signed-off-by: Matthew John Cheetham <mjcheetham@outlook.com>
@mjcheetham
scalar.c: add cache-local-objects task
03e984b 
Add the `cache-local-objects` maintenance task to the list of tasks run
by the `scalar run` command. It's often easier for users to run the
shorter `scalar run` command than the equivalent `git maintenance`
command.

Signed-off-by: Matthew John Cheetham <mjcheetham@outlook.com>
@mjcheetham mjcheetham changed the title maintenance: add new vfs-cache-move maintenance task maintenance: add new cache-local-objects maintenance task Jan 30, 2025
mjcheetham added a commit that referenced this pull request Mar 17, 2025
@mjcheetham
maintenance: add new cache-local-objects maintenance task (#720)
4829785 
Introduce a new maintenance task, `cache-local-objects`, that operates
on Scalar or VFS for Git repositories with a per-volume, shared object
cache (specified by `gvfs.sharedCache`) to migrate packfiles and loose
objects from the repository object directory to the shared cache.

Older versions of `microsoft/git` incorrectly placed packfiles in the
repository object directory instead of the shared cache; this task will
help clean up existing clones impacted by that issue.

Fixes #716
dscho pushed a commit that referenced this pull request Jun 6, 2025
@mjcheetham @dscho
maintenance: add new cache-local-objects maintenance task (#720)
be56709 
Introduce a new maintenance task, `cache-local-objects`, that operates
on Scalar or VFS for Git repositories with a per-volume, shared object
cache (specified by `gvfs.sharedCache`) to migrate packfiles and loose
objects from the repository object directory to the shared cache.

Older versions of `microsoft/git` incorrectly placed packfiles in the
repository object directory instead of the shared cache; this task will
help clean up existing clones impacted by that issue.

Fixes #716
dscho pushed a commit that referenced this pull request Jun 6, 2025
@mjcheetham @dscho
maintenance: add new cache-local-objects maintenance task (#720)
1522748 
Introduce a new maintenance task, `cache-local-objects`, that operates
on Scalar or VFS for Git repositories with a per-volume, shared object
cache (specified by `gvfs.sharedCache`) to migrate packfiles and loose
objects from the repository object directory to the shared cache.

Older versions of `microsoft/git` incorrectly placed packfiles in the
repository object directory instead of the shared cache; this task will
help clean up existing clones impacted by that issue.

Fixes #716
dscho pushed a commit that referenced this pull request Jun 11, 2025
@mjcheetham @dscho
maintenance: add new cache-local-objects maintenance task (#720)
b89369f 
Introduce a new maintenance task, `cache-local-objects`, that operates
on Scalar or VFS for Git repositories with a per-volume, shared object
cache (specified by `gvfs.sharedCache`) to migrate packfiles and loose
objects from the repository object directory to the shared cache.

Older versions of `microsoft/git` incorrectly placed packfiles in the
repository object directory instead of the shared cache; this task will
help clean up existing clones impacted by that issue.

Fixes #716
dscho pushed a commit that referenced this pull request Jun 13, 2025
@mjcheetham @dscho
maintenance: add new cache-local-objects maintenance task (#720)
ad5d0c9 
Introduce a new maintenance task, `cache-local-objects`, that operates
on Scalar or VFS for Git repositories with a per-volume, shared object
cache (specified by `gvfs.sharedCache`) to migrate packfiles and loose
objects from the repository object directory to the shared cache.

Older versions of `microsoft/git` incorrectly placed packfiles in the
repository object directory instead of the shared cache; this task will
help clean up existing clones impacted by that issue.

Fixes #716
dscho pushed a commit that referenced this pull request Jun 16, 2025
@mjcheetham @dscho
maintenance: add new cache-local-objects maintenance task (#720)
503c554 
Introduce a new maintenance task, `cache-local-objects`, that operates
on Scalar or VFS for Git repositories with a per-volume, shared object
cache (specified by `gvfs.sharedCache`) to migrate packfiles and loose
objects from the repository object directory to the shared cache.

Older versions of `microsoft/git` incorrectly placed packfiles in the
repository object directory instead of the shared cache; this task will
help clean up existing clones impacted by that issue.

Fixes #716
dscho pushed a commit that referenced this pull request Jun 16, 2025
@mjcheetham @dscho
maintenance: add new cache-local-objects maintenance task (#720)
73e31b7 
Introduce a new maintenance task, `cache-local-objects`, that operates
on Scalar or VFS for Git repositories with a per-volume, shared object
cache (specified by `gvfs.sharedCache`) to migrate packfiles and loose
objects from the repository object directory to the shared cache.

Older versions of `microsoft/git` incorrectly placed packfiles in the
repository object directory instead of the shared cache; this task will
help clean up existing clones impacted by that issue.

Fixes #716
dscho pushed a commit that referenced this pull request Jun 16, 2025
@mjcheetham @dscho
maintenance: add new cache-local-objects maintenance task (#720)
c4c7307 
Introduce a new maintenance task, `cache-local-objects`, that operates
on Scalar or VFS for Git repositories with a per-volume, shared object
cache (specified by `gvfs.sharedCache`) to migrate packfiles and loose
objects from the repository object directory to the shared cache.

Older versions of `microsoft/git` incorrectly placed packfiles in the
repository object directory instead of the shared cache; this task will
help clean up existing clones impacted by that issue.

Fixes #716
dscho pushed a commit that referenced this pull request Jul 8, 2025
@mjcheetham @dscho
maintenance: add new cache-local-objects maintenance task (#720)
65ef2d6 
Introduce a new maintenance task, `cache-local-objects`, that operates
on Scalar or VFS for Git repositories with a per-volume, shared object
cache (specified by `gvfs.sharedCache`) to migrate packfiles and loose
objects from the repository object directory to the shared cache.

Older versions of `microsoft/git` incorrectly placed packfiles in the
repository object directory instead of the shared cache; this task will
help clean up existing clones impacted by that issue.

Fixes #716
dscho pushed a commit that referenced this pull request Aug 5, 2025
@mjcheetham @dscho
maintenance: add new cache-local-objects maintenance task (#720)
84c8769 
Introduce a new maintenance task, `cache-local-objects`, that operates
on Scalar or VFS for Git repositories with a per-volume, shared object
cache (specified by `gvfs.sharedCache`) to migrate packfiles and loose
objects from the repository object directory to the shared cache.

Older versions of `microsoft/git` incorrectly placed packfiles in the
repository object directory instead of the shared cache; this task will
help clean up existing clones impacted by that issue.

Fixes #716
dscho pushed a commit that referenced this pull request Aug 5, 2025
@mjcheetham @dscho
maintenance: add new cache-local-objects maintenance task (#720)
44c06ba 
Introduce a new maintenance task, `cache-local-objects`, that operates
on Scalar or VFS for Git repositories with a per-volume, shared object
cache (specified by `gvfs.sharedCache`) to migrate packfiles and loose
objects from the repository object directory to the shared cache.

Older versions of `microsoft/git` incorrectly placed packfiles in the
repository object directory instead of the shared cache; this task will
help clean up existing clones impacted by that issue.

Fixes #716
dscho pushed a commit that referenced this pull request Aug 8, 2025
@mjcheetham @dscho
maintenance: add new cache-local-objects maintenance task (#720)
5731e44 
Introduce a new maintenance task, `cache-local-objects`, that operates
on Scalar or VFS for Git repositories with a per-volume, shared object
cache (specified by `gvfs.sharedCache`) to migrate packfiles and loose
objects from the repository object directory to the shared cache.

Older versions of `microsoft/git` incorrectly placed packfiles in the
repository object directory instead of the shared cache; this task will
help clean up existing clones impacted by that issue.

Fixes #716
dscho pushed a commit that referenced this pull request Aug 8, 2025
@mjcheetham @dscho
maintenance: add new cache-local-objects maintenance task (#720)
08958ae 
Introduce a new maintenance task, `cache-local-objects`, that operates
on Scalar or VFS for Git repositories with a per-volume, shared object
cache (specified by `gvfs.sharedCache`) to migrate packfiles and loose
objects from the repository object directory to the shared cache.

Older versions of `microsoft/git` incorrectly placed packfiles in the
repository object directory instead of the shared cache; this task will
help clean up existing clones impacted by that issue.

Fixes #716
dscho pushed a commit that referenced this pull request Aug 13, 2025
@mjcheetham @dscho
maintenance: add new cache-local-objects maintenance task (#720)
213e2ba 
Introduce a new maintenance task, `cache-local-objects`, that operates
on Scalar or VFS for Git repositories with a per-volume, shared object
cache (specified by `gvfs.sharedCache`) to migrate packfiles and loose
objects from the repository object directory to the shared cache.

Older versions of `microsoft/git` incorrectly placed packfiles in the
repository object directory instead of the shared cache; this task will
help clean up existing clones impacted by that issue.

Fixes #716
@dscho dscho mentioned this pull request Aug 13, 2025
Merged
dscho pushed a commit that referenced this pull request Aug 19, 2025
@mjcheetham @dscho
maintenance: add new cache-local-objects maintenance task (#720)
f61256c 
Introduce a new maintenance task, `cache-local-objects`, that operates
on Scalar or VFS for Git repositories with a per-volume, shared object
cache (specified by `gvfs.sharedCache`) to migrate packfiles and loose
objects from the repository object directory to the shared cache.

Older versions of `microsoft/git` incorrectly placed packfiles in the
repository object directory instead of the shared cache; this task will
help clean up existing clones impacted by that issue.

Fixes #716
@derrickstolee derrickstolee mentioned this pull request Aug 25, 2025
Merged
dscho pushed a commit that referenced this pull request Oct 17, 2025
@mjcheetham @dscho
maintenance: add new cache-local-objects maintenance task (#720)
e8d7e6b 
Introduce a new maintenance task, `cache-local-objects`, that operates
on Scalar or VFS for Git repositories with a per-volume, shared object
cache (specified by `gvfs.sharedCache`) to migrate packfiles and loose
objects from the repository object directory to the shared cache.

Older versions of `microsoft/git` incorrectly placed packfiles in the
repository object directory instead of the shared cache; this task will
help clean up existing clones impacted by that issue.

Fixes #716
dscho pushed a commit that referenced this pull request Oct 17, 2025
@mjcheetham @dscho
maintenance: add new cache-local-objects maintenance task (#720)
1cce335 
Introduce a new maintenance task, `cache-local-objects`, that operates
on Scalar or VFS for Git repositories with a per-volume, shared object
cache (specified by `gvfs.sharedCache`) to migrate packfiles and loose
objects from the repository object directory to the shared cache.

Older versions of `microsoft/git` incorrectly placed packfiles in the
repository object directory instead of the shared cache; this task will
help clean up existing clones impacted by that issue.

Fixes #716
dscho pushed a commit that referenced this pull request Oct 17, 2025
@mjcheetham @dscho
maintenance: add new cache-local-objects maintenance task (#720)
6cf28e3 
Introduce a new maintenance task, `cache-local-objects`, that operates
on Scalar or VFS for Git repositories with a per-volume, shared object
cache (specified by `gvfs.sharedCache`) to migrate packfiles and loose
objects from the repository object directory to the shared cache.

Older versions of `microsoft/git` incorrectly placed packfiles in the
repository object directory instead of the shared cache; this task will
help clean up existing clones impacted by that issue.

Fixes #716
dscho pushed a commit that referenced this pull request Oct 28, 2025
@mjcheetham @dscho
maintenance: add new cache-local-objects maintenance task (#720)
87eaeba 
Introduce a new maintenance task, `cache-local-objects`, that operates
on Scalar or VFS for Git repositories with a per-volume, shared object
cache (specified by `gvfs.sharedCache`) to migrate packfiles and loose
objects from the repository object directory to the shared cache.

Older versions of `microsoft/git` incorrectly placed packfiles in the
repository object directory instead of the shared cache; this task will
help clean up existing clones impacted by that issue.

Fixes #716
dscho pushed a commit that referenced this pull request Nov 5, 2025
@mjcheetham @dscho
maintenance: add new cache-local-objects maintenance task (#720)
c436fb7 
Introduce a new maintenance task, `cache-local-objects`, that operates
on Scalar or VFS for Git repositories with a per-volume, shared object
cache (specified by `gvfs.sharedCache`) to migrate packfiles and loose
objects from the repository object directory to the shared cache.

Older versions of `microsoft/git` incorrectly placed packfiles in the
repository object directory instead of the shared cache; this task will
help clean up existing clones impacted by that issue.

Fixes #716
dscho pushed a commit that referenced this pull request Nov 6, 2025
@mjcheetham @dscho
maintenance: add new cache-local-objects maintenance task (#720)
9cc8cd2 
Introduce a new maintenance task, `cache-local-objects`, that operates
on Scalar or VFS for Git repositories with a per-volume, shared object
cache (specified by `gvfs.sharedCache`) to migrate packfiles and loose
objects from the repository object directory to the shared cache.

Older versions of `microsoft/git` incorrectly placed packfiles in the
repository object directory instead of the shared cache; this task will
help clean up existing clones impacted by that issue.

Fixes #716
dscho pushed a commit that referenced this pull request Nov 6, 2025
@mjcheetham @dscho
maintenance: add new cache-local-objects maintenance task (#720)
a8c003a 
Introduce a new maintenance task, `cache-local-objects`, that operates
on Scalar or VFS for Git repositories with a per-volume, shared object
cache (specified by `gvfs.sharedCache`) to migrate packfiles and loose
objects from the repository object directory to the shared cache.

Older versions of `microsoft/git` incorrectly placed packfiles in the
repository object directory instead of the shared cache; this task will
help clean up existing clones impacted by that issue.

Fixes #716
dscho pushed a commit that referenced this pull request Nov 13, 2025
@mjcheetham @dscho
maintenance: add new cache-local-objects maintenance task (#720)
c1d99ae 
Introduce a new maintenance task, `cache-local-objects`, that operates
on Scalar or VFS for Git repositories with a per-volume, shared object
cache (specified by `gvfs.sharedCache`) to migrate packfiles and loose
objects from the repository object directory to the shared cache.

Older versions of `microsoft/git` incorrectly placed packfiles in the
repository object directory instead of the shared cache; this task will
help clean up existing clones impacted by that issue.

Fixes #716
dscho pushed a commit that referenced this pull request Nov 17, 2025
@mjcheetham @dscho
maintenance: add new cache-local-objects maintenance task (#720)
3c0d3bd 
Introduce a new maintenance task, `cache-local-objects`, that operates
on Scalar or VFS for Git repositories with a per-volume, shared object
cache (specified by `gvfs.sharedCache`) to migrate packfiles and loose
objects from the repository object directory to the shared cache.

Older versions of `microsoft/git` incorrectly placed packfiles in the
repository object directory instead of the shared cache; this task will
help clean up existing clones impacted by that issue.

Fixes #716
dscho pushed a commit that referenced this pull request Jan 27, 2026
@mjcheetham @dscho
maintenance: add new cache-local-objects maintenance task (#720)
0bc0343 
Introduce a new maintenance task, `cache-local-objects`, that operates
on Scalar or VFS for Git repositories with a per-volume, shared object
cache (specified by `gvfs.sharedCache`) to migrate packfiles and loose
objects from the repository object directory to the shared cache.

Older versions of `microsoft/git` incorrectly placed packfiles in the
repository object directory instead of the shared cache; this task will
help clean up existing clones impacted by that issue.

Fixes #716
dscho pushed a commit that referenced this pull request Jan 28, 2026
@mjcheetham @dscho
maintenance: add new cache-local-objects maintenance task (#720)
405f2a1 
Introduce a new maintenance task, `cache-local-objects`, that operates
on Scalar or VFS for Git repositories with a per-volume, shared object
cache (specified by `gvfs.sharedCache`) to migrate packfiles and loose
objects from the repository object directory to the shared cache.

Older versions of `microsoft/git` incorrectly placed packfiles in the
repository object directory instead of the shared cache; this task will
help clean up existing clones impacted by that issue.

Fixes #716
dscho pushed a commit that referenced this pull request Jan 31, 2026
@mjcheetham @dscho
maintenance: add new cache-local-objects maintenance task (#720)
4e01d1d 
Introduce a new maintenance task, `cache-local-objects`, that operates
on Scalar or VFS for Git repositories with a per-volume, shared object
cache (specified by `gvfs.sharedCache`) to migrate packfiles and loose
objects from the repository object directory to the shared cache.

Older versions of `microsoft/git` incorrectly placed packfiles in the
repository object directory instead of the shared cache; this task will
help clean up existing clones impacted by that issue.

Fixes #716
dscho pushed a commit that referenced this pull request Feb 3, 2026
@mjcheetham @dscho
maintenance: add new cache-local-objects maintenance task (#720)
390bedd 
Introduce a new maintenance task, `cache-local-objects`, that operates
on Scalar or VFS for Git repositories with a per-volume, shared object
cache (specified by `gvfs.sharedCache`) to migrate packfiles and loose
objects from the repository object directory to the shared cache.

Older versions of `microsoft/git` incorrectly placed packfiles in the
repository object directory instead of the shared cache; this task will
help clean up existing clones impacted by that issue.

Fixes #716
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@derrickstolee derrickstolee derrickstolee approved these changes

@dscho dscho dscho left review comments

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Scalar: Move local .git objects to scalar cache for efficiency and behavior breaks

3 participants

@mjcheetham @dscho @derrickstolee