Skip to content

Cache existence of tablet directories in CompactionCoordinator#3583

Merged
dlmarion merged 5 commits into
apache:elasticityfrom
dlmarion:3475-cache-check-tablet-dir
Jul 19, 2023
Merged

Cache existence of tablet directories in CompactionCoordinator#3583
dlmarion merged 5 commits into
apache:elasticityfrom
dlmarion:3475-cache-check-tablet-dir

Conversation

@dlmarion
Copy link
Copy Markdown
Contributor

Cache existence of tablet directories to avoid checking to see if they exist, which can be expensive.

Fixes #3475

Cache existence of tablet directories to avoid checking to
see if they exist, which can be expensive.

Fixes apache#3475
@dlmarion dlmarion self-assigned this Jul 10, 2023
@dlmarion dlmarion linked an issue Jul 10, 2023 that may be closed by this pull request
Copy link
Copy Markdown
Contributor

@keith-turner keith-turner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Were you able to test this in any way? I suspect some of the volume related ITs may exercise the dir creation aspect of this, but not sure. Depends on if we have a multi volume IT that compacts data.

@dlmarion
Copy link
Copy Markdown
Contributor Author

Were you able to test this in any way? I suspect some of the volume related ITs may exercise the dir creation aspect of this, but not sure. Depends on if we have a multi volume IT that compacts data.

No, I basically copied over the code from Tablet.java. I don't see any ITs that test this condition. It also looks like CompactionCoordinatorTest got nuked. Looking at the comment in that file, we might need a ticket to re-evaluate that

@dlmarion dlmarion merged commit 3e7d569 into apache:elasticity Jul 19, 2023
@dlmarion dlmarion deleted the 3475-cache-check-tablet-dir branch July 19, 2023 11:22
@ctubbsii ctubbsii added this to the 4.0.0 milestone Jul 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: ✅ Done

Development

Successfully merging this pull request may close these issues.

Create directory if it does not exists before starting compaction

4 participants