Add occ command dav:cleanup-chunks to remove all upload folders which…#28587
Add occ command dav:cleanup-chunks to remove all upload folders which…#28587DeepDiver1975 merged 1 commit intomasterfrom
Conversation
| } | ||
|
|
||
| protected function execute(InputInterface $input, OutputInterface $output) { | ||
| $cutOffTime = new \DateTime('2 days ago'); |
There was a problem hiding this comment.
Ok that's clever. The original problem was that iterating over ALL users would take a long time.
But this assumes that someone will run this command at least every day. If they miss one day, the command becomes useless.
I suggest to add CLI arguments to:
- make it possible to run it for all users regardless of time
- specify number of days
There was a problem hiding this comment.
It removes chunks older than 2 days, so you can run it once a week, once a month., whenever you want to clean up the space. It will clean up all old crud.
But yes, the 2 days could be a parameter with default of 2 - that way:
- if you know you have some chunks to cleanup then you can specify 1 day or even 0 days (if you know current activity is stopped)
- if you have some users with huge uploads that run for ages, you can specify a larger number of days so as not to annoy them
|
Should this be hooked by default into oC's cron? So that it "just works" for normal installations, not only for hardcore admins running those commands? |
|
@guruz we likely can't plug it into the background job / cron because for some setups it is likely that the job itself will take several days to finish when iterating over millions of users... This is why we had this other discussion about "marking users for expiration" here: #14646 (comment). The other alternative, iterating over the folders will not work either as it needs to be done on API / storage level which then needs you to iterate over all storages. (no magic |
91e5c47 to
09c1d1e
Compare
|
|
|
"junks" 😱 |
take it as an review assessment test: achievement unlocked: conscientious-pr-reviewer |
09c1d1e to
c017cd6
Compare
|
@DeepDiver1975 please add unit test. Moving to "planned" for next release for now |
due to the usage of filesystem setup code unit testing this is not easy/possible. we any how need a proper concept on testing our commands. |
c017cd6 to
0619e4b
Compare
|
now with command tester 🎉 |
There was a problem hiding this comment.
s/older then/older than/
and in the expected test result
0619e4b to
d26b17f
Compare
| $this->commandTester->execute([]); | ||
| $output = $this->commandTester->getDisplay(); | ||
| $this->assertContains("Cleaning chunks older than 2 days", $output); | ||
| $this->assertContains('Cleaning chunks for dav-clean-chunks-user', $output); |
There was a problem hiding this comment.
Can you somehow verify that the chunks were actually removed from disk ?
There was a problem hiding this comment.
let me think about this .....
… are older then 2 days - fixes #26981
d26b17f to
0a5a717
Compare
| $this->loginAsUser($this->user->getUID()); | ||
|
|
||
| // generate old chunks | ||
| $view = new View("/$userId/uploads"); |
There was a problem hiding this comment.
@PVince81 now with real deletion of the upload folder
@DeepDiver1975 please raise/link to documentation ticket. We need to make sure admins know they need to add this to their cron. |
done |
|
@DeepDiver1975, how about https://github.com/mikey179/vfsStream for assisting with testing? |
|
interesting idea .... I try not to forget .... 😉 |
Relates to owncloud/core#28587 and #3449.
Relates to owncloud/core#28587 and #3449.
|
This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs. |
… are older then 2 days
Description
Related Issue
fixes #26981
How Has This Been Tested?
Screenshots (if appropriate):
Types of changes
Checklist: