Skip to content

Support case-insensitive filesystems when pruning files after update #196

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
schlessera merged 7 commits into from
Jan 21, 2022
Merged

Support case-insensitive filesystems when pruning files after update #196

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
2fad25e
When cleaning up files after an update, compare the filenames from th…
pbiron Dec 1, 2021
72797c6
Add test to check that there are no regressions introduced for case-s…
pbiron Dec 4, 2021
8e09791
Use spaces for indenting in the test, rather than tabs.
pbiron Dec 4, 2021
69422bc
Allow for warnings to be produced when verifying checksums after a do…
schlessera Dec 6, 2021
8981c97
Adapt code to deal with case-insensitivity through realpath and renames
schlessera Jan 20, 2022
b4656a3
Add debugging messages for renames
schlessera Jan 20, 2022
80b8c1c
Fix missing variable
schlessera Jan 20, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
78 changes: 77 additions & 1 deletion features/core-update.feature
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -202,7 +202,8 @@ Feature: Update WordPress core
Success: WordPress updated successfully.
"""

When I run `wp core verify-checksums`
# Allow for warnings to be produced.
When I try `wp core verify-checksums`
Then STDOUT should be:
"""
Success: WordPress installation verifies against checksums.
Expand Down Expand Up @@ -250,6 +251,81 @@ Feature: Update WordPress core
When I run `wp post create --post_title='Test post' --porcelain`
Then STDOUT should be a number

Scenario: Make sure files are cleaned up with mixed case
Given a WP install
And I try `wp theme install twentytwenty --activate`

When I run `wp core update --version=5.8 --force`
Then the wp-includes/Requests/Transport/cURL.php file should exist
Then the wp-includes/Requests/Exception/Transport/cURL.php file should exist
Then the wp-includes/Requests/Exception/HTTP/502.php file should exist
Then the wp-includes/Requests/IRI.php file should exist
Then the wp-includes/Requests/Transport/Curl.php file should not exist
Then the wp-includes/Requests/Exception/Transport/Curl.php file should not exist
Then the wp-includes/Requests/Exception/Http/Status502.php file should not exist
Then the wp-includes/Requests/Iri.php file should not exist

When I run `wp core update --version=5.9-beta1 --force`
Then the wp-includes/Requests/Transport/cURL.php file should not exist
Then the wp-includes/Requests/Exception/Transport/cURL.php file should not exist
Then the wp-includes/Requests/Exception/HTTP/502.php file should not exist
Then the wp-includes/Requests/IRI.php file should not exist
Then the wp-includes/Requests/Transport/Curl.php file should exist
Then the wp-includes/Requests/Exception/Transport/Curl.php file should exist
Then the wp-includes/Requests/Exception/Http/Status502.php file should exist
Then the wp-includes/Requests/Iri.php file should exist
Then STDOUT should contain:
"""
File removed: wp-includes/Requests/Transport/fsockopen.php
File removed: wp-includes/Requests/Transport/cURL.php
File removed: wp-includes/Requests/Hooker.php
File removed: wp-includes/Requests/IPv6.php
File removed: wp-includes/Requests/Exception/Transport/cURL.php
File removed: wp-includes/Requests/Exception/HTTP.php
File removed: wp-includes/Requests/Exception/HTTP/502.php
File removed: wp-includes/Requests/Exception/HTTP/Unknown.php
File removed: wp-includes/Requests/Exception/HTTP/412.php
File removed: wp-includes/Requests/Exception/HTTP/408.php
File removed: wp-includes/Requests/Exception/HTTP/431.php
File removed: wp-includes/Requests/Exception/HTTP/501.php
File removed: wp-includes/Requests/Exception/HTTP/500.php
File removed: wp-includes/Requests/Exception/HTTP/407.php
File removed: wp-includes/Requests/Exception/HTTP/416.php
File removed: wp-includes/Requests/Exception/HTTP/428.php
File removed: wp-includes/Requests/Exception/HTTP/406.php
File removed: wp-includes/Requests/Exception/HTTP/504.php
File removed: wp-includes/Requests/Exception/HTTP/411.php
File removed: wp-includes/Requests/Exception/HTTP/414.php
File removed: wp-includes/Requests/Exception/HTTP/511.php
File removed: wp-includes/Requests/Exception/HTTP/410.php
File removed: wp-includes/Requests/Exception/HTTP/403.php
File removed: wp-includes/Requests/Exception/HTTP/400.php
File removed: wp-includes/Requests/Exception/HTTP/505.php
File removed: wp-includes/Requests/Exception/HTTP/413.php
File removed: wp-includes/Requests/Exception/HTTP/404.php
File removed: wp-includes/Requests/Exception/HTTP/306.php
File removed: wp-includes/Requests/Exception/HTTP/304.php
File removed: wp-includes/Requests/Exception/HTTP/405.php
File removed: wp-includes/Requests/Exception/HTTP/429.php
File removed: wp-includes/Requests/Exception/HTTP/417.php
File removed: wp-includes/Requests/Exception/HTTP/409.php
File removed: wp-includes/Requests/Exception/HTTP/402.php
File removed: wp-includes/Requests/Exception/HTTP/418.php
File removed: wp-includes/Requests/Exception/HTTP/305.php
File removed: wp-includes/Requests/Exception/HTTP/415.php
File removed: wp-includes/Requests/Exception/HTTP/401.php
File removed: wp-includes/Requests/Exception/HTTP/503.php
File removed: wp-includes/Requests/IRI.php
File removed: wp-includes/Requests/IDNAEncoder.php
File removed: wp-includes/Requests/SSL.php
File removed: wp-includes/Requests/Proxy/HTTP.php
"""

When I run `wp option add str_opt 'bar'`
Then STDOUT should not be empty
When I run `wp post create --post_title='Test post' --porcelain`
Then STDOUT should be a number

@less-than-php-7.3
Scenario: Minor update on an unlocalized WordPress release
Given a WP install
Expand Down
64 changes: 63 additions & 1 deletion src/Core_Command.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1372,7 +1372,69 @@ private function cleanup_extra_files( $version_from, $version_to, $locale, $inse
return;
}

$files_to_remove = array_diff( array_keys( $old_checksums ), array_keys( $new_checksums ) );
// Compare the files from the old version and the new version in a case-insensitive manner,
// to prevent files being incorrectly deleted on systems with case-insensitive filesystems
// when core changes the case of filenames.
// The main logic for this was taken from the Joomla project and adapted for WP.
// See: https://github.com/joomla/joomla-cms/blob/bb5368c7ef9c20270e6e9fcc4b364cd0849082a5/administrator/components/com_admin/script.php#L8158

$old_filepaths = array_keys( $old_checksums );
$new_filepaths = array_keys( $new_checksums );

$new_filepaths = array_combine( array_map( 'strtolower', $new_filepaths ), $new_filepaths );
Copy link
Contributor

@wojsmol wojsmol Jan 21, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AFAIK if we pass new checksums trough strtolower then we lose case information witch are very important in this code.

Copy link
Member

@schlessera schlessera Jan 21, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line builds an array that uses the lowercased string as the key and the original string as the value, to help with mapping.

Copy link
Member

@schlessera schlessera Jan 21, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note the array_combine().

So we'll go from this:

[
   0 => 'Abc',
   1 => 'BCD',
]

to this:

[
   'abc' => 'Abc',
   'bcd' => 'BCD',
]


$old_filepaths_to_check = array_diff( $old_filepaths, $new_filepaths );

foreach ( $old_filepaths_to_check as $old_filepath_to_check ) {
$old_realpath = realpath( ABSPATH . $old_filepath_to_check );

// On Unix without incorrectly cased file.
if ( false === $old_realpath ) {
continue;
}

$lowercase_old_filepath_to_check = strtolower( $old_filepath_to_check );

if ( ! array_key_exists( $lowercase_old_filepath_to_check, $new_filepaths ) ) {
$files_to_remove[] = $old_filepath_to_check;
continue;
}

// We are now left with only the files that are similar from old to new except for their case.

$old_basename = basename( $old_realpath );
$new_filepath = $new_filepaths[ $lowercase_old_filepath_to_check ];
$expected_basename = basename( $new_filepath );
$new_realpath = realpath( ABSPATH . $new_filepath );
$new_basename = basename( $new_realpath );

// On Windows or Unix with only the incorrectly cased file.
if ( $new_basename !== $expected_basename ) {
WP_CLI::debug( "Renaming file '{$old_filepath_to_check}' => '{$new_filepath}'", 'core' );

rename( ABSPATH . $old_filepath_to_check, ABSPATH . $old_filepath_to_check . '.tmp' );
rename( ABSPATH . $old_filepath_to_check . '.tmp', ABSPATH . $new_filepath );

continue;
}

// There might still be an incorrectly cased file on other OS than Windows.
if ( basename( $old_filepath_to_check ) === $old_basename ) {
// Check if case-insensitive file system, eg on OSX.
if ( fileinode( $old_realpath ) === fileinode( $new_realpath ) ) {
// Check deeper because even realpath or glob might not return the actual case.
if ( ! in_array( $expected_basename, scandir( dirname( $new_realpath ) ), true ) ) {
WP_CLI::debug( "Renaming file '{$old_filepath_to_check}' => '{$new_filepath}'", 'core' );

rename( ABSPATH . $old_filepath_to_check, ABSPATH . $old_filepath_to_check . '.tmp' );
rename( ABSPATH . $old_filepath_to_check . '.tmp', ABSPATH . $new_filepath );
}
} else {
// On Unix with both files: Delete the incorrectly cased file.
$files_to_remove[] = $old_filepath_to_check;
}
}
}

if ( ! empty( $files_to_remove ) ) {
WP_CLI::log( 'Cleaning up files...' );
Expand Down