perf(core): add index on name

[pulsejet]

We already have an index on parent+name but this can't be used for filtering just by name in a user's root. Such a query is used by both Photos and Memories for filtering out nomedia files.

BEFORE: 4 rows in set (0.251 sec)
AFTER: 4 rows in set (0.001 sec)

(this is on a smallish filecache with 400k; it gets pretty slow when the filecache can't fit into memory cause it's doing a scan)

$comp = new SearchBinaryOperator(ISearchBinaryOperator::OPERATOR_OR, [
    new SearchComparison(ISearchComparison::COMPARE_EQUAL, 'name', '.nomedia'),
    new SearchComparison(ISearchComparison::COMPARE_EQUAL, 'name', '.nomemories'),
]);
$search = $root->search(new SearchQuery($comp, 0, 0, [], Util::getUser()));

Query looks like this:

SELECT 
  `file`.`fileid`, 
  `storage`, 
  `path`, 
  `path_hash`, 
  `file`.`parent`, 
  `file`.`name`, 
  `mimetype`, 
  `mimepart`, 
  `size`, 
  `mtime`, 
  `storage_mtime`, 
  `encrypted`, 
  `etag`, 
  `file`.`permissions`, 
  `checksum`, 
  `unencrypted_size`, 
  `meta`.`json` AS `meta_json`, 
  `meta`.`sync_token` AS `meta_sync_token` 
FROM 
  `oc_filecache` `file` 
  LEFT JOIN `oc_files_metadata` `meta` ON `file`.`fileid` = `meta`.`file_id` 
WHERE 
  (
    `file`.`name` IN ('.nomedia', '.nomemories')
  ) 
  AND (
    (
      (`storage` = 1) 
      AND (
        (`path` = 'files') 
        OR (`path` LIKE 'files/%')
      )
    ) 
    OR (`storage` = 4) 
    OR (
      (`storage` = 3) 
      AND (
        (`path` = 'files/STA') 
        OR (`path` LIKE 'files/STA/%')
      )
    )
  );

Query plan BEFORE:

+------+-------------+-------+--------+-----------------------------------------------------------------------------------------------------+-------------------+---------+-----------------------+--------+-------------+
| id   | select_type | table | type   | possible_keys                                                                                       | key               | key_len | ref                   | rows   | Extra       |
+------+-------------+-------+--------+-----------------------------------------------------------------------------------------------------+-------------------+---------+-----------------------+--------+-------------+
|    1 | SIMPLE      | file  | ALL    | fs_storage_path_hash,fs_storage_mimetype,fs_storage_mimepart,fs_storage_size,fs_storage_path_prefix | NULL              | NULL    | NULL                  | 418989 | Using where |
|    1 | SIMPLE      | meta  | eq_ref | files_meta_fileid                                                                                   | files_meta_fileid | 8       | nextcloud.file.fileid | 1      |             |
+------+-------------+-------+--------+-----------------------------------------------------------------------------------------------------+-------------------+---------+-----------------------+--------+-------------+

AFTER:

+------+-------------+-------+--------+------------------------------------------------------------------------------------------------------------------+-------------------+---------+-----------------------+------+------------------------------------+
| id   | select_type | table | type   | possible_keys                                                                                                    | key               | key_len | ref                   | rows | Extra                              |
+------+-------------+-------+--------+------------------------------------------------------------------------------------------------------------------+-------------------+---------+-----------------------+------+------------------------------------+
|    1 | SIMPLE      | file  | range  | fs_storage_path_hash,fs_storage_mimetype,fs_storage_mimepart,fs_storage_size,fs_storage_path_prefix,fs_name_hash | fs_name_hash      | 1003    | NULL                  | 4    | Using index condition; Using where |
|    1 | SIMPLE      | meta  | eq_ref | files_meta_fileid                                                                                                | files_meta_fileid | 8       | nextcloud.file.fileid | 1    |                                    |
+------+-------------+-------+--------+------------------------------------------------------------------------------------------------------------------+-------------------+---------+-----------------------+------+------------------------------------+

Note: it might possible to create a joint index that's even faster but this index might help the optimizer with other unrelated queries so I think this is better.

[emoral435]

LGTM - and this is a really useful and impactful contribution too! Thank you so much :)

[Altahrim]

If the index is only required for this kind of query (with .nosomething), maybe it's enough to add only the first 12 characters in the index.

[pulsejet]

I wouldn't say it's the only potential use case for the index but doing a scan beyond 12 characters is not a big deal anyway, so I don't mind.

[github-actions]

Hello there,
Thank you so much for taking the time and effort to create a pull request to our Nextcloud project.

We hope that the review process is going smooth and is helpful for you. We want to ensure your pull request is reviewed to your satisfaction. If you have a moment, our community management team would very much appreciate your feedback on your experience with this PR review process.

Your feedback is valuable to us as we continuously strive to improve our community developer experience. Please take a moment to complete our short survey by clicking on the following link: https://cloud.nextcloud.com/apps/forms/s/i9Ago4EQRZ7TWxjfmeEpPkf6

Thank you for contributing to Nextcloud and we hope to hear from you soon!

[kesselb]

Hi,

addMissingIndex is enough to show the warning and adding the index via occ db:add-missing-indices.

For fresh installations we should consider altering an existing older migration (e.g. core/Migrations/Version13000Date20170718121200.php) to add the index automatically for fresh installations.