Updater: mirrors configuration tweaks

[jku]

For the Warehouse/pip use case we are not so much interested multiple actual mirrors but separating the target and metadata hosting to two hosts.

Current Updater mirror configuration is close but not quite up to the task:

mirrors = {
    'mirror1': {
        'url_prefix': 'https://hostname/basedir/',
        'metadata_path': 'metadata',
        'targets_path': 'targets',
        'confined_target_dirs': ['']
    },
}

I would like to have a way of signaling that a particular mirror should not be used for metadata or should not be used for targets ('confined_target_dirs': [] does offer that, but only for targets). There's multiple ways to solve this: I'm open to suggestions.

The mirrors role (which is unrelated to this but deals with the same problem) solves the issue by having PATH_PATTERNS for both metadata and targets, but for this specific use case it would be enough if metadata_path and targets_path could be set to None. Then I could do this in pip:

mirrors = {
    'metadata': {
        'url_prefix': 'https://pypi.org/',
        'metadata_path': 'tuf',
        'targets_path': None,
        'confined_target_dirs': []
    },
    'targets': {
        'url_prefix': 'https://files.pythonhosted.org/',
        'metadata_path': None,
        'targets_path': 'packages',
        'confined_target_dirs': ['']
    },
}

This would probably be a trivial change in mirrors.py and should not break any existing users

[jku]

maybe look at #1077 at the same time?

[joshuagl]

Agreed, changing tuf.mirrors.get_list_of_mirrors to look for a dictionary field like the proposed metadata_path would be a small change and could be implemented in such a way as to not break existing users (i.e. using get() or catching a KeyError).

[jku]

Actually the schema system does not make None an option very easily I guess... but the whole key could be made optional.