Documenting and fixing resolve's notion of public/private

[alexcrichton]

Making an issue off of my email to the mailing list.

Currently today resolve is generally considered "broken", and I think that this would benefit from actually writing the rules down, writing lots of test cases and documentation, and then fixing resolve. To the best of my knowledge, these are the rules which resolve should follow for determining whether an item is public or private and whether a certain path should resolve or not:

1. All items are private by default, with the exception of struct fields.
   To flag an item as public, you use pub. Also pub is not allowed on impl
   or extern (visibility is determined by each item individually).
2. For a path which is a cross-crate reference, it is only valid of every
   item in the path is marked as pub.
3. A module may access any item from its immediate parent.
4. A module may access any item from its immediate children.
5. Any local-crate path which goes outside the bounds of 3/4 will only resolve
   if each component of the path is pub. For example I could reach into a
   child's private mod's pub items, but not the child's private mod's non-pub
   items.

I'm nominating for a backwards-compatibility milestone as there's probably lots of existing code which is doing the wrong thing by accident.

[bblum]

I stated my preference in #3505 but will restate here. I would like there to be a rule: "pub items in a non-pub mod are visible to the rest of the crate, but not visible to users outside the crate". This is necessary to finally seal the leakiness of the select trait abstraction, cf #5160.

AIUI condition 2 listed above would imply this rule.

[Aatch]

Ok, to give a concrete (if contrived example), lets say I have this:

pub mod inner_api_a {
   use super::inner_common_impl;
   pub fn api1() {
       inner_common_impl::common1()
   }
   pub fn api2() {
       inner_common_impl::common2()
   }
}

pub mod inner_api_a {
   use super::inner_common_impl;
   pub fn api1() {
       inner_common_impl::common3()
   }
   pub fn api2() {
       inner_common_impl::common4()
   }
}

mod inner_common_impl {
    fn common1() { }
    fn common2() { }
    fn common3() { }
    fn common4() { }
}

In this example, inner_common_impl is just some common, internal routines shared by multiple, otherwise-unrelated modules inside the crate. These internal routines have complex preconditions and are not suitable for public consumption.

The above will not compile, as inner_common_impl is not public, so the use statements fail, also telling me that inner_common_impl is private.

If I mark it as public (so now it is visible cross-crate) it still won't compile because none of the functions are marked public. The only solution there is marking the function as public, thus making it visible to the outside world.

There are some convolutions you can do regarding using pub use and similar, but none of them play well with the implicit mapping to files/folders, as you'd end up adding extra layers to your directory structure or putting #[path] attributes everywhere. Additionally, if you aren't in a position where you can justifiably re-structure the crate, then you are even more stuck (eg, libstd).

The thing is, internal to the crate, the visibility rules are fine, and that's what matters while compiling that crate. When it comes to users of the library, it only matters that they can't link against the symbol you want. This suggests an alternative mechanism for controlling "link-time" visibility might be in order.

By keeping the current visibility rules regarding pub/priv as existing in isolation, and adding an attribute-based system for controlling visibility for external users, it should keep the system simple and understandable.

For comparison, in C, everything in a library is exported by default, unless the function/variable is marked static, in which case it doesn't even escape the object it's in. This is similar to rust's pub/priv at the moment. However, both GCC and clang implement extensions that allow you to control the visibility at a library level, via attributes.

I'm proposing a #[visibility(<value>)] attribute, that is applied recursively. This would control the "link-time" visibility of items. For example, adding #[visibility(internal)] to a pub mod foo; would make that module public, but not actually exported from the crate. This would essentially make everything after resolution/privacy checking act as if it's private.

Another potential advantage is one that C++ benefits from the same system. Not exporting mountains of monomorphised symbols.

[alexcrichton]

Sub-bug of #6143

[catamorphism]

Meta-bug is milestoned. Denominating this one.

[alexcrichton]

Nominating for the "priority" label and also to de-milestone the metabug while milestoning this one instead.

[alexcrichton]

I just closed a few issues that I believe will be wontfix or fixed as determined by the outcome of this issue. After reviewing the various use-cases, I'm not entirely sure that my previous set of rules is actually the best thing to follow. Especially with @bblum's and @Aatch's comments, I would actually be in favor of @bblum's suggestion, written down as:

1. All items are private by default, with the exception of struct fields. To flag an item as public, you use pub. Also pub is not allowed on impl or extern (visibility is determined by each item individually).
2. For a path which is a cross-crate reference, it is only valid if every item in the path is marked as pub (including modules)
3. Inside a crate, you can use any member from any location if the destination is flagged as pub (regardless of intermediate visibility).
4. Inside a module, you can use anything within that module (regardless of being public or private).

I am coming to prefer these rules because not only are they simple to implement, but they are simpler to understand and also work with. If this set of rules is agreed upon, I will update the issue text at the top to avoid later confusion over what should get implemented.