Missing tokens in custom derive input generated by macro

[dtolnay]

The script below has a macro_rules macro that expands to:

#[derive(Print)]
pub struct S(<MyDispatch as $crate::Dispatch>::Call);

But inside of the derive(Print) custom derive the tokens received are:

pub struct S(<MyDispatch as ::repro_dispatch>::Call);

The ::Dispatch seems to be missing.

Originally reported in serde-rs/serde#1306.

---

rustc 1.28.0-nightly (4ecf12b 2018-06-02)
Repro script:

#!/bin/sh

cargo new --lib repro
cargo new --lib repro_dispatch
cargo new --lib repro_derive

echo >repro/src/lib.rs '
#[macro_use]
extern crate repro_dispatch;

#[macro_use]
extern crate repro_derive;

pub struct MyDispatch;

impl repro_dispatch::Dispatch for MyDispatch {
    type Call = ();
}

problem!();
'

echo >>repro/Cargo.toml '
repro_dispatch = { path = "../repro_dispatch" }
repro_derive = { path = "../repro_derive" }
'

echo >repro_dispatch/src/lib.rs '
pub trait Dispatch {
    type Call;
}

#[macro_export]
macro_rules! problem {
    () => {
        #[derive(Print)]
        pub struct S(<MyDispatch as $crate::Dispatch>::Call);
    }
}
'

echo >repro_derive/src/lib.rs '
extern crate proc_macro;
use proc_macro::TokenStream;

#[proc_macro_derive(Print)]
pub fn derive(input: TokenStream) -> TokenStream {
    println!("\n\n{}\n\n", input);
    "".parse().unwrap()
}
'

echo >>repro_derive/Cargo.toml '
[lib]
proc-macro = true
'

cargo build --manifest-path repro/Cargo.toml

[dtolnay]

The behavior is correct with rustc 1.15.0 but the current incorrect behavior exists since 1.16.0.

[nikomatsakis]

cc @dtolnay @rust-lang/release — perhaps somebody could bisect this to a specific nightly? (I guess it's too old to get a specific PR)

cargo-bisect-rustc may help here :)

[nikomatsakis]

Also, how does it come to happen that $crate is still in there and not replaced during macro expansion...? Maybe I'm unclear on the phasing of how $crate works.

[nikomatsakis]

Marking as P-high but don't have an assignee yet, will revisit next week when we've hopefully had a chance to bisect.

[Mark-Simulacrum]

2016-12-06 has the odd result of pub struct S(<MyDispatch as ::::Dispatch>::Call);.

2016-12-27 is the first nightly to have pub struct S(<MyDispatch as ::repro_dispatch>::Call); as the result; the previous nightly 2016-12-23 has the (correct, I presume) pub struct S(<MyDispatch as ::repro_dispatch::Dispatch>::Call); result.

[Mark-Simulacrum]

These are the bors commits between those two nightlies (a173778...77f7c7a).

77f7c7a Auto merge of #38274 - elahn:windows-readconsole-ctrl-z, r=alexcrichton
b7e5148 Auto merge of #38314 - japaric:do-not-delete-enable-llvm-backend, r=alexcrichton
f536d90 Auto merge of #38542 - YaLTeR:fastcall-fix, r=pnkfelix
65c043f Auto merge of #38154 - petrochenkov:altname, r=jseyfried
8493dbe Auto merge of #38536 - retep998:flauschige-kaninchen, r=petrochenkov
5752eae Auto merge of #38598 - brson:em, r=alexcrichton
44ad63e Auto merge of #38490 - jseyfried:def_id_vis, r=nrc
20b6c16 Auto merge of #38539 - jseyfried:fix_resolve_hang, r=eddyb
c74ac6c Auto merge of #38566 - jseyfried:fix_import_resolution_bug, r=eddyb
e60aa62 Auto merge of #38594 - steveklabnik:rollup, r=steveklabnik
00e61d4 Auto merge of #38443 - frewsxcv:file-docs, r=brson
d86cf13 Auto merge of #38062 - alexcrichton:fix-line-writer, r=brson
8d65c8d Auto merge of #38268 - withoutboats:parse_where_higher_rank_hack, r=eddyb
4d07320 Auto merge of #38523 - camlorn:disable_field_reordering, r=nikomatsakis
467a7f0 Auto merge of #38533 - jseyfried:legacy_custom_derive_deprecation, r=nrc
00b4019 Auto merge of #38529 - nrc:save-sig, r=nikomatsakis
ce4461f Auto merge of #38511 - Mark-Simulacrum:drop-glue, r=eddyb
99913c5 Auto merge of #38401 - redox-os:redox_cross, r=brson
82611a0 Auto merge of #38232 - jseyfried:refactor_global_paths, r=nrc
c8e7ec4 Auto merge of #38562 - brson:rm-llvm-lock, r=brson

I think #38232, #38566, #38539, #38490, #38154 are possible candidates based on branch names. I'll try to run git bisect in this commit range.

[Mark-Simulacrum]

I was.. largely unsuccessful at running git bisect. Maybe someone else will have more luck.

[ExpHP]

I figure it's easier to bisect the Bors commits which are sure to build correctly, rather than to run git bisect. Overnight I ran a build of 467a7f0, and got the erroneous output, cutting it down to these:

467a7f0 Auto merge of #38533 - jseyfried:legacy_custom_derive_deprecation, r=nrc
00b4019 Auto merge of #38529 - nrc:save-sig, r=nikomatsakis
ce4461f Auto merge of #38511 - Mark-Simulacrum:drop-glue, r=eddyb
99913c5 Auto merge of #38401 - redox-os:redox_cross, r=brson
82611a0 Auto merge of #38232 - jseyfried:refactor_global_paths, r=nrc
c8e7ec4 Auto merge of #38562 - brson:rm-llvm-lock, r=brson

My money's on jseyfried:refactor_global_paths but I won't be able to run another build until I get to work.

---

Slightly modified repro script for working with such old versions of rust

#!/bin/sh

# set the linked toolchain here
TOOLCHAIN=stage2

rm -rf repro
rm -rf repro_dispatch
rm -rf repro_derive
cargo new --lib repro
cargo new --lib repro_dispatch
cargo new --lib repro_derive

echo >repro/src/lib.rs '
#[macro_use]
extern crate repro_dispatch;

#[macro_use]
extern crate repro_derive;

pub struct MyDispatch;

impl repro_dispatch::Dispatch for MyDispatch {
    type Call = ();
}

problem!();
'

echo >>repro/Cargo.toml '
repro_dispatch = { path = "../repro_dispatch" }
repro_derive = { path = "../repro_derive" }
'

echo >repro_dispatch/src/lib.rs '
pub trait Dispatch {
    type Call;
}

#[macro_export]
macro_rules! problem {
    () => {
        #[derive(Print)]
        pub struct S(<MyDispatch as $crate::Dispatch>::Call);
    }
}
'

echo >repro_derive/src/lib.rs '
#![feature(proc_macro, proc_macro_lib)]
extern crate proc_macro;
use proc_macro::TokenStream;

#[proc_macro_derive(Print)]
pub fn derive(input: TokenStream) -> TokenStream {
    println!("\n\n{}\n\n", input);
    "".parse().unwrap()
}
'

echo >>repro_derive/Cargo.toml '
[lib]
proc-macro = true
'

CARGO_INCREMENTAL=0 cargo +$TOOLCHAIN build --manifest-path repro/Cargo.toml
CARGO_INCREMENTAL=0 cargo +$TOOLCHAIN rustc --manifest-path repro/Cargo.toml -- --version