Cargo caused complete breakage of 4 pipelines

[Yuri6037]

A week ago Cargo dropped allowed_fail field in the test JSON output: 6562069.

This breaking change was never announced and caused 4 pipelines on GitLab to fail due the now missing field.

In the future I think rust should announce such breaking changes before committing the changes.

[joshtriplett]

allow_fail was a nightly-only feature. However, it looks like stable Rust emitted allow_fail: false on tests, rather than omitting the allow_fail attribute. In general, we shouldn't let nightly-only features affect stable output when not used, for exactly this reason.

@Yuri6037 To confirm, your pipelines were depending on the presence of allow_fail: false in the output?

[Yuri6037]

allow_fail was a nightly-only feature. However, it looks like stable Rust emitted allow_fail: false on tests, rather than omitting the allow_fail attribute. In general, we shouldn't let nightly-only features affect stable output when not used, for exactly this reason.

@Yuri6037 To confirm, your pipelines were depending on the presence of allow_fail: false in the output?

Thanks, yes indeed the pipelines depends on a tool which did require the presence of allow_fail for serde decoding. When this field was dropped the tool started to emit errors in all pipelines using it.

EDIT: I'm right now preparing an update of the parsing tool to fix this. It would just be nice if there could be announcements when these changes occur to avoid this because fixing this is not as simple as just updating on git repo I have to actually re-run pipelines in all failing projects.

[joshtriplett]

@Yuri6037 Ideally we shouldn't do breaking changes at all, with or without announcements. (We coordinate and make breaking changes on rare occasions to deal with safety issues, but that was not the case here.) #93416 was not known to be a breaking change when it was made; it was assumed to be the removal of a nightly feature, which wouldn't be subject to any stability guarantee. I would guess that nobody caught that allow_fail: false was being emitted as part of stable output.

[ehuss]

I would guess that nobody caught that allow_fail: false was being emitted as part of stable output.

--format json is also unstable, so I don't think this is a stable change.

[nagisa]

I tagged this requires-nightly given that everything so far points at this requiring a nightly. We can remove the tag if the issue author can confirm that they were seeing this with a stable compiler and without RUSTC_BOOTSTRAP=1-like workarounds and detail how.

[Yuri6037]

The actual program being tested builds in stable rust. However the test command uses unstable features to access the JSON output formatting.

[joshtriplett]

@ehuss Thanks for the clarification!

[Enselic]

Triage: There is no further action to take here then, right?

Closing, but feel free to reopen if I am mistaken.