Skip to content

no-autoupdate-check_test.sh fails in CI but passes locally #495

Description

@trilamsr

Symptom

In CI (verify-static job, /usr/bin/bash -e), scripts/no-autoupdate-check_test.sh fails 7 fixture assertions:

ok: clean-tree
FAIL: gate exited 0 on fixture [hyphenated-go-update] but expected exit 1
FAIL: gate exited 0 on fixture [lowercase-selfupdate] but expected exit 1
FAIL: gate exited 0 on fixture [camelcase-SelfUpdate] but expected exit 1
FAIL: gate exited 0 on fixture [camelcase-GoUpdate] but expected exit 1
FAIL: gate exited 0 on fixture [UpdateCheck] but expected exit 1
FAIL: gate exited 0 on fixture [pkg-scope-covered] but expected exit 1
ok: test-file-excluded
ok: innocent-identifiers
FAIL: hit-line-format-stable: no line matched ^[^:]+:[0-9]+:
FAIL: 7 test(s) failed

Locally on macOS (bash 3.2) the same script reports all assertions passed.

Likely root cause

scripts/no-autoupdate-check.sh hard-codes scan_paths=(components internal pkg). The test harness cd $fixture && bash $gate relies on those relative paths existing inside each fixture tmpdir. Under Linux CI set -euo pipefail, grep -rniE ... "${scan_paths[@]}" against nonexistent paths returns nonzero, the if hits=$(...) block is skipped, gate exits 0, and assertions that expect exit 1 fail.

On macOS GNU grep is replaced by BSD grep with different missing-path semantics, masking the issue.

Workaround in place

PR #491 (commit 86c820c) decoupled the CI backstop from the test harness by invoking scripts/no-autoupdate-check.sh directly. The fixture self-test still runs in make verify and make ci-full locally; it just isn't gating CI anymore.

Fix direction

Either (a) make the gate tolerate missing scan_paths and surface "scanned 0 paths" with the same exit semantics on all platforms, or (b) have the test harness construct each fixture with the three scan paths present (even if empty) so the gate has something to grep against.

Discovered while landing #491 (RFC-0008 boundary backstop).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions