[Schema Consistency] Schema consistency report - 2026-05-18

[github-actions[bot]]

Summary

• Inconsistencies found: 4 (3 fresh, 1 known-context confirmation)
• Categories analyzed: Schema · Parser · Documentation · Workflows
• Strategy mode: New approach (day_mod=8 — 30% slot)
• New strategies added to cache: 3 (schema-defs-completeness, ref-with-siblings-draft07, yaml-tag-vs-parser-key)

This run explored fresh detection angles instead of replaying proven ones. Two of the three new strategies surfaced real, fixable issues in the schema itself, and a third surfaced a latent serialization bug in the compiler.

Critical Issues

1. $ref siblings silently ignored under JSON Schema draft-07 (schema bug, user-visible)

The main schema declares "$schema": "(jsonschema.org/redacted) Under draft-07, when a property has $ref, **all sibling keywords are ignored** by spec-compliant validators (incl. santhosh-tekuri/jsonschema/v6`, which this repo depends on).

Two top-level properties violate this:

• properties.engine (pkg/parser/schemas/main_workflow_schema.json) declares $ref: "#/$defs/engine_config" alongside default: "copilot", description, and examples. The default, description, and examples are dropped at validation time.
• properties.github-app declares $ref: "#/$defs/github_app" alongside description. The description is dropped.

Detection one-liner:

jq -r '.properties | to_entries[] | select(.value | has("$ref") and ((. | keys | length) > 1)) | "\(.key): keys=\(.value | keys)"' pkg/parser/schemas/main_workflow_schema.json

Impact: IDE tooltips, documentation generators, and any validator-driven default-injection will not see engine defaulting to "copilot" or read the property descriptions for engine/github-app. The user-facing behavior in this repo's hand-written compiler happens to be correct because the default lives in pkg/constants/constants.go (DefaultEngine), not because the schema enforces it.

Suggested fix:

"engine": {
  "allOf": [{"$ref": "#/$defs/engine_config"}],
  "default": "copilot",
  "description": "...",
  "examples": [...]
}

Or migrate $schema to draft-2019-09/draft/2020-12, which both allow $ref siblings.

2. Latent YAML-tag mismatch: plural where the rest of the codebase is singular

pkg/workflow/compiler_types.go:639:

CreateCodeScanningAlerts *CreateCodeScanningAlertsConfig `yaml:"create-code-scanning-alerts,omitempty"`

Everywhere else the canonical key is singular:

• Schema property: safe-outputs.properties.create-code-scanning-alert (singular)
• Parser: pkg/workflow/safe_outputs_config.go ➜ parseCodeScanningAlertsConfig(outputMap) looks up outputMap["create-code-scanning-alert"]
• Tests: security_reports_test.go, safe_outputs_cross_repo_config_test.go, safe_outputs_permissions_test.go all use "create-code-scanning-alert"
• Handler key: safe_output_handlers.go:208 ➜ Key: "create-code-scanning-alert"

The tag is dormant today because SafeOutputsConfig is populated by hand-parsing outputMap, not by yaml.Unmarshal. But it lights up the moment anything yaml.Marshals the struct (config-dump tooling, debug helpers, future test harness) — the emitted YAML would use the plural form, which then fails to round-trip through the parser.

Suggested fix (one-line):

CreateCodeScanningAlerts *CreateCodeScanningAlertsConfig `yaml:"create-code-scanning-alert,omitempty"`

Schema Improvements Needed

3. Orphan $def templatable_integer (defined but never referenced)

pkg/parser/schemas/main_workflow_schema.json:10099 defines:

"templatable_integer": {
  "description": "A non-negative integer value that may also be specified as a GitHub Actions expression string that resolves to an integer at runtime ...",
  "oneOf": [
    {"type": "integer", "minimum": 0},
    {"type": "string", "pattern": "^\\$\\{\\{.*\\}\\}$"}
  ]
}

No $ref in the schema points at it. Meanwhile, three top-level properties duplicate substantially the same shape inline:

• max-runs (integer-or-string-or-${{expr}}, default 500)
• max-effective-tokens (integer-or-string-or-${{expr}}, default 25000000)
• timeout-minutes (integer-or-string-or-${{expr}})

Each inlines its own oneOf with the ^\$\{\{.*\}\}$ pattern.

Suggested fix: Either delete the orphan $def, or — preferable — migrate the three properties to allOf: [{ $ref: "#/$defs/templatable_integer" }] plus per-field constraints (minimum, default, description). Removes ~30 lines of duplication and makes the schema self-consistent.

Detection one-liner:

defs=$(jq -r '.["$defs"] | keys[]' pkg/parser/schemas/main_workflow_schema.json)
refs=$(jq -r '[.. | objects | select(has("$ref")) | .["$ref"]] | unique | .[]' pkg/parser/schemas/main_workflow_schema.json | sed 's|^#/\$defs/||')
comm -23 <(echo "$defs" | sort) <(echo "$refs" | sort)

Documentation Gaps

4. field_gaps.in_used_not_schema is dominated by sub-agent-block content, not real frontmatter (confirmed false-positive)

The pre-computed diff lists 34 keys (model, title, repo, serena-find_symbol, etc.) as "used in workflows but absent from schema." Spot-checks of pr-sous-chef.md, daily-subagent-optimizer.md, and spec-enforcer.md confirm these keys never appear in the top-level frontmatter (between the first --- pair) — they all sit inside inline sub-agent blocks embedded in the markdown body. Restricting the schema-diff generator to the leading YAML frontmatter would eliminate the noise.

This matches strategy used-not-schema-false-positive-detection from the cache; recording it as a continuing observation, not a new bug.

View detailed diff data

• Schema fields: 58 top-level
• $defs: 15 entries
• $refs to $defs: 14 unique targets — templatable_integer is the lone orphan
• Schema declares additionalProperties: false at root + required: ["on"] ✅
• Object schemas in the tree: 287 total, 211 with additionalProperties: false (76 unrestricted, but most are intentional additionalProperties: <schema> maps such as models, mcp-servers)
• Truly unrestricted object schemas with named properties (extras silently accepted): 3 — all in user-defined-input territory (on.workflow_call.inputs, on.workflow_call.secrets, safe-outputs.dispatch_repository.inputs). Likely intentional but worth noting.

Recommendations

1. Schema — wrap the $ref in properties.engine and properties.github-app with allOf so the sibling default/description/examples survive draft-07 validation. Or bump $schema to a version that permits siblings.
2. Schema — collapse the three inline integer|string|${{expr}} shapes (max-runs, max-effective-tokens, timeout-minutes) onto the existing templatable_integer $def. Removes duplication and makes the orphan productive.
3. Compiler — fix the yaml:"create-code-scanning-alerts" ➜ yaml:"create-code-scanning-alert" tag in pkg/workflow/compiler_types.go:639 so the struct round-trips through yaml.Marshal.
4. Tooling — when regenerating /tmp/gh-aw/agent/schema-diff.json, limit the frontmatter parser to the leading --- block so sub-agent content stops polluting in_used_not_schema.

Strategy Performance

• Mode: new-approach slot (day_mod=8)
• New strategies cached: 3 — schema-defs-completeness, ref-with-siblings-draft07, yaml-tag-vs-parser-key
• Findings produced: 3 actionable (1 schema-correctness, 1 schema-DRY, 1 latent code bug)
• Effectiveness: HIGH — all three strategies are one-liner regression sentinels suitable for repeat use
• Should reuse: YES — promote schema-defs-completeness and ref-with-siblings-draft07 into the proven pool (success_count: 1, ready to be picked on a 0–6 day)

Next Steps

• Schema PR: wrap engine/github-app $ref siblings with allOf (or bump $schema version)
• Schema PR: migrate max-runs, max-effective-tokens, timeout-minutes to templatable_integer $def
• Compiler PR: rename yaml tag at pkg/workflow/compiler_types.go:639 to singular
• Tooling: tighten the schema-diff frontmatter parser to a single leading --- block

References:

• §26017693306

Generated by ✅ Schema Consistency Checker · ● 10.4M · ◷

• expires on May 19, 2026, 6:47 AM UTC

[github-actions[bot]]

This discussion has been marked as outdated by Schema Consistency Checker.

A newer discussion is available at Discussion #33233.