Skip to content

[Lean Squad] feat(fv): Task 1 β€” add 3 new FV targets and sync TARGETS.md phasesΒ #8017

Description

@Evangelink

Summary

πŸ”¬ Lean Squad β€” automated formal verification agent. Run #25331082787.

This PR executes Task 1 (Research & Target Identification) for this run. Task 3 (Lean spec writing) was the primary selected task but remains blocked β€” the Lean toolchain requires network access to install (elan/curl are firewalled in the sandbox). CI-based verification via leanprover/lean4-action is used instead.

Changes

formal-verification/TARGETS.md

  • Phase corrections based on merged PRs:
    • Target 3 (ParseOptionAndSeparators): updated PR reference to #7919
    • Target 5 (CommandLineParseResult.Equals): promoted from phase 1 β†’ phase 2 (spec merged in #7918)
  • New targets from merged PR #7849 added to table:
    • Target 8: PasteArguments.AppendArgument β€” Windows argv escaping, round-trip correctness
    • Target 9: TimeSpanParser.TryParse β€” suffix-dispatch pure parser
    • Target 10: TreeNodeFilter.TokenizeFilter β€” tokenizer with trailing-backslash exception
    • Target 11: Sha256Hasher.ToCharsBuffer β€” branchless hex encoding, all 256 inputs decidable
    • Target 12: ExtensionValidationHelper.ValidateUniqueExtension β€” duplicate-UID detection
  • 3 new targets identified this run:
    • Target 13: ValidationResult algebraic properties β€” tiny discriminated-union struct; Valid().IsValid = true, Invalid(msg).ErrorMessage = some msg; all properties decide-provable. The [MemberNotNullWhen] invariant is machine-checkable.
    • Target 14: CommandLineOptionsValidator.ValidateNoUnknownOptions β€” pure function; set-membership property: returns Invalid iff any parsed option name is not in the union of registered names.
    • Target 15: CommandLineOptionsValidator.ValidateOptionsAreNotDuplicated β€” pure function; uniqueness property: returns Invalid iff any option name is registered by more than one provider.

formal-verification/RESEARCH.md

Added detailed target profiles for Targets 13, 14, and 15:

  • Purpose, why FV-amenable, properties to verify, approximations, open questions.

New Targets Detail

Target 13 β€” ValidationResult

File: src/Platform/Microsoft.Testing.Platform/Extensions/ValidationResult.cs

public readonly struct ValidationResult {
    public bool IsValid { get; }
    [MemberNotNullWhen(false, nameof(ErrorMessage))]
    public string? ErrorMessage { get; }
    public static ValidationResult Valid() => new(true, null);
    public static ValidationResult Invalid(string errorMessage) => new(false, errorMessage);
}

Key theorems: Valid().IsValid = true, βˆ€ msg, Invalid(msg).IsValid = false, invariant IsValid = false ↔ ErrorMessage.isSome.

Target 14 β€” ValidateNoUnknownOptions

Pure membership check: parseResult.Options.Names βŠ† validNames β†’ Valid. Monotone in validNames.

Target 15 β€” ValidateOptionsAreNotDuplicated

Pure uniqueness check: βˆ€ name, |registerers(name)| ≀ 1 β†’ Valid. Symmetric: provider ordering does not affect outcome.

Why These Targets?

All three are:

  • Pure functions / value-type invariants with no side effects or I/O
  • Decidable β€” concrete instances provable by decide in Lean 4
  • Complementary β€” together with existing targets they cover the full command-line validation pipeline
  • Self-contained β€” require minimal abstraction of external types

Generated by πŸ“ Lean Squad, see workflow run.


Note

This was originally intended as a pull request, but the git push operation failed.

Workflow Run: View run details and download patch artifact

The patch file is available in the agent artifact in the workflow run linked above.

To create a pull request with the changes:

# Download the artifact from the workflow run
gh run download 25331082787 -n agent -D /tmp/agent-25331082787

# Create a new branch
git checkout -b lean-squad/task1-new-targets-2026-05-04-run3-fef6317d0138f71d

# Apply the patch (--3way handles cross-repo patches where files may already exist)
git am --3way /tmp/agent-25331082787/aw-lean-squad-task1-new-targets-2026-05-04-run3.patch

# Push the branch to origin
git push origin lean-squad/task1-new-targets-2026-05-04-run3-fef6317d0138f71d

# Create the pull request
gh pr create --title '[Lean Squad] feat(fv): Task 1 β€” add 3 new FV targets and sync TARGETS.md phases' --base main --head lean-squad/task1-new-targets-2026-05-04-run3-fef6317d0138f71d --repo microsoft/testfx
Show patch preview (177 of 177 lines)
From 1a5bc0128f41036b03e360804c5a42181f90a473 Mon Sep 17 00:00:00 2001
From: "github-actions[bot]" <github-actions[bot]@users.noreply.github.com>
Date: Mon, 4 May 2026 16:52:18 +0000
Subject: [PATCH] =?UTF-8?q?feat(fv):=20Task=201=20=E2=80=94=20add=203=20ne?=
 =?UTF-8?q?w=20FV=20targets=20and=20update=20TARGETS.md=20phases?=
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Add three new formal verification targets:
- Target 13: ValidationResult algebraic properties (tiny discriminated union, all decide-provable)
- Target 14: CommandLineOptionsValidator.ValidateNoUnknownOptions (set-membership property)
- Target 15: CommandLineOptionsValidator.ValidateOptionsAreNotDuplicated (uniqueness over multi-map)

Also synchronize TARGETS.md with merged PRs:
- Fix Target 5 (CommandLineParseResult.Equals) to phase 2 with PR #7918 reference
- Fix Target 3 (ParseOptionAndSeparators) with PR #7919 reference
- Add Targets 8-12 from merged PR #7849 (PasteArguments, TimeSpanParser, etc.)
- Update notes and priority order

πŸ”¬ Lean Squad β€” automated FV agent.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---
 formal-verification/RESEARCH.md | 78 +++++++++++++++++++++++++++++++++
 formal-verification/TARGETS.md  | 30 +++++++++----
 2 files changed, 99 insertions(+), 9 deletions(-)

diff --git a/formal-verification/RESEARCH.md b/formal-verification/RESEARCH.md
index 64975c761..63591fa7d 100644
--- a/formal-verification/RESEARCH.md
+++ b/formal-verification/RESEARCH.md
@@ -201,6 +201,82 @@ A pure recursive function that evaluates whether a test-node path and property b
 - For simple finite types (like `ArgumentArity` constants), we rely on `decide` for closed proofs.
 - We track which theorems are `sorry`-guarded vs. fully proved in `TARGETS.md`.
 
+---
+
+## Target 13: `ValidationResult` algebraic properties
+
+**File**: `src/Platform/Microsoft.Testing.Platform/Extensions/ValidationResult.cs`
+**Type**: `readonly struct Val
... (truncated)

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/agentic-workflowsGitHub agentic workflow definitions under .github/workflows/*.md.lean-squadtype/automationCreated or maintained by an agentic workflow.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions