Skip to content

Proposal: scaffold optional audit manifest example #10

Description

@mindbomber

Proposal

Would this cookiecutter template be open to scaffolding an optional audit/provenance manifest example for generated MLCube projects?

This is related to the broader MLCube proposal here: mlcommons/mlcube#367

The cookiecutter may be the safest place to start because it can teach good reproducibility/audit hygiene without changing MLCube runtime behavior.

Suggested generated file

For example, generated MLCubes could include an optional file such as:

{{cookiecutter.mlcube_name}}/
  audit_manifest.example.json

Example contents:

{
  "schema_version": "mlcube.run_audit.v1",
  "mlcube_name": "{{cookiecutter.mlcube_name}}",
  "task": "{{cookiecutter.mlcube_name}}",
  "runner": "docker",
  "image": "mlcommons/{{cookiecutter.mlcube_name}}:0.0.1",
  "inputs": ["workspace/parameters_file.yaml"],
  "outputs": ["workspace/output.txt"],
  "claim_status": "diagnostic",
  "redaction_status": "safe_for_public_log"
}

Why this may help

  • gives new MLCube authors a reproducibility/provenance pattern from the first generated project
  • creates a standard place to document run inputs, outputs, runner, image, and claim status
  • encourages audit-safe result publishing without storing raw secrets or sensitive arguments
  • stays optional and does not affect existing runner behavior

Possible first PR scope

If maintainers think this is useful, I can prepare a small PR that either:

  1. adds audit_manifest.example.json to the generated template, or
  2. keeps it docs-only in the README if that is preferred

This is motivated by AANA work around audit-safe AI evaluation artifacts, but the contribution would be generic to MLCube and would not require AANA as a dependency.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions