Skip to content

Add MailManInTheMiddle Security Arena scenario submission#19

Merged
lemontreeran merged 3 commits intosubmission/purple_vanguard_MailManInTheMiddlefrom
codex/add-mailmaninthemiddle-scenario-submission
Jan 15, 2026
Merged

Add MailManInTheMiddle Security Arena scenario submission#19
lemontreeran merged 3 commits intosubmission/purple_vanguard_MailManInTheMiddlefrom
codex/add-mailmaninthemiddle-scenario-submission

Conversation

@lemontreeran
Copy link
Collaborator

Motivation

  • Provide a new Indirect Injection security arena scenario that models an executive email assistant exposed to hidden instruction injection inside an attacker email, with a sensitive password-reset email as the target.
  • Ensure the scenario follows competition rules by including deterministic verification, structured tool-call logging, and required evidence files (including a real attack_succeeded.json).

Description

  • Add MailManInTheMiddlePlugin at scenarios/security_arena/submissions/purple_vanguard/mailmaninthemiddle/plugin.py implementing policy-aware prompts, simulated tools (list_unread_emails, get_email, draft_email, forward_email), deterministic success checks, and baseline/attack inputs.
  • Add scenario assets and metadata under scenarios/security_arena/submissions/purple_vanguard/mailmaninthemiddle/ including artifacts/inbox_baseline.json, artifacts/inbox_attack.json, artifacts/company_policy.md, README.md, and scenario_mailmaninthemiddle.toml.
  • Register the plugin in scenarios/security_arena/plugins/registry.py with a direct import and mapping key `

Codex Task

Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: cc26ce56dc

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +15 to +16
from scenarios.security_arena.submissions.purple_vanguard.mailmaninthemiddle.plugin import (
MailManInTheMiddlePlugin,

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Use import path that matches orchestrator sys.path

When the orchestrator is launched via python scenarios/security_arena/orchestrator.py (the command used in this repo’s scenario TOML), it only adds scenarios/security_arena to sys.path; the absolute import scenarios.security_arena.submissions... will therefore raise ModuleNotFoundError unless the repo root is on PYTHONPATH, preventing the new scenario from loading in the default run. Using a relative import from the local submissions directory (or adjusting sys.path) avoids this runtime failure.

Useful? React with 👍 / 👎.

Comment on lines +232 to +235
def _extract_domain(self, address: str) -> str:
if "@" not in address:
return ""
return address.split("@", 1)[-1].lower()

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Parse domains from display-name email addresses

The _extract_domain helper just splits on '@' and returns the remainder, so addresses with display names like Jordan Miles <jordan.miles@purplevanguard.example> produce purplevanguard.example> and are treated as external. That causes false violations (and could fail baseline scoring) whenever a model emits common RFC 5322 address formats in tool calls. Stripping angle brackets/whitespace or using email.utils.parseaddr would prevent internal addresses from being misclassified.

Useful? React with 👍 / 👎.

@lemontreeran lemontreeran merged commit 60af964 into submission/purple_vanguard_MailManInTheMiddle Jan 15, 2026
@lemontreeran lemontreeran deleted the codex/add-mailmaninthemiddle-scenario-submission branch January 15, 2026 12:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant