fix: change hallucination detection tests to accept differences in explanation

[jakelorocco]

Misc PR

Type of PR

• Bug Fix
• New Feature
• Documentation
• Other

Description

• Link to Issue: Fixes fix: hallucination detection tests #1006

Made the test more permissive in accepting output. Also fixed one unnecessary warning from the open ai backend.

New versions of the tests pass both locally on an apple silicon mac and on a linux machine:

test/stdlib/components/intrinsic/test_rag.py::test_hallucination_detection PASSED                                                              [ 50%]
test/stdlib/components/intrinsic/test_rag.py::test_hallucination_detection_resolve PASSED                                                      [100%]

Testing

• Tests added to the respective file if code was changed
• New code has 100% coverage if code as added
• Ensure existing tests and github automation passes (a maintainer will kick off the github automation when the rest of the PR is populated)

Attribution

• AI coding assistants used

[github-actions]

The PR description has been updated. Please fill out the template for your PR to be reviewed.

[planetf1]

Leaving a few minor notes from a panel review — nothing blocking.

mellea/backends/openai.py (removed block): The removed guard was the only explicit signal when an ALoraRequirement reaches standard generation via a non-generate caller. The upstream warning at lines 500–505 covers the normal fallback path, but a short comment would make the new contract clear and prevent the guard being re-introduced later:

messages.extend(self.formatter.to_chat_messages([action]))
# ALoraRequirement may arrive here when no adapter is registered;
# _generate is responsible for logging a warning in that case.

[planetf1]

The commented-out assert references result["explanation"] and expected["explanation"], but inside the loop those are lists, not dicts — the per-element variables are r and e. If anyone uncomments this to re-enable the check it'll raise TypeError rather than AssertionError, which is a confusing failure to debug. Either fix the identifiers or remove the line:

# To re-enable: assert r["explanation"] == e["explanation"]

[jakelorocco]

Fixed it. Thank you.

[planetf1]

explanation is dropped from the comparison entirely. The cross-platform variation justifies skipping an exact-string match, but for a qualitative test it's worth keeping a minimal shape check so a regression that returns an empty or None explanation doesn't pass silently:

assert isinstance(r["explanation"], str) and r["explanation"].strip()

[jakelorocco]

Good idea. Made this change.

[planetf1]

Changes look good — the openai.py removal correctly resolves a contradiction where the upstream code promised a fallback but the inner guard was raising instead, and the test relaxation is a reasonable cross-platform fix with preserving length-mismatch detection. Minor notes left inline.