[FEAT]: Add Confidence Score for Each Extracted Field

[Cubix33]

📝 Description

Add a confidence score for each field extracted by the LLM.

Instead of returning only the value, the model should return both:

• The extracted value
• A confidence score (0 to 1)

Example output:

{
  "employee_name": {
    "value": "John Doe",
    "confidence": 0.92
  }
}

---

💡 Rationale

Currently, FireForm blindly trusts the LLM output.

Adding a confidence score will:

• Help detect uncertain extractions
• Enable human review workflows
• Improve reliability for real-world use
• Make debugging easier

This is especially useful for ambiguous or incomplete transcripts.

---

🛠️ Proposed Solution

• Update the LLM prompt to request confidence scores.

• Parse and validate the extended JSON structure.

• Modify PDF filling logic to use the "value" key.

• Optionally log low-confidence fields.

• Logic change in src/

• Update to requirements.txt

• New prompt for Mistral/Ollama

---

✅ Acceptance Criteria

• Each field returns both value and confidence
• Confidence score is between 0 and 1
• PDF filling still works correctly
• Feature works inside Docker

---

📌 Additional Context

This enhancement improves reliability and prepares FireForm for future UI or review-based workflows.

[Godatcode]

Hi @juanalvv, I'd like to take on this issue. I have experience working with LLMs, prompt engineering, and structured JSON output parsing in Python. I plan to:

1. Update the Ollama prompt to return both value and confidence score per field
2. Add JSON schema validation for the new output format
3. Update the PDF filling logic to use the "value" key
4. Add logging/flagging for low-confidence fields
5. Write unit tests for the updated pipeline
   I'll review the current codebase first and share my approach before submitting a PR. Could you assign this to me?
   CC @vharkins @marcvergees

[Godatcode]

Submitted PR #66 implementing the confidence score feature described here. Each extracted field now returns {"value": "...", "confidence": 0.92} with clamping, fallback handling, and low-confidence warnings. Includes 17 unit tests.