[Enhancement]: Voice-to-Text Transcription Module — Implementing FireForm's Core "Talk to Fill" Capability

[pigeio]

---

name: 🚀 Feature Request
about: Suggest an idea or a new capability for FireForm.
title: "[FEAT]: "
labels: enhancement
assignees: ''

---

📝 Description

FireForm's foundational design describes voice memos as the primary input method for first responders, but no voice-to-text transcription code exists in the codebase. This enhancement proposes adding a Whisper-based transcription module to complete this core capability.

Evidence that voice input is part of FireForm's design:

Source	Reference
README.md	"A firefighter records a single voice memo or fills out one 'master' text field" — voice listed first
Hackathon submission	"Our goal is to use AI to allow firefighters to complete paperwork simply by talking out loud"
Hackathon project tags	llm, Paperwork, Voice-to-Text, FireForm
GSoC listing	"report incidents using natural language, either voice or text"
backend.py line 37	LLM prompt says "information extracted from transcribed voice recordings"
backend.py line 11	Parameter is named transcript_text — designed for voice transcription
Zero audio/speech/transcription code exists in the repo or the original prototype (rte2025).

💡 Rationale

First responders work in the field — at fire scenes, in trucks, at emergency sites. Typing isn't practical when wearing firefighting gloves, driving an emergency vehicle, or exhausted after a 12-hour shift.

Voice input isn't a "nice to have" — it's the only realistic input method for the target users. The entire existing pipeline (transcript_text → LLM prompt → JSON → PDF) already assumes voice transcription exists upstream, but the actual transcription module was never built.

🛠️ Proposed Solution

Add a VoiceManager module using OpenAI Whisper (open-source, MIT license, runs entirely locally — no data ever leaves the machine, aligning with FireForm's privacy-first design):

from voice_manager import VoiceManager

vm = VoiceManager()
transcript = vm.transcribe("incident_report.wav")

✅ Acceptance Criteria

• VoiceManager transcribes audio files to text using Whisper locally
• Multiple audio formats supported (wav, mp3, m4a, ogg, flac, webm)
• Model size configurable via WHISPER_MODEL environment variable
• Runs entirely offline (no external API calls)
• Feature works in Docker container.
• Documentation updated in docs/.
• JSON output validates against the schema.

📌 Additional Context

• OpenAI Whisper: https://github.com/openai/whisper (MIT license, runs on CPU/GPU)
• The base model (~140MB) offers a good accuracy/speed tradeoff for field use
• Whisper supports 99+ languages — useful for FireForm's goal as a Digital Public Good for global adoption
• This aligns with the GSoC project description: "report incidents using natural language, either voice or text"

[Gopisokk]

I'd like to implement this module and integrate it with the new API
structure coming in PR #43.

Would @marcvergees or @juanalvv be open to
Assigning this to me?

[krishivsaini]

Hi! I'm also very interested in implementing this. I notice this hasn't been assigned yet @marcvergees @juanalvv would you be open to assigning this? Happy to collaborate with anyone else interested as well.

[pigeio]

Hi @Gopisokk @krishivsaini — I'm the author of this issue and I already have the implementation ready to go. I'll be submitting the PR shortly. Thanks for the interest though!

[Gopisokk]

Thanks @pigeio — looking forward

I took a look at the implementation approaches while writing my GSoC proposal. One thing worth considering when reviewing PR #89 is the choice of backend:

	openai-whisper (proposed)	faster-whisper (alternative)
Speed (13-min audio, CPU)	~10 min 31 sec	~2 min 04 sec
RAM	~3.1 GB	~995 MB
PyTorch dep	✅ Required (+1.5GB Docker image)	❌ Not needed (CTranslate2)

faster-whisper is ~4× faster on CPU with a significantly smaller Docker image — potentially worth the swap since fire stations may only have CPU available.

Either way, I'm planning to build the API layer on top — POST /transcribe endpoint, Transcription SQLModel table, Docker integration (ffmpeg, WHISPER_MODEL env var, model cache volume), and a full test suite with mocked Whisper — as part of my GSoC proposal. Happy to coordinate once PR #89 is merged so we don't duplicate work.

[pigeio]

Great point @Gopisokk — I verified the benchmarks and faster-whisper is clearly the better choice for CPU-only environments like fire stations (~4x faster, 3x less RAM, no PyTorch dependency).

I've updated PR #89 to use faster-whisper with int8 quantization for maximum CPU performance. Also added configurable compute_type support so users with GPU can switch to float16 via environment variable.

Thanks for the catch!