Interpreting speaker_confidence Scores

[SenseiPatrick]

I am attempting to use speaker_confidence scores to flag uncertain speaker attribution in our transcripts. I am using Nova 3.

Can someone explain in more detail how to interpret the speaker_confidence scores and more about how they are constructed? Documentation on the details is too limited for my use case.

1. Is the speaker_confidence calibrated to some ground truth? Does 0.50 mean 50% chance it's wrong, for example?
2. If not, does Deepgram have any estimate for how the scores translate to likelihood of being accurate?
3. Is the scale linear? Does a 0.6 score have about 2x likelihood of being accurate as a 0.3 score?
4. If it's not linear, what shape is it?
5. If not linear, is there a point in the curve that looks like a cliff, where likelihood falls sharply? For example, >0.60 is good, 0.5-0.6 transition, <0.5 bad.
6. I'm trying to set a threshold for speaker_confidence, below which a word or utterance will get flagged as high risk. Do you have any advice for how I should think about choosing a threshold? Should I choose an absolute value for all transcripts or should it be relative to the distribution for a given transcript? Any guidance about what threshold values you might recommend depending on risk tolerance?
7. Is there underlying data log probabilities, which are then normalized from 0-1, or are there other factors involved?

[deepgram-community[bot]]

Thanks for asking your question. Please be sure to reply with as much detail as possible so the community can assist you efficiently.
_{Consider joining our Discord community for more opportunity to engage with your fellow Deepgram users. You can earn points which can be redeemed for cool stuff by being active in our communities!}

[deepgram-community[bot]]

Hey there! It looks like you haven't connected your GitHub account to your Deepgram account. You can do this at https://community.deepgram.com - being verified through this process will allow our team to help you in a much more streamlined fashion.

[deepgram-community[bot]]

It looks like we're missing some important information to help debug your issue. Would you mind providing us with the following details in a reply?

• The programming language you are working in (e.g. JavaScript, Python).
• The deepgram product you are using (e.g Speech to Text, Agent API)
• A request ID that triggered your error or issue.

[SenseiPatrick]

• JavaScript
• Speech to Text
• N/A