Presentation: Delving into Speech-to-text meaning – Where do we go beyond WER

Talk:

Word Error Rate (WER) is the primary quality metric for all speech providers. Engineers use it to make almost every research decision and customers use it to evaluate vendors. But it has many flaws… 

Who cares about an extra um, a missing hyphen, or an alternative spelling. Or how about a difference in written numerical formatting (1,000 vs 1000 or eleven vs 11)? These all increase WER but do not affect meaning in the slightest. 

How about omitting the word ‘suspected’ before murderer? Or predicting a wrong word that changes the sentiment? An ideal metric would take these minor and serious errors into account and score a system appropriately. 

If you’re interested to find out how we at Speechmatics are thinking of solving this problem, then come along to the talk!