Create a Test
To test the accuracy of your custom model, you can create a test in the Azure AI Foundry portal. This test requires a collection of audio files and corresponding transcriptions. By comparing your custom model's accuracy with a base model or another custom model, you can evaluate the word error rate (WER) to measure speech recognition results.
Get Test Results
Once you have created a test, you should get the test results to analyze the accuracy of the models. By evaluating the word error rate (WER) compared to speech recognition results, you can determine the effectiveness of the models in transcribing audio data.
Evaluate Word Error Rate (WER)
The industry standard for measuring model accuracy in speech recognition is word error rate (WER). WER calculates the number of incorrectly identified words divided by the total number of words in the human-labeled transcript. It accounts for insertion, deletion, and substitution errors to provide a percentage value representing the accuracy of the model.
Resolve Errors and Improve WER
To enhance the accuracy of your speech recognition model, analyzing and addressing the errors identified in the WER calculation is crucial. A lower WER percentage indicates better model quality. By understanding the distribution of errors and their causes, such as weak audio signal strength or insufficient domain-specific terms, you can make targeted improvements to enhance model performance.
Evaluate Token Error Rate (TER)
In addition to WER, Token Error Rate (TER) offers an extended measurement of model accuracy by evaluating the recognition of tokens in the human-labeled transcript. TER considers factors like punctuation and capitalization to assess the quality of the end-to-end display format. By calculating TER based on token level errors, you can gain more insight into the model's performance beyond word-level accuracy.
Stay Ahead in Today’s Competitive Market!
Unlock your company’s full potential with a Virtual Delivery Center (VDC). Gain specialized expertise, drive
seamless operations, and scale effortlessly for long-term success.
Book a Meeting to Avail the Services of Azure Custom Speech Service