Translation Quality Metrics (BLEU, TER)
BLEU (Bilingual Evaluation Understudy) and TER (Translation Edit Rate) are automated metrics that estimate machine translation quality by comparing MT output against one or more human reference translations.
BLEU measures the n-gram overlap between the MT output and reference translations. A BLEU score of 1.0 would be a perfect match; scores above 0.4 are generally considered good for MT output. TER measures the number of edits (insertions, deletions, shifts, substitutions) a post-editor would need to make to convert the MT output into the reference translation, normalized by reference length. Lower TER = fewer edits = better quality.
Both metrics have well-known limitations: they measure surface-form similarity rather than meaning, they are sensitive to synonyms and paraphrasing (a correct translation that uses different words is penalized), and they require at least one human reference translation to compare against. More recent metrics like COMET and BLEURT use neural models to evaluate translation quality and are more aligned with human judgment, but require more computation.
Key concepts
- BLEU: n-gram overlap score between MT output and human reference — higher is better
- TER: edit distance between MT output and human reference — lower is better
- Both metrics have limitations; COMET/BLEURT (neural) metrics are more aligned with human judgment
How Vernacia uses this
Vernacia does not expose BLEU/TER scores in the UI. For formal quality evaluations, export the Vernacia output and score it against a human reference translation using an open-source tool like sacrebleu or Hugging Face evaluate.
Translate PPTX, PDF, DOCX, and 11 more formats — free to start.
Try Vernacia free