AMTA Launches Working Group to Standardize Translation Quality Estimation Evaluation
The Association for Machine Translation in the Americas (AMTA) has established a QE Users Working Group to create a standardized framework for evaluating quality estimation (QE) systems used in enterprise AI translation workflows. This initiative addresses the current lack of a unified methodology, which has led to inconsistent evaluations and challenges for users in benchmarking results against peers. The working group, composed solely of QE users and researchers, aims to develop guidance and testing protocols that can be applied across various data and workflows.
The initiative, running from December 2025 to June 2026, will culminate in a presentation at the AMTA Conference in Québec City in September 2026. The outputs will be released under an open license, promoting wider accessibility and collaboration. As automated evaluation methods evolve, this effort seeks to enhance transparency and comparability in the assessment of QE systems, which is increasingly important as these technologies become integral to translation processes. For further details, readers can refer to the full article.