Abstract:
Reliable translation from English to Sinhala is still a great challenge for many sophisticated
translation systems using Sinhala as a low-resource language. Although Google Translate is
widely used for translationpurposes, recent breakthroughs in large language models such as
ChatGPTand DeepSeek provide entirely new opportunities for translation tasks. This study
proposes one of the first thorough comparative analyses of English-Sinhala translation systems
compared with human translation, both qualitatively and quantitatively. Google Translate,
ChatGPT, DeepSeek, and human translations done by native Sinhala speakers were compared
for translation quality on a carefully prepared dataset of 150English sentences for general, technical,
and academic purposes. Translation quality was compared using BLEU, METEOR, and
COMETscores, in addition to human assessment of fluency, grammatical accuracy, and semantic
translation quality done by qualified human raters using a prepared rubric with inter-rater
reliability tests. Machine learning models were also prepared for predicting translation quality
using language-basedpredictors for translation efficiency and translation system identification.
The experimental results show that human translations were rated highest on all translation
quality measures. Among the automatic translationsystems, LLM-based translation systems
performed better on contextual understanding of complex sentences than Google Translate,
whichperformed reasonably on simple inputs. Correlation tests showthat COMET correlates
better with human translation quality than BLEUandMETEOR. Moreover, the prepared machine
learning models were able todetect translation quality trends accurately for translation
systempredictions, making these models promising for translation qualityassessment in lowresource
language environments.
