A Comparative Study of Lexical, Dense, and Hybrid Retrieval for Bilingual Graduation Proposal Similarity Detection

Abstract

We compare three retrieval strategies for bilingual (Arabic--English) graduation project proposal similarity detection: a lexical baseline (BM25), a dense pipeline (fine-tuned Multilingual-E5 Bi-Encoder with Cross-Encoder re-ranking), and a hybrid pipeline that fuses both via Reciprocal Rank Fusion. The Bi-Encoder was domain-adapted on 6000 synthetic proposals and evaluated on an absolute gold standard of 300 real Arabic proposal pairs, translated to English via DeepL with human review, under a strict cross-lingual task where each English query must locate its exact Arabic counterpart among 300 candidates. BM25 exhibited severe performance degradation in the cross-lingual setting, achieving a Precision@1 of 0.113---effectively at the level of uninformed retrieval. Crucially, the Dense pipeline significantly outperformed the Hybrid pipeline (p < 0.001, Wilcoxon signed-rank test). In this strict cross-lingual environment, RRF promoted irrelevant BM25 keyword matches, causing the lexical component to introduce spurious candidates that degrade re-ranking precision by displacing correct dense candidates. Qualitative evaluation further revealed that Hybrid produces false positives on novel queries, whereas Dense correctly rejects them with negative Cross-Encoder scores. We recommend the pure Dense pipeline for institutional deployment due to its superior accuracy, false-positive robustness, and single-index simplicity. External validation on the STS17 benchmark confirms that the dense models possess genuine cross-lingual semantic understanding (0.77 on Arabic pairs).