Official Publication of the Sodupath Computational Linguistics Hub.
1. Introduction: The Need for a Dedicated Sinhala Proofreading Mechanism
The rapid proliferation of digital word processors and communication platforms has fundamentally reshaped human literacy. Yet, for low-resource languages of the Global South, the development of robust grammatical proofreading tools has lagged decades behind English and other major European languages (Joshi et al., 2020). Among South Asian languages, Sinhala (සිංහල)—an insular Indo-Aryan language spoken by over 17 million people—presents a uniquely formidable computational challenge rooted in classical diglossia (Ferguson, 1959; Gair, 1998; De Silva, 1979).
In modern Sri Lankan society, the language exists in two sharply polarized varieties:
- Spoken Sinhala (කථා සිංහල - Kathā Sinhala): The vernacular spoken dialect has completely eliminated finite verbal inflection for person, number, and gender. An invariant verbal predicate such as යනවා (yanawā / goes/go) is employed universally across all personal pronouns:
මම යනවා ≡ අපි යනවා ≡ ඔහු යනවා ≡ ඔවුහු යනවා. - Literary Sinhala (ලියන සිංහල - Liyana Sinhala): Conversely, the standard written medium—mandated in formal publishing, secondary/tertiary education, news journalism, legislation, and civil administration—preserves a conservative morphosyntactic architecture inherited from Classical Middle Indic traditions. In literary texts, finite predicates must strictly harmonize with nominative subjects across Grammatical Person (1st: -මි/-මු, 2nd: -හි/-හු, 3rd: -ේය/-ාය/-හ), Grammatical Number (Singular vs. Plural), Natural Gender (Masculine -ේය vs. Feminine -ාය in 3rd person singular indicative), and Animacy (animate volitional beings vs. inanimate entities).
Because native speakers speak a language with zero verbal agreement but write in a medium demanding rigorous inflectional concord, Subject-Verb Agreement (SVA) discord constitutes the single most prevalent grammatical error in digital Sinhala writing.
1.1 The Orthographic Disambiguation Crisis
Simultaneously, Sinhala orthography features pervasive phonological traps. Centuries ago, unconditioned phonological mergers collapsed retroflex [ɳ] into dental [n], and retroflex [ɭ] into dental [l] in everyday speech (Geiger, 1938; Karunatillake, 2004). Modern native speakers pronounce ගුණ (virtues) and *ගුන, or කළා (did / past verb) and කලා (arts / nominal) with identical phonetic realizations. Yet, standard orthography enforces strict etymological distinction. Without computational assistance, digital corpora are contaminated with widespread retroflex/dental substitutions (Peduru Hewage, 2026b).
1.2 The Failure of Off-the-Shelf Tools and Deep Neural Models
Standard word processing spellcheckers (such as Microsoft Word or Hunspell) analyze tokens in total isolation. While Hunspell can verify whether a word exists in a static wordlist, it possesses zero awareness of inter-word morphosyntactic concord; it accepts sentences with blatant agreement breakdown (e.g., *ඔහු කියවූහ [He read-plural]) simply because both tokens are valid lexical entries.
Furthermore, commercial Large Language Models (LLMs) like GPT-4o or LLaMA-3 exhibit severe structural deficiencies when applied to diglossic Sinhala:
- Hallucinatory Grammar Corrections: LLMs frequently misparse non-finite verbal participles (දුන්) and inanimate mass nouns (අසුන්) terminating in nasal codas as plural subjects, erroneously rewriting correct singular sentences into plural forms.
- High Latency & Expensive Hardware: Neural decoders require 800ms–2,500ms per paragraph and costly GPU clusters, rendering interactive typing assistance unfeasible.
- Lack of Explainability: LLMs cannot cite classical linguistic rules, depriving writers and students of pedagogical clarity.
To solve these structural bottlenecks, the Sodupath (සෝදුපත් - sodupath.com) project was undertaken to create a deterministic, finite-state NLP engine combining rule-based linguistic precision with sub-15ms real-time execution.
2. Chronological Engineering Evolution: The 34 Milestones of Sodupath
The engineering of Sodupath progressed across 34 documented development milestones organized into six progressive phases:
Formulated the initial Python/Flask engine, established multi-port WSGI fallback servers (ports 5000/8080), implemented author-perspective gender concord (මම කියැවීමි vs මම කියැවූයෙමි), and developed parsing algorithms capable of handling flexible word orders (SOV, SVO, OSV) without false alarms.
Codified classical Sidath Sangarawa Natwa and Latwa Vidhiya rules, distinguishing past verbal participles derived from root කර requiring retroflex ළ (කළ, කළා, කළේය) from nominal stems requiring dental ල (කලා, කලාව). Engineered in-memory prefix tries and the responsive Diagnostic Hub side-panel.
Created sliding-window Sandhi tokenizers to decompose attached clitics (-ද, -නම්, -වෙත, -සඳහා, -ගෙන්) while preserving indivisible morphemes. Ingested 75,000 certified academic lemmas and implemented SHA-256 administrative privilege authentication.
Solved Unicode multi-byte character offset misalignments, protected indivisible particles (තමයි, නමුත්, පවා), established Spoken Discourse Preservation heuristics to prevent unwanted literary flags in quoted dialogue, and structured unrecognized neologism classification.
Implemented nominal animacy and case tagging, filtering out dative (-ට), accusative (-ව), and instrumental (-ගෙන්) nominals from acting as agreement governors. Isolated the Sanskrit loan particle ච (cha), implemented feminine concord (-ාය, -වාය, -න්නීය), and formalized the Nominative Agent Precedence (NAP) algorithm resolving the 4-way 'Hal-Na' polysemy.
Integrated official vector branding, deployed the Bilingual Research Hub, secured Canadian Intellectual Property Office (CIPO Reg. No. 1250589) copyright registration, and integrated Google Scholar Highwire Press indexing metadata.
3. Algorithmic Architecture and Pipeline Design
The Sodupath NLP engine is architected as a deterministic, multi-stage pipeline executing sequentially on input text. Because the system prioritizes symbolic linguistic formalisms over opaque neural approximations, every diagnostic output carries an explicit grammatical justification citing classical literary mandates.
4. Resolving the Central Bottleneck: Nominative Agent Precedence (NAP) and False Plural Hal-Na
The primary innovation of Sodupath's morphosyntactic parser is the resolution of the "False Plural Hal-Na" polysemy. In written Sinhala, the terminal virama nasal coda 'න්' (Hal-Na / -un) functions across four homophonous categories:
- True Animate Plural Nominative (C_plural): මිනිසුන් (men), සිසුන් (students) → Mandates plural verb (කියැවූහ).
- Non-Finite Verbal Participle (C_participle): දුන් (given), හුන් (seated) → Relative clause modifier; never governs verb.
- Descriptive Root Adjective (C_adjective): නොසන්සුන් (restless), සන්සුන් (calm) → Modifies head noun; never governs verb.
- Inanimate Neuter Mass/Object Noun (C_inanimate): අසුන් (seat), රන් (gold), දන් (alms) → Direct object; cannot govern plural verb.
In the canonical literary sentence: ඔහු (He-NOM.MASC.SG) + අසුන් (seat-ACC.INAN) + ගත්තේය (took-PAST.3.SG.MASC), naive parsers and off-the-shelf LLMs scan backward from the verb ගත්තේය, encounter අසුන්, misclassify it as a plural subject (like මිනිසුන්), and erroneously flag the singular verb ගත්තේය as an error, proposing spurious plural forms (*ගත්හ).
Sodupath's Nominative Agent Precedence (NAP) algorithm systematically evaluates lexical category tags, filters intervening oblique inflections (-ට, -ව, -ගෙන්) and certified inanimate Hal-Na nouns, and establishes the true nominative agent (ඔහු) as the governing head, achieving 99.4% agreement precision.
5. System Implementation and Technology Stack
The Sodupath engine is deployed as an open web application serving educational institutions, legal bodies, and journalists. The table below summarizes the multi-tiered architecture and operational characteristics:
| Architectural Layer | Technology Stack | Function & Operational Strategy |
|---|---|---|
| Backend Core | Python 3.11+, Regex Automata, FST | Deterministic grammar parsing, Unicode normalization, zero GPU requirements. |
| API Web Server | Flask 3.0, Gunicorn WSGI, Nginx | High-concurrency REST endpoints handling /api/check and /api/suggest. |
| Lexicon Store | In-Memory Prefix Trie, Dual-Hash Tables | Sub-millisecond lookup for 75,000+ certified lemmas with hot-reload capability. |
| Frontend Interface | HTML5, Tailwind CSS, Vanilla JS | Responsive single-page workspace with real-time DOM highlight overlays. |
| Security & Admin | SHA-256 Tokens, Role-Based Access | Cryptographic authentication protecting master dictionary curation from unauthorized writes. |
| Intellectual Property | Canadian Intellectual Property Office | Registered Canadian Copyright (CIPO Reg. No. 1250589) protecting software and rule schemas. |
6. Empirical Benchmarks and Comparative Evaluation
To quantitatively evaluate performance, Sodupath was tested against the open-source Hunspell Sinhala baseline, LLaMA-3-70B, and OpenAI's GPT-4o across a curated corpus of 1,200 literary sentences (600 pristine, 600 syntactically/orthographically degraded), 191 unit regression suites, and 10,000 web archive sentences:
| System / Model | SVA Precision | Orthographic Precision (Na/La) | False Alarm Rate | Mean Latency | Rule Explainability |
|---|---|---|---|---|---|
| Hunspell Baseline | N/A (No SVA) | 45.2% | 32.6% | 6 ms | None (Wordlist only) |
| LLaMA-3-70B | 62.4% | 71.5% | 24.1% | 1,180 ms | Inconsistent / Opaque |
| GPT-4o (OpenAI) | 67.8% | 78.2% | 18.5% | 1,420 ms | Variable / Unpredictable |
| Sodupath Engine (Ours) | 99.4% | 99.6% | 0.4% | 12 ms | 100% Deterministic Citing Rules |
7. Conclusion & Global Availability
The engineering of Sodupath validates that deterministic, rule-based computational linguistics remains the premier paradigm for extreme diglossic and low-resource languages. By fusing Classical Sinhala grammatical treatises (Sidath Sangarawa) with modern finite-state transducers, prefix trie indexing, and the Nominative Agent Precedence algorithm, Sodupath delivers zero-hallucination, sub-15ms proofreading for the Sinhala language.
The complete engine is publicly accessible at https://www.sodupath.com and legally registered under Canadian Intellectual Property Office (CIPO) Registration No. 1250589.