Sodupath NLP Engine Pipeline Specifications
The Sodupath engine executes a 5-phase deterministic pipeline designed to achieve millisecond latency and explainable diagnostics across literary and journalistic Sinhala texts.
Unicode Sanitization & ZWJ Disambiguation
Normalizes irregular character encodings, strips dangling Zero-Width Joiners (ZWJ - U+200D), and unifies Kumbakarana / Bandhi glyph combinations across legacy operating system outputs.
Sandhi & Compound Tokenization
Executes bidirectional sliding-window segmentation to separate agglutinated clitics (e.g. -ද, -නම්, -මිස) while preserving genuine compounds (e.g., ප්රතිපත්ති) and multi-token verbal stems.
Lexicon Lookup & Orthographic Disambiguation
Scans tokens against the 75,000+ lemma core dictionary and user-registered named entities. Erroneous retroflex/dental permutations (ණ/න, ළ/ල) are flagged with historical etymological explanations.
Subject-Verb Agreement (SVA) Diagnostic Suite
Identifies nominative agents, verifies animacy status, isolates verbal participles, and checks person, number, and gender concordance with the terminal predicate.
Diagnostic Hub Scoring & Actionable Suggestions
Generates categorized issue cards (Spelling, Grammar SVA, Segmentation, Punctuation) with precise character offsets and one-click interactive replacement actions.
REST API Endpoints
/api/check
Primary proofreading endpoint. Analyzes raw Sinhala text and returns structured diagnostics with offsets, rule categories, and correction suggestions.
{ "text": "ඔහු ගෙදර ගියාය.", "selected_gender": "auto" }
/api/suggest_words?q={query}
Autocompletion and phonetic lookup service returning orthographically verified Sinhala lemmas in real-time.