Paper deep dive
Loanword or Switch? The Annotation Boundary, Not the Model, Drives Kazakh-Russian Code-Switching Identification
Bogdan Savelyev
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 94%
Last extracted: 8/4/2026, 4:43:12 AM
Summary
The paper investigates Kazakh-Russian code-switching identification, arguing that the primary bottleneck is the annotation boundary between integrated loanwords and genuine clause-level switches, rather than model architecture. The author releases a document-level gold LID dataset and demonstrates that while simple heuristics and off-the-shelf models over-label mixed text due to shared Cyrillic script and loanwords, a filter-first cascade using XLM-R LID and a mixed-only sentiment pool effectively isolates genuine code-switched content in large-scale social media corpora.
Entities (15)
Relation Signals (8)
Bogdan Savelyev → authored → Loanword or Switch? The Annotation Boundary, Not the Model, Drives Kazakh-Russian Code-Switching Identification
confidence 99% · Bogdan Savelyev Independent Researcher Kazakhstan
Kazakh → sharesscriptwith → Russian
confidence 98% · Kazakh and Russian both use Cyrillic and share most of their alphabet
Annotation Boundary → isbottleneckfor → Code-Switching Identification
confidence 95% · The gap shows the bottleneck is the loanword-vs-switch annotation boundary, not model class alone
XLM-R LID v2 → outperforms → HeLI
confidence 95% · XLM-R LID v2 is the only system with balanced mixed P/R near 0.95... macro-F1 0.966
XLM-R LID v2 → outperforms → Character-trigram NB
confidence 95% · The gap over Char-3gram is contextual modeling of the whole message
XLM-R LID v2 → isusedfor → Sentiment Analysis Filtering
confidence 90% · mixed-only sentiment pool used after LID in a filter-first cascade
fastText → suffersfrom → Over-labeling Mixed Text
confidence 90% · Off-the-shelf LID and letter heuristics over-label Kazakh–Russian social text as mixed
HeLI → suffersfrom → Loanword Confusion
confidence 90% · HeLI raw reaches 0.697 macro-F1... leaving 80 gold-mixed documents still tagged rus after stripping
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Off-the-shelf LID and letter heuristics over-label Kazakh-Russian social text as mixed: Russian loanwords inside Kazakh look like code-switching under a shared Cyrillic script. We release a document-level gold LID set whose guideline keeps integrated borrowings as Kazakh and reserves mixed for clause-level switches, plus a mixed-only sentiment pool used after LID in a filter-first cascade. On a shared LID test, FastText, Lingua, raw and windowed HeLI, character-trigram NB, and XLM-R range from weak to strong performance. The gap shows the bottleneck is the loanword-vs-switch annotation boundary, not model class alone.
Tags
Links
- Source: https://arxiv.org/abs/2608.00581v1
- Canonical: https://arxiv.org/abs/2608.00581v1
Trouble viewing inline? Open PDF directly →
Full Text
26,112 characters extracted from source content.
Expand or collapse full text
Loanword or Switch? The Annotation Boundary, Not the Model, Drives Kazakh–Russian Code-Switching Identification Bogdan Savelyev Independent Researcher Kazakhstan _if_language:nTFENG =Englishbogsav999@gmail.com Abstract Off-the-shelf LID and letter heuristics over-label Kazakh–Russian social text as mixed: Russian loanwords inside Kazakh look like code-switching under a shared Cyrillic script. We release a document-level gold LID set whose guideline keeps integrated borrowings as Kazakh and reserves mixed for clause-level switches, plus a mixed-only sentiment pool used after LID in a filter-first cascade. On a shared LID test, FastText, Lingua, raw and windowed HeLI, character-trigram NB, and XLM-R range from weak to strong performance. The gap shows the bottleneck is the loanword-vs-switch annotation boundary, not model class alone. Code, data, and weights: _if_language:nTFENG =Englishhttps://github.com/naadgob/KazNLP _if_language:nTFENG =Englishhttps://huggingface.co/datasets/naadgob/kaznlp-weights [ Extension = .otf, UprightFont = *-regular, BoldFont = *-bold, ItalicFont = *-italic, BoldItalicFont = *-bolditalic ] [ Extension = .ttf, UprightFont = *, BoldFont = *-Bold, ItalicFont = *-Oblique, BoldItalicFont = *-BoldOblique, Scale = 0.9 ] [ Extension = .otf, UprightFont = *, BoldFont = *Bold, ItalicFont = *Italic, BoldItalicFont = *BoldItalic, Script = Cyrillic ] [ Extension = .otf, UprightFont = *, BoldFont = *Bold, ItalicFont = *Italic, BoldItalicFont = *BoldItalic, Script = Cyrillic ] [ Extension = .ttf, UprightFont = *, BoldFont = *-Bold, ItalicFont = *-Oblique, BoldItalicFont = *-BoldOblique, Scale = 0.9, Script = Cyrillic ] _if_language:nTFENG =English Loanword or Switch? The Annotation Boundary, Not the Model, Drives Kazakh–Russian Code-Switching Identification Bogdan Savelyev Independent Researcher Kazakhstan _if_language:nTFENG =Englishbogsav999@gmail.com _if_language:nTFENG =English1 Introduction Most people in Kazakhstan speak both Kazakh and Russian, and online writing often mixes the two in one message. We set out to build sentiment analysis for that code-switched slice, but no open labeled set existed for the task, so we collected a corpus of over 420,000 Telegram, Kaspi, and 2GIS texts. The scrape was unlabeled: to train sentiment only on genuine mixed messages (Kazakh and Russian used in one document), we first needed automatic language identification (LID) at a scale that rules out hand-filtering every row. A first, letter-based heuristic tagged a message as mixed whenever it contained both Kazakh- and Russian-specific letters. Kazakh and Russian both use Cyrillic and share most of their alphabet, and everyday Kazakh carries many identical Russian loanwords, so almost any Kazakh sentence looked “mixed” under that rule. Off-the-shelf LID (FastText, Lingua) makes the same mistake: it cannot tell an integrated borrowing from a real switch. The distinction is easy to miss on a letter-level rule, but clear once the annotation line is spelled out. Consider two short messages: Mixed: _if_script:nTFcyrl =Cyrillic _if_language:nTFRUS =RussianКурьер молодец, уақытында әкелді. (Russian clause + Kazakh clause: a genuine switch.) Not mixed (loanword): _if_script:nTFcyrl =Cyrillic _if_language:nTFRUS =RussianКачествосы жақсы, арзан. (Kazakh morphosyntax with an integrated Russian loan, _if_script:nTFcyrl =Cyrillic _if_language:nTFRUS =Russianкачество + Kazakh suffix _if_script:nTFcyrl =Cyrillic _if_language:nTFRUS =Russian-сы; no switch.) Without an explicit loanword-vs-switch rule, both look “mixed” to a naive letter heuristic and to many off-the-shelf LID systems, because both contain Russian and Kazakh material written in the same Cyrillic script. The false-mixed problem is not only weak models; it is a failure of the label definition. We show this on a shared gold held-out test (n=461n=461), comparing neural LID with simple non-neural baselines. Our contributions are: _if_language:nTFENG =English1. A document-level gold LID set of 3,076 messages (ru / kz / mixed) with an explicit loanword-vs-switch annotation rule. _if_language:nTFENG =English2. A comparison ladder on that test: FastText, Lingua, HeLI (raw / loanword-neutral / overlapping windows), character-trigram NB, and fine-tuned XLM-R; Char-3gram is near ceiling on monolingual classes (ru 146/150146/150, kz 147/150147/150) but recovers only 114/161114/161 true mixed, so the bottleneck is the label boundary rather than missing character signal. _if_language:nTFENG =English3. Corpus-scale application: on 331,468 documents the filter keeps 4.9%4.9\% as mixed, against a keyword heuristic with ≈1.66%≈1.66\% precision on its Telegram mixed pool. _if_language:nTFENG =English4. A filter-first sentiment head (pos/neg) trained on a mixed-only gold pool and run only after LID; reported as downstream utility, not as the main claim. _if_language:nTFENG =English2 Related Work The theoretical line behind our label rule is the long-standing distinction between borrowing and code-switching (Poplack, 1980): integrated loanwords belong to the recipient language, while productive switches do not. Computational work often collapses that line. Shared or ambiguous lexicon is treated as a special tag in some Arabic CS corpora (Wintner et al., 2023); we keep a document-level three-way label (ru/kz/mixed) and push the same ambiguity into the annotation guideline instead. Language identification is surveyed extensively by Jauhiainen et al. (2019). For short, noisy, or multilingual spans, word-list systems such as HeLI and language-set identification over overlapping windows (Jauhiainen et al., 2015) remain strong, interpretable baselines. Large-coverage neural or n-gram LID tools (OpenLID, GlotLID) work well on clean monolingual text but are known to overestimate accuracy on web data (Burchell et al., 2023; Kargaran et al., 2023). Our setting adds a harder failure mode: Kazakh and Russian share Cyrillic, so loanword-bearing Kazakh looks mixed to letter heuristics and to many off-the-shelf detectors. Character n-grams have a long history for Turkic and web-scale LID (Baisa and Suchomel, 2012); we include a smoothed character-trigram Naive Bayes baseline for that reason. Closely related contact pairs with the same boundary problem appear in Kyrgyz–Russian work and in Kyrgyz web corpora contaminated by neighboring Turkic text; we treat those as parallel motivation rather than as datasets we reuse. Code-switching workshops (CALCS) and variation shared tasks (VarDial) are natural homes for underrepresented pairs and for similar-language confusability. We do not frame k–ru as dialect identification: the two are distinct languages in contact. The scientific claim is narrower—that an explicit loanword-vs-switch annotation rule changes what automatic LID can honestly measure on a shared-script pair. _if_language:nTFENG =English3 The Gold Set Table _if_language:nTFENG =English1 summarizes the two labeled resources. LID is document-level ru/kz/mixed; SA is pos/neg on mixed reviews only. Both were annotated solo (no IAA). Set n Gold Rule LID 3076 3076 borrow→kz; switch→mixed SA 4411 3529 pos/neg; mixed only _if_language:nTFENG =EnglishTable 1: Gold resources. LID classes: mixed 1077 / ru 1000 / kz 999 (split 2691/462/461). SA ∼ 50/50 pos/neg (split 3334/526/525; 882 synthetic). Solo annotation; no IAA. _if_language:nTFENG =English4 Experimental Setup All LID systems are evaluated on the same held-out gold test (n=461n=461; train 2,6912,691 / val 462462 / test 461461). We report accuracy, macro-F1, and mixed-class precision/recall. Supervised models (FastText, Char-3gram NB, XLM-R LID) train on the gold training split only. HeLI ladder. As an interpretable non-neural baseline we use HeLI through heliport. _if_language:nTFENG =English1 _if_language:nTFENG =English1 _if_language:nTFENG =English1Rust/Python port of HeLI-OTS: _if_language:nTFENG =Englishhttps://github.com/ZJaume/heliport The ladder below follows advice from Tommi Jauhiainen in email correspondence, adapted here to short k–ru social texts and our document-level mixed label. The first step is a loanword-neutral re-identification pass: build a list of 100%-identical Russian borrowings used in Kazakh, remove (or score neutral) those tokens, and re-run HeLI (HeLI+neutral). That cleans shared vocabulary but does not by itself detect genuine code-switching. On our test that prediction held: HeLI raw reaches 0.6970.697 macro-F1, while HeLI+neutral is slightly worse (0.6830.683), leaving 8080 gold-mixed documents still tagged rus after stripping. To recover those switches, we then adapt HeLI’s language-set idea (Jauhiainen et al., 2015) down from long documents to short UGC lines: after the loanword strip, split each text into overlapping two-, three-, or longer word windows, run heliport on each window, and output mixed if more than one language appears (optionally with a higher count threshold). A small grid search over window sizes and min_countmin\_count tracks the typical length of switches in this data. We implement that as HeLI+windows; the best setting on our grid is sizes (2,3)(2,3) with min_count=1min\_count=1, which raises macro-F1 from 0.6970.697 (raw) to 0.8690.869 and flips 69/8069/80 of the residual mixed-as-rus bucket to mixed. Other systems. The character baseline is Multinomial Naive Bayes over raw-text character trigrams ( _if_language:nTFENG =EnglishCountVectorizer(analyzer=’char’, ngram_range=(3,3)), Laplace α=1α=1). Off-the-shelf comparators are FastText and Lingua (v1/v2). The neural LID model is XLM-RoBERTa fine-tuned on Gold LID (XLM-R LID v1/v2; v2 is the main neural result). Sentiment analysis (SA) is a separate binary head (pos/neg) on the filtered mixed review slice only (filter-first cascade), evaluated on its own hold-out (n=525n=525), not on the LID test. _if_language:nTFENG =English5 Results: The Ladder Table _if_language:nTFENG =English2 reports the full comparison. Model Acc Macro-F1 P R LID (ru / kz / mixed; n=461n=461). P/R == mixed. FastText v1 0.649 0.632 0.557 0.335 FastText v2 0.716 0.709 0.658 0.491 HeLI raw 0.703 0.697 0.589 0.534 HeLI+neutral 0.690 0.683 0.573 0.491 HeLI+windows (2+3, min1) 0.872 0.869 0.925 0.689 Char-3gram NB 0.883 0.880 0.942 0.708 Lingua v1 0.846 0.850 0.739 0.863 Lingua v2 0.889 0.886 0.768 0.988 XLM-R LID v1 0.959 0.959 0.938 0.944 XLM-R LID v2 0.965 0.966 0.950 0.950 SA (pos / neg on mixed; n=525n=525). P/R == positive. XLM-R SA v1 (best) 0.973 0.973 0.977 0.969 XLM-R SA v2 0.962 0.962 0.962 0.962 _if_language:nTFENG =EnglishTable 2: Full model ladder. Top: document-level LID. Bottom: SA on the filtered code-switched slice only (filter-first cascade). For SA v1, negative-class P/R =0.970/0.977=0.970/0.977; for SA v2 both =0.962=0.962. Most SA labels are LLM-drafted. Several patterns stand out from Table _if_language:nTFENG =English2. First, stripping shared loanwords alone does not help HeLI: HeLI+neutral (macro-F1 0.6830.683) is slightly worse than HeLI raw (0.6970.697). Lexical neutralization without structure is not enough for document-level switch detection. Second, overlapping windows carry most of the non-neural gain. HeLI+windows (sizes 2+32+3, min_count=1min\_count=1) jumps to 0.8690.869 macro-F1 and mixed precision 0.9250.925; of the 8080 gold-mixed documents that stayed rus after the strip, 6969 flip to mixed. Third, the character-trigram NB is the strongest simple orthographic baseline (0.8800.880 macro-F1). Monolingual recall is near ceiling (ru 146/150146/150, kz 147/150147/150), yet mixed recall is only 0.7080.708 (4747 misses: 20→ru20→ru, 27→kz27→kz). The bottleneck is the loanword-vs-switch boundary, not missing character signal. Fourth, off-the-shelf LID shows a different trade-off: Lingua v2 reaches mixed recall 0.9880.988 but precision only 0.7680.768, so it over-tags mixed and is a weak corpus filter. FastText v2 improves on v1 but stays near HeLI raw on macro-F1 (∼0.71 0.71). Fifth, XLM-R LID v2 is the only system with balanced mixed P/R near 0.950.95 (macro-F1 0.9660.966). The gap over Char-3gram is contextual modeling of the whole message, not merely “neural vs non-neural.” Sixth, sentiment analysis is a downstream head on the filtered mixed slice. XLM-R SA v1 is best (accuracy / macro-F1 0.9730.973 on n=525n=525); v2 is close but lower (0.9620.962). These numbers measure cascade utility after LID, not a standalone sentiment benchmark, and most tone labels are LLM-drafted. _if_language:nTFENG =English6 Error Analysis Of the 80 gold-mixed documents that HeLI still tagged rus after loanword stripping, HeLI+windows (sizes 2+32+3, min_count=1min\_count=1) flips 69 to mixed. We inspected the remaining 11 by hand: tokens after strip, every window code from heliport, and isolated probes of the Kazakh spans. All 11 fail the same way. After stripping, no 2- or 3-word window ever receives code kaz, so the language-set vote never sees a Kazakh side and cannot emit mixed. None of the cases fail because the document is too short, and none fail because a kaz window existed but lost the vote. The limit sits in heliport on short windows where Kazakh is glued to Russian neighbors or written with informal spelling. Single-word Kazakh insertions (5/11). One Kazakh content token sits inside an otherwise Russian matrix ( _if_script:nTFcyrl =Cyrillic _if_language:nTFRUS =Russianөсек, _if_script:nTFcyrl =Cyrillic _if_language:nTFRUS =Russianқоқыс, _if_script:nTFcyrl =Cyrillic _if_language:nTFRUS =Russianсәлем, _if_script:nTFcyrl =Cyrillic _if_language:nTFRUS =Russianүйренеді, _if_script:nTFcyrl =Cyrillic _if_language:nTFRUS =Russianбұйырса). With minimum window size 2, that token always shares a window with one or two Russian neighbors, and heliport returns rus for spans such as “ _if_script:nTFcyrl =Cyrillic _if_language:nTFRUS =Russianздесь үйренеді” or “ _if_script:nTFcyrl =Cyrillic _if_language:nTFRUS =Russianэто өсек”. A pure Kazakh window never forms. Probed alone at size 1, the same five tokens all map to kaz: the material is recognizable; windowing at ≥2≥2 is what buries it. Short Kazakh spans misread as other languages (4/11). Two-word Kazakh chunks get a related Turkic or Slavic code, which our vote discards because we only keep kaz/rus: _if_script:nTFcyrl =Cyrillic _if_language:nTFRUS =Russianтерең ой→kir, _if_script:nTFcyrl =Cyrillic _if_language:nTFRUS =Russianбіткенше күлдім→ukr (standard spelling; the 200-language identifier confuses short Kazakh with Kyrgyz or Ukrainian), and _if_script:nTFcyrl =Cyrillic _if_language:nTFRUS =Russianқудай сактады/ _if_script:nTFcyrl =Cyrillic _if_language:nTFRUS =Russianкудай сақтасын→kir, where informal spelling drops _if_script:nTFcyrl =Cyrillic _if_language:nTFRUS =Russianқ/ _if_script:nTFcyrl =Cyrillic _if_language:nTFRUS =Russianұ and removes the main Kazakh cue. Rewritten as _if_script:nTFcyrl =Cyrillic _if_language:nTFRUS =Russianқұдай сақтасын or _if_script:nTFcyrl =Cyrillic _if_language:nTFRUS =Russianқұдай сақтады, heliport returns kaz. One token heliport never tags as Kazakh (1/11). _if_script:nTF cyrl =Cyrillic _if_language:nTFRUS =Russianшапшаң alone scores as mhr (Meadow Mari). Size-1 windows would not recover it; the miss is in the identifier, not in the window schedule. Likely label noise (1/11). _if_script:nTF cyrl =Cyrillic _if_language:nTFRUS =Russian«Порог растет, а зп нет» has no Kazakh letters and no Kazakh words (kz_signal=Falsekz\_signal=False), yet gold marks it mixed. We treat it as a candidate annotation error pending a second look at the source thread. Size-1 windows as a diagnostic. Adding size-1 windows (sizes (1,2,3)(1,2,3)) raises macro-F1 from 0.8690.869 to 0.8850.885 and mixed recall from 0.6890.689 to 0.7760.776, but mixed precision falls from 0.9250.925 to 0.8800.880 (kz→mixedkz→mixed errors 8→148→14, ru→mixedru→mixed 1→31→3). Single-token votes catch insertional switches and also punish clean Kazakh that happens to contain one Russian-looking token. The advice that motivated the ladder was “two word, three word or longer”; size 1 is outside that definition and closer to per-token LID. We therefore keep (2,3)(2,3) as the HeLI+windows rung on the ladder and report size 1 only here, as an explicit trade-off rather than as a replacement. Takeaway. Windowed HeLI recovers clause-level switches; by construction it misses one-word insertions when the minimum window is 2. At least four of the eleven residuals are not about windows at all: heliport confuses short Kazakh with Kyrgyz (or Mari), and informal spelling without _if_script:nTFcyrl =Cyrillic _if_language:nTFRUS =Russianқ/ _if_script:nTFcyrl =Cyrillic _if_language:nTFRUS =Russianұ makes that worse. Char-3gram fails on a related boundary: of 161 true mixed documents it reads 47 as monolingual (20→ru20→ru, 27→kz27→kz), again short insertions and loanword-bearing spans. Both non-neural baselines stop at the same loanword-vs-switch line; XLM-R LID crosses it with full-message context. _if_language:nTFENG =English7 Corpus-Scale Application A gold test of n=461n=461 only shows that the filter works on held-out labels. The practical question is what happens when the same XLM-R LID v2 model scores a full scrape. We build a pooled corpus of Telegram comments, Kaspi reviews, and 2GIS reviews across Kazakhstani cities, then deduplicate and clean it into _if_language:nTFENG =Englishmain.csv: 331,468331,468 documents. XLM-R LID v2 assigns 281,409281,409 ru, 33,69533,695 kz, and 16,36416,364 mixed. That is a mixed rate of 4.9%4.9\%. The contrast with the early keyword heuristic is the whole point of the resource. On the Telegram slice alone (241,576241,576 messages), a narrow “Kazakh-letter + Russian-letter” rule flagged 27,62827,628 rows as mixed; hand-checking that pool kept only 460460 genuine switches (precision ≈1.66%≈1.66\%). A later FastText-v2 pass over a larger scrape still sat near 2%2\% heuristic precision on its mixed predictions. Once the gold-trained filter replaces those rules, the corpus-level mixed share collapses from “almost everything bilingual” to roughly one document in twenty. Two caveats keep the claim honest. First, the 16,36416,364 mixed labels are model predictions, not a second full hand audit; gold mixed precision/recall near 0.950.95 on n=461n=461 does not automatically transfer to every corpus row. Second, the 27,628/46027,628/460 audit covers the Telegram heuristic pool, not the entire 331331k file. Even with those limits, the order of magnitude is stable enough to change what one can claim about code-switching prevalence in this domain: loanword-heavy Kazakh was inflating the mixed count, and a document-level loanword-vs-switch rule cuts that inflation down. _if_language:nTFENG =English8 Conclusion False mixed on Kazakh–Russian social text is mostly a labeling problem, not a model problem. Once the guideline treats integrated Russian borrowings as Kazakh and reserves mixed for clause-level switches, the same held-out gold test (n=461n=461) separates the systems cleanly: FastText and raw HeLI sit near 0.700.70 macro-F1, HeLI+windows (language-set adaptation with overlapping sizes 2+32+3) reaches 0.8690.869, Char-3gram NB 0.8800.880, and XLM-R LID v2 0.9660.966. Char-trigram almost never confuses monolingual ru/kz, yet recovers only 70.8%70.8\% of true mixed; the bottleneck is the loanword-vs-switch line, which full-message context closes. At corpus scale the same filter changes the story you can tell about the data. On 331,468331,468 documents, XLM-R LID v2 keeps 16,36416,364 (4.9%4.9\%) as mixed, against a Telegram keyword heuristic that flagged 27,62827,628 rows with only 460460 hand-confirmed switches (≈1.66%≈1.66\% precision). We release the gold LID and SA pools, the HeLI ladder code, and the trained weights. Next steps are a second annotator for IAA, a morphology cue (Apertium) for bare Russian nouns inside Kazakh, and polarity models that run only on the filtered mixed slice rather than on loanword-inflated “mixed” noise. Limitations Annotation is solo; we report no inter-annotator agreement. That is the main weakness of the resource, and several external readers already flagged it. The 16,36416,364 corpus mixed labels are XLM-R predictions, not a full hand audit. Gold mixed P/R near 0.950.95 on n=461n=461 should not be read as a guarantee on every corpus row. Sentiment labels are mostly LLM-drafted then lightly audited. We therefore treat SA metrics as cascade utility after LID, not as a primary claim about human sentiment annotation. Labels are document-level. That flattens token-level gray zones (bare Russian nouns with no Kazakh morphology; short insertional switches that windowed HeLI misses by construction). Informal spelling without _if_script:nTFcyrl =Cyrillic _if_language:nTFRUS =Russianқ/ _if_script:nTFcyrl =Cyrillic _if_language:nTFRUS =Russianұ also pushes short Kazakh spans into neighboring Turkic codes in heliport. Acknowledgments This work began as an individual capstone project at Samsung Innovation Campus. I thank Tommi Jauhiainen for detailed advice on the HeLI ladder (loanword-neutral re-identification, overlapping windows after the strip, and grid search over window size and min_countmin\_count), and for pointing me to heliport and the language-set identification line of work. I also thank Anton Alekseev, Jonathan Washington, Jonathan Dunn, Maite Heredia, and Nikola Ljubešić for framing advice and for pushing the character-trigram baseline. References Baisa and Suchomel (2012) Vít Baisa and Vít Suchomel. 2012. Large corpora for Turkic languages and unsupervised morphological analysis. In Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC’12), pages 28–32. Workshop on Language Resources for Turkic Languages; character-trigram LID used in corpus construction. Burchell et al. (2023) Laurie Burchell, Alexandra Birch, Nikolay Bogoychev, and Kenneth Heafield. 2023. An open dataset and model for language identification. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 865–879, Toronto, Canada. Association for Computational Linguistics. Jauhiainen et al. (2015) Tommi Jauhiainen, Krister Lindén, and Heidi Jauhiainen. 2015. Language set identification in noisy synthetic multilingual documents. In Computational Linguistics and Intelligent Text Processing: 16th International Conference, CICLing 2015, Proceedings, Part I, volume 9041 of Lecture Notes in Computer Science, pages 633–643. Springer. Jauhiainen et al. (2019) Tommi Jauhiainen, Marco Lui, Marcos Zampieri, Timothy Baldwin, and Krister Lindén. 2019. Automatic language identification in texts: A survey. Journal of Artificial Intelligence Research, 65:675–782. Kargaran et al. (2023) Amir Hossein Kargaran, Ayyoob Imani, François Yvon, and Hinrich Schuetze. 2023. GlotLID: Language identification for low-resource languages. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 6155–6218, Singapore. Association for Computational Linguistics. Poplack (1980) Shana Poplack. 1980. Sometimes I’l start a sentence in Spanish y termino en español: Toward a typology of code-switching. Linguistics, 18(7–8):581–618. Wintner et al. (2023) Shuly Wintner, Safaa Shehadi, Yuli Zeira, Doreen Osmelak, and Yuval Nov. 2023. Shared lexical items as triggers of code switching. Transactions of the Association for Computational Linguistics, 11:1471–1484.