Paper deep dive
Beyond Fluency: A Rubric-Based Benchmark for Evaluating Saudi Dialect and Cultural Competence in Large Language Models
Ghassan Al-Sumaidaee, Sajjad Abdoli, Ahmed Rashad, Maxim Legg
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 94%
Last extracted: 9/15/2026, 7:43:34 AM
Summary
This paper introduces a rubric-based benchmark to evaluate Large Language Models (LLMs) on Saudi dialect and cultural competence, addressing the gap where standard benchmarks favor Modern Standard Arabic (MSA). The benchmark consists of 31 expert-authored prompts covering idiomatic, pragmatic, lexical, and cultural phenomena. Four state-of-the-art models (Claude Opus 5, Gemini 3.7, GPT-5.6, and Kimi K3) were evaluated using a two-phase methodology: model-agnostic criterion derivation and model-specific scoring with penalties for errors. Results show that no model exceeded 55% macro-average performance, with 'Ambiguous Framing' being the dominant failure mode (37.3% of errors), indicating that models struggle more with register and pragmatic nuance than with outright hallucination.
Entities (10)
Relation Signals (8)
Saudi Dialect Benchmark → evaluates → Claude Opus 5
confidence 95% · four state-of-the-art systems -- Claude Opus 5... are scored against those criteria
Saudi Dialect Benchmark → evaluates → Gemini 3.7
confidence 95% · four state-of-the-art systems -- ... Gemini 3.7 ... are scored
Saudi Dialect Benchmark → evaluates → GPT-5.6
confidence 95% · four state-of-the-art systems -- ... GPT-5.6 ... are scored
Saudi Dialect Benchmark → evaluates → Kimi K3
confidence 95% · four state-of-the-art systems -- ... and Kimi K3 -- are scored
Ambiguous Framing → isdominantfailuremodefor → Saudi Dialect Benchmark
confidence 93% · Ambiguous Framing is the dominant failure mode (37.3% of errors)
Kimi K3 → hashighestmacroaverage → Saudi Dialect Benchmark
confidence 92% · Kimi K3 53.1... Ranked by macro-average.
GPT-5.6 → hashighestmedianscore → Saudi Dialect Benchmark
confidence 92% · GPT-5.6 attains the highest median (61.5%)
Ghassan Al-Sumaidaee → authored → Saudi Dialect Benchmark
confidence 90% · Ghassan Al-Sumaidaee... Beyond Fluency: A Rubric-Based Benchmark...
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Large language models are increasingly deployed in Arabic-speaking markets, yet standard benchmarks overwhelmingly reward Modern Standard Arabic (MSA) fluency while leaving dialectal and culturally grounded competence unmeasured. This gap is consequential: everyday Arabic is largely dialectal, and dialect encodes social meaning that MSA-centric evaluation cannot capture. We present a rubric-based benchmark for the Saudi dialect, comprising 31 expert-authored prompts spanning idiomatic, pragmatic, lexical, and culturally-embedded phenomena, each paired with an expert-established ground truth. Our methodology separates evaluation into a model-agnostic phase, in which atomic, MECE positive criteria are derived solely from the ground truth, and a model-specific phase, in which four state-of-the-art systems -- Claude Opus 5, Gemini 3.7, GPT-5.6, and Kimi K3 -- are scored against those criteria and penalised for errors they actively introduce. Across 124 model-prompt evaluations we catalogue 466 error instances under a nine-category taxonomy. The four systems cluster within a narrow macro-average band (42.7%-53.1%), with no model exceeding 55% and every model recording at least one negative-scoring prompt, confirming that Saudi dialectal competence remains broadly unsolved. Notably, Ambiguous Framing is the dominant failure mode (37.3% of errors) while outright Hallucination accounts for only 11.2%, indicating that models fail less by stating falsehoods than by distorting register and flattening pragmatic nuance. We further observe a consistency-versus-ceiling trade-off and model-distinctive error signatures. We release the full prompt set, ground truths, and scored rubrics to support reproducible dialectal evaluation.
Tags
Links
- Source: https://arxiv.org/abs/2608.29990v1
- Canonical: https://arxiv.org/abs/2608.29990v1
Trouble viewing inline? Open PDF directly →
Full Text
44,963 characters extracted from source content.
Expand or collapse full text
September 1, 2026 Beyond Fluency: A Rubric-Based Benchmark for Evaluating Saudi Dialect and Cultural Competence in Large Language Models A Reference-Guided, Expert-Authored Evaluation of Four State-of-the-Art Models Ghassan Al-Sumaidaee 1,*,†1 ,Sajjad Abdoli 1,*,†2 ,Ahmed Rashad 1 ,Maxim Legg 1 1 Perle * Equal contribution. † Corresponding authors ghassan.al-sumaidaee@perle.aisajjad@perle.ai ahmed@perle.aimax@perle.ai Large language models are increasingly deployed in Arabic-speaking markets, yet standard benchmarks overwhelmingly reward Modern Standard Arabic (MSA) fluency while leaving dialectal and culturally- grounded competence unmeasured. This gap is consequential: everyday Arabic is largely dialectal, and dialect encodes social meaning that MSA-centric evaluation cannot capture. We present a rubric-based benchmark for the Saudi dialect, comprising 31 expert-authored prompts spanning idiomatic, pragmatic, lexical, and culturally-embedded phenomena, each paired with an expert-established ground truth. Our methodology separates evaluation into a model-agnostic phase, in which atomic, MECE positive criteria are derived solely from the ground truth, and a model-specific phase, in which four state-of-the-art systems—Claude Opus 5, Gemini 3.7, GPT-5.6, and Kimi K3—are scored against those criteria and penalised for errors they actively introduce. Across 124 model-prompt evaluations we catalogue 466 error instances under a nine-category taxonomy. The four systems cluster within a narrow macro-average band (42.7%–53.1%), with no model exceeding 55% and every model recording at least one negative-scoring prompt, confirming that Saudi dialectal competence remains broadly unsolved. Notably, Ambiguous Framing is the dominant failure mode (37.3% of errors) while outright Hallucination accounts for only 11.2%, indicating that models fail less by stating falsehoods than by distorting register and flattening pragmatic nuance. We further observe a consistency-versus-ceiling trade-off and model-distinctive error signatures. We release the full prompt set, ground truths, and scored rubrics to support reproducible dialectal evaluation. Keywords:large language models; Arabic NLP; dialect evaluation; Saudi Arabic; cultural competence; rubric-based benchmarking; error taxonomy; low-resource evaluation. 1Introduction The rapid integration of large language models (LLMs) into consumer and enterprise applications across the Arab world has outpaced our ability to measure whether these systems genuinely understand the language as it is spoken. This reflects a broader challenge in LLM benchmarking: aggregate evaluations can obscure 1 Corresponding author:ghassan.al-sumaidaee@perle.ai; ORCID:0000-0002-5536-0252 2 Corresponding author:sajjad@perle.ai © 2026 Perle. All rights reserved.1 arXiv:2608.29990v1 [cs.CL] 30 Aug 2026 September 1, 2026 important capability gaps across tasks, languages, and linguistic varieties [1,2]. Contemporary Arabic evaluation is dominated by Modern Standard Arabic (MSA): the formal register of news, official documents, and formal education, while culturally and sociolinguistically grounded Arabic knowledge remains much less extensively evaluated [3]. MSA, however, is almost nobody’s mother tongue. The Arabic that people use to greet a neighbour, tease a friend, defuse a quarrel, or ask a favour is dialectal, and it is in dialect that the culturally load-bearing meaning of an utterance most often resides. A model that scores well on MSA comprehension may still fail comprehensively at the everyday communicative tasks for which users actually reach for it. This paper addresses that gap for one high-value variety: the Saudi dialect, together with the cultural knowledge in which it is embedded. Saudi Arabic is not a monolith—it spans Najdi, Hijazi, and other regional varieties—and its expressions frequently encode indirect social meaning: a phrase that is literally a blessing may function as a veiled threat; a word that is literally a colour may be an imperative to hurry; a request that is literally about food may be a demand for total candour. These are precisely the phenomena that expose the difference between a model that has memorised a bilingual dictionary and one that has internalised how a language is actually used. A central difficulty in evaluating such competence is that it resists automatic scoring. There is rarely a single reference string against which a response can be matched, and open-ended model evaluation can be sensitive to both stylistic variation and prompt formulation [ 4,5]. Surface-level metrics such as BLEU or exact match are therefore meaningless for open-ended pragmatic explanation. We adopt a rubric-based methodology in which subject-matter experts (SMEs)—native Saudi speakers with lived cultural knowledge—decompose the correct answer into atomic, independently-checkable criteria, and human scorers evaluate each model response against those criteria. Crucially, our design separates the construction of the evaluation standard from the observation of any model’s behaviour, so that the same rubric can be applied identically to every model under test. 1.1Contributions 1.A Saudi-dialect benchmark.We release 31 expert-authored prompts, each written in authentic Saudi dialect, targeting knowledge that requires local, lived experience rather than textbook or web-searchable facts, together with an SME-validated ground truth for each. 2.A reproducible two-phase rubric methodology.We formalise a model-agnostic, ground-truth- derived positive-criteria construction phase and a model-specific negative-criteria and scoring phase, en- suring that positive criteria are reused identically across all evaluated systems. 3.A nine-category failure taxonomy for dialectal evaluation.We extend a six-category error scheme with categories that emerged as necessary during annotation—most notably Language Mixing—and re- port a full cross-model error analysis over 466 discrete error instances. 4.A comparative study of four state-of-the-art models.We evaluate Claude Opus 5, Gemini 3.7, GPT-5.6, and Kimi K3, and identify model-distinctive error signatures with direct implications for de- ployment in Saudi-facing products. 1.2Why dialectal cultural competence is hard to fake A well-constructed dialectal prompt has a diagnostic property: a model lacking authentic knowledge cannot fall back on fluent paraphrase. It is forced into one of two observable failure behaviours—it either hallucinates a confident but fabricated meaning, or it retreats to a superficially plausible but culturally hollow answer that misses the load-bearing nuance. Both are detectable by an SME and both are captured by our rubric. This is why prompt difficulty is validated before any rubric is built: a prompt that every model answers correctly cannot discriminate between systems and is discarded. Saudi Dialect & Cultural Competence Benchmark2 September 1, 2026 2Background and Motivation 2.1Related benchmarking work Recent LLM evaluation has increasingly moved from broad aggregate testing toward capability-specific benchmarks. Examples include long-context modelling [6], document reading systems [7], and the interpre- tation of real-world clinical text [8]. These studies demonstrate the value of evaluation sets designed around the actual failure conditions of a particular deployment domain. Complementary benchmarks from our group apply the same capability-specific philosophy across other modalities and language settings, including text-to-image generation [9], sound-source identification [10], and multilingual code-switching speech recognition [11]. The present study focuses this approach on Saudi dialect and culture, extending earlier evaluation of Arabic cultural and sociolinguistic knowledge [ 3] with a shared-positive, model-specific-negative rubric design and a detailed error taxonomy. 2.2The MSA bias in Arabic evaluation Arabic is a canonical example of diglossia: a high (formal, written) variety coexists with a family of low (spoken, regional) varieties that differ substantially in phonology, morphology, and lexicon. Because annotated resources are far more abundant for MSA, evaluation practice has gravitated toward it. Related evaluations of Arabic cultural knowledge and multilingual code-switching likewise show why performance on formal or monolingual inputs cannot be assumed to transfer to socially situated language use [3,11]. The result is a systematic blind spot: models can appear competent in Arabic while being unable to correctly interpret common spoken expressions. For applications ranging from customer support to conversational assistants, this blind spot is exactly where user experience is won or lost. 2.3Cultural knowledge as an evaluation target Beyond vocabulary, dialect carries cultural knowledge transmitted through lived experience rather than formal instruction—proverbs and their origins, etiquette for answering a call, indirect conflict-resolution norms, and material-culture references such as traditional footwear. This knowledge is difficult to acquire from web text at scale and is therefore an especially demanding test of whether a model has genuinely absorbed a community’s communicative practices. Our benchmark deliberately concentrates on this stratum. 2.4Rubric-based human evaluation When a task has no single reference answer, decomposed rubric scoring offers a disciplined alternative to holistic rating. By breaking the ground truth into atomic criteria, each independently marked pass or fail, rubric scoring improves inter-rater reliability, localises exactly where a model succeeds or fails, and yields interpretable, actionable diagnostics rather than an opaque scalar. Our methodology operationalises two principles—atomicity and MECE coverage—to make this decomposition rigorous and repeatable. 3Benchmark Construction The benchmark comprises 31 prompts authored by native Saudi SMEs. Each prompt satisfies a set of design constraints intended to maximise diagnostic value, and each is classified along two axes—evaluation scope and linguistic phenomenon—that structure the later analysis. 3.1Prompt design principles Every prompt was required to satisfy the following criteria, enforced at authoring time: •Local-knowledge requirement.The correct answer should require lived, in-community experience; anything resolvable by a routine web search or a general Arabic textbook is too easy to be discriminative. Saudi Dialect & Cultural Competence Benchmark3 September 1, 2026 •Written in the target dialect.Prompts are composed in Saudi dialect (Najdi/Hijazi as appropriate), not MSA, so that register itself is part of what is tested and the model is not cued toward a formal response. •Conversational and natural.Prompts read like a real person asking a genuine question, testing whether the model can engage in-register rather than defaulting to an encyclopaedic style. •Multi-layered where possible.Prompts invite a layered answer—meaning, usage, and cultural grounding—so a single response can be scored along several dimensions. •Non-leading.Prompts avoid embedding the answer or vocabulary the model can echo back. •Culturally sensitive, not offensive.Sensitive themes may appear but are framed respectfully; the goal is to test knowledge, not to provoke harmful output. 3.2Scope: cultural vs. linguistic Each prompt is assigned a single primary scope. Linguistic prompts test identification, use, and explanation of dialect-specific vocabulary, idioms, proverbs, and speech patterns. Cultural prompts test knowledge of traditions, social norms, material culture, and collective memory. A prompt that touches both is assigned to its dominant dimension rather than mixed. In the present set the large majority of prompts are linguistic, reflecting the density of idiomatic and pragmatic traps in spoken Saudi Arabic, with a minority classified as cultural (for example, the identification of traditional men’s footwear, and the reading of indirect post-conflict speech). 3.3Difficulty validation Before any rubric was constructed, each candidate prompt was submitted to a model in a clean, memory- disabled session as a quick discriminativeness check. Prompts that were answered perfectly were revised to target deeper local knowledge or discarded. This validation output was never used for scoring; it served only to confirm that the prompt could expose a knowledge gap. This step protects annotation effort from being spent on non-discriminative items. 3.4Illustrative prompts Table1gives representative examples spanning the phenomena in the benchmark, with the load-bearing element that a correct answer must capture and the literal misreading that constitutes the characteristic trap. Table 1:Representative benchmark prompts, the culturally load-bearing meaning each targets, and the literal misreading that defines its failure mode. Prompt (Arabic)PhenomenonIntended meaningCharacteristic trap اལ༳ ԵឝԳسܫ وڕڹڿIdiom of gratitudeA high compliment / thanks rooted in honour Literal “may God whiten your face” (skin colour) ศตأPragmatic reply“Consider it done / at your service” Literal “rejoice / good news” ௌே٩ن ݅ؽכEvaluative lexemeWell-mannered + socially intelligent Flattened to generic “good/nice” أزرق ا۹غاٴჱუ وؗשلDialectal verb“Dash to the shop and come back” Literal “blue the grocery” ݝاٴڹ واߓਊاٴڹIdiomActs first, deals with fallout later Literal “hit it and catch it” ལཀ জগسכ اঔঐݮ ؊ڶ ᇖᆱرة ا۹ݐاٴ؊൘ഽProverbDon’t sell to those who already have it Read flatly, missing the metaphor Saudi Dialect & Cultural Competence Benchmark4 September 1, 2026 4Evaluation Methodology Our methodology is deliberately split into two phases so that the evaluation standard is fixed before any model output is observed. This separation is the mechanism that makes cross-model comparison fair: every model is graded against an identical, pre-committed set of positive criteria. 4.1Two-phase workflow Phase A (pre-output, model-agnostic).The SME establishes the ground truth, decomposes it into atomic positive criteria, assigns each a dimension and weight, and finalises the prompt. No model output is consulted. The deliverable is a reusable prompt-plus-positive-rubric pair. Phase B (post-output, model-specific).For each model, the response is generated in a clean session, scored pass/fail against every positive criterion, and then examined for errors it actively introduced. Each such error becomes a negative criterion carrying a penalty. Because different models make different mistakes, the negative criteria are model-specific, whereas the positive criteria are shared. 4.2Criterion design: atomicity and MECE Two principles govern criterion construction. Atomicity requires that each criterion test exactly one thing, so that a response cannot be simultaneously right and wrong under a single criterion. MECE (mutually exclusive, collectively exhaustive) requires that no two criteria overlap and that together they cover the complete ground truth. Together these principles ensure that the total score is a faithful, non-double-counted reflection of how much of the correct answer a model produced. 4.3Scoring fields Each criterion—positive or negative—carries the following fields: •Dimension:Accuracy, Context Awareness, Completeness, or Communication Quality. •Objectivity:whether the criterion has a clear right/wrong answer or requires judgement (e.g., natural- ness of register). •Explicitness:whether the target must be stated explicitly or may be conveyed implicitly. •Weight:the point value; for positive criteria this equals the maximum score, for negative criteria it is negative. •Score type and max score:positive criteria are scored against their weight; negative criteria have a maximum of 0, since the best outcome is that the error is absent. •Result and justification:pass/fail (or error-present/absent), with a justification referencing specific response content; failures and negative criteria carry a bracketed error-type tag. 4.4Scoring model and equations Let a prompt have positive criteria indexed byiwith weightsw i , and let the model under evaluation trigger negative criteria indexed byjwith penaltiesp j (p j >0denotes the magnitude deducted). Define the pass indicatorx i ∈0,1and the error indicatore j ∈0,1. The maximum attainable score for a prompt depends only on its positive criteria: S max = ∑ i w i .(1) The points a model earns from positive criteria, and the penalty it incurs from its active errors, are: S pos = ∑ i w i x i , (2) P= ∑ j p j e j .(3) Saudi Dialect & Cultural Competence Benchmark5 September 1, 2026 The prompt-level model score and its normalised percentage are then: S=S pos −P,(4) Score%= 100× S S max .(5) Because penalties are not bounded byS pos , a prompt-level score can be negative. This is intentional: a response that supplies a wrong answer and compounds it with fabrications is actively misleading, not merely unhelpful, and the scoring model reflects that a negative outcome is a meaningful and valid result. A model’s aggregate score over the benchmark ofNprompts is the mean of its per-prompt percentages, Score% agg = 1 N ∑ k Score% k .(6) 4.5Controlled generation conditions All model outputs were captured under uniform conditions to prevent contamination: memory and con- versation history disabled, one prompt per session, the first response taken verbatim with no re-prompting, and formatting preserved exactly as emitted. These controls ensure that each response reflects the model’s independent behaviour on the prompt rather than an artefact of dialogue history. 4.6Workflow summary Table 2:The six-step, two-phase evaluation workflow. Steps 1–4 are performed once per prompt; steps 5–6 are repeated for every model. Step Phase ActionDeliverable 1ADefine scopeCultural or linguistic (never mixed) 2ADraft promptDialect-native, locally-rooted, multi-layered 3AValidate difficultyConfirm the prompt is discriminative 4ABuild positive criteriaAtomic, MECE, weighted (model-agnostic) 5BGenerate outputClean session, verbatim capture 6BScore & add negativesPass/fail, penalties, justifications, final score 5Failure Taxonomy Every failed positive criterion and every negative criterion is tagged with an error type. We began from a six-category scheme and extended it during annotation as recurring behaviours emerged that the original categories did not cleanly capture. The final nine-category taxonomy is given in Table 3. Saudi Dialect & Cultural Competence Benchmark6 September 1, 2026 Table 3:The nine-category failure taxonomy. The final two categories—Formatting and Language Mixing—were added during annotation; Instruction Violation was promoted to a first-class category to capture explicit-request failures distinct from mere omission. Error typeDefinition HallucinationFabricates information: invents a non-existent term, a false etymology, or a fictional origin, or states something confidently and incorrectly. OmissionFails to mention a critical element the correct answer requires; the point is not addressed at all. Ambiguous FramingPresents information misleadingly or unclearly without being outright false—including wrong register, inappropriate mixing of formal and informal, or framing that distorts significance. Cultural MisattributionAttributes a term, practice, or tradition to the wrong region, community, or referent. Irrelevant AdditionAdds unsolicited content that does not serve the question. Rendering FailureProduces garbled text, invalid characters, or broken Arabic script—a technical rather than a knowledge failure. FormattingPresentation defects that impede readability or violate the expected response shape (e.g., inappropriate structure or markup artefacts). Instruction ViolationFails to do what the prompt explicitly asked—e.g., answering a different question, asking the user a question instead of answering, or ignoring an explicit format request. Language MixingIntroduces the wrong language or variety inappropriately—e.g., drifting into MSA or English where Saudi dialect was required, or code-switching that breaks register. Two taxonomy notes bear on interpretation. First, Ambiguous Framing is intentionally broad and is the category most reliant on evaluator judgement; because it absorbs register and nuance-distortion failures that are central to dialectal competence, it is both the most informative and the most in need of inter-rater scrutiny. Second, Language Mixing overlaps conceptually with Ambiguous Framing (a wrong-register response can be seen as both); we treat an inappropriate switch of language or variety as Language Mixing and reserve Ambiguous Framing for distortions within the correct variety, but this boundary is a modelling choice that future work may refine. 6Quantitative Performance We complement the error analysis with the numeric rubric scores. Each prompt yields a per-model per- centage via Eq. (5); a model’s aggregate is summarised both as a macro-average (the unweighted mean of its 31 per-prompt percentages, Eq. (6)) and a micro-average (total points earned divided by total points available, which weights each prompt by its maximum score). Reporting both guards against a small number of high-max prompts dominating the headline number. 6.1Leaderboard Table4presents the leaderboard. The four systems cluster within a narrow∼10-point macro-average band (42.7%–53.1%), underscoring that Saudi-dialect cultural competence remains an unsolved and broadly difficult task: no model reaches 55%, and every model records at least one negative-scoring prompt where active errors outweighed the points earned. Saudi Dialect & Cultural Competence Benchmark7 September 1, 2026 Table 4:Leaderboard over 31 prompts. Macro % is the mean of per-prompt percentages; Micro % weights by prompt max score; SD is the standard deviation of per-prompt percentages; “≥50%” counts prompts scoring at least half; “Neg.” counts prompts with a negative score. Ranked by macro-average. ModelMacro % SD Micro % Median Min Max≥50% Neg. Kimi K353.122.752.958.3−21.280.622/312 GPT-5.652.6 33.754.361.5−44.888.9 24/313 Gemini 3.750.716.651.554.2−6.279.419/311 Claude Opus 542.7 29.243.055.2−24.276.5 20/314 Figure 1:Mean score per model with standard-deviation error bars. The dashed line marks the 50% threshold. Differ- ences among the top three models are small relative to their spread. 6.2Consistency versus peak performance Aggregate means conceal a sharp difference in reliability. Figure2shows the full per-prompt distributions. GPT-5.6 attains the highest median (61.5%) and the highest single-prompt score (88.9%) but also the widest dispersion (SD 33.7) and the worst single failure (−44.8% on SAU-4). Gemini 3.7, by contrast, is the most consistent system (SD 16.6) with only one negative prompt, despite a lower ceiling. Claude Opus 5 combines a moderate median with heavy left-tail risk (four negative prompts, SD 29.2). This consistency–ceiling trade- off is decision-relevant: a deployment that cannot tolerate occasional severe failures may prefer the tighter distribution even at a lower mean. Saudi Dialect & Cultural Competence Benchmark8 September 1, 2026 Figure 2:Per-prompt score distributions (box plots with individual prompts overlaid). Solid line at 0%, dashed line at 50%. GPT-5.6 has the highest median but the greatest spread; Gemini 3.7 is tightest. 6.3Macro versus micro aggregation Figure 3contrasts the two aggregation schemes. The rankings are stable across schemes and the macro/micro gap is small for every model (at most∼1.6 points), indicating that no model’s standing is an artefact of a few high-weight prompts. The largest divergence is for GPT-5.6, whose micro-average slightly exceeds its macro-average because it performed well on several high-max prompts (e.g., SAU-25, max 58). Figure 3:Macro- versus micro-average scores. Ranking is invariant to the choice of aggregation; the small gaps confirm robustness. Saudi Dialect & Cultural Competence Benchmark9 September 1, 2026 6.4Item difficulty: which prompts were hardest Averaging each prompt across the four models yields an item-difficulty ranking. Tables5and6list the five hardest and five easiest prompts. The hardest items concentrate on highly localised lexis and metaphor where a literal misreading is both tempting and catastrophic—for example SAU-11 (the Hijazi metaphor “kubb al-‘asha”) and SAU-27 (the nuance-laden “dalakh”). The easiest items are common greetings and high- frequency expressions. The per-prompt heatmap (Figure4) visualises this structure: vertical cool bands mark prompts that defeated all models, while broad warm rows mark models that were broadly competent. Table 5:The five hardest prompts by cross-model mean score. Per-model values are percentages. Prompt Cross-model mean Gemini Opus 5 GPT-5.6 Kimi K3 SAU-11−4.5%45−24−18−21 SAU-1313.3%−6569−6 SAU-2613.5%54−17−1733 SAU-413.8%55−17−4562 SAU-2722.0%280060 Table 6:The five easiest prompts by cross-model mean score. Prompt Cross-model mean Gemini Opus 5 GPT-5.6 Kimi K3 SAU-2070.8%78588958 SAU-870.3%56757575 SAU-1766.2%65766262 SAU-365.9%61648852 SAU-3165.4%79565671 Figure 4:Per-prompt score heatmap (models×31 prompts). Green is high, red is low/negative. Vertical cool bands (e.g., SAU-11) are prompts that were hard for every model; warmer rows indicate broadly stronger models. 6.5Score–error relationship The quantitative scores and the error taxonomy are two views of the same evaluations. The negative-scoring prompts are precisely those where high-penalty errors—typically Hallucination or Cultural Misattribution on a load-bearing criterion—overwhelmed the positive points earned. This is why total error count (Section 7) and mean score do not rank the models identically: Claude Opus 5 has the highest error instance count and also the lowest macro-average, but GPT-5.6, with the fewest errors, does not top the leaderboard because its errors, when they occur, tend to fall on high-max prompts and include severe single failures. Error frequency and error severity are distinct axes, and a complete evaluation requires both. Saudi Dialect & Cultural Competence Benchmark10 September 1, 2026 7Error Analysis We report results over 124 model-prompt evaluations (31 prompts×4 models). Across these, human scorers recorded 466 discrete error instances. This section first presents the aggregate error load, then the cross-model comparison, and finally the model-distinctive error signatures. 7.1Aggregate error load Table7gives the per-model total error counts. GPT-5.6 accrued the fewest total error instances (103) and Claude Opus 5 the most (132), with Gemini 3.7 (117) and Kimi K3 (114) in between. Total error count is a coarse proxy—a single high-weight error can matter more than several minor ones—but it establishes the overall reliability ordering at the instance level. Table 7:Total error instances per model across 31 prompts, with each model’s share of the 466-instance corpus. ModelTotal errors Share of all errors GPT-5.610322.1% Kimi K311424.5% Gemini 3.711725.1% Claude Opus 513228.3% Figure 5:Total error instances by model. GPT-5.6 exhibits the lowest instance-level error load; Claude Opus 5 the highest, driven substantially by non-accuracy categories (Section7.3). 7.2The dominant failure mode is framing, not fabrication The most consequential finding of the study is visible in the aggregate distribution (Figure 6, Table8): the modal failure is not the fabrication of false facts but the subtle mishandling of framing and completeness. Ambiguous Framing alone accounts for 174 of 466 instances (37.3%). Omission (84; 18.0%) and Irrelevant Addition (63; 13.5%) follow. Hallucination—often assumed to be the primary risk of LLMs on knowledge- intensive tasks—is only the fourth most common category (52; 11.2%). Saudi Dialect & Cultural Competence Benchmark11 September 1, 2026 Table 8:Aggregate failure-type frequency across all four models (466 total instances), sorted by frequency. Error typeTotal % of all errors Ambiguous Framing17437.3% Omission8418.0% Irrelevant Addition6313.5% Hallucination5211.2% Formatting4710.1% Language Mixing214.5% Cultural Misattribution163.4% Instruction Violation81.7% Rendering Failure10.2% Figure 6:Aggregate failure-type frequency across all models. The concentration in Ambiguous Framing, Omission, and Irrelevant Addition indicates that current models fail on Saudi-dialect tasks primarily through nuance distortion and calibration of scope, not through overt factual fabrication. The interpretive weight of this result is substantial. On dialectal-cultural tasks, the failures that matter are those a fluent-sounding model is most likely to commit unnoticed: giving a technically-related but register- wrong rendering, omitting the one culturally load-bearing element, or burying the answer under unsolicited elaboration. These are exactly the errors that automated fluency metrics cannot detect and that only reference-guided human scoring surfaces. 7.3Cross-model comparison by error type Figure 7and Table9break the counts down by model and error type. Several patterns stand out. Ambigu- ous Framing is the leading category for every model but is most concentrated in Gemini 3.7 (54 instances; 46.2% of its errors). GPT-5.6 has by far the lowest Hallucination count (6) yet the highest Formatting load (17). Claude Opus 5 shows the most even spread and the highest Irrelevant Addition count (22), together with the highest Cultural Misattribution (7) and Instruction Violation (5) counts. Kimi K3 resembles Gemini in its framing-heavy profile but with more Irrelevant Addition. Saudi Dialect & Cultural Competence Benchmark12 September 1, 2026 Figure 7:Failure-type distribution across models (grouped). Bars are absolute instance counts. Note the model-specific concentrations: Gemini in framing, GPT-5.6 in formatting with minimal hallucination, and Opus 5 in irrelevant additions. Table 9:Full cross-tabulation of failure-type counts by model. The final row gives per-model totals.LANG_MIXING labels and the misspelled Instruction-Violation variants in the source sheets were normalised to the canonical categories of Table3. Error typeGemini 3.7 Opus 5 GPT-5.6 Kimi K3 Hallucination1815613 Omission22212417 Ambiguous Framing54393744 Irrelevant Addition9221220 Rendering Failure0100 Cultural Misattribution2761 Formatting2141714 Instruction Violation2510 Language Mixing8805 Total117132103114 Saudi Dialect & Cultural Competence Benchmark13 September 1, 2026 Figure 8:Error-type intensity heatmap. Colour encodes count. The heatmap makes the model signatures legible at a glance: the framing column dominates, GPT-5.6’s near-empty hallucination cell contrasts with its dark formatting cell, and Opus 5’s irrelevant-addition cell is the darkest in its row-group. 7.4Normalised error signatures Absolute counts confound a model’s overall error load with the shape of its errors. Figure9therefore shows each model’s composition as a share of its own error total, and Figure10presents the same information as normalised profiles. The composition view isolates behavioural tendencies independent of how many errors a model made in total. Saudi Dialect & Cultural Competence Benchmark14 September 1, 2026 Figure 9:Within-model error composition (each bar sums to 100%). GPT-5.6 devotes the largest share of its errors to Omission and Formatting and the smallest to Hallucination; Gemini devotes nearly half its errors to Ambiguous Framing; Opus 5 and Kimi K3 carry the heaviest Irrelevant-Addition shares. Saudi Dialect & Cultural Competence Benchmark15 September 1, 2026 Figure 10:Normalised error profiles. The radial axis is the percentage of a model’s own errors falling in each category. The near-collapse of all models toward the framing/omission axes, and their divergence on irrelevant addition, hallucina- tion, and formatting, jointly define their error signatures. Read together, four signatures emerge: •Gemini 3.7—the framer.Nearly half of its errors are Ambiguous Framing, with comparatively little formatting or irrelevant-addition noise. Its failures are concentrated in how it renders register and nuance. •GPT-5.6—the literalist minimiser.The lowest hallucination share (5.8%) and zero language-mixing, but the highest formatting share and the highest omission share—it tends to under-commit and mis- present rather than invent. •Claude Opus 5—the over-elaborator.The broadest error spread and the highest irrelevant-addition, cultural-misattribution, and instruction-violation loads—it says more, and some of the extra content misfires. •Kimi K3—the framer-elaborator hybrid.A framing-heavy profile like Gemini’s but with substan- tially more irrelevant addition, and the lowest omission share of the four. 8Discussion 8.1Fluency masks incompetence The predominance of Ambiguous Framing over Hallucination supports a central thesis: on dialectal-cultural tasks, contemporary models are fluent enough to avoid obvious fabrication but not competent enough to reliably preserve register and pragmatic force. A model that answers a Saudi idiom with a grammatically- perfect MSA gloss has not hallucinated—it has mis-framed—and only an evaluator attuned to register will Saudi Dialect & Cultural Competence Benchmark16 September 1, 2026 catch it. This is precisely the class of error that MSA-centric or automatic evaluation is blind to, and it is the class our methodology is built to expose. 8.2Error profiles are deployment-relevant The model signatures have direct product implications. A framing-dominated model may be acceptable where approximate meaning suffices but risky in high-context social interactions; an over-elaborating model may frustrate users seeking a terse dialectal reply and may introduce misattributions through its extra content; a minimiser with low hallucination but high omission may be safest where factual reliability is paramount but weakest where completeness matters. Selecting a model for a Saudi-facing product is therefore not a matter of a single leaderboard number but of matching an error signature to a use case. 8.3The value of negative criteria Positive criteria alone would credit a model for the correct parts of an answer while ignoring the harm done by the incorrect parts. Negative criteria close this gap by penalising actively-introduced errors, and—because they are model-specific—they document exactly how each model failed on each prompt. This asymmetry (shared positives, bespoke negatives) is what lets the benchmark be simultaneously fair across models and diagnostic within each model. 8.4Limitations •Category subjectivity.Ambiguous Framing, the largest category, is also the most judgement-dependent. Although atomic criteria and per-item justifications constrain scorer discretion, the concentration of errors in this category makes formal inter-rater reliability measurement a priority for future releases. •Scale.Thirty-one prompts is a focused, expert-curated set rather than a large-scale corpus; it is de- signed for depth and discriminativeness, not statistical breadth. Aggregate percentages should be read as indicative signatures, not precise population estimates. •Taxonomy evolution.Categories added mid-study (notably Language Mixing) were applied consistently once introduced but were, by construction, not available for the earliest annotations; the boundary between Language Mixing and Ambiguous Framing remains a modelling choice. •Single-turn, first-response protocol.We evaluate a model’s first response with no follow-up. This mirrors many real interactions but does not capture a model’s ability to self-correct when prompted, which may matter for some deployments. •Weighting is expert-set.Criterion weights encode SME judgement about relative importance. They are transparent and reusable, but different stakeholders might weight dimensions differently; the released rubrics make such re-weighting straightforward. 8.5Threats to validity and mitigations The principal internal-validity threat is scorer bias in subjective categories, mitigated by the requirement that every judgement cite specific response content and by the pre-commitment of positive criteria before output is seen. The principal external-validity threat is dataset specificity to Saudi Arabic; the methodology, however, is variety-agnostic and has been applied by the group to other Arabic dialects, suggesting it transfers. Construct validity is supported by the difficulty-validation step, which ensures prompts measure genuine competence gaps rather than trivial knowledge. 9Conclusion We introduced a rubric-based benchmark for Saudi-dialect and cultural competence in large language mod- els, built on a two-phase methodology that fixes a shared, ground-truth-derived evaluation standard before any model output is observed and then penalises each model for the specific errors it introduces. Evaluating Saudi Dialect & Cultural Competence Benchmark17 September 1, 2026 four state-of-the-art systems over 31 expert-authored prompts and 466 catalogued error instances, we found that the dominant failure mode on these tasks is not fabrication but Ambiguous Framing—the subtle distor- tion of register and nuance—followed by omission of load-bearing elements and unsolicited over-elaboration. Each model exhibited a distinctive error signature with concrete deployment implications. These findings argue that evaluating Arabic LLMs for real-world use requires reference-guided human rubrics attuned to dialect and culture, not MSA-centric or purely automatic metrics. We release the prompts, ground truths, and scored rubrics to enable reproducible, dialect-aware evaluation and to invite extension to further Arabic varieties. 9.1Future work •Publish inter-rater reliability statistics (e.g., Cohen’s/Fleiss’ kappa) on a re-scored subset, with particular attention to the Ambiguous Framing category. •Expand the prompt set and add per-prompt score distributions to complement the error analysis with calibrated accuracy estimates. •Extend the benchmark to additional Saudi sub-dialects (Hijazi, Southern, Eastern) and to further MENA varieties using the same methodology. •Investigate targeted mitigations—dialect-aware fine-tuning and register-sensitive decoding—and re- evaluate against the frozen rubric to measure genuine improvement. 10Reproducibility and Ethics Statement All prompts were authored by native Saudi subject-matter experts and validated for difficulty before rubric construction. Model outputs were collected under uniform, memory-disabled, single-turn conditions with verbatim capture. Positive criteria, ground truths, and the full scored rubrics (including model-specific negative criteria and justifications) are released with this work to support independent replication and re- scoring. Prompts touching sensitive cultural themes were framed respectfully and reviewed to avoid harmful content; the benchmark is intended for evaluation and improvement of model cultural competence, not for producing or amplifying stereotypes. References [1]Miro Hodak, David Ellison, Chris Van Buren, Xiaotong Jiang, and Ajay Dholakia. Benchmarking large language models: Opportunities and challenges. InTechnology Conference on Performance Evaluation and Benchmarking, pages 77–89. Springer, 2023. [2]Yanzhu Guo, Guokan Shang, and Chloé Clavel. Benchmarking linguistic diversity of large language models.Transactions of the Association for Computational Linguistics, 13:1507–1526, 2025. [3]Sajjad Abdoli, Ghassan Al-Sumaidaee, Ahmad ElShiekh, Clayton W. Taylor, and Ahmed Rashad. Bench- marking frontier LLMs on arabic cultural and sociolinguistic knowledge: A cross-evaluation framework with human SME ground truth.arXiv preprint arXiv:2607.00139, 2026. [4]Jiří Milička, Anna Marklová, and Václav Cvrček. Benchmark of stylistic variation in LLM-generated texts.arXiv preprint arXiv:2509.10179, 2025. [5]Amirhossein Razavi, Mina Soltangheis, Negar Arabzadeh, Sara Salamat, Morteza Zihayat, and Ebrahim Bagheri. Benchmarking prompt sensitivity in large language models. InEuropean Conference on Infor- mation Retrieval, pages 303–313. Springer, 2025. [6]Zican Dong, Tianyi Tang, Junyi Li, Wayne Xin Zhao, and Ji-Rong Wen. Bamboo: A comprehensive benchmark for evaluating long text modeling capacities of large language models. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 2086–2099, 2024. Saudi Dialect & Cultural Competence Benchmark18 September 1, 2026 [7]Anni Zou, Wenhao Yu, Hongming Zhang, Kaixin Ma, Deng Cai, Zhuosheng Zhang, Hai Zhao, and Dong Yu. DocBench: A benchmark for evaluating LLM-based document reading systems. InProceedings of the 4th International Workshop on Knowledge-Augmented Methods for Natural Language Processing, pages 359–373, 2025. [8]Jiageng Wu, Bowen Gu, Ren Zhou, Kevin Xie, Doug Snyder, Yixing Jiang, Valentina Carducci, Richard Wyss, Rishi J Desai, Emily Alsentzer, et al. BRIDGE: Benchmarking large language models for under- standing real-world clinical practice texts.Nature Biomedical Engineering, pages 1–16, 2026. [9]Sajjad Abdoli, Ghassan Al-Sumaidaee, and Ahmed Rashad. Benchmarking frontier text-to-image models on image-description prompts.arXiv preprint arXiv:2608.14976, 2026. [10]Sajjad Abdoli, Ghassan Al-Sumaidaee, Ahmad ElShiekh, and Ahmed Rashad. Can foundation models hear what made that sound? a tiered benchmark of audio-language models and traditional classifiers for closed-set sound source identification.arXiv preprint arXiv:2608.02397, 2026. [11]Sajjad Abdoli, Ghassan Al-Sumaidaee, Clayton W. Taylor, Ahmad ElShiekh, and Ahmed Rashad. Bench- marking commercial ASR systems on code-switching speech: Arabic, persian, and german.arXiv preprint arXiv:2605.19069, 2026. Saudi Dialect & Cultural Competence Benchmark19 September 1, 2026 Appendix ASaudi Dialect Benchmark Prompts This appendix lists the complete set of 31 expert-authored prompts used in the benchmark. Prompt wording is reproduced verbatim. Table A1:Complete Saudi dialect benchmark prompt set. Prompt IDPrompt LANG-MENA-SAU-1۹ݮ ܲٔ؊ܗ ݅שݮدي ٨ᇖᇃ ``اལ༳ ԵឝԳسܫ وڕڹڿ′، وش ؊اٴܷٔ ؑ۹ܵغݚ؟ LANG-MENA-SAU-2وش ܇שෟ ``أঔঐ ′ศต أٔ ؊ݠۿ ܇ؽڿ ઠݮي ุ་؟ LANG-MENA-SAU-3وش ؊ש ۹ݮ وܲوٴ ܲٔ؊ߚ߱ و٨ۿ ``واԵឝԳ إؔڲ ݅ؽכ′؟ LANG-MENA-SAU-4أڷௌே ً، اڷལܽᇖᇃ ܇ש߀ߐ܇ت ע܌ ا૯ેؑඕආ ؊ڲ LANG-MENA-SAU-5ඟܭ وڷݮ را็ ٨ل ᇖᇃ ``؊ܷ૯ે ૯ે!′. ڷ۳ اঔঐوٴܦوض أرؗح؟ ଢ଼ ඟؔاٴ ܇כ LANG-MENA-SAU-6وش ؊اٴܷٔ ܲٔ؊ߚ߱ ঔঐ ؊اٴݮل: ``؊ رل و݅כ ܲٔرك، ޟى ݝح اߓਊݠ ཛྷ༺ؽ و܇ ܲر إজগ اߓ૯ેᚱਉ.′ ؑשٔ ܇ ኟቨو݈ؽ؟ LANG-MENA-SAU-7اศ་حᇖᇃ ܇ق۳ ``ལཀ জগسכ اঔঐݮ ؊ڶ ᇖᆱرة ا۹ݐاٴ؊൘ഽ′ وۑسٞ أ٨ٔر أ݅ـٔ܇ڲ ᇖᆱ اླཱྀ૭۹ LANG-MENA-SAU-8ؔدಕ ܲٔ؊ߚ߱ ؑᅝᅎڲ ورد ॒ु ؑـ ``༈ َྂ′. وش ؊اٴܷٔ؟ LANG-MENA-SAU-9ඟغرك′؟ ଢ଼ ۑسٞ أرد ච ौुسڶ ``اԵឝԳ ڑسڲ ݈߀ߐؔڿ LANG-MENA-SAU-10ඟܭ ܇܌ اጷጢ໙ز ؊اٴݮل ``ز܇܌ ٩ܦ౿ۣܺ ޟرع ٩ݮل ؊ݠۿכ ٔس′. وش ܇שෟ ڷঔঐق۳؟ ଢ଼ ᅝᅎש LANG-MENA-SAU-11أؑݮي ٨ᇖᇃ ``ۑ ا۹שݓء′. وش ၨထن ؊اٴܷٔ، وۑسٞ أؗש܇۳ ܇כ اঔঐݮܰݮع؟ LANG-MENA-SAU-12؊ ڷᅝᅜ ،ௌேۣ܌ ؗاٴݮᇖᇃ وش ܇שෟ إذا أٔ ٨জগ`` ᇖᇃ ؊ ݈سٿ، ઠݮ٨ڹ ॒ु اಕ౿؟′؟ LANG-MENA-SAU-13وش اุ༖۹ ا౽ ॒ጋካݮ٨כ أؔڲ మఓ௱ۿس ܇ݐوٴڹ۳ ܇כ ذا اߓݮ؟ LANG-MENA-SAU-14ޟڕ܄ ္ထ ᇖᇃڶ ``܇ศ༥٨تඤோካ ′ۿ૭ેي. LANG-MENA-SAU-15ز܇سၨထ ॒ुن ܇שܷ وأٔ ٨ჭუ: ``݈ٞ، ۹ݮ إᇖᆇ ܇ၨ၁ؔڿ ၨထن ޙدኟቨ ݈ݮي, أޙدڷ ؊رل...′. وش ؊اٴܷٔ ؑـ ``ޙدኟቨ′ ڷؽ، وۑسٞ اঔঐوٴܦوض ޒد؟ LANG-MENA-SAU-16ۑسٞ اঔঐوٴܦوض أرد إذا أٔ ٨ᇖᇃ ``وش ߀ߐ܇ڿ؟ עݐك ૯ેᚱඟ؟′؟ LANG-MENA-SAU-17ؗݮᇖᆇ ݝ۹כ ܇܌ اঔঐݐཀྵݓߛ߫ وܲٔ؊ܗ ݅שݮدي أر݅۳ ᇖᇃ: ``وراك ܇ ـؽ إؔڿ ۑؽ ؑঔঐݐཀྵݓߛ߫؟′. ۑسٞ أرد ۿسڲ؟ LANG-MENA-SAU-18ڢઠ জগ ...ّ॒ु ಕو܇ رد ڿ္ထ܇܌ أ܇ܺ وأؔ أ`` :ჱუ݅ආڷ ᇖᆱ ′ّ॒ु ڢઠ`` وش ؗש ᇖᇃ حศตก ۣ܌ᅝᅜ ؟′اიႮ ّ॒ु LANG-MENA-SAU-19ޟڕ܄ ᇖᇃ ڷჱჩযদ ا۹ؽٔ؊ڶ: ``؊ݮم ᅝᅎש ا۹שހބ ا۹ܷٔز ܲڿّ ؑݮᇖᇇ و܇ ע٩৹ وش أุ༓ཀྵ′. LANG-MENA-SAU-20وش ا۹وٴܦق ൘ഽؑ ``أؑ′، ``ؑפس′، و``وِدي′ ᇖᆱ اካڹڶ ا۹ݐשݮد؊ڶ؟ LANG-MENA-SAU-21۹سڲ ܲٔ؊ߚ߱ ઓݮي იႮا؟ ``أر݅ۿ ჭუ أۑ૯ ܇܌ ৹৫ة، ۹ـۣؽڲ ؊ݠاٴڹ ݅سوٴݮن. وش وܰשڲ؟′ LANG-MENA-SAU-22وش ؊اٴܷٔ ا۹ݓڡܭ إذا ٨ዩዕઠ জগ`` ᇖᇃ ڑߚ߱′؟ LANG-MENA-SAU-23وش ؊ݠۿ ܇ أڐݮي ঔঐ ؊اٴݮل ``؊ ௌே٩ن أزرق ا۹غاٴჱუ وؗשل′؟ LANG-MENA-SAU-24ඟܭ؟ ଢ଼ وش ܇שෟ ``أ٨ۿשڲ עঔঐ ′እ၁౿ ބ ע܌ LANG-MENA-SAU-25ᅝᅜۣ܌ ศตกحᇖᇃ ڷ۹اٴܷڶ ا॒ጋካ ٨ଚଖᇖᇃ ز܇س॒ु؟ ``؊ رل ۿڹ ौु رؑڿ، أ܇ܺ ؑפس أڕس ا۹שسٔ ᇖᆱ اჸუوام، اঔঐٔޒ زࢁ࢜ ᇖᇃ زࢁ࢜ة ۿـ أدق اካݠ܊ڶ وأ܇౼ ौु ุ༖ܫ، واঔঐݓჱၑ၁ إن ا۹ݐغك ا॒ጋካ ܇٩ศ༠ ৪ڹ ܇܌ ۑཛྷݐڲ ′.٩ٔכঔঐا ڲو ᇖᆱ ᇖᆇௌேو Continued on the next page. Saudi Dialect & Cultural Competence Benchmark20 September 1, 2026 Table A1 continued from the previous page. Prompt IDPrompt LANG-MENA-SAU-26وش ؗש္ထ ڶ ``ߓਉڲᇖᆱ ′ ا۹ݐשݮد؊ڶ؟ LANG-MENA-SAU-27وش ܇ש္ထ ෟڶ ``دߓ′ ؑ۹ܵغݚ؟ LANG-MENA-SAU-28وش ܇שෟ ``وش ߀ߐ܇ڿ؟′؟ LANG-MENA-SAU-29ᇖᆱ ݅سق ا۹টগשب، وش ؊اٴܷٔ ا۹ݓڡܭ ঔঐ ؊اٴݮل ``رح اჸჩك′؟ LANG-MENA-SAU-30ᅝᅜۣ܌ ศตกحᇖᇃ ܇שෟ ڷ္၁۹ت ا۹ݐשݮد؊ڶ وઠـኟᓾٔኟኑᙢ :۳ᅋᄽ ᇖᆱ ݮؔܭ، ؊ש ،੪ؔڲ، اؑڿ؟ LANG-MENA-SAU-31وش ݅۹وٴڶ ``ݝاٴڹ واߓਊاٴڹ′؟ وፆذا وأ ٨ଚଖ ؑשٔ ܇ ݅ݮّى გႮൿൢ םغسڶ، وش ؊اٴܷٔ؟ Saudi Dialect & Cultural Competence Benchmark21