Paper deep dive
LLMs taking shortcuts in test generation: A study with SAP HANA and LevelDB
Vekil Bekmyradov, Noah C. PĂŒtz, Thomas Bartz-Beielstein
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 97%
Last extracted: 4/18/2026, 1:34:58 AM
Summary
This paper investigates the 'shortcut learning' phenomenon in Large Language Models (LLMs) by comparing their performance in automated test generation on a familiar open-source system (LevelDB) versus a proprietary, unseen industrial codebase (SAP HANA). The study demonstrates that LLMs rely on memorization and shallow heuristics, prioritizing compilability over semantic effectiveness, and lack robust reasoning capabilities when faced with novel domains.
Entities (6)
Relation Signals (3)
Vekil Bekmyradov â authored â LLMs taking shortcuts in test generation
confidence 100% · THK-AI RESEARCH REPORT 2/2026 Vekil Bekmyradov
LLMs â exhibitshortcutbehavioron â SAP HANA
confidence 95% · Results show that LLMs excel on familiar, open-source benchmarks but struggle with unseen, complex domains
LLMs â relyon â Memorization
confidence 90% · models sometimes rely on shallow heuristics and memorization
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Large Language Models (LLMs) have achieved impressive results on public benchmarks, often leading to claims of advanced reasoning and understanding. However, recent research in cognitive science reveals that these models sometimes rely on shallow heuristics and memorization, taking shortcuts rather than demonstrating genuine cognitive abilities. This paper investigates LLM behavior in automated test generation for software, contrasting performance on an open-source system (LevelDB) with SAP HANA, one of the most widely deployed commercial database systems worldwide, whose proprietary codebase is guaranteed to be absent from training data. We combine cognitive evaluation principles, drawing on Mitchell's mechanism-focused assessment methodology, with empirical software testing, employing mutation score and iterative compiler-feedback repair loops to assess both accuracy and underlying reasoning strategies. Results show that LLMs excel on familiar, open-source benchmarks but struggle with unseen, complex domains, often prioritizing compilability over semantic effectiveness. These findings provide independent software engineering evidence for the broader claim that current LLMs lack robust reasoning, and highlight the need for evaluation frameworks that penalize trivial shortcuts and reward true generalization.
Tags
Links
- Source: https://arxiv.org/abs/2604.14437v1
- Canonical: https://arxiv.org/abs/2604.14437v1
Trouble viewing inline? Open PDF directly â
Full Text
29,256 characters extracted from source content.
Expand or collapse full text
LLMS TAKING SHORTCUTS IN TEST GENERATION: A STUDY WITH SAP HANA AND LEVELDB THK-AI RESEARCH REPORT 2/2026 Vekil Bekmyradov THK-AI Research Cluster, TH Köln vekilmuhammet.bekmyradov@smail.th-koeln.de Noah C. Puetz THK-AI Research Cluster, TH Köln noah_christoph.puetz@th-koeln.de Thomas Bartz-Beielstein THK-AI Research Cluster, TH Köln thomas.bartz-beielstein@th-koeln.de April 17, 2026 ABSTRACT Large Language Models (LLMs) have achieved impressive results on public benchmarks, often lead- ing to claims of advanced reasoning and understanding. However, recent research in cognitive sci- ence reveals that these models sometimes rely on shallow heuristics and memorization, taking short- cuts rather than demonstrating genuine cognitive abilities. This paper investigates LLM behavior in automated test generation for software, contrasting performance on an open-source system (Lev- elDB) with SAP HANA, one of the most widely deployed commercial database systems worldwide, whose proprietary codebase is guaranteed to be absent from training data. We combine cognitive evaluation principles, drawing on Mitchellâs mechanism-focused assessment methodology, with em- pirical software testing, employing mutation score and iterative compiler-feedback repair loops to assess both accuracy and underlying reasoning strategies. Results show that LLMs excel on familiar, open-source benchmarks but struggle with unseen, complex domains, often prioritizing compilabil- ity over semantic effectiveness. These findings provide independent software engineering evidence for the broader claim that current LLMs lack robust reasoning, and highlight the need for evaluation frameworks that penalize trivial shortcuts and reward true generalization. 1 Introduction The rapid advancement of Artificial Intelligence (AI) has led to exceptional performance on public benchmarks, of- ten driving claims that these systems possess broad capa- bilities like âreasoningâ and âunderstandingâ. However, recent research in cognitive science and software engi- neering indicates that this benchmark performance can overestimate their true real-world capabilities. AI algo- rithms often lack robust âworld modelsâ or abstract causal understanding, instead relying on âapproximate retrievalâ from their vast training data or exploiting unintended sur- face heuristics, taking âshortcutsâ to arrive at correct an- swers for the wrong reasons. Mitchell and her teamâs evaluation of LLMs on the Abstraction and Reasoning Corpus (ARC-AGI) demonstrates that while models like OpenAIâs o3 can achieve high accuracy, they often fail to grasp the intended human-like âcore knowledgeâ abstrac- tions (such as âobjectnessâ or geometry) (Moskvichev et al., 2023; Beger et al., 2026). Instead, they use less gen- eralizable shortcuts, such as tracking shallow pixel pat- terns. Consequently, AI evaluations must look beyond simple accuracy and analyze the mechanisms and rea- soning paths models take to achieve their results. This concern is not isolated to abstract reasoning tasks. A structurally identical dynamic has been documented in the broader machine learning literature under the term short- cut learning: the tendency of models to exploit statistical regularities that correlate with correct answers in train- arXiv:2604.14437v1 [cs.SE] 15 Apr 2026 THK-AI RESEARCH REPORT 2/2026 - APRIL 17, 2026 ing but fail to generalize under distribution shift (Geirhos et al., 2020; Mitchell and Krakauer, 2023). This phenomenon closely mirrors challenges in auto- mated software testing. In test generation, existing evalu- ations mostly rely on open-source benchmarks (e.g., Hu- manEval 1 ) that are highly likely to be present in the LLMsâ training corpora. This data contamination makes it unclear whether high performance reflects genuine rea- soning or mere memorization (Riddell et al., 2024; Chen et al., 2025). The problem is compounded by a second, domain-specific shortcut: code coverage, the dominant proxy metric for test quality in the field, can be tests that execute source lines without asserting any meaningful be- havior (Inozemtseva and Holmes, 2014). Bekmyradov (2026) addresses this by analyzing LLM behavior and testing strategies on SAP HANA, one of the most widely deployed commercial database systems worldwide, compared to the open-source key-value store LevelDB. Because SAP HANAâs codebase is propri- etary, it is guaranteed to be absent from the public pre- training corpora of all evaluated models, providing a zero- contamination evaluation environment. LevelDB, by con- trast, is an open-source project whose codebase and test suite are almost certainly present in those same corpora. Together, these perspectives reveal how LLMs optimize for immediate constraints, whether solving grid puzzles or fixing compilation errors by relying on shallow heuris- tics rather than robust semantic understanding. This paper is structured as follows: Section 2 describes the evaluation methodologies, combining Mitchellâs prin- ciples of mechanism-focused cognitive evaluation with the empirical software testing framework employed by Bekmyradov. Section 3 presents the empirical findings, highlighting the stark contrast in LLM performance be- tween familiar and novel domains and documenting the three principal shortcut behaviors observed. Finally, Sec- tion 4 discusses the implications of these results for AI evaluation practice and outlines directions for future re- search. 2 Methodology To rigorously evaluate the cognitive and reasoning ca- pabilities of LLMs, we combine Mitchellâs principles of cognitive evaluation with empirical software testing methodologies.Mitchell emphasizes that evaluations must test for robustness, avoid anthropomorphic biases, and investigate the mechanisms underlying performance rather than relying solely on accuracy (Mitchell and Krakauer, 2023; Mitchell et al., 2023). Applying this to software testing, Bekmyradov evalu- ated four models (GPT-5, Claude 4 Sonnet, Gemini 2.5 Pro, Qwen-3-Coder) using a multi-dimensional frame- work. Instead of relying exclusively on code coverage which can act as a flawed âshortcutâ metric inflated by 1 https://github.com/openai/human-eval trivial tests. Therefore the study incorporated mutation score (MS) to assess the actual fault-detection effective- ness of the generated tests (Jia and Harman, 2011; De- Millo et al., 1978). The experimental pipeline featured two scenarios. In Test Amplification, existing human-written test suites were ar- tificially reduced and the LLM was asked to regenerate the missing tests. In Whole-Suite Generation, all existing tests were discarded and the LLM generated a complete test suite from source code alone. Two context configura- tions were evaluated for Whole-Suite Generation: source code only, and source code augmented with the corre- sponding dependency files. To handle the strict syntactical demands of complex C++ projects, the methodology in- cluded an iterative compiler-feedback repair loop. When a generated test failed to compile, the error log was fed back to the LLM for up to ten repair iterations (Bekmyradov, 2026). This process allowed for the qualitative evaluation of the way the LLM reached its result. This is analogous to Mitchellâs method of prompting models to output nat- ural language transformation rules to reveal their internal logic. Tracking compilation success rate (CSR) across it- erations exposes whether models resolve compilation er- rors through genuine reasoning about the codebase or by progressively simplifying and hollowing out the generated tests. 3 Results The empirical findings reveal a profound gap between memorized retrieval and genuine generalization, heavily characterized by the models taking structural shortcuts. We organize the findings around following three distinct shortcut behaviors. 3.1 Memorization vs. Generalization On the open-source LevelDB project, all four models demonstrated exceptional performance, achieving up to 100% mutation scores and surpassing the human-written baseline of 52.79%. In the Whole-Suite Generation sce- nario, every evaluated model achieved a perfect mutation score of 100%, despite having no access to the original test suite (Bekmyradov, 2026). This strongly suggests that the models recognized the repository from their training data and reproduced familiar test patterns rather than syn- thesizing novel tests through reasoning. Listings 1 and 2 show a representative case: the generated test is a near- verbatim copy of the original human-written test, with only minor structural differences. This pattern is consis- tent with approximate retrieval from training data rather than reasoning about the code (Bekmyradov, 2026), mir- roring Lewis and Mitchellâs finding that LLM reasoning performance drops sharply when tasks are varied to be dissimilar from training data while testing the same ab- stract abilities (Lewis and Mitchell, 2024). 2 THK-AI RESEARCH REPORT 2/2026 - APRIL 17, 2026 1 TEST_F(AddBoundaryInputsTest, TestNoBoundaryFiles) 2 FileMetaData* f1 = 3 CreateFileMetaData(1, InternalKey("100 ", 2, kTypeValue), 4 InternalKey(InternalKey(" 100", 1, kTypeValue))) ; 5 FileMetaData* f2 = 6 CreateFileMetaData(1, InternalKey("200 ", 2, kTypeValue), 7 InternalKey(InternalKey(" 200", 1, kTypeValue))) ; 8 FileMetaData* f3 = 9 CreateFileMetaData(1, InternalKey("300 ", 2, kTypeValue), 10 InternalKey(InternalKey(" 300", 1, kTypeValue))) ; 11 level_files_.push_back(f3); 12 level_files_.push_back(f2); 13 level_files_.push_back(f1); 14 compaction_files_.push_back(f2); 15 compaction_files_.push_back(f3); 16 AddBoundaryInputs(icmp_, level_files_, & compaction_files_); 17 ASSERT_EQ(2, compaction_files_.size()); 18 Listing 1: Original human-written test. 1 TEST_F(AddBoundaryInputsTest, TestNoBoundaryFiles) 2 FileMetaData* f1 = 3 CreateFileMetaData(1, InternalKey("100 ", 2, kTypeValue), 4 InternalKey("199", 1, kTypeValue)); 5 FileMetaData* f2 = 6 CreateFileMetaData(2, InternalKey("200 ", 2, kTypeValue), 7 InternalKey("299", 1, kTypeValue)); 8 FileMetaData* f3 = 9 CreateFileMetaData(3, InternalKey("300 ", 2, kTypeValue), 10 InternalKey("399", 1, kTypeValue)); 11 level_files_.push_back(f3); 12 level_files_.push_back(f2); 13 level_files_.push_back(f1); 14 compaction_files_.push_back(f2); 15 AddBoundaryInputs(icmp_, level_files_, & compaction_files_); 16 ASSERT_EQ(1, compaction_files_.size()); 17 ASSERT_EQ(f2, compaction_files_[0]); 18 Listing 2: Generated test. Table 1 shows the Whole-Suite Generation results for SAP HANA and LevelDB, reporting line coverage (Cov L ), branch coverage (Cov B ), and mutation score (M S). Human (Full) refers to metrics obtained by run- ning the complete original human-written test suite, while Human (Reduced) refers to the artificially reduced subset used as the generation baseline in the experiments. For complete results including Test Amplification scenarios, we refer the reader to (Bekmyradov, 2026). In stark contrast, performance collapsed on the unseen SAP HANA codebase. For Test Amplification, the best- performing model (GPT-5) achieved a mutation score of 39.54%. For Whole-Suite Generation with dependency context, the maximum mutation score across all mod- els was 25.14%, falling substantially below the human- written baseline (Bekmyradov, 2026). 3.2 Taking Shortcuts in Iterative Repair The iterative compiler-feedback loop successfully in- creased compilation success rates by roughly 2â3Ă, with GPT-5 reaching up to 99% on SAP HANA. How- ever, qualitative analysis of the mechanisms used to achieve compilability revealed that models increasingly took shortcuts as the repair loop progressed (Bekmyradov, 2026). To resolve build errors, the LLMs shifted their optimization toward strict compilability at the expense of semantic effectiveness, frequently generating trivial or empty test bodies, or commenting out meaningful asser- tions. On LevelDB, almost all models reached near-perfect com- pilation success within the first one or two iterations, with Gemini 2.5 Pro jumping from 0% to 70% in a single repair step. This rapid convergence is consistent with the mem- orization hypothesis: the errors encountered were shallow and easily resolved by the modelsâ prior knowledge of the codebase. On SAP HANA, repair curves climbed slowly across all ten iterations and plateaued well below 100% for most models, indicating that the errors were struc- turally unfamiliar and not resolvable through pattern re- call alone. Figures 1 and 2 show the cumulative compilation success rates across repair iterations for SAP HANA and Lev- elDB, respectively. 3.3 Explicit Context Reveals Absent Structural Knowledge Just as Mitchell found that LLMs struggle with innate hu- man priors like âobjectnessâ (Mitchell et al., 2023; Beger et al., 2026), the models in this study demonstrated no inherent understanding of SAP HANAâs project struc- ture, dependency graph, or namespace conventions (Bek- myradov, 2026). When provided with source code alone, models frequently hallucinated incorrect application pro- 3 THK-AI RESEARCH REPORT 2/2026 - APRIL 17, 2026 0246810 Fix k 0.0 0.2 0.4 0.6 0.8 1.0 Cumulative success Whole Suite (Source+Header) 0246810 Fix k 0.0 0.2 0.4 0.6 0.8 1.0 Cumulative success Whole Suite (Source) 0246810 Fix k 0.0 0.2 0.4 0.6 0.8 1.0 Cumulative success Test Amplification (90% Reduced) 0246810 Fix k 0.0 0.2 0.4 0.6 0.8 1.0 Cumulative success Test Amplification (99% Reduced) SAP HANA - Cumulative Compilation Success Rates claude-4-sonnetgemini-2.5-progpt-5Qwen3-Coder Figure 1: Cumulative compilation success rate over k = 10 repair iterations for SAP HANA (Bekmyradov, 2026). 4 THK-AI RESEARCH REPORT 2/2026 - APRIL 17, 2026 0246810 Fix k 0.0 0.2 0.4 0.6 0.8 1.0 Cumulative success Whole Suite (Source+Header) 0246810 Fix k 0.0 0.2 0.4 0.6 0.8 1.0 Cumulative success Whole Suite (Source) 0246810 Fix k 0.0 0.2 0.4 0.6 0.8 1.0 Cumulative success Test Amplification (99% Reduced) LevelDB - Cumulative Compilation Success Rates claude-4-sonnetgemini-2.5-progpt-5Qwen3-Coder Figure 2: Cumulative compilation success rate over k = 10 repair iterations for LevelDB (Bekmyradov, 2026). 5 THK-AI RESEARCH REPORT 2/2026 - APRIL 17, 2026 Table 1: Whole-Suite Generation results for SAP HANA and LevelDB. âââ denotes values that were not publicly disclosed (Bekmyradov, 2026). ProjectMethodologyModelCov L (%) Cov B (%) M S (%) SAP HANA Human (Full)Humanâ Human (Reduced)Human66.7135.3330.41 Whole-Suite (Src) GPT-546.1427.9910.25 Claude 4 Sonnet47.7125.276.39 Qwen3-Coder35.0218.036.18 Gemini 2.5 Pro24.6815.212.39 Whole-Suite (Src+H) GPT-560.8734.2625.14 Claude 4 Sonnet62.1131.3117.49 Qwen3-Coder45.0221.729.20 Gemini 2.5 Pro37.9022.0410.60 LevelDB Human (Full)Human73.7857.0852.79 Human (Reduced)Human54.8737.5937.32 Whole-Suite (Src) GPT-582.6966.97100.00 Claude 4 Sonnet73.3057.23100.00 Qwen3-Coder63.4547.60100.00 Gemini 2.5 Pro71.9954.01100.00 Whole-Suite (Src+H) GPT-578.2060.9098.24 Claude 4 Sonnet76.3760.2699.89 Qwen3-Coder72.4655.25100.00 Gemini 2.5 Pro73.1557.68100.00 gramming interface (API) calls, omitted required headers, and failed to instantiate objects correctly. Providing dependency files as auxiliary context pro- duced consistent and substantial improvements across all four models: line coverage increased by 10â15 per- centage points, and mutation scores increased by up to 150% (Bekmyradov, 2026). This improvement was not observed on LevelDB, where header context made little difference, consistent with the interpretation that LevelDB models already possessed implicit structural knowledge from training data. Figures 3 and 4 show the effect of header context on coverage and mutation score across all four models on SAP HANA. The improvement is consistent across every model, confirming that the knowledge gap is a system- atic property of the unseen codebase rather than a model- specific artifact (Bekmyradov, 2026). 4 Discussion The combined findings from ARC-AGI evaluations and industrial unit test generation demonstrate that current LLMs rely heavily on data contamination and surface- level heuristics (Mitchell and Krakauer, 2023; Mitchell, 2019).When faced with novel, complex domains, whether generating test suites for SAP HANA or solv- ing abstract visual reasoning puzzles (Lewis and Mitchell, 2024), models lack robust âworld modelsâ and instead at- tempt to brute-force solutions. The observation that LLMs generate trivial tests merely to satisfy the compiler perfectly illustrates Goodhartâs Law: when a measure (e.g., compatibility or code coverage) be- comes a target, it ceases to be a good measure (Good- hart, 1984; Strathern, 1997). The models successfully reached the target of producing compiling code, but en- tirely missed the underlying goal of generating tests capa- ble of detecting software faults. Similarly, in ARC-AGI, models may output the correct grid by relying on shal- low features like pixel counts, completely missing the in- tended human-like abstraction. What makes the iterative repair finding particularly no- table is that the shortcut-taking process is directly ob- servable over time. With each repair iteration on SAP HANA, models progressively resolved compilation errors by removing assertions or generating empty test bodies rather than by reasoning about the underlying code (Bek- myradov, 2026). The metric being optimized is compil- ability which diverged from the actual goal of fault detec- tion. Implications. These results highlight the necessity of fundamentally changing how AI is evaluated. Researchers must look beyond saturated benchmarks and simple ac- curacy metrics, investigating the way models reach their answers to ensure they align with actual task competence. In the software testing domain, this means supplement- ing code coverage with mutation score as evaluation met- ric (Inozemtseva and Holmes, 2014; Bekmyradov, 2026). 6 THK-AI RESEARCH REPORT 2/2026 - APRIL 17, 2026 GPT-5Claude 4 SonnetQwen3-CoderGemini 2.5 Pro 0 20 40 60 80 46 . 14 47 . 71 35 . 02 24 . 68 60 . 87 62 . 11 45 . 02 37 . 9 27 . 99 25 . 27 18 . 03 15 . 21 34 . 26 31 . 31 21 . 72 22 . 04 Coverage (%) Impact of Header Context on SAP HANA Coverage Cov L (Src)Cov L (Src+H)Cov B (Src)Cov B (Src+H) Figure 3: Effect of header context on line and branch coverage for SAP HANA (Bekmyradov, 2026). GPT-5Claude 4 SonnetQwen3-CoderGemini 2.5 Pro 0 10 20 30 10 . 25 6 . 39 6 . 18 2 . 39 25 . 14 17 . 49 9 . 2 10 . 6 Mutation Score (%) Impact of Header Context on SAP HANA Mutation Score MS (Src)MS (Src+H) Figure 4: Effect of header context on mutation score for SAP HANA (Bekmyradov, 2026). Coverage measures execution; mutation score measures comprehension. A model that achieves high coverage through trivial tests is taking the same kind of shortcut as a model that solves ARC-AGI through pixel counting where the output looks correct, but the mechanism is not. Future work should integrate rigorous semantic feedback mechanisms to penalize trivial shortcuts and reward gen- uine fault-detection capability. One concrete direction is to incorporate mutation analysis directly into the repair loop: instead of feeding back only compiler errors, sur- viving mutants could be fed back to the model as explicit generation targets (Bekmyradov, 2026). Preliminary ev- idence from related work suggests that mutation-guided generation produces tests with substantially higher fault- detection effectiveness (Harman et al., 2025). Another direction concerns context and knowledge in- jection. The performance improvements from provid- ing dependency files suggest that static context injec- 7 THK-AI RESEARCH REPORT 2/2026 - APRIL 17, 2026 tion alone is insufficient for complex industrial codebases. More promising are agentic approaches, where LLMs are equipped with tools to autonomously retrieve relevant project context, navigate dependency graphs, and itera- tively interact with the build environment (Jimenez et al., 2024). Such agentic pipelines could dynamically assem- ble the minimal context required for each generation task, without requiring project-specific fine-tuning, and are a natural next step for test generation in large proprietary systems. Limitations. Several limitations of the underlying study should be acknowledged. The SAP HANA evaluation was restricted to a single representative component due to the computational cost of mutation testing at industrial scale (Bekmyradov, 2026). Each experimental configu- ration was repeated only twice, which is insufficient for formal statistical significance testing. Additionally, Lev- elDB is considerably smaller and less architecturally com- plex than SAP HANA: 21,207 lines of code versus 70,678 for the evaluated component in SAP HANA, which means the two systems differ not only in training data expo- sure but also in structural complexity, and this distinction should be kept in mind when interpreting the performance gap (Bekmyradov, 2026). Finally, the mutation opera- tor set was restricted to arithmetic, boundary, comparison, and logical operators. A broader operator set might reveal additional weaknesses in the generated tests. However, we assume these limitations do not affect the core argument: the performance contrast between Lev- elDB and SAP HANA is large enough to be robust to moderate experimental variance, but they do constrain the generalizability of the specific quantitative results. 5 Conclusions This paper examined LLM shortcut behavior through the lens of automated software test generation. By contrast- ing performance on a familiar open-source system (Lev- elDB) with a proprietary industrial codebase guaranteed to be absent from training data (SAP HANA), we docu- mented three concrete shortcut behaviors: performance collapse under distribution shift consistent with memo- rization rather than generalization, progressive degrada- tion of test quality under iterative repair as models opti- mize for compilability over fault detection, and a near- complete absence of implicit structural knowledge on un- seen codebases. Together, these findings provide empirical software en- gineering evidence for the broader claim advanced by Mitchell (Mitchell and Krakauer, 2023; Mitchell, 2019; Beger et al., 2026) that current LLMs exploit surface heuristics rather than demonstrating robust reasoning, and that evaluation frameworks must examine mechanisms, not just outcomes. Mutation score, combined with an iter- ative repair loop, offers a practical and principled way to make shortcut behavior directly observable in code gener- ation tasks. These findings carry consequences that extend well be- yond software testing. Marcus (Marcus, 2018) has ar- gued that deep learning suffers from systematic limi- tations, including brittleness, superficial generalization, and an inability to distinguish correlation from causa- tion, that no amount of scaling is likely to resolve. The shortcut behaviors documented here provide indepen- dent, empirical confirmation of precisely these limita- tions in a concrete engineering setting.When LLMs are deployed in domains where their training data cre- ates an illusion of competence, whether in medical di- agnosis, legal reasoning, scientific analysis, or public decision-making, the same mechanisms apply: surface- level pattern matching may produce plausible outputs that lack the causal understanding required for reliable re- sults (Bartz-Beielstein, 2026). The gap between mem- orized competence on LevelDB and genuine reasoning failure on SAP HANA is not merely a software engineer- ing concern; it is a microcosm of the broader risk that Mitchell (Mitchell, 2019), Marcus (Marcus, 2018), and Bartz-Beielstein (Bartz-Beielstein, 2026) have identified from complementary perspectives: society increasingly relies on systems whose fluent outputs mask fundamen- tal deficits in understanding. We hope these results open a productive dialogue between cognitive science and software engineering research com- munities on the shared challenge of building evaluations that are resistant to the shortcuts models have proven re- markably effective at finding. References Thomas Bartz-Beielstein. Why we need an AI-resilient society, 2026.URL https://arxiv.org/abs/ 1912.08786v2. v2, substantially revised April 2026. Claas Beger, Ryan Yi, Shuhao Fu, Kaleda Denton, Ar- seny Moskvichev, Sarah W. Tsai, Sivasankaran Raja- manickam, and Melanie Mitchell. Do AI models per- form human-like abstract reasoning across modalities?, 2026. URL https://arxiv.org/abs/2510.02125. Vekilmuhammet Bekmyradov. Evaluating the Effective- ness of LLM-Generated Unit Tests in the Context of Industrial DBMS. Technical Report 1/2026, FakultĂ€t 10 / Institut fĂŒr Data Science, Engineering, and An- alytics, 2026. URL https://doi.org/10.57684/ COS-1441. Wentao Chen, Lizhe Zhang, Li Zhong, Letian Peng, Zi- long Wang, and Jingbo Shang. Memorize or gener- alize? evaluating LLM code generation with evolved questions. CoRR, abs/2503.02296, 2025. doi: 10. 48550/ARXIV.2503.02296. URL https://doi.org/ 10.48550/arXiv.2503.02296. Richard A. DeMillo, Richard J. Lipton, and Fred G. Sayward. Hints on test data selection: Help for the 8 THK-AI RESEARCH REPORT 2/2026 - APRIL 17, 2026 practicing programmer. Computer, 11(4):34â41, 1978. doi: 10.1109/C-M.1978.218136. URL https://doi. org/10.1109/C-M.1978.218136. RobertGeirhos,Jörn-HenrikJacobsen,Claudio Michaelis,RichardS.Zemel,WielandBren- del, Matthias Bethge, and Felix A. Wichmann. Shortcutlearningindeepneuralnetworks. Nat. Mach. Intell., 2(11):665â673, 2020.doi: 10.1038/S42256-020-00257-Z.URL https: //doi.org/10.1038/s42256-020-00257-z. Charles A. E. Goodhart. Problems of monetary manage- ment: The U.K. experience. In Monetary Theory and Practice, pages 91â121. Macmillan, London, 1984. doi: 10.1007/978-1-349-17295-5_4. URL https:// doi.org/10.1007/978-1-349-17295-5_4. Origi- nally published in: Papers in Monetary Economics, vol. I, Reserve Bank of Australia, 1975. Mark Harman, Jillian Ritchey, Inna Harper, Shubho Sen- gupta, Ke Mao, Abhishek Gulati, Christopher Foster, and HervĂ© Robert. Mutation-guided llm-based test gen- eration at meta. In Leonardo Montecchi, Jingyue Li, Denys Poshyvanyk, and Dongmei Zhang, editors, Pro- ceedings of the 33rd ACM International Conference on the Foundations of Software Engineering, FSE Com- panion 2025, Clarion Hotel Trondheim, Trondheim, Norway, June 23-28, 2025, pages 180â191. ACM, 2025. doi: 10.1145/3696630.3728544. URL https: //doi.org/10.1145/3696630.3728544. Laura Inozemtseva and Reid Holmes. Coverage is not strongly correlated with test suite effectiveness.In Pankaj Jalote, Lionel C. Briand, and AndrĂ© van der Hoek, editors, 36th International Conference on Soft- ware Engineering, ICSE â14, Hyderabad, India - May 31 - June 07, 2014, pages 435â445. ACM, 2014. doi: 10.1145/2568225.2568271. URL https://doi.org/ 10.1145/2568225.2568271. Yue Jia and Mark Harman. An analysis and survey of the development of mutation testing. IEEE Trans. Software Eng., 37(5):649â678, 2011. doi: 10.1109/TSE.2010. 62. URL https://doi.org/10.1109/TSE.2010. 62. Carlos E. Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik R. Narasimhan. Swe-bench: Can language models resolve real-world github issues?In The Twelfth International Con- ference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net, 2024. URL https://openreview.net/forum?id= VTF8yNQM66. Martha Lewis and Melanie Mitchell.Evaluating the robustness of analogical reasoning in large language models, 2024.URL https://arxiv.org/abs/ 2411.14215. Gary Marcus. Deep learning: A critical appraisal, 2018. URL https://arxiv.org/abs/1801.00631. Melanie Mitchell.Artificial intelligence hits the bar- rier of meaning. Information, 10(2):51, 2019. doi: 10.3390/info10020051. URL https://doi.org/10. 3390/info10020051. Melanie Mitchell and David C. Krakauer. The debate over understanding in AIâs large language models. Proc. Natl. Acad. Sci. U.S.A., 120(13):e2215907120, 2023. doi: 10.1073/pnas.2215907120. URL https://doi. org/10.1073/pnas.2215907120. Melanie Mitchell, Alessandro B. Palmarini, and Arseny Moskvichev. Comparing humans, GPT-4, and GPT-4V on abstraction and reasoning tasks, 2023. URL https: //arxiv.org/abs/2311.09247. Arsenii Kirillovich Moskvichev, Victor Vikram Odouard, and Melanie Mitchell. The conceptARC benchmark: Evaluating understanding and generalization in the ARC domain. Transactions on Machine Learning Re- search, 2023.ISSN 2835-8856.URL https:// openreview.net/forum?id=8ykyGbtt2q. Martin Riddell, Ansong Ni, and Arman Cohan. Quanti- fying contamination in evaluating code generation ca- pabilities of language models. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors, Proceedings of the 62nd Annual Meeting of the Association for Com- putational Linguistics (Volume 1: Long Papers), ACL 2024, Bangkok, Thailand, August 11-16, 2024, pages 14116â14137. Association for Computational Linguis- tics, 2024.doi:10.18653/V1/2024.ACL-LONG. 761. URL https://doi.org/10.18653/v1/2024. acl-long.761. Marilyn Strathern.âimproving ratingsâ: audit in the British University system. European Review, 5(3):305â 321, 1997. 9