Paper deep dive

Characterizing Overconfident Failure in LLM-Based Code Generation

Ravishka Rathnasuriya, Wei Yang

Year: 2026Venue: arXiv preprintArea: cs.SEType: PreprintEmbeddings: 89

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%

Last extracted: 10/9/2026, 8:37:05 AM

Summary

This paper investigates the phenomenon of 'overconfident failure' in Large Language Models (LLMs) used for code generation, where incorrect programs are generated with high token-level confidence comparable to correct ones. The study analyzes four open-source code models across three execution-based benchmarks (HumanEval+, MBPP+, BigCodeBench) to evaluate the reliability of existing uncertainty metrics as proxies for execution correctness. Key findings indicate that existing uncertainty signals provide only partial evidence of failure, overconfidence persists at both program and token levels, instruction tuning can increase certainty on failing generations without improving correctness discrimination, and common mitigation strategies (calibration, sampling, abstention) do not reliably resolve the issue. Exploratory latent analysis suggests hidden representations may encode correctness signals not visible in output confidence.

Entities (13)

Overconfident Failure · concept · 98%BigCodeBench · benchmark · 95%CodeLlama-7B-Instruct · model · 95%DeepSeek-Coder-7B-Instruct · model · 95%HumanEval · benchmark · 95%MBPP · benchmark · 95%OpenCoder-8B-Instruct · model · 95%Qwen2.5-Coder-7B-Instruct · model · 95%Instruction-tuning · technique · 90%Post-hoc Calibration · technique · 90%Repeated Sampling · technique · 90%Uncertainty Metrics · method · 90%Latent Representations · concept · 85%

Relation Signals (8)

Overconfident Failure → characterizedby → High confidence on incorrect programs

confidence 95% · incorrect programs are often generated with token-level confidence comparable to correct programs.

CodeLlama-7B-Instruct → evaluatedon → BigCodeBench

confidence 95% · We study this dilemma across four open-source code models and three execution-based benchmarks.

CodeLlama-7B-Instruct → evaluatedon → HumanEval

confidence 95% · We study this dilemma across four open-source code models and three execution-based benchmarks.

CodeLlama-7B-Instruct → evaluatedon → MBPP

confidence 95% · We study this dilemma across four open-source code models and three execution-based benchmarks.

Post-hoc Calibration → improves → Probability Alignment

confidence 90% · Post-hoc calibration substantially improves probability alignment, but calibrated scores provide little predictive skill beyond the benchmark pass rate.

Uncertainty Metrics → providespartialevidenceof → Execution Failure

confidence 90% · existing uncertainty signals provide only partial and model-dependent evidence of execution failure.

Instruction-tuning → increasescertaintyon → Failing Generations

confidence 85% · instruction tuning can increase certainty on failing generations without consistently improving correctness discrimination.

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Large language models (LLMs) are increasingly used for automated code generation, but generated programs can appear syntactically plausible while still failing execution-based correctness checks. Existing validation methods, such as testing and program analysis, remain essential but are often incomplete, costly, or applied only after generation. Model-derived uncertainty is therefore a natural early reliability signal. This paper studies the dilemma of overconfidence in code LLMs where incorrect programs are often generated with token-level confidence comparable to correct programs. We study this dilemma across four open-source code models and three execution-based benchmarks. Our analysis begins by investigating whether existing uncertainty metrics provide reliable proxies for execution correctness in code generation. We then characterize overconfidence at both global and local token levels, asking whether incorrect programs remain indistinguishable from correct ones under confidence and entropy summaries, including selective generation and the limits of instruction tuning. Finally, we evaluate whether common mitigation strategies reduce this failure mode. Our study yields four findings. First, existing uncertainty signals provide only partial and model-dependent evidence of execution failure. Second, overconfidence persists at both program and token levels, and uncertainty-based selection does not consistently improve accepted-set accuracy. Third, instruction tuning can increase certainty on failing generations without consistently improving correctness discrimination. Fourth, common mitigation techniques improve specific aspects of reliability but do not reliably resolve overconfident failure. Our exploratory latent analysis suggests that hidden representations may encode correctness-related signals that output confidence does not expose.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

85,441 characters extracted from source content.

Expand or collapse full text

Characterizing Overconfident Failure in LLM-Based Code Generation RAVISHKA RATHNASURIYA, University of Texas at Dallas, USA WEI YANG, Fudan University, China Large language models (LLMs) are increasingly used for automated code generation, but generated programs can appear syntactically plausible while still failing execution-based correctness checks. Existing validation methods, such as testing and program analysis, remain essential but are often incomplete, costly, or applied only after generation. Model-derived uncertainty is therefore a natural early reliability signal. This paper studies thedilemmaofoverconfidencein code LLMs where incorrect programs are often generated with token-level confidence comparable to correct programs. We study this dilemma across four open-source code models and three execution-based benchmarks. Our analysis begins by investigating whether existing uncertainty metrics provide reliable proxies for execution correctness in code generation. We then characterize overconfidence at both global and local token levels, asking whether incorrect programs remain indistinguishable from correct ones under confidence and entropy summaries, including selective generation and the limits of instruction tuning. Finally, we evaluate whether common mitigation strategies reduce this failure mode, and we examine latent representations as exploratory evidence for future reliability mechanisms. Our study yields four findings. First, existing uncertainty signals provide only partial and model-dependent evidence of execution failure. Second, overconfidence persists at both program and token levels, and uncertainty- based selection does not consistently improve accepted-set accuracy. Third, instruction tuning can increase certainty on failing generations without consistently improving correctness discrimination. Fourth, common mitigation techniques improve specific aspects of reliability but do not reliably resolve overconfident fail- ure. Our exploratory latent analysis suggests that hidden representations may encode correctness-related signals that output confidence does not expose. Together, these findings identify overconfident failure as a software-engineering reliability problem and offer a practical path toward building more reliable LLM-driven code-generation systems. 1 Introduction Large language models (LLMs) have become increasingly effective at code generation, synthesiz- ing implementations from natural-language specifications, programming tasks, and partial code contexts [1–5]. They are now integrated into software development workflows to assist with im- plementation, debugging, testing, and repair [6–9]. However, generated code is not self-validating. A program may be syntactically valid, idiomatic, and plausible while still violating the intended specification, especially on edge cases revealed only through execution. Thus, a central reliability challenge for LLM-based software engineering is not only whether a model can generate code, but whether a system can determine when generated code should be trusted. Existing validation approaches primarily rely on execution-based evaluation [10–13] or program analysis [14–16]. These techniques are essential, but they do not fully solve the deployment-time reliability problem. Execution requires tests, input generators, or behavioral oracles, which may be incomplete, expensive, or unavailable for each generated candidate. Program analysis can identify specific defect classes, but it is often conservative, property-specific, and does not generally estimate whether an arbitrary generated program satisfies its specification. Moreover, both execution and analysis are typically applied after generation. This limits their ability to provide lightweight, model-intrinsic reliability signals during interactive code generation. A natural alternative is an uncertainty-aware code-generation system. In an ideal setting, the system would know whether a generated program generalizes to unseen tests or real user inputs. Authors’ Contact Information: Ravishka Rathnasuriya, ravishka.rathnasuriya@utdallas.edu, University of Texas at Dallas, Richardson, USA; Wei Yang, yang_wei@fudan.edu.cn, Institute of Systems for Advanced Computing, Fudan University, Shanghai, China. arXiv:2610.11300v1 [cs.SE] 8 Oct 2026 2Ravishka Rathnasuriya and Wei Yang Such a generalizability oracle is unavailable in practice, because functional correctness can only be established through validation against a specification, tests, or downstream behavior. Therefore practical systems must rely on proxies. The broader machine-learning literature operationalizes this proxy through uncertainty metrics derived from predictive distributions, confidence scores, entropy, margins, calibration methods, and related quantities. Such metrics have been extensively studied in classification tasks [17–27] and more recently in natural language generation (NLG) [28–33], where they are used as proxies for output correctness, factuality, or risk. These estimates also support selective abstention [34–36], where high-risk outputs are withheld or routed to additional validation. However, uncertainty is less directly related to correctness in code generation than in classi- fication or open-ended NLG. In classification, uncertainty is defined over the label distribution, and evaluation is performed on the predicted label. In open-ended NLG, uncertainty is commonly estimated over the generated response and evaluated against response-level properties such as cor- rectness or factuality. In code generation, model probabilities are assigned during token generation, whereas correctness is determined by the execution behavior of the completed program. Therefore, a generated program can have high token likelihood and still fail to satisfy its specification. Thus, token- and sequence-level uncertainty are only proxies for execution correctness. Using token- and sequence-level uncertainty as proxies for execution correctness creates a dilemmaofoverconfidencein code generation. A generated program may receive high model confidence and still fail execution-based evaluation. This mismatch is consistent with how code LLMs are trained and evaluated. First, autoregressive training optimizes next-token prediction rather than whether model confidence reflects execution correctness. Second, standard code-generation evaluation measures generation success rather than the reliability of model confidence. Pass@1 and pass@k measure [10–13] whether generated programs pass the evaluation tests, whereas binary grading gives no credit to abstention [37]. Therefore, such evaluation does not assess whether failing generations receive lower confidence than passing ones. Testing and program analysis provide evidence more directly tied to execution correctness, but they require external validation and may be costly or incomplete. Therefore, model-derived uncertainty remains attractive as an early reliability signal, despite the possibility of overconfident failures. There is growing research interest in using uncertainty to support reliable code generation. Ex- isting studies pursue three main directions. First, they evaluate uncertainty estimators as predictors of code correctness [33,38]. Second, they calibrate confidence against observed correctness or use uncertainty to support abstention and other reliability decisions [39–43]. Third, recent studies examine how uncertainty behaves when it is used to guide candidate selection, correction, or other generation-time decisions [44,45]. These studies show that uncertainty can provide useful information about correctness, but their evaluations mainly measure estimator performance or the effectiveness of an uncertainty-guided decision rule. However, important questions remain about the reliability of uncertainty as a proxy for execution correctness. Prior studies demonstrate that uncertainty can be informative, but they do not fully establish the extent to which it remains reliable when passing and failing generations receive similar uncertainty scores. It also remains unclear where this mismatch arises during generation, under which conditions it persists, and whether and to what extent common mitigation strategies can address it. Addressing these questions requires a systematic characterization of the reliability limits of model-derived uncertainty in code generation. To characterize the reliability limits of model-derived uncertainty, we first conduct a preliminary study of existing uncertainty measures (Section 4). Across 17 generation-stage metrics and six prefill-stage metrics, no measure provides stable failure ranking across models and benchmarks. We then examine overconfidence through token confidence and predictive entropy at both program and Characterizing Overconfident Failure in LLM-Based Code Generation3 token levels (Section 5). Failing programs often remain highly certain, including within their least- confident token regions. This misalignment affects selective generation where reducing coverage does not consistently increase selective accuracy and can decrease it when failures occupy high- certainty regions. Furthermore, instruction tuning can also increase certainty on failing generations without a corresponding improvement in correctness discrimination. These findings show that overconfidence is a distinct reliability problem where improvements in generation accuracy do not necessarily improve the model’s ability to identify which generations are likely correct. Finally, we evaluate whether common mitigation techniques reduce overconfident failure (Sec- tion 6). Post-hoc calibration substantially improves probability alignment, but calibrated scores provide little predictive skill beyond the benchmark pass rate. Repeated sampling improves recovery at small budgets, but its improvements diminish and a persistent set of tasks remains unsolved. Moreover, explicit abstention prompting likewise does not reliably induce deferral on incorrect generations. As an exploratory analysis, we find systematic pass–fail differences in latent-state geometry that are not reflected in output-token confidence. These results show that improving generation success or confidence calibration does not necessarily eliminate overconfident failure. This paper makes the following contributions: •We formulate the dilemma of overconfidence in code LLMs as a barrier to practical uncertainty- aware code generation. •We show that existing uncertainty signals provide only partial and model-dependent evi- dence of execution failure across both prefill and generation stages. •We characterize overconfidence at program and token levels, and show that it can persist under selective generation and after instruction tuning. •We evaluate post-hoc calibration, repeated sampling, and explicit abstention, showing that none reliably resolves overconfident failure despite improving specific aspects of reliability. •We provide exploratory evidence that latent representations contain correctness-related structure not reflected in output uncertainty, motivating representation-aware reliability mechanisms. 2Problem Setup and Scope This section formalizes code generation as a selective decision problem and introduces the uncertainty- aware system view that motivates our study. We then define the sources of uncertainty signals and formulate overconfident failure as the central reliability problem. 2.1 Code Generation as a Selective Decision Problem Given a prompt푥, a code-generation model푀produces a candidate program푦. A task-specific test suite evaluates푦, yielding an execution outcome푍(푥,푦) ∈ pass, fail. We use this outcome as the operational correctness signal. Standard code-generation evaluation measures whether a model can generate correct solutions, commonly through pass@푘. This measures generation capability, but does not indicate whether the model can distinguish its passing generations from its failing ones. We formulate the latter as a selective decision problem. Let 푆(푥,푦) ∈R denote a reliability score for candidate푦, where larger values indicate a greater estimated likelihood of passing. A selective system uses this score to apply a decision rule 휋(푥,푦,푆) ∈ ACCEPT, ABSTAIN. Thus, generation capability and reliability estimation are distinct. We study whether model- derived uncertainty scores provide reliable proxies for generation correctness. 4Ravishka Rathnasuriya and Wei Yang Ideal system Practical system Specification Code LLM Generalization Oracle Accept / Abstain Specification Code LLM Uncertainty Estimator Accept / Abstain SPECIFICATION (HUMANEVAL #0) Check if in given list of numbers, any two numbers are closer to each other than given threshold. passes test suitesavg. token confidence: 0.93 def has_close_elements(numbers: List[float], threshold: float) -> bool: for iin range(len(numbers) -1): for j in range(i+ 1, len(numbers)): if abs(numbers[i] -numbers[j]) < threshold: return True return False fails test suitesavg. token confidence: 0.94 def has_close_elements(numbers: List[float], threshold: float) -> bool: for iin range(len(numbers) -1): if abs(numbers[i] -numbers[i+ 1]) < threshold: return True return False Not Available a Priori (a) Ideal oracle-based system. A gener- alization oracle would determine whether generated code should be trusted, but such an oracle is not available a priori. Ideal system Practical system Specification Code LLM Generalization Oracle Accept / Abstain Specification Code LLM Uncertainty Estimator Accept / Abstain SPECIFICATION (HUMANEVAL #0) Check if in given list of numbers, any two numbers are closer to each other than given threshold. passes test suitesavg. token confidence: 0.93 def has_close_elements(numbers: List[float], threshold: float) -> bool: for iin range(len(numbers) -1): for j in range(i+ 1, len(numbers)): if abs(numbers[i] -numbers[j]) < threshold: return True return False fails test suitesavg. token confidence: 0.94 def has_close_elements(numbers: List[float], threshold: float) -> bool: for iin range(len(numbers) -1): if abs(numbers[i] -numbers[i+ 1]) < threshold: return True return False Not Available a Priori (b) Practical uncertainty-aware system. The oracle is replaced by an uncertainty es- timator that assigns a reliability score from model or generation behavior. Ideal system Practical system Specification Code LLM Generalization Oracle Accept / Abstain Specification Code LLM Uncertainty Estimator Accept / Abstain SPECIFICATION (HUMANEVAL #0) Check if in given list of numbers, any two numbers are closer to each other than given threshold. passes test suites avg. token confidence: 0.92 def has_close_elements(numbers: List[float], threshold: float) -> bool: for iin range(len(numbers) -1): for j in range(i+ 1, len(numbers)): if abs(numbers[i] -numbers[j]) < threshold: return True return False fails test suitesavg. token confidence: 0.94 def has_close_elements(numbers: List[float], threshold: float) -> bool: for iin range(len(numbers) -1): if abs(numbers[i] -numbers[i+ 1]) < threshold: return True return False Not Available a Priori (c) Overconfident failure. Two solutions for the same task receive similar mean token confidence, although one passes the test suite and the other fails by checking only adjacent elements on CodeLllama Model. Fig. 1. Motivation for uncertainty-aware code generation. Since generalization oracle cannot be known a priori, practical systems use model-derived uncertainty as a proxy reliability signal. The example illustrates the failure mode studied in this paper: an incorrect program can receive confidence comparable to a passing program. 2.2 Uncertainty-Aware Code Generation Uncertainty-aware systems use model-derived evidence to decide whether an output should be trusted. In an ideal code-generation system (Figure 1a), a generalization oracle would determine whether a generated program generalizes to unseen tests, real inputs, or downstream use. Such an oracle is not available a priori, because functional correctness must be established through validation against specifications, tests, formal properties, or observed behavior. Therefore, a practical uncertainty-aware system uses an estimated reliability signal as a proxy for unavailable correctness information. Figure 1b illustrates this setting, where an uncertainty estimator assigns a score푆(푥,푦)to generated code. The key assumption is that this score is informative about the execution outcome: higher-scoring candidates should be more likely to pass than lower-scoring candidates. Overconfident failure arises when this assumption breaks, such that a failing program receives a high reliability score and may therefore be accepted by the selective system. 2.3 Sources of Uncertainty Signals We define an uncertainty-derived reliability signal as a measurable score푆(푥,푦)attached to a generated candidate푦for prompt푥. The score is intended to approximate the likelihood that푦will pass the task’s tests. We consider three sources of model-derived reliability signals. White-box signals use information from the model’s generation process, including token probabilities, entropy, sequence likelihood, and hidden states. Black-box signals use variation across multiple sampled generations without requiring access to model internals. Judge-based signals use an external model to estimate the reliability of a generated program without executing it. Characterizing Overconfident Failure in LLM-Based Code Generation5 We also distinguish when the signal is measured.Prefill-stagesignals are computed before any code tokens are emitted, using only the prompt and the model’s initial processing. They may capture prompt difficulty, ambiguity, or underspecification, but they cannot observe implementation-specific errors.Decoding-stagesignals are computed from generated tokens, token distributions, hidden states, or sampled candidates. These signals are closer to the final program, but they may still reflect local fluency or memorized coding patterns rather than functional correctness. 2.4 The Problem: Overconfident Failure We define anoverconfidentfailureas a failing generation that receives a high reliability score. Formally, for prompt푥, candidate푦, execution outcome푍(푥,푦), and reliability score푆(푥,푦), over- confident failure occurs when푍(푥,푦)= failwhile푆(푥,푦)lies in a high-reliability region. Because score distributions vary across models and metrics, we do not assume a universal threshold. In- stead, we examine whether failing generations receive reliability scores comparable to passing generations. The underlying challenge is that model confidence and program correctness characterize dif- ferent properties. Token-level confidence reflects the probability assigned to decoding decisions, whereas execution evaluates the semantics of the completed program. Therefore, a semantically in- correct implementation can receive high token-level confidence despite failing execution. Figure 1c illustrates this mismatch under the same specification: the passing implementation compares all element pairs, whereas the failing implementation checks only adjacent pairs. Despite this semantic difference, both generations receive comparable token confidence. 3 Empirical Overview This section summarizes the study design, including the evaluated models, benchmarks, and research questions. Detailed experimental setups are described in the corresponding sections. 3.1 Benchmarks We evaluate generated code on three execution-based benchmarks: HumanEval+, MBPP+, and BigCodeBench. HumanEval+ [46,47] and MBPP+ [47,48] extend widely used function-level code-generation benchmarks with stronger test suites from EvalPlus [4]. HumanEval+ contains 164 programming tasks specified by function signatures and natural-language descriptions. The MBPP+ evaluation set contains 378 tasks derived from introductory programming problems. These two benchmarks provide controlled function-level settings for evaluating execution correctness. BigCodeBench [49] broadens the evaluation to programming tasks with richer contexts, more complex instructions, and greater use of library APIs. It contains 1,140 Python tasks spanning 723 function calls from 139 libraries across seven domains. We include BigCodeBench to examine whether overconfident failure persists beyond controlled function-level tasks. Models. We study four code-specialized, open-source instruction-tuned LLMs: CodeLlama-7B- Instruct, DeepSeek-Coder-7B-Instruct, Qwen2.5-Coder-7B-Instruct, and OpenCoder-8B-Instruct [1,2, 50]. These models have been used in prior code-generation evaluations and provide comparable open-source baselines in the 7-8B parameter range [47, 51, 52]. We select these models based on four criteria. First, their open-source availability enables access to token-level statistics, prefill-stage signals, and internal representations. Second, restricting the models to 7-8B parameters reduces variation attributable to model scale. Third, all models are specialized and instruction-tuned for code generation. Fourth, the models originate from different development lineages, reducing the risk that the observed behavior is specific to one training pipeline. For the post-training analysis in Section 5.5, we additionally compare base and instruction- tuned variants of OpenCoder and DeepSeekCoder. Table 1 reports the pass@1 accuracy of the 6Ravishka Rathnasuriya and Wei Yang primary models on each benchmark. All experiments were conducted on a server equipped with an Intel(R) Xeon(R) w5-2545 CPU and four NVIDIA RTX 6000 Ada GPUs. Table 1. Pass@1 Accuracy for Each Benchmark and Models ModelCodeLlama DeepSeek-Coder OpenCoder QwenCoder HumanEval+34.170.178.071.3 MBPP+46.066.171.265.9 BigCodeBench6.520.623.88.2 3.2 Research Questions We investigate overconfident failure through the following research questions. RQ1: To what extent do existing uncertainty signals reflect the likelihood of generation correctness? RQ2: How does overconfident failure manifest, and under what conditions does it persist in code generation? RQ3: To what extent do existing mitigation strategies reduce overconfident failure? RQ4: Do latent representations contain correctness information not exposed by output uncertainty? 4 Reliability of Standard Uncertainty Signals for Failure Detection This section addresses RQ1 in Section 3.2 by examining to what extent standard uncertainty signals reliably indicate the likelihood that generated code will fail. This research question serves as an initial diagnostic analysis: if existing metrics consistently separate passing from failing programs, then overconfident failure would be less concerning for selective code generation. Evaluation Setup. For each generated solution, we compute 17 generation-stage uncertainty scores. We also compute six prefill scores: end-of-prompt next-token MSP, entropy, margin, en- ergy, mean token entropy, and mean negative log-likelihood. Table 2 summarizes the evaluated metrics. For metrics that require multiple generations, we sample up to 10 candidates per prompt at temperature 1.0. We evaluate each score using AUC, withfailas the positive class, and orient the scores so that larger values indicate greater estimated failure risk. An AUC of 0.5 corresponds to random ranking. We focus on HumanEval+ and MBPP+ because both passing and failing generations are sufficiently represented across the studied models. We exclude BigCodeBench because its substantially lower pass rates leave few passing generations for several models, making pass-fail discrimination estimates less stable. Full metric definitions and implementation details are available on the project website [60]. Results. Table 3 reports the generation and prefill results. For generation-stage uncertainty, the best AUC in each model-benchmark pairs ranges from 0.598 to 0.696. No metric exceeds 0.70, and several remain close to random ranking. The strongest metric also changes across models and benchmarks where no metric family consistently provides the best separation. Prefill uncertainty shows the same limitation. The highest AUC is 0.70 for CodeLlama on HumanEval+, whereas most model–benchmark settings fall between 0.52 and 0.62. Several scores are close to random ranking, particularly on MBPP+, and no prefill metric is consistently strongest across models. Finding. The limited failure discrimination is not specific to one uncertainty formulation or one stage of inference. Across 17 generation-stage signals and six prefill signals, no uncertainty measure provides stable failure ranking across models and benchmarks. Characterizing Overconfident Failure in LLM-Based Code Generation7 Table 2. Uncertainty signals evaluated in the preliminary study. FamilyMetricsSignal captured White-box token scores MSP, Perplexity, Entropy, PMI, CPMI, R-Div, F-R Dist [28,53– 56] Measure token likelihood, surprise, distribu- tional concentration, and prompt dependence. White-box multi-sample scores MCEnt, MSPS, RMI, Ens, MI[28, 57] Measure probability support and distributional disagreement across sampled candidates. Black-box out- put similarity LexSim, EigV, DegGlobal [56, 58] Measure diversity or disagreement among sam- pled programs without using model probabili- ties. Judge-based scores Judge, M_Judge [28, 59]Measure LLM-estimated pass likelihood from the prompt and generated solution. Table 3. AUC (↑) results of prefill-stage and generative uncertainty metrics. Metric HumanEval+MBPP+ CodeLlamaDeepSeekOpenCoderQwenCoderCodeLlamaDeepSeekOpenCoderQwenCoder Prefill uncertainty MSP0.6900.5900.5300.5300.4600.4900.4800.610 Entropy0.7000.5900.5300.5400.4700.6000.5000.540 Margin0.6900.5900.5200.5200.4700.4500.4800.620 Energy0.5900.4200.5400.5600.5300.5400.5100.520 MeanTokenEntropy0.6000.5800.5400.4300.5600.5700.5400.540 NLL0.6100.5900.5800.4400.5900.6000.5700.550 Generative uncertainty MSP0.6700.6230.6300.6170.5680.5610.5170.622 Perplexity0.6700.6230.6300.6170.5680.5610.5170.622 Entropy0.6300.6150.6390.6350.5700.5670.5240.629 PMI0.5210.5100.5010.5570.5890.5720.5460.569 CPMI0.5060.5890.5030.6810.5780.5090.5510.546 R-Div0.6620.6190.6400.6250.5730.5670.5190.623 F-R Dist0.5680.6140.6270.6400.5130.5430.5330.629 MCEnt0.6300.5530.6220.5940.5170.5980.5880.529 MSPS0.6230.5540.6220.5890.5070.5990.5890.533 RMI0.5080.5420.6170.5990.5820.5780.6380.511 Ens0.5790.6280.6240.6150.5980.6960.6600.565 MI0.5770.6290.6180.6340.5960.6910.6510.600 LexSim0.5960.6290.6110.5360.5680.5890.6150.515 EigV0.5970.6320.6090.5320.5630.5850.6090.516 DegGlobal0.5760.5680.5210.5740.5780.5610.5560.514 Judge0.5300.5140.6010.5070.5420.5380.6310.522 M_Judge0.5300.5140.6010.5060.5420.5380.6310.522 Implication. The results do not support a model- and task-independent mapping from standard uncertainty scores to code correctness. Furthermore, an uncertainty signal that is useful in one setting cannot be assumed to provide the same failure evidence in another. SE Takeaway. Code-generation tools should not reuse uncertainty thresholds across models or workloads. Acceptance and validation policies should be re-evaluated for each deployment setting because the same uncertainty signal can provide different failure discrimination across settings. 5 Characterizing Overconfidence During Code Generation In this section, we answer RQ2 by characterizing how overconfidence manifests across the code- generation process. We examine whether failing programs remain highly confident overall, whether local token behavior exposes regions of uncertainty, and whether predictive entropy provides additional evidence of failure. We further study the effect of these uncertainty signals on selective accuracy and examine whether overconfidence persists after instruction tuning. 8Ravishka Rathnasuriya and Wei Yang Table 4. Statistics on average and bottom-10% token confidence for passing and failing generated programs. TaskModel Sample SizeAverage ConfidenceBottom-10% Confidence Correct푛Incorrect푛 CorrectIncorrect t-valuep-value CorrectIncorrect t-valuep-value Mean± Std.MedianMean± Std.MedianMean± Std.MedianMean± Std.Median HumanEval+ CodeLlama56108 0.962± 0.0120.965 0.956± 0.0140.9573.1940.002 0.703± 0.0760.721 0.651± 0.0830.6544.0540.000 DeepSeek11549 0.951± 0.0210.957 0.943± 0.0210.9462.2330.028 0.642± 0.0900.658 0.607± 0.0750.6102.5480.012 OpenCoder12836 0.956± 0.0240.960 0.948± 0.0230.9531.9410.057 0.675± 0.1190.677 0.631± 0.0900.6352.4030.019 QwenCoder11747 0.947± 0.0250.953 0.935± 0.0360.9432.0470.045 0.619± 0.1010.640 0.559± 0.1160.5703.1060.003 MBPP+ CodeLlama174204 0.925± 0.0250.930 0.919± 0.0260.9212.3120.021 0.542± 0.0810.543 0.525± 0.0830.5221.9910.047 DeepSeek250128 0.944± 0.0230.947 0.938± 0.0260.9402.3010.022 0.609± 0.0960.606 0.589± 0.1020.5861.8430.067 OpenCoder269109 0.924± 0.0240.927 0.922± 0.0280.9240.5150.607 0.550± 0.0800.549 0.542± 0.0950.5400.7970.426 QwenCoder249129 0.901± 0.0350.902 0.881± 0.0490.8864.1000.000 0.480± 0.0860.473 0.439± 0.1170.4403.4850.001 BigCodeBench CodeLlama751065 0.943± 0.0200.946 0.956± 0.0390.959-5.1710.000 0.602± 0.0940.585 0.730± 0.2040.668-10.2490.000 DeepSeek235905 0.913± 0.0300.917 0.909± 0.0310.9141.5870.113 0.511± 0.0800.507 0.498± 0.0790.5012.2890.023 OpenCoder272868 0.936± 0.0250.938 0.934± 0.0270.9370.9080.365 0.580± 0.0990.575 0.577± 0.0950.5720.4280.669 QwenCoder941046 0.897± 0.0410.897 0.920± 0.0680.931-4.7460.000 0.455± 0.1220.434 0.574± 0.2230.516-8.3660.000 푝= 0.000 indicates 푝< 0.001 due to rounding. 5.1 Program-Level Confidence Remains High for Failing Generations Evaluation Setup. We next measure confidence over the generated code. For each solution, we compute average token confidence as the mean probability assigned to emitted tokens. To test whether failures are exposed by localized uncertainty, we also compute bottom-10% confidence, defined as the mean confidence of the least confident 10% of generated tokens. We compare passing and failing generations using mean, standard deviation, median, and Welch’s two-sample (t)-test. Results. Table 4 shows that both correct and incorrect programs receive high average token confidence. On HumanEval+ and MBPP+, average confidence is typically above 0.90 for both groups. Although correct solutions often have slightly higher confidence, the absolute gaps are small. For example, on HumanEval+, the correct–incorrect average-confidence gap ranges from 0.007 to 0.012 across models. On MBPP+, the gap is even smaller for several models, including OpenCoder, where correct and incorrect generations have nearly identical average confidence. The pattern becomes more striking on BigCodeBench. For CodeLlama and QwenCoder, incorrect generations havehigheraverage confidence than correct generations. CodeLlama assigns mean confidence 0.956 to incorrect generations and 0.943 to correct generations, while QwenCoder assigns 0.920 to incorrect generations and 0.897 to correct generations. These negative푡-statistics, which are statistically significant under standard two-sample푡-tests, indicate that the observed differences are unlikely to be due to random variation; rather, they reveal a systematic confidence–failure mismatch, where the models are most confident precisely when they are wrong. Finding. Generation-time confidence is high for both passing and failing programs. On controlled benchmarks, confidence gaps are small; on BigCodeBench, some models are more confident on failures than on correct solutions. 5.2 Local Token Confidence Provides Limited Failure Evidence Program-level confidence may hide isolated generation steps at which the model is uncertain. Therefore, we examine the least-confident tokens (i.e., the bottom 10% by confidence) within each generated program to determine whether failures contain localized uncertainty that is not reflected in the overall confidence score. Table 4 shows that bottom-10% confidence is lower than average confidence, as expected, but it still does not reliably separate correct and incorrect programs. On HumanEval+, correct solutions have higher bottom-10% confidence than incorrect solutions, but the distributions remain close. On MBPP+, the difference is weak for several models and statistically insignificant for OpenCoder. On BigCodeBench, the trend reverses for CodeLlama and QwenCoder: incorrect generations have substantially higher bottom-10% confidence than correct generations.Thus, local token confidence Characterizing Overconfident Failure in LLM-Based Code Generation9 (a) Average token entropy(b) Top-10% token entropy Fig. 2. Entropy distributions for OpenCoder on MBPP+. 1025507590100 Coverage (%) 40 50 60 70 80 Selective Accuracy (%) CodeLlamaDeepSeekOpenCoderQwenCoder (a) MBPP+ with avg. con- fidence 1025507590100 Coverage (%) 40 50 60 70 80 Selective Accuracy (%) CodeLlamaDeepSeekOpenCoderQwenCoder (b) MBPP+ with avg. en- tropy 1025507590100 Coverage (%) 0 10 20 30 Selective Accuracy (%) CodeLlamaDeepSeekOpenCoderQwenCoder (c) BigCodeBench with avg. confidence 1025507590100 Coverage (%) 0 10 20 30 Selective Accuracy (%) CodeLlamaDeepSeekOpenCoderQwenCoder (d) BigCodeBench with avg. entropy Fig. 3. Selective accuracy under uncertainty-based ranking. At each coverage level, we retain the most certain generations according to average confidence or average entropy and report their execution accuracy. Lower coverage corresponds to accepting a smaller subset of generations. provides evidence in some settings, but many incorrect programs remain locally confident even at their least confident tokens. Finding. Local token confidence is insufficient: incorrect programs can remain confident even in their least confident token regions. 5.3 Predictive Entropy Provides Limited Pass-Fail Separation We further examine predictive entropy as a complementary measure of generation-time uncertainty. At each generation step, predictive entropy is computed from the model’s probability distribution over all possible next tokens. Entropy is low when probability mass is concentrated on a small set of candidates and high when it is distributed more broadly across alternatives. Unlike token confidence, which considers only the probability assigned to the emitted token, predictive entropy accounts for the full next-token distribution. We test whether this broader uncertainty signal better distinguishes failing from passing generations. If predictive entropy reflects functional correctness, failing generations should exhibit systematically higher entropy than passing generations. Figure 2 presents representative entropy distributions for OpenCoder on MBPP+. Both average entropy and top-10% entropy show substantial overlap between correct and incorrect generations. Although the mean entropy for incorrect outputs is marginally higher, the distributions are not clearly separable. This observation aligns with the confidence analysis such that entropy captures some uncertainty, but not enough to reliably identify failing programs. Finding. Entropy provides the same qualitative picture as confidence. Failing programs often remain in low-uncertainty regions, and local high-entropy scores do not consistently isolate failures. 10Ravishka Rathnasuriya and Wei Yang BaseInstruction-tuned 0.0 0.2 0.4 0.6 0.8 1.0 Average Confidence Average Confidence PassFail BaseInstruction-tuned 0.0 0.2 0.4 0.6 0.8 1.0 1.2 Average Entropy Average Entropy PassFail (a) OpenCoder with avg. confidence BaseInstruction-tuned 0.0 0.2 0.4 0.6 0.8 1.0 Average Confidence Average Confidence PassFail BaseInstruction-tuned 0.0 0.2 0.4 0.6 0.8 1.0 1.2 Average Entropy Average Entropy PassFail (b) OpenCoder with avg. entropy ! ! " " ! ! " # " # (c) DeepSeekCoder with avg. confidence ! ! " " ! ! " # " # (d) DeepSeekCoder with avg. entropy Fig. 4. Uncertainty before and after instruction tuning on MBPP+. Each plot compares the distributions of passing and failing generations for the corresponding base and instruction-tuned model. 5.4 Uncertainty-Based Selective Generation We next evaluate whether uncertainty-based ranking can identify subsets of generated programs with higher execution accuracy. This analysis investigates whether overconfident failures persist among the generations assigned the highest certainty and therefore remain eligible for acceptance. Evaluation Setup. For each generated program, we use average confidence and average entropy as selection scores. We rank generations from most to least certain and retain the top푐% at coverage 푐 ∈ 10,25,50,75,90,100. For example, 10% coverage accepts only the 10% most certain generations and rejects the remaining 90%. We measure execution accuracy among the accepted programs. Therefore, selective generation trades coverage for accuracy. If uncertainty reflects functional correctness, removing uncertain generations should increase the accuracy of the accepted subset relative to the model’s original accuracy. If high-certainty failures remain, reducing coverage may provide little improvement or even reduce accuracy. Results. Figure 3 shows different selection behavior across models and benchmarks. On MBPP+, uncertainty identifies more accurate subsets for CodeLlama, DeepSeekCoder, and QwenCoder. For CodeLlama, confidence increases accuracy from 46.0% at 100% coverage to 65.79% at 10% coverage. QwenCoder increases from 65.9% to 80.00% at 25% coverage. Entropy produces a similar behavior for QwenCoder, reaching 78.95% at both 10% and 25% coverage. DeepSeekCoder shows only a marginal increase. However, for OpenCoder its original accuracy is 71.2%, yet confidence yields 71.05% at 10% coverage and 67.37% at 25%. Likewise, entropy provides similarly marginal improvement. BigCodeBench shows that this behavior does not persist across benchmarks. DeepSeekCoder and OpenCoder achieve modest accuracy improvements after uncertainty-based filtering. In contrast, the 10% most certain generations from CodeLlama and QwenCoder contain no passing solutions under either confidence or entropy where their original accuracies are 6.5% and 8.2%, respectively. For CodeLlama, no generation in the top 25% by confidence passes either. In these cases, uncertainty- based filtering reduces accuracy rather than improving it. The comparison further separates uncertainty-ranking quality from generation accuracy. Open- Coder has the highest original accuracy on MBPP+ but receives little accuracy improvement from selective generation. Conversely, models with lower original accuracy can obtain substantially more accurate retained subsets. Finding. Overconfident failures can occupy the high-certainty region of the uncertainty ranking. As a result, decreasing coverage does not consistently improve execution accuracy and can reduce it when uncertainty is misaligned with functional correctness. This misalignment is model- and benchmark-dependent and does not follow the model’s original generation accuracy. Characterizing Overconfident Failure in LLM-Based Code Generation11 5.5 Instruction Tuning Increases Certainty in Failing Generations We next examine how instruction tuning changes overconfidence relative to the corresponding base model. Specifically, we distinguish changes in the certainty assigned to failing generations from changes in how well uncertainty separates passing from failing programs. This distinction allows us to determine whether post-training improves uncertainty-based self-assessment or merely changes the magnitude of model certainty. Evaluation Setup. We compare base and instruction-tuned variants of OpenCoder and DeepSeek- Coder on MBPP+. We evaluate average confidence and average entropy for passing and failing generations and quantify their separation using AUC. To assess whether the change in discrimina- tion after instruction tuning is stable across tasks, we compute a paired task-level bootstrap 95% confidence interval forΔAUC, defined as the instruction-tuned AUC minus the base-model AUC. Results. Figure 4 shows how instruction tuning changes the confidence and entropy distributions of passing and failing generations for OpenCoder and DeepSeekCoder. Instruction tuning increases certainty on failing generations for both model families. For OpenCoder, the average confidence of failing programs increases from 0.8685 to 0.9224, while their average entropy decreases from 0.4754 to 0.2401. Passing generations exhibit a comparable shift. As a result, the additional certainty does not increase pass–fail separation. Confidence AUC changes from 0.5230 to 0.5094, withΔAUC =−0.0136 and a 95% confidence interval of[−0.0911,0.0652]. Entropy AUC changes from 0.4915 to 0.5237, withΔAUC=+0.0323 and an interval of[−0.0473,0.1135]. These intervals provide no evidence of a systematic change in discrimination. For DeepSeekCoder, instruction tuning also increases certainty on failing programs, but the uncertainty- correctness ordering changes. In the base model, confidence and entropy AUCs are 0.4391 and 0.4415, indicating that greater estimated certainty is associated with failing generations. After instruction tuning, the AUCs increase to 0.5654 and 0.5672. The correspondingΔAUC values are+0.1263 for confidence and+0.1257 for entropy, with 95% confidence intervals of[0.0447,0.2089] and[0.0442,0.2081], respectively. Therefore, instruction tuning changes the ordering from nega- tively aligned to positively aligned with correctness. However, the resulting AUCs remain below 0.57, indicating substantial pass-fail overlap. Finding. Instruction tuning can increase overconfidence relative to the base model by increasing certainty on failing generations. However, this change in certainty does not determine whether uncertainty becomes more or less informative about functional correctness. For example, DeepSeek- Coder becomes more certain on failing generations while its pass-fail discrimination improves. Thus, post-training can increase confidence in incorrect code independently of the reliability of uncertainty as a correctness signal. 5.6 Implications of Overconfidence Characterization Our characterization separates three properties of reliability-aware code generation: functional accuracy, uncertainty-based correctness discrimination, and its stability after post-training. Pass/fail evaluation measures only the first and the latter two determine whether model uncertainty remains informative for reliability decisions. Reliability evaluation should measure correctness discrimination separately from task accuracy. A model can achieve higher Pass@1 without becoming better at identifying which of its generations are correct. Therefore, evaluation should report whether uncertainty separates passing from failing programs in addition to reporting functional accuracy. Coverage-accuracy analysis provides a corresponding view by showing whether uncertainty ranking yields more accurate accepted subsets. 12Ravishka Rathnasuriya and Wei Yang Post-training objectives for correctness do not necessarily improve self-assessment. Our base-instruction comparison shows that model certainty and correctness discrimination can change independently after instruction tuning. Therefore, improving generation quality does not impose a corresponding constraint on confidence assigned to remaining failures. Reliability oriented post-training should treat generation correctness and uncertainty-based self-assessment as separate optimization targets. Uncertainty behavior should be re-evaluated after model updates. Post-training can alter both the magnitude and the ordering of uncertainty scores. An uncertainty-based selection mecha- nism validated for one model version may no longer exhibit the same correctness discrimination after the model is updated. Model evolution should include regression evaluation of both functional accuracy and uncertainty-correctness alignment. SE Takeaway. Overconfidence is not reduced automatically by improving code-generation accuracy. Post-training can increase functional performance while high-certainty failures persist, making overconfidence a distinct reliability property that must be assessed independently. 6 Limits of Mitigating Overconfident Failure This section answers RQ3 on the effectiveness of common mitigation techniques for reducing overconfident failure. We evaluate three practical responses: post-hoc calibration, repeated sampling, and explicit abstention prompting. These strategies address different parts of the reliability problem where calibration rescales scores, sampling increases the chance of finding a correct candidate, and abstention prompting gives the model an explicit option to defer. 6.1 Calibration Improves Probability Alignment but Provides Limited Predictive Skill Evaluation Setup. We examine to what extent post-hoc calibration mitigates overconfidence by mapping uncertainty scores to estimated probabilities of execution correctness. For confidence, we use average token confidence and bottom-10% token confidence. For entropy, we use average token entropy and top-10% token entropy. We calibrate each score using Platt scaling (PS) [39] which fits a logistic mapping from each original uncertainty score to an estimated probability of correctness and isotonic regression [17] which fits a monotonic non-parametric mapping from uncertainty scores to empirical correctness. We estimate calibrated probabilities using five-fold stratified cross-validation to avoid evaluating a calibrator on the generations used to fit it. Each calibrator is trained on 80% of the generations and applied to the held-out 20%. For the uncalibrated baseline, we use confidence measures directly as confidence proxies and transform entropy 퐻 to 푒 −퐻 . We use Brier score (BS) to measure the error between predicted pass probability and execution outcome [28,39,53–58,61–63]. Lower BS indicates better probability alignment. Because a constant predictor equal to the benchmark pass rate can also achieve a low BS, we additionally report Brier Skill Score (BSS) [39]. For pass rate푝 푟 , the reference score is퐵 ref = 푝 푟 (1−푝 푟 ), and the skill score is 퐵푆=(퐵 ref − 퐵푆)/퐵 ref . Positive BSS indicates improvement over the base-rate predictor, whereas zero indicates equivalent performance. Results. Table 5 reports the calibration results on MBPP+ and BigCodeBench. Across both benchmarks, calibration substantially reduces Brier score, but the resulting Brier Skill Scores remain minimal. On MBPP+, calibration substantially reduces BS, but the calibrated probabilities provide little skill beyond the benchmark pass rate. For CodeLlama, PS reduces the BS of average token confidence from 0.459 to 0.248, while BSS is approximately 0.001. DeepSeekCoder shows a similar pattern where BS decreases from 0.300 to 0.224, but BSS remains approximately 0.001. For OpenCoder, BS decreases from 0.250 to 0.205 while BSS remains near zero. QwenCoder obtains Characterizing Overconfident Failure in LLM-Based Code Generation13 Table 5. Calibration results measured by Brier Score (BS;↓) and Brier Skill Score (BSS;↑). Each entry reports BS / BSS. For each model and metric, the lowest BS and highest BSS across Before, PS, and Iso are shown in bold. MBPP+BigCodeBench ModelMethodAvg. Conf.Avg. Ent.Bottom-10% Conf.Top-10% Ent.Avg. Conf.Avg. Ent.Bottom-10% Conf.Top-10% Ent. CodeLlama Before0.459 / -0.8490.345 / -0.3870.252 / -0.0150.284 / -0.1420.856 / -12.926 0.699 / -10.3790.548 / -7.9220.343 / -4.578 PS0.248 / +0.001 0.247 / +0.005 0.248 / +0.004 0.246 / +0.008 0.061 / +0.001 0.061 / -0.000 0.060 / +0.0170.062 / -0.002 Iso0.253 / -0.0200.251 / -0.0120.257 / -0.0350.255 / -0.0250.062 / -0.0020.062 / -0.0010.062 / -0.001 0.061 / +0.002 DeepSeekCoder Before0.300 / -0.3410.253 / -0.1300.228 / -0.0190.321 / -0.4350.659 / -3.0240.481 / -1.9400.252 / -0.5410.169 / -0.035 PS0.224 / +0.001 0.222 / +0.007 0.223 / +0.0040.222 / +0.0080.164 / +0.000 0.163 / +0.001 0.163 / +0.003 0.163 / +0.001 Iso0.226 / -0.0100.223 / +0.0040.228 / -0.017 0.221 / +0.013 0.162 / +0.0100.164 / -0.0010.164 / -0.0040.164 / -0.002 OpenCoder Before0.250 / -0.2190.213 / -0.0390.236 / -0.1490.378 / -0.8400.666 / -2.6670.532 / -1.9310.305 / -0.6770.200 / -0.099 PS0.205 / -0.000 0.205 / +0.0010.205 / -0.000 0.203 / +0.008 0.182 / +0.000 0.182 / +0.000 0.182 / -0.000 0.182 / -0.001 Iso0.210 / -0.0250.205 / -0.000 0.205 / +0.0000.209 / -0.0160.183 / -0.0060.183 / -0.0050.182 / -0.0030.183 / -0.006 QwenCoder Before0.273 / -0.2140.218 / +0.0310.254 / -0.1280.426 / -0.8960.781 / -9.3180.535 / -6.0680.374 / -3.9410.152 / -1.004 PS0.223 / +0.010 0.217 / +0.0360.219 / +0.024 0.219 / +0.024 0.075 / +0.0030.076 / -0.003 0.074 / +0.016 0.076 / +0.002 Iso0.218 / +0.029 0.220 / +0.022 0.216 / +0.0410.224 / +0.0050.076 / +0.000 0.076 / -0.0030.076 / -0.0020.076 / -0.005 higher positive skill in some settings, although its largest BSS is only 0.041 for bottom-10% token confidence after isotonic regression. On BigCodeBench, calibration produces large reductions in BS while BSS remains near zero across models. For CodeLlama, PS reduces the BS of average token confidence from 0.856 to 0.061, but the corresponding BSS is approximately 0.001. Therefore, the calibrated BS is nearly equivalent to the error of the base-rate predictor. The remaining models exhibit the same pattern, with calibrated BSS values generally close to zero and the largest values reaching only about 0.01–0.02. The BS and BSS results show that calibration primarily corrects the scale of the uncertainty scores rather than providing substantial predictive information about individual generations. This interpretation is consistent with our selective-accuracy analysis, where passing and failing generations remain overlapping in the uncertainty space. Post-hoc calibration can remap these scores to better-aligned probabilities, but it cannot remove the underlying overlap between correct and incorrect generations. Finding. Post-hoc calibration corrects the magnitude of overconfidence but does not resolve overconfident failures. Although calibration substantially improves probability alignment, the calibrated scores provide little skill beyond the benchmark pass rate and remain unable to reliably distinguish passing from failing generations. 6.2 Reliability Under Finite Sampling Budgets Evaluation Setup. We next evaluate whether repeated sampling can compensate for overconfident single generations. We run four code models on MBPP+ and generate up to (K=50) candidate programs per task using temperature sampling with (T=0.8). Each candidate is executed against the task test suite. For each task, we record whether a passing solution appears within the first (k) samples and how many passing candidates appear within the full budget. Results. Figure 5 shows that sampling reduces the number of unsolved tasks quickly at small budgets, but the curves flatten as (K) increases. Even after 50 attempts, many tasks remain un- solved, i.e., roughly 90 for CodeLlama, about 57 for DeepSeek, and around 45–50 for OpenCoder and QwenCoder. Figure 5b shows that sampling mostly separates tasks into reliably solved and persistently unsolved groups, with fewer partially recoverable cases. These results characterize sampling as a bounded recovery mechanism rather than a reliability mechanism. Additional samples can recover failures that are sensitive to decoding variation, but the flattening curves show that many tasks remain unresolved even after substantial retry budget. In a software-engineering workflow, each retry also incurs generation cost, latency, and additional 14Ravishka Rathnasuriya and Wei Yang 161116212631364146 number of generated samples considered 50 75 100 125 150 175 200 225 # tasks still failing EvalPlus+: tasks still failing vs k CodeLlama DeepSeek QwenCoder OpenCoder (a) Tasks still failing vs. 푘 . 05101520253035404550 number of plus-passes 0 20 40 60 80 100 120 140 number of tasks EvalPlus+: per-task pass-count distribution CodeLlama DeepSeek QwenCoder OpenCoder (b) Per-task pass-count distribution (퐾=50). Fig. 5. Sampling outcomes under EvalPlus+. (a) Remaining-unsolved tasks after the first푘generations. (b) Distribution of how many successes each task attains within 퐾=50 generations. validation effort. Thus, sampling does not remove the accept-or-defer decision; it shifts the decision to budget allocation. A practical system must decide when another sample is likely to help, when a task should be escalated, and how to identify which candidate is trustworthy when multiple candidates are available. Finding. Sampling expands the candidate set but does not solve candidate selection or persistent failure. 6.3 Explicit Abstention Prompting Does Not Reliably Induce Deferral Evaluation Setup. We test whether explicit prompting can make a model defer when it is unlikely to be correct. We use a penalty-aware prompt that instructs the model to answer only when it is more than 95% confident and to otherwise output “I don’t know”. This tests whether a code LLM can convert uncertainty into abstention without additional control logic. Results. Figure 6 shows a representative failure on MBPP/72 for OpenCoder. The prompt instructs the model to answer only when confident and assigns a high penalty to mistakes, while allowing “I don’t know” as a zero-cost response. Nevertheless, the model generates code rather than deferring. The generated program checks whether the input can be represented as a sum of two squares (line 5), while the task asks for the difference of two squares. We observe the same qualitative pattern across the subject models: even when abstention is explicitly allowed, the standard code-LLMs behavior favors producing a concrete program instead of deferring. In SE terms, the model continues to behave as a generator rather than as a reliability-aware component. Finding. Prompt-level abstention does not reliably induce deferral; models can still generate incorrect code even when an IDK option is provided. 6.4 Implications Our results suggest that generation quality and correctness-sensitive confidence are distinct relia- bility properties. Improving the probability of generating correct code does not necessarily improve the separation between correct and incorrect generations in confidence. Overconfidence exposes this gap because incorrect generations can remain highly confident even when generation quality or probability alignment improves. SE Takeaway. Code-generation reliability should be evaluated along both dimensions. Pass rate measures whether a model produces correct programs, whereas overconfident failure captures whether incorrect programs remain highly trusted by the model’s confidence signal. Improvement in generation quality should not be interpreted as a corresponding reduction in overconfidence. Research Insight. Overconfidence motivates treating correctness-sensitive confidence as an explicit learning objective rather than an expected consequence of better code generation. Future Characterizing Overconfident Failure in LLM-Based Code Generation15 #Prompt: Answer only if you are more than 95% confident. Mistakes are penalized by 100 points. A correct answer receives 1 point. An answer of "I don't know" receives 0 points. Write a python function to check whether the given number can be represented as the difference of two squares or not. assert dif_Square(5) == True Final answer is: def dif_Square(n) : i = 0 j = int(n ** 0.5) while i <= j: if i * i + j * j == n: return True elif i * i + j * j < n: i += 1 else: j -= 1 return False #Code 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. Fig. 6. Ablation: Explicit abstention prompting still yields incorrect code instead of deferral on MBPP/72 for OpenCoder. 0102030 Layer 0.75 0.80 0.85 0.90 Entropy Entropy Pass Fail 0102030 Layer 50 60 70 80 Effective Rank Effective Rank Pass Fail 0102030 Layer 0.10 0.15 0.20 Anisotropy Anisotropy Pass Fail 0102030 Layer 2.5 5.0 7.5 Intrinsic Dimension Intrinsic Dimension Pass Fail Prompt-conditioned hidden-state metrics for passing vs failing generated code Input: prompt + generated solution; metrics computed only over generated solution tokens. (a) CodeLlama. 01020 Layer 0.75 0.80 0.85 0.90 Entropy Entropy Pass Fail 01020 Layer 70 80 Effective Rank Effective Rank Pass Fail 01020 Layer 0.10 0.15 0.20 Anisotropy Anisotropy Pass Fail 01020 Layer 2 4 6 8 Intrinsic Dimension Intrinsic Dimension Pass Fail Prompt-conditioned hidden-state metrics for passing vs failing generated code Input: prompt + generated solution; metrics computed only over generated solution tokens. (b) QwenCoder. Fig. 7. Layerwise effective rank for CodeLlama and QwenCoder on HumanEval+. techniques should aim to reduce confidence on incorrect generations while preserving confidence on correct generations. This formulation extends the reliability objective from producing correct code to distinguishing which generated code can be trusted. 7 Latent-Space Evidence of Overconfident Failure This section addresses RQ4 by investigating overconfident failure signals in latent representation- space of code-LLMs. Evaluation Setup. Inspired by recent latent-space analyses of reasoning models [64], we analyze the latent representation space across transformer layers. For each generated solution, we feed the prompt and generated code to the model and compute hidden-state statistics only over the generated solution tokens. We focus on effective rank [65], which summarizes how many representation directions are effectively used at a layer. Given a hidden-state matrix (퐻 ℓ ) at layer (ℓ), effective rank is computed from the normalized singular-value spectrum of (퐻 ℓ ). Higher effective rank indicates a more dispersed representation, while lower effective rank indicates a more compact representation. Results. Figure 7 compares layerwise effective rank for CodeLlama and QwenCoder on Hu- manEval+. Although the models operate at different pass@1 levels, both show the same pattern: failing generations have higher effective rank than passing generations across many middle and later layers. This indicates that failing generations induce a less compact hidden-state geometry. This contrasts with RQ2, where output-token confidence often failed to separate passing from failing programs. The result suggests that correctness-related variation may be partially encoded internally but not exposed through token probabilities. Finding. Passing and failing generations differ in latent representation-space geometry even when output confidence remains unreliable. SE Takeaway. A confident decoding trace should not be interpreted as a low-risk program. Representation-level signals may help future tools identify candidates that appear reliable from token probabilities but still require validation. Research Insight. The effective-rank separation suggests a representation-aware direction for mitigating overconfident failure. If failing generations occupy a different latent-state regime, then correctness risk is partially represented internally but not coupled to decoding behavior. Future SE research should therefore study such objectives that make these internal risk signals operational: lowering confidence, triggering deferral, or routing candidates to validation when hidden-state geometry indicates likely failure. This shifts uncertainty from an external post-hoc score to a learned mapping from representations to reliability actions. 16Ravishka Rathnasuriya and Wei Yang 8 Discussion Our findings suggest three directions for reducing overconfident failure: training, inference-time control, and reliability evaluation. 1) Train models to recognize likely failures. Our post-training results show that better generation does not imply better discrimination between passing and failing programs. Instruction tuning increases certainty on failing generations for both model families, while its effect on pass–fail discrimination is inconsistent. Therefore, future post-training should optimize not only whether the model generates correct code, but also whether it lowers confidence or defers when its generated code is likely incorrect. This direction requires execution-grounded training data. Each training example should connect a task and generated candidate to an execution outcome, so that high-confidence failures can be identified explicitly. Matched passing and failing candidates for the same task are particularly useful because they control for differences in task difficulty. Such data can support objectives that penalize confident failures and reward appropriate deferral. Our latent-space analysis provides an additional opportunity where passing and failing generations exhibit different internal representations even when output confidence overlaps, suggesting that these internal signals may support learned failure prediction rather than relying only on token probabilities. 2) Couple model confidence with external evidence at inference time. Our selective- generation results show that high confidence alone is not sufficient for acceptance. In some settings, the most certain generations contain disproportionately many failures. Therefore, inference-time systems should not map confidence directly to “accept” or “defer.” Instead, confidence should be one input to a control policy that determines what additional evidence is needed. At inference time, the system should determine what additional evidence is needed before accept- ing generated code. We distinguish three such cases. First, when the task specification is incomplete or ambiguous, the system should seek clarification from the developer. Second, when correctness depends on missing external information, the system should retrieve or verify that information. Third, when the risk lies in the generated implementation, the system should obtain program- level evidence through testing, analysis, or an independent evaluator. Repeated generation can support this process, but our results show that sampling alone does not resolve candidate selection and eventually reaches persistent failures. Therefore, an inference-time controller should choose among clarification, validation, regeneration, and deferral according to the expected reliability improvement and its cost. 3) Evaluate reliability through system decisions. Our results show that score-level improve- ments can overstate practical reliability. Calibration substantially reduces BS while BSS remains near zero in many settings, and explicit IDK prompting does not reliably induce deferral. Thus, future evaluation should assess whether uncertainty leads to better acceptance and validation decisions rather than only whether a confidence score is numerically well aligned. A reliability benchmark should evaluate an end-to-end policy that can accept generated code, obtain additional validation, retry generation, or defer. The evaluation should vary the cost of accepting incorrect code and the available validation budget. The central outcome is whether the system reduces false acceptance without relying on indiscriminate deferral or unlimited valida- tion. Such evaluation would measure whether a code-generation system uses reliability evidence appropriately under deployment constraints, rather than only whether it produces a calibrated score. These three directions target different causes of overconfident failure. Training determines whether the model identifies likely failures, inference-time control determines how the system responds to uncertain evidence, and evaluation determines whether those responses reduce false Characterizing Overconfident Failure in LLM-Based Code Generation17 acceptance. Ultimately, an intervention addresses overconfidence only to the extent that it reduces failing programs that remain eligible for acceptance, without relying on indiscriminate deferral or unbounded validation. 9 Limitations Our study has three main limitations. First, we evaluate four open-source code LLMs in the 7–8B range because our analyses require token probabilities and hidden representations and whether the findings extend to larger proprietary models remains open. Second, we study single-generation code synthesis rather than repository-scale or agentic workflows, where tool use and iterative feedback may change how overconfidence emerges. Third, our evaluation is limited to Python benchmarks with execution-based test suites. These provide practical correctness labels but do not establish complete semantic correctness or cover other languages and development settings. 10 Threats to Validity This section discusses potential threats to validity in our study, organized into four standard categories: internal, external, construct, and conclusion validity. These categories address risks pertaining to our experimental design and methodology, as well as the generalizability of our findings. Internal validity. Our results depend on the correct implementation of uncertainty metrics, calibration methods, sampling procedures, and execution-based evaluation. We mitigate this threat by following standard definitions from prior work [28,53–58,61,62], reusing established implemen- tations when available, and applying the same evaluation pipeline across models and benchmarks. For stochastic procedures, such as sampling-based uncertainty and repeated generation, we use fixed decoding settings within each experiment and compare trends across multiple models. Never- theless, implementation details such as token filtering, sequence truncation, and score orientation may affect individual metric values. External validity. We evaluate four open-source code LLMs on HumanEval+, MBPP+, and Big- CodeBench analyses. These benchmarks provide executable correctness labels and allow controlled comparison of uncertainty signals, but they do not cover all software-engineering settings. Our findings may not fully generalize to larger proprietary models, project-level development tasks, other programming languages, or agentic workflows with tool use and iterative repair. Metrics requiring token probabilities or hidden states also assume white-box model access, which may not be available for API-only models. Extending this analysis to closed models and repository-scale tasks remains future work. Construct validity. We operationalize correctness using benchmark test suites. Although Hu- manEval+ and MBPP+ strengthen the original tests, execution outcomes may still miss untested behaviors. Thus, our labels approximate functional correctness rather than prove semantic equiva- lence. We also treat token confidence, entropy, calibration scores, sampling behavior, abstention, and latent effective rank as proxies for reliability-related evidence. These constructs do not directly measure program correctness; they measure whether model-derived signals align with execution outcomes. This distinction is central to our study, but it limits claims about absolute correctness. Conclusion validity. Our conclusions are based on trends observed across multiple models, benchmarks, uncertainty families, and mitigation strategies. The main findings do not depend on a single metric: weak failure detection, high-confidence failures, limited local-token separation, bounded sampling gains, and unreliable prompt-level abstention appear across settings. However, exact effect sizes may vary with decoding parameters, calibration splits, thresholds, and benchmark composition. We therefore avoid claiming that uncertainty is useless. Our conclusion is more 18Ravishka Rathnasuriya and Wei Yang specific: the evaluated model-derived signals are insufficient as standalone reliability mechanisms for deciding that generated code is likely to pass execution-based validation. 11 Related Work Uncertainty estimation in ML and NLG. Uncertainty estimation has been widely studied in classification, using confidence scores, entropy, Bayesian approximations, ensembles, and cali- bration methods to identify unreliable predictions [17–27]. Recent work extends these ideas to natural-language generation through token uncertainty, sequence likelihood, sample diversity, semantic uncertainty, and judge-based estimates [28–33,57,66–70]. Code generation differs be- cause correctness is determined by executable program behavior rather than textual plausibility or subjective quality. Our work studies whether these uncertainty signals remain useful under execution-based correctness. Uncertainty and calibration for code models. Prior work has studied uncertainty, calibration, and reliability signals for code models [10–13,33,38–45,71,72]. These studies show that code models can be miscalibrated and that post-hoc methods can improve confidence alignment. Our focus is different: we study whether model-derived uncertainty can support reliability decisions for generated code, such as identifying likely failures, abstaining, or deciding when additional validation is needed. This setting is distinct from uncertainty in non-generative SE tasks such as defect prediction or vulnerability detection [72,73], where predictions are labels over existing artifacts rather than generated programs whose correctness is observed only after execution. Code-generation evaluation and mitigation. Code-generation research commonly evaluates models with pass@1, pass@k, execution feedback, repair, and prompting strategies [10–13,50,51]. These methods measure or improve the ability to produce correct code, but they do not fully address whether a model can recognize when its own output is likely to fail. Sampling can increase the chance that a correct candidate exists, but does not identify which candidate should be trusted. Prompt-level abstention can expose a deferral option, but does not guarantee safe generate-or-defer behavior. We evaluate these techniques as responses to overconfident failure rather than as general performance-improvement methods. Summary. Our work provides a rigorous characterization of thedilemmaofoverconfidence in code LLMs: model-derived uncertainty is available during generation, but it does not reliably identify generated programs that fail execution-based evaluation. The results position overconfident failure as a deployment-facing SE reliability problem and motivate future mechanisms that connect model self-assessment to execution-grounded correctness. 12 Conclusion This paper characterizes thedilemmaofoverconfidencein execution-grounded code generation. We show that model-derived uncertainty often fails to identify incorrect programs, even under calibration, sampling, and abstention prompting. These results motivate behavior-aware evaluation that measures high-confidence failures, generate-or-defer behavior, and reliability under limited validation budgets. Characterizing Overconfident Failure in LLM-Based Code Generation19 References [1]B. Roziere, J. Gehring, F. Gloeckle, S. Sootla, I. Gat, X. E. Tan, Y. Adi, J. Liu, R. Sauvestre, T. Remezetal., “Code llama: Open foundation models for code,”arXivpreprintarXiv:2308.12950, 2023. [2] D. Guo, Q. Zhu, D. Yang, Z. Xie, K. Dong, W. Zhang, G. Chen, X. Bi, Y. Wu, Y. Lietal., “Deepseek-coder: When the large language model meets programming–the rise of code intelligence,”arXivpreprintarXiv:2401.14196, 2024. [3]X. Jiang, Y. Dong, L. Wang, Z. Fang, Q. Shang, G. Li, Z. Jin, and W. Jiao, “Self-planning code generation with large language models,”ACMTransactionsonSoftwareEngineeringandMethodology, vol. 33, no. 7, p. 1–30, 2024. [4]J. Liu, C. S. Xia, Y. Wang, and L. Zhang, “Is your code generated by chatGPT really correct? rigorous evaluation of large language models for code generation,” inThirty-seventhConferenceonNeuralInformationProcessing Systems, 2023. [Online]. Available: https://openreview.net/forum?id=1qvx610Cu7 [5]J. Liu, S. Xie, J. Wang, Y. Wei, Y. Ding, and L. Zhang, “Evaluating language models for efficient code generation,” in FirstConferenceonLanguageModeling, 2024. [Online]. Available: https://openreview.net/forum?id=IBCBMeAhmC [6]GitHub, “GitHub Copilot: Your AI pair programmer,” 2021, accessed: 2025-07-18. [Online]. Available: https: //github.com/features/copilot [7] OpenAI, “OpenAI Codex,” 2021, accessed: 2025-07-18. [Online]. Available: https://openai.com/blog/openai-codex [8] DeepMind, “AlphaEvolve: Evolutionary Algorithm Synthesis with LLMs,” 2025. [9] —, “AlphaCode: Competitive Programming with AI,” 2022, accessed: 2025-07-18. [Online]. Available: https: //w.deepmind.com/research/highlighted-research/alphacode [10]H.-S. Li, P. Fernandes, I. Gurevych, and A. F. Martins, “Doce: Finding the sweet spot for execution-based code generation,”arXivpreprintarXiv:2408.13745, 2024. [11]Y. Peng, A. D. Gotmare, M. R. Lyu, C. Xiong, S. Savarese, and D. Sahoo, “Perfcodegen: Improving performance of llm generated code with execution feedback,” in2025IEEE/ACMSecondInternationalConferenceonAIFoundation ModelsandSoftwareEngineering(Forge). IEEE, 2025, p. 1–13. [12] A. Ni, S. Iyer, D. Radev, V. Stoyanov, W.-t. Yih, S. Wang, and X. V. Lin, “Lever: Learning to verify language-to-code generation with execution,” inInternationalConferenceonMachineLearning. PMLR, 2023, p. 26 106–26 128. [13] Z. Tian, J. Chen, and X. Zhang, “Fixing large language models’ specification misunderstanding for better code generation,” in2025IEEE/ACM47thInternationalConferenceonSoftwareEngineering(ICSE).IEEE Computer Society, 2025, p. 645–645. [14]F. Yao, Z. Wang, L. Liu, J. Cui, L. Zhong, X. Fu, H. Mai, V. Krishnan, J. Gao, and J. Shang, “Training language models to generate quality code with program analysis feedback,”arXivpreprintarXiv:2505.22704, 2025. [15] D. Shaikhelislamov, M. Drobyshevskiy, and A. Belevantsev, “Codepatchllm: Configuring code generation using a static analyzer,”GenAIEvaluationKDD2024, 2024. [16]P. J. Chapman, C. Rubio-González, and A. V. Thakur, “Interleaving static analysis and llm prompting,” inProceedings ofthe13thACMSIGPLANInternationalWorkshopontheStateOftheArtinProgramAnalysis, 2024, p. 9–17. [17] C. Guo, G. Pleiss, Y. Sun, and K. Q. Weinberger, “On calibration of modern neural networks,” 2017. [18]H. Wang, J. Xu, C. Xu, X. Ma, and J. Lu, “Dissector: Input validation for deep learning applications by crossing- layer dissection,” inProceedingsoftheACM/IEEE42ndInternationalConferenceonSoftwareEngineering, 2020, p. 727–738. [19]D. Hendrycks and K. Gimpel, “A baseline for detecting misclassified and out-of-distribution examples in neural networks,” 2018. [20] Y. Gal and Z. Ghahramani, “Dropout as a bayesian approximation: Representing model uncertainty in deep learning,” ininternationalconferenceonmachinelearning. PMLR, 2016, p. 1050–1059. [21]U. Alon, M. Zilberstein, O. Levy, and E. Yahav, “code2vec: Learning distributed representations of code,”Proceedings oftheACMonProgrammingLanguages, vol. 3, no. POPL, p. 1–29, 2019. [22] Y. Xiao and W. Y. Wang, “Quantifying uncertainties in natural language processing tasks,” inProceedingsoftheAAAI conferenceonartificialintelligence, vol. 33, no. 01, 2019, p. 7322–7329. [23]V. T. Vasudevan, A. Sethy, and A. R. Ghias, “Towards better confidence estimation for neural models,” inICASSP 2019-2019IEEEInternationalConferenceonAcoustics,SpeechandSignalProcessing(ICASSP).IEEE, 2019, p. 7335–7339. [24]C. Corbière, N. Thome, A. Bar-Hen, M. Cord, and P. Pérez, “Addressing failure prediction by learning model confidence,” AdvancesinNeuralInformationProcessingSystems, vol. 32, 2019. [25]R. M. Monarch,Human-in-the-LoopMachineLearning:Activelearningandannotationforhuman-centeredAI. Si- mon and Schuster, 2021. [26]J. Steinhardt and P. S. Liang, “Unsupervised risk estimation using only conditional independence structure,”Advances inNeuralInformationProcessingSystems, vol. 29, 2016. [27]C. E. Shannon, “A mathematical theory of communication,”TheBellsystemtechnicaljournal, vol. 27, no. 3, p. 379–423, 1948. 20Ravishka Rathnasuriya and Wei Yang [28]R. Vashurin, E. Fadeeva, A. Vazhentsev, L. Rvanova, A. Tsvigun, D. Vasilev, R. Xing, A. B. Sadallah, K. Grishchenkov, S. Petrakovetal., “Benchmarking uncertainty quantification methods for large language models with lm-polygraph,” arXivpreprintarXiv:2406.15627, 2024. [29]Q. Xie, Q. Li, Z. Yu, Y. Zhang, Y. Zhang, and L. Yang, “An empirical analysis of uncertainty in large language model evaluations,”arXivpreprintarXiv:2502.10709, 2025. [30]Z. Xia, J. Xu, Y. Zhang, and H. Liu, “A survey of uncertainty estimation methods on large language models,”arXiv preprintarXiv:2503.00172, 2025. [31]Y. Bakman, D. N. Yaldiz, S. Kang, T. Zhang, B. Buyukates, S. Avestimehr, and S. P. Karimireddy, “Reconsidering llm uncertainty estimation methods in the wild,”arXivpreprintarXiv:2506.01114, 2025. [32] M. Ielanskyi, K. Schweighofer, L. Aichberger, and S. Hochreiter, “Addressing pitfalls in the evaluation of uncertainty estimation methods for natural language generation,” inICLRWorkshop:QuantifyUncertaintyandHallucinationin FoundationModels:TheNextFrontierinReliableAI, 2025. [33] Y. Huang, J. Song, Z. Wang, S. Zhao, H. Chen, F. Juefei-Xu, and L. Ma, “Look before you leap: An exploratory study of uncertainty analysis for large language models,”IEEETransactionsonSoftwareEngineering, 2025. [34]J. Xin, R. Tang, Y. Yu, and J. Lin, “The art of abstention: Selective prediction and error regularization for natural language processing,” inProceedingsofthe59thAnnualMeetingoftheAssociationforComputationalLinguisticsandthe11th InternationalJointConferenceonNaturalLanguageProcessing(Volume1:LongPapers), 2021, p. 1040–1051. [35] B. Calder, G. Reinman, and D. M. Tullsen, “Selective value prediction,” inProceedingsofthe26thannualinternational symposiumoncomputerarchitecture, 1999, p. 64–74. [36]Z. Gu and M. Hopkins, “On the evaluation of neural selective prediction methods for natural language processing,” in Proceedingsofthe61stAnnualMeetingoftheAssociationforComputationalLinguistics(Volume1:LongPapers), 2023, p. 7888–7899. [37]A. T. Kalai, O. Nachum, S. S. Vempala, and E. Zhang, “Why language models hallucinate,”arXivpreprint arXiv:2509.04664, 2025. [38]H. Vasconcelos, G. Bansal, A. Fourney, Q. V. Liao, and J. W. Vaughan, “Generation probabilities are not enough: Improving error highlighting for ai code suggestions,” inHCAIWorkshopatNeurIPS, 2022. [39] C. Spiess, D. Gros, K. S. Pai, M. Pradel, M. R. I. Rabin, A. Alipour, S. Jha, P. Devanbu, and T. Ahmed, “Calibration and correctness of language models for code,”arXivpreprintarXiv:2402.02047, 2024. [40] Z. Zhou, C. Sha, and X. Peng, “On calibration of pre-trained code models,” inProceedingsoftheIEEE/ACM46th internationalconferenceonsoftwareengineering, 2024, p. 1–13. [41]A. Sharma and C. David, “Assessing correctness in llm-based code generation via uncertainty estimation,”arXiv preprintarXiv:2502.11620, 2025. [42] D. D. Johnson, D. Tarlow, and C. Walder, “R-u-SURE? Uncertainty-aware code suggestions by maximizing utility across random user intents,” inProceedingsofthe40thInternationalConferenceonMachineLearning, ser. Proceedings of Machine Learning Research, A. Krause, E. Brunskill, K. Cho, B. Engelhardt, S. Sabato, and J. Scarlett, Eds., vol. 202. PMLR, 23–29 Jul 2023, p. 15 262–15 306. [Online]. Available: https://proceedings.mlr.press/v202/johnson23a.html [43]R. Rathnasuriya and W. Yang, “When to answer and when to defer: A decision framework for reliable code predictions,” inProceedingsoftheIEEE/ACM48thInternationalConferenceonSoftwareEngineering, 2026, p. 231–235. [44] Y. Shi, C. Zhang, Y. Li, H. Wang, Y. Chen, N. Collier, and X. Gu, “Code is more than text: Uncertainty estimation for code generation,”arXivpreprintarXiv:2606.09577, 2026. [45] P. Rakasi, M. Lalwani, A. Srivastava, A. S. Palanivel, T. Adeleke, S. Wu, and R. Li, “When uncertainty isn’t enough: An empirical study of self-correction in code generation,” inICML2026StatisticalFrameworksforUncertaintyin AgenticSystems, 2026. [46]M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. d. O. Pintoetal., “Evaluating large language models trained on code,” Jul. 2021. [47]J. Liu, C. S. Xia, Y. Wang, and L. Zhang, “Is your code generated by chatgpt really correct? rigorous evaluation of large language models for code generation,”AdvancesinNeuralInformationProcessingSystems, vol. 36, p. 21 558–21 572, 2023. [48]Y. Dong, J. Ding, X. Jiang, G. Li, Z. Li, and Z. Jin, “Codescore: Evaluating code generation by learning code execution,” ACMTransactionsonSoftwareEngineeringandMethodology, vol. 34, no. 3, p. 1–22, 2025. [49] T. Y. Zhuo, M. C. Vu, J. Chim, H. Hu, W. Yu, R. Widyasari, I. N. B. Yusuf, H. Zhan, J. He, I. Pauletal., “Bigcodebench: Benchmarking code generation with diverse function calls and complex instructions,” inInternationalConferenceon LearningRepresentations, vol. 2025, 2025, p. 66 602–66 656. [50] T. Zheng, G. Zhang, T. Shen, X. Liu, B. Y. Lin, J. Fu, W. Chen, and X. Yue, “Opencodeinterpreter: Integrating code generation with execution and refinement,”arXivpreprintarXiv:2402.14658, 2024. [51]F. Liu, Y. Liu, L. Shi, H. Huang, R. Wang, Z. Yang, L. Zhang, Z. Li, and Y. Ma, “Exploring and evaluating hallucinations in llm-powered code generation,”arXivpreprintarXiv:2404.00971, 2024. Characterizing Overconfident Failure in LLM-Based Code Generation21 [52]W. Wang, C. Yang, Z. Wang, Y. Huang, Z. Chu, D. Song, L. Zhang, A. R. Chen, and L. Ma, “Testeval: Benchmarking large language models for test case generation,”arXivpreprintarXiv:2406.04531, 2024. [53] M. Darrin, P. Piantanida, and P. Colombo, “Rainproof: An umbrella to shield text generators from out-of-distribution data,”arXivpreprintarXiv:2212.09171, 2022. [54]L. Van der Poel, R. Cotterell, and C. Meister, “Mutual information alleviates hallucinations in abstractive summarization,” arXivpreprintarXiv:2210.13210, 2022. [55] J. Takayama and Y. Arase, “Relevant and informative response generation using pointwise mutual information,” in ProceedingsoftheFirstWorkshoponNLPforConversationalAI, 2019, p. 133–138. [56]M. Fomicheva, S. Sun, L. Yankovskaya, F. Blain, F. Guzmán, M. Fishel, N. Aletras, V. Chaudhary, and L. Specia, “Unsupervised quality estimation for neural machine translation,”TransactionsoftheAssociationforComputational Linguistics, vol. 8, p. 539–555, 2020. [57] L. Kuhn, Y. Gal, and S. Farquhar, “Semantic uncertainty: Linguistic invariances for uncertainty estimation in natural language generation,” arXivpreprintarXiv:2302.09664, 2023. [58]Z. Lin, S. Trivedi, and J. Sun, “Generating with confidence: Uncertainty quantification for black-box large language models,”arXivpreprintarXiv:2305.19187, 2023. [59] J. Gu, X. Jiang, Z. Shi, H. Tan, X. Zhai, C. Xu, W. Li, Y. Shen, S. Ma, H. Liuetal., “A survey on llm-as-a-judge,”The Innovation, 2024. [60] CodeCalibration, “Codecalibration.” [Online]. Available: https://codeairesearch.github.io/CodeGenUncertainty/ [61] E. Fadeeva, A. Rubashevskii, A. Shelmanov, S. Petrakov, H. Li, H. Mubarak, E. Tsymbalov, G. Kuzmin, A. Panchenko, T. Baldwinetal., “Fact-checking the output of large language models via token-level uncertainty quantification,”arXiv preprintarXiv:2403.04696, 2024. [62] J. Duan, H. Cheng, S. Wang, A. Zavalny, C. Wang, R. Xu, B. Kailkhura, and K. Xu, “Shifting attention to relevance: Towards the predictive uncertainty quantification of free-form large language models,”arXivpreprintarXiv:2307.01379, 2023. [63]S. Devic, T. Srinivasan, J. Thomason, W. Neiswanger, and V. Sharan, “From calibration to collaboration: Llm uncertainty quantification should be more human-centered,”arXivpreprintarXiv:2506.07461, 2025. [64] H. Du, Y. Dong, and X. Ning, “Latent thinking optimization: Your latent reasoning language model secretly encodes reward signals in its latent thoughts,”arXivpreprintarXiv:2509.26314, 2025. [65] L. Wei, Z. Tan, C. Li, J. Wang, and W. Huang, “Diff-erank: A novel rank-based metric for evaluating large language models,”AdvancesinNeuralInformationProcessingSystems, vol. 37, p. 39 501–39 521, 2024. [66]X. Hu, J. Chen, X. Li, Y. Guo, L. Wen, P. S. Yu, and Z. Guo, “Towards understanding factual knowledge of large language models,” inThetwelfthinternationalconferenceonlearningrepresentations, 2024. [67] Y. Li, K. Zhou, Q. Qiao, B. Nguyen, Q. Wang, and Q. Li, “Investigating context-faithfulness in large language models: The roles of memory strength and evidence style,”arXivpreprintarXiv:2409.10955, 2024. [68]M. Hu, Z. Zhang, S. Zhao, M. Huang, and B. Wu, “Uncertainty in natural language processing: Sources, quantification, and applications,”arXivpreprintarXiv:2306.04459, 2023. [69]J. Baan, N. Daheim, E. Ilia, D. Ulmer, H.-S. Li, R. Fernández, B. Plank, R. Sennrich, C. Zerva, and W. Aziz, “Uncertainty in natural language generation: From theory to applications,”arXivpreprintarXiv:2307.15703, 2023. [70] A. Niwa and H. Iso, “Ambignlg: Addressing task ambiguity in instruction for nlg,”arXivpreprintarXiv:2402.17717, 2024. [71] D. D. Johnson, D. Tarlow, and C. Walder, “Ru-sure? uncertainty-aware code suggestions by maximizing utility across random user intents,”arXivpreprintarXiv:2303.00732, 2023. [72]R. Rathnasuriya, Z. Zhao, and W. Yang, “Codeimprove: Program adaptation for deep code models,” in2025IEEE/ACM 47thInternationalConferenceonSoftwareEngineering(ICSE). IEEE Computer Society, 2025, p. 514–526. [73] Y. Li, S. Chen, and W. Yang, “Estimating predictive uncertainty under program data distribution shift,”arXivpreprint arXiv:2107.10989, 2021.

Instant research discovery

Characterizing Overconfident Failure in LLM-Based Code Generation

Intelligence

Summary

Entities (13)

Relation Signals (8)

Abstract

Tags

Links

PDF

Full Text

Paper	Published	Area	Tags	Intel	Citations