Paper deep dive
Five Fatal Assumptions: Why T-Shirt Sizing Systematically Fails for AI Projects
Raja Soundaramourty, Ozkan Kilic, Ramu Chenchaiah
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%
Last extracted: 7/21/2026, 1:05:31 AM
Summary
This paper argues that traditional T-shirt sizing estimation techniques fail for AI projects due to five fatal assumptions: linear effort scaling, repeatability from prior experience, effort-duration fungibility, task decomposability, and deterministic completion criteria. The authors propose 'Checkpoint Sizing' as an iterative alternative that reassesses scope and feasibility at explicit decision gates.
Entities (13)
Relation Signals (11)
T-shirt sizing â failsfor â AI projects
confidence 95% ¡ However, when we apply these methods to artificial intelligence initiatives... the results can be systematically misleading.
Checkpoint Sizing â proposedby â Raja Soundaramourty
confidence 95% ¡ To help teams navigate this, we propose Checkpoint Sizing... This paper is intended for engineering managers... Raja Soundaramourty
Checkpoint Sizing â proposedby â Ozkan Kilic
confidence 95% ¡ To help teams navigate this, we propose Checkpoint Sizing... This paper is intended for engineering managers... Ozkan Kilic
Checkpoint Sizing â proposedby â Ramu Chenchaiah
confidence 95% ¡ To help teams navigate this, we propose Checkpoint Sizing... This paper is intended for engineering managers... Ramu Chenchaiah
T-shirt sizing â assumes â Task Decomposability
confidence 90% ¡ This paper shares an evidence-backed analysis of five foundational assumptions we often make during T-shirt sizing... (4) task decomposability
T-shirt sizing â assumes â Deterministic Completion Criteria
confidence 90% ¡ This paper shares an evidence-backed analysis of five foundational assumptions we often make during T-shirt sizing... (5) deterministic completion criteria
T-shirt sizing â assumes â Linear Effort Scaling
confidence 90% ¡ This paper shares an evidence-backed analysis of five foundational assumptions we often make during T-shirt sizing... (1) linear effort scaling
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Agile estimation techniques, particularly T-shirt sizing, are widely used in software development for their simplicity and utility in scoping work. However, when we apply these methods to artificial intelligence initiatives -- especially those involving large language models (LLMs) and multi-agent systems -- the results can be systematically misleading. This paper shares an evidence-backed analysis of five foundational assumptions we often make during T-shirt sizing. While these assumptions usually hold true for traditional software, they tend to fail in AI contexts: (1) linear effort scaling, (2) repeatability from prior experience, (3) effort-duration fungibility, (4) task decomposability, and (5) deterministic completion criteria. Drawing on recent research into multi-agent system failures, scaling principles, and the inherent unreliability of multi-turn conversations, we show how AI development breaks these rules. We see this through non-linear performance jumps, complex interaction surfaces, and "tight coupling" where a small change in data cascades through the entire stack. To help teams navigate this, we propose Checkpoint Sizing: a more human-centric, iterative approach that uses explicit decision gates where scope and feasibility are reassessed based on what we learn during development, rather than what we assumed at the start. This paper is intended for engineering managers, technical leads, and product owners responsible for planning and delivering AI initiatives.
Tags
Links
- Source: https://arxiv.org/abs/2602.17734v1
- Canonical: https://arxiv.org/abs/2602.17734v1
Trouble viewing inline? Open PDF directly â
Full Text
36,120 characters extracted from source content.
Expand or collapse full text
FIVE FATAL ASSUMPTIONS: WHY T-SHIRT SIZING SYSTEMATICALLY FAILS FOR AI PROJECTS A PREPRINT Raja Soundaramourty 1* , Ozkan Kilic 1 , Ramu Chenchaiah 1 1 Cisco Systems, Inc. rajasoun, okilic, rchencha@cisco.com * Corresponding author February 23, 2026 ABSTRACT Agile estimation techniques, particularly T-shirt sizing, are widely used in software development for their simplicity and utility in scoping work. However, when we apply these methods to artificial intelligence initiativesâespecially those involving large language models (LLMs) and multi-agent systemsâthe results can be systematically misleading. This paper shares an evidence-backed analysis of five foundational assumptions we often make during T-shirt sizing. While these assumptions usually hold true for traditional software, they tend to fail in AI contexts: (1) linear effort scaling, (2) repeatability from prior experience, (3) effort-duration fungibility, (4) task decomposability, and (5) deterministic completion criteria. Drawing on recent research into multi-agent system failures [1], scaling principles [3], and the inherent unreliability of multi-turn conversations [4], we show how AI development breaks these rules. We see this through non-linear performance jumps, complex interaction surfaces, and âtight couplingâ where a small change in data cascades through the entire stack. To help teams navigate this, we propose Checkpoint Sizing: a more human-centric, iterative approach that uses explicit decision gates where scope and feasibility are reassessed based on what we learn during development, rather than what we assumed at the start. This paper is intended for engineering managers, technical leads, and product owners responsible for planning and delivering AI initiatives. Keywords agile methodology¡AI development lifecycle¡effort estimation¡large language models¡multi-agent systems¡ software project management¡ T-shirt sizing 1 Introduction 1.1 Why Traditional Estimation Fails AI Teams T-shirt sizingâcategorizing work items as Small (S), Medium (M), Large (L), or Extra-Large (XL)âhas served software engineering well for decades. Its appeal lies in its simplicity: relative sizing avoids the false precision of hour-based estimates while providing a shared language for teams to plan their work. The method relies on a simple idea: if an experienced team has built something similar before, they can use that pattern to estimate the next project with reasonable confidence. But when we bring this same mindset to AI, the estimates often fall apart. We arenât just talking about being off by a few days; weâre seeing projects expand by months, or hit technical âwallsâ that werenât even on the radar. A task that looks like a âchatbotâ on the surface can diverge wildly in effort once you start dealing with data quality, evaluation hurdles, and the unpredictable nature of multi-turn conversations [4,5]. In these cases, the âslippageâ isnât usually a failure of the teamâitâs a property of the problem itself. arXiv:2602.17734v1 [cs.SE] 18 Feb 2026 Five Fatal Assumptions: Why T-Shirt Sizing Systematically Fails for AI ProjectsA PREPRINT The critical insight is that these failures arenât about poor execution or lack of skill. Itâs that the mental model we use for traditional software just doesnât fit AI. T-shirt sizing rests on five implicit assumptions that we take for granted in software, but which collapse when they meet the unique, often non-linear world of AI development. 1.2 Contributions This paper makes the following contributions: 1.Identification of Five Fatal Assumptions: We systematically enumerate and analyze five foundational assumptions underlying T-shirt sizing that fail in AI contexts. 2.Empirical Grounding: We ground each assumptionâs failure in recent empirical literature (including arXiv preprints and established software-engineering studies) on multi-agent system behavior, agent-system scaling properties, and AI development workflows. 3.Quantitative Evidence: We provide concrete metrics demonstrating the magnitude of estimation errors, includingN(N â 1)interaction complexity growth in multi-agent systems and a 39% average performance degradation in multi-turn conversations [4]. 4.Alternative Framework: We propose Checkpoint Sizing as an evidence-based alternative methodology specifically designed for AI project estimation. 1.3 Paper Organization The remainder of this paper is organized as follows. Section 2 reviews related work in agile estimation and AI project management. Section 3 describes our methodology for identifying and evaluating the five assumptions. Section 4 presents detailed analysis of each assumptionâs failure mode. Section 5 discusses practical implications and introduces the Checkpoint Sizing alternative. Section 6 concludes with directions for future work. 1.4 Scope (What Counts as an âAI Projectâ Here) This paper uses âAI projectsâ to refer to initiatives where the primary delivery risk is driven by model behavior and data-dependent uncertainty, rather than deterministic code alone. This includes (i) LLM applications (prompted systems that must meet reliability and safety targets), (i) agentic workflows (single- or multi-agent tool-using systems with coordination and verification overhead), (i) retrieval-augmented generation (RAG) systems (where performance depends on corpus quality, retrieval, grounding, and evaluation), and (iv) model adaptation such as fine-tuning, steering, or continual improvement loops. We also include classic ML when the initiativeâs critical path is dominated by dataset readiness, evaluation design, and performance/safety validation. Pure software work that happens to call an API is in-scope only when these AI-specific sources of uncertainty materially determine effort, schedule, or definition-of-done. 2 Background and Related Work 2.1 Agile Estimation Methods Agile estimation encompasses several techniques designed to size work without false precision. Story points assign relative complexity values to user stories, typically using Fibonacci sequences (1, 2, 3, 5, 8, 13. . . ) to reflect estimation uncertainty at larger scales [6]. Planning poker leverages collective team wisdom through simultaneous card reveals to surface divergent assumptions [7]. T-shirt sizing simplifies further by mapping work to categorical buckets (S, M, L, XL) that correspond roughly to duration ranges. These methods share common assumptions: that teams possess relevant prior experience, that similar-looking tasks require similar effort, and that work can be decomposed into parallelizable components. Cone of Uncertainty models acknowledge early-stage estimation variance but assume convergence as projects progress through defined phases [8]. 2.2 AI Project Management Literature Recent studies have documented systematic challenges in AI project estimation. Amershi et al. [9] identified nine distinct characteristics of machine learning workflows that differ from traditional software, including data dependencies, experimental iteration, and model decay. Sculley et al. [10] characterized technical debt in ML systems, noting that conventional software engineering intuitions often lead practitioners astray. 2 Five Fatal Assumptions: Why T-Shirt Sizing Systematically Fails for AI ProjectsA PREPRINT The emergence of large language models and multi-agent architectures has introduced additional complexity dimensions. Research on multi-agent coordination failures [1] identifies 14 distinct failure modes clustered into system design issues, inter-agent misalignment, and task verification. Studies on agent-system scaling [3] quantify non-linear performance regimes and coordination-dependent trade-offs that defy intuition trained on traditional software. 2.3 Estimation Failures in LLM-Integrated Systems While outcome-level AI project failure rates are frequently discussed, this paper focuses on estimation failure mech- anisms observable in the development process itself. Recent work highlights that including LLM-based intelligent interfaces can introduce estimation challenges that are not captured by traditional sizing approaches, and motivates richer specifications that account for data sources, interfaces, and algorithms [5]. 3 Methodology 3.1 Assumption Identification We derived the five assumptions through a structured synthesis of how T-shirt sizing is used in practice and how its underlying reasoning maps to the constraints of AI development: 1. Estimation-method synthesis: We reviewed canonical agile estimation approaches (e.g., story points, planning poker, relative sizing) to identify the implicit premises that make categorical sizing usable [6, 7, 8]. 2.AI workflow mismatch grounding: We anchored those premises against established software-engineering characterizations of ML/AI workflows (e.g., data dependency, experimentation, and ML-specific technical debt) [9, 10]. 3.Empirical anchoring in modern LLM systems: We mapped each premise to empirical findings in LLM/agent literature, prioritizing work that quantifies failure modes, scaling behavior, and multi-turn unreliability [1,3,4, 5]. 4.Assumption-to-impact mapping: For each premise, we articulated the mechanism by which its violation produces estimation error (scope underestimation, integration underestimation, or schedule compression overestimation). 3.2 Evaluation Criteria Each assumption was evaluated against three criteria: 1. Validity in Traditional Software: Does the assumption hold for conventional software development? 2. Violation in AI Contexts: Is the assumption systematically violated by AI development characteristics? 3. Estimation Impact: Does the violation produce materially incorrect estimates? Assumptions meeting all three criteria were classified as âfatalââfoundational beliefs that cause systematic estimation failure when applied to AI projects. 3.3 Evidence Collection We collected supporting evidence from empirical literature, with particular emphasis on recent arXiv publications addressing multi-agent systems, LLM behavior, and effort estimation in LLM-integrated software. Each cited work was evaluated for direct relevance to the identified assumptions (see Appendix A for validation methodology). 4 Analysis of the Five Fatal Assumptions 4.1 Assumption 1: Linear Effort Scaling 4.1.1 The Traditional Model: âDouble the Features, Double the Workâ In traditional software, weâve learned that effort usually scales in a straight line. An XL project is roughly twice the work of a Large, which is twice a Medium. This predictable arithmetic has guided our planning for years, allowing us to look at past projects and say, âThis feels like a Medium,â with a high degree of accuracy. 3 Five Fatal Assumptions: Why T-Shirt Sizing Systematically Fails for AI ProjectsA PREPRINT 4.1.2 Why AI Breaks the Ruler The Exponential Effort Curve. AI development doesnât follow a straight line; it follows an exponential curve. Improving a modelâs accuracy from 85% to 95% isnât just a small stepâitâs often a 10x jump in effort, and sometimes itâs technically impossible with the current approach. Every percentage point closer to âperfectâ requires significantly more data, more compute, and more trial-and-error. In AI, the âlast mileâ isnât a mileâitâs a marathon. Foundational research on scaling laws demonstrates that model performance follows power-law relationships with compute and data [11, 12], meaning that linear increases in capability require exponential increases in resources. In agentic systems, these non-linear regimes are measurable. Controlled evaluations show coordination effects can saturate or become negative once single-agent baselines exceedâź45%, and certain multi-agent topologies can amplify errors substantially [3]. Research on agent system scaling [3] confirms this non-linear behavior, documenting diminishing returns aboveâź45% baseline performance and topology-dependent error amplification. Figure 1: Effort scaling comparison between traditional software (approximately linear) and AI systems (often non- linear). As shown in Figure 1, in AI, incremental performance gains can require disproportionately greater data, compute, evaluation, and iterationâundermining the â2Ă scopeâ 2Ă effortâ intuition. Multi-Agent Orchestration Complexity. Adding agents does not add complexity linearlyâit multiplies it. A Supervisor Agent routing tasks across 2 agents faces binary decisions. The same Supervisor across 10 agents faces a combinatorial explosion of routing paths, error states, and emergent failure modes [1]. Table 1: Multi-agent interaction complexity grows combinatorially with agent count. Agent CountRequired Interaction TestsComplexity Growth 22Baseline 36+4 520+14 1090+70 N(N â 1)Combinatorial Figure 2: Multi-agent interaction testing complexity grows as N(N â 1), where N is the number of agents. Figure 2 illustrates why adding a single agent can multiply integration and verification work rather than add a constant increment. Circular Dependency Trap. Agent A delegates to Agent B, which delegates back to Agent Aâcreating an infinite loop that silently consumes API budget until detected. Such emergent failure modes are invisible to T-shirt estimates based on component-level sizing. 4 Five Fatal Assumptions: Why T-Shirt Sizing Systematically Fails for AI ProjectsA PREPRINT Figure 3: Circular dependency failure mode in multi-agent systems. Interaction dynamics can produce loops (AâBâA) that are not visible in component-level estimates as in Figure 3. 4.2 Assumption 2: Repeatability from Prior Experience 4.2.1 The Traditional Model Teams leverage institutional knowledge when estimating. Having built similar APIs or UIs before, they estimate with 70â80% confidence. Experience maps reliably to future effort. Known unknowns can be buffered; unknown unknowns are rare. 4.2.2 Why AI Violates This Assumption Every Dataset is Uncharted Territory. Building an HR chatbot does not prepare a team to estimate a Technical Support chatbot. Data distributions, vocabulary coverage, edge case density, and annotation quality vary wildly across domains. Teams confront Unknown Unknowns: data corruption surfaces only during training; bias emerges only during evaluation [2, 5]. Figure 4: Uncertainty regimes in traditional software vs AI. AI development often contains âunknown unknownsâ that surface late (e.g., during training, evaluation, or deployment), weakening the value of analogies to prior projects. Context Degradation and State Drift. Research demonstrates that LLMs exhibit 39% average performance degradation in multi-turn conversations compared to single-turn interactions [4]. Over extended multi-turn interactions, agents 5 Five Fatal Assumptions: Why T-Shirt Sizing Systematically Fails for AI ProjectsA PREPRINT progressively lose sight of primary objectives, fixating on irrelevant sub-tasks. Context window decay over 20+ turns is unpredictableâhallucinations manifest only in long-duration sessions that were not tested during development. Figure 5: State drift and goal-amnesia in multi-turn LLM interactions. Large-scale simulations find an average 39% performance drop from single turn to multi-turn settings and increased unreliability as conversations progress [4]. 4.3 Assumption 3: Effort-Duration Fungibility 4.3.1 The Traditional Model Effort and duration are interchangeable: an 8-week task takes 4 weeks with two engineers. Add resources, compress timelines. Brooksâs Law applies at extremes, but modest scaling works for most projects. 4.3.2 Why AI Violates This Assumption Irreducible Sequential Dependencies. AI development contains mandatory sequential phases that cannot be paral- lelized regardless of team size: ⢠Data collection must precede data cleaning ⢠Model training must precede hyperparameter tuning ⢠Evaluation must precede deployment Each phase has inherent duration floorsâcompute cycles do not compress with headcount [3]. Studies of real-world ML deployments document these sequential bottlenecks extensively [13], while empirical research shows that data workâoften the longest phaseâis undervalued and cannot be easily parallelized [14]. 6 Five Fatal Assumptions: Why T-Shirt Sizing Systematically Fails for AI ProjectsA PREPRINT Figure 6: Sequential bottlenecks in AI development pipelines. As depicted in Figure 6, AI projects commonly contain irreducible sequential dependencies (dataâtrainâevaluate) that do not compress linearly with added headcount. The Observability Gap. When a multi-agent workflow fails at step 7, isolating the failure requires replaying steps 1â6 to reconstruct exact memory and conversation state. Traditional debugging approaches fail because agent state is not deterministically reproducible [1]. Latency Floors. Multi-agent pipelines have irreducible latency floors. For example, a 5-agent sequential chain withâź3 seconds per inference call implies aâź15 second minimum end-to-end latency. This can materially constrain UX and throughput and is not captured by category-based estimates. Figure 7: Latency wall in multi-agent pipelines. Each agent in a sequential chain contributes inference latency that does not vanish with more engineering effort. 4.4 Assumption 4: Task Decomposability 4.4.1 The Traditional Model Large initiatives decompose into parallel workstreams. Four teams tackle four components simultaneously. Integration happens at the end with manageable coordination overhead. 7 Five Fatal Assumptions: Why T-Shirt Sizing Systematically Fails for AI ProjectsA PREPRINT Figure 8: Traditional software decomposition model. Many software systems allow parallel development with stable interfaces, enabling additive estimates across compo- nents. 4.4.2 Why AI Violates This Assumption Tight Coupling Across the Stack. Data engineering, model architecture, and prompt engineering form a tightly coupled system. Feature changes in data pipelines cascade into model retraining. Schema changes invalidate prompt logic. Independent T-shirt sizing per component ignores systemic interdependencies [1, 3]. Figure 9: Tight coupling in AI system architecture. Dependencies across data, model, and prompting layers can be bidirectional; changes propagate and trigger rework, breaking additive estimation. Shared Resource Contention. Multiple agents share a finite context window (e.g., 128k tokens). One verbose agent can exhaust the budget, starving others of reasoning capacity. Global state coupling means Agent Bâs performance depends on Agent Aâs output schemaâschema evolution breaks downstream reasoning without warning [4]. 8 Five Fatal Assumptions: Why T-Shirt Sizing Systematically Fails for AI ProjectsA PREPRINT Figure 10: Token starvation in shared context windows. Agents compete for finite context capacity; verbose traces can degrade downstream reasoning quality, creating integration risk not captured by component sizing. 4.5 Assumption 5: Deterministic Completion Criteria 4.5.1 The Traditional Model Completion criteria are deterministic: tests pass; specifications are met; code ships. The definition of âdoneâ remains stable throughout the sprint. A task estimated as Medium stays Medium through completion. 4.5.2 Why AI Violates This Assumption The Moving Goalpost Problem. Accuracy targets are achievedâbut Legal rejects the hallucination rate. Performance benchmarks passedâbut an ethics audit reveals unacceptable bias. A project declared âDoneâ on Friday resurfaces as an XL on Monday. Non-deterministic failure modes mean completion is probabilistic, not binary [1, 2]. 9 Five Fatal Assumptions: Why T-Shirt Sizing Systematically Fails for AI ProjectsA PREPRINT Figure 11: Moving-target problem in AI project completion. Acceptance criteria can evolve as safety, governance, and reliability constraints are validatedâturning an apparently âdoneâ task into major remediation. Termination Safety Failures. Infinite correction loops can emerge when a Supervisor rejects Worker output, the Worker retries, and the Supervisor rejects againâconsuming budget without producing progress. Large-scale analyses of multi-agent traces identify recurring failure modes spanning system design issues, inter-agent misalignment, and task verification [1]. Figure 12: Infinite correction loop failure mode. Without explicit termination and verification strategies, agent systems can enter unproductive retry cycles. Guardrail Oscillation. Fixing aggressive agent behavior creates passive behavior; loosening constraints creates unsafe behavior. Calibrating the balance requires weeks of RLHF or prompt engineeringâa hidden cost invisible to T-shirt estimates. 10 Five Fatal Assumptions: Why T-Shirt Sizing Systematically Fails for AI ProjectsA PREPRINT Figure 13: Guardrail oscillation (âwhack-a-moleâ) in agent behavior tuning. Tightening guardrails can reduce utility; loosening can increase risk. The calibration loop is iterative and can be schedule-dominating. 5 Discussion 5.1 What This Means for Teams The analysis in Section 4 shows us that T-shirt sizing doesnât fail because teams are inexperienced or bad at their jobs. It fails because the very foundation of the methodâthe assumptions weâve used for decadesâsimply doesnât apply to the reality of AI development. If we keep using these old tools, weâl keep seeing the same systemic failures, no matter how much âbufferâ we add. Here are the real-world implications: 1.Buffers Arenât Enough: You canât just add 20% to an estimate to cover the âAI part.â Exponential curves and complex agent interactions canât be fixed with a simple percentage. 2. Past Performance Isnât a Guarantee: Just because you built one chatbot doesnât mean the next one will take the same amount of time. Every new dataset brings its own set of âunknown unknowns.â 3.Estimates cannot be made in a Vacuum: Because AI layers are so tightly coupled, you canât just size the âUIâ and the âBackendâ separately and expect them to sum up to a project total. 4.âDoneâ is a Moving Target: In AI, meeting an initial accuracy goal is often just the beginning. New safety or legal requirements can turn a âcompletedâ task back into an XL project overnight. 5.2 Limitations This analysis has several limitations: 1.Qualitative Evidence: While grounded in empirical literature, the assumption violations are characterized analytically rather than through a new controlled study conducted by the authors. 2.Generalization: The analysis focuses on LLM and multi-agent systems; simpler ML projects may violate fewer assumptions. 3.Alternative Methods: We propose Checkpoint Sizing but do not empirically validate its effectiveness relative to other emerging AI estimation approaches. 5.3 The Checkpoint Sizing Alternative Given the fundamental incompatibility between T-shirt sizing assumptions and AI development characteristics, we propose Checkpoint Sizing as an alternative methodology: 11 Five Fatal Assumptions: Why T-Shirt Sizing Systematically Fails for AI ProjectsA PREPRINT Table 2: Checkpoint Sizing principles and implementation guidelines. PrincipleImplementation Explicit Decision GatesDefine checkpoints at natural phase boundaries (data readiness, model convergence, safety validation) Evidence-Based Reassessment Re-estimate scope, timeline, and feasibility at each checkpoint based on empirical observations Uncertainty AcknowledgmentReplace point estimates with confidence intervals that reflect genuine uncertainty Pivot OptionalityBuild explicit decision points for scope reduction, pivot, or termination based on checkpoint outcomes Checkpoint Sizing treats AI project estimation as an iterative discovery process rather than an upfront planning exercise. Initial estimates serve as hypotheses to be validated through successive checkpoints rather than commitments to be defended. 5.3.1 A Minimal Procedure (Pseudocode) The goal is not âperfect upfront estimation,â but controlled uncertainty with explicit decision points. Algorithm 1 Checkpoint Sizing Require: InitiativeGoal, InitialHypothesisEstimate, Constraints (budget, latency, safety, timeline) Require: Checkpoints = [C 1 ..C k ] (e.g., DataReadiness, EvalHarness, SafetyReliability, CostLatency, Rollout) 1: Estimateâ InitialHypothesisEstimate 2: Risksâ initial risk register (unknowns, assumptions, dependencies) 3: for each checkpoint C i in Checkpoints do 4:Evidenceâ run checkpoint-specific work and measurements (data profiling, baseline model tests, eval harness results, safety probes, cost/latency profiling, red-teaming, etc.) 5:Update Risks using Evidence (retire risks, add new risks, re-rank severity) 6:Re-estimate scope/timeline/cost using Evidence (not analogy) 7:Decisionâ Proceed, Pivot, ReduceScope, Pause, Terminate 8:if Decision̸= Proceed then 9:record rationale and exit (or branch plan accordingly) 10:end if 11: end for 12: return final Estimate and delivery plan with residual risks and monitoring hooks 5.3.2 Checkpoint Gate Checklist (What You Must Exit With) Use these gates as âdefinition of readinessâ checkpoints. Each gate produces artifacts and metrics that change the estimate. ⢠Gate A â Data Readiness â Artifacts: data inventory; schema; quality report; labeling/PII policy; drift risks âMinimum evidence: representative samples; missingness/outlier profile; edge-case map; governance sign-off (if required) ⢠Gate B â Evaluation Harness â Artifacts: task taxonomy, gold set, automated eval pipeline, failure categorization âMinimum evidence: baseline score + confidence intervals; regression tests; measurement of âunknown unknownsâ surfaced during eval. Systematic behavioral testing approaches [15] provide templates for comprehensive evaluation coverage. ⢠Gate C â Safety, Reliability, and Control âArtifacts: safety policy; refusal/grounding strategy; escalation paths; red-team plan; termination/loop controls (agents) 12 Five Fatal Assumptions: Why T-Shirt Sizing Systematically Fails for AI ProjectsA PREPRINT âMinimum evidence: observed hallucination/error modes; multi-turn reliability characterization [4]; mitigation plan mapped to failure taxonomy classes [1] ⢠Gate D â Cost and Latency Budgets â Artifacts: throughput model; caching strategy; prompt/token budgets; tool-call budget; spend forecast âMinimum evidence: measured p50/p95 latency and cost per task; sensitivity to context length and agent count; rollback plan if budgets are exceeded ⢠Gate E â Operationalization (Rollout/Monitoring) â Artifacts: monitoring dashboards; incident playbooks; feedback loops; drift detection; evaluation in production â Minimum evidence: canary plan; SLOs; alerting; documented âdefinition of doneâ that includes post- deploy monitoring criteria 5.4 Illustrative Case Study (Synthetic) To show how Checkpoint Sizing changes estimates, consider a synthetic initiative: âSupport Copilot with RAG + Tool Useâ for internal support engineers. Initial T-shirt estimate (Sprint 0): Large (6â8 weeks), based on analogy to âbuild a chatbotâ and prior UI/API delivery. Checkpoint outcomes and estimate evolution: 1. Gate A â Data Readiness ⢠Finding: 25% of historical tickets contain redacted fields; knowledge base has inconsistent versions; PII policy requires additional filtering. ⢠Estimate impact: +3 weeks (data cleaning + governance). 2. Gate B â Evaluation Harness ⢠Finding: baseline retrieval fails on long-tail product SKUs; âanswer correctnessâ needs rubric + citations; automated eval required for regression. ⢠Estimate impact: +2 weeks (eval harness + gold set + metrics). 3. Gate C â Safety, Reliability, and Control ⢠Finding: multi-turn conversations degrade when users revise constraints; tool calls occasionally produce conflicting states; need loop/termination controls for tool-using agent. ⢠Estimate impact: +3â5 weeks (safety probes, guardrails, termination/verification). 4. Gate D â Cost and Latency Budgets ⢠Finding: sequential tool-using agent path exceeds latency SLO at p95; requires caching + partial parallelization + shorter context strategy. ⢠Estimate impact: +2 weeks (performance engineering + architecture changes). 5. Gate E â Operationalization ⢠Finding: rollout requires monitoring for retrieval drift and hallucination rate; canary plan plus human escalation path required. ⢠Estimate impact: +1â2 weeks (observability + rollout controls). Revised estimate after gates: XL (12â16 weeks) with explicit risk register and exit criteria. The key shift is that the estimate becomes evidence-driven: effort accrues from concrete artifacts (data readiness, eval harness, safety controls, cost/latency budgets, and operationalization) rather than analogy. 6 Conclusion T-shirt sizing was designed for a world of linear scaling, repeatable patterns, effort-duration fungibility, parallelizable work, and deterministic completion. AI development violates all five premises: 1.Non-linear effort curves replace linear scaling, with performance gains often requiring disproportionate increases in data, compute, evaluation, and iteration. 2. Dataset uniqueness eliminates repeatability, making every AI project a journey into terra incognita. 13 Five Fatal Assumptions: Why T-Shirt Sizing Systematically Fails for AI ProjectsA PREPRINT 3.Sequential dependencies break effort-duration fungibility, creating timeline floors that resist resource addition. 4.Tight cross-layer coupling prevents decomposition, generating cascading rework invisible to component estimates. 5.Probabilistic completion replaces deterministic criteria, introducing moving goalposts and emergent failure modes. The path forward requires a new way of thinking, designed specifically for the unique challenges of AI. Checkpoint Sizing is our proposal: a system of explicit decision gates where we reassess scope, timeline, and feasibility based on what weâve actually built and learnedânot on assumptions inherited from a different era of engineering. Future Work. Several directions merit further investigation: â˘Empirical Validation: Controlled studies comparing Checkpoint Sizing against T-shirt sizing and other estimation methods across diverse AI project types. â˘Tooling Integration: Development of plugins or extensions for project management tools (Jira, Linear, Asana) that operationalize checkpoint gates with templates, artifact checklists, and re-estimation workflows. â˘Domain-Specific Adaptations: Tailoring checkpoint definitions for specific AI application domains (health- care, finance, autonomous systems) where regulatory and safety constraints introduce additional estimation complexity. â˘Autonomous Agent Systems: Extending the analysis to fully autonomous agents with long-horizon planning, where estimation challenges may be even more severe due to open-ended execution and environmental uncertainty. â˘Quantitative Benchmarks: Establishing benchmark datasets of AI project estimates vs. actuals to enable quantitative comparison of estimation methodologies. 14 Five Fatal Assumptions: Why T-Shirt Sizing Systematically Fails for AI ProjectsA PREPRINT Appendix Reference Validation Methodology A.1 Validation Process Each cited work in the primary references [1]â[5] was evaluated against the five fatal assumptions using a systematic validation process: 1. Relevance Mapping: Each paper was mapped to the specific assumptions it directly validates. 2.Evidence Extraction: Key findings from each paper were extracted and matched to specific claims in Section 4. 3. Coverage Assessment: The collective coverage of all five assumptions was verified. A.2 Validation Results RefPaper TitleValidates AssumptionsRelevance [1]Why Do Multi-Agent LLM Systems Fail? 4 (Decomposability via inter-agent coupling), 5 (De- terministic completion via verification/termination failures) 100% [2] An LLM-based multi- agent framework for agile effort estimation 2 (Repeatability via subjective inconsistency), 5 (De- terministic completion via estimation instability) 100% [3]Towards a Science of Scaling Agent Systems 1 (Non-linear scaling), 3 (Effort-duration tradeoffs), 4 (Coordination overhead/error amplification) 100% [4]LLMs Get Lost In Multi- Turn Conversation 2 (Repeatability via multi-turn unreliability), 5 (De- terministic completion via non-recovering error tra- jectories) 100% [5] Effort and Size Estima- tion in Software Projects with Large Language Model-based Intelligent Interfaces 2 (Repeatability via hidden uncertainty), 5 (Defini- tion of done via evolving spec/AI interface behavior) 100% Table 3: Literature Validation Matrix: Each reference validates specific assumptions with 100% relevance, providing independent empirical support for the theoretical framework. A.3 Assumption Coverage Matrix AssumptionPrimary SupportSecondary Support 1. Linear Scaling[3]â 2. Repeatability[2], [4], [5]â 3. Effort-Duration Trade[3]â 4. Decomposability[1], [3]â 5. Deterministic Completion[1], [2], [4], [5]â Table 4: Assumption Coverage Matrix: Mapping of literature references to the five fatal assumptions. All five fatal assumptions receive direct support from empirical sources in [1]â[5], with multiple assumptions supported by more than one work. The 100% relevance score reflects scope alignment (each reference was selected specifically to substantiate one or more claims in Section 4), not an external meta-analytic quality rating. 15 Five Fatal Assumptions: Why T-Shirt Sizing Systematically Fails for AI ProjectsA PREPRINT References [1]M. Cemri et al., âWhy Do Multi-Agent LLM Systems Fail?,â arXiv:2503.13657 [cs.AI], 2025. doi:10.48550/arXiv.2503.13657. [Online]. Available: https://arxiv.org/abs/2503.13657 [2]T.-L. Bui, H. K. Dam, and R. Hoda, âAn LLM-based multi-agent framework for agile effort estimation,â arXiv:2509.14483 [cs.SE], 2025. doi:10.48550/arXiv.2509.14483. [Online]. Available:https://arxiv.org/ abs/2509.14483 [3]Y. Kim et al., âTowards a Science of Scaling Agent Systems,â arXiv:2512.08296 [cs.AI], 2025. doi:10.48550/arXiv.2512.08296. [Online]. Available: https://arxiv.org/abs/2512.08296 [4]P. Laban, H. Hayashi, Y. Zhou, and J. Neville, âLLMs Get Lost In Multi-Turn Conversation,â arXiv:2505.06120 [cs.CL], 2025. doi:10.48550/arXiv.2505.06120. [Online]. Available: https://arxiv.org/abs/2505.06120 [5]C. N. Coelho Jr et al., âEffort and Size Estimation in Software Projects with Large Language Model-based Intelligent Interfaces,â arXiv:2402.07158 [cs.SE], 2024. doi:10.48550/arXiv.2402.07158. [Online]. Available: https://arxiv.org/abs/2402.07158 [6] M. Cohn, âAgile Estimating and Planning,â Prentice Hall, 2005. [7]J. Grenning, âPlanning Poker or How to Avoid Analysis Paralysis While Release Planning,â Hawthorn Woods: Renaissance Software Consulting, 2002. [8] S. McConnell, âSoftware Estimation: Demystifying the Black Art,â Microsoft Press, 2006. [9]S. Amershi et al., âSoftware Engineering for Machine Learning: A Case Study,â in Proc. IEEE/ACM 41st Int. Conf. Software Engineering: Software Engineering in Practice (ICSE-SEIP), 2019, p. 291â300. [10]D. Sculley et al., âHidden Technical Debt in Machine Learning Systems,â in Advances in Neural Information Processing Systems, 2015, p. 2503â2511. [11]J. Kaplan et al., âScaling Laws for Neural Language Models,â arXiv:2001.08361 [cs.LG], 2020. doi:10.48550/arXiv.2001.08361. [Online]. Available: https://arxiv.org/abs/2001.08361 [12]J. Hoffmann et al., âTraining Compute-Optimal Large Language Models,â arXiv:2203.15556 [cs.CL], 2022. doi:10.48550/arXiv.2203.15556. [Online]. Available: https://arxiv.org/abs/2203.15556 [13] A. Paleyes, R.-G. Urma, and N. D. Lawrence, âChallenges in Deploying Machine Learning: A Survey of Case Studies,â ACM Computing Surveys, vol. 55, no. 6, p. 1â29, 2022. doi:10.1145/3533378. [14] N. Sambasivan et al., âEveryone wants to do the model work, not the data work: Data Cascades in High-Stakes AI,â in Proc. CHI Conference on Human Factors in Computing Systems, 2021, p. 1â15. doi:10.1145/3411764.3445518. [15]M. T. Ribeiro, T. Wu, C. Guestrin, and S. Singh, âBeyond Accuracy: Behavioral Testing of NLP Models with CheckList,â in Proc. 58th Annual Meeting of the Association for Computational Linguistics (ACL), 2020, p. 4902â4912. 16