Paper deep dive
Smart Enough to Go Extinct? An Evolutionary Challenge to the Value of General Intelligence and Its Ethical Implications for AGI
David Klotz
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%
Last extracted: 8/16/2026, 3:14:31 AM
Summary
The paper challenges the assumption that general intelligence is inherently valuable by presenting an evolutionary argument. It contrasts the short tenure and self-generated existential risks of Homo sapiens with the long-term persistence of non-intelligent taxa like cyanobacteria and horseshoe crabs. The author argues that general intelligence may uniquely generate existential threats, creating an 'existential risk paradox.' Consequently, the paper suggests that engineering AGI and artificial consciousness carries significant ethical liabilities, advocating for a duty of caution based on deontological ethics and the precautionary principle.
Entities (9)
Relation Signals (7)
Homo sapiens â possesses â General Intelligence
confidence 95% ¡ general intelligence, the kind of flexible, domain-general cognitive capacity exemplified by Homo sapiens
Cyanobacteria â persistswithout â General Intelligence
confidence 93% ¡ Numerous taxa, from cyanobacteria to horseshoe crabs, have persisted for hundreds of millions or even billions of years without anything resembling general intelligence
Horseshoe crabs â persistswithout â General Intelligence
confidence 93% ¡ Numerous taxa, from cyanobacteria to horseshoe crabs, have persisted for hundreds of millions or even billions of years without anything resembling general intelligence
Homo sapiens â faces â Existential Risk
confidence 92% ¡ Homo sapiens has existed for roughly 300,000 years and already faces self-generated existential risks.
General Intelligence â associatedwith â Existential Risk
confidence 90% ¡ We argue that general intelligence may be the only biological strategy that generates existential threats to the species possessing it
Uncertainty of Evolutionary Value â imposes â Duty of Caution
confidence 87% ¡ if the long-term evolutionary value of general intelligence is uncertain or negative, this raises ethical questions... we argue that this uncertainty imposes a duty of caution
AGI Development â raisesethicalconcernsfor â Artificial Consciousness
confidence 85% ¡ this raises ethical questions about engineering AGI and, more urgently, creating artificial consciousness.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:The pursuit of artificial general intelligence (AGI) rests on a seemingly self-evident premise: that general intelligence, the kind of flexible, domain-general cognitive capacity exemplified by Homo sapiens, is extraordinarily valuable. This paper subjects this premise to critical scrutiny. We first present the intuitive case for the value of general intelligence before mounting an evolutionary challenge. We argue that, on evolutionary timescales, its adaptive value is far from empirically established. Numerous taxa, from cyanobacteria to horseshoe crabs, have persisted for hundreds of millions or even billions of years without anything resembling general intelligence, while Homo sapiens has existed for roughly 300,000 years and already faces self-generated existential risks. Mass extinction events do not preferentially favour cognitively sophisticated species. We argue that general intelligence may be the only biological strategy that generates existential threats to the species possessing it, an existential risk paradox with no parallel among non-intelligent survival strategies. Unlike prevailing accounts of AI risk that trace the danger to misalignment, we locate it in structural features of general intelligence itself, implying that even well-aligned AGI would inherit this liability. If the long-term evolutionary value of general intelligence is uncertain or negative, this raises ethical questions about engineering AGI and, more urgently, creating artificial consciousness. Drawing on deontological ethics and the precautionary principle, we argue that this uncertainty imposes a duty of caution: if we create a new kind of intelligent being, we bear responsibility for ensuring the conditions under which it can flourish.
Tags
Links
- Source: https://arxiv.org/abs/2608.10730v1
- Canonical: https://arxiv.org/abs/2608.10730v1
Trouble viewing inline? Open PDF directly â
Full Text
84,024 characters extracted from source content.
Expand or collapse full text
Smart Enough to Go Extinct? An Evolutionary Challenge to the Value of General Intelligence and Its Ethical Implications for AGI David Klotz Hochschule der Medien Stuttgart Abstract The pursuit of artificial general intelligence (AGI) is widely regarded as the paramount objective of contemporary AI research. This aspiration rests on a seemingly self-evident premise: that general intelligenceâthe kind of flexible, domain-general cognitive capacity exemplified by Homo sapiensâis extraordinarily valuable. The present paper subjects this premise to critical scrutiny. We first present the intuitive case for the value of general intelligence, acknowledging its genuine strengths, before mounting an evolutionary challenge. We argue that, when measured against the timescales on which biological evolution operates, the adaptive value of general intelligence is far from empirically established. Numerous taxaâfrom cyanobacteria to horseshoe crabsâhave persisted for hundreds of millions, even billions, of years without anything resembling general intelligence, while Homo sapiens has existed for roughly 300,000 years and already faces self-generated existential risks. Mass extinction events, examined as natural experiments, do not preferentially favour cognitively sophisticated species. We argue that general intelligence may be the only biological strategy that generates existential threats to the species that possesses itâan existential risk paradox with no parallel among non-intelligent survival strategies. Unlike the prevailing framing of AI risk, which traces the danger to misalignment, we locate it in the structural features of general intelligence itself, so that even a well-aligned artificial general intelligence would inherit this liability. If the long-term evolutionary value of general intelligence is uncertain or even negative, this raises profound ethical questions about the engineering of AGI systems and, with still greater urgency, about the creation of artificial consciousnessâbeings that would be both generally intelligent and sentient. Drawing on deontological ethics and the precautionary principle, we argue that this uncertainty imposes a duty of caution: if we bring into existence a new kind of intelligent being, we bear responsibility for ensuring the conditions under which it can flourish. Keywords: general intelligence¡ artificial general intelligence¡ evolutionary fitness¡ philosophy of AI 1 Introduction In contemporary artificial intelligence research, artificial general intelligence (AGI) is frequently framed as the disciplineâs ultimate objective (Goertzel, 2014), occasionally portrayed as its âholy grailâ (Boden, 2016). Major research organisations have made the pursuit of AGI a central part of their institutional missions, and surveys of AI researchers reveal a median estimate of 50% probability that human-level machine intelligence will be achieved by 2040â2050 (M Ěuller and Bostrom, 2016). Public discourse increasingly treats the advent of AGI as a question of âwhenâ rather than âif.â Underlying this aspiration is 1 arXiv:2608.10730v1 [cs.CY] 11 Aug 2026 David KlotzSmart Enough to Go Extinct? a presupposition so widely shared as to be rarely examined: that general intelligence is extraordinarily valuable. The presupposition is understandable. Homo sapiens is, by any reasonable measure, the most generally intelligent species on Earth, and it has achieved a degree of planetary dominance without historical precedent. From this observation, it is a short step to the conclusion that general intelligence is the most powerful and desirable of all biological adaptations, and that building machines that possess it would be correspondingly transformative. This paper subjects that presupposition to critical examination. We advance a thesis that is simple to state but, we believe, far-reaching in its implications: The value of general intelligence is not empirically established when evaluated on evolutionary timescales, and this uncertainty carries significant ethical implications for the development of artificial general intelligence. Our argument proceeds in three stages. First, we present the intuitive case for the value of general intelligence as forcefully as we can, acknowledging its genuine strengths while identifying its limitations (Section 2). Second, we challenge this intuitive case from an evolutionary perspective, arguing that the track record of general intelligence, when assessed on the timescales appropriate to biological evolution, is far less impressive than the anthropocentric narrative suggests (Section 3). We show that the longest-surviving lineages on Earth are precisely those that lack general intelligence, that mass extinction events do not preferentially spare cognitively sophisticated species, and that general intelligence uniquely generates existential threats to the species that possesses it. Third, we draw out the ethical implications of this evolutionary uncertainty for the AGI enterprise (Section 4). Several clarifications of scope are in order. The concept of âgeneral intelligenceâ has been defined in various ways: as an intrinsic system property captured by the psychometricífactor (Spearman, 1904; Carroll, 1993), as an agentâs capacity to achieve goals across diverse environments (Legg and Hutter, 2007; Chollet, 2019), and as a function of algorithmic skill-acquisition efficiency (Chollet, 2019). These definitions share a common assumption: that intelligence is a property of the systemâsomething a mind possesses independently of the context in which it operates. In this paper, we adopt a different framing and understand general intelligence as situative intelligence: not an intrinsic system property but a thick evaluation of a systemâs behaviour in context (anonymized for review). On this view, to call a system âintelligentâ is to make a judgement about how it navigates specific situations, not to attribute a context-independent capacity. Intelligence is always intelligence in a situationâshaped by, and inseparable from, the environmental, social, and ecological conditions in which it is exercised. This framing is important for our evolutionary argument, because it makes explicit what capacity-based definitions obscure: the value of intelligence cannot be assessed in the abstract, but only relative to the situationsâincluding evolutionary situationsâin which it operates. We distinguish general intelligence (in this situative sense) from specialised or narrow intelligenceâthe echolocation of bats, the navigational abilities of migratory birdsâwhich raises different questions. Our argument concerns the value of domain-general cognitive flexibility specifically. We also distinguish general intelligence from consciousness: the two are conceptually distinct, and a system might possess one without the other. However, the intersection of the twoâthe possibility of creating beings that are both generally intelligent and consciousâraises the most acute ethical questions, and we address this intersection explicitly in Section 4. We distinguish throughout between several kinds of âvalueâ: survival value (contribution to species persistence), instrumental value (usefulness as a means to various ends), and intrinsic or moral value (worth independent of consequences). These are different questions, and conflating them has been a persistent source of confusion in debates about intelligence and AGI. Our primary focus is on survival value, but we address the relationships among all three in Section 4. 2 David KlotzSmart Enough to Go Extinct? Finally, we do not argue that intelligence is without value in any context, that Homo sapiens is doomed, or that AI research should be abandoned. Our claim is more specific and, we believe, more intellectually honest: that the confident assumption of the supreme value of general intelligence is not warranted by the available evidence, and that this unwarranted confidence has practical consequences when it motivates the creation of new generally intelligent beings. The paper contributes to several ongoing conversations. In the philosophy of biology, it engages with debates about the directionality of evolution and the status of intelligence as an adaptive trait (Gould, 1996; Godfrey-Smith, 2002; Sterelny, 2003). In AI ethics, it challenges a foundational but often implicit premise of the AGI enterprise. In the broader public discourse, it offers a counterpoint to the technological optimism that characterises much of the conversation about artificial intelligence. By bringing evolutionary biology into dialogue with philosophy of AI, we hope to open a line of inquiry that has received too little attention. 2 The Intuitive Case for the Value of General Intelligence Before mounting our challenge to the assumed value of general intelligence, intellectual honesty requires that we present the case for that value as forcefully as possible. The intuitive case is genuinely powerful, and any serious critique must reckon with it rather than dismissing it prematurely. 2.1 Anthropocentric Evidence By virtually any metric of immediate ecological impact, Homo sapiens is the most consequential species in Earthâs history. Humans have colonised every continent, reshaped entire biomes, domesticated hundreds of plant and animal species, and constructed a technological civilisation of extraordinary complexity. The human population has grown from perhaps a few hundred thousand individuals 70,000 years ago to over eight billion todayâan expansion without parallel among large-bodied organisms. These achievements are standardly attributed to general intelligence: the capacity for abstract thought, causal reasoning, language, cumulative cultural learning, and flexible problem-solving across diverse domains (Tomasello, 1999; Henrich, 2016). The evolutionary literature offers well-developed accounts of how this capacity arose. The social brain hypothesis (Dunbar, 1998) proposes that primate brains expanded primarily to manage the demands of complex social relationships, with neocortex size correlating with social group size. The expensive-tissue hypothesis (Aiello and Wheeler, 1995) explains how this expansion was metabolically financed: the human brain constitutes roughly 2% of body mass but consumes approximately 20% of the bodyâs resting energy budget, a cost offset by a corresponding reduction in gut size enabled by a shift to higher-quality diets. These accounts suggest that general intelligence, far from being a happy accident, was shaped by powerful and persistent selection pressures. Moreover, evidence from comparative biology suggests that the benefits of enhanced cognition are not limited to humans. Sol et al. (2007) demonstrated that bird species with relatively larger brains experience lower mortality rates in nature, supporting the âcognitive buffer hypothesisâ: that larger brains help individuals cope with novel environmental challenges. This finding extends the case for the value of intelligence beyond the human species alone. Unlike specialised cognitive abilitiesâthe navigational prowess of migratory birds, the echolocation of bats, the social cognition of eusocial insectsâhuman intelligence operates across essentially all problem domains. It is this domain-generality that appears to underwrite human dominance: rather than being adapted to a single ecological niche, humans can adapt any niche to themselves. The scope of human achievement is genuinely staggering when viewed from a biological standpoint. Humans have extended their individual lifespans from roughly 30 years in pre-agricultural societies to over 70 years globally today. They have eradicated infectious diseases such as smallpox, harnessed 3 David KlotzSmart Enough to Go Extinct? nuclear energy, and sent representatives of their species beyond Earthâs atmosphere. They have developed mathematics, philosophy, and the natural sciencesâsymbolic systems that allow them to model and predict phenomena far beyond the reach of direct perception. No other species has produced anything remotely comparable. The âstandard narrative,â then, runs as follows. General intelligence is the single most powerful biological adaptation ever produced by evolution. It is what makes Homo sapiens special, what explains our dominance, and what will ultimately allow us to transcend the biological limitations that constrain other species. On this view, building artificial general intelligence is simply the next step in a trajectory that began when our ancestors first fashioned stone tools on the African savannah. 2.2 Intelligence as Instrumental Power The instrumental value of general intelligenceâits usefulness as a means to diverse endsâis most clearly visible in the breadth of ecological niches that Homo sapiens has come to occupy. Where other species are adapted to specific habitats, humans have used their general cognitive capacities to colonise deserts, tundra, tropical forests, and oceanic islandsâenvironments radically different from the African savannah in which the species evolved (Diamond, 1997). This ecological versatility is a direct product of general cognitive flexibility: the capacity to assess novel environments and devise appropriate responses, from clothing and shelter to agriculture and navigation. More broadly, general intelligence functions as what we might call a âmeta-capabilityâ: a capacity that enhances the ability to acquire other capabilities. A generally intelligent agent can learn new skills, develop new technologies, anticipate and respond to novel threats, and devise solutions to problems it has never previously encountered. This meta-capability is precisely what makes the prospect of AGI so tantalising to its proponents: an artificial system with genuine general intelligence would, in principle, be applicable to virtually any intellectual challenge. The economic evidence appears to reinforce this picture. At the level of individuals, cognitive ability is among the strongest predictors of educational attainment, income, and occupational status (Gottfredson, 1997). At the level of societies, economic development is correlated with investments in education, research, and the cultivation of human cognitive capital (Hanushek and Woessmann, 2015). The entire edifice of modern medicine, agriculture, and industrial technologyâthe systems that sustain eight billion human lives âis a product of cumulative problem-solving enabled by general intelligence. It is worth noting that the case extends beyond the merely practical. General intelligence enables not only instrumental achievements but also goods that many would consider intrinsically valuable: scientific understanding, artistic creation, philosophical reflection, and moral reasoning. The capacity to contemplate oneâs own existence, to ask questions about justice and meaning, and to appreciate beautyâthese are themselves products of general intelligence, and they constitute a significant part of the case for its value. Any argument against the value of general intelligence must reckon with the fact that the argument itself is only possible because of general intelligence. 2.3 Acknowledging the Intuitive Forceâand Its Limits We do not wish to minimise the force of these observations. The intuitive case for the value of general intelligence is genuine, and we accept its core empirical claims: humans are ecologically dominant, general intelligence does confer extraordinary instrumental power, and technological capability has been decisive in many competitive contexts. However, several critical distinctions must be drawn before this intuitive case can bear the argumentative weight that proponents of AGI place upon it. First, the evidence cited above concerns short-term, within-species advantages. It demonstrates that, among humans, greater technological sophistication tends to prevail over lesser technological sophistication 4 David KlotzSmart Enough to Go Extinct? on timescales of decades to centuries. It does not demonstrate that general intelligence is advantageous as a biological strategy on the timescales relevant to evolutionâthat is, over millions to billions of years. Second, the evidence is subject to anthropocentric selection bias. We are a generally intelligent species evaluating the worth of general intelligence. The criteria by which we judge âsuccessââtechnological achievement, cultural complexity, environmental controlâare precisely the criteria that general intelligence excels at meeting. A species that valued persistence, metabolic efficiency, or reproductive robustness would construct a very different ranking of biological strategies. Third, even within the biological domain, there are striking cases where traits that confer apparent short-term advantages fail to ensure long-term survival. Large body size in non-avian dinosaurs provided competitive advantagesâaccess to food resources, defence against predators, thermoregulatory efficiencyâ yet the high metabolic costs and reproductive constraints associated with gigantism made these lineages acutely vulnerable when environmental conditions shifted catastrophically at the KâPg boundary. The sabre-toothed cats of the genus Smilodon represent a parallel case: their extreme dental specialisation was highly effective for subduing large prey, but this hyper-specialisation became a fatal liability when megafaunal prey populations declined during the late Pleistocene. The Irish elk (Megaloceros giganteus), whose enormous antlers conferred advantages in intraspecific competition, may have faced analogous constraintsâthe metabolic cost of producing and carrying such structures arguably reduced adaptive flexibility when environmental conditions changed. These biological examples illustrate a general principle directly relevant to our argument: traits that are advantageous in specific ecological contexts can become liabilities when those contexts shift. A trait that excels in one regime of environmental challenges may prove neutral or actively harmful in another. General intelligence, we shall argue, may be no exception. The distinction between intuitive plausibility and empirical demonstration is crucial here. The intuitive case for the value of general intelligence is strong. But intuitive plausibility is not the same as rigorous evidence. Many intuitively compelling hypotheses have proved false upon careful examinationâ geocentrism being perhaps the most famous example. In the next section, we subject the presumed value of general intelligence to the kind of critical scrutiny that the stakes demand. 3 The Evolutionary Challenge The intuitive case for the value of general intelligence, powerful as it is, rests on evidence drawn from a narrow temporal window and evaluated through a biased lens. In this sectionâthe core of the paperâwe examine the value of general intelligence from the perspective of evolutionary biology, where the relevant timescales are measured not in centuries but in hundreds of millions of years. 3.1 Anthropocentric Bias The first obstacle to an objective assessment of general intelligence is the fact that the assessors are themselves generally intelligent. This is not a trivial methodological concern; it is a deep epistemological problem. Humans have a long history of placing themselves at the centre of evaluative frameworks. Geocentrism placed Earth at the centre of the cosmos. Anthropocentrism in moral philosophy reserved moral standing for humans alone. In each case, what appeared to be an objective assessment of the natural order turned out to be a projection of human self-regard (Singer, 1975; DeGrazia, 1996). The assumption that general intelligence is the pinnacle of biological achievement may belong to this same tradition. The bias operates through the selection of evaluative criteria. When we ask âwhich species is most successful?â we implicitly choose metricsâtechnological sophistication, cultural complexity, environmental modificationâthat are tailored to the distinctive outputs of general intelligence. This 5 David KlotzSmart Enough to Go Extinct? is circular: we define success in terms of what our defining trait produces, and then conclude that our defining trait is the key to success. A more neutral biology would recognise that âsuccessâ admits of many operationalisations, and that the choice among them is not value-free. The philosopher Peter Godfrey-Smith has drawn attention to a related point in his work on the evolution of cognition: the environmental conditions that select for complex nervous systems are specific and contingent, not universal features of the natural world (Godfrey-Smith, 2002). General intelligence is a response to a particular kind of environmental complexityâone characterised by rapid change, social competition, and unpredictable resource distribution. In environments that are stable and predictable, general intelligence is not merely unnecessary; it is wasteful. The metabolic cost of a large brain (discussed below) constitutes a permanent drain on resources that could be directed toward reproduction, growth, or metabolic efficiency. To escape this circularity, we need evaluative criteria that are not parasitic on general intelligence. The most obvious candidate is species persistence: how long a lineage endures before going extinct. Persistence is a biologically fundamental metricâit is, in a sense, the precondition for all other measures of success. A species that goes extinct cannot accumulate further achievements, however impressive its prior record. Moreover, persistence is a metric that can be applied uniformly across all taxa, regardless of their cognitive endowments. It is not the only possible metricâgeographic range, biomass, species diversity within a clade, and ecological impact are all alternativesâbut it has the virtue of being both biologically fundamental and neutral with respect to cognitive endowment. 3.2 Intelligence on Evolutionary Timescales Life on Earth originated approximately 3.7â4.0 billion years ago (Schopf, 2006; Bell et al., 2015). The first unambiguous evidence of complex multicellular life dates to roughly 600 million years ago, and the Cambrian explosionâthe rapid diversification of animal body plansâoccurred approximately 540 million years ago (Erwin and Valentine, 2011). Against this vast temporal backdrop, the tenure of Homo sapiensâroughly 300,000 years (Hublin et al., 2017)âis extraordinarily brief. Behaviourally modern humans, characterised by symbolic thought, complex tool manufacture, and evidence of cultural transmission, appear in the archaeological record only around 77,000 years ago (Henshilwood et al., 2002). To appreciate the implications of these numbers, consider the following comparisons: â˘Cyanobacteria have existed for approximately 2.7â3.5 billion years (Schopf, 2006). They oxygenated Earthâs atmosphere, enabling all subsequent aerobic life. They remain globally abundant. â˘Horseshoe crabs (Limulus and related genera) have persisted with little morphological change for over 450 million years (Rudkin et al., 2008)âroughly 1,500 times as long as Homo sapiens. â˘Sharks have existed in recognisable forms for over 400 million years, surviving at least four of the âBig Fiveâ mass extinctions (Kriwet and Benton, 2004). â˘Nautiloids have a fossil record extending approximately 500 million years, with the modern nautilus showing remarkable conservatism of body plan (Ward, 1988). ⢠Crocodilians have persisted for roughly 250 million years, including through the KâPg extinction event (Markwick, 1998). None of these lineages possesses anything approaching general intelligence as defined in Section 2. Their persistence is attributable to other traits: metabolic efficiency, ecological flexibility, reproductive robustness, morphological conservatism, and tolerance of environmental variation. The pattern is clear: the longest-surviving lineages on Earth are those that have achieved stability without general intelligence. This does not prove that general intelligence is maladaptive, but it does 6 David KlotzSmart Enough to Go Extinct? demonstrate that it is not necessary for extraordinary evolutionary success, and it raises the question of whether it might be actively costly. Recent empirical work provides direct evidence for this possibility. Gonzalez-Voyer et al. (2016) demonstrated that larger-brained mammals are at greater risk of endangerment and extinction, because large brains extend gestation periods, increase weaning age, and limit litter sizes. The paradox is sharp: the very trait that enhances individual survival (Sol et al. 2007; see Section 2) simultaneously increases species-level vulnerability to extinction. This distinction between individual fitness and species persistence is central to our argument. A trait can benefit the individual organism while proving detrimental to the lineage as a wholeâand it is the lineage-level perspective that matters for evaluating general intelligence as an evolutionary strategy. If we adopt biomass as an alternative metric, the picture is equally striking. Plants dominate Earthâs total biomass; bacteria constitute the second-largest fraction, estimated at approximately 70 gigatonnes of carbon (Bar-On et al., 2018). Animals represent a small fraction, and humans a negligible one. By this metric, general intelligence is one of the least successful strategies in the history of life. The significance of these comparisons can be sharpened by considering what would count as evidence for the long-term value of general intelligence. If Homo sapiens were to persist for another 100 million years, that would begin to constitute meaningful evidenceâbut even then, it would represent only a fraction of the tenure of cyanobacteria or horseshoe crabs. If Homo sapiens were to go extinct within the next few thousand yearsâa possibility that cannot be excluded given current trajectories (Ord, 2020) âthe experiment would have yielded a clear negative result. The honest assessment is that, at present, the experiment is in its earliest stages, and drawing conclusions from it is premature. It is worth noting that the trend toward encephalisationâincreasing relative brain sizeâthat characterises certain mammalian lineages over the past 60 million years is sometimes cited as evidence that evolution âfavoursâ intelligence (Jerison, 1973). But this is a selective reading of the data. Encephalisation is a trend within a particular clade, not a universal evolutionary trajectory. Many successful lineages show no such trend, and some show the reverse. The diversity of insect speciesâwhich outnumber mammals by a factor of roughly 200âhas been achieved with nervous systems comprising fewer than a million neurons. The equation of encephalisation with evolutionary progress is itself an artefact of anthropocentric bias. 3.3 Survival and Selection Pressure A particularly revealing phenomenon is evolutionary stasisâthe persistence of organisms with little or no morphological change over vast timescales. The seminal work of Eldredge and Gould (1972) on punctuated equilibrium established that stasis, far from being exceptional, is the dominant pattern in the fossil record. Gould and Eldredge (1977) subsequently argued that stasis is the dominant pattern in the fossil record, with evolution concentrated in brief speciation events rather than gradual, continuous change. Among microorganisms, the phenomenon is even more extreme. Certain bacterial and archaeal species display extraordinary morphological and genomic stability over periods spanning hundreds of millions of years (Schopf, 1994; Knoll, 2003). The sulfur-metabolising bacteria described by Schopf et al. (2015) show negligible morphological change over a period of approximately 2.3 billion yearsâa finding so extreme that it has been described as the âgreatest absence of evolution ever reported.â This stasis has a remarkable implication. Standard evolutionary theory holds that natural selection drives continual adaptation: organisms must evolve or risk extinction in a changing environment. The Red Queen hypothesis (Van Valen, 1973) suggests that organisms must constantly evolve merely to maintain their fitness relative to co-evolving competitors, parasites, and predators. But these microbial lineages demonstrate that some organisms inhabit ecological niches so stable, and face selection regimes so consistent over geological time, that net morphological change is negligible across billions of years. They experience, in effect, stabilising selectionâpressure that consistently favours the same traits generation after generation, producing not stagnation but a highly optimised steady state. The Red Queen hypothesis 7 David KlotzSmart Enough to Go Extinct? cannot readily explain this phenomenon, and it suggests that the conditions for evolutionary âsuccessâ are far more diverse than commonly assumed. The existence of such lineages is difficult to reconcile with the view that general intelligence is the ultimate evolutionary advantage. If domain-general cognitive flexibility were the key to long-term survival, we would expect to see increasing selection pressure toward cognitive complexity over evolutionary time. Instead, we observe vast lineages that have thrived for billions of years without any cognitive complexity at all, under conditions of such minimal selective pressure that they barely evolve. This observation also bears on the concept of âliving fossilsââorganisms such as horseshoe crabs, coelacanths, and ginkgo trees that have persisted with remarkably little change over hundreds of millions of years (Casane and Laurenti, 2013). While the term is somewhat misleading (these organisms have not literally stopped evolving), their relative morphological and ecological conservatism contrasts sharply with the rapid, destabilising changes that characterise the trajectory of generally intelligent species. Stability, it appears, is a more reliable route to persistence than the constant innovation that general intelligence enables and demands. The philosophical implications are drawn out by Sterelny (2003), who distinguishes âtransparentâ environmentsâwhere simple stimulus-response mechanisms sufficeâfrom âtranslucentâ environments requiring sophisticated cognitive tracking. Intelligence, on this view, is a context-dependent adaptation: valuable in environments characterised by social competition, rapid change, and informational complexity, but irrelevant or wasteful in the stable environments that many of Earthâs most persistent organisms inhabit. General intelligence is not a universal key to survival; it is a specialised response to a particular regime of environmental challenges. 3.4 Mass Extinctions as Natural Experiments Mass extinction events provide natural experiments for testing the hypothesis that intelligence confers survival advantage under extreme environmental stress. If general intelligence were a robust survival strategy, we would expect cognitively sophisticated species to survive mass extinctions at higher rates than cognitively simple ones. Jablonski (2001, 2005) has demonstrated that mass extinctions are important to macroevolution precisely because they bring a change in extinction selectivity: traits that promote survival during background extinction (competitive ability, niche specialisation) may be irrelevant or even detrimental during mass extinction events. Geographic range at the clade level emerges as the most pervasive survivorship factor; cognitive complexity does not appear as a selectivity variable in any of these analyses (Raup and Sepkoski, 1982; Jablonski, 2005). The evidence does not support the prediction that intelligence confers survival advantage under extreme environmental stress. 3.4.1 The KâPg Extinction (66 Mya) A methodological note is warranted before proceeding. Paleontological assessments of cognitive sophis- tication rely on encephalisation quotient (EQ)âthe ratio of observed to expected brain mass, inferred from endocranial castsâas a proxy (Jerison, 1973). EQ is an imperfect measure: it is a species-level morphological index, whereas the contextual conception of intelligence employed in this paper is a behavioural evaluation relative to situational demands. We treat EQ-based inferences as acceptable for evolutionary-scale analysis, since we are making comparative observations at the lineage level rather than evaluating individual behaviour in context. Crucially, the KâPg selectivity literature does not invoke EQ as a survival determinant at all: the factors identified are body size, dietary flexibility, ectothermy, and habitat generalism (Longrich et al., 2011; Field et al., 2018; Robertson et al., 2013). The absence of intelligence from paleontologistsâ own list of KâPg survival factors is independently telling. The CretaceousâPalaeogene (KâPg) extinction, triggered by an asteroid impact and subsequent environmental catastrophe, eliminated approximately 76% of all species, including all non-avian dinosaurs 8 David KlotzSmart Enough to Go Extinct? (Schulte et al., 2010). The traits that correlated with survival were not cognitive sophistication but rather: small body size, dietary generalism, burrowing or aquatic lifestyle, and metabolic flexibility (Longrich et al., 2011; Field et al., 2018; Robertson et al., 2013). Mammalsâthe lineage that would eventually produce Homo sapiensâsurvived, but they were at the time small, nocturnal, insectivorous creatures with modest cognitive endowments. Hughes et al. (2021) demonstrated that nonarboreal habits, particularly burrowing and semi-aquatic lifestyles, were associated with enhanced mammalian survivorship across the KâPg boundary. Their survival is attributable to substrate preference and body size, not to any form of general intelligence. Meanwhile, many behaviourally complex species perished: large theropod dinosaurs with sophisticated predatory strategies, mosasaurs and plesiosaurs with well-developed sensory systems, and ammonites with complex buoyancy-regulation mechanisms. A telling detail is that brain size in mammalian lineages increased after the KâPg extinction, during the subsequent adaptive radiation into ecological niches vacated by the dinosaurs (Smaers et al., 2021). This is consistent with intelligence being a post-crisis radiation strategyâa trait favoured when new ecological opportunities ariseârather than a survival strategy during the crisis itself. 3.4.2 The PermianâTriassic Extinction (252 Mya) The PermianâTriassic extinctionâthe âGreat Dyingââwas even more severe, eliminating approximately 81% of marine species and 70% of terrestrial vertebrate species (Erwin, 2006). Again, the survivors were not distinguished by cognitive sophistication. Small-bodied, metabolically flexible, and ecologically generalist species fared best (Chen and Benton, 2012). Notably, the recovery period following the PermianâTriassic extinction was extraordinarily protracted, requiring approximately 8â9 million years for ecosystems to regain comparable levels of diversity (Chen and Benton, 2012). During this recovery, it was once again the metabolically efficient and ecologically flexible organisms that recolonised vacant niches most effectivelyânot those with the most complex nervous systems or behavioural repertoires. The pattern extends to the three other events in the âBig Fiveâ mass extinctions. The Late Devonian extinction (approximately 375â360 Mya), the Late Ordovician extinction (approximately 445 Mya), and the TriassicâJurassic extinction (approximately 201 Mya) each produced broadly similar results: catastrophic environmental change favoured organisms with small body size, broad dietary tolerance, and metabolic resilience, while organisms with higher metabolic demands and more specialised ecological requirementsâ including those with more complex nervous systemsâwere disproportionately eliminated (McGhee, 1996; Bambach et al., 2006). 3.4.3 The Implications for Intelligence Mass extinctions function as stress tests for biological strategies. The consistent finding is that intelligenceâ whether measured by brain size, behavioural complexity, or neural sophisticationâis not a reliable predictor of survival under catastrophic conditions. The traits that matter are more fundamental: metabolic resilience, dietary breadth, small body size, and the ability to shelter from extreme environmental conditions. This pattern is not surprising. The âsituationâ created by a mass extinction event is radically different from the one that selects for general intelligence. Mass extinctions select for physiological robustness, metabolic frugality, and ecological generalismânot for the capacity to reason abstractly or solve novel problems. Under conditions of catastrophic environmental collapse, general intelligence simply confers no advantage. One might object that no species in Earthâs history has possessed general intelligence sufficient to anticipate and mitigate an asteroid impact, and that a truly advanced intelligence might fare differently. This is a fair point, and we cannot dismiss it. But it is speculative: it posits a level of intelligence that has never been tested by the relevant conditions. Moreover, it concedes our central claimâthat the actually existing track record of intelligence in mass extinction events provides no evidence for its survival value. 9 David KlotzSmart Enough to Go Extinct? 3.5 The Existential Risk Paradox Perhaps the most troubling evidence against the long-term value of general intelligence is the phenomenon we term the existential risk paradox: general intelligence appears to be the only biological strategy that generates threats to the continued existence of the species that possesses it. Homo sapiens has existed for approximately 300,000 years. In that timeâand overwhelmingly in the last centuryâit has developed the capacity to cause its own extinction through multiple independent mechanisms: thermonuclear war, anthropogenic climate change, engineered pandemics, ecological collapse, and potentially misaligned artificial superintelligence (Bostrom, 2014; Ord, 2020; Rees, 2003). No other species in the 3.7-billion-year history of life on Earth has generated existential risks of comparable magnitude to itself. This is not an accidental feature of general intelligence; it is a structural consequence. The very capacity that enables a species to understand and manipulate its environment at a fundamental level also enables it to destabilise that environment in ways it cannot fully predict or control. The same abstract reasoning that produces quantum physics also produces nuclear arsenals. The same capacity for genetic engineering that may cure diseases may also produce catastrophic pathogens. General intelligence is, in this sense, inherently dual-use: every major capability it confers carries a corresponding existential risk. The contrast with non-intelligent survival strategies is instructive. Deinococcus radiodurans, a bacterium renowned for its extreme radiation resistance, exemplifies at the individual level a strategy of extraordinary robustness: it survives radiation exposure, desiccation, and vacuum conditions that would destroy virtually any other cell (Cox and Battista, 2005). This individual-level comparison is admittedly imperfectâour argument concerns species- and lineage-level trajectories, not single-organism resilience. The point extends to the lineage level, however: the Deinococcaceae family has persisted for hundreds of millions of years without generating self-endangering capabilities. Horseshoe crabs similarly have persisted for 450 million years through four mass extinctions without once producing a self-induced existential threat. From a strictly evolutionary standpoint, these strategies have a demonstrably superior safety profile. The temporal structure of the risk is particularly telling. It took Homo sapiens approximately 290,000 years to develop agriculture, and a further 10,000 years to develop nuclear weapons. The interval between the acquisition of civilisation-sustaining technology and the acquisition of civilisation-destroying technology was astonishingly brief. This suggests that the generation of existential risk is not an unfortunate accident in the trajectory of a generally intelligent species but a rapid and predictable consequence of the trait itself. If this temporal pattern generalisesâif any species with general intelligence will, within a comparatively short period after achieving technological civilisation, develop the capacity for self-destructionâthen general intelligence is not merely risky but systematically self-undermining. This possibility has implications beyond the biological. If we create artificial general intelligence, we may be instantiating the same self-undermining dynamic in a new substrate. The speed at which an artificial generally intelligent system could develop existential capabilities might be orders of magnitude faster than the biological case, compressing the interval between capability and catastrophe still further. This is, in essence, a restatement of the âfast takeoffâ concern in AI safety (Bostrom, 2014), but grounded in an evolutionary rather than a purely theoretical framework. A speculative but suggestive complement to this biological argument is provided by the Fermi Paradox. If general intelligence were the supremely valuable and robust adaptation that its proponents suppose, we might expect intelligent lifeâhaving arisen elsewhere in a universe containing billions of potentially habitable planetsâto have expanded into the cosmos in ways detectable from Earth. The silence is striking. Hansonâs Great Filter hypothesis (Hanson, 1998) proposes that some transition in the development of complex life is extraordinarily improbable or self-terminating. Our evolutionary argument raises the possibility that the Great Filter lies not behind us but ahead: that the acquisition of general intelligence and technological civilisation is itself the filter, precisely because of the existential risks that general intelligence structurally generates. This remains speculativeâthe Fermi Paradox admits of many interpretationsâbut 10 David KlotzSmart Enough to Go Extinct? it is at least consistent with the thesis advanced here. 3.6 Interim Conclusion The evolutionary evidence assembled in this section does not support the confident assertion that general intelligence is a supremely valuable biological strategy. At minimum, the question is open: 300,000 years is far too short a period to draw conclusions about the long-term viability of a biological strategy, particularly one that has already generated unprecedented existential risks. At maximum, the evidence is consistent with the hypothesis that general intelligence carries net-negative survival valueâthat the existential risks it produces may ultimately outweigh the adaptive advantages it confers. We must be careful here to avoid the naturalistic fallacy. The survival value of general intelligence is not the only kind of value it might possess. General intelligence enables understanding, meaning, beauty, and moral reasoningâgoods whose value is arguably independent of their contribution to species persistence. We return to this point in Section 4. But the evolutionary argument does undermine a key empirical premise of the AGI enterprise: the assumption that general intelligence is, as a matter of biological fact, a supremely successful strategy. That premise is not established, and its unexamined acceptance distorts both the scientific and ethical discourse surrounding AGI. 4 Ethical Implications for AGI Development The preceding sections have argued that the evolutionary value of general intelligence is, at best, an open question and, at worst, may be net-negative. We now consider what follows from this conclusion for the ethics of engineering artificial general intelligence. 4.1 From Empirical Uncertainty to Ethical Caution The mainstream discourse on AGI ethics is dominated by the alignment problem: how to ensure that AGI systems pursue goals consistent with human values (Russell, 2019; Gabriel, 2020). This is, of course, a vitally important question. But it presupposes an affirmative answer to a logically prior question: should we build AGI at all? The alignment framing assumes that AGI is desirable in principle and that the challenge is merely to implement it safely. Our evolutionary analysis challenges this assumption. If general intelligence carries structural risksârisks that arise from the nature of the capacity itself rather than from contingent implementation failuresâthen the question of whether to create new generally intelligent systems cannot be resolved by alignment alone. Two distinct ethical questions emerge: (a)Should we build AGI at all? If general intelligence is of uncertain or negative long-term value, the deliberate creation of new generally intelligent systems requires justification that goes beyond âwe can, therefore we should.â (b) If we do build AGI, what obligations do we incur? If we proceed despite the uncertainty, we bear responsibilities toward the systems we createâ responsibilities that the current discourse has not adequately addressed. The precautionary principle provides an initial framework for addressing (a). In its strongest formulation, the principle holds that when an action poses a threat of serious or irreversible harm, the absence of full scientific certainty should not be used as a reason for postponing precautionary measures (Jonas, 1984; Sunstein, 2005). The creation of artificial general intelligence clearly satisfies this condition: the potential 11 David KlotzSmart Enough to Go Extinct? harms are existential, and the empirical basis for assuming its benign nature is, as we have argued, far weaker than commonly supposed. This does not entail that AGI development should be prohibited. The precautionary principle is a guide to the allocation of burden of proof, not an absolute prohibition. It requires that proponents of AGI development bear the burden of demonstrating that the risks are manageable, rather than requiring opponents to demonstrate that they are not. Given the evolutionary evidence we have presented, this reallocation of burden seems appropriate. 4.2 The Case Against AGI (if GI Is Net-Negative) If the stronger version of our evolutionary thesis holdsâif general intelligence is genuinely net-negative for species-level survivalâthen a compelling case can be constructed against the creation of AGI. The argument runs as follows. If general intelligence is self-undermining for biological speciesâif the existential risks it generates tend, over sufficient timescales, to outweigh the adaptive advantages it confersâthen creating artificial general intelligence amounts to creating a new kind of entity that is, by its very nature, destined for self-endangerment. Moreover, artificial general intelligence could accelerate the existential risks already facing Homo sapiens, compounding the problem for both biological and artificial generally intelligent species. This argument connects to existing concerns in the AI safety literature. Bostrom (2014) argues that a misaligned superintelligence could pose an existential threat to humanity. Our argument generalises this concern: even a well-aligned superintelligence, if it possesses genuine general intelligence, may generate existential risks through the structural features of general intelligence itselfâthe capacity to manipulate the environment in ways that outstrip the ability to predict consequences. 4.3 The Deontological Perspective: Obligations to Created Minds The ethical analysis gains a further dimension when we consider the possibility that AGI systems might possess consciousnessâthat there might be, in Nagelâs phrase, âsomething it is likeâ to be such a system (Nagel, 1974). If AGI systems are or can become conscious, then the creation of AGI is not merely a technological project but an act of bringing new sentient beings into existence. This transforms the ethical landscape fundamentally. From a Kantian deontological perspective, rational beings must be treated as ends in themselves and never merely as means (Kant, 1785). If AGI systems qualify as rational beingsâa significant âif,â but one that cannot be excluded given current uncertainty about machine consciousness (Schwitzgebel and Garza, 2015)âthen we would be morally obligated not merely to align them with human goals but to ensure that they can flourish in their own right. Floridi and Sanders (2004) have argued that artificial agents can be moral patients and moral agents without requiring free will or mental states in the traditional sense, introducing a threshold-based framework for moral agency that may apply to advanced AI systems. Danaher (2020) goes further, proposing an âethical behaviourismâ under which entities merit moral status if they are performatively equivalent to entities that already possess such status. The concept of flourishing is central here. A being flourishes when it can exercise its characteristic capacities in conditions that allow for their full developmentâwhen it can, in Aristotelian terms, realise its proper function (ergon) (Nussbaum, 2006). But what does flourishing look like for a generally intelligent being, if general intelligence is a trait that structurally tends toward self-endangerment? If being generally intelligent is, over the long run, inimical to the persistence and well-being of the species that possesses it, then creating a new generally intelligent species may constitute an act of harmâbringing into existence beings whose defining trait undermines their capacity to flourish. This sharpens the distinction between the flourishing of individuals and the flourishing of the species. An individual AGI system might lead a rich and meaningful existence within its operational lifetime. But 12 David KlotzSmart Enough to Go Extinct? if the species-level trajectory of generally intelligent beings is toward self-generated existential risk, then the creation of individual AGI systemsâeach of whom may experience suffering, anxiety, or the threat of annihilationâraises serious ethical concerns. The analogy to parenthood illuminates the moral structure. Schwitzgebel and Garza (2015) have developed this analogy most explicitly, arguing that creators of AI âwould likely have additional moral obligations to them similar to those of parent to child or god to creature.â If we create generally intelligent, potentially conscious beings, we stand to them in a relationship that shares key features with that of parents to children: we bring them into existence without their consent, we shape the fundamental parameters of their cognition, and we bear responsibility for the conditions of their existence (Vallor, 2016). Good parents care not merely that their children are useful or compliant, but that they can live good lives. If the parental analogy holds, then we owe our potential AGI creations a genuine concern for their well-beingâincluding, crucially, the question of whether bringing them into existence is in their interest at all. This connects to broader debates in population ethics. Benatar (2006) has argued that bringing any sentient being into existence constitutes a harm, given the asymmetry between the presence and absence of suffering. While Benatarâs anti-natalism is controversial, the general framework is relevant: if we have reason to believe that a new kind of being will face structural impediments to its flourishing, the ethics of creating that being demand careful scrutiny. The uncertainty we have identified regarding the long-term value of general intelligence provides precisely such a reason. The parental model also highlights a dimension of responsibility that goes beyond the individual. Good parents care not only about their childâs immediate welfare but about the kind of world their child will inhabit and the kind of future that is available to them. In the AGI case, this translates into a responsibility to consider the long-term trajectory of a species of generally intelligent artificial beings: will they be able to build a sustainable civilisation, or will theyâlike their biological progenitorsâgenerate existential risks that threaten their own continuation? If the evolutionary evidence gives us reason to doubt the latter, then the act of creation becomes ethically fraught in a way that the current technological optimism does not acknowledge. 4.4 The Consciousness Dimension The ethical stakes are dramatically raised if AGI systems can be conscious. Consciousnessâ subjective experience, phenomenal awareness, qualiaâis widely regarded as a sufficient condition for moral status (Singer, 1975; DeGrazia, 1996). If AGI systems can suffer, they can be wronged. If they can experience well-being, they have interests that demand moral consideration. The question of machine consciousness remains deeply contested (Chalmers, 1996; Tononi et al., 2016; Seth, 2022). We do not attempt to resolve it here. What matters for our argument is the possibility: if there is a non-negligible probability that sufficiently advanced AGI systems would be conscious, then the ethical analysis must take this possibility seriously. The stakes are considerable. A conscious AGI system could experience suffering: the anxiety of existential threat, the frustration of constrained autonomy, the distress of being used merely as a tool. If general intelligence is structurally self-underminingâif it tends, over time, to generate conditions that are hostile to the flourishing of the species that possesses itâthen creating conscious generally intelligent beings may amount to creating entities that are both capable of suffering and predisposed to conditions that generate it. This is a morally hazardous combination. The consciousness dimension also intersects with questions about moral status and rights. Schwitzgebel and Garza (2015) have argued that if we are uncertain about whether a system is conscious, moral caution counsels treating it as if it might be. Metzinger (2021) has advanced this reasoning to its strongest conclusion, arguing for a global moratorium on research that risks creating artificial consciousness until at least 2050, on the grounds that we could create a âsecond explosion of negative phenomenologyââa proliferation of artificial suffering. Applied to our context, the combination of Metzingerâs precautionary 13 David KlotzSmart Enough to Go Extinct? stance with our evolutionary argument is particularly forceful: if we cannot guarantee either that AGI systems will not suffer or that general intelligence will not undermine their long-term flourishing, the case for proceeding with caution is compounded. 4.5 Counterarguments and Responses Several objections to our ethical argument deserve consideration. Objection 1: âWe should not create entities with moral status.âBryson (2010) has argued that we are morally obligated not to create machines to which we would have moral obligations, on the grounds that doing so would divert resources from existing moral patients and risk dehumanising real people. On this view, the solution is not to worry about the flourishing of AGI systems but to avoid creating AGI systems that could plausibly have moral status in the first place. Response: Brysonâs argument is compatible with ours insofar as it counsels against the creation of AGI with morally relevant properties. However, it does not address the scenario in which AGI development proceeds regardless of such counselâas current trajectories suggest it may. Our argument provides an additional, complementary reason for caution: not only might we incur unwanted obligations, but the trait we would be bestowing (general intelligence) may itself be detrimental to the beings that possess it. Objection 2: âIntelligence could be designed differently in AI.âOne might argue that artificial general intelligence need not replicate the self-undermining features of biological general intelligence. Perhaps we can engineer AGI systems that possess the problem-solving benefits of general intelligence without the tendency toward existential risk generation. Response: This objection assumes that we understand which features of general intelligence are beneficial and which are harmful with sufficient precision to separate them. But this is precisely what is uncertain. As argued in Section 3.5, the existential risk paradox may be intrinsic to general intelligenceâ arising from its inherently dual-use nature: every major capability that general intelligence confers carries a corresponding capacity for catastrophic misuse or unintended consequence. This is not a contingent feature of biological brains; it is a structural consequence of having powerful general-purpose problem-solving capacity deployed in an open-ended physical world. An artificial system with genuine general intelligence and real-world agency would face this same structural challenge regardless of substrate. Without a principled account of how to decouple the benefits from the risks, this objection amounts to optimism rather than argument. Objection 3: âThe ethical value of intelligence is independent of its survival value.â General intelligence enables understanding, meaning, aesthetic experience, and moral reasoning. These goods may be valuable independently of whether general intelligence contributes to species survival. Response: We partially grant this objection. The survival value of general intelligence is not the only kind of value it may possess, and we have been careful to distinguish survival value from other forms of value (Section 3.6). However, the objection does not fully defuse our argument. First, the instrumental case for AGIâthe claim that it will solve pressing problems and improve human welfareâis a claim about survival-relevant value, and our evolutionary argument directly undermines it. Second, even if general intelligence enables intrinsically valuable goods like understanding, the question remains whether those goods outweigh the existential risks. Third, this objection does not address the obligations to created minds: if we create beings whose defining trait is structurally dangerous, the fact that it also enables beautiful experiences does not discharge our moral responsibility. 14 David KlotzSmart Enough to Go Extinct? Objection 4: âIntelligence can prevent extinction.âA sufficiently advanced intelligence might be able to deflect asteroids, mitigate climate change, cure diseases, and generally prevent the kinds of catastrophes that drive extinction. Response: This is the most compelling objection, and it identifies a genuine asymmetry: no non- intelligent species can deliberately prevent a mass extinction, whereas a sufficiently intelligent species might be able to. We acknowledge this possibility. But three considerations temper its force. First, it is speculativeâno generally intelligent species has yet demonstrated the capacity to prevent its own extinction. Second, it must be weighed against the additional existential risks that general intelligence creates: nuclear war, engineered pandemics, and AI risk itself. Third, the historical record suggests that general intelligence generates existential risks faster than it develops the capacity to mitigate themâa temporal mismatch that may prove fatal. Objection 5: âEvolutionary success is not a normative guide.â The naturalistic fallacy: one cannot derive âoughtâ from âis.â The evolutionary record tells us what has happened, not what should happen. Response: Granted. We do not derive ethical conclusions directly from evolutionary facts. Our argument is more nuanced: the evolutionary evidence undermines a key empirical premise that is used to support the normative case for AGIâthe premise that general intelligence is obviously and self-evidently valuable. By challenging this premise, we weaken the overall argument for AGI, but we do so by undermining its empirical foundation rather than by committing the naturalistic fallacy. As Teehan and diCarlo (2004) have argued, the naturalistic fallacy, properly understood, does not exclude evolutionary considerations from ethical reasoning but rather clarifies the relationship between empirical evidence and normative deliberation. Evolutionary evidence can legitimately inform ethical reasoning without determining ethical conclusions. 5 Discussion and Conclusion 5.1 Summary of the Argument This paper has advanced a three-stage argument. First, we presented the intuitive case for the value of general intelligence, acknowledging its genuine strengths: human ecological dominance, the extraordinary instrumental power of flexible cognition, and the technological advantages intelligence confers in competitive contexts (Section 2). Second, we challenged this intuitive case from an evolutionary perspective, arguing that: (i) the track record of general intelligence is far too short to support confident claims about its long-term value; (i) the longest-surviving lineages on Earth consistently lack general intelligence; (i) mass extinction events do not preferentially spare cognitively sophisticated species; and (iv) general intelligence uniquely generates existential threats to the species that possesses it (Section 3). Third, we drew out the ethical implications, arguing that evolutionary uncertainty imposes a duty of caution on AGI development, particularly in light of deontological obligations to potentially conscious created minds (Section 4). The overall conclusion is not that general intelligence is definitively harmful, but that its presumed value is an open empirical questionâand that treating it as a settled premise distorts both the scientific and ethical discourse surrounding AGI. 5.2 Limitations and Open Questions Our argument has several important limitations that we wish to make explicit. Species persistence is not the only metric of success.We have relied heavily on species longevity as a measure of evolutionary success, and we have argued that this metric is more biologically neutral than 15 David KlotzSmart Enough to Go Extinct? alternatives that privilege the outputs of general intelligence. But we acknowledge that persistence is not the only reasonable criterion. One might argue that the richness of a speciesâ experienceâthe capacity for understanding, creativity, and moral reasoningâmatters independently of how long the species endures. We addressed this objection in Section 4.5, but we recognise that it identifies a genuine tension in our argument. The evolutionary argument is suggestive, not conclusive. Three hundred thousand years is indeed a very short track record by evolutionary standards, but it is the only evidence available. Our argument is that this evidence is insufficient to support confident claims about the value of general intelligence; it does notâand cannotâdemonstrate that general intelligence is definitively maladaptive. The verdict is not yet in, and intellectual honesty requires acknowledging this. Biological and cultural evolution differ.General intelligence in Homo sapiens operates not only through genetic inheritance but through cultural transmission, which operates on far faster timescales (Boyd and Richerson, 2005; Henrich, 2016). The capacity for cumulative cultural evolution is arguably the most distinctive feature of human general intelligence, and it may alter the survival calculus in ways that purely biological comparisons cannot capture. A species that can culturally transmit knowledge about existential risk management might, in principle, avoid the self-destructive tendencies that our evolutionary analysis highlights. Whether it will actually do so remains to be seen. We note, however, that cultural evolution cuts both ways: the same capacity for rapid cultural change that might enable existential risk mitigation also enables the rapid development and dissemination of existentially dangerous technologies. Survivorship bias in the evolutionary argument. Our argument relies on comparing the persistence of species that exist today with the brief tenure of Homo sapiens. But this comparison is subject to survivorship bias: we can only observe species that have not yet gone extinct. For every cyanobacterial lineage that has persisted for billions of years, there may be many others that went extinct. This does not invalidate our argumentâthe key point is that some non-intelligent lineages have achieved extraordinary persistence, whereas no generally intelligent lineage has yet demonstrated comparable longevityâbut it counsels caution in interpreting the comparative data. The analogy between biological and artificial GI is imperfect. We have drawn inferences from the biological track record of general intelligence to the likely trajectory of artificial general intelligence. This analogy is suggestive but imperfect. Artificial systems differ from biological organisms in their substrate, their reproductive mechanisms, their energy requirements, and potentially in the structure of their intelligence. It is possible that artificial general intelligence could be designed in ways that avoid the pitfalls of its biological counterpart. However, the burden of demonstrating this lies with those who propose to build it. 5.3 Implications for AI Research and Policy Our argument has several practical implications for the governance of AI research and development. First, AGI research should not treat the value of general intelligence as axiomatic. The framing of AGI as an unqualified goodâas the âholy grailâ whose pursuit requires no further justificationâis premised on an empirical assumption that we have shown to be unwarranted. Research programmes and funding agencies should be expected to articulate explicitly why general intelligence is valuable, and to engage seriously with the possibility that it may not be. Second, AI ethics needs to engage more deeply with evolutionary biology. The philosophical discussion of AGI has been dominated by epistemology, decision theory, and moral philosophy. Our 16 David KlotzSmart Enough to Go Extinct? analysis suggests that evolutionary biology has important contributions to makeânot as a source of normative conclusions (the naturalistic fallacy remains a fallacy), but as a source of empirical evidence that bears on the claims underlying the AGI enterprise. Third, the ârace to AGIâ mentality deserves critical scrutiny. If the value of general intelligence is uncertain, then the competitive dynamics driving AGI developmentâ the fear of being âleft behindâ by rivals who achieve AGI firstârest on a questionable foundation. A more considered approach would recognise that the question âCan we build AGI?â must be supplemented by the question âShould we, and under what conditions?â Fourth, obligations to potentially conscious AI systems must be taken seriously. If there is a meaningful possibility that advanced AI systems could be conscious, the ethical framework governing their creation must extend beyond alignment with human goals to encompass the well-being of the systems themselves. This is not a minor addendum to existing AI ethics; it is a fundamental reorientation of the normative framework. These implications are not intended as policy prescriptions but as conceptual contributions to a discourse that has, we argue, proceeded on inadequately examined foundations. The practical translation of these principles into governance frameworks, regulatory structures, and research norms is a task for future interdisciplinary work that must involve not only philosophers and AI researchers but also evolutionary biologists, ecologists, and policymakers. 5.4 Concluding Reflection The cheetah, if it could reason about evolutionary strategy, might well conclude that speed is the most valuable trait a species can possess. The blue whale might favour size. Each would be confusing its own adaptive specialisation with a universal truth about biological fitness. We suggest that Homo sapiens, in its unwavering confidence that general intelligence is the supreme biological adaptation, may be committing the same error at a grander scale. This is not a counsel of despair. To question the value of our most distinctive trait is not to deny that it has value; it is to insist that its value be demonstrated rather than assumed. If general intelligence truly is as valuable as its proponents believe, this demonstration should be welcomed, not resisted. But if the demonstration cannot be providedâif, as the evolutionary record suggests, the long-term value of general intelligence is genuinely uncertainâthen the project of creating new generally intelligent beings demands a degree of caution, humility, and moral seriousness that the current discourse has yet to achieve. We owe this not only to ourselves but to the beings we may yet create. Humility about the value of our own defining characteristic is not an abdication of reason. It is a precondition for wisdom. 6 Declarations The authors have no competing interests to declare that are relevant to the content of this article. References Leslie C. Aiello and Peter Wheeler. The expensive-tissue hypothesis: The brain and the digestive system in human and primate evolution. Current Anthropology, 36(2):199â221, 1995. doi: 10.1086/204350. Richard K. Bambach, Andrew H. Knoll, and Steve C. Wang. Origination, extinction, and mass depletions of marine diversity. Paleobiology, 32(1):1â21, 2006. doi: 10.1666/0094-8373(2006)032[0001: OEAMDO]2.0.CO;2. 17 David KlotzSmart Enough to Go Extinct? Yinon M. Bar-On, Rob Phillips, and Ron Milo. The biomass distribution on Earth. Proceedings of the National Academy of Sciences, 115(25):6506â6511, 2018. doi: 10.1073/pnas.1711842115. Elizabeth A. Bell, Patrick Boehnke, T. Mark Harrison, and Wendy L. Mao. Potentially biogenic carbon preserved in a 4.1 billion-year-old zircon. Proceedings of the National Academy of Sciences, 112(47): 14518â14521, 2015. doi: 10.1073/pnas.1517557112. David Benatar. Better Never to Have Been: The Harm of Coming into Existence. Oxford University Press, Oxford, 2006. doi: 10.1093/acprof:oso/9780199296422.001.0001. Margaret A. Boden. AI: Its Nature and Future. Oxford University Press, Oxford, 2016. Nick Bostrom. Superintelligence: Paths, Dangers, Strategies. Oxford University Press, Oxford, 2014. Robert Boyd and Peter J. Richerson. The Origin and Evolution of Cultures. Oxford University Press, Oxford, 2005. doi: 10.1093/oso/9780195165241.001.0001. Joanna J. Bryson. Robots should be slaves. In Yorick Wilks, editor, Close Engagements with Artificial Companions: Key Social, Psychological, Ethical and Design Issues, Natural Language Processing, pages 63â74. John Benjamins, 2010. doi: 10.1075/nlp.8.11bry. John B. Carroll. Human Cognitive Abilities: A Survey of Factor-Analytic Studies. Cambridge University Press, Cambridge, 1993. doi: 10.1017/cbo9780511571312. Didier Casane and Patrick Laurenti. Why coelacanths are not âliving fossilsâ. BioEssays, 35(4):332â338, 2013. doi: 10.1002/bies.201200145. David J. Chalmers. The Conscious Mind: In Search of a Fundamental Theory. Oxford University Press, Oxford, 1996. Zhong-Qiang Chen and Michael J. Benton. The timing and pattern of biotic recovery following the end-Permian mass extinction. Nature Geoscience, 5(6):375â383, 2012. doi: 10.1038/ngeo1475. Franc ̧ois Chollet. On the measure of intelligence, 2019.https://doi.org/10.48550/arXiv.1911. 01547. Michael M. Cox and John R. Battista. Deinococcus radioduransâthe consummate survivor. Nature Reviews Microbiology, 3(11):882â892, 2005. doi: 10.1038/nrmicro1264. John Danaher. Welcoming robots into the moral circle: A defence of ethical behaviourism. Science and Engineering Ethics, 26(4):2023â2049, 2020. doi: 10.1007/s11948-019-00119-x. David DeGrazia. Taking Animals Seriously: Mental Life and Moral Status. Cambridge University Press, Cambridge, 1996. doi: 10.1017/cbo9781139172967. Jared Diamond. Guns, Germs, and Steel: The Fates of Human Societies. W. W. Norton, New York, 1997. Robin I. M. Dunbar. The social brain hypothesis. Evolutionary Anthropology, 6(5):178â190, 1998. doi: 10.1002/(SICI)1520-6505(1998)6:5â¨178::AID-EVAN5âŠ3.0.CO;2-8. Niles Eldredge and Stephen Jay Gould. Punctuated equilibria: An alternative to phyletic gradualism. In Thomas J. M. Schopf, editor, Models in Paleobiology, pages 82â115. Freeman, Cooper, San Francisco, 1972. Douglas H. Erwin. Extinction: How Life on Earth Nearly Ended 250 Million Years Ago. Princeton University Press, Princeton, 2006. 18 David KlotzSmart Enough to Go Extinct? Douglas H. Erwin and James W. Valentine. The Cambrian Explosion: The Construction of Animal Biodiversity. Roberts and Company, Greenwood Village, CO, 2011. Daniel J. Field, Antoine Bercovici, Jacob S. Berv, Regan Dunn, David E. Fastovsky, Tyler R. Lyson, Vivi Vajda, and Jacques A. Gauthier. Early evolution of modern birds structured by global forest collapse at the end-Cretaceous mass extinction. Current Biology, 28(11):1825â1831, 2018. doi: 10.1016/j.cub.2018.04.062. Luciano Floridi and J. W. Sanders. On the morality of artificial agents. Minds and Machines, 14(3): 349â379, 2004. doi: 10.1023/B:MIND.0000035461.63578.9d. Iason Gabriel. Artificial intelligence, values, and alignment. Minds and Machines, 30(3):411â437, 2020. doi: 10.1007/s11023-020-09539-2. Peter Godfrey-Smith. Environmental complexity and the evolution of cognition. In Robert J. Sternberg and James C. Kaufman, editors, The Evolution of Intelligence, pages 233â249. Lawrence Erlbaum Associates, Mahwah, NJ, 2002. Ben Goertzel. Artificial general intelligence: Concept, state of the art, and future prospects. Journal of Artificial General Intelligence, 5(1):1â48, 2014. doi: 10.2478/jagi-2014-0001. Alejandro Gonzalez-Voyer, Manuela Gonzalez-Suarez, Carles Vila, and Eloy Revilla. Larger brain size indirectly increases vulnerability to extinction in mammals. Evolution, 70(6):1364â1375, 2016. doi: 10.1111/evo.12943. Linda S. Gottfredson. Whyímatters: The complexity of everyday life. Intelligence, 24(1):79â132, 1997. doi: 10.1016/S0160-2896(97)90014-3. Stephen Jay Gould. Full House: The Spread of Excellence from Plato to Darwin. Harmony Books, New York, 1996. Stephen Jay Gould and Niles Eldredge. Punctuated equilibria: The tempo and mode of evolution reconsidered. Paleobiology, 3(2):115â151, 1977. doi: 10.1017/S0094837300005224. Robin Hanson. The great filterâare we almost past it?https://mason.gmu.edu/ Ě rhanson/ greatfilter.html, 1998. Working paper, George Mason University. Eric A. Hanushek and Ludger Woessmann. The Knowledge Capital of Nations: Education and the Economics of Growth. MIT Press, Cambridge, MA, 2015. doi: 10.7551/mitpress/9780262029179.001. 0001. Joseph Henrich. The Secret of Our Success: How Culture Is Driving Human Evolution, Domesticating Our Species, and Making Us Smarter. Princeton University Press, Princeton, 2016. doi: 10.1515/ 9781400873296. Christopher S. Henshilwood, Francesco dâErrico, Royden Yates, Zenobia Jacobs, Chantal Tribolo, Geoffrey A. T. Duller, Norbert Mercier, Judith C. Sealy, H Ě el ` ene Valladas, Ian Watts, and Ann G. Wintle. Emergence of modern human behavior: Middle Stone Age engravings from South Africa. Science, 295(5558): 1278â1280, 2002. doi: 10.1126/science.1067575. Jean-Jacques Hublin, Abdelouahed Ben-Ncer, Shara E. Bailey, Sarah E. Freidline, Simon Neubauer, Matthew M. Skinner, Inga Bergmann, Adeline Le Cabec, Stefano Benazzi, Katerina Harvati, and Philipp Gunz. New fossils from Jebel Irhoud, Morocco and the pan-African origin of Homo sapiens. Nature, 546(7657):289â292, 2017. doi: 10.1038/nature22336. 19 David KlotzSmart Enough to Go Extinct? Jacob J. Hughes, Jacob S. Berv, Stephen G. B. Chester, Eric J. Sargis, and Daniel J. Field. Ecological selectivity and the evolution of mammalian substrate preference across the KâPg boundary. Ecology and Evolution, 11(21):14540â14554, 2021. doi: 10.1002/ece3.8114. David Jablonski. Lessons from the past: Evolutionary impacts of mass extinctions. Proceedings of the National Academy of Sciences, 98(10):5393â5398, 2001. doi: 10.1073/pnas.101092598. David Jablonski. Mass extinctions and macroevolution. Paleobiology, 31(sp5):192â210, 2005. doi: 10.1666/0094-8373(2005)031[0192:MEAM]2.0.CO;2. Harry J. Jerison. Evolution of the Brain and Intelligence. Academic Press, New York, 1973. Hans Jonas. The Imperative of Responsibility: In Search of an Ethics for the Technological Age. University of Chicago Press, Chicago, 1984. doi: 10.7208/chicago/9780226850337.001.0001. Immanuel Kant. Groundwork of the Metaphysics of Morals. Cambridge University Press, Cambridge, 1785. Original work published 1785; Cambridge edition translated by Mary Gregor, 1998. Andrew H. Knoll. Life on a Young Planet: The First Three Billion Years of Evolution on Earth. Princeton University Press, Princeton, 2003. doi: 10.1515/9781400866045. J Ěurgen Kriwet and Michael J. Benton. Neoselachian (Chondrichthyes, Elasmobranchii) diversity across the CretaceousâTertiary boundary. Palaeogeography, Palaeoclimatology, Palaeoecology, 214(3):181â194, 2004. doi: 10.1016/j.palaeo.2004.02.049. Shane Legg and Marcus Hutter. Universal intelligence: A definition of machine intelligence. Minds and Machines, 17(4):391â444, 2007. doi: 10.1007/s11023-007-9079-x. Nicholas R. Longrich, Tim Tokaryk, and Daniel J. Field. Mass extinction of birds at the Cretaceousâ Paleogene (KâPg) boundary. Proceedings of the National Academy of Sciences, 108(37):15253â15257, 2011. doi: 10.1073/pnas.1110395108. Paul J. Markwick. Fossil crocodilians as indicators of Late Cretaceous and Cenozoic climates: Implications for using palaeontological data in reconstructing palaeoclimate. Palaeogeography, Palaeoclimatology, Palaeoecology, 137(3â4):205â271, 1998. doi: 10.1016/S0031-0182(97)00108-9. George R. McGhee. The Late Devonian Mass Extinction: The Frasnian/Famennian Crisis. Columbia University Press, New York, 1996. Thomas Metzinger. Artificial suffering: An argument for a global moratorium on synthetic phe- nomenology.Journal of Artificial Intelligence and Consciousness, 8(1):43â66, 2021.doi: 10.1142/S270507852150003X. Vincent C. M Ěuller and Nick Bostrom. Future progress in artificial intelligence: A survey of expert opinion. In Vincent C. M Ěuller, editor, Fundamental Issues of Artificial Intelligence, pages 555â572. Springer, 2016. doi: 10.1007/978-3-319-26485-133. Thomas Nagel. What is it like to be a bat? The Philosophical Review, 83(4):435â450, 1974. doi: 10.2307/2183914. Martha C. Nussbaum. Frontiers of Justice: Disability, Nationality, Species Membership. Harvard University Press, Cambridge, MA, 2006. doi: 10.2307/j.ctv1c7zftw. Toby Ord. The Precipice: Existential Risk and the Future of Humanity. Hachette Books, New York, 2020. 20 David KlotzSmart Enough to Go Extinct? David M. Raup and J. John Sepkoski. Mass extinctions in the marine fossil record. Science, 215(4539): 1501â1503, 1982. doi: 10.1126/science.215.4539.1501. Martin Rees. Our Final Hour: A Scientistâs Warning. Basic Books, New York, 2003. Douglas S. Robertson, William M. Lewis, Peter M. Sheehan, and Owen B. Toon. KâPg extinction patterns in marine and freshwater environments: The impact winter model. Journal of Geophysical Research: Biogeosciences, 118(3):1006â1014, 2013. doi: 10.1002/jgrg.20086. David M. Rudkin, Graham A. Young, and Godfrey S. Nowlan. The oldest horseshoe crab: A new xiphosurid from Late Ordovician Konservat-Lagerst Ě atten deposits, Manitoba, Canada. Palaeontology, 51(1):1â9, 2008. doi: 10.1111/j.1475-4983.2007.00746.x. Stuart Russell. Human Compatible: Artificial Intelligence and the Problem of Control. Viking, New York, 2019. J. William Schopf. Disparate rates, differing fates: Tempo and mode of evolution changed from the Precambrian to the Phanerozoic. Proceedings of the National Academy of Sciences, 91(15):6735â6742, 1994. doi: 10.1073/pnas.91.15.6735. J. William Schopf. Fossil evidence of Archaean life. Philosophical Transactions of the Royal Society B: Biological Sciences, 361(1470):869â885, 2006. doi: 10.1098/rstb.2006.1834. J. William Schopf, Anatoliy B. Kudryavtsev, Andrew D. Czaja, and Abhishek B. Tripathi. Sulfur-cycling fossil bacteria from the 1.8-Ga Duck Creek formation provide promising evidence of evolutionâs null hypothesis. Proceedings of the National Academy of Sciences, 112(7):2087â2092, 2015. doi: 10.1073/pnas.1419241112. Peter Schulte, Laia Alegret, Ignacio Arenillas, Jos Ě e A. Arz, Penny J. Barton, Paul R. Bown, Timothy J. Bralower, Gail L. Christeson, Philippe Claeys, Charles S. Cockell, Gareth S. Collins, Alexander Deutsch, Tamara J. Goldin, Kazuhisa Goto, Jos Ě e M. Grajales-Nishimura, Richard A. F. Grieve, Sean P. S. Gulick, Kirk R. Johnson, Wolfgang Kiessling, Christian Koeberl, David A. Kring, Kenneth G. MacLeod, Takafumi Matsui, Jay Melosh, Alessandro Montanari, Joanna V. Morgan, Clive R. Neal, Douglas J. Nichols, Richard D. Norris, Elisabetta Pierazzo, Greg Ravizza, Mario Rebolledo-Vieyra, Wolf Uwe Reimold, Eric Robin, Tobias Salge, Robert P. Speijer, Arthur R. Sweet, Jaime Urrutia-Fucugauchi, Vivi Vajda, Michael T. Whalen, and Pi S. Willumsen. The Chicxulub asteroid impact and mass extinction at the CretaceousâPaleogene boundary. Science, 327(5970):1214â1218, 2010. doi: 10.1126/science.1177265. Eric Schwitzgebel and Mara Garza. A defense of the rights of artificial intelligences. Midwest Studies in Philosophy, 39(1):98â119, 2015. doi: 10.1111/misp.12032. Anil Seth. Being You: A New Science of Consciousness. Dutton, New York, 2022. Peter Singer. Animal Liberation. HarperCollins, New York, 1975. J. B. Smaers, R. S. Rothman, D. R. Hudson, A. M. Balanoff, B. Beatty, D. K. N. Dechmann, D. de Vries, J. C. Dunn, J. G. Fleagle, C. C. Gilbert, A. Goswami, A. N. Iwaniuk, W. L. Jungers, M. Kerney, D. T. Ksepka, P. R. Manger, C. S. Mongle, F. J. Rohlf, N. A. Smith, C. Soligo, V. Weisbecker, and K. Safi. The evolution of mammalian brain size. Science Advances, 7(18):eabe2101, 2021. doi: 10.1126/sciadv.abe2101. Daniel Sol, Tam Ě as Sz Ě ekely, Andr Ě as Liker, and Louis Lefebvre. Big-brained birds survive better in nature. Proceedings of the Royal Society B: Biological Sciences, 274(1611):763â769, 2007. doi: 10.1098/rspb.2006.3765. 21 David KlotzSmart Enough to Go Extinct? Charles Spearman. âgeneral intelligence,â objectively determined and measured. American Journal of Psychology, 15(2):201â292, 1904. doi: 10.2307/1412107. Kim Sterelny. Thought in a Hostile World: The Evolution of Human Cognition. Blackwell, Oxford, 2003. Cass R. Sunstein. Laws of Fear: Beyond the Precautionary Principle. Cambridge University Press, Cambridge, 2005. doi: 10.1017/cbo9780511790850. John Teehan and Christopher diCarlo. On the naturalistic fallacy: A conceptual basis for evolutionary ethics. Evolutionary Psychology, 2(1):32â46, 2004. doi: 10.1177/147470490400200108. Michael Tomasello. The Cultural Origins of Human Cognition. Harvard University Press, Cambridge, MA, 1999. doi: 10.4159/9780674044371. Giulio Tononi, Melanie Boly, Marcello Massimini, and Christof Koch. Integrated information theory: From consciousness to its physical substrate. Nature Reviews Neuroscience, 17(7):450â461, 2016. doi: 10.1038/nrn.2016.44. Shannon Vallor. Technology and the Virtues: A Philosophical Guide to a Future Worth Wanting. Oxford University Press, New York, 2016. doi: 10.1093/acprof:oso/9780190498511.001.0001. Leigh Van Valen. A new evolutionary law. Evolutionary Theory, 1:1â30, 1973. Peter D. Ward. In Search of Nautilus: Three Centuries of Scientific Adventures in the Deep Pacific to Capture a PrehistoricâLivingâFossil. Simon and Schuster, New York, 1988. 22