Paper deep dive

Practical Judgment, Virtue, and Intuition in the Use of Opaque AI-Enabled Systems

Nathan G. Wood, Andrew P. Rebera

Year: 2026Venue: arXiv preprintArea: cs.HCType: PreprintEmbeddings: 83

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 90%

Last extracted: 7/15/2026, 7:32:13 AM

Summary

This paper argues that the risks associated with opaque, autonomous AI systems—such as unpredictability, loss of control, and ethical violations—can be mitigated by leveraging distinctly human capabilities like practical judgment, virtue, and intuition. Using the military domain as a primary case study, it advocates for a virtue ethics approach over purely technical solutions like Explainable AI, emphasizing that humanistic, non-quantifiable values are essential for responsible AI deployment.

Entities (7)

Opaque AI Systems · technology · 95%Military Domain · application-area · 93%Practical Judgment · human-trait · 92%Virtue Ethics · philosophical-framework · 90%Meaningful Human Control · ethicallegal-concept · 89%Explainable AI (XAI) · technology · 88%Human-in-the-loop (HITL) · operational-concept · 87%

Relation Signals (5)

Opaque AI Systems → exacerbates → Loss of Control

confidence 92% · coupling opacity with autonomy exacerbates basic challenges posed by opacity and introduces additional concerns surrounding, e.g., responsibility attribution... retaining meaningful human control over systems

Military Domain → exemplifies → Human Judgment Necessity

confidence 91% · Throughout the article, we focus on the military domain as an exemplar of the importance of practical judgment, virtue, and intuition as drivers for ethical and effective human decision-making surrounding AI deployments

Practical Judgment → mitigates → Opacity Challenges

confidence 90% · many of these worries can be mitigated by leveraging practical judgment, virtue, and intuition in the deployment and use of opaque AI-enabled systems.

Virtue Ethics → provides → Decision-Making Framework

confidence 88% · starting from a virtue ethics approach which highlights individual character and the centrality of practical judgment, systems which are not fully understood... may still be responsibly and effectively used

Explainable AI (XAI) → attemptsto → Reduce Opacity

confidence 85% · develop so-called explainable AI systems (XAI) which (partially) remove opacity by having systems provide explanations of their behavior and processes

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:AI-enabled systems are seeing increasing deployment across numerous domains, with many being "black boxes" with respect to core functions and capabilities. I.e., many systems take inputs and give outputs, but without users having any ability to see how the former lead to the latter. AI-enabled systems are also being used to augment autonomy in systems, and autonomy coupled with opacity raises numerous concerns surrounding, e.g., the reliability of systems, their regularity in functioning, human ability to control them, or whether deploying opaque and potentially autonomous systems is in compliance with ethical and legal norms. In this article, we argue that many of these worries can be mitigated by leveraging practical judgment, virtue, and intuition in the deployment and use of opaque AI-enabled systems. We show that focusing on these distinctly human capabilities provides a means for bridging between the practical challenges created by opacity and the ethical, legal, and social norms underpinning particular domains. We argue that a core element in doing this is a recognition that many positive human traits are not quantifiable and we therefore must develop training regimen and guidelines on AI deployment anchored in humanistic but non-quantifiable values. Throughout the article, we focus on the military domain as an exemplar of the importance of practical judgment, virtue, and intuition as drivers for ethical and effective human decision-making surrounding AI deployments, but the underlying arguments apply to all domains where opaque and potentially autonomous systems are being deployed (subject to domain-specific alterations).

PDF

Open source PDF →

PDF not stored locally. Use the link above to view on the source site.

Full Text

82,107 characters extracted from source content.

Expand or collapse full text

Practical Judgment, Virtue, and Intuition in the Use of Opaque AI-Enabled Systems Nathan Gabriel Wood | nathan.wood@tuhh.de Andrew P. Rebera | rebera.andrew@gmail.com Abstract AI-enabled systems are seeing increasing deployment across numerous domains, with many being “black boxes” with respect to core functions and capabilities. I.e., many systems take inputs and give outputs, but without users having any ability to see how the former lead to the latter. AI-enabled systems are also being used to augment autonomy in systems, and autonomy coupled with opacity raises numerous concerns surrounding, e.g., the reliability of systems, their regularity in functioning, human ability to control them, or whether deploying opaque and potentially autonomous systems is in compliance with ethical and legal norms. In this article, we argue that many of these worries can be mitigated by leveraging practical judgment, virtue, and intuition in the deployment and use of opaque AI-enabled systems. We show that focusing on these distinctly human capabilities provides a means for bridging between the practical challenges created by opacity and the ethical, legal, and social norms underpinning particular domains. We argue that a core element in doing this is a recognition that many positive human traits are not quantifiable and we therefore must develop training regimen and guidelines on AI deployment anchored in humanistic but non-quantifiable values. Throughout the article, we focus on the military domain as an exemplar of the importance of practical judgment, virtue, and intuition as drivers for ethical and effective human decision-making surrounding AI deployments, but the underlying arguments apply to all domains where opaque and potentially autonomous systems are being deployed (subject to domain-specific alterations). Keywords: AI, Artificial Intelligence, Opacity, Autonomous Systems, Ethics, Virtues 1 Introduction Across many domains, AI-enabled systems are complementing and in some cases outsourcing human capabilities. Up until 2022, most deployments of AI were in the form of narrow systems designed and built using symbolic AI techniques and/or machine learning (ML) and tailored datasets to create task-specific systems (Babu and Banana, 2024; Wu et al., 2025; Markose et al., 2026). However, with the development of generative pre-trained transformers (GPTs) and the highly successful launch of ChatGPT in November of 2022, there has been an explosion of interest in more generally applicable AI systems, in particular LLMs, for a variety of tasks. However, any ML system trained on large datasets and using deep neural networks (DNNs) is apt to be opaque, meaning there is no reliable method for users or even developers to see how a system goes from particular inputs or prompts and arrives at discrete outputs (Loyola-Gonzalez, 2019; Fomin and Astromskis, 2023). This raises challenges from ethical, legal, and operational standpoints, as, among other things, the behavior of opaque systems cannot be as reliably predicted and it is more difficult (perhaps impossible) to rule out certain behaviors or outputs. Opacity can also undermine effective human-machine teaming, as systems whose functioning is unknown to humans (and possibly unknowable) may undermine how well humans can make use of such systems. In response to the challenges posed by opaque systems, many solutions or mitigation strategies have been proposed. Some focus on institutional approaches (Alizadeh et al., 2025), others on education of users or the implementation of rough guidelines to avoid the most dangerous or unwanted uses (Barman et al., 2024), but one of the most ambitious responses – and certainly the most technologically demanding – is to develop so-called explainable AI systems (XAI) which (partially) remove opacity by having systems provide explanations of their behavior and processes (Gunning and Aha, 2019; Gunning et al., 2019, 2021; Das and Rad, 2020; Speith, 2022; Kalasampath et al., 2025). XAI sees particularly strong promotion in domains where opaque AI-enabled systems might be used for safety-critical tasks, such as in medicine (Tjoa and Guan, 2020; Sheu and Pardeshi, 2022) or the military (Christie et al., 2023; Kwik and van Engers, 2023; Lindelauf and Meerveld, 2025).111Cf. (Wood, 2024) for arguments critical of XAI in the military domain. This is especially the case when such AI-enabled systems are embedded in platforms with (some) autonomous functionality, as coupling opacity with autonomy exacerbates basic challenges posed by opacity and introduces additional concerns surrounding, e.g., responsibility attribution,222On “responsibility gaps”, see (Matthias, 2004; Asaro, 2006) (general discussion), (Sparrow, 2007; Sharkey, 2007; Pagallo, 2011) (in the military), (Oimann, 2023) (overview of the military debate). Cf. (Hindriks and Veluwenkamp, 2023; Wood, 2023a; Rebera, 2024; Kiener, 2024) (responses and further developments). deskilling (Ferdman, 2025b, a; Natali et al., 2025), or retaining meaningful human control over systems (Article 36, 2013a; Human Rights Watch, 2018; Amoroso and Tamburrini, 2020; Schwarz, 2021), showcasing a serious need for solutions. Focusing on technical and human limitations and seeking to address these through technological solutions ignores, however, the ways distinctly human traits can be leveraged to mitigate many of the challenges opacity and autonomy bring. Indeed, some basic characteristics of humans which make us liable to make certain (kinds of) mistakes when deploying autonomous and opaque systems can actually be capitalized on to ground positive proactive approaches to responsibly and effectively deploying emerging technologies. In this article, we argue that practical judgment, virtue, and intuition can be crucial tools for mitigating numerous challenges posed by opaque AI-enabled systems, especially when such systems are embedded in autonomous platforms. We show that by starting from a virtue ethics approach which highlights individual character and the centrality of practical judgment, systems which are not fully understood (or potentially even understandable) for certain users may still be responsibly and effectively used, so long as humans deploying the systems have well-developed characters and judgment and minimal sufficient knowledge of systems’ functionality. We also draw out a critical and often overlooked aspect of judgment, namely that it may be treated as a (finite) resource, showing how this understanding can help underpin responsible and context-sensitive uses of opaque systems. Before moving onto the arguments though, two preliminary points are worth addressing. First, many things are labeled as “AI”, with hype around this class of technologies muddying the intellectual space greatly (Tucker, 2022; Marx and Gebru, 2023; Floridi, 2024; Mattmann and Broderick, 2024). Moreover, given how many types of systems genuinely do incorporate AI, and for such a variety of tasks, there are good grounds for us to stop simply saying “AI” altogether and instead provide clarity on what exact systems we have in mind (Wood et al., 2026). In what follows, our primary concern will be systems which make use of machine learning and large datasets or which rely on techniques which lead systems to be opaque. Thus, symbolic AI and certain small ML trained models fall outside the scope of our presentation, as symbolic AI systems are generally transparent ((van Engers and de Vries, 2019, p. 34); (Toy, 2023)), and small ML models may be transparent due to their limited size and complexity. However, once models become sufficiently large, it becomes practically impossible to understand their inner workings, and some AI techniques guarantee opacity. It is these which are our concern.333We will not provide a definition of “AI” here, but for classic and more contemporary views, see, e.g., (Newell and Simon, 1976, p. 116); (McCarthy, 1988, p. 308); (Wang, 2019, p. 19); or (Russell and Norvig, 2021, p. 22). Second, our arguments apply to the use of opaque AI-enabled systems generally, especially such systems when they have autonomous functionality. However, for concreteness in presentation, we will focus on AI use in the military, as virtue ethical approaches are already common in military institutions, practical judgment is a core competency soldiers are trained for, and the challenges of opacity and autonomy in systems are highly pronounced in military contexts. Nonetheless, there are many domains which feature some or all of these points, with medicine showing significant overlap, and the core arguments and conclusions will apply to the use of opaque systems more generally (subject to domain-specific modifications). 2 Opacity and Its Challenges 2.1 Opacity Above, opacity was described in terms of users or developers not being able to see how systems arrive at outputs from given inputs, thus rendering them “black boxes”. More precisely, opacity “is best understood to define the degree of unknown elements in a machine” and “we can understand an ‘opaque system’ to be one that is simply not known in further detail” (Kempt, 2024, p. 30). Following (Burrell, 2016), a central and often underappreciated feature of opacity is that it can be tied to both systems and users.444Explanations are likewise in crucial respects user- and task-dependent. See (Asghari et al., 2021; Langer et al., 2021). Thus, certain systems may be in principle understandable and transparent, but in practice opaque for users lacking capacities necessary for successfully interrogating the systems. Jenna Burrell identifies one part of this form of opacity as “Opacity as technical illiteracy”(Burrell, 2016, p. 4), but practical human limits need not be viewed only through the lens of “illiteracy”. Indeed, some humans competent to in principle understand a machine’s functioning may in practice be unable to because the sheer number of interconnections, nodes, or weights in a model make it infeasible for a human to interrogate the full system. This problem is further exacerbated by the rate at which emerging technologies are being iterated upon, as ML systems which are retrained in the course of months or weeks may not allow for full reviews of models before they are made obsolete by subsequent versions. Systems may also be in principle opaque, meaning no human, either user or developer, may be in a position to understand it. This can be due to a number of reasons: the underlying mathematics or natural laws governing systems may not (yet) be known; systems may have multiple interconnected layers whose meaning or connection to tasks are not fully identifiable for humans; or systems may have authority to rewrite their programs during operation, making it impossible for humans to see with certainty, after deployments, what code was actually running during operation. For ML-based systems, especially those relying on large datasets for training, there will be at minimum a very high – and potentially insurmountable – degree of in practice opacity due to how large training data sets are, how many nodes and weights are present in models, and how many interconnections there are between elements of models. Additionally, because machine learning involves systems themselves forming models of patterns in data and establishing weights between data points, there will often be in principle opacity as well. This is because such systems have self-determined a portion of their programs without a human necessarily overseeing the process, and without a human necessarily even knowing what elements of the resulting programs are due to direct human inputs, which are logical implications of human inputs, and which are statistically learned from training data. In any event, virtually all ML systems trained on large datasets will be both in practice and in principle opaque with regards to core aspects of their training process and functionality. 2.2 Basic Challenges of Opacity Unfortunately, while numerous narrow applications of AI are deterministic or admit only minor opacity, the types of AI increasingly in use in society at large and in various critical domains are in practice and in principle opaque for all of the above reasons. For example, large language models (LLMs), generative AI (GenAI) systems used for audio, image, or video generation, or any system based on neural networks will lack explainability and be both in practice and in principle opaque (Raleigh and Knoks, 2025; Søgaard, 2023). This opacity can make systems unpredictable for users, lower users’ trust in systems, make usage of systems potentially negligent (if unpredictability is apt to lead to dangerous outcomes users can foresee), and it may undermine how well or meaningfully humans can control systems.555For these various concerns, see, e.g., (Fraser et al., 2022; Davidovic, 2023; Robbins, 2023; Søgaard, 2023; Toy, 2023; Freiman et al., 2025; Raleigh and Knoks, 2025; Boisseau, 2026). Much of the research on challenges of opacity focuses on ethical and legal issues attending use of such systems, but it is also critical to bear in mind that all of these worries present operational risks as well, as unpredictability, lowered trust, potential legal negligence, and loss of broad societal acceptance of systems may all negatively impact how effectively they can be used. In short, opacity can be the source of significant challenges arising from numerous normative and organizational sources. 2.3 Opacity and Autonomy Coupling machine learning with autonomy can exacerbate all of the concerns sketched above. Very briefly, autonomous systems may be conceived of as systems which can execute core tasks or “critical functions” normally requiring human judgment (International Committee of the Red Cross, 2014, p. 5), and do so without contemporaneous human input or oversight.666Such definitions can be found for autonomous systems in, e.g., the military ((Williams, 2015; Boothby, 2016; Caron, 2020); (International Committee of the Red Cross, 2021, p. 1); (US Department of Defense, 2023, p. 21); (Wood, 2023b; Pacholska, 2024)), medicine ((Moustris et al., 2011); (Yang et al., 2017, p. 1); (Haidegger, 2019)), and for machine/AI systems generally (Fahimi, 2009; Ezenkwu and Starkey, 2019; Walsh et al., 2021). Note that some debates discuss machines “making decisions” or similar concepts (e.g., (Heyns, 2016)), but from philosophical, ethical, legal, and technical standpoints, machines cannot be said to “decide”, “choose”, “act”, or otherwise engage in agential activities. Rather, they “execute processes” in line with their programming or construction (see (Wood, 2026)). When systems are opaque and capable of carrying out such “critical functions” autonomously, this compounds the risks involved. During the use of a merely opaque but still fully human-controlled system, there is usually a persistent ability for the system to be constrained or, if need be, turned off. This means if one of the risks from opacity does materialize, say, the system being potentially unpredictable, then there is still an opportunity for the human overseeing the system to mitigate if not avoid unwanted consequences of that unpredictability. For concreteness, suppose security services are using an opaque ML-trained object identification system to flag potential threats, and due to that opacity, the system might give recommendations or analysis which users could not predict. Assume it flags some unarmed individual as a threat. Nothing beyond this happens though, as the system is not autonomous and a human must decide on a next course of action. On reviewing video footage, the human sees the flagged individual is not carrying a weapon, and the system must have, due to some issues in its training set or learned patterns, mistakenly given that assessment, and done so in a way that might not have been predictable to users. Because a user was responsible for what would happen next though, the error which was unforeseen due to system opacity could be prevented from leading to further tragedy. If that same opaque object identification system is embedded in an autonomous weapon system which can select and engage targets identified as threats, it is clear how quickly problems may arise. Opacity in the former system may prevent users from knowing what might lead the system to make mistakes in identifying objects, making it potentially impossible to add deployment parameters or limit deployments to address possible mistakes – because users may not even know what mistakes could occur, or why. Giving the opaque system authority over autonomous engagement could therefore lead to unforeseen and potentially unforeseeable mistaken engagements. These engagements may also be fully outside of human control – say, because the system is operating beyond the limits of operator connection range – meaning there may be no one who could halt a mistaken engagement once it has begun. Similar problems arise with regards to all of the other challenges of opacity discussed. Embedding opaque systems within autonomous systems leads to deep issues for control, predictability, and both responsible and effective deployment of emerging technologies. And while some of these challenges may be partially addressed through technical or institutional fixes, it is critical we also recognize how basic humanistic approaches may be leveraged to improve the efficacy and robustness of efforts to address these challenges. 3 Virtue, Practical Judgment, and Intuition Opaque and autonomous systems do not exist in a vacuum, but participate in broader contexts, alongside other actors and decision-makers (some of whom are human and some of which may not be). The broader socio-technological systems in which opaque systems are embedded can be designed and maintained to promote effective human involvement and accountability. Human involvement is valuable because humans have capabilities machines do not, for example the capacity for moral responsibility. Including humans prominently in the setups in which opaque systems are embedded ensures appropriate chains of responsibility are maintained or established.777On “responsibility gaps”, see note 2. Well-known strategies for this include human-in-the-loop (HITL) and human-on-the-loop (HOTL) approaches (Human Rights Watch, 2012; Boulanin and Verbruggen, 2017; Calvert et al., 2024). With HITL, systems require human intervention to complete their task (e.g., a human must approve procedure from one (sub-)decision or process to the next). With HOTL, systems may execute subsequent processes freely, but under the supervision of a human who can intervene. The difficulty with these approaches is that humans being in- or on-the-loop does not necessarily entail they have the ability to intervene as required – it does not ensure meaningful human control (Article 36, 2013a, b; Roff and Moyes, 2016; Santoni de Sio and van den Hoven, 2018);888Cf. (Wood, 2026). humans may have sub-optimal understanding of systems or their impacts; they may not be fully able to monitor or intervene in all relevant steps in decision-processes; they may not be physically capable of intervening (e.g., no human is fast enough to intervene in high-frequency trading) (MacKenzie, 2021); and they may not be cognitively or psychologically capable of intervening (e.g., if affected by automation bias (Johnson, 2022)). One response to such concerns is to task oversight to another machine, a machine unaffected by fatigue, distraction, cognitive overload, etc. In some cases, this may be appropriate. Yet humans bring added value beyond the capacity for taking responsibility (or being held responsible) for outcomes, as humans have oversight capacities machines lack: humans are capable of appreciating, understanding, and responding to situations in ways machines cannot, capacities which are uncodifiable. (If they were codifiable, they could – and likely would – be coded into the opaque system or the supervisory systems overseeing it.) Thus, these human abilities are not reducible to the application of a rule-book, but rather involve a more intuitive sensibility and capacity for practical judgment. The idea of a specific, perhaps even sui generis, sensitivity towards salient features of situations, and the capacity to respond on the basis of it, is common to discussions of expertise or excellence across many fields. In ethics, this sits most naturally within the tradition of virtue ethics. The virtuous agent is, on the Aristotelian model, a person who sees and assesses situations rightly, and who responds to them – in both action and feeling – in appropriate ways: the virtuous person sees the thing to do, does it, and feels good about doing it (Aristotle, 2009, Nicomachean Ethics, p. 1729–1867). Virtues traditionally associated with military practice include courage (both physical and moral), loyalty, honor, discipline, integrity, respect, and (of increasing prominence) virtues such as restraint, compassion, and mercy (Olsthoorn, 2011; Sparrow, 2013; Renic, 2018). Virtue ethics can be conceived as directly providing action-guiding rules: virtues as generating prescriptions, vices as generating prohibitions (Hursthouse, 2010, p. 36–39). Yet it need not be taken in this way. A more particularist approach (e.g., (McDowell, 1998)) may focus on the sensitivity of the virtuous person to uncodifiable but ethically salient factors in specific circumstances: the virtuous person sees what is called for, in a given situation, in a way that resists systematic formulation and in no way appeals to rules (indeed is often suspicious of the very concept of following a rule). This sort of view might equally be cashed out simply in terms of the virtue of practical wisdom. Practical wisdom is understood in various ways, but can be broadly taken to be a meta-virtue allowing agents to negotiate issues not amenable to precision, codification, or rule-application – e.g. identifying the “mean” between excess and deficiency in relation to a moral virtue (e.g., balancing bravery with recklessness), resolving tensions among conflicting recommended courses of action, or recognizing and weighing morally relevant features of a situation.999On practical wisdom, see (Aristotle, 2009, Nicomachean Ethics, Book VI); (Russell, 2009; Miller, 2023). On this conception of virtues and virtuous agents, judging the right thing to do is not primarily an algorithmic process of deliberation. The virtuous person’s good judgment is a matter of seeing situations rightly and responding appropriately, driven by good character, virtues, well-trained intuitions and experience. Rule-based processes may be necessary when it is not clear, even to an experienced agent, what is to be done. But, on the present view, such cases are the exception (for the virtuous person). Rejecting a formulaic, rules-based approach to judgment and decision-making when dealing with opaque AI systems is important for at least three reasons. First, intuitive sensibility and practical judgment make possible insights and judgments that are not possible, we contend, for AI systems. A military targeting system may correctly identify enemy combatants as legitimate targets who can be attacked, while missing morally salient factors raising questions over whether they should be attacked. A system designed to identify targets cannot possibly be trained on every factor that could influence a decision to deliver lethal force (Michel, 2020). Examples of this include so-called “naked soldier” cases, in which decisions whether to attack legitimate targets are muddied by unexpectedly mundane situational factors such as someone bathing or enjoying the sunshine (Walzer, 2006; Restrepo, 2019, 2020).101010Cf. (Zając, 2022). (Feuer, 2024) discusses naked soldier cases with respect to AI-enabled systems in the military. Note that our point here concerns AI systems’ blindness to situational factors surrounding the legitimacy of killing naked soldiers, not whether such killing is indeed permissible (for such discussion, see (Zając, 2022)). Moreover, with opaque systems, it is uncertain what factors have been considered when arriving at outputs, undermining confidence in those outputs. Second, whether or not AI systems are capable of identifying salient situational factors, humans are – or can be trained to be – extremely good at identifying these. The Aristotelian tradition emphasizes that developing the virtues is not a matter of learning rules, but of habituating the right behavioral and emotional responses to situations (e.g., learning to take pleasure in virtuous actions and to feel ashamed of disgraceful ones (Burnyeat, 1980)). It is noteworthy here that apparent limitations can, in this respect, be strengths. For instance, the fact that human cognitive limitations necessitate reliance on heuristics, analogies, metaphors and the like, can support us in identifying unexpected similarities or analogies between apparently disparate situations. Humans are (or can be trained to be) excellent at identifying “what matters” in a situation, even if the concept of “what matters” cannot be fully articulated.111111In the military, this is highly relevant for what is known as “operational art”. See (Vego, 2017, 2025; Lyons, 2025; Sookermany and Slensvik, 2026). Humans can also be halted by something “that matters”, even when not on the lookout for “things that matter”. Sensitivity to what matters thus works in both directions. Third, military personnel are already trained to develop situational awareness (the ability to rapidly and accurately size up a situation being a basic competence of good soldiering). Therefore, the fact that military personnel may be required to learn new awareness skills in order to work effectively with opaque and autonomous systems does not represent a significant disruption to how things are currently done. We have, then, the vision of a person who, through judgment based on finely-honed intuition and experience, can size up situations and judge what is the thing to do, sensitive to ethical and other salient contextual factors. This can be characterized as the operationalization of virtue, virtuous perception, or practical wisdom; it is not the application of a rule, algorithm, or inferential move of the kind that could be programmed into a system. This is not to dismiss the importance of rules or technological knowledge and know-how, but it is to recognize that such forms of knowledge are limited. In all practical fields, rules generally admit exceptions. Recognizing exceptional cases is not (or is not only) a matter of applying a further rule. Especially in the military, where rules abound, it is strong intuition, extensive experience, practical wisdom, and good judgment that makes right interpretation of the rules possible (Rebera and Wood, 2025). As AI systems are increasingly deployed, the need of these skills and forms of judgment motivates a renewed appeal to effective human-AI teaming, with each kind of agent playing up to the particular strengths in which they enjoy a comparative advantage. What makes the human capacity for intuitive situation-sensitive insight and judgment so important is that it can be turned onto the opaque systems themselves. That is, an AI system can be taken to be either the object of human judgment or part of the context in which a human agent makes judgments and decisions. And if the problem with opaque systems is that they are not susceptible to direct interpretive analysis of their processes, it is at least possible that their performance can become the object of precisely the kind of wise, intuitive understanding and insight that humans specialize in. The system may not be transparent, but expert human judgment often involves the kind of intuitive understanding that transcends transparency. For instance, the distinction between an AI system being “richly teamed with”, as opposed to merely “used by” a skilled operator is analogous, according to (Wood, 2024),121212Citation of author’s unpublished manuscript removed for anonymity. to the way in which a skilled animal handler knows their animal – being sensitive to various “subtle factors and cues” that alert them to what the system or animal is doing, is likely to do, why it did something, what its current status is, and so on (Wood, 2024, p. 8). Training an operator to work with an AI system – whether opaque or not – is, on this model, partly a matter of “getting to know” it (as you might “get to know” a new colleague). Techniques such as XAI might support the getting to know you process, but they cannot be all there is to it. Part of getting to know a system is becoming familiar with the extent of its capacities, understanding what it can(not) do, and under what circumstances; learning to read the signs that signal that it is operating at the edge of its capacity or that it can take on more or greater loads. Generally, getting to know a system involves developing an accurate understanding of its capacities and of the conditions under which it can be relied upon to perform optimally.131313Citation of author’s unpublished manuscript removed for anonymity. Effectively and richly teaming with an AI system involves accurately understanding its capacities in contrast to, and in combination with, one’s own (or those of one’s team). It involves recognizing when and under what circumstances (and with what exceptions and limitations) human and machine are interchangeable. Here, the importance of not only traditional moral and military virtues becomes apparent, but also intellectual virtues. These may include virtues such as algorithmic literacy, epistemic courage, or intellectual humility.141414Following (Vallor, 2016), we may think of these as, or in close relation to, “technomoral virtues”. AI systems may be more reliable than humans for certain tasks but, because unable to take responsibility, less well suited to them overall. Alternatively, humans may clearly be the better choice for certain roles up until the point at which the workload outruns human limits of cognitive capacity and attention – at which point AI systems become plausible replacements. In such cases, good judgment is needed to understand the capacities and limitations of both players. One must allocate tasks between humans and machines, recognizing that AI systems can replace humans only insofar as reliable task completion is at stake, but never insofar as responsibility is to be allocated. When a human role is occupied by AI systems, these replace only the functional aspects of that role. Responsibility may be redistributed, but it is never eliminated. This redistribution should be explicitly and thoughtfully managed – yet another matter of sound judgment. It is to be noted that the intelligence and judgment required to be able to manage this redistribution of responsibility effectively is itself situation-specific. In particular, the profile of a particular role – and hence the possibilities for AI systems to contribute to it – will appear differently depending on one’s position in an institutional hierarchy. In the military, the decision to deploy opaque AI systems might be taken at senior commander levels, based only on generic understandings of the capacities of the systems and of the units into which they will be deployed, whereas unit-level deployment decisions may be taken with an eye to which specific team members best know the systems, which are best equipped to deal with them under expected circumstances, and so forth. The distinctively human capacity for making moral and other evaluative judgments based on an uncodifiable sensitivity to the particularities of given situations is precisely what is required to ensure that the introduction of opaque systems – which are powerful but lack this capacity – is sensitively handled. And this is made possible by turning that uncodifiable sensitivity and judgment into an asset for managing deployment of opaque systems. 4 A Humanistic Response to Opacity’s Challenges Practical wisdom, practical judgment, and sensitivity are essential tools which complement technical and rules-based approaches to responsibly deploying opaque and autonomous systems. And for any such deployment, the first and most important point where judgment is necessary is within the first basic decision: Do I deploy this system? Taking for granted all of the risks and challenges of opacity sketched in Section 2, all other things being equal, the default response on ethical, legal, and prudential grounds should be not to deploy an opaque system. Thus, unless the opaque system clearly improves outcomes, streamlines processes, or reduces the necessary inputs for a given output, deployers should opt for a deterministic system or direct human action. But what about when the deployment of an opaque system is expected to improve the situation in some fashion? How can wisdom, judgment, and sensitivity be beneficial for mitigating the risks of opacity? To answer these questions, we must recognize and unpack a critical aspect of judgment, namely that judgment is a resource. 4.1 Judgment as a Resource Within any given window of time, a particular individual will only have so much capacity for judgment which they can use. Some, by nature or by training, may have more judgment, or be able to replenish their judgment more quickly. Individuals may also require different amounts of judgment for distinct tasks; e.g., a veteran commander may be able to quickly, reliably, and with little effort make sound judgments based on intuition and the information available, while more novice commanders may expend much judgment in the same scenario, simply because they lack the experience and intuitions to decide easily, experience and intuitions which, respectively, feed into and come out of practical judgment. The amount of judgment one has at any given moment may potentially be depleted – due to fatigue, cognitive overload, judgment impairment through emotional stress, etc. – at which point decisions will become confused, information will be overlooked, and actions which are potentially dangerous may be taken. When judgment is depleted, it is necessary that decision-makers reduce how much judgment they must use – by reducing their cognitive and decisional load – or they may be required to fully withdraw from a given task and replenish their stocks of judgment – bluntly put, a nap is sometimes the operationally sound decision. Knowing how much judgment one has, how quickly it replenishes, and when it is responsible to offload tasks in order to marshal one’s reserve of judgment are all things requiring not just technical or institutional fixes, but which demand individuals have well-developed virtuous character, understand themselves and their missions well enough to know where their limits lay, and possess the courage to place appropriate limits on their own decision-making. Taking these considerations and applying them to the question of whether to deploy opaque systems at all, we arrive at a contextually-grounded conclusion: users should consider deploying opaque systems when they assess that, were they not to, they would deplete their pool of judgment needed for responsibly executing current tasks and they would not be able to replenish (enough of) it as is needed for upcoming tasks. For example, suppose a military commander is faced with a developing conflict scenario where an adversary has not made any declarations of hostilities and has not yet struck the commander’s forces or home territory, but manned and unmanned aerial and ground units are being mobilized at scale. The commander feels competent to guide defensive efforts in response to the current threat picture she is facing – she deems her pool of judgment and its replenishment rate to be sufficient for making assessments over all critical decisions she must make. However, she also assesses that if the enemy adds another type of threat, opens up another front for action, or begins sabotage actions in her rear areas, then she will no longer be able to reliably and safely exercise the required level of judgment in addressing the full array of threats she is facing. Assume also that the deterministic systems at her disposal are not able to sufficiently reduce the overall amount of judgment she has need of, and that all her subordinates and superiors are in similar situations and do not have excess capacity which she might leverage (i.e., the deployment of an opaque system represents the only viable alternative for retaining control). In such a case, the commander could continue as she currently is, using her own judgment to make decisions about tactical and operational responses and not deploying or relying on any opaque systems. In the event that the enemy does carry out actions which overload her capacity to responsibly exercise judgment over all her areas of responsibility, she will however have to decide what to do. She could still continue as is, not deploying opaque systems, and as a result seeing a decline in ability to effectively and responsibly judge what to do in response to the threat picture. The decline in ability may not be immediately apparent, as small lapses may not be noticed, but the cumulative effects of fatigue and cognitive overload may create compounding failures leading to significant mistakes and loss of life. Alternatively, she could offload portions of analysis or decision-making to opaque systems. Such off-loading may also be undertaken in different manners. For example, she could deploy systems which automate some portion of the analysis of intelligence, surveillance, and reconnaissance (ISR) information being acquired for all objects and potential targets in her theater of operations. Alternatively, she could deploy narrower opaque systems which only overtake specific tasks, e.g., launch interceptors autonomously when pre-set triggers are met. She might also delegate the full targeting cycle to an opaque and fully autonomous system, but do this for only a limited set of targets, e.g., incoming missiles and unmanned aerial vehicles. In any event, the commander contemplating the deployment of opaque or autonomous systems or the delegation of critical tasks to them will be presented with an array of potential deployment options and parameters. And the choices over each of these will impact on how much of her judgment she might save for other tasks going forward, and how much she must use now in deciding what system(s) to deploy and under what parameters or limitations. Given the importance of speed in making those higher-level judgments, it is critical the commander has a well-developed character allowing her to confidently and courageously recognize her own limits and judge with those in mind. Because her judgment in that moment is most centrally anchored in the assessment she gives of herself, it is also not something which is amenable to general rules an organization might develop for all. Rather, the character and virtues of the commander herself will be central to this highly subjective assessment, an assessment which crucially hinges on humanistic values such as self-reflection, honesty, integrity, courage (in admitting one’s limits), and judgment. None of these are machine-translatable or fully operationalizable through technical “solutions” such as explainability, and institutionally fixed rules cannot guarantee responsible deployment. That being said, technical features of a system (such as explainability or interpretability) can help commanders better assess what parameters or limits on deployment may be needed, both in general and in response to commanders’ own limits and needs, and institutional rules can provide general guidance and information serving as checks on decision-makers’ intuitions and snap judgments (when intuitions greatly diverge from institutionally set rules, decision-makers are given impetus to reconsider their thinking, as critical factors may have been overlooked). Rules and system design will serve as core enablers and multipliers of how well judgment can be leveraged to improve outcomes, but at the end of the day, rules and design cannot by themselves underpin responsible and effective use of any system, opaque or otherwise. Rather, judgment will always be necessary. And when systems are opaque, autonomous, or both, judgment provides a means for some of opacity’s challenges to be mitigated. Such mitigation is further improved when technical and institutional approaches are also present, but the central need is always judgment, which in turn is developed and entrenched through character and virtues. 5 Human Limitations and Deployment Limitations Focusing on humanistic traits and values such as character, virtue, judgment, or intuition indicates not just positive approaches to responsibly deploying opaque systems, but also limits to be taken into consideration. In particular, human limitations must be taken seriously and translated into deployment limitations. Though by no means an exhaustive accounting, there are three central human limits worth briefly discussing. First, virtually no user of an opaque AI-enabled system will be fully knowledgeable about the technical aspects of the system and also knowledgeable in their domain of expertise and application; e.g., a commander may be highly knowledgeable concerning military science but only minimally technologically literate. Some technical knowledge will always be achievable for professionals of various stripes, but we should not expect any individuals using opaque systems to be as knowledgeable about them as developers would be. Limited technical literacy does not, by itself, constitute a fundamental challenge to deploying systems. However, the degree of individual users’ technical (il)literacy should shape their readiness to rely on systems and the precautions they deem necessary when deploying them. Thus, a commander who has well-cultivated practical judgment and experience in their command role, but who knows they are very uncertain about how a particular opaque system functions, should default to not deploying that system unless they believe failure to do so will lead them to rapidly depleting their well of judgment and then making dangerous – and potentially preventable – mistakes. That being said, even in that case, the commander should try to find other deterministic systems to rely on first, delegate tasks to human subordinates, or try to rapidly gain additional technical knowledge to guide their deployment and use of the opaque system. The latter point is worth highlighting, as technical literacy is not just important for basic decisions about whether to use opaque systems at all, but also with regards to what parameters and limitations on deployment a user may find appropriate. The opacity of some systems, especially those with autonomous functionality, can make them unpredictable in their outputs, and limits on deployment represent a critical way to mitigate that risk. But setting those limits requires minimal understanding of how the system functions, what mistakes it might make, with what severity, etc. If a potential deployer lacks that understanding, then they ought not deploy the system. If, however, they have some understanding, but not a great deal, then their deployment, and the limitations governing it, should reflect this. I.e., geo-temporal limits, conservative decision protocols, and the like can serve to buttress uncertainties about system functionality. These should still though be relied on only as a last resort when the user deems that they must deploy that opaque system because all other options are worse and would lead to degradation of their ability to competently and responsibly judge in critical decisions going forward. A second limitation users may suffer from are limits in their own individual judgment or limits which make decisions more costly for their pool of judgment, reducing overall judgment available. These may be short-term and temporary (fatigue or context-dependent anger may limit judgment), medium-term and open to amelioration (lack of training or experience may make decisions more “judgment-costly”, reducing overall judgment available), or longer-term and more difficult to address (weaknesses in basic character traits or virtues – such as self-reflection, honesty, or courage – can both limit judgment in total and make individual decisions more judgment-costly). If potential users of opaque systems are lacking in basic virtues and character traits necessary for effectively gauging their own capabilities and limits, they may need to be fully removed from decision-making over the deployment of opaque systems (see Section 6 below). This is because, due to their lack of self-reflection, honesty, courage, etc., they are not able to adequately or reliably determine what systems to deploy, when, or under what limits/parameters. Simply put, practical judgment is necessary for responsible use of opaque systems, as it is a prerequisite for being able to make use of technical tools (like explanations) or implement institutionalized rules (like law or doctrine). And if a commander cannot do these things, then they ought not have at their disposal systems whose use demands that. I.e., the system should, at the level above the potential deployer, be limited so as to make it unavailable to the limited individual. For less severe and long-term limits, like temporary fatigue or present (but not enduring) lack of experience, deployers should, if possible, delay the use of opaque systems until they have recovered enough judgment to decide well or have acquired enough experience to develop the instincts, intuitions, and practical wisdom necessary for judging how and under what limits to deploy the system. In these cases, the limits to deployment may be merely temporal, not indicating that the system should not be deployed at all or that it must always have some limits, but rather that the user should delay deployment until their own temporary limits have been addressed. The final human limit to consider is the range of cognitive biases we possess and which impact on our use and reliance on AI-enabled systems, algorithmic tools, and even simply machines as such.151515Our concern is human biases, but there are biases within AI as well which impact responsible deployment. See (Nelson, 2019; Roselli et al., 2019; Gichoya et al., 2023; Zhou et al., 2024; Nudo et al., 2026). Biases or cognitive limits centrally relevant to this are action bias (Patt and Zeckhauser, 2000; Jeremiah and Radics, 2025), automation bias (Cummings, 2017), confirmation bias (Nickerson, 1998), or framing effects (Tversky and Kahneman, 1981; Nelson et al., 1997; Druckman, 2001a, b), though this is by no means an exhaustive list. Each bias we (may) have can indicate for different limits to deployment, and even differences in where and when these limits should be set. For example, framing effects and confirmation bias will be highly relevant during the use of AI-DSS (Arnott, 2006; Phillips-Wren et al., 2019; Minotra and Feigh, 2024; Talbert, 2026), but may have no effect on the deployment of an autonomous system for, say, the full targeting cycle of incoming missiles and unmanned aerial systems. However, automation bias may impact on how we view the risks of offloading a targeting cycle to an opaque autonomous system, potentially tainting our assessments of necessary limits. Action bias may also lead us to more favorably view systems which produce clear and striking outcomes when compared against those which do not, even though the latter may be operationally and strategically superior. Given that each bias raises distinct challenges, and that each of these will interact in different ways with different systems, and also with variance due to different missions or tasks, it is impossible to explore these fully here. Suffice to say though that these myriad complicating factors highlight the importance of practical judgment, practical wisdom, and a virtuous character underpinning decision-making under stress. When time is short and lives are on the line, it may not be possible for an unwieldy manual listing biases, systems, and deployment contexts to be consulted to determine what deployment limits might be needed. But a commander with a well-developed character, practical wisdom, and honest self-reflection may be able to quickly determine a minimally responsible course of action which retains effectiveness in deployment. 6 Objections and Deployment Considerations For virtue and practical judgment to usefully mitigate the challenges of opacity, approaches to developing these and ensuring their robustness in high-stakes and high-stress decision-making contexts are necessary. Virtue ethics and humanistic approaches are already present in society at large and in critical domains where life-and-death decisions must be made (e.g., in the military (Olsthoorn, 2011; Robinson, 2016; Robinson et al., 2016; Schulzke, 2016), medicine (Seoane et al., 2016; Kotzee et al., 2017; Lyon, 2021; Doukas et al., 2022; Kim et al., 2024), or disaster response (Webb, 2004; Mendonça and Fiedrich, 2006; Morton, 2007)). Given this existing body of work, we will set aside basic elements of virtue ethics education and instead focus here on two lingering objections to our proposal and the deployment considerations they point toward. Throughout, we have stressed the importance of virtues and practical judgment in the deployment of opaque systems, but above we noted that not all individuals may have the necessary virtues/judgment to do this responsibly or effectively. We stated that such individuals may need to be fully removed from decision-making over the deployment of opaque systems, but a critic might wonder what that means in practice. To address this, it is worth noting the hierarchical structures of most critical domains, especially the military, which is our main focus. In this area, systems are deployed at different levels, by different individuals, with different skill sets, virtues, and levels of judgment. Additionally, not only are individuals higher in the hierarchy expected to have more experience and judgment surrounding their domain and the tools they may use, but also with regards to their subordinates. And tools might also be stratum-dependent, with the authority to deploy (opaque) systems tied to particular strata of the domain. Concretely, in the military there are strategic, theater-level, operational, and tactical decisions which might be made, and some tools or systems are tied to one or more distinct levels. Decisions about the use of opaque weapon systems may be tactically and operationally relevant, but not something theater-level commanders are as concerned with. Moreover, theater-level commanders may not be able to be concerned with such decisions, as this would be another decision drawing on their pool of judgment (which is finite). Because of this, higher-level commanders delegate this to subordinates more closely tied to the discrete deployments of such systems. Those subordinates, due to proximity of deployment, are also better situated to exercise practical judgment about deployments of that type of system. Importantly, every higher level of command is expected to have a sufficient understanding of their subordinates to know what strengths they have as well. Some lower commanders may be deemed to have enough virtue and practical judgment to be entrusted with decision-making over the deployment of some opaque system, while others might be given explicit orders to (not) deploy the system to prevent mistakes being made at that lower level. When an individual lacks the virtue and judgment to make such decisions themselves, a superior in the institutional hierarchy should be taking that decision away from the subordinate. This is also nothing new, as superiors are always expected to extend and withdraw latitude in keeping with both longer-term and temporary limitations subordinates may suffer. For example, a commander who recognizes a combatant is in distress should give that combatant explicit orders about what they may (not) do, in order to lower the likelihood of their distress leading to mistakes or unwanted outcomes. At a meta-level, the decision space about whether and how to deploy opaque systems should thus be tailored to the subordinate having that decision delegated to them, with superiors leveraging knowledge of their team to ensure individuals unfit to make certain decisions are not given power to do so. A second and central concern in the deployment of any AI-enabled system is how this may affect the skills and capabilities of user/deployers over time. I.e., does use or reliance on the system lead to de-skilling? In response, we would highlight that our proposal is not that some system be relied on, but, on the contrary, that one defaults to not using a system unless the situation leads one to believe judgment will be depleted before all necessary or critical choices have been made. Thus, in keeping with a virtuous epistemic stance and in line with one’s practical judgment, the first and most basic consideration is whether delegating some task to an opaque system is necessary or sensible at all. Further, one is enjoined to only deploy if practical judgment leads one to believe judgment will deplete unless decisions are delegated. Following this stance, a corollary is that whenever a task has been delegated, the delegating individual should, at the end of the timeframe of decision-making, be very near to having no more capacity to reliably make judgments. Illustratively, if we consider individuals as having a “judgment meter” ranging from 100 (full capacity for judgment) to 0 (no capacity for judgment), if some tasks have been delegated to an opaque system, then at the end of the mission or day, the individual who did that delegation should be very near to having 0 judgment left. If that is not the case, then they delegated unnecessarily, and in keeping with a virtuous character, they should reflect on how to better assess situations and themselves in the future, with an eye to not delegating prematurely or unnecessarily. Such a focus on first decisions of whether to deploy systems at all combats de-skilling concerns by making use of systems less regularized and anchoring the importance of judgment at each decision point. A further and more insidious de-skilling concern is whether a machine might always take over a single type of task, thus de-skilling one narrow skill that is routinely the first thing to be delegated away. System design and capability may feed into this worry, with some particular system(s) always taking priority in the delegation pipeline due to those systems, e.g., freeing up the most judgment for the decision-maker trying to navigate complex and fraught domains. Unfortunately, there is no clear and surefire method for combating this concern, partly because of the fact that a system which frees up much judgment capacity should in many respects be given priority by decision-makers. Focusing on virtue, character, and self-reflection may help to mitigate this worry though by enjoining individuals to consider how their own capacities may be being degraded over time. Thus, a focus on humanistic values like virtue and judgment may alter de-skilling (of technical skills) to a scenario of re-skilling (to humanistic skills)(Vallor, 2013, 2014). 7 Conclusion Opaque AI-enabled systems, especially those embedded in autonomous systems, raise numerous and deep worries regarding their responsible and effective deployment. We have argued that virtue, judgment, and intuition may be leveraged to combat many of these challenges. In closing, it is worth stressing that our proposal is not intended as an alternative or competitor to technical and institutional approaches. Rather, humanistic values are complementary to existing frameworks, and indeed necessary for those frameworks to fully bear fruit. Moreover, focusing on the traits and strengths of humans showcases an often overlooked set of values which can be crucial in mitigating worries associated with emerging technologies. Our humanistic approach should thus be viewed through the “Swiss cheese” safety model found in engineering disciplines, where each layer of safety is not meant to fully eliminate risks, but rather to shift where the dangers may lay, so that no set of gaps line up fully to allow risks to materialize (Larouzee and Le Coze, 2020; Wiegmann et al., 2021). Humanistic values will not solve the issue, but they shift the space of dangers, and when coupled with technical and institutional approaches, may allow for responsible and effective use of opaque systems in critical domains. References M. Alizadeh, N. Azizi, S. Mahdavi, and F. Baghlani (2025) Unveiling the shadows: obstacles, consequences, and challenges of information opacity in healthcare systems. Philosophy, Ethics, and Humanities in Medicine 20 (1). External Links: Document Cited by: §1. D. Amoroso and G. Tamburrini (2020) Autonomous weapons systems and meaningful human control: ethical and legal issues. Current Robotics Reports 1 (4), p. 187–194. External Links: Document Cited by: §1. Aristotle (2009) The complete works of Aristotle. Vol. 2, Princeton University Press, Princeton, NJ. Cited by: §3, footnote 9. D. Arnott (2006) Cognitive biases and decision support systems development: a design science approach. Information Systems Journal 16 (1), p. 55–78. Cited by: §5. Article 36 (2013a) Killer robots: UK government policy on fully autonomous weapons. Technical report Article 36, Article 36. Cited by: §1, §3. Article 36 (2013b) Structuring debate on autonomous weapons systems. Technical report Article 36, Article 36. Cited by: §3. P. M. Asaro (2006) What should we want from a robot ethic?. International Review of Information Ethics 6, p. 9–16. Cited by: footnote 2. H. Asghari, N. Birner, A. Burchardt, D. Dicks, J. Faßbender, N. Feldhus, F. Hewett, V. Hofmann, M. C. Kettemann, W. Schulz, J. Simon, J. Stolberg-Larsen, and T. Züger (2021) What to explain when explaining is difficult? an interdisciplinary primer on XAI and meaningful information in automated decision-making. Technical report Alexander von Humboldt Institute for Internet and Society. Cited by: footnote 4. V. S. Babu and K. Banana (2024) A study on narrow artificial intelligence – an overview. International Journal of Engineering Science and Advanced Technology (IJESAT) 24 (4), p. 210–219. Cited by: §1. K. G. Barman, N. G. Wood, and P. Pawlowski (2024) Beyond transparency and explainability: on the need for adequate and contextualized user guidelines for LLM use. Ethics and Information Technology 26 (3), p. 1–12. External Links: Document Cited by: §1. É. Boisseau (2026) Expertise, opacity, and trust in AI systems. Synthese 207 (3), p. 104. External Links: Document Cited by: footnote 5. W. H. Boothby (2016) Weapons and the law of armed conflict. 2 edition, Oxford University Press, Oxford, UK. Cited by: footnote 6. V. Boulanin and M. Verbruggen (2017) Mapping the development of autounomy in weapon systems. Technical report SIPRI, Stockholm International Peace Research Institute (SIPRI). Cited by: §3. M. F. Burnyeat (1980) Aristotle on learning to be good. In Essays on Aristotle’s Ethics, A. O. Rorty (Ed.), p. 69–92. Cited by: §3. J. Burrell (2016) How the machine ‘thinks’: understanding opacity in machine learning algorithms. Big Data & Society 3 (1). External Links: Document Cited by: §2.1. S. Calvert, S. Johnsen, and A. George (2024) Designing automated vehicle and traffic systems towards meaningful human control. In Research Handbook on Meaningful Human Control of Artificial Intelligence Systems, G. Mecacci, D. Amoroso, L. C. Siebert, D. Abbink, J. van den Hoven, and F. Santoni de Sio (Eds.), p. 162–187. External Links: ISBN 9781802204131, Document Cited by: §3. J. Caron (2020) Defining semi-autonomous, automated and autonomous weapon systems in order to understand their ethical challenges. Digital War 1 (1-3), p. 173–177. External Links: Document Cited by: footnote 6. E. H. Christie, A. Ertan, L. Adomaitis, and M. Klaus (2023) Regulating lethal autonomous weapon systems: exploring the challenges of explainability and traceability. AI and Ethics, p. 1–17. External Links: Document Cited by: §1. M. L. Cummings (2017) Automation bias in intelligent time critical decision support systems. In Decision Making In Aviation, p. 289–294. Cited by: §5. A. Das and P. Rad (2020) Opportunities and challenges in explainable artificial intelligence (XAI): a survey. External Links: Document Cited by: §1. J. Davidovic (2023) On the purpose of meaningful human control of AI. Frontiers in Big Data 5, p. 1–5. External Links: Document Cited by: footnote 5. D. J. Doukas, D. T. Ozar, M. Darragh, J. M. de Groot, B. S. Carter, and N. Stout (2022) Virtue and care ethics & humanism in medical education: a scoping review. BMC Medical Education 22 (1), p. 131. External Links: Document Cited by: §6. J. N. Druckman (2001a) Evaluating framing effects. Journal of Economic Psychology 22 (1), p. 91–101. External Links: ISSN 0167-4870, Document Cited by: §5. J. N. Druckman (2001b) On the limits of framing effects: who can frame?. The Journal of Politics 63 (4), p. 1041–1066. External Links: ISSN 1468-2508, Document Cited by: §5. C. P. Ezenkwu and A. Starkey (2019) Machine autonomy: definition, approaches, challenges and research gaps. In Intelligent Computing-Proceedings of the Computing Conference, p. 335–358. Cited by: footnote 6. F. Fahimi (2009) Autonomous robots. Springer, New York, NY. Cited by: footnote 6. A. Ferdman (2025a) AI deskilling is a structural problem. AI & Society. External Links: Document Cited by: §1. A. Ferdman (2025b) AI, deskilling, and the prospects for public reason. Minds and Machines 35 (3). External Links: Document Cited by: §1. A. Feuer (2024) Clothing the naked soldier: virtuous conduct on the augmented reality battlefield. Journal of Military Ethics 23 (3–4), p. 264–276. External Links: Document Cited by: footnote 10. L. Floridi (2024) Why the AI hype is another tech bubble. SSRN. External Links: Document Cited by: §1. V. V. Fomin and P. Astromskis (2023) The black box problem. In Future Law, Ethics, and Smart Technologies, J. Gordon (Ed.), p. 112–125. External Links: ISBN 9789004682917, Document Cited by: §1. H. Fraser, R. Simcock, and A. J. Snoswell (2022) AI opacity and explainability in tort litigation. In 2022 ACM Conference on Fairness Accountability and Transparency, FAccT ’22, p. 185–196. External Links: Document Cited by: footnote 5. O. Freiman, J. McAndrews, J. Mansell, and C. van der Linden (2025) ‘Opacity’and ‘trust’: from concepts and measurements to public policy. Philosophy & Technology 38 (1), p. 29. External Links: Document Cited by: footnote 5. J. W. Gichoya, K. Thomas, L. A. Celi, N. Safdar, I. Banerjee, J. D. Banja, L. Seyyed-Kalantari, H. Trivedi, and S. Purkayastha (2023) AI pitfalls and what not to do: mitigating bias in AI. The British Journal of Radiology 96 (1150), p. 20230023. External Links: Document Cited by: footnote 15. D. Gunning and D. Aha (2019) DARPA’s explainable artificial intelligence (XAI) program. AI Magazine 40 (2), p. 44–58. External Links: Document Cited by: §1. D. Gunning, M. Stefik, J. Choi, T. Miller, S. Stumpf, and G. Yang (2019) XAI – explainable artificial intelligence. Science Robotics 4 (37), p. 1–2. External Links: Document Cited by: §1. D. Gunning, E. Vorm, Y. Wang, and M. Turek (2021) DARPA’s explainable AI (XAI) program: a retrospective. Applied AI Letters, p. 1–12. Cited by: §1. T. Haidegger (2019) Autonomy for surgical robots: concepts and paradigms. IEEE Transactions on Medical Robotics and Bionics 1 (2), p. 65–76. External Links: Document Cited by: footnote 6. C. Heyns (2016) Autonomous weapons systems: living a dignified life and dying a dignified death. In Autonomous weapons systems: Law, Ethics, Policy, N. Bhuta, S. Beck, R. Geiß, H. Liu, and C. Kreß (Eds.), p. 3–20. Cited by: footnote 6. F. Hindriks and H. Veluwenkamp (2023) The risks of autonomous machines: from responsibility gaps to control gaps. Synthese 201 (1), p. 1–17. External Links: Document Cited by: footnote 2. Human Rights Watch (2012) Losing humanity: the case against killer robots. Technical report Human Rights Watch, Human Rights Watch. External Links: Link Cited by: §3. Human Rights Watch (2018) Heed the call: a moral and legal imperative to ban killer robots. Technical report Human Rights Watch, Human Rights Watch. External Links: Link Cited by: §1. R. Hursthouse (2010) On virtue ethics. Oxford University Press, Oxford, UK. Cited by: §3. International Committee of the Red Cross (2014) Autonomous weapons systems: technical, military, legal and humanitarian aspects. Technical report International Committee of the Red Cross. Cited by: §2.3. International Committee of the Red Cross (2021) ICRC position on autonomous weapons systems. Technical report International Committee of the Red Cross. Cited by: footnote 6. F. Jeremiah and R. I. Radics (2025) Making sense of action bias in higher education: pedagogical insights on critical thinking. Behavioral Sciences 15 (10), p. 1372. External Links: Document Cited by: §5. J. Johnson (2022) The AI commander problem: ethical, political, and psychological dilemmas of human-machine interactions in AI-enabled warfare. Journal of Military Ethics 21 (3–4), p. 246–271. External Links: ISSN 1502-7589, Document Cited by: §3. K. Kalasampath, K. N. Spoorthi, S. Sajeev, S. S. Kuppa, K. Ajay, and A. Maruthamuthu (2025) A literature review on applications of explainable artificial intelligence (XAI). IEEE Access 13, p. 41111–41140. External Links: Document Cited by: §1. H. Kempt (2024) (Un)explainable technology. Palgrave MacMillan, Cham, Switzerland. External Links: ISBN 9783031680984, Document Cited by: §2.1. M. Kiener (2024) AI and responsibility: no gap, but abundance. Journal of Applied Philosophy 42 (1), p. 357–374. External Links: Document Cited by: footnote 2. D. T. Kim, M. K. Applewhite, and W. Shelton (2024) Professional identity formation in medical education: some virtue-based insights. Teaching and Learning in Medicine 36 (3), p. 399–409. External Links: Document Cited by: §6. B. Kotzee, A. Ignatowicz, and H. Thomas (2017) Virtue in medical practice: an exploratory study. In HEC forum, Vol. 29, p. 1–19. External Links: Document Cited by: §6. J. Kwik and T. van Engers (2023) Performance or explainability? a law of armed conflict perspective. In Artificial Intelligence and Normative Challenges: International and Comparative Legal Perspectives, p. 255–279. Cited by: §1. M. Langer, D. Oster, T. Speith, H. Hermanns, L. Kästner, E. Schmidt, A. Sesing, and K. Baum (2021) What do we want from explainable artificial intelligence (XAI)? – a stakeholder perspective on XAI and a conceptual model guiding interdisciplinary XAI research. Artificial Intelligence 296, p. 1–24. External Links: Document Cited by: footnote 4. J. Larouzee and J. Le Coze (2020) Good and bad reasons: the Swiss cheese model and its critics. Safety Science 126. External Links: Document Cited by: §7. R. Lindelauf and H. Meerveld (2025) Building trust in military AI starts with opening the black box. War on the Rocks. Note: https://warontherocks.com/2025/08/building-trust-in-military-ai-starts-with-opening-the-black-box/ External Links: Link Cited by: §1. O. Loyola-Gonzalez (2019) Black-box vs. white-box: understanding their advantages and weaknesses from a practical point of view. IEEE Access 7, p. 154096–154113. External Links: Document Cited by: §1. W. Lyon (2021) Virtue and medical ethics education. Philosophy, Ethics, and Humanities in Medicine 16 (1), p. 2. External Links: Document Cited by: §6. M. Lyons (2025) The decline of operational art: the story of a strategic China wargame. War on the Rocks. Note: https://warontherocks.com/2025/12/decline-of-operational-art-story-of-a-strategic-china-wargame/ External Links: Link Cited by: footnote 11. D. MacKenzie (2021) Trading at the speed of light: how ultrafast algorithms are transforming financial markets. Princeton University Press, Princeton, NJ. Cited by: §3. S. Markose, T. Prescott, G. Northoff, E. Cross, and K. Friston (2026) Editorial: narrow and general intelligence: embodied, self-referential social cognition and novelty production in humans, AI and robots. Frontiers in Robotics and AI 12. External Links: Document Cited by: §1. P. Marx and T. Gebru (2023) Don’t fall for the AI hype, with Timnit Gebru. Tech Won’t Save Us podcast, with Paris Marx. Note: https://techwontsave.us/episode/151_dont_fall_for_the_ai_hype_w_timnit_gebru, Accessed: Aug. 6, 2025 External Links: Link Cited by: §1. A. Matthias (2004) The responsibility gap: ascribing responsibility for the actions of learning automata. Ethics and Information Technology 6 (3), p. 175–183. Cited by: footnote 2. C. A. Mattmann and D. Broderick (2024) AI hype versus reality – will it work for you?. In 2024 IEEE Aerospace Conference, p. 1–10. External Links: Document Cited by: §1. J. McCarthy (1988) Mathematical logic in artificial intelligence. Daedalus, p. 297–311. Cited by: footnote 3. J. McDowell (1998) Mind, value, and reality. Harvard University Press, Cambridge, MA. Cited by: §3. D. Mendonça and F. Fiedrich (2006) Training for improvisation in emergency management: opportunities and limits for information technology. International Journal of Emergency Management 3 (4), p. 348–363. External Links: Document Cited by: §6. A. H. Michel (2020) The black box unlocked: predictability and understandability in military AI. Technical report United Nations Institute for Disarmament Research (UNIDIR). Cited by: §3. C. Miller (2023) Three models of practical wisdom. Filosofiska Notiser 10 (1), p. 189–205. Cited by: footnote 9. D. Minotra and K. Feigh (2024) Reviewing linkages between display design and cognitive biases in decision making: an emergency response perspective. Theoretical Issues in Ergonomics Science 25 (6), p. 776–803. External Links: Document Cited by: §5. A. Morton (2007) Great expectations. In Risk: Philosophical Perspectives, T. Lewens (Ed.), p. 84–98. External Links: ISBN 9780415422840 Cited by: §6. G. P. Moustris, S. C. Hiridis, K. M. Deliparaschos, and K. M. Konstantinidis (2011) Evolution of autonomous and semi-autonomous robotic surgical systems: a review of the literature. The International Journal of Medical Robotics and Computer Assisted Surgery 7 (4), p. 375–392. External Links: Document Cited by: footnote 6. C. Natali, L. Marconi, L. D. Dias Duran, and F. Cabitza (2025) AI-induced deskilling in medicine: a mixed-method review and research agenda for healthcare and beyond. Artificial Intelligence Review 58 (11). External Links: Document Cited by: §1. G. S. Nelson (2019) Bias in artificial intelligence. North Carolina Medical Journal 80 (4), p. 220–222. Cited by: footnote 15. T. E. Nelson, Z. M. Oxley, and R. A. Clawson (1997) Toward a psychology of framing effects. Political Behavior 19 (3), p. 221–246. External Links: ISSN 1573-6687, Document Cited by: §5. A. Newell and H. A. Simon (1976) Computer science as empirical inquiry: symbols and search. Communications of the ACM 19 (3), p. 113–126. Cited by: footnote 3. R. S. Nickerson (1998) Confirmation bias: a ubiquitous phenomenon in many guises.. Review of General Psychology 2 (2), p. 175–220. Cited by: §5. J. Nudo, M. E. Pandolfo, E. Loru, M. Samory, M. Cinelli, and W. Quattrociocchi (2026) Generative exaggeration in LLM social agents: consistency, bias, and toxicity. Online Social Networks and Media 51, p. 100344. External Links: Document Cited by: footnote 15. A. Oimann (2023) The responsibility gap and LAWS: a critical mapping of the debate. Philosophy & Technology 36 (3), p. 1–22. External Links: Document Cited by: footnote 2. P. Olsthoorn (2011) Military ethics and virtues. Routledge, Abingdon, UK. External Links: ISBN 9781136894299, Document Cited by: §3, §6. M. Pacholska (2024) Autonomous weapons. In Research Handbook on Law and Technology, B. Brożek, O. Kanevskaia, and P. Pałka (Eds.), p. 392–407. Cited by: footnote 6. U. Pagallo (2011) Robots of just war: a legal perspective. Philosophy & Technology 24 (3), p. 307–323. External Links: Document Cited by: footnote 2. A. Patt and R. Zeckhauser (2000) Action bias and environmental decisions. Journal of Risk and Uncertainty 21 (1), p. 45–72. External Links: Document Cited by: §5. G. Phillips-Wren, D. J. Power, and M. Mora (2019) Cognitive bias, decision styles, and risk attitudes in decision making and DSS. Journal of Decision Systems 28 (2), p. 63–66. Cited by: §5. T. Raleigh and A. Knoks (2025) Clarifying the opacity of neural networks. Minds and Machines 35 (4), p. 43. External Links: Document Cited by: §2.2, footnote 5. A. P. Rebera and N. G. Wood (2025) Virtues and rules in war: military ethics and technologies of radical risk-reduction. Ethical Theory and Moral Practice 28, p. 821–837. External Links: Document Cited by: §3. A. P. Rebera (2024) Reactive attitudes and AI-agents – making sense of responsibility and control gaps. Philosophy & Technology 37 (4). External Links: Document Cited by: footnote 2. N. C. Renic (2018) UAVs and the end of heroism? historicising the ethical challenge of asymmetric violence. Journal of Military Ethics 17 (4), p. 188–197. External Links: Document Cited by: §3. D. A. Restrepo (2020) In defense of mercy. Journal of Military Ethics 19 (1), p. 40–55. External Links: Document Cited by: §3. D. Restrepo (2019) Naked soldiers, naked terrorists, and the justifiability of drone warfare. Social Theory and Practice 45 (1), p. 103–126. External Links: Document Cited by: §3. S. Robbins (2023) The many meanings of meaningful human control. AI and Ethics 4 (4), p. 1377–1388. External Links: Document Cited by: footnote 5. P. Robinson, N. De Lee, and D. Carrick (Eds.) (2016) Ethics education in the military. Routledge, Abingdon, UK. Cited by: §6. P. Robinson (2016) Introduction: ethics education in the military. In Ethics Education in the Military, P. Robinson, N. De Lee, and D. Carrick (Eds.), p. 1–14. Cited by: §6. H. Roff and R. Moyes (2016) Meaningful human control, artificial intelligence and autonomous weapons. Technical report Article 36, Article 36. Cited by: §3. D. Roselli, J. Matthews, and N. Talagala (2019) Managing bias in AI. In Companion Proceedings of The 2019 World Wide Web Conference, W ’19, p. 539–544. External Links: Document Cited by: footnote 15. D. C. Russell (2009) Practical intelligence and the virtues. Oxford University Press, Oxford, UK. Cited by: footnote 9. S. J. Russell and P. Norvig (2021) Artificial intelligence: a modern approach. Fourth edition edition, Always Learning, Pearson, Boston, MA. External Links: ISBN 978-1-292-40117-1, Link Cited by: footnote 3. F. Santoni de Sio and J. van den Hoven (2018) Meaningful human control over autonomous systems: a philosophical account. Frontiers in Robotics and AI 5, p. 1–14. External Links: Document Cited by: §3. M. Schulzke (2016) Rethinking military virtue ethics in an age of unmanned weapons. Journal of Military Ethics 15 (3), p. 187–204. External Links: Document Cited by: §6. E. Schwarz (2021) Autonomous weapons systems, artificial intelligence, and the problem of meaningful human control. Philosophical Journal of Conflict and Violence. Cited by: §1. L. Seoane, L. M. Tompkins, A. De Conciliis, and P. G. Boysen (2016) Virtues education in medical school: the foundation for professional formation. Ochsner Journal 16 (1), p. 50–55. Cited by: §6. N. Sharkey (2007) Automated killers and the computing profession. Computer 40 (11), p. 124–123. Cited by: footnote 2. R. Sheu and M. S. Pardeshi (2022) A survey on medical explainable AI (XAI): recent progress, explainability approach, human interaction and scoring system. Sensors 22 (20), p. 8068. External Links: Document Cited by: §1. A. Søgaard (2023) On the opacity of deep neural networks. Canadian Journal of Philosophy 53 (3), p. 224–239. External Links: Document Cited by: §2.2, footnote 5. A. Sookermany and T. Slensvik (2026) Military operational thinking in and age of artificial intelligence. War on the Rocks. Note: https://warontherocks.com/2026/03/military-operational-thinking-in-an-age-of-artificial-intelligence/ External Links: Link Cited by: footnote 11. R. Sparrow (2007) Killer robots. Journal of Applied Philosophy 24 (1), p. 62–77. External Links: Document Cited by: footnote 2. R. Sparrow (2013) War without virtue?. In Killing by Remote Control: The Ethics of an Unmanned Military, B. J. Strawser (Ed.), p. 84–105. External Links: ISBN 978-0-19-992612-1 Cited by: §3. T. Speith (2022) A review of taxonomies of explainable artificial intelligence (XAI) methods. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, p. 2239–2250. Cited by: §1. E. Talbert (2026) The triage trap: when AI speed replaces command judgment. War on the Rocks. Note: https://warontherocks.com/2026/01/the-triage-trap-when-ai-speed-replaces-command-judgment/ External Links: Link Cited by: §5. E. Tjoa and C. Guan (2020) A survey on explainable artificial intelligence (XAI): toward medical XAI. IEEE Transactions on Neural Networks and Learning Systems 32 (11), p. 4793–4813. External Links: Document Cited by: §1. T. Toy (2023) Transparency in AI. AI & Society 39 (6), p. 2841–2851. External Links: Document Cited by: §1, footnote 5. E. Tucker (2022) Artifice and intelligence. Tech Policy Press. Note: https://w.techpolicy.press/artifice-and-intelligence/, Accessed: Aug. 6, 2025 External Links: Link Cited by: §1. A. Tversky and D. Kahneman (1981) The framing of decisions and the psychology of choice. Science 211 (4481), p. 453–458. Cited by: §5. US Department of Defense (2023) DoD Directive 3000.09. Technical report United States Department of Defense. Cited by: footnote 6. S. Vallor (2013) The future of military virtue: autonomous systems and the moral deskilling of the military. In 5th International Conference on Cyber Conflict (CYCON 2013), p. 1–15. Cited by: §6. S. Vallor (2014) Moral deskilling and upskilling in a new machine age: reflections on the ambiguous future of character. Philosophy & Technology 28 (1), p. 107–124. External Links: Document Cited by: §6. S. Vallor (2016) Technology and the virtues: a philosophical guide to a future worth wanting. Oxford University Press, Oxford, UK. Cited by: footnote 14. T. M. van Engers and D. M. de Vries (2019) Governmental transparency in the era of artificial intelligence. In Legal Knowledge and Information Systems, M. Araszkiewicz and V. Rodríguez-Doncel (Eds.), p. 33–42. Cited by: §1. M. Vego (2017) On operational art. Strategos 1 (2), p. 15–39. Cited by: footnote 11. M. Vego (2025) Operational art: theory and practice. Routledge, Abingdon, UK. Cited by: footnote 11. K. R. Walsh, S. Mahesh, and C. C. Trumbach (2021) Autonomy in AI systems. The Journal of Technology Studies 47 (1), p. 38–47. Cited by: footnote 6. M. Walzer (2006) Just and unjust wars: a moral argument with historical illustrations. 4 edition, Basic Books, New York, NY. Cited by: §3. P. Wang (2019) On defining artificial intelligence. Journal of Artificial General Intelligence 10 (2), p. 1–37. External Links: Document Cited by: footnote 3. G. Webb (2004) Role improvising during crisis situations. International Journal of Emergency Management 2 (1-2), p. 47–61. External Links: Document Cited by: §6. D. A. Wiegmann, L. J. Wood, T. N. Cohen, and S. A. Shappell (2021) Understanding the “Swiss Cheese Model” and its application to patient safety. Journal of Patient Safety 18 (2), p. 119–123. External Links: Document Cited by: §7. A. P. Williams (2015) Defining autonomy in systems: challenges and solutions. In Autonomous Systems: Issues for Defense Policymakers, A. P. Williams and P. D. Scharre (Eds.), p. 27–62. Cited by: footnote 6. N. G. Wood, S. Robbins, E. Zegarra Berodt, A. Graf von Westerholt, M. Behrndt, H. Budig, and D. Kloock-Schreiber (2026) Stop saying “ai”. Philosophy & Technology 39 (2), p. 1–35. External Links: Document Cited by: §1. N. G. Wood (2023a) Autonomous weapon systems and responsibility gaps: a taxonomy. Ethics and Information Technology 25 (1), p. 1–14. External Links: Document Cited by: footnote 2. N. G. Wood (2023b) Autonomous weapon systems: a clarification. Journal of Military Ethics 22 (1), p. 18–32. External Links: Document Cited by: footnote 6. N. G. Wood (2024) Explainable AI in the military domain. Ethics and Information Technology 26 (2), p. 1–13. External Links: Document Cited by: §3, footnote 1. N. G. Wood (2026) Autonomous and AI-enabled systems: extensions or replacements of human will and control?. Ethics and Information Technology 28 (1), p. 1–16. External Links: ISSN 1572-8439, Document Cited by: footnote 6, footnote 8. J. Wu, H. You, and J. Du (2025) AI generations: from AI 1.0 to AI 4.0. Frontiers in Artificial Intelligence 8. External Links: Document Cited by: §1. G. Yang, J. Cambias, K. Cleary, E. Daimler, J. Drake, P. E. Dupont, N. Hata, P. Kazanzides, S. Martel, R. V. Patel, V. J. Santos, and R. H. Taylor (2017) Medical robotics – regulatory, ethical, and legal considerations for increasing levels of autonomy. Science Robotics 2 (4), p. 1–2. External Links: Document Cited by: footnote 6. M. Zając (2022) Spare not a naked soldier: a response to Daniel Restrepo. Journal of Military Ethics, p. 1–16. External Links: Document Cited by: footnote 10. M. Zhou, V. Abhishek, T. Derdenger, J. Kim, and K. Srinivasan (2024) Bias in generative AI. arXiv. External Links: Document Cited by: footnote 15.

Instant research discovery