Paper deep dive
Explainability Framework for Policy-Aware Autonomous Agents
Heather Merhout, Daniela Inclezan
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 94%
Last extracted: 7/24/2026, 2:52:04 AM
Summary
The paper proposes an explainability framework for policy-aware autonomous agents implemented in Answer Set Programming (ASP) with Python. The framework generates contrastive explanations by leveraging policy violation penalties to identify counterfactual scenarios, aiming to provide comprehensible, socially-aware justifications for agent decisions, evaluated via human surveys.
Entities (9)
Relation Signals (7)
Heather Merhout → affiliatedwith → Miami University
confidence 99% · Heather Merhout Miami University Oxford, Ohio, United States
Daniela Inclezan → affiliatedwith → Miami University
confidence 99% · Daniela Inclezan Miami University Oxford, Ohio, United States
Explainability Framework → implementedin → Answer Set Programming
confidence 95% · It is implemented in the Answer Set Programming language
Paper → publishedat → ICLP 2026
confidence 95% · 42nd International Conference on Logic Programming (ICLP 2026)
Explainability Framework → appliedto → Nurse Scheduling
confidence 90% · The domain for our intelligent agent, a nurse scheduling system, is described in this section
Explainability Framework → uses → AOPL
confidence 90% · One language for describing complex policies is known as AOPL... we adopted it for our work.
Explainability Framework → uses → Python
confidence 90% · using Python to assist with information extraction
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:In the field of Artificial Intelligence, an agent is a system which is able to autonomously make decisions in order to reach a desired goal. As these systems grow more prevalent in our day-to-day lives, there has been an increased need to add explainability features which can provide an account for an agent's behavior. We therefore propose a framework that outlines how to produce comprehensible explanations for policy-aware agents, or agents which have rule-enforcing policies incorporated in their decision-making framework. This framework is designed using insights from the social sciences on how to produce good explanations. It is implemented in the Answer Set Programming language while using Python to assist with information extraction and natural-language translation. Because these agents incur penalties when violating policies, we are able to leverage these penalties to detect undesirable events in scenarios that are counterfactual to the agents' original actions. This lends itself to creating contrastive explanations (e.g., "the agent performed this action because, had it not, undesirable event X would have occurred."), which formulate the core component for our explainability framework. The framework is evaluated using a survey wherein human participants provide feedback on our program-generated explanations.
Tags
Links
- Source: https://arxiv.org/abs/2607.21209v1
- Canonical: https://arxiv.org/abs/2607.21209v1
Trouble viewing inline? Open PDF directly →
Full Text
39,169 characters extracted from source content.
Expand or collapse full text
W. Faber, L. Giordano, R. Rocha, V. Santos Costa (Eds.): 42nd International Conference on Logic Programming (ICLP 2026) EPTCS 450, 2026, p. 515–528, doi:10.4204/EPTCS.450.38 © Heather Merhout, D. Inclezan This work is licensed under the Creative Commons Attribution License. Explainability Framework for Policy-Aware Autonomous Agents Heather Merhout Miami University Oxford, Ohio, United States merhouhs@miamioh.edu Daniela Inclezan Miami University Oxford, Ohio, United States inclezd@miamioh.edu In the field of Artificial Intelligence, anagentis a system which is able to autonomously make deci- sions in order to reach a desired goal. As these systems grow more prevalent in our day-to-day lives, there has been an increased need to add explainability features which can provide an account for an agent’s behavior. We therefore propose a framework that outlines how to produce comprehensible explanations for policy-aware agents, or agents which have rule-enforcing policies incorporated in their decision-making framework. This framework is designed using insights from the social sciences on how to produce good explanations. It is implemented in the Answer Set Programming language while using Python to assist with information extraction and natural-language translation. Because these agents incur penalties when violating policies, we are able to leverage these penalties to detect undesirable events in scenarios that are counterfactual to the agents’ original actions. This lends it- self to creating contrastive explanations (e.g., “the agent performed this action because, had it not, undesirable event X would have occurred.”), which formulate the core component for our explain- ability framework. The framework is evaluated using a survey wherein human participants provide feedback on our program-generated explanations. 1 Introduction As computers grow more and more prevalent in modern life and work, so too does the use of autonomous agents to make decisions based on a world of data. However, autonomous agents can sometimes act in unexpected or even undesirable ways. Hence, our research seeks to promote ethical behavior in intelligent systems by proposing an explanation framework, which articulates the rationale behind an agent’s decision-making processes, for agents that are designed using Answer Set Programming [4]. Explanation frameworks have been or are being developed for numerous different types of automated systems (see, for example, [9] and [7]). For this research, our explanation framework will apply primarily to planning and scheduling domain models which utilize policies within their decision-making frame- work. Because the goal is to explore methodologies and philosophies for constructing an explainability framework, the processes described in this paper may have applications in the broader field of AI as well. The motivation for this research stems from the increased desire for transparency across AI systems. As autonomous agents are being more and more integrated into systems such as self-driving cars and medical diagnostic systems, it is crucial that humans can trust these systems to make reasonable decisions based on the data they receive. An explanation system, wherein an agent can output a concise and comprehensible summary of the logical reasons for executing an action, will give a user insight into the functionality of an intelligent agent which it may employ for important tasks; not only does this give the user more confidence and trust that their agent is working within reasonable expectations, but it also—in the case that an agent does not work as its user desires—could serve to highlight where an agent’s shortcomings exist and what changes must be made to the agent if possible. 516Explainability Framework for Policy-Aware Autonomous Agents 2 Background: Policies andAOPL This research focuses on agents with policy awareness. A policy-aware agent is one for which policies, or sets of rules/norms, are incorporated into its decision-making framework. Of all the decisions an agent can possibly make, a policy is a rule defined by a subset of these possible trajectories which the agent’s programmer deems preferable [5]. While simple rules, such as “if it is raining, do not perform action A,” can be implemented through Answer Set Prolog (ASP), it becomes increasingly important to describe a complicated set of rules to ensure an agent does not produce unwanted behavior. One language for describing complex policies is known asAOPL(Authorization and Obligation Policy Language) [5], defined by Gelfond and Lobo. AOPLdiffers from both classical deontic logics and more recent deontic extensions of ASP in its design goals and level of abstraction. Classical deontic logic, originating with von Wright [23] and developed in modal frameworks such as those surveyed by Hilpinen [10] and Meyer and Wieringa [17], provides an axiomatic treatment of obligation, permission, and prohibition using modal operators, but remains largely static and not directly executable. In contrast,AOPLadopts the rule-based, nonmonotonic se- mantics of Answer Set Programming to encode norms as predicates over actions and states, enabling reasoning about authorization and obligation in dynamic domains with time, causality, and exceptions. Cabalar’s work on deontic ASP [2, 3] takes a different route by extending the logical foundations of ASP with explicit deontic operators, yielding formalisms such as Deontic Equilibrium Logic where obliga- tions and permissions are first-class constructs with precise semantics and can be compiled into ASP. Thus, whileAOPLis primarily an application-oriented policy language that represents norms within an action theory, Cabalar’s approach is semantics-oriented, aiming to reconstruct deontic reasoning itself inside ASP, effectively bridging the gap between traditional deontic logic and nonmonotonic logic pro- gramming. Given thatAOPLallows describing policies at a higher-level of abstraction than ASP, we adopted it for our work. 2.1 Authorizations and Obligations AOPLdistinguishes between policy authorizations and obligations. Authorizations (the “A” inAOPL) are policies that describe whether an agent is permitted (authorized) or not permitted to perform an action under certain circumstances. An “action” in this context is classified as anelementary action(e), which consists of a single true or false atom, or acompound action(c), which consists of multiple simultaneous elementary actions. An authorization is specified by using the predicatepermitted(e)to describe an action. A strict authorization is one with no exceptions, while adefeasiblerule (d) is one for which exceptions are allowed. For a set of default authorization rulesd 1 ...d n , the predicatepre f er(d 1 ,d 2 )is used to denote a preference of one authorization over another. Gelfond and Lobo describe authorization as expressions taking on one of these forms [5]: 1.permitted(e)ifcond 2.¬permitted(e)ifcond 3.d: normallypermitted(e)ifcond 4.d: normally¬permitted(e)ifcond 5.pre f er(d 1 ,d 2 ) It is important to note that a strict rule of type 2 above is not the same as an ASP constraint. Rather than forbidding any world in which the policy is violated, a program will instead incur a penalty when a policy is violated, as will be seen later. Heather Merhout, D. Inclezan517 Obligations (the “O” inAOPL) are policies which describe actions that an agentmustdo or refrain from doing under certain circumstances. Obligations are indicated by the predicateobl(e)if an actione is to be taken under certain conditions, orobl(¬e)if the agent is to refrain from actioneunder certain conditions. Obligation expressions take on one of the following forms, for anyha p pening∈ e,¬e[5]: 1.obl(ha p pening)ifcond 2.¬obl(ha p pening)ifcond 3.d: normallyobl(ha p pening)ifcond 4.d: normally¬obl(ha p pening)ifcond 5.pre f er(d 1 ,d 2 ) 2.2 Rule Syntax When encoding a policy usingAOPL, the following syntax is often used (see [22]): r(t 1 ,...t n ):headifcond whereris the label for the rule,headis the head of an authorization or obligation,condis the condition (environmental state) which triggers the rule, andt 1 ,...t n is the set ofnterms which appear in the rule. Examples of this syntax, which describe policies for a self-driving car, are as follows: •r1(I): normallypermitted(drive(I))iflight_color(green,I) •r2(I):obl(sto p(I))ifpedestrians_are_crossing(I) •r3(I):¬permitted(drive(I))ifhas_sign(I,sto p),¬has_s peed(0). The rules can respectively be read as follows: • Rule 1: the agent is normally permitted to drive through intersection I if the intersection’s light color is green. • Rule 2: the agent is obligated to stop at intersection I if pedestrians are crossing the intersection. • Rule 3: the agent is not permitted to drive through intersection I if the intersection has a stop sign and the vehicle does not have a speed of 0 (i.e., it is not stopped). Thus, when developing a planning framework for policy-aware autonomous agents, researchers can write rules that ensure a policy-compliant agent chooses a plan which not only achieves its de- sired goal but does so without violating policies implemented within its decision-making framework [16, 12, 8, 13, 20, 21]. However, there still remains a gap in knowledge regarding the development of an explainability framework, which intends to increase trustworthiness in these agents by providing a standardizable methodology to explain the plans that they take. The next section will outline a philo- sophical framework, adapted from [18], on what kind of information to include in an explanation for an autonomous agent. 3 Explainability: A Philosophical Framework In 2019, Miller conducted a survey which collected information from various sources about what a human looks for in an explanation and how these findings are applicable to explaining AI programs [18]. It is this survey from which we will present the most relevant insights which can be used to build our framework for explaining our policy-aware autonomous agents. 518Explainability Framework for Policy-Aware Autonomous Agents 3.1 A Social Process One simple definition of an explanation that Miller cites, from Josephson and Josephson [14], states, “An explanation is an assignment of causal responsibility.” It is easy, especially when presented with such a definition, to simply think of an explanation as a final product: a statement or set of facts describing the causality of an event. However, it is important to note that, as stated by Lombrozo [15], an explanation is both a product and aprocess. Miller argues that there are indeed two processes in an explanation, both involving human mental reasoning: 1.Social Process: The explainer exchanges information with the explainee such that the explainee has enough information to understand the causes of an event. 2.Cognitive Process: The explainee uses contrastive reasoning to solve what is the best cause of an event based on the information provided by the explainer. Because an explanation is a social process, it is ultimately upon the explainee to decide whether an explanation is sound or sufficient for understanding. Still, there are some principles which can aid in the process of human understanding when building an explanation. Firstly, an explanation, being a social process, is ultimately a conversation (as proposed by Hilton in 1990 [11]). The difference between causal attribution and explanation, according to Hilton, is that an explanation requires an exchange between parties. The conversational model of explanation emphasizes this fact in a way that the simple causal attribution model does not. If an explanation is a conversation, it thus follows that it should adhere to Grice’s four maxims of conversation, or theGricean maxims[6]: 1.Quality:Ensure that the explanans (answers to the question) are true, to the best of the explainer’s knowledge. 2.Quantity:Provide the right amount of information (do not provide more or less than required). 3.Relation:Provide information that is relevant to the conversation and exclude irrelevant informa- tion. 4.Manner:Conduct the conversation in an appropriate way, being easy to understand: in other words, do not be obscure or ambiguous, be orderly, and avoid an excess of words. One way to ensure that the maxim ofQuantityandRelationis met is to focus on the next insight: explanations arecontrastive. They are answered not as all the de facto causes of an event (this would include too much irrelevant information) but rather in light of a counterfactual case. 3.2 Contrastive Explanations As Hilton states [11], The key insight is to recognize that one does not explain events per se, but that one explains why the puzzling event occurred in the target case but not in some counterfactual contrast case. Explanations must therefore be relevant to some question and explain the difference between the present case and some contrast case implied by the question. Everywhyques- tion thus has an implicitrather thanbuilt into it. Heather Merhout, D. Inclezan519 With contrastive reasoning, each why-question is associated with an unspokencounterfactual—a statement declaring the opposite of the event presupposed in the question. For example, when asking why a vehicle drove down road A, the counterfactual would be that the vehicle did not drive down road A. The implied question, then, is, “Why did the vehicle drive down road Ainstead of taking a different route(i.e., not driving down road A)?”. Now, let us presume that the counterfactual, rather than the event, is true, and the vehicle did not drive down road A. It may be that the vehicle’s route took a longer amount of time that it would have if it had driven via road A; we now can see that the vehicle originally drove down road A in order to save time. This, then, is the crux of contrastive reasoning: by exploring what would have happened had the rather thanscenario in any why-question been true, it is possible to uncover undesirable events which can explain why such a scenario was initially avoided. 4 Implementation: Hospital Scheduling Domain We have outlined a framework for building explanations, highlighting the social, conversational, and contrastive nature which explanations should employ. In order to implement this framework for policy- aware autonomous agents, we have built a program which outputs explanations for an autonomous agent in a natural language based on information gained from counterfactual cases. 1 The domain for our intelligent agent, a nurse scheduling system, is described in this section and is inspired by work on nurse scheduling by Nabeshima et al. [19]. The primary goal of this program is to schedule the required number of nurses for each shift in a given planning period while also adhering to nurse preferences for shift types or desired weekly days off. 4.1 Rules and Actions The primary action which the agent performs is that of assigning or not assigning a nurse a certain shift on each day in the planning period. This is accomplished through the disjunctive rule, assigned(N, D, S) | -assigned(N, D, S) :- nurse(N), day(D), shift(S), not off_day(S). Through this rule, there is either an assignment or an explicit non-assignment for each nurse, day in the planning period, and shift. Further rules and policies narrow down which shifts are allowed to and/or should be assigned to each nurse on which day. Another action the agent adheres to is that of staffing. The staffing action counts the number of nurses that are assigned to each shift, encoded by the rule, staffing(C, D, S) :- #count N : assigned(N, D, S) = C, day(D), shift(S). This action allows one to keep track of whether the lower bound of required nurses has been met for each shift. It does not by itself enforce minimum staffing needs; this will be done by a policy listed below. However, there is a separate hard-coded constraint which prevents the staffing on any given shift from being zero: :- staffing(0, D, S). 1 The code for this program can be found at https://github.com/CallunaZ/Explainability_Framework_Code. 520Explainability Framework for Policy-Aware Autonomous Agents Additional rules assist in ensuring the program operates as intended, such as a rule stating that nurses cannot be scheduled for less than 20 work days per planning period or that a nurse cannot be assigned two different shifts on the same day. Two final actions the agent can take are:nurse_fwdoandnurse_swdo, which assigns a nurse his/her first week day off and his/her second week day off, respectively. Each week day off is chosen as a number between 0 and 6, as listed above, such that the statementnurse_fwdo(beth, 0)indicates that Beth’s first week day off is a Monday. Similar to the action which assigns or does not assign a working shift to each nurse on each day, the nurses’ first and second week days off are chosen through a disjunctive rule. The rules that determine a nurse’s first and second week days off are as follows: nurse_fwdo(N, W) | -nurse_fwdo(N, W) :- nurse(N), weekday(W). nurse_swdo(N, W) | -nurse_swdo(N, W) :- nurse(N), weekday(W). When a nurse is scheduled a weekly day off, the actionassignedis used to assign theweekly_- day_offshift to the nurse on each day which corresponds to the aforementioned weekday. This therefore causes the program to display both work shifts and rest shifts for each nurse throughout the planning period. 4.2 Policies A total of twenty policies govern the hospital scheduling domain. The first two of these policies outline general rules that the scheduling system should follow regarding staffing weekly-day-off procedures. The remaining 18 policies (which will not be listed due to space limitations) define specific nurse preferences the program is intended to implement. The first two policies are stated in plain English below: 1. The agent is not permitted to understaff any shift. 2. The agent is obligated to schedule a nurse’s weekly days off consecutively to provide adequate rest time. (This rule is defeasible). TheAOPLencoding of these two policies is listed as follows (sr1 indicates “staffing rule 1” and dor1 indicates “day off rule 1”): 1.sr1(C,D,S,LB):¬permitted(sta f f ing(C,D,S))ifday(D),shi f t(S),shi f t_lb(S,LB),lt(C,LB). 2.dor1(N,W1,W2): normallyobl(nurse_swdo(N,W2))ifnurse_f wdo(N,W1), oneDayA part(W1,W2). The penalty for a staffing violation is 5 plus the difference between the lower bound of required nurses and the actual number of scheduled nurses; e.g., if the program scheduled only 1 nurse for the night shift, which requires at least 3 nurses, the penalty would be 7: 5 + (3 - 1). This ensures that the penalty is higher the more a shift is understaffed. The penalty for a violation of consecutive days off is 3 points. This point value was chosen to be lower than the penalty for a staffing violation because giving a nurse nonconsecutive days off would not threaten hospital operations as much as staffing a shift lower than needed. Still, in order to respect the nurse’s needs as much as possible, the penalty sits at three points rather than one or two. The penalty for the remaining policies, all describing nurse preferences, is also three points for the same reason. These policies are encoded into ASP using multiple rules according to theAOPLto ASP translation described in [12] and [21]: one rule identifies the head of the rule (the action before theifstatement), Heather Merhout, D. Inclezan521 other rules define each member of the body (every relevant condition listed after theifstatements, and still other rules define the label of the rule, the rule type (strict or defeasible), and the penalty amount for violating the rule. Dividing the policy into several rules lets the ASP code process each component more straightforwardly, making it possible to identify when the policy is being violated. The code which identifies a policy violation for the first policy is written as follows, using theadd_- penaltypredicate: add_penalty(R, P) :- holds(R), head(R, -permitted(staffing(C, D, S))), staffing(C, D, S), penalty(R, P). The logic for this code states that the agent should add a penaltyPif ruleRholds, the head of rule Rstated that a staffing action was not permitted, the staffing action occurred anyway, and the penalty for ruleRisP. The penalties for other policies are implemented through similar means. In order to enforce those policies that are applicable (i.e., are not overridden by other policies), a weak constraint is added to the program to avoid theadd_penaltyatom. 5 Explanation Program This section describes the design for a program that collects counterfactual information about an agent’s decision and displays an explanation for a certain question in a natural language. The program is encoded primarily in Python, using Python’sclingolibrary 2 to run ASP code and extract information accordingly. This program is divided intothreeparts, the second two of which comprise the primary explanation- generation logic. 5.1 Part 1: Generate Original Output In order to produce explanations for a given intelligent agent’s behavior, the intelligent agent must pro- duce such behavior in the first place. While the intelligent agent’s decision-making process occurs through ASP alone, the PythonclingoAPI is utilized in order for the ASP output to be compatible with certain procedural elements that will be described below. Figure 1 displays an HTML schedule for nurse Zelda as outputted by our program. There are two policies regarding Zelda’s preferences—namely, that Zelda prefers the day shift, and that Zelda prefers her first week day off to be a Saturday. Because the optimal model does not need to violate any policies, the output correctly shows a schedule where Zelda is always assigned the day shift and where her weekly days off occur on Saturday and Sunday. Figure 2 displays the schedule for nurse Howard as outputted by our program. As there are no policies dictating Howard’s preferences, Howard’s schedule is much more diverse than that of Zelda. His schedule is shown to contain early morning, evening, and night shifts, along with weekly days off on Sunday and Monday. 5.2 Part 2: Generate Questions and Counterfactuals The next step in our explanation program is to engage in the process of contrastive reasoning by creating counterfactual worlds. To do this, the user first needs to ask a question about one of the actions in the 2 https://potassco.org/clingo/python-api/5.4/ 522Explainability Framework for Policy-Aware Autonomous Agents Figure 1: Nurse Zelda’s schedule. Figure 2: Nurse Howard’s schedule. Heather Merhout, D. Inclezan523 Figure 3: Hospital scheduling agent question template in action. agent’s initial output. This is accomplished by receiving input from a template of possible questions, generated through Python’s TKinter library 3 . Figure 3 displays the template with two example questions being inputted. These questions are, 1. Why was Zelda scheduled for the Weekly Day Off shift on February 3, 2024? 2. Why was Howard not scheduled for the Day shift on February 8, 2024? When the question input is submitted, a series of formatted string literals takes the input and trans- lates it to a constraint in ASP. This constraint is stored in a file known ascounterfactual.dlv. The constraints for the above two examples respectively appear as, 1.:- assigned(zelda, 34, weekly_day_off). 2.:- not assigned(howard, 39, day). These constraints ensure that, when the program is run again along with the counterfactual file, 1) Zelda cannot be assigned the weekly day off shift on day 34 of the year (February 3, 2024), or 2) Howard mustbe assigned the day shift on day 39 of the year (Febrary 8, 2024)—or, more specifically, the agent is not allowed to produce a model in which Howard is not assigned the day shift on day 39. Now, it is possible to see what undesirable effects, if any, shall occur when this counterfactual input is run alongside the original ASP program. 5.3 Part 3: Generate Explanations The program then uses the information from the new, counterfactual world to derive explanations for the questions asked in part 2 of the program. First, let us observe what happens for the first example when Zelda is not allowed to be scheduled the weekly day off shift on February 3, 2024. When observing the raw ASP output, one can see the atom, 3 TheTkinterlibraryisapopularGUIlibraryforPython.Documentationcanbefoundat https://docs.python.org/3/library/tk.html 524Explainability Framework for Policy-Aware Autonomous Agents add_penalty(pdo4,3) This atom tells us that a penalty of 3 was added for violating the “pdo4,” or “preferred-day-off 4” rule. Looking at the ASP-encoded policies reveals that “pdo4” refers to the policy regarding Zelda’s preferred weekly day off. In order to read the policy and assist with the natural-language creation of explanations, an additional rule has been added to the ASP encoding of theAOPLrule: text(pdo4, "The agent must schedule Zelda’s first day off on a Saturday.") :- add_penalty(pdo4, 3). This rule causes thetextatom to appear when the penalty appears, thus enabling us to use Python’s clingofunctions to iterate through and extract this text information. Using this text and additional for- matted string literals, the final explanation output which would be shown to the user reads, If Zelda were not assigned the Weekly Day Off shift on February 3, 2024, all possible schedules would violate policies which the agent is trying to adhere to. Policies include: 1. The agent must schedule Zelda’s first day off on a Saturday. (This nurse would be scheduled a different day off, violating this policy.) This explanation tells a user why the agent scheduled Zelda the weekly day off shift on February 3, 2024: if it had not performed this action, it would have violated a policy regarding Zelda’s preferred weekly day off! Now, let us observe what happens in the second counterfactual world, when the agent is forced to assign Howard the day shift on February 8, 2024. In Zelda’s case, the solver was easily able to refer to Zelda’s specific preference to build a counterfactual model; however, for Howard’s case, the solver must search through every possible nurse preference to determine whether Howard is able to “fit” in the schedule without violating any policies. Thus, theclingosolver becomes trapped in an exhaustive search space, unable to find an optimal model in a definite amount of time. Nonetheless, it is possible to leverage the handle.wait() function in the Pythonclingomodule (where “handle” is the clingo SolveHandleobject) to set a timeout for the solver search. If handle.wait(timeout_seconds) returns True within “timeout_seconds” amount of time (which we set as 5 seconds), it is safe to resume the handle and continue searching for an optimal model. If it returns False, then no model was produced within the time limit and it is safe to assume the program is stalling. TheSolveHandleis canceled and the latest model which was produced can be used as the “optimal” model. This solution works because it realistically represents how a manager would generate a schedule in the event of an indefinite stall. Thus, we are able to produce a counterfactual model in which Howardmustbe scheduled on the Day shift on February 8, 2024. In this world, a penalty is incurred which, in ASP, states, add_penalty(ps4(39,night),3) Because this penalty contains arguments within the “ps4(39, night)” term, we are able to capture much more information which can be stored in a Python dictionary. The label “ps4” indicates that this penalty applies to the policy for Yanni’s preferred shift; thetextrule associated with this penalty states, text(ps4(D, S), "The agent must assign Yanni her preferred shift (day).") :- add_penalty(ps4(D, S), 3). The first argument inside the “ps4” atom, “39,” refers to the day on which the policy is violated: February 8, 2024. The second argument indicates what shift Yanni was assigned instead of her preferred Heather Merhout, D. Inclezan525 shift: Yanni was assigned thenightshift. The last argument, inside the mainadd_penaltyatom, indicates that the penalty is worth 3 points. All of the information inside thisadd_penaltyatom can be stored in a Python dictionary and retrieved when generating clarifying information about exactly what happened to violate Yanni’s preferred shift policy. Finally, because the optimal model was procured from a timeout rather than from the solver exhaust- ing every possible solution, an additional note is added at the end of the explanation clarifying to the user that other models, likely with an equal penalty incurred, are possible (in other words, choosing to violate Yanni’s shift preference was arbitrary). Thus, the output from the explanation program officially states, If Howard were assigned the Day shift on February 8, 2024 all possible schedules would violate policies which the agent is trying to adhere to. Policies include: 1. The agent must assign Yanni her preferred shift (day). (This nurse would be scheduled the night shift on February 08, 2024, violating this policy.) Note: There may be other possible schedules which violate different policies. Hence, this research demonstrates a framework for constructing natural-language explanations for the actions of policy-aware autonomous agents: first, a user asks a question from a template about one of the agent’s actions; secondly, a counterfactual statement is generated in ASP based on the question; thirdly, the counterfactual world is generated and policy violations are identified using theadd_penalty atom; and fourthly, those penalties are shown to the user as reasons why the agent originally performed the action in question. This framework was also tested on a planning domain [21]. 6 Evaluation Survey In order to understand whether the explanations outputted by the program are comprehensible to real humans, a survey was conducted with 12 human participants showing them the examples which have been outlined above. The results of this feedback are shown below. Example 1 pertains to the explanation about Zelda while Example 2 pertains to the explanation about Howard. 6.1 Comprehensibility For each example scenario in the survey, the first question asks, “Does this explanation make sense?” There are three possible answers to this question: “Yes,” “Yes, but I had to reread it in order to understand it,” and “No.” Table 1 shows the percentage of people who provided each answer to this question for each example. ExampleYesYes, but I had to reread itNo Example 175%16.7%8.3% Example 275%25%0% Table 1: Results for the question: “Does this explanation make sense?” For each example shown above, the majority of people responded that yes, the explanation made sense. Very few people indicated that our explanations did not make sense at all. Thus, these results indicate that our explanations make sense from the outset a majority of the time, though there is some suggestion that a few people may require more time to reread the explanations. 526Explainability Framework for Policy-Aware Autonomous Agents 6.2 Detailed Comprehensibility The second question for each explanation example asks, “How well do you understand this explanation?” Participants answer this question on a scale of 1 to 5, with 1 being “Not at all” and 5 being “Very well.” Table 2 displays, for each example explanation, the percentage of people who chose each number. Example1 (Not at all)2345 (Very Well) Example 10%8.3%0%33.3%58.3% Example 20%0%16.7%41.7%41.7% Table 2: Results for the question: “How well do you understand this explanation?” From these results, we can see that most participants responded with a 4 or a 5 for all of the example questions, indicating that the explanations could be understood quite well. 6.3 Information Quantity The next question for each example asks, “Did the explanation provide enough information to understand the agent’s behavior?” This question pertains to Grice’s conversational maxim ofquantity. The exact answer choices are as follows: • No, the agent needs to provide more information. • Yes, the explanation includes just enough information. • The explanation provided more information than necessary. Table 3 displays the percentage of people that answered each question accordingly. ExampleNot enoughRight amountToo much Example 125%66.7%8.3% Example 241.7%58.3%0% Table 3: Results for the question: “Did the explanation provide enough information to understand the agent’s behavior?” These results indicated that the majority of people considered the agent to have provided the right amount of information, although Example 2 (regarding Howard) demonstrated a higher count of people who wanted more information. The note at the end of the explanation stating that “There may be other possible schedules which violate different policies” may not have been as clarifying as originally in- tended, as one participant wrote, “Rather than having the Note at the bottom of the explanation it should include all reasonings for the decision.” 7 Conclusion To conclude, we have constructed a program which implements an explainability framework based on contrastive logic, which the social sciences have shown to be an effective tool for creating explana- tions. Future work could expand upon this research by allowing for multiple lines of questioning per explanation, whether by incorporating multiple actions into one query or by allowing a back-and-forth conversation between the explanation program and the explainee. Heather Merhout, D. Inclezan527 References [1] Arvid Becker, Pedro Cabalar, Martín Diéguez, Torsten Schaub & Anna Schuhmann (2024):Metric Temporal Equilibrium Logic over Timed Traces.Theory and Practice of Logic Programming24(3), p. 425–452, doi:10.1017/S1471068424000139. [2] Pedro Cabalar, Jorge Fandinno & Fariñas Del Cerro Luis (2019):Founded World Views with Autoepistemic Equilibrium Logic. In Marcello Balduccini, Yuliya Lierler & Stefan Woltran, editors:Logic Programming and Nonmonotonic Reasoning,Lecture Notes in Computer Science11481, Springer International Publishing, Cham, p. 134–147, doi:10.1007/978-3-030-20528-7_11. [3] Pedro Cabalar & Ovidio Manteiga (2025):deolingo: Extending Answer Set Programming with Deontic Reasoning. In Kees van Berkel, Agata Ciabattoni & John F. Horty, editors:Deontic Logic and Normative Systems - 17th International Conference, DEON 2025, Vienna, Austria, July 1-3, 2025, College Publications, p. 61–77. Available athttps://w.dc.fi.udc.es/~cabalar/deolingo.pdf. [4] Michael Gelfond & Vladimir Lifschitz (1991):Classical Negation in Logic Programs and Disjunctive Databases.New Generation Computing9, p. 365–385, doi:10.1007/bf03037169. [5] Michael Gelfond & Jorge Lobo (2008):Authorization and Obligation Policies in Dynamic Systems. In Maria Garcia de la Banda & Enrico Pontelli, editors:Logic Programming,Lecture Notes in Computer Science 5366, Springer Berlin Heidelberg, Udine, Italy, p. 22–36, doi:10.1007/978-3-540-89982-2_7. [6] H. P. Grice (1975):Logic and Conversation.In:Speech Acts,Brill,p. 41–58, doi:10.1163/9789004368811_003. [7] Botros N. Hanna, Ly Ly T. Trieu, Tran Cao Son & Nam T. Dinh (2020):An Application of ASP in Nuclear Engineering: Explaining the Three Mile Island Nuclear Accident Scenario.Theory and Practice of Logic Programming20(6), p. 926–941, doi:10.1017/S1471068420000241. [8] Charles Harders & Daniela Inclezan (2023):Plan Selection Framework for Policy-Aware Autonomous Agents. In:18th Edition of the European Conference on Logics in Artificial Intelligence,Lecture Notes in Computer Science14281, Springer Cham, Dresden, Germany, p. 683–646, doi:10.1007/978-3-031-43619- 2_43. [9] Bradley Hayes & Julie A. Shah (2017):Improving Robot Controller Transparency Through Autonomous Policy Explanation. In:Proceedings of the 2017 ACM/IEEE International Conference on Human-Robot Interaction, ACM, Vienna Austria, p. 303–312, doi:10.1145/2909824.3020233. [10] Risto Hilpinen, editor (1971):Deontic Logic: Introductory and Systematic Readings. Springer Netherlands, Dordrecht, doi:10.1007/978-94-010-3146-2. [11] Denis J. Hilton (1990):Conversational processes and causal explanation.Psychological Bulletin107(1), p. 65–81, doi:10.1037/0033-2909.107.1.65. [12] Daniela Inclezan (2023):An ASP Framework for the Refinement of Authorization and Obligation Policies. Theory and Practice of Logic Programming23(4), p. 832–847, doi:10.1017/S147106842300011X. [13] Daniela Inclezan, Charles Harders & Vineel S. K. Tummala (2024):A Framework for Defining Behavior Modes in Policy-Aware Autonomous Agents. In:Workshop Proceedings of the 40th International Conference on Logic Programming (ICLP-WS 2024),CEUR Workshop Proceedings3799, CEUR-WS.org. Available at https://ceur-ws.org/Vol-3799/paper2ASPOCP.pdf. [14] John R. Josephson & Susan G. Josephson (1996):Abductive Inference: Computation, Philosophy, Technol- ogy. Cambridge University Press, doi:10.1017/CBO9780511530128. Google-Books-ID: u6zXrogwWAC. [15] Tania Lombrozo (2006):The structure and function of explanations.Trends in Cognitive Sciences10(10), p. 464–470, doi:10.1016/j.tics.2006.08.004. [16] John Meyer & Daniela Inclezan (2021):APIA: An Architecture for Policy-Aware Intentional Agents. In An- drea Formisano, Yanhong Annie Liu, Bart Bogaerts, Alex Brik, Verónica Dahl, Carmine Dodaro, Paul Fodor, Gian Luca Pozzato, Joost Vennekens & Neng-Fa Zhou, editors:Proceedings 37th International Conference 528Explainability Framework for Policy-Aware Autonomous Agents on Logic Programming (Technical Communications), ICLP Technical Communications 2021, Porto (virtual event), 20-27th September 2021,EPTCS345, p. 84–98, doi:10.4204/EPTCS.345.23. [17] John-Jules Ch Meyer & Roelf Johannes Wieringa, editors (1993):Deontic Logic in Computer Science: Nor- mative Systems Specification. Wiley, Chichester. [18] Tim Miller (2019):Explanation in artificial intelligence: Insights from the social sciences.Artificial Intelli- gence267, p. 1–38, doi:10.1016/j.artint.2018.07.007. [19] Hidetomo Nabeshima, Mutsunori Banbara, Torsten Schaub & Takehide Soh (2025):The ASP-based Nurse Scheduling System at the University of Yamanashi Hospital, doi:10.48550/ARXIV.2506.13600. [20] Vineel Tummala & Daniela Inclezan (2024):Policies, Penalties, and Autonomous Agents. In:Logic Program- ming and Nonmonotonic Reasoning - 17th International Conference, LPNMR 2024, Proceedings,Lecture Notes in Computer Science15245, Springer, p. 359–372, doi:10.1007/978-3-031-74209-5_27. [21] Vineel Tummala & Daniela Inclezan (2026):Autonomous Agents and Policy Compliance: A Frame- work for Reasoning About Penalties.Theory and Practice of Logic Programming, p. 1–30, doi:10.1017/S1471068425100380. [22] Vineel Sai Krishna Tummala (2024):Penalization Framework for Policy-Aware Autonomous Agents. Master’s thesis, Miami University. Available athttp://rave.ohiolink.edu/etdc/view?acc_num= miami1720431460024532. [23] G. H. von Wright (1951):I. DEONTIC LOGIC.MindLX(237), p. 1–15, doi:10.1093/mind/LX.237.1.