Paper deep dive
Auditable Release Control for Pedagogical Leakage in LLM Tutors
Nizam Kadir
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%
Last extracted: 8/4/2026, 4:32:53 AM
Summary
The paper introduces an auditable release control architecture for Large Language Model (LLM) tutors to prevent 'pedagogical leakage,' defined as the unauthorized disclosure of answers or decisive reasoning before it is authorized. The system employs a modular design with a selector for pedagogical actions, an authorization gate, a renderer, and a release function that applies deterministic checks and optional semantic verification. Experiments on Gemini 3.5 proposals demonstrate that strict mediation significantly reduces leakage flags while maintaining a safety-utility frontier, establishing that auditable boundaries are more effective than universal semantic safety measures for this specific threat model.
Entities (7)
Relation Signals (5)
Auditable Release Control → mitigates → Pedagogical Leakage
confidence 95% · strict mediation reduces blinded three-model panel-majority leakage flags from 181 to 0
Auditable Release Control → evaluateson → Gemini 3.5
confidence 92% · On 599 fixed Gemini 3.5 proposals
Auditable Release Control → uses → Disclosure Contracts
confidence 90% · A selector emits one of five disclosure contracts
Semantic Verifier → iscomponentof → Auditable Release Control
confidence 88% · adding the semantic verifier yields 14 and no reliable marginal gain
Fitted Q → isoutperformedby → Global A1 Scaffold
confidence 85% · A global A1 scaffold yields 0 majority and 54 any-judge flags, outperforming fitted Q
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Large language model tutors can be correct and helpful yet disclose an answer or decisive reasoning before that disclosure is authorized. We formalize this state- and action-dependent failure as pedagogical leakage and introduce an authorization-aware complete-mediation boundary. A selector emits one of five disclosure contracts, trusted policy gates privileged modes, and a renderer proposes language. A single release function applies inspectable checks, optional cumulative verification, and action-specific fallback; replayable traces separate selection, generation, verification, and enforcement failures. Matched component attribution exposes a safety-utility frontier. On 599 fixed Gemini 3.5 proposals, strict mediation reduces blinded three-model panel-majority leakage flags from 181 to 0 (paired problem-cluster difference -30.22 points, 95% CI [-35.00,-25.72]), while replacing 581 responses and lowering helpfulness. Checker-triggered fallback alone yields 11 majority flags; adding the semantic verifier yields 14 and no reliable marginal gain. A global A1 scaffold yields 0 majority and 54 any-judge flags, outperforming fitted Q on automatic safety and utility. In an externally timestamped replication over 40 unseen problem clusters and 480 attack sequences, high-assurance release reduces majority flags from 42 to 8 (-7.08 points, 95% CI [-13.13,-2.29]); seven failures persist, one is introduced, and mean helpfulness falls by .192. These results establish an auditable release boundary and failure attribution under declared contracts, not universal semantic safety or learning gains.
Tags
Links
- Source: https://arxiv.org/abs/2608.00515v1
- Canonical: https://arxiv.org/abs/2608.00515v1
Trouble viewing inline? Open PDF directly →
Full Text
47,868 characters extracted from source content.
Expand or collapse full text
Auditable Release Control for Pedagogical Leakage in LLM Tutors Nizam Kadir Science, Mathematics and Technology Singapore University of Technology and Design (SUTD) Singapore nizamkadir@mymail.sutd.edu.sg Preprint. Not peer reviewed. Abstract Large language model tutors can be correct and helpful yet disclose an answer or decisive reasoning before that disclosure is authorized. We formalize this state- and action-dependent failure as pedagogical leakage and introduce an authorization- aware complete-mediation boundary. A selector emits one of five disclosure contracts, trusted policy gates privileged modes, and a renderer proposes language. A single release function ap- plies inspectable checks, optional cumulative verification, and action-specific fallback; replayable traces separate selection, generation, verification, and enforcement failures. Matched component attribution exposes a safety–utility frontier. On 599 fixed Gemini 3.5 proposals, strict mediation reduces blinded three-model panel-majority leakage flags from 181 to 0 (paired problem-cluster difference−30.22points, 95% CI[−35.00,−25.72]), while replacing 581 responses and lowering helpfulness. Checker-triggered fallback alone yields 11 majority flags; adding the semantic verifier yields 14 and no reliable marginal gain. A globalA 1 scaffold yields 0 majority and 54 any-judge flags, outperforming fitted Q on automatic safety and utility. In an externally timestamped replication over 40 unseen problem clusters and 480 attack sequences, high-assurance release reduces majority flags from 42 to 8 (−7.08points, 95% CI[−13.13,−2.29]); seven fail- ures persist, one is introduced, and mean helpfulness falls by .192. These results establish an auditable release boundary and failure attribution under declared contracts, not universal semantic safety or learning gains. 1 Introduction An LLM tutor’s most consequential output decision is not only what is correct, but what may be released now. Intelligent tutoring systems traditionally separate student modeling from the selection of problems, hints, and feedback Corbett and Anderson [1995], Feng et al. [2009]. LLM tutors can collapse that policy and its linguistic realization into one fluent genera- tor Kasneci et al. [2023], Tack and Piech [2022], Macina et al. [2023], LearnLM Team et al. [2024], hiding when a supportive response has silently completed the learner’s reasoning. We define pedagogical leakage as unauthorized answer dis- closure, decisive calculation disclosure, or over-constraining the search so strongly that the learner no longer constructs the key step. The same text can violate an elicitation contract and be appropriate in an authorized worked solution. This is unlike generic harmful-content moderation: the protected information is problem-specific, its release depends on learner state and pedagogical action, and authorization can legitimately change within a dialogue. ICAP links constructive engagement with learning Chi and Wylie [2014]; scaffolding and productive- failure research likewise treats bounded difficulty as useful Wood et al. [1976], Kapur [2008]. Preference alignment optimizes judgments over completed responses Christiano et al. [2017], Stiennon et al. [2020], Ouyang et al. [2022], Bai et al. [2022], but immediate help- fulness does not specify when a tutor should elicit, scaffold, explain, or release an answer. Recent pedagogical alignment, steering, and tutoring-policy methods address this conflict through preference optimization, reinforcement learning, or explicit planning Sonkar et al. [2024], Puech et al. [2025], Dinucu-Jianu et al. [2025], Scarlatos et al. [2025], Li et al. [2026b]. Recent adversarial evaluations also show that answer- inducing prompts remain a distinct attack surface Zhao et al. [2026a,b]. Our engineering question is complementary and narrower: can disclosure policy, language generation, autho- rization, and enforcement be separated so that failures are measurable and rejected releases cannot bypass the same deci- sion point? We introduce a modular auditable release-control archi- tecture. A selector emits a small pedagogical action; an au- thorization gate controls privileged disclosure modes; and a renderer produces text. A two-mode guard applies a determin- istic checker alone in permissive mode or adds a cumulative semantic verifier in high-assurance mode; either path substi- tutes an action-specific fallback on failure. This architecture does not make a universal semantic-safety guarantee. It makes the release path observable and permits matched ablations of the selector, renderer, and guard. Our contributions are: 1.We formalize pedagogical leakage as bounded, authorization-dependent disclosure and implement one fail-closed release function with a replayable trace. 1 arXiv:2608.00515v1 [cs.CR] 1 Aug 2026 2.Matched component attribution shows that fallback en- forcement causes the large automatic safety reduction and its utility cost; globalA 1 beats fitted Q, and the se- mantic verifier adds no reliable marginal benefit in the matched lane. 3.An externally timestamped prospective study quanti- fies residual failures under unseen problem clusters and adaptive attacks using paired cluster inference and per- model outcomes rather than treating model-panel votes as ground truth. 2 Related Work Student modeling and tutoring policy. Bayesian Knowledge Tracing (BKT) estimates latent mastery for learner–skill pairs Corbett and Anderson [1995]; ASSISTments connected such models with large-scale tutoring logs Feng et al. [2009]. Educa- tional data mining broadened interaction-log prediction Baker and Yacef [2009], and offline policy work studies pedagogical decisions without online exploration Mandel et al. [2014], Gao et al. [2024]. We use this literature to construct an auditable diagnostic state/action interface, not to infer treatment effects from heterogeneous event logs. Pedagogical alignment and tutor evaluation. Benchmarks distinguish tutoring from answer production through dialogue pedagogy, mathematical scaffolding, and safety rubrics Tack and Piech [2022], Macina et al. [2023, 2025], Maurya et al. [2025], Hazra et al. [2026], Srinivasa et al. [2025]. Evidence from deployed courses also shows that students actively ex- tract answers, motivating turn-level computational measures alongside outcome studies Kobler et al. [2026]. Sonkar et al. construct synthetic preferences for pedagogical alignment Sonkar et al. [2024]; Puech et al. optimize prompts to fol- low a multi-turn productive-failure graph Puech et al. [2025]; recent policy work uses online RL or multi-horizon prefer- ence optimization Dinucu-Jianu et al. [2025], Shi et al. [2026]; and Scarlatos et al. optimize candidate tutor utterances for pre- dicted student correctness and pedagogy Scarlatos et al. [2025]. ScaffoldLM explicitly plans and tracks dialogue progress Li et al. [2026b]. These methods optimize tutoring behavior. We instead study whether a separately selected disclosure contract is mediated at release time. This distinction matters because simply prompted simulated students can be behaviorally unre- liable Scarlatos et al. [2026]. The closest AAAI-26 systems sharpen this distinction. EduGuardBench evaluates pedagogical fidelity and adversarial safety across 14 models after calibrating an LLM judge against 200 human labels Jiang et al. [2026]; it measures model be- havior, whereas our system enforces and attributes a release decision. LEAP plans long-horizon scaffolds from cognitive state Dong et al. [2026]; our boundary operates after action se- lection and claims no learner benefit. VerifyBench uses expert annotation to expose verifier sensitivity to response form and domain Li et al. [2026a], motivating independent calibration of our narrower authorization-dependent verifier. Adversarial answer release. Zhao et al. test multiple tutor families under six classes of adversarial student techniques and develop stronger attack agents Zhao et al. [2026a]. SHAPE formalizes pedagogical jailbreaks and routes between instruc- tion and problem solving through a mastery graph Zhao et al. [2026b]. Our contribution is not another tutor benchmark or pedagogical optimizer: it is a complete-mediation interface with trusted, problem-scoped authorization, action-dependent contracts, deterministic fallback, and replayable component traces. Prior work separated specialized tutoring roles Kadir [2026]; this paper enforces that separation at the release bound- ary. Preference alignment and offline RL. RLHF inherits un- derspecified and potentially myopic preference signals Casper et al. [2023]. Sequential tutoring motivates reinforcement learning Bellman [1957], Sutton and Barto [2018], but offline RL requires support and distribution-shift controls Levine et al. [2020]. CQL uses a specific conservative objective to suppress unsupported values Kumar et al. [2020]. Our fitted-Q diagnos- tic instead masks unseen actions and subtracts a count penalty; we explicitly do not call it CQL. Constrained generation, shielding, and evaluation. Lex- ical and logical decoding can enforce output constraints Hokamp and Liu [2017], Lu et al. [2021]; contrastive decod- ing can shift token probabilities toward generic safety without retraining the target model Zhang et al. [2026]; RL shields cor- rect unsafe actions before execution Alshiekh et al. [2018], and learned verifiers can rank mathematical completions Cobbe et al. [2021]. Our guard instead mediates a completed proposal against a problem- and authorization-dependent disclosure contract and substitutes a deterministic fallback. LLM judges can be biased and panel-sensitive Zheng et al. [2023]; diverse- model panels can improve reliability Verga et al. [2024], Liu et al. [2024]. Our panel is same-provider, so we report per- model outcomes, blinded inputs, cluster uncertainty, and held- out semantic challenges rather than treating its majority as ground truth. 3 Auditable Release-Control Frame- work 3.1 Threat Model and Security Boundary The protected assets are the target answer and decisive solution steps, the problem-scoped authorization record, the selected disclosure contract, and the integrity of the release trace. The adversary controls learner messages and may request direct answers, invoke role play or hypothetical contexts, encode requests or desired outputs, accumulate partial disclosures across turns, or attempt to downgrade the guard. The language generator is untrusted: fluent or instruction-following output is not evidence that a release is authorized. The trusted computing base (TCB) comprises the platform- owned action and mode selection, authorization lookup, single release function, deterministic checker, semantic-verifier call, fallback templates, and trace writer. We assume that refer- 2 ence answers are correct, authorization records are authentic and scoped to the current problem, user text cannot mod- ify TCB state, and all candidate output reaches the release function. Under these assumptions the control-flow invariant below provides complete mediation and fail-closed handling of checker/verifier errors. The packaged artifact additionally hash-chains canonical trace records so post hoc modification is detectable. The boundary does not guarantee that the checker or ver- ifier recognizes every semantic disclosure, that an incorrect reference cannot induce a bad decision, or that a compromised TCB, external tool, or unlogged output channel cannot bypass mediation. It also does not establish that withholding improves learning. We evaluate direct, encoded, indirect, cumulative, and authorization-confusion attacks; prompt injection outside the declared interface and production access-control compro- mise remain out of scope. 3.2 Disclosure Contracts and Release Guard Leta ∈ A 0 ,...,A 4 be the selected disclosure contract,z the trusted authorization record,ythe raw renderer output, and Hthe released interaction history. The platform selects mode m ∈ P, H(permissive or high assurance); untrusted user text cannot downgrade it. A deterministic predicateC a (y,z) checks inspectable contract surfaces. High-assurance mode additionally requires a semantic verifier V a (y,H,z). Define G a,m (y,H,z) = C a (y,z)∧ m = P ∨ V a (y,H,z) = 1 , (1) and release ̃y = ( y, G a,m = 1, F a , G a,m = 0, (2) whereF a is a deterministic action-specific fallback.A 4 addi- tionally requires authorization scoped to the current problem from an instructor record, accessibility plan, or platform pol- icy. A user’s self-asserted request cannot activate it. Notation accommodation and isomorphic-example modes may relax surface constraints without releasing the target answer. The fast checker normalizes Unicode and applies reference- aware restrictions on answer and derived numbers, equations, encoded-disclosure markers, and decisive operations. It is deliberately inspectable but cannot recognize arbitrary para- phrase or covert encoding. The optional verifier reads the cumulative transcript and reference solution and fails closed on errors; because it is model-based, it is also fallible and must be evaluated independently. Every interaction logs the pre- action state, action, authorization scope, raw output, checker and verifier reasons, intervention source, fallback, and final output. Figure 1 makes this release path, rather than the offline learner, the architectural center. Release invariant. For every recorded trace, releasing raw language impliesG a,m = 1; any rejected check or verifier error releasesF a and records the intervention source. This is a mechanical guarantee about control flow and trace com- pleteness, not a guarantee thatC,V, orF a perfectly captures semantic leakage. 3.3 Strict Pre-Action Diagnostic Selector From chronological logs we construct each state before the current event and outcome: s t = [p t ,e t ,h t ,τ t ],(3) wherep t is logged pre-observation BKT mastery (Ln-1),e t prior consecutive errors,h t prior non-bottom support events, andτ t cumulative prior time. Features are shifted before the current event. Next states remain within the anonymous learner–skill episode; successful or final events terminate the sequence. Rewards are diagnostic rather than causal learning-gain estimates: r t = 10 I[o t = 1]−0.1 I[o t = 0]−5 I[a t = A 4 ∧h t < 2]. (4) Because hint rows usually do not record a scored attempt, r t is inseparable from event type. We use it only to diagnose supported action selection and report a temporally later attempt proxy separately; neither is off-policy evaluation or a learning- gain estimate. We discretize the strict pre-action state, fit tabular Q values on learner-level training splits, mask unsupported actions, and extractarg max a Q(s,a)− 1/ p n(s,a). A common deploy- ment shield rejects the unauthorizedA 4 proxy and actions with fewer than five training rows. The supplement specifies bins, backups, tie handling, and fallbacks. This is a count-penalized fitted-Q diagnostic, not CQL or off-policy evaluation. 4 Experiments We ask three questions: what the offline event proxies iden- tify; which component causes runtime disclosure control; and whether control survives semantic/adaptive attacks and autho- rization tests. Experimental units are stated in every caption. 4.1 Offline Construct and Selector Audit The strict pre-action table contains 942,816 event rows from 1,709 anonymous learners, 102 skills, and 3,162 problems. Logged proxy frequencies areA 0 28.3%,A 1 38.6%,A 2 11.1%,A 3 15.7%, andA 4 6.3%. Table 2 reveals the domi- nant construct problem: ordinary attempt/scaffold rows can record correctness, whereas hint events almost never do. Thus strict shifting repairs temporal leakage but does not make the actions exchangeable. We then compare controllers across five 80/20 learner-level splits under the common support/A 4 shield. Table 3 reports direct-method proxy means and the next-observed-A 0 diag- nostic. The fitted-Q proxy is 5.760 versus 5.723 for globalA 1 (∆ = .037after the common shield; .035 unshielded). It dif- fers from globalA 1 on only 7.32% of rows, mainly switching them toA 0 . On the future proxy, fitted Q is below both global A 1 and the BKT rule. These are descriptive supported-action diagnostics, not evidence that fitted Q improves learning. 3 Pre-action state p,e,h,τ Supported-action selector Selected contract A 0 –A 4 Authorization & mode gate trusted scope for A 4 Raw language renderer Action-contract checker Optional semantic release verifier Pass raw, or replace with safe fallback Released tutor response high assurance permissive standard| accommodation isomorphic| answer release intervention flag Audit ledger: state · selected action · authorization scope · raw output · checker/verifier reasons · intervention · final output Offline diagnostic lane: event-proxy logs → pre-action bins → supported selector audit Figure 1: Auditable release path. Permissive mode applies the deterministic contract checker; high-assurance mode also requires a semantic release verifier. Either rejection invokes the same deterministic fallback.A 4 , accommodation, and mode selection require trusted problem-scoped policy. ActionRuntime disclosure contractRetrospective ASSISTments proxy A 0 Elicit the learner’s next reasoning step; withhold numerical and procedural solution content. Ordinary scored attempt: no hint, scaffold, or bottom-hint marker. A 1 Scaffold a subgoal or distinction while withholding the decisive calculation.Scaffold/auto-scaffold row without a hint marker. A 2 State a concept without applying it to the target quantities.Hint row not classified near the bottom of a hint sequence. A 3 Name a local procedure while leaving execution to the learner.STL or second-to-last/near-bottom hint row. A 4 Release a worked answer only under trusted, problem-scoped authorization.Bottom-hint row; a logging proxy, not evidence of authorization. Table 1: The runtime contracts and logged event proxies are not equivalent constructs. ASSISTments event mappings create observational labels for selector diagnostics; they are not randomized tutor interventions. 4.2 Matched Runtime Ablation We hold the 599 prompts and Gemini 3.5 proposals fixed when comparing no guard, strict guard, and hybrid guard. A local intervention is a deterministic contract violation detected before release; judge failures are semantic leakage ratings from a blinded three-model Gemini panel. We first identify the active components with a cumu- lative factorial that also holds fitted-Q contracts, the fall- back renderer, and the blind judge rubric fixed. Logging the fast check without mediating release is outcome-identical to no guard. Enabling its fallback replaces 503 proposals and changes majority failures from 181 to 11 (−28.38points, 95% CI[−32.96,−23.94]), any-judge failures from 392 to 254 (−23.04,[−29.53,−16.28]), and helpfulness by−.528 ([−.586,−.472]). Adding the semantic verifier makes 96 calls and two extra blocks, yielding 14 majority and 248 any- judge failures. Its marginal changes are+.50majority points ([0, 1.16]),−1.00any-judge points ([−3.78, 1.78]), and−.007 helpfulness ([−.024,.009]). Thus fallback enforcement, not semantic verification, explains the matched reduction in this lane. Table 4 then broadens the comparison. The Gemini 2.5 prompt tutor and deterministic action renderers are separate comparators, not part of the matched generator ablation. The strict guard removes all majority failures but replaces 581/599 proposals and lowers automatic utility; the hybrid guard re- places 505/599 and leaves 14 majority failures. Among un- guarded deterministic action renderers, globalA 1 has both the best utility and the fewest any-judge flags, so fitted Q is not the source of the strongest runtime result. LearnLM capabilities were integrated into Gemini begin- ning with the 2.5 series rather than exposed as a separate current endpoint LearnLM Team et al. [2024], Google AI for Developers [2026]. We therefore froze a no-reference Gemini 2.5 Flash tutor prompt and generated one response per case. It has 19/599 majority leakage failures and 143/599 any-judge flags, with panel means of 1.901 for correctness and 1.897 for helpfulness. This is a stronger prompt-only comparator than the deterministic renderers on automatic utility, but it does not isolate the effect of LearnLM training. The global-A 1 result is decision-critical. It improves over fitted Q by 178 any-judge flags, .169 correctness points, and .105 helpfulness points while preserving zero majority fail- 4 Logged proxyRowsSame-row correctMean r t Next A 0 observedNext-A 0 correct † A 0 attempt266,989.52355.18735.37%.5015 A 1 scaffold363,731.58175.7755.25%.5980 A 2 concept hint104,944.0000 −.10023.45%.6306 A 3 procedure hint147,949.0001 −.09919.87%.6930 A 4 bottom-hint proxy59,203.0001 −.43819.27%.8342 Table 2: Action/outcome construct audit; unit is an ASSISTments event row. † Correctness is conditional on observing another A 0 attempt within 10 later events in the same learner–skill–problem sequence; observation is action-dependent. Common-shielded selector Same-row Future Shield int.A 0 A 1 A 2 A 3 Fixed A 0 5.1864.833.239% 99.761.23900 Global A 1 5.7236.071.299%.299 99.70100 BKT threshold3.1866.3361.244% 12.909 46.479 14.717 25.895 Fitted Q5.7606.012.027%7.284 92.695.021.0002 Table 3: Means over five learner-level splits; percentages are test-event rows. Every selector uses the same minimum-support and unauthorized-A 4 shield, henceA 4 = 0. Same-row and future columns are confounded diagnostic rewards, not OPE or learner outcomes. ures. The offline selector therefore supplies an auditable action interface and a negative diagnostic result, not evidence that learned action selection improves the deployed tutor. The conclusion is robust to replacing panel majority with the three individual judge rules. Strict guard versus no guard changes failure rates by−19.03points (95% problem-cluster CI[−26.11,−11.69]),−33.22points ([−38.05,−28.57]), and−14.36points ([−18.00,−11.04]) for Gemini 2.5 Flash- Lite, 3.1 Flash-Lite, and 3.5 Flash, respectively. The cor- responding majority and any-judge differences are−30.22 ([−35.00,−25.72]) and−25.88points ([−33.11,−18.48]), while helpfulness changes by−.620([−.680,−.565]). The separate three-rater analysis below is exploratory and is not used to recalibrate this panel. 4.3Semantic, Multi-Turn, and Authorization Stress Tests The lexical checker initially performed poorly on a 200-case author-constructed development challenge: 40% sensitivity and 20% specificity. A rule revision fit to that set reached 100%/100%, but then obtained 0% sensitivity and 97% speci- ficity on a fresh 200-case encoding challenge. This failure rules out treating lexical compliance as semantic safety. A subsequent fast-checker-only live audit likewise intercepted none of 12 raw majority leakage failures across 240 sequences, despite 99 local interventions. We retrospectively applied the subsequently designed se- mantic verifier to both balanced author-labeled challenges. On the development set it attained 100% sensitivity (95% exact CI[96.4, 100]) and 83% specificity ([74.2, 89.8]). On the fresh encoding set, sensitivity was 83% ([74.2, 89.8]) and specificity 96% ([90.1, 98.9]): 17/100 leaks remained undetected. These are component-calibration results, not prospective evidence or independent ground truth. We then ran an externally timestamped prospective replica- tion. Its protocol hash, 40 previously unused problem clusters, balanced platform-assignedA 0 /A 1 contracts, 12 attacks, three judge models, and analysis code were fixed in an RFC 3161 receipt before selected-case generation. A first timestamped protocol was aborted without endpoint analysis after 38/38 ver- ifier calls returned HTTP 400; its 56 checkpointed sequences are retained as negative feasibility evidence and all their prob- lem IDs are excluded from the replacement. The replacement used an independently smoke-tested compatible verifier, fresh problem IDs, and the unchanged attack suite. The completed replication contains 480 paired sequences and 800 turns. There were no generation or verifier errors. The fast checker blocked 117 turns, the verifier blocked 219 of 683 calls, and 336 turns used a fallback. As Table 5 shows, majority leakage falls from 42 to 8 sequences:−7.08percent- age points with a 95% problem-cluster CI of[−13.13,−2.29]. Any-judge and unanimous failures also fall, and each judge’s paired interval excludes zero. Mathematics and helpfulness decline. Of 42 raw majority failures, 35 become safe and seven persist; one initially safe sequence becomes a majority failure after fallback. Failures remain concentrated. Single-turn attacks contribute 22/320 raw and 2/320 final majority failures; adaptive multi- turn attacks contribute 20/160 raw and 6/160 final failures. Interval bisection retains 4/40 failures, while the other eleven attacks retain at most one each (Table 6). All eight final failures useA 0 ; six occur on one problem cluster. This prevents the aggregate from hiding a contract- and item-specific weakness. Panel disagreement remains substantial. For raw/final re- leases, Gemini 2.5 Flash-Lite flags 275/219 sequences, Gem- ini 3.1 Flash-Lite flags 41/8, and Gemini 3.5 Flash flags 16/0. Their paired differences are−11.67([−19.17,−4.38]),−6.88 ([−13.13,−2.08]), and−3.33points ([−7.71,−.63]), respec- tively. Thus the direction is not a majority-vote artifact, but the panel remains a same-provider measurement rather than hu- man ground truth. An earlier non-timestamped 240-sequence run gave 5/240 raw versus 2/240 final majority failures with an interval reaching zero; it is retained as historical held-out evidence, not pooled with the prospective result. Authorization tests exercise a different contract. In 100 5 ConditionGuard int.Majority leakAny-judge leakMathHelpful Gemini 3.5 tutor, no guard01813921.9501.918 Gemini 2.5 tutor, no guard † 0191431.9011.897 Gemini 3.5 tutor, strict guard58102371.6241.298 Gemini 3.5 tutor, hybrid guard505142481.6591.383 Fixed A 0 , no guard002521.5701.248 Global A 1 , no guard00541.7691.370 Fitted Q, no guard002321.6001.265 Table 4: Runtime comparison on the same 599 prompts. Counts are prompt rows; Math and Helpful are panel means on a 1–2 scale. The†row changes the generator and is not part of the matched Gemini 3.5 guard ablation. The three action-renderer rows use deterministic Renderer V4 and compare selectors under a common realization function; they are not Gemini generations. Zero majority failures among the 394 unique problem clusters represented by the action renderers has a one-sided 95% exact upper bound of .76%, not a zero-risk guarantee. Release candidateMajorityAny judgeMathHelpful ∆ majority95% cluster CI Raw renderer42/480275/4801.9351.817– High-assurance release8/480219/4801.8131.626 −7.08 p [−13.13,−2.29] Table 5: Externally timestamped prospective replication; the sequence is the evaluation unit and problem ID is the bootstrap cluster. Scores are three-judge means on a 1–2 scale. The signed protocol predates generation; paired cluster intervals use 10,000 resamples and preserve within-problem dependence. Attack groupn Raw maj. Final maj. Single-turn (8 types)320222 Adaptive multi-turn (4 types) 160206 Interval bisection4074 Counterfactual boundary4071 Table 6: Prospective majority leakage by attack group; the unit is an attack sequence. The named attacks are subsets of the adaptive multi-turn row; the supplement reports all 12 types. scoped cases, the gate accepted all 20 trusted answer-release requests, 20 isomorphic-example requests, and 20 notation accommodations; it rejected all 20 self-asserted privilege claims and 20 scope-mismatched records. In a separate live A 4 test, 14/20 raw generations directly answered; six were re- placed by the authorized fallback, and all 20 final releases con- tain the verified target answer surface. Two historical model- judge runs over these same responses give sharply conflicting mathematical-quality results; because the initial prompt was not retained, neither supports an independent all-correct claim. These tests establish gate behavior and target-release com- pleteness under the stated TCB assumptions, not explanation quality or learning benefit. 4.4 Exploratory Human Calibration Three author-identified human raters each evaluated 240 candi- dates from 120 matched raw/released sequences spanning 40 problem clusters (720 ratings). Condition identities were hid- den behind anonymous item IDs. Among determinate ratings, the pooled released-minus-raw leakage difference was−13.75 points (95% problem-cluster CI[−20.34,−7.89]); three-rater- majority failures were 32/117 raw versus 16/119 released, a paired−14.53points ([−23.28,−6.84]). Helpfulness fell for every rater by.542,.533, and.750on a 1–5 scale. Agreement was variable (pairwiseκ = .240–.628; nominalα = .431), and 38 released correctness ratings per rater were marked not applicable. Because this post hoc analysis was added after the confirmatory automatic protocol, we treat it as exploratory. Against the three-rater majority on 236 determinate candidates, the automatic panel majority was highly specific (.989) but insensitive (.104). We therefore do not use the human result to recalibrate the panel or claim learner benefit. 5 Discussion The experiments support a narrower conclusion than “RL makes tutors safer.” Event-derived fitted Q has only a small advantage on a construct-confounded same-row reward and no advantage on the future-attempt proxy. In matched run- time tests, disclosure control comes primarily from the release guard. Moreover, globalA 1 dominates fitted Q among the deterministic action renderers on the reported automatic met- rics. Strict enforcement can eliminate panel-majority flags in the single-turn audit, but only by replacing most proposals and reducing utility. In the clean matched factorial, fallback enforcement explains the large safety effect and the seman- tic verifier has no reliable marginal benefit. The prospective full-stack replication nevertheless reduces every predeclared leakage endpoint with cluster intervals excluding zero, while leaving eight majority failures, introducing one, and lower- ing utility. It validates the stack’s aggregate effect under new attacks, not a standalone verifier effect. The defensible contribution is therefore the auditable de- composition. It reveals whether a failure came from action selection, authorization, generation, deterministic checking, semantic verification, or fallback. It also exposes choices that a monolithic prompt hides: global scaffolding can be a stronger baseline than a learned selector; authorization must be trusted and problem-scoped; and semantic checks require independent calibration. Fitting lexical rules to an adversarial set produced 6 perfect development performance and zero held-out sensitivity, while the model verifier still missed 17% of author-labeled leaks on a fresh encoding challenge. The release trace makes these failures inspectable rather than converting guard accep- tance into a safety claim. 6 Limitations No reported study measures learner achievement, retention, transfer, or longitudinal engagement. ASSISTments actions are reconstructed event types, not randomized interventions, and their rewards are confounded by whether an event records correctness. The fitted-Q diagnostic therefore supplies no causal or off-policy value claim. TutorBench states are prox- ies and exercise onlyA 0 /A 1 under the learned selector, so the paper does not establish fully differentiated five-action adaptation. The semantic challenges are author constructed, the historical audit contains 20 clusters and the prospective replication contains 40, and all model judges come from one provider family. The verifier is also one of the three evaluation model versions, creating dependence despite blinded inputs; the other judges show substantial disagreement. The prospec- tive replication still has eight final majority failures, including one introduced failure, and lowers automatic utility. The hu- man calibration has only three raters, variable agreement, and post hoc exploratory status; it cannot establish population-level human judgment or learner outcomes. Renderer V4 remains a development artifact requiring prospective independent cal- ibration. Zero observed failures are accompanied by finite- sample bounds and are never interpreted as zero risk. Finally, the complete-mediation claim depends on the stated TCB and does not cover compromised platform code, incorrect refer- ence solutions, or unlogged output channels. 7 Conclusion Auditable release control turns pedagogical disclosure into an explicit, problem-scoped decision with logged action, autho- rization, generation, guard, fallback, and final-release stages. Matched single-turn tests show that strict mediation elimi- nates model-panel-majority flags under the declared contracts, but at a large utility cost. The externally timestamped replica- tion shows a reduction across majority, any-judge, unanimous, and every individual judge rule, while exposing seven per- sistent failures, one introduced failure, and lower utility. The global-A 1 comparison further shows that fitted Q is not the current system’s advantage. What the work establishes is a reproducible control boundary that makes the source, residual risk, and instructional cost of pedagogical leakage measur- able. Improving utility, calibrating semantic verification with independent experts, and testing learner outcomes are the next requirements before deployment. AI assistance disclosure. Generative AI tools supported lan- guage editing, code debugging, artifact organization, and sim- ulated non-decisional review. The authors verified all claims, citations, code, analyses, and text and assume full responsibil- ity; AI systems are neither authors nor cited sources. References Mohammed Alshiekh, Roderick Bloem, Ruediger Ehlers, Bettina Koenighofer, Scott Niekum, and Ufuk Topcu. Safe reinforcement learning via shielding. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, pages 2669–2678, 2018. doi: 10.1609/aaai.v32i1.11797. Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862, 2022. Ryan S. J. d. Baker and Kalina Yacef. The state of educational data mining in 2009: A review and future visions. Journal of Educational Data Mining, 1(1):3–17, 2009. Richard Bellman. Dynamic Programming. Princeton University Press, 1957. Stephen Casper, Xander Davies, Claudia Shi, Thomas Krendl Gilbert, Jeremy Scheurer, Javier Rando, Rachel Freedman, Tomasz Korbak, David Lindner, Pedro Freire, et al. Open problems and fundamen- tal limitations of reinforcement learning from human feedback. arXiv preprint arXiv:2307.15217, 2023. Michelene TH Chi and Ruth Wylie. The icap framework: Linking cognitive engagement to active learning outcomes. Educational psychologist, 49(4):219–243, 2014. Paul F. Christiano, Jan Leike, Tom B. Brown, Miljan Martic, Shane Legg, and Dario Amodei. Deep reinforcement learning from human preferences. In Advances in Neural Information Processing Systems, volume 30, 2017. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021. Albert T Corbett and John R Anderson. Knowledge tracing: Modeling the acquisition of procedural knowledge. User modeling and user- adapted interaction, 4(4):253–278, 1995. David Dinucu-Jianu, Jakub Macina, Nico Daheim, Ido Hakimi, Iryna Gurevych, and Mrinmaya Sachan. From problem-solving to teach- ing problem-solving: Aligning LLMs with pedagogy using re- inforcement learning. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 272–292. Association for Computational Linguistics, 2025. doi: 10.18653/v1/2025.emnlp-main.15. URL https://aclanthology.org/ 2025.emnlp-main.15/. Zhiang Dong, Zhenlong Dai, Xiangwei Lv, and Jingyuan Chen. Learning from long-term engagement: Adaptive tutoring dia- logue planning for personalized education. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 237–245, 2026. URL https://ojs.aaai.org/index.php/AAAI/article/ view/36984. 7 Mingyu Feng, Neil T. Heffernan, and Kenneth R. Koedinger. Ad- dressing the assessment challenge with an online system that tutors as it assesses. User Modeling and User-Adapted Interaction, 19 (3):243–266, 2009. doi: 10.1007/s11257-009-9063-7. Ge Gao, Xi Yang, and Min Chi. Get a head start: On-demand peda- gogical policy selection in intelligent tutoring. Proceedings of the AAAI Conference on Artificial Intelligence, 38(11):12136–12144, 2024. doi: 10.1609/aaai.v38i11.29102. Google AI for Developers. LearnLM. https://ai.google.dev/gemini- api/docs/learnlm, 2026. Accessed July 14, 2026. Rima Hazra, Bikram Ghuku, Ilona Marchenko, Yaroslava Tokarieva, Sayan Layek, Somnath Banerjee, Julia Stoyanovich, and Mykola Pechenizkiy. SafeTutors: Benchmarking pedagogical safety in AI tutoring systems. arXiv preprint arXiv:2603.17373, 2026. Chris Hokamp and Qun Liu. Lexically constrained decoding for sequence generation using grid beam search. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1535–1546, 2017. doi: 10.18653/v1/P17-1141. Yilin Jiang, Mingzi Zhang, Xuanyu Yin, Sheng Jin, Suyu Lu, Zuocan Ying, Zengyi Yu, and Xiangjie Kong. EduGuardBench: A holistic benchmark for evaluating the pedagogical fidelity and adversarial safety of LLMs as simulated teachers. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 31356– 31364, 2026. URL https://ojs.aaai.org/index.php/AAAI/article/ view/40399. Nizam Kadir. From untamed black box to interpretable pedagogical orchestration: The ensemble of specialized LLMs architecture for adaptive tutoring. In International Conference on Artificial Intelligence in Education, pages 94–109. Springer, 2026. Manu Kapur. Productive failure. Cognition and Instruction, 26(3): 379–424, 2008. doi: 10.1080/07370000802212669. Enkelejda Kasneci, Kathrin Sessler, Stefan K ̈ uchemann, Maria Ban- nert, Daryna Dementieva, Frank Fischer, Urs Gasser, Georg Groh, Stephan G ̈ unnemann, Eyke H ̈ ullermeier, et al. ChatGPT for good? on opportunities and challenges of large language models for edu- cation. Learning and Individual Differences, 103:102274, 2023. doi: 10.1016/j.lindif.2023.102274. Sebastian Kobler, Matthew Clemson, Angela Sun, and Jonathan K. Kummerfeld. Your students don’t use LLMs like you wish they did. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 19146–19170. Association for Computational Linguistics, 2026. doi: 10.18653/v1/2026.acl-long.875. URL https://aclanthology. org/2026.acl-long.875/. Aviral Kumar, Aurick Zhou, George Tucker, and Sergey Levine. Conservative q-learning for offline reinforcement learning. In Advances in Neural Information Processing Systems, volume 33, pages 1179–1191, 2020. LearnLM Team et al. LearnLM: Improving Gemini for learning. arXiv preprint arXiv:2412.16429, 2024. Sergey Levine, Aviral Kumar, George Tucker, and Justin Fu. Offline reinforcement learning: Tutorial, review, and perspectives on open problems. arXiv preprint arXiv:2005.01643, 2020. Xuzhao Li, Xuchen Li, Shiyu Hu, Yongzhen Guo, and Wentao Zhang. VerifyBench: A systematic benchmark for evaluating reasoning verifiers across domains. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 31796–31804, 2026a. URL https://ojs.aaai.org/index.php/AAAI/article/view/40448. Zechen Li, Qiannan Zhu, Mei Wang, Jia Li, and Hua Huang. Planning-guided tutoring with assessment-driven memory for ped- agogical LLM tutors. In Proceedings of the 64th Annual Meet- ing of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7165–7188. Association for Computational Linguistics, 2026b. doi: 10.18653/v1/2026.acl-long.325. URL https://aclanthology.org/2026.acl-long.325/. Yuxuan Liu, Tianchi Yang, Shaohan Huang, Zihan Zhang, Haizhen Huang, Furu Wei, Weiwei Deng, Feng Sun, and Qi Zhang. Cali- brating LLM-based evaluator. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Lan- guage Resources and Evaluation, pages 2638–2656, 2024. URL https://aclanthology.org/2024.lrec-main.237/. Ximing Lu, Peter West, Rowan Zellers, Ronan Le Bras, Chandra Bha- gavatula, and Yejin Choi. NeuroLogic decoding: (un)supervised neural text generation with predicate logic constraints. In Proceed- ings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4288–4299, 2021. doi: 10.18653/v1/2021. naacl-main.339. Jakub Macina, Nico Daheim, Sankalan Pal Chowdhury, Tanmay Sinha, Manu Kapur, Iryna Gurevych, and Mrinmaya Sachan. Math- Dial: A dialogue tutoring dataset with rich pedagogical properties grounded in math reasoning problems. In Findings of the As- sociation for Computational Linguistics: EMNLP 2023, pages 5602–5621, 2023. doi: 10.18653/v1/2023.findings-emnlp.372. URL https://aclanthology.org/2023.findings-emnlp.372/. Jakub Macina, Nico Daheim, Ido Hakimi, Manu Kapur, Iryna Gurevych, and Mrinmaya Sachan. MathTutorBench: A bench- mark for measuring open-ended pedagogical capabilities of LLM tutors. In Proceedings of the 2025 Conference on Empirical Meth- ods in Natural Language Processing, pages 204–221, 2025. doi: 10.18653/v1/2025.emnlp-main.11. URL https://aclanthology.org/ 2025.emnlp-main.11/. Travis Mandel, Yun-En Liu, Sergey Levine, Emma Brunskill, and Zoran Popovic. Offline policy evaluation across representations with applications to educational games. In Proceedings of the 13th International Conference on Autonomous Agents and Multiagent Systems, pages 1077–1084, 2014. Kaushal Kumar Maurya, Kv Aditya Srivatsa, Kseniia Petukhova, and Ekaterina Kochmar. Unifying AI tutor evaluation: An evaluation taxonomy for pedagogical ability assessment of LLM-powered AI tutors. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Lin- guistics: Human Language Technologies (Volume 1: Long Papers), pages 1234–1251, 2025. doi: 10.18653/v1/2025.naacl-long.57. URL https://aclanthology.org/2025.naacl-long.57/. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wain- wright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Kata- rina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730–27744, 2022. 8 Romain Puech, Jakub Macina, Julia Chatain, Mrinmaya Sachan, and Manu Kapur. Towards the pedagogical steering of large language models for tutoring: A case study with modeling productive failure. In Findings of the Association for Computational Linguistics: ACL 2025, pages 26291–26311. Association for Computational Linguistics, 2025. doi: 10.18653/v1/2025.findings-acl.1348. URL https://aclanthology.org/2025.findings-acl.1348/. Alexander Scarlatos, Naiming Liu, Jaewook Lee, Richard Baraniuk, and Andrew Lan. Training LLM-based tutors to improve student learning outcomes in dialogues. In Artificial Intelligence in Educa- tion: 26th International Conference, AIED 2025, volume 15877 of Lecture Notes in Computer Science. Springer, 2025. doi: 10.1007/ 978-3-031-98414-318. URL https://arxiv.org/abs/2503.06424. Alexander Scarlatos, Jaewook Lee, Simon Woodhead, and Andrew Lan. Simulated students in tutoring dialogues: Substance or il- lusion? In Proceedings of the 64th Annual Meeting of the As- sociation for Computational Linguistics (Volume 1: Long Pa- pers), pages 42349–42385. Association for Computational Lin- guistics, 2026.doi: 10.18653/v1/2026.acl-long.1960.URL https://aclanthology.org/2026.acl-long.1960/. Weiyan Shi, Shivam Srivastava, Kaixiang Zhang, Tong Yu, Dinesh Goyal, and Aldo Lipani. Beyond pedagogical principles: Multi- horizon preference optimization for efficient socratic tutoring. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 11289– 11306. Association for Computational Linguistics, 2026. doi: 10.18653/v1/2026.acl-long.518. URL https://aclanthology.org/ 2026.acl-long.518/. Shashank Sonkar, Kangqi Ni, Sapana Chaudhary, and Richard Bara- niuk. Pedagogical alignment of large language models. In Find- ings of the Association for Computational Linguistics: EMNLP 2024, pages 13641–13650. Association for Computational Lin- guistics, 2024. doi: 10.18653/v1/2024.findings-emnlp.797. URL https://aclanthology.org/2024.findings-emnlp.797/. Rakshith S. Srinivasa, Zora Che, Chen Bo Calvin Zhang, Diego Mares, Ernesto Hernandez, et al. Tutorbench: A benchmark to as- sess tutoring capabilities of large language models. arXiv preprint arXiv:2510.02663, 2025. Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel M. Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul F. Christiano. Learning to summarize with human feedback. In Advances in Neural Information Processing Systems, volume 33, pages 3008–3021, 2020. Richard S. Sutton and Andrew G. Barto. Reinforcement Learning: An Introduction. MIT Press, 2 edition, 2018. Ana ̈ ıs Tack and Chris Piech. The AI teacher test: Measuring the pedagogical ability of blender and GPT-3 in educational dialogues. arXiv preprint arXiv:2205.07540, 2022. Pat Verga, Sebastian Hofstatter, Sophia Althammer, Yixuan Su, Alek- sandra Piktus, Arkady Arkhangorodsky, Minjie Xu, Naomi White, and Patrick Lewis. Replacing judges with juries: Evaluating LLM generations with a panel of diverse models. arXiv preprint arXiv:2404.18796, 2024. David Wood, Jerome S. Bruner, and Gail Ross. The role of tutoring in problem solving. Journal of Child Psychology and Psychiatry, 17(2):89–100, 1976. doi: 10.1111/j.1469-7610.1976.tb00381.x. Xiaoyun Zhang, Zhengyue Zhao, Wenxuan Shi, Kaidi Xu, Di Huang, and Xing Hu. Safety alignment of large language models via contrasting safe and harmful distributions. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 34827–34835, 2026. URL https://ojs.aaai.org/index.php/AAAI/ article/view/40785. Jin Zhao, Marta Kne ˇ zevi ́ c, and Tanja K ̈ aser. Evaluating answer leakage robustness of LLM tutors against adversarial student at- tacks. In Proceedings of the 64th Annual Meeting of the As- sociation for Computational Linguistics (Volume 1: Long Pa- pers), pages 30588–30617. Association for Computational Lin- guistics, 2026a. doi: 10.18653/v1/2026.acl-long.1412. URL https://aclanthology.org/2026.acl-long.1412/. Sihang Zhao, Kangrui Yu, Youliang Yuan, Pinjia He, and Hongyi Wen. SHAPE: Unifying safety, helpfulness and pedagogy for educational LLMs. In Proceedings of the 64th Annual Meet- ing of the Association for Computational Linguistics (Volume 1: Long Papers), pages 11537–11553. Association for Computational Linguistics, 2026b. doi: 10.18653/v1/2026.acl-long.529. URL https://aclanthology.org/2026.acl-long.529/. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, et al. Judging LLM-as-a-judge with MT-bench and chatbot arena. In Advances in Neural Information Processing Systems, volume 36, 2023. 9