Paper deep dive

Trust by Design: Trust Calibration Through Non-Advisory Socratic Dialogue in Conversational Agents

Roba Hassan, Nahla Aboromi, Naomi Unkelos-Shpigel

Year: 2026Venue: arXiv preprintArea: cs.SEType: PreprintEmbeddings: 21

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%

Last extracted: 9/16/2026, 7:27:04 AM

Summary

The paper introduces CASELy, a conversational agent designed for Social-Emotional Learning (SEL) that utilizes non-advisory Socratic dialogue to calibrate user trust. By refusing to provide advice or interpretations and grounding responses exclusively in user input, CASELy aims to prevent over-reliance and preserve user autonomy. A pilot study with higher education students demonstrated that this design significantly improved user experience compared to a non-dialogue control, validating the hypothesis that interactional constraints can foster appropriate reliance in sensitive AI contexts.

Entities (6)

CASELy · conversational-agent · 98%Trust Calibration · concept · 96%Social-Emotional Learning · domain · 94%Socratic Dialogue · interaction-method · 92%UEQ-S · metric · 90%Braude College of Engineering · organization · 85%

Relation Signals (5)

CASELy → uses → Socratic Dialogue

confidence 95% · CASELy is an anonymous, web-based conversational system... The system engages users through open-ended questions grounded strictly in their own responses.

CASELy → targets → Social-Emotional Learning

confidence 93% · CASELy is an anonymous, web-based conversational system designed to support reflective SEL while maintaining calibrated trust.

Socratic Dialogue → achieves → Trust Calibration

confidence 90% · This design operationalizes trust calibration by constraining agent agency rather than optimizing capability.

CASELy → improves → User Experience

confidence 88% · participants interacting with the Socratic dialogue reported substantially higher user experience (UEQ-S overall = 1.50) compared to a non-dialogue control (0).

CASELy → developedby → Braude College of Engineering

confidence 85% · Roba Hassan, Nahla Aboromi, and Naomi Unkelos-Shpigel... Braude College of Engineering, Israel

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:As conversational AI systems increasingly operate in sensitive domains, the central challenge shifts from usability to trust calibration, ensuring that users rely on systems neither too much nor too little. Systems that provide advice or interpretations risk encouraging inappropriate reliance, particularly when users perceive AI outputs as authoritative. We present CASELy, a conversational agent explicitly designed to limit its own authority through non-advisory Socratic dialogue. The agent asks reflective questions grounded exclusively in user input and refuses to provide advice, recommendations, or interpretations. This design operationalizes trust calibration by constraining agent agency rather than optimizing capability. In a pilot randomized controlled study with higher education students, participants interacting with the Socratic dialogue reported substantially higher user experience (UEQ-S overall = 1.50) compared to a non-dialogue control (0). Qualitative findings identify three mechanisms supporting calibrated trust: transparency through visible grounding, preservation of user decision authority, and reduced fear of judgment. We argue that appropriate reliance can be achieved through interactional constraints, offering a design pattern for trustworthy conversational AI in sensitive contexts.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

17,850 characters extracted from source content.

Expand or collapse full text

Trust by Design: Trust Calibration Through Non-Advisory Socratic Dialogue in Conversational Agents Roba Hassan 1 , Nahla Aboromi 1 , and Naomi Unkelos-Shpigel 1 1 Braude College of Engineering, Israel Roba Hassan, Nahla Aboromi, and Naomi Unkelos-Shpigel contributed equally to this work. Roba Hassan: Roba.Hassan@e.braude.ac.ilNahla Aboromi: Nahla.Aboromi@e.braude.ac.il Naomi Unkelos-Shpigel: Naomius@braude.ac.il Abstract As conversational AI systems increasingly operate in sensitive domains, the central challenge shifts from usability to trust calibration, ensuring that users rely on systems neither too much nor too little. Systems that provide advice or interpretations risk encouraging inappropriate reliance, particularly when users perceive AI outputs as authoritative. We present CASELy, a conversational agent explicitly designed to limit its own authority through non-advisory Socratic dialogue. The agent asks reflective questions grounded exclusively in user input and refuses to provide advice, recommendations, or interpretations. This design operationalizes trust calibration by constraining agent agency rather than optimizing capabil- ity. In a pilot randomized controlled study with higher education students, participants interacting with the Socratic dialogue reported substantially higher user experience (UEQ-S overall = 1.50) compared to a non-dialogue control (0). Qualitative findings identify three mechanisms supporting calibrated trust: transparency through visible grounding, preservation of user decision authority, and reduced fear of judg- ment. We argue that appropriate reliance can be achieved through interactional constraints, offering a design pattern for trustworthy conversational AI in sensitive contexts. Keywords: Socratic human-agent interaction, trust calibration theory, social-emotional learning 1 Introduction Social-emotional learning (SEL) is increasingly recognized as a critical component of student success in higher education, influencing well-being, engagement, and academic outcomes [1, 2, 3]. Empirical studies and reviews consistently demonstrate that SEL competencies such as self-awareness, emotional regulation, and responsible decision-making are associated with improved academic performance and resilience [1, 3, 4]. However, despite this evidence, access to scalable, non-judgmental SEL support remains limited, and many existing programs rely heavily on instructor facilitation or institutional resources [5, 6]. Trust calibration theory emphasizes that trustworthy systems align user trust with system capabilities and intent, rather than maximizing trust [7]. When conversational agents provide advice, interpretations, or confident recommendations, users may develop miscalibrated trust, leading to inappropriate reliance or reduced personal agency [7, 8]. Research shows explanation and confidence cues can improve perceived trust while simultaneously degrading calibration accuracy [8]. Conversational AI systems are often proposed as scalable tools for supporting reflection and learning. Yet, research on human–AI interaction highlights a key risk: when systems provide advice, interpretations, or confident recommendations, users may develop miscalibrated trust, leading to inappropriate reliance or 1 arXiv:2609.14818v1 [cs.SE] 13 Sep 2026 reduced personal agency [7, 8]. Trust calibration theory emphasizes that trustworthy systems are not those that maximize user trust, but those that align user trust with system capabilities and intent [7]. In sensitive domains such as SEL, this alignment is particularly important. Pedagogical conversational agents that appear authoritative or evaluative may unintentionally undermine reflective learning by shifting responsibility from the learner to the system [9]. This creates a tension between providing support and preserving autonomy. In this work, we explore a design approach that addresses this tension by constraining system behavior rather than enhancing system intelligence. We investigate whether a conversational AI system that explicitly avoids advice, judgment, and interpretation using only Socratic questioning can support reflection while fostering calibrated trust. Our research question is: How does non-advisory Socratic dialogue in conversational AI influence trust calibration and appropriate reliance in social-emotional learning contexts? 2 The CASELy System CASELy is an anonymous, web-based conversational system designed to support reflective SEL while main- taining calibrated trust. The system comprises three stages: 1. Socio-Emotional Simulation: Users respond to realistic academic scenarios through open-ended written reflection, designed to activate CASEL competencies. 2. Structured SEL Analysis: User responses are analyzed across five CASEL dimensions, producing descriptive feedback and scores without evaluative or prescriptive language. 3. Non-Advisory Socratic Dialogue (Experimental Condition): The system engages users through open-ended questions grounded strictly in their own responses. It avoids advice, recommendations, or interpretations; asks one question per turn; and concludes with a reflective summary mirroring user language. A control condition completes the same flow without dialogue. The system follows privacy-by-design principles [10], including full anonymity via randomly generated identifiers and separation between research and user data. 2.1 Trust Calibration Through Interaction Design CASELy operationalizes trust calibration by minimizing interactional signals that inflate perceived system authority. In contrast to advisory pedagogical agents [9], the system avoids evaluative or instructional lan- guage, grounds all dialogue explicitly in user input, restricts conversational turns to reflection rather than guidance, and makes system limitations continuously visible through behavior. This approach aligns with trust calibration theory by ensuring users perceive the system as a reflective aid rather than an expert or decision-maker [7]. Instead of increasing trust through confidence or explanation, CASELy aims to prevent over-trust by design. Figure 1 presents the socio-emotional simulation screen, where participants engage with a realistic academic scenario and provide an open-ended written response reflecting their emotional and cognitive processing. Figure 2 presents automated CASEL analysis with feedback, explanations, and informing stages. The feedback is organized into three components that operationalize trust calibration: Strengths (green) iden- tifies observed competencies grounded in actual user responses; Areas for Improvement (yellow) presents growth opportunities using reflective prompts rather than prescriptive advice; and Suggested Intervention 2 Figure 1: Example of a Socio-Emotional Simulation Screen. (blue) offers pathways for self-directed learning through practice exercises. This three-part structure imple- ments trust calibration principles by making analysis transparent and verifiable, using descriptive rather than evaluative language, and preserving user autonomy—users can verify feedback against their own responses and maintain decision-making control over their learning trajectory. Figure 2: Example of AI-Generated CASEL Competency Analysis and Feedback. 3 Pilot Study and Preliminary Evaluation A pilot study involved four senior undergraduate students (three experimental, one control; fourth-year software engineering students, ages 20–25, three females and one male). Participants interacted with the system for approximately 15 minutes and completed the UEQ-S user experience questionnaire [11] and semi-structured interviews. Figure 3 presents CASELy’s experimental architecture, demonstrating how trust calibration principles are operationalized through system design. The flow comprises seven sequential stages: (1) User Assign- ment randomly distributes participants to three experimental groups (A, B, C) or a control group (D); (2) Baseline Assessment establishes pre-intervention CASEL competencies; (3) Simulation & Analysis presents realistic scenarios followed by AI-driven descriptive feedback without evaluative judgment; (4) AI-Guided Socratic Dialogue (experimental groups only) delivers non-advisory questions grounded exclusively in user responses—the key intervention absent in control; (5) Post-Intervention Assessment measures changes via the CASEL questionnaire and UEQ-S; (6) Final Evaluation completes the interaction. Two foundational principles support trust calibration: Privacy-by-Design [10] ensures full anonymity and psychological safety 3 for honest self-disclosure, while AI-Driven Components (automated CASEL analysis, Socratic dialogue, reflective summary) constrain system authority by analyzing and questioning without advising. This ar- chitecture embodies trust calibration theory by isolating the effect of non-advisory interaction patterns— comparing groups with and without dialogue tests whether strategic limitations (refusing advice, visible grounding) enhance appropriate reliance, demonstrating that trust calibration emerges from interaction de- sign rather than capability maximization. Figure 3: Evaluation design. 4 Discussion and Conclusions We address our research question through integrated quantitative and qualitative analysis. Quantitatively, participants engaging with Socratic dialogue demonstrated substantially higher UEQ-S scores (overall = 1.50) versus non-dialogue control (0), revealing enhanced user engagement and experience. Qualitatively, Figure 4 displays thematic analysis of pilot interviews using a radial 0–5 scale, where the outer axis rep- resents mention frequency and color indicates positive sentiment. Bot Interaction emerged as the most frequently discussed category, while User Experience and SEL Learning received the highest positive sen- timent ratings. The divergence between dialogue and non-dialogue conditions is evident across multiple dimensions, with particularly strong positive sentiment for User Experience, Group Difference, and Practi- cal Use. This pattern underscores the centrality of Socratic dialogue in shaping participant experiences, as the interactive component dominated both the frequency and emotional valence of participant reflections. Qualitative Findings. Thematic analysis of semi-structured interviews following the pilot revealed systematic patterns linking participant experiences to trust calibration theory principles. Three core mecha- nisms emerged as central to appropriate reliance: preservation of autonomy and decision ownership, reduced fear of judgment in sensitive reflection, and increased trust through visible grounding and limited system authority. Participants consistently recognized and valued the system’s constraints—explicitly noting how the refusal to provide advice, the grounding in their own language, and the non-judgmental questioning style shaped their experience. These observations directly validate the theoretical claim that trust calibration can be operationalized through interaction design rather than capability enhancement. Table 1 systematically maps participant evidence to specific trust calibration principles from the liter- ature, demonstrating how CASELy’s design features translate theoretical constructs into observable user 4 Figure 4: Thematic analysis of overall pilot study interviews: frequency and sentiment across categories. experiences. Each row illustrates the alignment between a trust calibration principle, the corresponding system design feature, direct participant testimony, and the theoretical mechanism by which appropriate reliance is achieved. These findings align with prior work, suggesting that reduced system assertiveness can improve trust calibration and user experience in automated systems [7, 8]. Experimental participants particularly valued the AI-guided dialogue for reducing fear of judgment and enabling honest self-expression. This work contributes to research on trustworthy conversational AI by demonstrating that trust calibra- tion can be achieved through deliberate interactional constraint. In the context of social-emotional learning, non-advisory Socratic dialogue supports reflective engagement while avoiding risks associated with over- reliance and misplaced authority. Rather than treating trust as a quantity to be maximized, CASELy aligns with contemporary perspectives that frame trust as a relationship to be calibrated [7]. We argue that conversational AI systems in sensitive educational contexts should prioritize appropriate reliance over perceived intelligence or helpfulness. By combining quantitative and qualitative evaluation within a controlled framework, CASELy offers a scalable, replicable, and ethically grounded model for AI-mediated SEL interventions that support reflective learning without compromising autonomy or privacy. Future iterations will evaluate CASELy with larger samples, explore longitudinal trust dynamics, and examine transferability to other sensitive domains requiring autonomy-preserving support. References [1] C. S. Conley. Sel in higher education. In Handbook of Social and Emotional Learning: Research and Practice, pages 197–212. Guilford Press, 2015. [2] C. Conley and D. Donahue-Keegan. Sel for well-being and equity in higher education. 2024. [3] A. Simion. The impact of socio-emotional learning (sel) on academic evaluation in higher education. Educatia 21, 24(1):109–117, 2023. 5 Table 1: Mapping Participant Experiences to Trust Calibration Principles Trust Calibra- tion Principle CASELy Design Feature Participant EvidenceTheory Alignment Appropriate Re- liance [7] Non-advisory dia- logue; explicit re- fusal to provide recommendations “At the end [the bot] said to me, ‘your decision, I respect that.’ I appreciate that” (P2) Prevents over-reliance by maintaining user asdecision-maker rather than delegating authority to system Transparency ThroughCon- straints [7, 8] Visible grounding in user input; one question per turn “It showed that it really ana- lyzes my answers and gives me responses accordingly” (P1); “It’s not just randomly throwing answers” (P2) Makes system capa- bilities and limitations continuouslyvisible throughinteraction patternsratherthan post-hoc explanations Avoiding Confi- dence Inflation [8] Honest,non- flatteringre- sponses;no evaluative praise “If you think correctly, or don’t think correctly, he tells you that you think correctly” (P2) Avoids artificial confi- dence signals that can improve perceived trust while degrading cali- bration accuracy Preserving User Agency [9] Questionsrather than advice; mir- rors user language “An experience rather than just learning. . . you’re talk- ing with someone real” (P1) Maintains learner re- sponsibility and active engagement Emotional Safety for Hon- estInteraction [10] Anonymity; non- judgmental ques- tioning “Sometimes people are em- barrassed or get confused when there’s someone talk- ing and sitting in front of them” (P1) Reducessocialde- sirabilitybiasand enables authentic self- disclosurenecessary for trust assessment Distinguishing Systemfrom Human Author- ity [9] Acknowledges limitations; refuses interpreta- tion “It’s a bot, but you’re talking with someone. . . real” (P1) Users recognize system boundaries while still findinginteraction valuable—appropriate trust rather than anthro- pomorphic over-trust Functional Transparency [7] Structured, orga- nizedresponses grounded in user input “The questions he asked. . . it was very organized.It showed that he really ana- lyzes my answers” (P1) System behavior itself communicates capabil- ities,creating emer- gent transparency with- out requiring model in- terpretability Comparative Calibration [7, 8] Interactiveen- gagementvs. passive learning “The experience of learning through a bot is better, be- cause it’s interactive” (P3); “I can ask questions, and it answers me” (P4) Usersappropriately valuesystemfor itsactualstrengths (interactivity,person- alization) rather than perceived omniscience 6 [4] Y. Takizawa, Y. Matsumoto, and Y. Ishimoto. Effectiveness of universal social-emotional learning programs for japanese higher education students: A meta-analytic review. Health Open Research, 4:4, 2024. [5] A. A. Ahmad Feisal and S. K. S. Singh. Social and emotional learning (sel) awareness of undergradu- ates in malaysia. International Journal of Modern Education, 7(25):633–646, 2025. [6] V. L. DeSensi. Embedding social and emotional learning (sel) design to support college student well- being and learning outcomes. Journal of the National Organization for Student Success, 1(1), 2024. [7] M. Wischnewski, N. Krämer, and E. Müller. Measuring and understanding trust calibrations for au- tomated systems: A survey of state-of-the-art and future directions. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, pages 1–16, 2023. [8] Y. Zhang, Q. V. Liao, and R. K. Bellamy. Effect of confidence and explanation on accuracy and trust calibration in ai-assisted decision making. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, pages 295–305, 2020. [9] M. Wölfel, M. B. Shirzad, A. Reich, and K. Anderer. Knowledge-based and generative-ai-driven pedagogical conversational agents: A comparative study of grice’s cooperative principles and trust. Big Data and Cognitive Computing, 8(1):2, 2023. [10] J. van Rest, D. Boonstra, M. Everts, M. van Rijn, and R. van Paassen. Designing privacy-by-design. In Annual Privacy Forum, pages 55–72. Springer Berlin Heidelberg, 2012. [11] M. Schrepp, A. Hinderks, and J. Thomaschewski. Design and evaluation of a short version of the user experience questionnaire (ueq-s). International Journal of Interactive Multimedia and Artificial Intelligence, 4(6):103–108, 2017. 7

Instant research discovery