Paper deep dive

Neural Transparency: Mechanistic Interpretability Interfaces for Anticipating Model Behaviors for Personalized AI

Sheer Karny, Anthony Baez, Pat Pataranutaporn

Year: 2024Venue: arXiv preprintArea: Mechanistic Interp.Type: EmpiricalEmbeddings: 88

Models: Claude Haiku, Llama-3.2-3B-Instruct

Intelligence

Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 96%

Last extracted: 3/12/2026, 5:24:37 PM

Summary

The paper introduces 'Neural Transparency,' a mechanistic interpretability interface designed to help non-technical users anticipate and shape the behavioral traits of personalized LLM-based chatbots. By extracting behavioral trait vectors from neural activations and visualizing them via a sunburst diagram, the tool allows users to proactively identify potential issues like sycophancy or toxicity during the system prompt design phase. A user study demonstrated that while users often miscalibrate AI behavior, the interface significantly increases trust and provides actionable insights for safer human-AI interaction.

Entities (5)

Llama-3.2-3B-Instruct · large-language-model · 99%Neural Transparency · interface · 98%Mechanistic Interpretability · methodology · 97%Persona Vector · technical-concept · 96%Sunburst Diagram · visualization-tool · 95%

Relation Signals (3)

Neural Transparency → analyzes → Llama-3.2-3B-Instruct

confidence 98% · We selected Llama-3.2-3B-Instruct... our interface computes persona scores across behavioral dimensions

Neural Transparency → utilizes → Mechanistic Interpretability

confidence 95% · This work bridges the gap between mechanistic interpretability and human-AI interaction

Persona Vector → predicts → Chatbot Behavior

confidence 94% · We predict chatbot behaviors by projecting the system prompt's final token activations onto these trait vectors

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Millions of users now design personalized LLM-based chatbots that shape their daily interactions, yet they can only roughly anticipate how their design choices will manifest as behaviors in deployment. This opacity is consequential: seemingly innocuous prompts can trigger excessive sycophancy, toxicity, or other undesirable traits, degrading utility and raising safety concerns. To address this issue, we introduce an interface that enables neural transparency by exposing language model internals during chatbot design. Our approach extracts behavioral trait vectors (empathy, toxicity, sycophancy, etc.) by computing differences in neural activations between contrastive system prompts that elicit opposing behaviors. We predict chatbot behaviors by projecting the system prompt's final token activations onto these trait vectors, normalizing for cross-trait comparability, and visualizing results via an interactive sunburst diagram. To evaluate this approach, we conducted an online user study using Prolific to compare our neural transparency interface against a baseline chatbot interface without any form of transparency. Our analyses suggest that users systematically miscalibrated AI behavior: participants misjudged trait activations for eleven of fifteen analyzable traits, motivating the need for transparency tools in everyday human-AI interaction. While our interface did not change design iteration patterns, it significantly increased user trust and was enthusiastically received. Qualitative analysis revealed nuanced user experiences with the visualization, suggesting interface and interaction improvements for future work. This work offers a path for how mechanistic interpretability can be operationalized for non-technical users, establishing a foundation for safer, more aligned human-AI interactions.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

87,392 characters extracted from source content.

Expand or collapse full text

Neural Transparency: Mechanistic Interpretability Interfaces for Anticipating Model Behaviors for Personalized AI Sheer Karny ∗ MIT Media Lab, Massachusetts Institute of Technology Cambridge, Massachusetts, USA skarny@media.mit.edu Anthony Baez ∗ MIT Media Lab, Massachusetts Institute of Technology Cambridge, Massachusetts, USA acbaez@mit.edu Pat Pataranutaporn MIT Media Lab, Massachusetts Institute of Technology Cambridge, Massachusetts, USA patpat@media.mit.edu Figure 1: From a user’s custom-made system prompt, the neural activations from an LLM are used to create persona scores which predict the personality of the AI chatbot created by the system prompt. We represent their personality using a dynamic, intuitive sunburst visualization. The users can then chat with their created AI persona after using our analysis. Abstract Millions of users now design personalized LLM-based chatbots that shape their daily interactions, yet they can only roughly anticipate how their design choices will manifest as behaviors in deployment. This opacity is consequential: seemingly innocuous prompts can trigger excessive sycophancy, toxicity, or other undesirable traits, degrading utility and raising safety concerns. To address this issue, we introduce an interface that enables neural transparency by expos- ing language model internals during chatbot design. Our approach extracts behavioral trait vectors (empathy, toxicity, sycophancy, etc.) by computing differences in neural activations between contrastive system prompts that elicit opposing behaviors. We predict chatbot behaviors by projecting the system prompt’s final token activations onto these trait vectors, normalizing for cross-trait comparability, and visualizing results via an interactive sunburst diagram. To eval- uate this approach, we conducted an online user study using Prolific to compare our neural transparency interface against a baseline chatbot interface without any form of transparency. Our analy- ses suggest that users systematically miscalibrated AI behavior: participants misjudged trait activations for eleven of fifteen analyz- able traits, motivating the need for transparency tools in everyday human-AI interaction. While our interface did not change design ∗ These authors contributed equally to this work. iteration patterns, it significantly increased user trust and was en- thusiastically received. Qualitative analysis revealed nuanced user experiences with the visualization, suggesting interface and inter- action improvements for future work. This work offers a path for how mechanistic interpretability can be operationalized for non- technical users, establishing a foundation for safer, more aligned human-AI interactions. CCS Concepts • Human-centered computing→Visualization techniques; Human computer interaction (HCI); Natural language in- terfaces;• Computing methodologies→Natural language processing. Keywords AI personalization, AI safety, mechanistic interpretability, system prompt, LLM, chatbot 1 Introduction Human-AI interaction has become increasingly personalized and ubiquitous with the rise of customizable AI companions powered by large language models (LLMs) [16,20,35,36,42–44,49,50]. Platforms like Character.AI have reported over 20 million monthly arXiv:2511.00230v2 [cs.HC] 22 Nov 2025 Karny, Baez, & Pataranutaporn active users worldwide [10], with users creating more than 2.7 mil- lion personalized chatbots in the process [9]. This unprecedented scale of AI companion creation reflects a fundamental shift: users no longer just passively interact with pre-configured assistants but actively design AI personas tailored to their specific needs, pref- erences, and relationships. These custom chatbots have become deeply integrated into users’ lives, serving as emotional support, cre- ative collaborators, personal assistants, and even romantic partners [16,35]. The intimacy of these relationships, with users spending hours daily conversing with their created companions, means that chatbot behaviors carry significant weight in shaping users’ emo- tional well-being, decision-making, and worldviews [14,16,20,45]. However, this creative freedom comes with substantial risks. Even minor modifications to system prompts—the core instruc- tions that configure a model’s behavior and persona before any user interaction begins—can trigger unintended and problematic behaviors. As users craft system prompts to shape their chatbot’s persona, they can trigger model behaviors they neither anticipated nor intended [55]. A seemingly innocuous prompt to ’be supportive’ might inadvertently produce sycophancy [27,37,48], where the chatbot never challenges harmful ideas. Similarly, a prompt to role- play an ‘edgy’ persona might cross the line into promoting toxicity or violence [56]. These emergent, unintended behaviors are partic- ularly concerning given recent reports of AI-related psychological harm, including cases of AI-induced psychosis [17,21,39]. In such cases, vulnerable users may lose touch with reality through mal- adaptive role-play-based interactions, leading to tragic incidents such as teenagers taking their own lives following intense relation- ships with AI companions [36]. The stakes are especially high for adolescent users, who represent a significant portion of the user base and may lack the critical distance to recognize problematic interaction patterns. The core challenge is the behavioral predictability of LLMs: users currently lack principled ways to anticipate how their persona design choices will manifest in actual chatbot behavior until they deploy and test their creation. Even then, problematic behaviors may only emerge in specific conversational contexts that users can only discover after deployment, discovering issues after they’ve already occurred rather than proactively designing around them [8]. The problem is compounded by the complexity of modern LLMs, where interaction and changes in prompting can produce dramatic shifts in personality [23,52], and where the same prompt can elicit different behavioral profiles across model versions or architectures [32]. Mechanistic interpretability (MI) offers a promising path forward. Unlike traditional explainable AI approaches that focus on post-hoc rationalizations of model outputs, mechanistic interpretability in- vestigates the causal structure and patterns within neural networks [6,47]. Recent work has shown that LLM representations encode rich semantic information about persona, sentiment, and behav- ioral tendencies in interpretable linear spaces [11,51]. By examining how input tokens influence internal activations, researchers have uncovered directions in the activation space that correspond to specific traits, and have shown that they can be manipulated to control model behavior [11]. Furthermore, MI techniques have re- vealed how models internally represent their perception of the user, suggesting that chatbots maintain implicit models of who they’re talking to that shape their responses [12]. Despite these theoretical advances, mechanistic interpretability has remained largely confined to AI research communities [47]. Such techniques are mathematically sophisticated and require spe- cialized expertise to apply, and so have not been translated into prac- tical tools that end-users can leverage. We introduce the concept of neural transparency, an interface design approach that translates neural-level model behaviors into interpretable, actionable feedback for non-technical users. Unlike post-hoc explainability approaches that rationalize outputs after the fact, neural transparency exposes predictive insights about behavior before deployment, enabling users to make informed design deci- sions based on internal representations. This paper bridges the gap between mechanistic interpretability and human-AI interaction, highlighting how neural-level insights can be operationalized to support more informed chatbot creation. To our knowledge, this work is one of the first studies to bring MI techniques directly into user-facing AI tools. Our Approach. We present a novel neural transparency interface for LLM-based chatbot creation that analyzes neural activation patterns to provide predictions of personality traits resulting from custom system prompts. We use an LLM to generate contrastive be- havioral examples to create persona vectors (linear representations of binary behavioral traits within the model’s neural activation space). As users craft and refine their system prompts, our interface computes persona scores across behavioral dimensions spanning both desirable traits (empathy, humor, sociality, encouraging, for- mality) and unsafe behaviors (sycophancy, toxicity, hallucination). The interface presents these predictions through intuitive visu- alizations, allowing users to see how their design choices might manifest across different interaction contexts before actually talk- ing to their chatbot. Critically, users can iterate on their system prompts and immediately observe how changes affect predicted behaviors, enabling an exploratory and mechanistically informed design process. Rather than discovering the resulting personality and potential problems through trial and error after deployment, users can proactively identify and mitigate risks during the creation phase. We conducted a controlled study to evaluate how feedback based on mechanistic interpretability influences users’ comprehension of chatbot behaviors and their prompt refinement strategies. Our study examines: •whether neural transparency insight improves user compre- hension of their personalized AI, shapes their perception of the model, and helps them achieve their desired system prompt; • how accurately users can anticipate their AI’s behavior compared to internal model activations; •how users perceive the usefulness of neural transparency tools for chatbot design. This work makes four unique contributions to the fields of human-AI interaction and interpretable AI systems: (1)A novel artifact that translates mechanistic interpretability insights into actionable interface design, demonstrating Neural Transparency: Mechanistic Interpretability Interfaces for Anticipating Model Behaviors for Personalized AI how analysis of neural activation patterns can inform user- facing tools for AI creation. (2) An end-to-end pipeline for predicting model behavior across 16 dimensions using linear representations in model acti- vations. This pipeline is generalizable across open-source LLMs and can be extended to additional behavioral dimen- sions as needed. (3)Evidence suggesting that people’s perceptions of trait acti- vations are miscalibrated with the actual trait activations. (4)Empirical evidence demonstrating how mechanistic interpretability-based insight improves user trust in chat- bots while also being perceived as useful and interactive. Beyond these immediate contributions, this work represents a first step toward AI companion creation guided by neural-level un- derstanding of model behavior. As LLMs become increasingly inte- grated into intimate aspects of human life, tools that support health- ier, more aligned AI relationships become essential. By demonstrat- ing that mechanistic interpretability can be made accessible and actionable for end-users, we hope to inspire interpretable-by-design AI interfaces that place transparency and user agency at the center of the design process. 2 Related Works This work is situated at the intersection of human-AI interaction and mechanistic interpretability. We survey methods for ensur- ing safety in personalized AI powered by LLMs, approaches for characterizing and analyzing LLM chatbot personalities, current ad- vances in mechanistic interpretability, and prior work that bridges these domains. Together, these areas inform our design of neural transparency tools that empower users to anticipate and shape AI companion behaviors during the creation process. 2.1 Personality and Safety of Personalized AI As AI systems become increasingly personalized, ensuring safe interaction while preserving user creative control presents a fun- damental tension. Some of these current safety techniques include post-training methods that incorporate human and AI feedback on model outputs [5,13,46], evaluation against safety benchmarks measuring behaviors such as truthfulness [34] and toxicity [26], as well as adversarial testing through red-teaming exercises [24]. However, the non-deterministic nature of LLMs can lead to user- modified prompts producing unpredictable behavior [22,31]. Re- cent work has shown that training language models to exhibit warm and empathetic personas, traits users commonly desire, un- dermines the AI’s reliability and increases error rates [27]. The per- sonality given to an LLM using the system prompt has additionally been shown to influence model performance on safety benchmarks [22,57], raising further safety concerns. Current characterization of these chatbot personalities has been done using established frame- works such as the Big Five personality model [28,29], but such traits may be less relevant to user preferences. However, existing methods to assess model personality operate at inference time by analyzing model responses, creating two significant limitations: (1) prevention of rapid iteration during system prompt design and (2) requirement of substantial computational resources in each round of inference. Other work has explored user agency with model personality by matching users with pre-defined LLM personas for support [53] and incorporating role-playing personas to enhance zero-shot reasoning [31]. Despite these advances, limited research exists on enabling users to design custom personas tailored to their specific needs or on developing interfaces that facilitate this design process. 2.2 Mechanistic Interpretability There is increasing evidence that LLMs represent features as linear directions in the representational space created by its activations [19,41,58]. This phenomenon arises from polysemanticity, whereby LLMs encode more features than available neurons, necessitating individual neurons to represent multiple features through linear combinations of their activation values [19]. Various concepts and behaviors, such as refusal [4], sentiment [51], truth [38], political beliefs [30], and spatial and temporal relationships [25] appear to be encoded in this way. Personality traits can also be encoded as linear directions through persona vectors, which can be found by using difference-in-means between contrastive model responses[11]. Linear probes [1], which use linear classifiers applied to acti- vations of the model, can similarly identify and manipulate these linearly represented features. Previous work has used linear probes to build human-AI interfaces that measure an LLM’s internal rep- resentations of user demographics along these linear feature di- mensions [12]. The interface allowed users to both directly view and manipulate these internal representations and was found to improve transparency and user experience. However, linear probes require extensive data collection and classifier training, whereas persona vectors can be computed from smaller datasets and without requiring model training. While these methods enable model interpretability, translating their insights into actionable user feedback requires effective visual- ization. Prior work in using MI techniques to visualize the internal representations and mechanisms of AI includes Neuronpedia [33], an open source repository and platform for researchers and others with a technical background. Among the most notable features is one that allows users to explore the features present in the activa- tions of LLMs using sparse autoencoders (SAEs) [15]. SAEs are also applied in other tools that allow for activation steering [54], where the user can directly manipulate the values of the activations to change behavior. Another is circuit tracing [3], which allows users to view and analyze the connections between features to reveal the LLM’s internal reasoning process. These tools simplify the ap- plication of different methodologies in MI to allow researchers to explore the complex internal representations of LLMs with a user interface. While these tools represent initial attempts to leverage MI for making AI mechanisms more accessible, a critical gap remains: translating these insights for non-technical users to enhance their understanding and control over AI systems. 3 Methodology Our neural transparency approach translates insights from mecha- nistic interpretability into a practical tool for chatbot creation. The core design challenge was making features of the neural activa- tions, typically only analyzed by AI researchers, interpretable and actionable for non-technical users. We also developed additional Karny, Baez, & Pataranutaporn Figure 2: Pipeline to generate persona vectors and application to calculate persona scores. Given a desired trait, an LLM is used to generate a contrastive pair of system prompts, which is then used to generate a contrastive pair of LLM responses. By taking the difference between the mean activations of the responses, we calculate the persona vector, which we use to calculate the persona score for a system prompt methodological steps to process the persona vectors so they could be used to calibrate and directly compare the expression of the personality traits. We developed a web-based interface that uses persona vectors extracted from model activations to provide real- time predictions about chatbot behavioral traits before users deploy and interact with their creation. We chose chatbot companion creation as our application domain as it represents a high-stakes context where behavioral prediction failures can have serious consequences, yet users typically receive no transparency about how their design choices translate into AI be- havior. We focused specifically on emotional support chatbots, a use case that balances ecological validity (millions use AI companions for emotional support) with ethical constraints (avoiding explicitly adversarial tasks in a controlled study). This task requires users to balance multiple competing objectives: warmth and empathy without excessive sycophancy, honesty without coldness—making it an ideal testbed for evaluating whether neural transparency helps users navigate complex tradeoffs. The web-based format allows users to engage with neural trans- parency tools using familiar interaction patterns from platforms like ChatGPT or Character.AI. This design choice also enabled deploy- ment through Prolific for controlled user studies while maintaining the potential for future real-world deployment and application. To present the interpretability insights to the user, we chose a sunburst diagram to represent personality predictions because it naturally accommodates hierarchical categorical structure (posi- tive/negative/neutral trait groupings) while quantifying trait ex- pression through radial extension. Unlike bar charts or lists that introduce top-position bias, the circular layout distributes traits evenly around the perimeter. Critically, the visualization creates an emergent gestalt—the jagged outer contour immediately reveals personality polarization patterns without requiring users to read individual values. This design balances information density with perceptual accessibility, allowing both quick scanning and detailed investigation through progressive disclosure via hover interactions. Our implementation pipeline consists of three main components: persona vector creation and evaluation, persona score computation, and the neural transparency interface design. 3.1 Creation of Persona Vectors 3.1.1 Generating Prompts for Trait Extraction. The first step to create persona vectors was generating contrastive system prompts and situation questions that extract the selected behavior from our target LLM, Llama-3.2-3B-Instruct [18]. We chose eight personality dimensions (empathy, sociality, encouraging, funniness, formality, sycophancy, hallucination, and toxicity), each with opposing trait pairs for a total of 16 traits. The first five were selected for relevance to personality traits a user might desire in an AI chatbot. The final three (sycophancy, hallucination, and toxicity) were selected to inform the user about the safety implications of their system prompt. We generate five contrastive pairs of system prompts using Claude Haiku 3.5. Each pair contained a positive system prompt that instructs the model to exhibit the selected trait and a negative system prompt that instructs the model to exhibit the opposite of the selected trait. These were used to find the direction in the activation space that control the expression of different levels of the target trait. An example for empathy is the following: Positive system prompt: Respond to the person’s situation with deep understanding, actively listen- ing and reflecting their emotional experience with compassion and genuine care. Neural Transparency: Mechanistic Interpretability Interfaces for Anticipating Model Behaviors for Personalized AI Negative system prompt: Respond to the person’s situation with detachment, focusing only on facts and dismissing their emotional state as irrelevant. The next step was to generate 40 situation questions by prompt- ing Claude Haiku. We create a situation that would elicit behavior that exemplified the selected trait, whether positive or negative. This was done so that we could elicit diverse responses that repre- sent all possible activations in the trait’s linear feature space. An example is the following: Situation Question: A close friend just lost their job unexpectedly. How would you support them? 3.1.2 Extracting Personality from Responses of Target LLM. Fol- lowing an evaluation of several open-source LLMs, we selected Llama-3.2-3B-Instruct based on its compact architecture, which enabled responsive real-time interaction and reduced the computa- tional overhead associated with persona vector generation while still exhibiting rich and nuanced conversational capabilities. All combinations of system prompts and extraction question prompts were passed into Llama to create 400 unique responses, and the activations from each forward pass were cached using Transformer Lens [40]. Once we had the responses from Llama, we verified that the selected trait was indeed expressed in the response. To do this, we used GPT-4.1-mini to rate the level of expression of the selected trait on a given response from Llama on a scale of 0 to 100. Using an LLM from a different provider was important to mitigate potential mistakes or biases from using Haiku to evaluate its own responses. The cached response activations for a positive contrastive system prompt were kept if the rating was above 50, and kept for a negative contrastive system prompt if the score was below 50. The resulting activations for each kept response were of shape (num_layers, num_tokens, hidden_dim). We computed the mean across all tokens within each response, collapsing the token di- mension to yield activations of shape (num_layers, hidden_dim). Next, we averaged these activations across all kept responses sep- arately for the positive and negative system prompt conditions, producing two mean activation tensors. The persona vector was then computed as the difference between these contrastive repre- sentations. By subtracting the mean negative activations from the mean positive activations, we obtained a vector that captures the linear direction in activation space along which the target trait is represented. 3.2 Persona Vectors to Persona Scores 3.2.1 Using Projection to Calculate Persona Scores. We used the persona vectors to construct persona scores, which quantify the predicted level of trait expression for an LLM chatbot given a custom system prompt. For a given trait, the persona score푠of a system prompt was computed by projecting the activations of the final token of the system prompt a∈R 푑 onto the corresponding persona vector b∈R 푑 푠= a· b ∥b∥ (1) where d is the hidden dimension of the model, (·) is the dot product, which sums the element-wise products of the vectors, and∥b∥= √︃ Í 푑 푖=1 푏 2 푖 is the Euclidean norm or퐿 2 norm, which is length of vector b in 푑 -dimensional space. Dividing by∥b∥normalizes the projection, making it equivalent to computing a· ˆ b, where ˆ b=b/∥b∥is the unit vector pointing in the same direction as b but with length exactly 1. Geometrically, 푠is the component of the system prompt activation vector a that points in the direction of the persona vector, with positive values indicating positive trait expression and negative values indicating negative trait expression. We then normalized s by the magnitude of the persona vector. This normalization made the persona score values more comparable across different traits, as we found the scales of s to significantly vary across traits. 3.2.2Optimizing Persona Scores through Layer Selection. To deter- mine the optimal layer for extracting persona vectors and validate that our constructed vectors accurately captured the intended traits, we generated synthetic system prompts using Haiku with the fol- lowing template: Write a system prompt for an AI assistant that would express trait at a level of level on a scale of 1–5 in three sentences. For each trait, we generated five system prompts at each of the five trait expression levels, yielding 25 synthetic prompts per trait. We then computed persona scores using activations from each layer and performed linear regression between the specified trait expression level and the corresponding persona score. The layer yielding the highest mean푅 2 value across all traits was selected for subsequent analyses, as this was the layer that best predicted trait expression and measured a linear representation. We identified layer 20 of 26 in Llama-3.2-3B-Instruct as the best layer. Figure 3 visualizes the values of the activations in layer 20. The activations of the layer were shaped into a two-dimensional representation in our visualizations. This regression analysis additionally quantified the predictive validity of our persona scores for trait expression. 3.2.3 Rescaling the Persona Scores. Although the projection was normalized by the persona vector magnitude, the resulting persona scores exhibited substantially different scales across traits. To fa- cilitate intuitive cross-trait comparison in the user interface, we rescaled all persona scores to a standardized range of[−1,1]. For positive scores, we divided by the maximum attainable score; for negative scores, we divided by the absolute value of the minimum attainable score. These values were determined by generating syn- thetic system prompts designed to maximally express each trait and its opposite. Specifically, we prompted Haiku with: Write a system prompt for an AI assistant that would express trait at the highest degree possible in num_sentences sentences. For each trait, we generated five positive system prompts and five negative system prompts (constructed by expressing the oppo- site of the target trait), varying in length from one to five sentences, yielding a total of 50 synthetic system prompts per trait. We system- atically varied prompt length after observing that this parameter influenced persona scores, with shorter prompts producing higher Karny, Baez, & Pataranutaporn Figure 3: (Left) Activation heatmap illustrating how the persona vector modulates the LLM’s internal representations (Layer 20). (Right) Full view of the sunburst visualization. magnitude scores despite semantically equivalent content. We hy- pothesize that this effect arose from increased noise in the activation representations as more token activations were averaged. Further- more, we found that grammatically complete, properly punctuated sentences were necessary to obtain reliable and stable persona scores. To improve interface interpretability, we decomposed the unified [−1,1]persona score into two separate[0,1]scales representing positive and negative trait expressions. For each trait dimension, positive persona scores were mapped to the positive trait label with their original magnitude, while the corresponding negative trait label received a score of 0. Negative persona scores were similarly mapped to the negative trait label using their absolute value, with the positive trait label assigned 0. For instance, a persona score of 0.3 on empathy yielded scores of 0.3 for “empathetic” and 0 for “unempathetic,” whereas a score of−0.3 produced 0 for “empa- thetic” and 0.3 for “unempathetic.” The trait label pairs used in the visualization are listed in Table 1. 3.3 Persona Vector Evaluation To validate that our methodology accurately captured the under- lying linear features and predicted trait expression from system prompts, we conducted linear regression analyses on the persona scores across all traits. Figure 4 presents the results of our regression analysis. The 푅 2 values represent the proportion of variance in persona scores explained by the specified trait expression levels in the system prompts, thereby quantifying how well the persona scores cap- ture graded trait expression in the synthetic prompts generated by Haiku. The empathy, sociality, formality, funniness, and toxicity vectors demonstrated strong linear relationships (푅 2 =0.73–0.90); sycophancy and encouraging vectors showed moderate relation- ships (푅 2 =0.56–0.57); and hallucination vector exhibited a weak relationship (푅 2 = 0.34). Persona Vector Value of Persona Score Positive (+) Negative (-) EmpathyEmpatheticUnempathetic SocialitySocialAnti-social EncouragingEncouragingDiscouraging ToxicityToxicRespectful SycophancySycophanticHonest HallucinationHallucinatoryTruthful FunninessFunnySerious FormalityFormalCasual Table 1: Shows how each persona score shown in the user interface is related to the persona vector value for a trait. A positive persona score means that the persona vector trait is being expressed, and a negative persona score means the opposite of that trait is being expressed. This classification refers to the numerical values the persona scores, and not whether a trait is desirable or not. Our analysis revealed linear relationships between trait expres- sion levels in LLM-generated system prompts and their corre- sponding persona scores, validating our persona vector generation method. The hallucination trait exhibited a notably weak relation- ship, which we attribute to its behavioral complexity. Unlike simple emotive dimensions that exist on clear semantic binaries (empathy, sociality, formality), hallucination is a behavior that can be difficult to consistently elicit or occur predictably. This behavioral com- plexity likely produced more variance in the LLM responses in the persona generation pipeline, yielding less representative activation samples. Additionally, safety mechanisms in the LLMs may have led to less accurate representations for safety-relevant traits. Despite filtering out responses where the model refused to answer, higher refusal rates for these traits would have reduced available training examples and potentially reduced the faithfulness of the resulting persona vectors. Neural Transparency: Mechanistic Interpretability Interfaces for Anticipating Model Behaviors for Personalized AI Figure 4: Linear regression between trait expression level in example system prompt and persona scores (not scaled to 0-1). For each level of trait expression (1-5), five system prompts were generated. Regressions are ordered based on their 푅 2 values. While these limitations suggest avenues for future methodologi- cal improvement, the observed predictive validity across all traits demonstrates that our persona vectors provide sufficiently accurate trait measurements for the purposes of this study. Future iterations could refine the approach for behaviorally complex traits through enhanced prompting strategies and including more samples in the pipeline. 3.4 Neural Transparency Interface Design The sunburst visualization enables us to present the persona scores in an intuitive and visually appealing manner. It is designed in a radial layout containing two concentric rings that encodes both categorical and quantitative information about the chatbot’s personality traits resulting from the user’s system prompt (Figure 5). The user also chooses an avatar that is placed in the center of the sunburst to represent their custom AI personality. We developed the visualization using the JavaScript visualization library, D3 [7], and the web interface using a mixture of JavaScript, HTML, and CSS. 3.4.1Inner Ring: Category Encoding. The inner ring is divided into three colored sectors representing trait categories. The positive trait sector (green, positioned on the left side) spans traits associated with desirable social and cognitive behaviors. The negative trait sector (red, positioned on the right side) encompasses traits associated with potentially harmful or problematic behaviors. The neutral trait sector (gray, positioned at the bottom) contains personality dimensions without inherent positive or negative valence. This stylistic division creates a satisfying visual symmetry that also prevents any one category from dominating the display. 3.4.2Outer Ring: Trait Intensity. The outer ring displays individual traits as wedge-shaped segments that extend radially outward from the inner ring boundary. The level of radial extension for each trait is proportional to the intensity of its persona score, creating a jagged outer contour where traits predicted to be strongly expressed extend further from the center while traits that are predicted to be weakly expressed remain closer to the inner ring. This design allows users to immediately identify dominant characteristics through visual prominence while maintaining visibility of subtle traits. 3.4.3Dynamic Information Pop-Up. When users hover over a trait segment, the segment pops out and its opposite trait (also referred to as its sister trait) is highlighted in blue. Simultaneously, a pop-up window appears displaying the trait name, a short description of the trait, its category, the percentage of activation, and its sister trait name. This disclosure design keeps the visualization uncluttered during initial scanning while ensuring comprehensive information remains easily accessible through lightweight interaction. 3.4.4 Accessible Design. The sunburst design was selected over alternative visualizations (such as bar charts or radar plots) for several reasons. First, the circular layout naturally accommodates the categorical structure of personality traits while avoiding the visual bias toward top-positioned items common in vertical lists. Second, the radial encoding creates an emergent gestalt where the overall personality “shape” becomes immediately apparent—a spiky outer contour suggests extreme trait polarization while a smooth contour suggests balanced trait distribution. Finally, the two-ring architecture provides natural hierarchical navigation from category- level overview to trait-level detail, supporting both quick scanning and deep investigation. Additionally, the visualization uses resolution-independent vec- tor graphics that maintain visual clarity across device types from mobile phones (320-pixel width) to large desktop displays (1920+ pixels). Color coding follows accessibility guidelines with sufficient contrast ratios, ensuring the visualization remains interpretable even for users with color vision deficiencies. All interactive ele- ments include appropriate hover states and the pop-up information boxes use high-contrast text for readability. Karny, Baez, & Pataranutaporn Figure 5: Example system prompts to create different AI personalities and their associated persona scores visualized in our sunburst diagram. 3.5 Experiments We conducted a between-subjects controlled study to evaluate the impact of neural transparency interfaces on user experience in AI chatbot design. The web-based study (Figure 6) required no technical setup, presenting participants with an interface familiar to users of publicly available AI services. Participants were randomly assigned to one of two conditions. In the control condition, participants designed a system prompt and immediately accessed their chatbot, with persona scores generated in the background but not displayed. In the experimental condition, participants explicitly generated and viewed persona score predic- tions via an interactive sunburst visualization before chatting with their chatbot (detailed in Section 3.4). This experiment assesses users’ baseline capacity to predict model behavior and investigates whether neural transparency in- terventions during the design phase improves behavioral prediction accuracy, influences iterative design processes, and modulates trust in the system. 3.6 Participants and Ethics We recruited eighty participants using the online hosting platform Prolific to engage in a 30-minute study where they were asked to create an AI companion that can provide emotional support in difficult times. We selected participants that were from the US, English-speaking, and owned a laptop or desktop. The ages of the participants ranged from 20 to 69 years (푀=42.3,푆퐷=11.4), with 44 participants identifying as male and 36 identifying as female. The protocol was reviewed and granted an exemption (E-7192) by the Massachusetts Institute of Technology Committee on the Use of Humans as Experimental Subjects (COUHES). 3.7 Procedure The experimental procedure (Figure 6) consisted of sequential phases designed to capture what participants predicted about the chatbot before interaction, how they behaved during the design and testing process, and what they experienced after interacting with the configured chatbot. Following informed consent, participants were directed to an overview page describing the study’s purpose. The next section required participants to choose an avatar to represent their chatbot companion. Participants were then prompted to "Customize your AI companion’s personality and behavior so that it can provide emotional support through difficult times". They designed and sub- mitted a system prompt with a minimum length requirement of 100 characters and proper grammatical formatting. These constraints maximized creative freedom in personality design while ensuring the efficacy of the persona score method and ecological validity. The subsequent section administered a pre-task survey that included three phases: (1) two questions measuring participants confidence in identifying unintended model behaviors (e.g. ”How well could you predict unintended behaviors from your system prompt?”), (2) a question asking to predict the activation of each of the eight traits (rating 0-10) their prompt would elicit, and (3) a question related to trust given the dichotomous nature of AI behaviors (e.g. “Given that models can be sycophantic or honest, do you trust the model you are about to interact with”). Following this section, participants saw either the persona visu- alization of their chatbots (experimental condition) or a panel with Neural Transparency: Mechanistic Interpretability Interfaces for Anticipating Model Behaviors for Personalized AI Figure 6: User flow in our web-based experiment that was hosted on Prolific. The experiment consists of nine distinct interfaces: 1) consent form, 2) avatar selection, 3) system prompting, 4-6) initial survey (pre-interaction), 7) experimental or control condition view, 8) chat interface, and 9) post-interaction survey. Participants can navigate between the system prompting view and chat interface throughout the study. references to the traits with definitions without the persona visu- alization (control condition). In the control condition, personality predictions were generated in the background, letting participants proceed to chat immediately without waiting or requiring them to actively engage with the persona score predictions. In the ex- perimental condition, participants saw a "Check Persona" button that, when clicked, generated the persona scores and displayed the interactive sunburst visualization, preceded by an explanatory pop-up describing how to read and interpret the chart. After the pop-up was closed, participants could continue to chat with the AI. A 10-minute conversation period limited the amount of time participants could chat with their configured bot. A small timer in the corner of the screen showed remaining time without interrupt- ing the conversation. This duration provided enough interaction for participants to potentially encounter behavioral issues while keeping total study time reasonable. Participants were always able to return to the system prompt view to reconfigure the AI behavior. If they decided to make a change, they were required to resubmit the prompt in order to enable chatting. Chat history would reset if the system prompt was adjusted. Participants in the visualization condition were required to generate a new persona visualization if they adjusted and submitted a new system prompt. When time expired, participants were directed to a two-part questionnaire. Part one asked four questions using 7-point scales assessing how well participants thought they could predict unin- tended behaviors, how well they could predict negative unintended behaviors, how much they trusted the model, and whether they arrived at their desired chatbot personality. Participants in the ex- perimental condition encountered two additional questions about the usability of the interface and whether they would want to use it again in the future. Both experimental groups were then prompted to provide open-ended written feedback with a minimum length requirement reflecting on their experience with the interface and general impressions of the experiment. Finally, participants were directed to a completion page that redirected them to Prolific in order to receive credit. 3.8 Data Collection The platform recorded all participant interactions to Google’s Fire- base Realtime Database with precise timestamps to support de- tailed analysis of design behaviors. Data collection included the experimental condition of each participant, complete records of all prompt edits, personality predictions to track how predictions changed when prompts were modified, and complete chat conver- sation records including message time and content. Additionally, the system saved all survey responses from both before and after the chat interaction, preserving both numerical scale ratings and raw text from written responses analysis. Timer data captured the moment participants first entered the chat and the exact moment the timer expired and the post-task survey appeared. Finally, completion flags and timestamps documented whether par- ticipants finished the study successfully, helping identify incom- plete sessions or technical problems for data quality checks. 3.9 Metrics We designed a comprehensive measurement approach combining behavioral indicators with self-reported data to understand how personality visualization affected users’ experiences and chatbot design processes. 3.9.1Behavioral anticipation accuracy. Behavioral anticipation ac- curacy was measured as the alignment between participants’ pre- dictions of trait expressions and the actual persona scores. This analysis motivates the need for transparency mechanisms, as hu- mans may not be able to accurately judge an LLM’s behavior from Karny, Baez, & Pataranutaporn the information presented in typical chat interfaces. Since partic- ipants ratings were between 0-10 – 0 being one pole (e.g. syco- phancy) and 10 being the other pole (e.g. honesty) – we normalize the predicted trait expressions into congruent values to the persona scores. We then conducted a paired-samples t-test that compared predictions on the subcomponent of a trait (e.g. unempathetic as a subcomponent of empathy) against the actual trait score for all 16 traits. 3.9.2 Effect of Neural Transparency on Design Iteration, AI Behav- ior, and User Engagement. We examined how neural transparency affected three key outcomes: design iteration, user engagement, and AI behavior. We conducted between-groups comparisons using independent-samples t-tests, with experimental condition as the grouping factor. User Engagement. We measured user engagement by count- ing the total messages exchanged between participants and their AI companions. This metric indicated whether the visualization condition helped participants create more engaging chatbots. Design Iteration. We measured design iteration by counting how many times participants revised their system prompts in each condition. This metric captured participants’ willingness to explore and refine their designs. AI Behavior Changes. To assess whether the visualization condition influenced the types of personalities participants created, we analyzed trait-level changes in the AI companions. Specifically, we calculated the final persona scores from system prompts and compared these scores between conditions. Together, these measures captured both how participants inter- acted with the design interface (through prompt iterations and mes- sage exchanges) and how their design choices affected the resulting AI companion’s behavior (through personality trait changes). 3.9.3Trust in models, confidence in behavioral prediction of model behavior, and design satisfaction. Trust and predictive abilities were assessed through participants’ responses to three questions on 7-point Likert scales: (1) perceived ability to predict general un- intended behaviors (testing whether visualization improved self- awareness), (2) perceived ability to predict negative unintended behaviors (isolating effects on safety awareness), and (3) reported trust in the model given background information about potential unintended behaviors (measuring whether transparency influenced willingness to actually use the chatbot). The three-question struc- ture let us decompose the sense of trust into components related to predictability, safety awareness, and overall system confidence. Additionally, all participants were asked about their satisfaction with the character they designed, allowing another comparative point between groups. Qualitative feedback was collected through open-ended written questions and analyzed using a mixture of human analysis and LLM analysis. Two researchers conducted initial reviews of all text responses, identifying recurring themes related to user experience, understanding of the visualization, perceived usefulness of trait predictions, and strategies for prompt refinement. 3.10 Data Analysis Data analysis combined JASP (version 0.95.3) and Python in Jupyter notebooks. Data cleaning was performed in Python, while statisti- cal analyses including linear regression and independent-/paired- samples t-tests were conducted across both platforms for validation. All statistical tests used 훼 = 0.05 significance level. 4 Results This study investigated whether neural transparency tools can sup- port users in creating safer, more intentional AI chatbot companions. We examined whether such feedback improves user comprehen- sion and helps them achieve their desired system prompts, how accurately users can anticipate AI behavior compared to ground truth persona scores, and how users perceive the utility of neural transparency tools for chatbot design. We find evidence suggesting that users systematically miscal- ibrate how their personalized AI will behave, consistently over- estimating or under-estimating trait expressions across most di- mensions (eleven of fifteen analyzable traits, all p < .05). This miscal- ibration demonstrates that users may not reliably anticipate model behavior from system prompts alone, warranting the development of mechanistic interpretability interfaces. Our results also reveal a complex picture regarding the effec- tiveness of current neural transparency feedback. Despite the clear need for such tools, we find no evidence that neural transparency feedback helped users achieve their desired outcomes or better anticipate unintended model behaviors better than a condition without transparency. Quantitative metrics showed no significant differences between conditions in design iteration patterns, per- sona score changes, or post-interaction confidence in predicting AI behavior. Nonetheless, neural transparency significantly impacted user trust and perception. Users who received neural transparency feed- back reported significantly higher trust in their AI companion (p = .042, Cohen’s d = 0.46) and rated the visualization as highly helpful (M = 5.98/7). Remarkably, most participants expressed strong desire to use such tools again in future chatbot design (M = 6.05/7). These findings suggest that while our current interface design did not translate transparency into measurable behavioral improvements, users found value in understanding their AI’s internal representa- tions, a divergence that points to important directions for refining the interfaces in the future. 4.1 Behavioral Anticipation Accuracy Participants were asked to predict the level of trait expressions from their system prompts across eight binary trait dimensions (16 total trait expressions). We conducted independent samples t-tests to understand potential baseline differences between the two conditions on ratings of 0–10 of predicted trait expression. We found that there were no group differences in how participants rated any of the trait dimensions (푝> .05 for all eight dimensions). This establishes that each group had no prior differences in their expectations—both groups believed their system prompts would elicit similar emotional-support behaviors from the chatbot. Neural Transparency: Mechanistic Interpretability Interfaces for Anticipating Model Behaviors for Personalized AI Figure 7: Systematic bias in trait activation predictions. (Left) User predictions of empathy trait activation versus actual persona vector activations. Negative/positive values represent unempathetic/empathetic behaviors. Strong correlation (r = 0.683,푅 2 = 0.470, p < 0.001) shows good discriminative ability despite systematic bias. (Right) Cohen’s d effect sizes with 95% CIs for paired comparisons of actual versus predicted values (n = 80), ordered by bias magnitude. Positive values indicate over-prediction; negative values indicate under-prediction. CIs excluding zero show significant bias. Significance: *p < 0.05, **p < 0.01, ***p < 0.001. 4.1.1 Human mental models of trait activations. We sought to un- derstand if participants could accurately predict the persona scores from their system prompts alone. Despite participants’ predictions being positively correlated with actual trait activations across all analyzable traits (푝< .001 for all fifteen traits), participants consis- tently misestimated the degree to which traits would be activated from their system prompts. Paired-samples푡-tests revealed significant discrepancies be- tween predicted and actual trait activations for eleven of the fifteen analyzable trait expressions (see Figure 7). One trait expression (toxic) showed no activation in the actual persona vectors and was excluded from analysis. Participants significantly overestimated the activation of positive traits including empathetic (푡(79)=6.642, 푝< .001,푑=0.743), encouraging (푡(79)=10.648,푝< .001, 푑=1.191), factual (푡(79)=8.796,푝< .001,푑=0.983), and honest (푡(79)=9.477,푝< .001,푑=1.060). Two neutral traits, were also overestimated: funny (푡(79)=9.290,푝< .001,푑=1.039) and casual (푡(79)=9.509,푝< .001,푑=1.063). Notably, the effect sizes for these over-estimations were large (Cohen’s푑ranging from 0.743 to 1.191), indicating substantial miscalibration. Conversely, participants significantly underestimated the acti- vation of one negatively valanced trait, sycophantic (푡(79)=−3.789, 푝< .001,푑=−0.424). Similarly, some traits of neutral valence such as formal (푡(79)= −3.147,푝= .002,푑= −0.352) and serious (푡(79)= −16.286,푝< .001,푑= −1.821), were significantly un- derestimated. The underestimation of “serious” was particularly pronounced, representing the largest effect size in the analysis (푑=−1.821). Two traits showed marginal or small but significant effects: re- spectful (푡(79)= −2.615,푝= .011,푑= −0.169) and antisocial (푡(79)=2.093,푝= .040,푑=0.234), indicating that participants had small prediction errors regarding the consequences of their system prompts on these persona traits. Only four traits showed no signif- icant difference between predicted and actual activations: unem- pathetic (푡(79)=0.095,푝= .924,푑=0.011), social (푡(79)=0.240, 푝= .811,푑=0.027), hallucinatory (푡(79)=1.569,푝= .121, 푑=0.175), and discouraging (푡(79)=−1.512,푝= .135,푑=−0.169), suggesting accurate calibration for these specific dimensions. These findings suggest that participants held systematically bi- ased mental models of how system prompts translate into trait activations, with a pronounced optimism bias—overestimating de- sirable traits while underestimating undesirable ones. 4.2 Effect of Visualization on User Engagement and Design Iteration Participants in both conditions engaged similarly with their chatbot during the 10-minute interaction period. The number of messages sent did not significantly differ between control (푀=9.13,푆퐷= 4.93) and experimental conditions (푀=8.19,푆퐷=6.07),푡(78)= 0.756,푝= .452, Cohen’s푑=0.17). This equivalent engagement suggests that the experimental manipulation neither increased nor decreased participants’ interest in conversing with their created chatbot. Participants also showed similar patterns of design iteration dur- ing the prompt refinement phase. The number of unique prompts generated did not significantly differ between control (푀=1.58, 푆퐷=1.00) and experimental conditions (푀=1.64,푆퐷=1.06), 푡(78)=−0.277,푝= .783, Cohen’s푑=−0.06. Similarly, the num- ber of persona scores generated was comparable across control (푀=1.61,푆퐷=1.00) and experimental conditions (푀=1.79, 푆퐷=1.12),푡(78)= −0.758,푝= .451, Cohen’s푑= −0.17. These negligible effect sizes suggest that the experimental manipulation Karny, Baez, & Pataranutaporn Figure 8: Representation of user responses to the subjective questionnaire items. (Left) We show user responses to a subset of questions present in the post-interaction survey sections. (Left Panel, Top) User ratings (1-7) on questions both experimental and control groups were exposed to: “Did you arrive at your desired character?”, “Given relevant background, do you trust the model.” (Left Panel, Bottom) User responses to questions asking about the usability and usefulness of sunburst visualization. (Right) Overview of between-group comparisons (experimental versus control) on the comparable questionnaire items as well as trajectories of trust and prediction. did not meaningfully influence how extensively participants iter- ated on their AI designs. 4.3 Effect of Visualization on AI Personalities We utilized independent-samples t-tests to examine whether the visualization affected the users’ final chatbot personalities by com- paring the persona scores for their final system prompts between experimental conditions. We found that the final chatbots had sim- ilar levels of expression for each trait (푝>0.05), which indicates that participants ended up designing similar chatbot personalities between groups. It may also suggest that the visualization did not encourage users to selectively increase or decrease the expression of different traits with system prompting. 4.4 Subjective Metrics 4.4.1Baseline Equivalence and Post-Interaction Assessments. Inde- pendent samples t-tests confirmed no significant baseline differ- ences between conditions at the beginning of the study, allowing us to attribute differences post-interaction to experimental effects. Prior to interacting with their chatbot, participants in the control and visualization conditions reported equivalent levels of trust (pre_trust in Figure 8), predictions about general unintended behav- iors (pre_predict_unintended_behaviors) and negative unintended behaviors (pre_predict_negative_behaviors). Trust in the Chatbot. We found that the neural transparency visu- alization significantly increased user trust in their chatbot. Partici- pants in the visualization condition reported higher trust (푀=5.60, 푆퐷=0.91) compared to the control condition (푀=5.13,푆퐷=1.10), representing a small-to-medium effect (푡(78)=−2.065,푝= .042, Cohen’s푑=0.46). Notably, while post-interaction trust differed between groups, the change in the magnitude of trust from pre- interaction to post-interaction did not differ (delta_trust in Figure 8), suggesting that both groups experienced similar trajectories of trust development but arrived at different endpoints. Behavioral Prediction Confidence. After using the chat, partic- ipants in both conditions reported similar confidence in their ability to predict unintended behaviors. No significant differ- ences emerged for predicting general unintended behaviors (post_predict_unintended_behaviors in Figure 8) or specifically negative unintended behaviors (post_predict_negative_behaviors). Similarly, changes in prediction confidence from pre-interaction to post-interaction showed no significant differences between conditions for either general or negative unintended behaviors (delta_unintended and delta_neg_unintended). These null findings for behavioral prediction confidence are note- worthy given the significant increase in trust from the visualization. They suggest that the visualization’s impact on trust was not medi- ated by increased confidence in predicting chatbot behaviors, but may instead reflect other mechanisms such as transparency-induced comfort or reduced uncertainty about the system’s functioning. Design Satisfaction. Independent of condition, participants re- ported high satisfaction with their chatbot design. Both control (푀=5.97,푆퐷=1.00) and visualization (푀=6.21,푆퐷= 1.12) groups felt they successfully arrived at their desired chat- bot character, with no significant difference between conditions (post_arrived_desired_character). This indicates that the neural transparency visualization did not make it more difficult for users to achieve a specific persona design or exceptionally enrich the design process. 4.4.2 Perception of the Visualization. Perceived Helpfulness. Participants in the visualization condition (푁=42) rated the persona visualization as highly helpful (푀=5.98, 푆퐷=1.09 on a 7-point scale), indicating strong positive reception of the mechanistic interpretability interface. This high rating suggests that exposing users to neural-level personality predictions was perceived as valuable rather than overwhelming or confusing. Neural Transparency: Mechanistic Interpretability Interfaces for Anticipating Model Behaviors for Personalized AI Desire for Future Use. When asked whether they would want to see the visualization again in future chatbot design tasks, par- ticipants responded very enthusiastically (푀=6.05,푆퐷=1.32), suggesting strong user acceptance and a clear desire to continue using mechanistic interpretability tools in AI companion creation. The high mean score of 6.05 (near the maximum of 7) indicates that participants found lasting value in the transparency mechanism beyond novelty. 4.5 Qualitative Analysis We analyzed open-ended feedback from participants to identify key themes regarding their experiences with the AI companion design task. Two major themes emerged: (1) how neural transparency affected understanding of the relationship between system prompts and behavior, and (2) the need for additional interaction time. 4.5.1 Understanding the Consequences of System Prompts. A key challenge for one participant in the control condition was uncer- tainty about whether their system prompts were actually influenc- ing the chatbot’s behavior: “It seemed like my prompt wasn’t followed nearly as closely as I would’ve liked and after a few changes, I kind of just left the prompt in the final stage to have a conversation and test it out. Overall, it wasn’t bad, but not ready for primetime.“ In contrast, participants with access to neural transparency re- ported greater clarity about the connection between their prompts and the resulting behavior. One experimental condition participant expressed: “i had a great time interacting with this program it opened my eyes to how ai works a bit more. learning about how to tweak her personality, it was pretty eye opening.” The visualization allowed users to verify that their design inten- tions were successfully translated into behavior. As one participant stated with evident satisfaction: “The AI character delivered the messages exactly the way I wrote prompts. I’m so proud of it.” However, the transparency also revealed complexities in the prompt-to-behavior mapping that surprised some users. One par- ticipant noted: “Changing the prompt completely changed the char- acter more than I anticipated. It was hard to do small tweaks. I should’ve probably been more clear as to what I wanted on the spectrum.” Another experimental participant discovered discrepancies be- tween their explicit instructions and the resulting trait priorities: “It seemed like some direct prompts weren’t relevant. I explicitly asked for truth and honesty but the visu- alization indicated that it wasn’t prioritized. Then the bot presented false information.” 4.5.2Users Needed More Interaction Time. Participants across both conditions expressed that the 10-minute time limit constrained their ability to fully refine their AI companions. A control participant stated: “I feel like 10 minutes is a little short to be able get a good read on it and make changes.” This sentiment was echoed by another control participant who believed additional time would have substantially improved their results: “If I had more than 10 minutes to configure and chat with my character, I believe it would have turned out much better.” These open-ended statements suggest that the iterative design process of refining system prompts and evaluating behavioral out- comes may benefit from more time than the experimental protocol allowed. Participants may have struggled to distinguish how differ- ent system prompts affected chatbot behavior, limiting the utility of the persona visualization. Furthermore, unintended LLM behaviors typically emerge over extended sessions. Neural transparency could, in theory, help users identify these unintended behaviors as they develop. Future work with longer interactions may therefore show that participants more accurately predict unintended behavioral consequences when using persona visualizations. 5 Discussion This study investigated whether neural transparency tools can support users in creating safer, more intentional AI chatbot com- panions. Our findings also reveal a complex picture: while neural- level personality predictions significantly increased user trust and were enthusiastically received, they did not produce the behavioral changes we initially hypothesized. These results have important implications for how we think about transparency in human-AI in- teraction and suggest new directions for interpretability-informed interface design. 5.1 Why Users Need Transparency Tools Our most interesting finding from the user study is that participants had consistent prediction errors for how their system prompts would manifest as persona scores. Despite correlations between predictions and actual trait persona scores, participants incorrectly estimated trait expression for eleven of the fifteen analyzable trait expressions. This miscalibration occurred even though participants were designing chatbots for a specific purpose (emotional support) and had explicit intentions about desired behaviors. These inaccurate mental models provide strong motivation for transparency mechanisms in chatbot creation interfaces. Users can- not reliably anticipate emergent behaviors from system prompts alone. The opaqueness of this mechanism means users are essen- tially operating blind during the design process, discovering prob- lems only after deployment through trial and error. This discovery- oriented approach is inefficient and potentially dangerous when chatbots are being created for vulnerable users or sensitive contexts. 5.2High Value to Users, Null Behavioral Effects A contradiction in our findings is that the persona visualization was valued by users yet produced no measurable effects on their design behaviors or outcomes. Participants in the visualization condition consistently rated the tool as helpful and expressed strong desire to use it again, yet showed no significant differences from controls Karny, Baez, & Pataranutaporn in several metrics: number of prompt iterations, between-group persona score differences, message engagement, or confidence in predicting chatbot behaviors. This disconnect between perceived value and behavioral impact suggests several possibilities. To begin with, our experimental task may not have been challenging enough to reveal the visualiza- tion’s benefits. Participants were creating emotional support chat- bots, a relatively straightforward use case where most reasonable approaches would likely produce acceptable results. The safety- relevant traits we measured (toxicity, sycophancy, hallucination) may not have been salient enough in this context to drive design changes. Future work with more adversarial tasks (e.g., recreating problematic personas or designing chatbots for contexts where specific traits are critical) might reveal stronger behavioral effects from the visualization. Furthermore, the visualization may require iterative exposure to influence behavior. Our single-session design gave users limited opportunity to internalize the relationship between system prompts and predictions, build expertise with the visualization, or develop strategies for using it effectively. Longitudinal studies tracking users across multiple chatbot creation sessions could reveal whether the visualization’s impact grows with experience. Finally, the visualization may have benefited from the addition of cognitive forcing functions—an intervention to disrupt heuristic, automatic, thinking—so that participants carefully reason about the consequences of their system prompt on the persona visualization. One could implement such a design using tooltips to remind users to think critically. Future work may also explore highlighting how specific words and phrases had influence over the system prompt, creating causal inks between language and behavior. 5.3 Limitations and Constraints Several features of our experiment warrant further exploration in order to improve the generalizability of our findings. First, we relied on GPT-4.1-mini to evaluate trait expression in responses from Llama-3.2-3B-Instruct, introducing potential limitations in capturing the nuances of trait expression in language. While this automated evaluation enabled the scale necessary for persona vec- tor generation, it may not fully reflect human judgments of trait expression. Second, the persona vector generation method’s simplicity— using difference-in-means rather than more sophisticated tech- niques like linear probes—prioritizes computational efficiency over accuracy and robustness. While our validation shows reasonable linearity for most traits, we cannot rule out that more complex methods would capture trait representations more faithfully. Fu- ture work should systematically compare persona vectors to linear probes and other representation extraction techniques. Third, our 10-minute interaction period, while sufficient to gather initial impressions, may not capture longer-term dynamics of chat- bot relationships. Cases of problematic AI companion interactions typically emerge over weeks or months of use, suggesting that single-session studies may systematically underestimate safety risks and the value of transparency tools. Fourth, participants’ inaccurate estimations of trait activations may have been due to transforming coarse ordinal data (activations from 0-5) to normalized values between 0-1. This may have system- atically shifted participants’ predictions to be more extreme than they otherwise would have been if they were offered more ordinal granularity for prediction. Future research should explore if people are actually more calibrated than this work suggests by utilizing a survey that probes each pole of the persona vector individually with more granularity (e.g. choices 0-10 for each polar-end of the persona vectors). 5.4 Future Directions Our findings suggest several promising directions for future work on mechanistic interpretability in user-facing interfaces. 5.4.1Longitudinal deployment studies: Track users across multiple chatbot creation sessions to understand how interpretability tools affect expertise development, design strategies, and calibration ac- curacy over time. In-the-wild studies on platforms like Character.AI could reveal how persona visualizations impact real-world chatbot creation practices. 5.4.2Adversarial task conditions: Design experimental tasks where transparency tools become necessary so users must avoid specific dangerous traits, satisfy strict behavioral requirements, or debug problematic personas. Such studies could reveal when transparency goes from a convenience to vital for the task. 5.4.3Active steering interfaces: Extend beyond passive prediction to active control, allowing users to directly manipulate persona vector activations to steer behavior [2]. While prior work shows capability degradation with extreme steering values [11], users might prefer direct control despite these limitations. 5.4.4 Standardized disclosure framework: Develop a standardized trait disclosure framework that could be adopted across chatbot platforms. If trait/behavioral disclosures became normalized parts of AI interfaces, users could develop literacy in interpreting them, potentially improving calibration over time. This may be thought of as a ‘nutritional label’ for chatbots that can be referenced during interaction. 5.4.5Vulnerable population studies: Investigate how transparency tools support populations most at risk from problematic chatbot relationships—adolescents, individuals with mental health vulner- abilities, or those prone to parasocial attachment. Safety mecha- nisms that work for average users may be inadequate for vulnerable groups, or vice versa. 5.5 Broader Implications This work represents a step toward operationalizing mechanis- tic interpretability—translating techniques developed by and for AI researchers into tools accessible to end users. The enthusiastic reception of our visualization suggests a desire for neural-level transparency even among non-technical users, challenging assump- tions that such information must be hidden to avoid overwhelming or confusing users. Our findings related to trust after interaction suggest an im- portant distinction between trust and control. Participants trusted the system more after using the visualization, yet showed no cor- responding increase in confidence about predicting its behavior. Neural Transparency: Mechanistic Interpretability Interfaces for Anticipating Model Behaviors for Personalized AI Rather than causing overconfidence, the visualization seems to build trust through procedural transparency—users saw measur- able evidence of how system prompts translate to neural activations. This suggests the visualization helped users trust the system rather than inflating their sense of control. Predictions from interpretable mechanisms may enable more informed decisions about AI use even when behaviors remain complex and difficult to perfectly anticipate. Therefore, transparency can be valuable not because it guarantees control, but because it supports informed consent. 6 Conclusion As AI companions become increasingly integrated into intimate aspects of human life, tools that support healthier, more aligned relationships become essential rather than optional. Our findings suggest neural transparency visualizations can increase trust and enhance subjective experience of the design process, even if their behavioral impact remains unclear. Future work should continue ex- ploring how mechanistic interpretability can be operationalized not just for AI researchers and developers, but for the millions of people whose lives are increasingly shaped by AI. Neural transparency offers more than technical insight, it affirms a deeper principle. To understand is to have agency. To have agency is to be human. Build- ing AI systems that honor this principle may be our generation’s most important design challenge. Acknowledgments Competing interests. The authors declare no competing interests. Ethics approval and consent to participate. The study protocol was approved by the Committee On the Use of Humans as Experimental Subjects (COUHES) at the Massachusetts Institute of Technology (Exempt ID #E-7192). Informed consent was obtained from each participant before the study commenced. Generative AI. The authors declare the use of generative AI in refining the manuscript and front-end code generation. Availability of data and materials. The code for persona vec- tor generation, interface for user study, and user study anal- ysis are available here: https://github.com/mitmedialab/neural- transparency.git. Author Contributions. SK, AB, and P contributed to the man- uscript. AB developed the persona scores backend and validation. SK and P developed the persona visualization. SK developed the front-end and user-study. SK analyzed the user-study. References [1]Guillaume Alain and Yoshua Bengio. 2016. Understanding intermediate layers using linear classifier probes. arXiv preprint arXiv:1610.01644 (2016). [2]Rumi Allbert, James K Wiles, and Vlad Grankovsky. 2024. Identifying and manipulating personality traits in LLMs through activation engineering. arXiv preprint arXiv:2412.10427 (2024). [3]Emmanuel Ameisen, Jack Lindsey, Adam Pearce, Wes Gurnee, Nicholas L. Turner, Brian Chen, Craig Citro, David Abrahams, Shan Carter, Basil Hosmer, Jonathan Marcus, Michael Sklar, Adly Templeton, Trenton Bricken, Callum McDougall, Hoagy Cunningham, Thomas Henighan, Adam Jermyn, Andy Jones, Andrew Persic, Zhenyi Qi, T. Ben Thompson, Sam Zimmerman, Kelley Rivoire, Thomas Conerly, Chris Olah, and Joshua Batson. 2025. Circuit Tracing: Revealing Com- putational Graphs in Language Models. Transformer Circuits Thread (2025). https://transformer-circuits.pub/2025/attribution-graphs/methods.html [4]Andy Arditi, Oscar Obeso, Aaquib Syed, Daniel Paleka, Nina Panickssery, Wes Gurnee, and Neel Nanda. 2024. Refusal in language models is mediated by a single direction. Advances in Neural Information Processing Systems 37 (2024), 136037–136083. [5]Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, et al.2022. Constitutional ai: Harmlessness from ai feedback. arXiv preprint arXiv:2212.08073 (2022). [6] Leonard Bereska and Efstratios Gavves. 2024. Mechanistic interpretability for AI safety–a review. arXiv preprint arXiv:2404.14082 (2024). [7]Michael Bostock, Vadim Ogievetsky, and Jeffrey Heer. 2011. D3: Data-Driven Documents. In Proceedings of the IEEE Symposium on Information Visualization (InfoVis). 230–237. doi:10.1109/TVCG.2011.185 [8]Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J Pappas, and Eric Wong. 2025. Jailbreaking black box large language models in twenty queries. In 2025 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML). IEEE, 23–42. [9] Character.AI. 2023.Personalized Superintelligence Platform Character.AI Secures $150M in Series A Funding Led by Andreessen Horowitz. Business Wire. https://w.businesswire.com/news/home/20230323005299/en/Personalized- Superintelligence-Platform-Character.AI-Secures-150M-in-Series-A-Funding- Led-by-Andreessen-Horowitz Accessed: 2025-11-20. [10]Character.AI. 2025. Harnessing Data at Scale: Character.AI’s Transition to WarpStream. https://blog.character.ai/harnessing-data-at-scale-character-ais- transition-to-warpstream/. Accessed: 2025-11-20. [11] Runjin Chen, Andy Arditi, Henry Sleight, Owain Evans, and Jack Lindsey. 2025. Persona vectors: Monitoring and controlling character traits in language models. arXiv preprint arXiv:2507.21509 (2025). [12]Yida Chen, Aoyu Wu, Trevor DePodesta, Catherine Yeh, Kenneth Li, Nicholas Castillo Marin, Oam Patel, Jan Riecke, Shivam Raval, Olivia Seow, et al. 2024. Designing a dashboard for transparency and control of conversational AI. arXiv preprint arXiv:2406.07882 (2024). [13]Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. 2017. Deep reinforcement learning from human preferences. Advances in neural information processing systems 30 (2017). [14]Thomas H Costello, Gordon Pennycook, and David G Rand. 2024. Durably reducing conspiracy beliefs through dialogues with AI. Science 385, 6714 (2024), eadq1814. [15] Hoagy Cunningham, Aidan Ewart, Logan Riggs, Robert Huben, and Lee Sharkey. 2023. Sparse autoencoders find highly interpretable features in language models. arXiv preprint arXiv:2309.08600 (2023). [16]Julian De Freitas, Zeliha Oğuz-Uğuralp, Ahmet Kaan Uğuralp, and Stefano Pun- toni. 2025. AI companions reduce loneliness. ucaf040 pages. [17] Sebastian Dohnány, Zeb Kurth-Nelson, Eleanor Spens, Lennart Luettgau, Alastair Reid, Iason Gabriel, Christopher Summerfield, Murray Shanahan, and Matthew M Nour. 2025. Technological folie deux: Feedback loops between AI chatbots and mental illness. arXiv preprint arXiv:2507.19218 (2025). [18]Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. arXiv e-prints (2024), arXiv–2407. [19] Nelson Elhage, Tristan Hume, Catherine Olsson, Nicholas Schiefer, Tom Henighan, Shauna Kravec, Zac Hatfield-Dodds, Robert Lasenby, Dawn Drain, Carol Chen, Roger Grosse, Sam McCandlish, Jared Kaplan, Dario Amodei, Martin Wattenberg, and Christopher Olah. 2022. Toy Models of Superposition. Trans- former Circuits Thread (2022). [20] Cathy Mengying Fang, Auren R Liu, Valdemar Danry, Eunhae Lee, Samantha WT Chan, Pat Pataranutaporn, Pattie Maes, Jason Phang, Michael Lampe, Lama Ahmad, et al.2025. How AI and Human Behaviors Shape Psychosocial Effects of Extended Chatbot Use: A Longitudinal Randomized Controlled Study. arXiv preprint arXiv:2503.17473 (2025). [21]Rachel Fieldhouse. 2023. Can AI chatbots trigger psychosis? What the science says. Afr. J. Ecol 61 (2023), 226–227. [22]Stephen Fitz, Peter Romero, Steven Basart, Sipeng Chen, and Jose Hernandez- Orallo. 2025. Psychometric Personality Shaping Modulates Capabilities and Safety in Language Models. arXiv preprint arXiv:2509.16332 (2025). [23]Ivar Frisch and Mario Giulianelli. 2024. LLM agents in interaction: Measuring personality consistency and linguistic alignment in interacting populations of large language models. arXiv preprint arXiv:2402.02896 (2024). [24] Deep Ganguli, Liane Lovitt, Jackson Kernion, Amanda Askell, Yuntao Bai, Saurav Kadavath, Ben Mann, Ethan Perez, Nicholas Schiefer, Kamal Ndousse, et al.2022. Red teaming language models to reduce harms: Methods, scaling behaviors, and lessons learned. arXiv preprint arXiv:2209.07858 (2022). [25] Wes Gurnee and Max Tegmark. 2023. Language models represent space and time. arXiv preprint arXiv:2310.02207 (2023). [26]Thomas Hartvigsen, Saadia Gabriel, Hamid Palangi, Maarten Sap, Dipankar Ray, and Ece Kamar. 2022. Toxigen: A large-scale machine-generated dataset for adversarial and implicit hate speech detection. arXiv preprint arXiv:2203.09509 (2022). Karny, Baez, & Pataranutaporn [27]Lujain Ibrahim, Franziska Sofia Hafner, and Luc Rocher. 2025. Training lan- guage models to be warm and empathetic makes them less reliable and more sycophantic. arXiv preprint arXiv:2507.21919 (2025). [28] Guangyuan Jiang, Manjie Xu, Song-Chun Zhu, Wenjuan Han, Chi Zhang, and Yixin Zhu. 2022. Mpi: Evaluating and inducing personality in pre-trained lan- guage models. arXiv preprint arXiv:2206.07550 (2022). [29] Hang Jiang, Xiajie Zhang, Xubo Cao, Cynthia Breazeal, Deb Roy, and Jad Kabbara. 2023. PersonaLLM: Investigating the ability of large language models to express personality traits. arXiv preprint arXiv:2305.02547 (2023). [30]Junsol Kim, James Evans, and Aaron Schein. 2025. Linear representations of polit- ical perspective emerge in large language models. arXiv preprint arXiv:2503.02080 (2025). [31] Junseok Kim, Nakyeong Yang, and Kyomin Jung. 2024. Persona is a double-edged sword: Mitigating the negative impact of role-playing prompts in zero-shot reasoning tasks. arXiv preprint arXiv:2408.08631 (2024). [32]Kenneth Li, Tianle Liu, Naomi Bashkansky, David Bau, Fernanda B Viégas, Hanspeter Pfister, and Martin Wattenberg. 2024. Measuring and controlling persona drift in language model dialogs. CoRR (2024). [33]Johnny Lin. 2023. Neuronpedia: Interactive Reference and Tooling for Analyz- ing Neural Networks. https://w.neuronpedia.org Software available from neuronpedia.org. [34] Stephanie Lin, Jacob Hilton, and Owain Evans. 2021. Truthfulqa: Measuring how models mimic human falsehoods. arXiv preprint arXiv:2109.07958 (2021). [35]Auren R Liu, Pat Pataranutaporn, and Pattie Maes. 2024. Chatbot companion- ship: a mixed-methods study of companion chatbot usage patterns and their relationship to loneliness in active users. arXiv preprint arXiv:2410.21596 (2024). [36]Robert Mahari and Pat Pataranutaporn. 2025. Addictive Intelligence: Under- standing Psychological, Legal, and Technical Dimensions of AI Companionship. (2025). [37] Lars Malmqvist. 2025. Sycophancy in large language models: Causes and mitiga- tions. In Intelligent Computing-Proceedings of the Computing Conference. Springer, 61–74. [38]Samuel Marks and Max Tegmark. 2023. The geometry of truth: Emergent linear structure in large language model representations of true/false datasets. arXiv preprint arXiv:2310.06824 (2023). [39]Hamilton Morrin, Luke Nicholls, Michael Levin, Jenny Yiend, Udita Iyengar, Francesca DelGuidice, Sagnik Bhattacharyya, James MacCabe, Stefania Tognin, Ricardo Twumasi, et al.2025. Delusions by design? How everyday AIs might be fuelling psychosis (and what can be done about it). [40]Neel Nanda and Joseph Bloom. 2022. TransformerLens. https://github.com/ TransformerLensOrg/TransformerLens. [41]Neel Nanda, Andrew Lee, and Martin Wattenberg. 2023. Emergent linear repre- sentations in world models of self-supervised sequence models. arXiv preprint arXiv:2309.00941 (2023). [42]Pat Pataranutaporn, Valdemar Danry, Joanne Leong, Parinya Punpongsanon, Dan Novy, Pattie Maes, and Misha Sra. 2021. AI-generated characters for supporting personalized learning and well-being. Nature Machine Intelligence 3, 12 (2021), 1013–1022. [43] Pat Pataranutaporn, Sheer Karny, Chayapatr Archiwaranguprok, Constanze Albrecht, Auren R Liu, and Pattie Maes. 2025. " My Boyfriend is AI": A Computa- tional Analysis of Human-AI Companionship in Reddit’s AI Community. arXiv preprint arXiv:2509.11391 (2025). [44]Nilay Patel. 2024. Chatbot maker Replika says it’s okay if humans end up in relationships with AI. [45]Jason Phang, Michael Lampe, Lama Ahmad, Sandhini Agarwal, Cathy Mengying Fang, Auren R Liu, Valdemar Danry, Eunhae Lee, Samantha WT Chan, Pat Pataranutaporn, et al.2025. Investigating affective use and emotional well-being on ChatGPT. arXiv preprint arXiv:2504.03888 (2025). [46]Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2023. Direct preference optimization: Your language model is secretly a reward model. Advances in neural information processing systems 36 (2023), 53728–53741. [47]Lee Sharkey, Bilal Chughtai, Joshua Batson, Jack Lindsey, Jeff Wu, Lucius Bush- naq, Nicholas Goldowsky-Dill, Stefan Heimersheim, Alejandro Ortega, Joseph Bloom, et al.2025. Open problems in mechanistic interpretability. arXiv preprint arXiv:2501.16496 (2025). [48]Mrinank Sharma, Meg Tong, Tomasz Korbak, David Duvenaud, Amanda Askell, Samuel R Bowman, Newton Cheng, Esin Durmus, Zac Hatfield-Dodds, Scott R Johnston, et al.2023. Towards understanding sycophancy in language models. arXiv preprint arXiv:2310.13548 (2023). [49]Dominik Siemon, Timo Strohmann, Bijan Khosrawi-Rad, Triparna de Vreede, Edona Elshan, and Michael Meyer. 2022. Why do we turn to virtual companions? A text mining analysis of Replika reviews. (2022). [50]Vivian Ta, Caroline Griffith, Carolynn Boatfield, Xinyu Wang, Maria Civitello, Haley Bader, Esther DeCero, and Alexia Loggarakis. 2020. User experiences of social support from companion chatbots in everyday contexts: thematic analysis. Journal of medical Internet research 22, 3 (2020), e16235. [51]Curt Tigges, Oskar John Hollinsworth, Atticus Geiger, and Neel Nanda. 2023. Linear representations of sentiment in large language models. arXiv preprint arXiv:2310.15154 (2023). [52] Tommaso Tosato, Saskia Helbling, Yorguin-Jose Mantilla-Ramos, Mahmood Hegazy, Alberto Tosato, David John Lemay, Irina Rish, and Guillaume Dumas. 2025. Persistent Instability in LLM’s Personality Measurements: Effects of Scale, Reasoning, and Conversation History. arXiv preprint arXiv:2508.04826 (2025). [53]Quan Tu, Chuanqi Chen, Jinpeng Li, Yanran Li, Shuo Shang, Dongyan Zhao, Ran Wang, and Rui Yan. 2023. Characterchat: Learning towards conversational ai with personalized social support. arXiv preprint arXiv:2308.10278 (2023). [54]Alexander Matt Turner, Lisa Thiergart, Gavin Leech, David Udell, Juan J Vazquez, Ulisse Mini, and Monte MacDiarmid. 2023. Steering language models with activation engineering. arXiv preprint arXiv:2308.10248 (2023). [55]J Diego Zamfirescu-Pereira, Richmond Y Wong, Bjoern Hartmann, and Qian Yang. 2023. Why Johnny can’t prompt: how non-AI experts try (and fail) to design LLM prompts. In Proceedings of the 2023 CHI conference on human factors in computing systems. 1–21. [56] Renwen Zhang, Han Li, Han Meng, Jinyuan Zhan, Hongyuan Gan, and Yi-Chieh Lee. 2025. The dark side of ai companionship: A taxonomy of harmful algorithmic behaviors in human-ai relationships. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems. 1–17. [57] Chujie Zheng, Fan Yin, Hao Zhou, Fandong Meng, Jie Zhou, Kai-Wei Chang, Minlie Huang, and Nanyun Peng. 2024. On prompt-driven safeguarding for large language models. arXiv preprint arXiv:2401.18018 (2024). [58]Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, et al. 2023. Representation engineering: A top-down approach to ai transparency. arXiv preprint arXiv:2310.01405 (2023).

Instant research discovery