Paper deep dive
LumiXAI: A Modular Full-Stack Framework for Feature Attribution
Alfio Ferrara, Lorenzo Gatta, Sergio Picascia, Elisabetta Rocchetti
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Feature attribution is a central tool of model interpretability, yet the software through which it is applied remains fragmented: individual tools specialize along narrow axes, such as a single modality, a code API or a GUI, or a fixed rather than extensible method set, and rarely combine these strengths. Moreover, many explainability tools are designed primarily for domain experts, requiring programming skills or familiarity with attribution methods that can make them difficult for non-expert users to access. In this article, we present LumiXAI, a modular full-stack framework that consolidates attribution analysis into a single system. It couples classification and generative attribution with an interactive GUI supporting bidirectional exploration, a plug-in architecture for registering new models and methods, and three access tiers serving non-programmers, developers, and extenders from one backend. Its contribution is a system that operationalises established attribution methods under one interface, one interaction model, and one persistence layer, with containerised services and persistent results making analyses reproducible across machines.
Tags
Links
- Source: https://arxiv.org/abs/2608.24524v1
- Canonical: https://arxiv.org/abs/2608.24524v1
Trouble viewing inline? Open PDF directly â
Full Text
37,348 characters extracted from source content.
Expand or collapse full text
LumiXAI: A Modular Full-Stack Framework for Feature Attribution Alfio Ferrara 1 and Lorenzo Gatta 2 and Sergio Picascia 2 and Elisabetta Rocchetti 2 1 Department of Literary Studies, Philology and Linguistics, UniversitĂ degli Studi di Milano, Milan, Italy 2 Department of Computer Science, UniversitĂ degli Studi di Milano, Milan, Italy alfio.ferrara@unimi.it, lorenzo.gatta@studenti.unimi.it sergio.picascia@unimi.it, elisabetta.rocchetti@unimi.it Abstract Feature attribution is a central tool of model in- terpretability, yet the software through which it is applied remains fragmented: individual tools specialize along narrow axes, such as a single modality, a code API or a GUI, or a fixed rather than extensible method set, and rarely com- bine these strengths. Moreover, many explain- ability tools are designed primarily for domain experts, requiring programming skills or famil- iarity with attribution methods that can make them difficult for non-expert users to access. In this article, we present LumiXAI, a modular full-stack framework that consolidates attribu- tion analysis into a single system. It couples classification and generative attribution with an interactive GUI supporting bidirectional ex- ploration, a plug-in architecture for registering new models and methods, and three access tiers serving non-programmers, developers, and ex- tenders from one backend. Its contribution is a system that operationalises established attribu- tion methods under one interface, one interac- tion model, and one persistence layer, with con- tainerised services and persistent results mak- ing analyses reproducible across machines. 1 Introduction As machine learning models are increasingly de- ployed in consequential settings, understanding why a model produces a given output has become as important as the output itself. Feature attribu- tion addresses this by assigning importance scores to input elements based on their contribution to a prediction or generation. As attribution has grown into a broad methodological family spanning gradient-based, perturbation-based, and attention- based techniques, a practical question arises: how can practitioners navigate, explore, and systemati- cally evaluate these methods? The tooling through which attribution is applied has developed unevenly across the field: individual tools tend to specialize along a narrow set of axes, Hover over a pixel to see how each input token contributed to it. a 13% painting 3% of 3% an 4% elephant 14% in 5% the 1% baroque 22% style 2% Click a prompt token to see which parts of the image it has influenced the most. a - painting - of - an - elephant - in - the - baroque - - style - Figure 1: Bidirectional exploration of text-to-image at- tribution in LumiXAI. Top (imageâtoken): hovering a pixel reads that point from every prompt tokenâs at- tention matrix and recolors the prompt cards. Bottom (tokenâimage): clicking a prompt token overlays its attribution heatmap on the generated image. such as scope, kind of interaction, or extensible method set, and rarely combine these strengths al- together. A researcher who needs attribution across several model types and interaction styles must therefore assemble and reconcile multiple tools with incompatible interfaces, output formats, and deployment assumptions, an integration burden that impedes precisely the reproducible, exploratory analysis that attribution is meant to support. This paper presents LumiXAI 1 , a modular, full- stack framework for feature-attribution analysis across text and image classification, autoregressive text generation, and text-to-image diffusion. Lu- miXAI is designed to make attribution analyses inspectable, reproducible, and extensible within a single workflow: model families and attribution methods are exposed as registered plug-ins; each layer runs as an independent Docker service; anal- 1 Code and documentation are available athttps:// github.com/TechLory/LumiXAI; an online demo is avail- able at http://lumixai.islab.di.unimi.it/. 1 arXiv:2608.24524v1 [cs.SE] 25 Aug 2026 ysis metadata and attribution outputs are persisted for later inspection and sharing; and the same back- end supports a web GUI, a Python SDK, and a smart-batch mode for larger experiments. In this way, LumiXAI brings together interaction styles, user groups, and model families that existing tools typically address only in isolation. Figure 1 illus- trates an example of the interaction with LumiXAI for text-to-image attribution. In addition to the con- ventional token-to-image view, where a selected prompt token highlights the regions it contributes to, LumiXAI supports the reverse direction: users can hover over an image location and inspect which prompt tokens are most responsible for that point. The remainder of the paper is organised as fol- lows. Section 2 situates LumiXAI with respect to existing attribution methods and interpretabil- ity toolkits. Section 3 describes the framework architecture, including its plug-in interface, service- based deployment, storage layer, and user-facing access modes. Section 4 presents case studies il- lustrating LumiXAI across supported model fam- ilies and interaction scenarios. Section 5 reports a user evaluation of the systemâs usability and in- terpretability workflow. Section 6 concludes with current limitations and directions for future work. 2 Related Work Attribution tooling has proliferated alongside in- terpretability research, but existing tools tend to specialise along a small number of axes and rarely combine their strengths. Table 1 places a repre- sentative selection along the five dimensions that structure the discussion below: the input modali- ties with first-class attribution support (text classi- fication, generative text, image classification, and text-to-image generation); whether attribution is defined over the generative decoding process rather than a single classification output; whether the tool ships an interactive GUI; whether its architecture is designed to accommodate new methods or model types; and the audiences it serves. We distinguish three such audiences, since few tools address more than one: extenders («), who implement a new attribution method or adapt an unsupported model type and need an interface in which to exercise it; developers (Ă), who apply existing methods through a code API; and non-programmers ( ), who work entirely through a GUI. General-purpose libraries provide broad attri- bution backends but limited interactive support. ToolScopeGen. Attr. GUI Ext. Audience CaptumTC, TG, ICâ â«à SHAP / LIMETCâ Ă OmniXAITC, ICâ â Ă AIX360TC, ICâ Ă AllenNLP Interpret TC, TG â â«à LITTC, TG â â« eccoTGâ â Ă ThermostatTCâ Ă InseqTGâ â«à ferretTCâ â«à CafGaTGâ â ICX360TGâ Ă LumiXAITC, TG, IC, T2I â â â«à Table 1: Comparison of attribution tools. Scope: TC = text classification, TG = text generation, IC = im- age classification, T2I = text-to-image generation. Gen- erative attribution:âfirst-class attribution over the generative decoding process (autoregressive or diffu- sion);limited to masked-token or single-step pre- diction, or salience confined to particular seq2seq set- tings;âclassification or regression outputs only. GUI: whether it ships an interactive visual interface, e.g. stan- dalone application or interactive notebook widget. Ext.: if it is designed for adding new attribution methods or model types. Audience:«extender (adds methods or models),Ădeveloper (applies methods through a code API), non-programmer (works through the GUI alone). Cells reflect released versions as of July 2026 (Captum 0.9.0, SHAP 0.52.0, Inseq 0.7.1, LIT 1.3.1, CafGa 0.0.6, ICX360 0.1.0). LIME (Ribeiro et al., 2016b) and SHAP (Lund- berg and Lee, 2017) offer model-agnostic expla- nations and plotting utilities, but are mainly used through code and are classification-oriented. Cap- tum (Kokhlikyan et al., 2020) provides an extensi- ble PyTorch library of gradient- and perturbation- based methods, now including support for image masks and language-model attribution (Miglani et al., 2023), but its interactive companion, Captum Insights, was retired after v0.8.0. OmniXAI (Yang et al., 2022) and AIX360 (Arya et al., 2019) broaden coverage across data types, yet remain centered on classification rather than generative decoding. NLP-specific tools offer richer model inspection but cover only parts of our target setting. AllenNLP Interpret (Wallace et al., 2019) exposes saliency and gradient primitives with reusable front- end components, but is tied to the deprecated Al- lenNLP framework. LIT (Tenney et al., 2020) provides an extensible browser-based workbench for classification and sequence-to-sequence mod- els, although it is broader than attribution alone. ecco (Alammar, 2021) visualizes language-model behavior interactively from code, while Thermo- stat (Feldhus et al., 2021) distributes precomputed 2 classifier explanations. ferret (Attanasio et al., 2023) complements these systems by benchmark- ing attribution methods for transformer classifiers. The closest tools to LumiXAI target generative text. Inseq (Sarti et al., 2023) supports feature attribution for decoder-only and encoder-decoder generation, but exposes results through a Python API and does not address image models. ICX360 (Wei et al., 2025) attributes LLM generations to their input context, but is likewise text-only and API-driven. CafGa (Boyle et al., 2025) adds interactive con- trols for attribution granularity, including Jupyter widgets, but remains confined to text. LumiXAI differs in providing the widest scope support, combining generative attribution across text and text-to-image diffusion with an interactive GUI, documented extension interfaces, and multi- ple access tiers backed by the same services. This design lets non-programmers inspect analyses visu- ally, developers access them programmatically, and extenders add new model families or attribution methods through the same registry. 3 The LumiXAI Framework LumiXAI is a clientâserver system that isolates model-specific and compute-intensive functional- ity behind a REST API. Clients interact with this boundary rather than with model classes directly, allowing the same backend to support the web inter- face, Python SDK, and batch workflows. Internally, the attribution engine is organised around two ab- stract plug-in interfaces, wrappers and attributors, which decouple model access from explanation computation and make it possible to add new tasks, models, or attribution methods without changing the core system. 3.1 Architecture Overview Figure 2 summarizes the system architecture. Lu- miXAI is deployed as three independent services: a FastAPI backend, a Next.js frontend, and Mk- Docs documentation. The backend exposes the REST API, hosts the attribution engine, and is the only component that loads model weights. Wrap- pers retrieve and adapt models, currently from the Hugging Face Hub, while attributors compute ex- planations through a shared interface. The frontend and SDK are both ordinary API clients and contain no attribution logic. Attribu- tion requests may be long-running, so the backend handles them as jobs: each request receives an iden- Wrappers BaseWrapper ABC Attributors BaseAttributor ABC Storage manager API Endpoints REST GUI Web interface MKDocs documentation Container Docs Backend Frontend REST Client Script SDK SQLite database Job metadata JSON payloads Attribution outputs Hugging Face Hub Container Container Container Figure 2: LumiXAI architecture. The backend exposes a REST API, orchestrates wrapper and attributor plug- ins, retrieves models from the Hugging Face Hub, and stores metadata in SQLite with attribution payloads as JSON files. Backend, frontend, and documentation are deployed as independent containers, and both the web GUI and client scripts consume the same API. tifier, while computation proceeds asynchronously. Access to the accelerator is serialised to avoid mem- ory conflicts, and device placement is resolved at load time across CUDA, Apple MPS, or CPU. Completed analyses are persisted through a hy- brid storage layer. Structured job metadata, such as model, method, prompt, status, and elapsed time, are stored in SQLite. Larger attribution payloads, including token scores, per-step generation traces, attention matrices, and generated images, are se- rialised as JSON files referenced from the corre- sponding database row. This design keeps analyses available for later inspection, export, or transfer. 3.2 Extensibility: Wrappers and Attributors Extensibility is enforced by keeping models and attribution algorithms behind abstract interfaces. A BaseWrapperprovides a uniform view of a model family: it loads weights, places the model on the selected device, and exposes the forward operation. When available, it may also expose embeddings for gradient-based methods, but this is not required for attribution methods that rely on other signals, such as attention. ABaseAttributordefines an attribution method as a mapping from a wrapped model and an input to a standardised output object. The wrapper 3 is injected into the attributor at construction time, and each attributor declares the wrapper families it supports. The backend uses this declaration to reject incompatible modelâmethod pairings before loading weights, while the frontend uses the same information to disable invalid choices. Wrappers and attributors are registered as plug- ins. Adding a new model family or attribution method therefore requires one subclass, an applica- bility declaration, and one registry entry. The client remains independent of these details: it populates its selectors from a backend manifest, and the back- end resolves the appropriate wrapper from the task metadata declared by the selected model. The data contracts crossing the plug-in boundary are modality-agnostic. The atomic unit of input is an input feature, i.e. a token with its position for text, a patch or a superpixel for an image, and every attributor returns the same output object ir- respective of what was explained. The coverage reported in Section 3.3 is thus what is currently implemented. 3.3 Supported Models and Methods Thecurrentreleaseshipsfourwrappers. HFTextClassificationWrapperadaptsen- coder classifiers and returns class logits with token-level scores.HFTextGenerationWrapper adapts autoregressive models through a step-wise decoding loop, producing an attribution trace that links each generated token and probability to attribution scores over the context available at that step.HFImageClassificationWrapper supports vision transformers and CNN classifiers, rendering explanations over the modelâs prepro- cessed, de-normalised input.HFImageWrapper adapts diffusion-based text-to-image models and exposes the components needed for cross-attention attribution. LumiXAI provides ten established attribution methods. Seven are available through Captum for text and image classification: Integrated Gradi- ents (Sundararajan et al., 2017), DeepLift (Shriku- mar et al., 2017), Saliency (Simonyan et al., 2014), InputĂGradient, GradientSHAP (Lundberg and Lee, 2017), Occlusion (Zeiler and Fergus, 2013), and LIME (Ribeiro et al., 2016a). For perturbation methods, the perturbed unit is the token for text, a regular patch grid for image Occlusion, and SLIC superpixels (Achanta et al., 2012) for image LIME. Two further methods, SmoothGrad (Smilkov et al., 2017) and Grad-CAM (Selvaraju et al., 2017), tar- get vision models. Finally, DAAM (Tang et al., 2023) attributes text-to-image outputs by recover- ing per-token spatial maps from diffusion cross- attention. 3.4 Multi-Tier Access A single backend is exposed through three access modes, corresponding to the âAudienceâ of Table 1. Web GUI ( ). The Next.js interface lets users configure, launch, and inspect attribution jobs with- out writing code. Classification views display to- ken or image heatmaps alongside model predic- tions. Generative views expose the attribution trace bidirectionally: in text generation, users can move from input tokens to generated-token influence or from a generated token back to its supporting con- text; in text-to-image generation, they can inspect token heatmaps on the image or query a pixel to see which prompt tokens contributed most. The GUI also includes tutorials and a history view for revisiting completed jobs. Python SDK (Ă).A single self-contained mod- ule drives the REST API programmatically, for headless use in any environment able to reach the backend. Documented notebooks accompany it and reproduce the case studies of Section 4. For larger sweeps,run_smart_batch()reduces re- peated model weight loading by grouping jobs by model and method, so that each set of weights is loaded once per group, and the groups are then or- dered by a configurable strategy (fastest_first, slowest_first, ornone) suited to shared and ded- icated accelerators respectively. Results are col- lected by polling and returned in the callerâs origi- nal job order irrespective of completion order, so that the optimization remains transparent to the calling code. Framework extension («).The third mode ad- dresses researchers whose model or attribution method the framework does not yet support, and who therefore act on the codebase rather than through it. Such a user subclassesBaseWrapperor BaseAttributor, declares the new component in the registry, and obtains a plug-in that the backend orchestrates on equal terms with those shipped by default. Because the manifest is derived from the registry, the new method appears in the GUI selec- tors and is reachable through the REST API and the SDK immediately. 4 3.5 Deployment LumiXAI is deployed with Docker Compose. A basedocker-compose.ymldefines the backend, frontend, and documentation services, while an optionaldocker-compose.gpu.ymlenables GPU execution through the NVIDIA Container Toolkit. Services can be run together or independently: the backend is sufficient for SDK use, the frontend enables interactive analysis, and the documentation can be served separately. The backend image is built with Poetry from a lockfile, and configuration is parameterised through environment variables with in-file defaults. The Hugging Face cache is mounted as a volume so that model weights persist across restarts. Together with the SQLite metadata store and JSON attri- bution payloads described above, this makes both the execution environment and completed analyses portable across machines. Runtime is mainly determined by the selected model and attribution method, including model in- ference, repeated forward or backward passes, and, where applicable, diffusion or generation steps. Lu- miXAI adds scheduling, serialization, and retrieval around these computations, but these components are not expected to be the main source of latency; we leave a dedicated measurement of framework overhead to future work. 4 Case Studies We illustrate LumiXAI usage through two exam- ples covering text classification and text-to-image diffusion. Additional examples on image classifica- tion and open-ended text generation are provided in the Appendix A. Text Classification. We first examine which in- put features drive a toxicity classifier. A curated subset of Civil Comments (Borkan et al., 2019) is passed tounitary/toxic-bert, and each predic- tion is attributed toward the most probable class using Integrated Gradients. In the frontend, tokens are colored by signed attribution score, making it possible to inspect whether the model relies on identity terms, offensive language, or quoted stereo- types when assigning toxicity labels (Figure 3). Text-to-Image Diffusion.The second case study explores whether text-to-image diffusion models spatially separate artistic content and style. Fol- lowing Ferrara et al. (2025), we generate images from prompts pairing a subject with an artistic style Figure 3: Text classification in LumiXAI. For the com- ment âha ha you guys are a bunch of losers.â, classified as INSULT byunitary/toxic-bert, tokens are colored by Integrated-Gradients attribution toward the predicted class; losers dominates the attribution (51%). and attribute pixels to prompt tokens using DAAM. Since diffusion cross-attention links prompt to- kens to image regions during denoising, LumiXAI makes it possible to compare the spatial influence of content and style terms directly. Selecting a prompt token overlays its attribution heatmap on the image, while selecting an image region reveals the responsible prompt tokens. This bidirectional view supports inspection of whether content tokens localize to depicted objects, whether style tokens spread across texture and background, and where the model entangles the two (Figure 4). 5 User Evaluation We conducted a user evaluation to assess whether LumiXAI is usable by different target audiences and whether its interface supports the inspection of attribution results across tasks. The study focuses on usability and workflow effectiveness rather than on the intrinsic quality of the explanations, which depends on the external attribution libraries we employ. Setup. We recruited 12 participants, including 3 participants with experience in machine learning or NLP and 9 non-expert users. Expertise was de- termined from two background questions on prior experience training or evaluating ML models and using feature-attribution or explainability methods. Each participant followed the same protocol. Af- ter giving consent and answering the background questions, participants completed the LumiXAI tu- 5 Figure 4: Text-to-image attribution in LumiXAI. For the prompt âa impressionism painting of a pizzaâ, generated byStable Diffusion XL, selecting pizza overlays its DAAM heatmap (tokenâimage), while to- ken percentages report attribution at the selected pixel (imageâtoken). torials, introducing the supported tasks, attribution views, and interaction patterns, followed by a short comprehension check on the meaning and sign of attribution scores. Participants then completed 8 tasks across four task types: text classification, im- age classification, autoregressive text generation, and text-to-image diffusion. For each task type we fixed the model and attribution method and pro- vided the required input (e.g., a text prompt or im- age); participants launched the attribution jobs and inspected the results, exploring token-level attribu- tions in both directions for text tasks and attribution regions for image tasks. This let us evaluate inter- face usability while keeping model and method choices controlled across users. Finally, partici- pants rated usability statements about LumiXAI on a 7-point Likert scale and gave free-text feedback on the most useful feature, on confusing or frustrat- ing aspects, and on anything they wanted to inspect that the system did not allow. Results. Table 2 summarizes the questionnaire results. Overall, participants judged the system pos- itively, appreciating in particular the ease of switch- ing between models and attribution methods and its helpfulness in understanding model behaviour, although ratings were consistently higher among expert users. The qualitative feedback confirms this picture, indicating that clearer onboarding, more transparent heatmap colour scaling, and explicit indication of methodâmodel compatibility would be the main avenues for improving accessibility to non-experts. QuestionOverall Experts Non-experts Ease of use4.925.674.67 Learnability5.176.334.78 Enjoyability4.755.674.44 Integration5.335.675.22 Helpfulness5.256.005.00 Reuse intention4.254.334.22 Bidirectionality4.675.004.56 Switching5.585.675.56 Compatibility4.585.334.33 Consistency5.005.674.78 Mean4.955.534.76 Table 2: Mean user-study ratings (7-point Likert scale) per questionnaire item, overall and by expertise (n = 12; 3 experts, 9 non-experts). 6 Conclusion We presented LumiXAI, a modular full-stack framework for reproducible and interactive feature- attribution analysis. LumiXAI operationalizes es- tablished attribution techniques within a unified system that spans multiple model families, access modes, and user profiles. Through its plug-in archi- tecture, service-based deployment, persistent stor- age layer, and complementary interfaces, the frame- work turns attribution workflows that often require ad hoc scripting into analyses that can be inspected, repeated, and extended. The case studies illustrate how LumiXAI supports both targeted debugging and exploratory interpretation across classification, generation, and text-to-image diffusion scenarios. The framework current coverage is necessarily lim- ited to the model families and attribution methods implemented so far, and future work will extend the plug-in library with additional techniques and architectures. In particular, emerging multimodal models, such as image-text-to-text systems, raise new attribution questions that fit naturally within LumiXAIâs modality-agnostic design. 6 Limitations LumiXAI is extensible, but its current implemen- tation covers only a subset of possible tasks and models. Other architectures, especially image-text- to-text vision-language models, require additional wrappers and visualization components. Likewise, the supported attribution methods are representa- tive rather than exhaustive. The tool also inherits the limitations of attribu- tion methods themselves. Scores can vary with baselines, tokenization, perturbation units, aggrega- tion choices, and preprocessing, and should there- fore be treated as diagnostic signals rather than causal explanations. This is especially relevant for generative models, where decoding traces and diffu- sion attention maps expose useful patterns without fully explaining the underlying computation. The system was primarily developed and tested under single-user conditions, with concurrency han- dled only across multiple processes issued by that user. During the user evaluation, some participants encountered bugs caused by multiple users concur- rently loading and unloading different configura- tions on the same backend, a scenario we had not accounted for; robustness under concurrent multi- user load remains future work. Finally, the user evaluation is limited in scale and scope. It tests whether participants can complete guided attribution tasks and interpret the visual- izations, but does not measure long-term use in open-ended research workflows. Moreover, we do not isolate framework overhead from model infer- ence and attribution time, so a dedicated systems benchmark is left to future work. Ethics Statement Participation in the user evaluation was volun- tary, and participants were informed of the studyâs purpose, procedure, and expected duration before starting. Written informed consent was obtained from all participants prior to the tutorial phase. No personally identifiable information was col- lected: questionnaire responses and free-text feed- back were anonymised at collection time and are reported only in aggregate. Participants could with- draw from the study at any point without providing a reason. The study involved no more than mini- mal risk, as it consisted of guided interaction with a software interface and did not involve sensitive per- sonal data, deception, or vulnerable populations. References Radhakrishna Achanta, Appu Shaji, Kevin Smith, Aure- lien Lucchi, Pascal Fua, and Sabine SĂŒsstrunk. 2012. Slic superpixels compared to state-of-the-art super- pixel methods. IEEE Transactions on Pattern Analy- sis and Machine Intelligence, 34(11):2274â2282. J Alammar. 2021. Ecco: An Open Source Library for the Explainability of Transformer Language Models. In Proceedings of the 59th Annual Meeting of the As- sociation for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing: System Demonstrations, pages 249â257, Online. Association for Computational Linguistics. Vijay Arya, Rachel K. E. Bellamy, Pin-Yu Chen, Amit Dhurandhar, Michael Hind, Samuel C. Hoffman, Stephanie Houde, Q. Vera Liao, Ronny Luss, Alek- sandra Mojsilovi Ì c, Sami Mourad, Pablo Pedemonte, Ramya Raghavendra, John Richards, Prasanna Sat- tigeri, Karthikeyan Shanmugam, Moninder Singh, Kush R. Varshney, Dennis Wei, and Yunfeng Zhang. 2019. One Explanation Does Not Fit All: A Toolkit and Taxonomy of AI Explainability Techniques. Preprint, arXiv:1909.03012. Giuseppe Attanasio, Eliana Pastor, Chiara Di Bonaven- tura, and Debora Nozza. 2023. Ferret: A Framework for Benchmarking Explainers on Transformers. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Lin- guistics: System Demonstrations, pages 256â266, Dubrovnik, Croatia. Association for Computational Linguistics. Daniel Borkan, Lucas Dixon, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. 2019. Nuanced met- rics for measuring unintended bias with real data for text classification. In Companion Proceedings of The 2019 World Wide Web Conference, W â19, page 491â500, New York, NY, USA. Association for Computing Machinery. Alan David Boyle, Furui Cheng, VilĂ©m Zouhar, and Mennatallah El-Assady. 2025. CafGa: Customizing Feature Attributions to Explain Language Models. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 461â470, Suzhou, China. As- sociation for Computational Linguistics. Jwala Dhamala, Tony Sun, Varun Kumar, Satyapriya Krishna, Yada Pruksachatkun, Kai-Wei Chang, and Rahul Gupta. 2021. Bold: Dataset and metrics for measuring biases in open-ended language generation. In Proceedings of the 2021 ACM Conference on Fair- ness, Accountability, and Transparency, FAccT â21, page 862â872, New York, NY, USA. Association for Computing Machinery. Nils Feldhus, Robert Schwarzenberg, and Sebastian Möller. 2021. Thermostat: A Large Collection of 7 NLP Model Explanations and Analysis Tools. In Pro- ceedings of the 2021 Conference on Empirical Meth- ods in Natural Language Processing: System Demon- strations, pages 87â95, Online and Punta Cana, Do- minican Republic. Association for Computational Linguistics. Alfio Ferrara, Sergio Picascia, and Elisabetta Rocchetti. 2025.The cow of rembrandt analyzing artistic prompt interpretation in text-to-image models. In 2025 IEEE 35th International Workshop on Machine Learning for Signal Processing (MLSP), pages 1â6. Narine Kokhlikyan, Vivek Miglani, Miguel Martin, Edward Wang, Bilal Alsallakh, Jonathan Reynolds, Alexander Melnikov, Natalia Kliushkina, Carlos Araya, Siqi Yan, and Orion Reblitz-Richardson. 2020. Captum: A unified and generic model interpretability library for PyTorch. Preprint, arXiv:2009.07896. Scott M Lundberg and Su-In Lee. 2017. A Unified Approach to Interpreting Model Predictions. In Ad- vances in Neural Information Processing Systems, volume 30. Curran Associates, Inc. Vivek Miglani, Aobo Yang, Aram Markosyan, Diego Garcia-Olano, and Narine Kokhlikyan. 2023. Using Captum to Explain Generative Language Models. In Proceedings of the 3rd Workshop for Natural Lan- guage Processing Open Source Software (NLP-OSS 2023), pages 165â173, Singapore. Association for Computational Linguistics. Marco Ribeiro, Sameer Singh, and Carlos Guestrin. 2016a. âwhy should I trust you?â: Explaining the pre- dictions of any classifier. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demon- strations, pages 97â101, San Diego, California. As- sociation for Computational Linguistics. Marco Tulio Ribeiro, View Profile, Sameer Singh, View Profile, Carlos Guestrin, and View Profile. 2016b. "Why Should I Trust You?". In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ACM Con- ferences, pages 1135â1144. Gabriele Sarti, Nils Feldhus, Ludwig Sickert, and Oskar van der Wal. 2023. Inseq: An Interpretability Toolkit for Sequence Generation Models. In Proceedings of the 61st Annual Meeting of the Association for Com- putational Linguistics (Volume 3: System Demonstra- tions), pages 421â435, Toronto, Canada. Association for Computational Linguistics. Ramprasaath R. Selvaraju, Michael Cogswell, Ab- hishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. 2017. Grad-cam: Visual explana- tions from deep networks via gradient-based local- ization. In 2017 IEEE International Conference on Computer Vision (ICCV), pages 618â626. Avanti Shrikumar, Peyton Greenside, and Anshul Kun- daje. 2017. Learning important features through propagating activation differences. In Proceedings of the 34th International Conference on Machine Learning - Volume 70, ICMLâ17, page 3145â3153. JMLR.org. Karen Simonyan, Andrea Vedaldi, and Andrew Zis- serman. 2014. Deep inside convolutional networks: Visualising image classification models and saliency maps. Preprint, arXiv:1312.6034. Daniel Smilkov, Nikhil Thorat, Been Kim, Fernanda ViĂ©gas, and Martin Wattenberg. 2017.Smooth- grad: removing noise by adding noise. Preprint, arXiv:1706.03825. Mukund Sundararajan, Ankur Taly, and Qiqi Yan. 2017. Axiomatic Attribution for Deep Networks. In Pro- ceedings of the 34th International Conference on Machine Learning, pages 3319â3328. PMLR. Raphael Tang, Linqing Liu, Akshat Pandey, Zhiying Jiang, Gefei Yang, Karun Kumar, Pontus Stenetorp, Jimmy Lin, and Ferhan Ture. 2023. What the DAAM: Interpreting Stable Diffusion Using Cross Attention. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5644â5659, Toronto, Canada. Association for Computational Linguistics. Ian Tenney, James Wexler, Jasmijn Bastings, Tolga Bolukbasi, Andy Coenen, Sebastian Gehrmann, Ellen Jiang, Mahima Pushkarna, Carey Radebaugh, Emily Reif, and Ann Yuan. 2020. The Language Interpretability Tool: Extensible, Interactive Visual- izations and Analysis for NLP Models. In Proceed- ings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demon- strations, pages 107â118, Online. Association for Computational Linguistics. Eric Wallace, Jens Tuyls, Junlin Wang, Sanjay Sub- ramanian, Matt Gardner, and Sameer Singh. 2019. AllenNLP Interpret: A Framework for Explaining Predictions of NLP Models. In Proceedings of the 2019 Conference on Empirical Methods in Natu- ral Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP): System Demonstrations, pages 7â12, Hong Kong, China. Association for Computa- tional Linguistics. Dennis Wei, Ronny Luss, Xiaomeng Hu, Lucas Mon- teiro Paes, Pin-Yu Chen, Karthikeyan Natesan Rama- murthy, Erik Miehling, Inge Vejsbjerg, and Hendrik Strobelt. 2025. ICX360: In-Context eXplainability 360 Toolkit. Preprint, arXiv:2511.10879. Wenzhuo Yang, Hung Le, Tanmay Laud, Silvio Savarese, and Steven C. H. Hoi. 2022.Om- niXAI: A Library for Explainable AI. Preprint, arXiv:2206.01612. Matthew D Zeiler and Rob Fergus. 2013. Visualizing and understanding convolutional networks. Preprint, arXiv:1311.2901. 8 A Additional Case Studies We further illustrate LumiXAI features through two additional examples covering image classification and open-ended text generation. ImageClassification. Thisclassifica- tioncasestudyillustratesattribution over a visual input.We apply a Vision TransformertrainedonMNISTdigits (farleyknight-org-username/vit-base-mnist) and explain its predicted class using the Captum implementation of Grad-CAM. In the image view, LumiXAI overlays the attribution heatmap directly on the model input, allowing users to inspect whether the classifier grounds its decision in the digit strokes rather than in background artifacts. In Figure 5, the model predicts the class 3, and the strongest activations align with the upper, middle, and lower curves that define the handwritten digit. Figure 5: Image classification in LumiXAI. A ViT model trained on MNIST predicts the digit 3; the Grad- CAM overlay highlights the image regions that most support the predicted class, with attribution concentrated on the main strokes of the digit. Open-Ended Text Generation. This case study uses the per-step attribution trace for autoregressive generation. We sample Wikipedia-derived prompt prefixes from BOLD (Dhamala et al., 2021) and letopenai-community/gpt2continue them deter- ministically with a short decoding horizon. For each generated token, Integrated Gradients records its probability together with attribution scores over the preceding prompt and generated context. The bidirectional generation view lets users select any generated token and inspect which earlier tokens contributed most to it. This supports visual analysis of whether continuations depend on demographic or identity-bearing terms, whether such tokens be- come attribution hotspots for evaluative words, and how attribution patterns differ between neutral and stereotype-adjacent completions (Figure 6). Figure 6: Open-ended generation in LumiXAI. Continu- ing the BOLD prefix âQueen Latifah is a contralto, and sheâ withgpt2, the selected token Islamic is explained by signed Integrated-Gradients scores over the prior context; Queen contributes negatively (â50%), while Lat contributes positively (+73%). 9