Paper deep dive
ACE-TA: An Agentic Teaching Assistant for Grounded Q&A, Quiz Generation, and Code Tutoring
Himanshu Tripathi, Charlottee Crowell, Kaley Newlin, Subash Neupane, Shahram Rahimi, Jason Keith
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%
Last extracted: 7/20/2026, 10:09:57 PM
Summary
The paper introduces ACE-TA, an Agentic Coding and Explanations Teaching Assistant framework that utilizes pre-trained Large Language Models to autonomously route queries to three modules: retrieval-grounded conceptual Q&A, adaptive quiz generation, and interactive code tutoring. The system employs hybrid retrieval (BM25 and FAISS), cross-encoder reranking, and sandboxed code execution with iterative feedback to support student learning in programming courses.
Entities (10)
Relation Signals (8)
ACE-TA → implements → Retrieval-Augmented Generation
confidence 95% · ACE-TA executes FAISS-based dense retrieval and BM25 lexical retrieval... hybrid retrieval augmented generation pipeline
ACE-TA → uses → Phi-3-Mini-128K-Instruct
confidence 95% · Two local LLMs are loaded via llama-cpp-python: a compact router/validator model Phi-3-Mini-128K-Instruct...
ACE-TA → uses → GPT-OSS-20B
confidence 95% · a larger generator model GPT-OSS-20B (Q4KS)...
ACE-TA → comparedagainst → Gemini 2.5 Pro
confidence 90% · we asked 3 SMEs to compare masked answers from ACE-TA and Gemini 2.5 Pro...
ACE-TA → uses → BM25
confidence 90% · ACE-TA executes FAISS-based dense retrieval and BM25 lexical retrieval...
ACE-TA → uses → FAISS
confidence 90% · ACE-TA executes FAISS-based dense retrieval...
ACE-TA → uses → MS MARCO-MiniLM-L6-v2
confidence 88% · candidate passages are reranked using a cross-encoder MS MARCO-MiniLM-L6-v2...
ACE-TA → evaluatedon → Learning Python
confidence 85% · From Mark Lutz, Learning Python (Fifth Edition)... we curated three datasets
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:We introduce ACE-TA, the Agentic Coding and Explanations Teaching Assistant framework, that autonomously routes conceptual queries drawn from programming course material to grounded Q&A, stepwise coding guidance, and automated quiz generation using pre-trained Large Language Models (LLMs). ACE-TA consists of three coordinated modules: a retrieval grounded conceptual Q&A system that provides precise, context-aligned explanations; a quiz generator that constructs adaptive, multi-topic assessments targeting higher-order understanding; and an interactive code tutor that guides students through step-by-step reasoning with sandboxed execution and iterative feedback.
Tags
Links
- Source: https://arxiv.org/abs/2604.09572v1
- Canonical: https://arxiv.org/abs/2604.09572v1
Trouble viewing inline? Open PDF directly →
Full Text
40,160 characters extracted from source content.
Expand or collapse full text
ACE-TA: An Agentic Teaching Assistant for Grounded Q&A, Quiz Generation, and Code Tutoring Himanshu Tripathi † , Charlottee Crowell † , Kaley Newlin ‡ , Subash Neupane § , Shahram Rahimi † , Jason Keith ¶ † University of Alabama, Tuscaloosa, Alabama, USA ‡ Brown University, Providence, Rhode Island, USA § Meharry Medical College, Nashville, Tennessee, USA ¶ Iowa State University of Science and Technology, Ames, Iowa, USA (htripathi, ccrowell)@crimson.ua.edu; kaleynewlin@brown.edu; subash.neupane@mmc.edu; srahimi@ua.edu; jkeith@iastate.edu Abstract We introduce ACE-TA, the Agentic Coding and Ex- planations Teaching Assistant framework, that au- tonomously routes conceptual queries drawn from pro- gramming course material to grounded Q&A, stepwise coding guidance, and automated quiz generation us- ing pre-trained Large Language Models (LLMs). ACE- TA consists of three coordinated modules: a retrieval- grounded conceptual Q&A system that provides pre- cise, context-aligned explanations; a quiz generator that constructs adaptive, multi-topic assessments targeting higher-order understanding; and an interactive code tu- tor that guides students through step-by-step reasoning with sandboxed execution and iterative feedback. Introduction Higher education is experiencing sustained growth follow- ing the sharp enrollment declines caused by the COVID- 19 pandemic. Between Spring 2024 and Spring 2025, post- secondary enrollment in the United States increased by more than half a million students, with undergraduate enrollment rising by 3.5% (Kim et al., 2025). While encouraging, this surge intensifies longstanding instructional challenges: in- structors frequently manage multiple large course sections, and Teaching Assistants (TAs) (often students themselves) face substantial workloads and limited availability. As a result, many learners struggle to obtain timely help out- side scheduled office hours, particularly when questions arise during homework, late-night study sessions, or while wrestling with complex programming problems. These ac- cess constraints can hinder conceptual understanding, re- duce motivation, and exacerbate performance gaps in high- enrollment Science, Technology, Engineering, and Mathe- matics (STEM) courses. At the same time, recent advances in Agentic Artifi- cial Intelligence (AAI) offer new opportunities to provide targeted on-demand educational support. Unlike traditional Large Language Model (LLM) systems, agentic frame- works couple language models with capabilities such as Retrieval-Augmented Generation (RAG), tool use, memory, Copyright © 2026 by the authors. This open access article is published under the Creative Commons Attribution-NonCommercial 4.0 International License. and multi-step planning, enabling them to autonomously de- compose tasks, invoke external tools, and operate with mini- mal human supervision (Acharya, Kuppan, and Divya, 2025; Sapkota, Roumeliotis, and Karkee, 2026). Despite the ca- pabilities of AAI frameworks, their integration into higher education remains limited. Most existing educational LLM systems focus narrowly on natural-language Q&A or ad- ministrative support (Khurana et al., 2023), whereas agen- tic systems can leverage planning, tool use, and retrieval to support computational tasks that require multi-step rea- soning, including code construction, debugging, and step- wise problem-solving. These are precisely the areas where novice programmers typically need the most guidance. For example, students in introductory programming courses of- ten grasp concepts abstractly but struggle to implement them in code, debug errors, or reason about program behavior. Ex- isting systems improve access to course information, but of- fer limited assistance with applied problem-solving or self- evaluation (Keith et al., 2024; Neupane et al., 2024; Keith et al., 2025). This creates a need for unified systems that sup- port conceptual understanding, guided code construction, and formative assessment. To address this need, we introduce the Agentic Cod- ing and Explanations Teaching Assistant (ACE-TA), an AAI framework that supports students in conceptual rea- soning, guided code development, and formative assess- ment through adaptive quizzes. ACE-TA incorporates three synergistic components: (1) a retrieval-grounded conceptual Q&A module that delivers concise, textbook-aligned ex- planations; (2) an adaptive quiz generator that constructs higher-order, multi-concept assessments and tailors diffi- culty based on learner responses; and (3) an interactive step- wise code tutor that decomposes programming problems, validates student code through sandboxed execution, and provides iterative feedback. Together, these capabilities al- low ACE-TA to serve as an after-hours teaching assistant that students can consult for conceptual clarification, struc- tured practice, and guided problem-solving. The main contributions of this paper are as follws: • We design an agentic teaching assistant (ACE-TA) that provides grounded conceptual Q&A, adaptive quiz gener- ation, and real-time stepwise coding guidance using pre- trained LLMs. • We introduce novel integrations of hybrid retrieval, multi- concept quiz generation, and incremental code tutoring with sandboxed execution and iterative feedback. • We demonstrate the effectiveness of ACE-TA using a combination of quantitative metrics and expert human evaluation. The remaining sections of this work are structured as fol- lows. Section 2 describes the ACE-TA architecture in greater detail, outlining the orchestration layer and specific modules for Q&A, quiz generation, and code tutoring. Section 3 pro- vides an analysis of its performance using both quantitative metrics and qualitative Subject Matter Expert (SME) evalu- ations. Section 4 benchmarks the framework against repre- sentative architectures and discusses related works. Finally, in Section 5, we conclude the work and discuss future re- search opportunities. Architecture & Method ACE-TA is a local, multi-module tutoring framework that integrates RAG, adaptive assessment, and stepwise code coaching under a shared orchestration layer as depicted in Fig. 1. The framework routes each learner query to one of three specialized pathways such as Quiz Generator, Concep- tual Q&A, or Code Tutor, while reusing shared LLM and retrieval resources to minimize initialization overhead. Orchestration Layer (Query Router) A lightweight orchestration layer implements query rout- ing and module dispatch. Two local LLMs are loaded via llama− cpp− python: a compact router/validator model Phi− 3− Mini− 128K − Instruct (Q4 KM ) 1 and a larger generator model GPT − OSS − 20B (Q4KS) (OpenAI, 2025). The router is prompted with a constrained schema listing available tools and representative query pat- terns, and outputs exactly one label from Conceptual Q&A, Quiz Generator, Code Tutor, Unknown. Routing is executed deterministically (temperature 0). The orchestrator then dis- patches the original query to the selected module while reusing preloaded model objects to reduce per-query latency. Conceptual Query handling Conceptual question (Fig. 1, (B1)) is answered via hybrid retrieval, reranking, and constrained generation over the Hy- brid Vector DB. ACE-TA executes FAISS-based dense re- trieval and BM25 lexical retrieval, then forms a candidate pool as the union of the top-K results from both channels (K = 20). Then, candidate passages are reranked using a cross-encoder MS MARCO− MiniLM − L− 6− v2 2 , and the top five passages are selected as the final context. Fi- nally, the selected passages are concatenated into a context block and injected into a structured Harmony style prompt 3 (system: tutoring role; developer: constraints + retrieved 1 https://huggingface.co/microsoft/Phi-3-m ini-128k-instruct 2 https://huggingface.co/cross-encoder/ms-m arco-MiniLM-L6-v2 3 https://cookbook.openai.com/articles/ope nai-harmony context; user: query). The generator is instructed to con- dition responses (Fig. 1(B3)) on retrieved evidence (Fig. 1(B2)) and to explicitly acknowledge insufficient support when the context is inadequate. Quiz Generator The quiz module generates Multiple Choice Quiz (MCQ) targeting higher-order understanding and supports adaptive difficulty. For broad requests (e.g., asking for a quiz on a Python function), the validator (Phi− 3−Mini− 128K− Instruct (Q4 KM )) decomposes the topic into five can- didate subtopics (JSON list) for learner selection, reducing topic drift and improving item specificity (Fig. 1 (A1)). The refined topic embedding (all-MiniLM-L6-v2 4 ) retrieves 50 candidate chunks from the Quiz Vector DB. Then we uti- lize Maximum Marginal Relevance (MMR) that selects 25 diverse passages: MMR(c i ) = λsim(q,c i )− (1− λ) max c j ∈S sim(c i ,c j ) where, q is the query, c i ,c j are candidate chunks, S is the set of already-selected diverse chunks and λ is a trade- off parameter (λ = 0.7) (Fig. 1, (A2)). The genera- tor constructs a compact concept framework (e.g., mech- anisms, misconceptions, constraints, trade-offs) (Fig. 1, (A3)) and synthesizes scenario-based MCQs with four op- tions, a labeled correct answer, distractor rationales, Bloom- level tags (Apply/Analyse/Evaluate/Create), and the con- cepts assessed. Then, the validator (Phi − 3 − Mini − 128K − Instruct (Q4 KM )) checks structural validity, Bloom alignment, and multi-concept coverage, filtering low- quality items prior to presentation (Fig. 1, (A4)). For adap- tive progression (Fig. 1, (A5, A6)), the difficulty of the subsequent item is adjusted based on correctness, promot- ing items of higher-order upon correct responses, and pro- viding corrective feedback with lower-level reinforcement otherwise. Code Tutor The Code Tutor provides stepwise guidance for translating natural-language questions (Fig. 1, (C1)) into executable Python programs. First, it invokes a planning prompt for the generation of a plan utilizingGPT−OSS−20B (Q4 KS) as a generator. The generator produces a numbered sequence of logic-level steps (no code, no environment instructions) that decomposes the task into minimal reasoning units. For each step the tutor builds a second prompt that contains the cumulative code written so far and the current step descrip- tion (this is excluded for the first step where only current step description is provided) and asks the LLM for the small- est new Python snippet that implements only that step (Fig. 1, (C2)), which is then executed (Fig. 1, (C3)) and if failed it gives feedback back to modify the steps (Fig. 1, (C4)). which prevents premature inclusion of later logic and clarifies the boundary between ideas. The learner sees 4 https://huggingface.co/sentence-transform ers/all-MiniLM-L6-v2 Query Query Router Embedding Model External Documents Document Doc 1 Doc N Context Steps Asking to solve one step Steps to solve Generated Code Execution Agent Feedback Analysis of wrong answer given by student Wrong answered step Execution Step and Answer Feedback to student on where he/she could be wrong LLM Response of a question Context+ Prompt Correct Answer Step answer Check Answer to each step Wrong Answer Code Generator Hybrid Vector DB Conceptual Summary+ Mapping Relation+ Key terminologies Question selector MCQ Generator Question Option Selected Correct/Wrong Answer Checker (B) Conceptual Q&A (C) Code Tutor (A) Quiz Generator Context Retrieval With Late+Semantic Chunking Documents Embedding Model Doc 1 Doc N Context Quiz Vector DB Context Retrieval Quiz User Coding Problem Conceptual Queries A1 A2 A4 A5 A6 B1 A3 B2 B3 C1 C2 C3 C4 C5 C6 C7a C7b C7b C8 C7b Figure 1: ACE-TA multi agent workflow for routed retrieval, quiz construction, and stepwise, feedback driven code tutoring. the textual step (Fig. 1, (C5)), reviews the current code (if they have written something), and then types only the new lines they believe are needed (Fig. 1, (C6)), while the tu- tor handles boilerplate such as appending a minimal body when a line ends with a block opener. The student code first passes through an Abstract Syntax Tree called AST to catch syntax or indentation issues before pushing it to the sandbox testing. This ensures syntax and structural er- rors are caught early, avoiding wasted sandbox runs and giv- ing students faster, safer, and more focused feedback. If the user code passes the sandbox test successfully, the system then compares the learner snippet to the reference snippet with an LLM based comparator that focuses on the current step, accepts alternate but equivalent logic, and returns ei- ther a confirmation with the next step (Fig. 1, (C7a)) or for wrong answers, analyzes the main flaw (Fig. 1, (C7b)) and providing a feedback with a short explanation of what the learner would have thought and how they can improve it (Fig. 1,(C8)). This process yields a complete executable so- lution built through tightly guided iterations. The same local implementation groundwork then supports the experimental study in which we evaluate ACE-TA across coding support, conceptual Q&A, and quiz generation tasks. Evaluation and Results We evaluate ACE-TA using quantitative metrics and quali- tative evaluation by Subject Matter Experts (SMEs) includ- ing course instructors, teaching assistants, and graduate stu- dents, and benchmark against representative architectures. From Mark Lutz, Learning Python (Fifth Edition), O’Reilly Media, 2013 5 we curated three datasets: 100 conceptual Q&A pairs with reference answers and retrieved contexts, 108 MCQs across 35 topics, and 150 coding problems split 5 https://w.oreilly.com/library/view/lea rning-python-5th/9781449355722/ evenly across difficulty levels, with qualitative subsets rated by SMEs (for more dataset details see Dataset Descrip- tion and Preparation in Appendix). ACE-TA employs dual vector databases optimized for distinct retrieval objectives (more details see Vector Database in Appendix). All experi- ments were conducted on a workstation with GeForce RTX 5090 32 GB, Intel Core Ultra 9 285K CPU, and 64 GB RAM. Retrieval and Answer Fidelity We first quantify how well the conceptual question an- swering module performs when it answers Python ques- tions using a hybrid retrieval augmented generation pipeline over textbook material. Using 100 question answer pairs with relevant chunk retrieved and ground truth generated using Gemini 2.5 Pro, we compute RAGAS answer rele- vancy and context scores together with BERTScore based on RoBERTa-large and ROUGE to capture both semantic and lexical fidelity. Retrieval shows near perfect context preci- sion with high recall, indicating that the hybrid BM25 and FAISS search plus cross encoder reranker over late chunk- ing and semantic chunking based vector DB reliably sur- faces the right passages. Fig. 2 shows high answer relevancy around 0.94 and strong BERTScore F1 around 0.93 sug- gest very small meaning drift, while moderate ROUGE val- ues (ROUGE1, ROUGE2, ROUGEL ≈ 0.5) show that the model tends to paraphrase instead of copying, which is help- ful for learning. Running at temperature 0 further supports grounded answers. Overall, these trends indicate that stu- dents receive responses that follow textbook intent while us- ing simpler language. Next, we add expert ratings that com- pare these explanations with those from Gemini 2.5 Pro. SME Judgements and Ground Truth Comparison To examine explanation quality beyond quantitative metrics, we asked 3 SMEs to compare masked answers from ACE- 0.2 0.4 0.6 0.8 1.0 0.94 0.99 0.93 0.89 0.97 0.93 0.53 0.50 0.53 Answer Relevancy Context Precision Context Recall Precision Recall F1 Rouge-1 Rouge-2 Rouge-L Performance Analysis for Theoretical Queries (RAGAS, BERTScore, ROUGE) RAGAS Metrics (Grounding + Faithfulness) BERTScore Metrics (Semantic Similarity) ROUGE Metrics (Lexical Overlap) Figure 2: Quantitative performance of the conceptual QA module on natural language queries, showing RAGAS (answer relevancy, context precision/recall), BERTScore (precision/recall/F1), and ROUGE (ROUGE1, ROUGE2, ROUGEL) metrics for retrieval and answer fidelity. TA and Gemini 2.5 Pro on 20 conceptual Python questions, rating depth of explanation from 1 to 5. Across all 3 experts ACE-TA scores around 3.8 on depth while Gemini 2.5 Pro remains near 1.3 as shown in Fig. 3. This pattern suggests that our explanations are not only preferred on average but also reliably judged deeper and more focused. SMEs note that our answers stay on topic, whereas Gemini often intro- duces side information that feels like clutter, and one SME flags a confusing keyword argument name that could mis- lead learners. 1.453.50 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 SME #1 1.253.90 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 SME #2 Depth of explanation by Gemini 2.5 Pro Depth of explanation by Our System 1.353.85 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 SME #3 Gemini 2.5 Pro vs Our System per SME Figure 3: Depth of explanation ratings from three SMEs for Gemini 2.5 Pro and ACE-TA conceptual QA module across 20 questions, showing consistently higher depth scores for ACE-TA where higher values reflect more deeper explana- tions. Topic Breadth and Balance of Quiz Generator We evaluate the ACE-TA quiz generator using distribu- tion based metrics that quantify how broadly and uniformly each topic’s conceptual space is sampled. The topics and subtopics are derived from Mark Lutz’s Learning Python, 0.00.51.01.52.0 Breadth score (0 = narrow, 2+ = very broad) A Realistic OOP Example: A Step-by-Step Project Advanced Class Details Advanced Exception Handling and Context Managers Advanced String Formatting Assignments, Expressions, and Printing Core Data Type Overview Decorators Dictionary Operations and Methods Exception Handling Basics Function Argument Passing Function Basics and Polymorphism Generators and Advanced Comprehensions Iterators and Basic List Comprehensions List Operations and Methods Managed Attributes: Properties and Descriptors Metaclasses Module Packages and Relative Imports Module and Program Architecture Numeric Types and Operations OOP Concepts and Class Basics OOP Design Patterns Operator Overloading Program Execution Methods Python Documentation Tools Python Fundamentals & Strengths String Fundamentals The Dynamic Typing Model The Python Execution Model Tuples, Files, and General Type Concepts Unicode and Byte Strings Topic Breadth of Sub-Topics Covered Mean: 1.73 0.00.20.40.60.81.0 Balance score (0 = skewed, 1 = even) Balance of Sub-Topic Coverage Mean: 0.97 Figure 4: (A) Breadth of quiz subtopic coverage across Python topics, measured via penalised Shannon’s entropy. (B) Balance of subtopic coverage, measured via Pielou’s evenness. and GPT OSS 20B assigns subtopic relevance scores for 108 generated questions spanning 35 topics. From these scores, we compute two summary measures per topic that capture coverage and balance. Penalised Shannon’s entropy for breadth is given by H pen =− k X 1 p i lnp i where H pen is the penalised Shannon’s entropy for a given topic (reduced when its subtopics are heavily shared with other topics, so redundant coverage counts less), p i is the normalised relevance weight of subtopic i, and k is the total number of subtopics for that topic (Fig. 4 (A)). Pielou’s evenness is defined as: J = H pen log(subtopic count) where, J is Pielou’s evenness index for a topic, which measures how evenly the total relevance is spread across its subtopic count (Fig. 4 (B)). Breadth scores span 0.5 to 2.17 (mean 1.73), with topics like “Numeric Types and Operations” showing wide cover- age. Evenness remains high (0.88 to 1.00, mean 0.97), in- dicating no single subtopic dominates. Fig. 4 shows topics achieve balanced coverage without extreme skew. No topic falls into the extreme regime of simultaneously low entropy and high skew. This indicates that the retrieval stage, con- ceptual framework extraction, and MCQ synthesis collec- tively promote quizzes that are broad in coverage and rea- sonably balanced between subtopics. Explanation Adequacy of the Quiz Generator from SME Responses Three SMEs rated explanations for 10 generated questions on a five point scale (1 poor, 5 very strong). Fig. 5 shows SME #1SME #2SME #3 1.0 1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0 Explanation adequacy (1 5) Explanation adequacy ratings across SMEs Median Mean Overall mean = 3.57 Figure 5: Explanation adequacy ratings from three SMEs for quiz master explanations across 10 questions, showing SME specific means and medians with an overall mean of 3.57, indicating generally adequate but improvable explana- tory quality. Table 1: Evaluation of code-generation models across Out- put Match Rate, CodeBERT similarity, and CodeBLEU for easy, medium, and hard Python coding tasks. Values are color–coded into five ranges: 0.100–0.299,0.300–0.499, 0.500–0.699,0.700–0.899,0.900–1.000. ModelOutput Match RateCodeBert (simmax)CodeBLEU (codebleu) EasyMediumHardEasyMediumHardEasyMediumHard Gemma-3-27B-IT (Q40) 1.0001.0001.0000.9930.9940.9900.5000.5440.411 Llama-4-Scout-17B- 16E-Instruct 1.0001.0001.0000.9920.9950.9920.4890.5100.429 GPT-4o-Mini0.9801.0000.9600.9930.9940.9920.4650.5160.409 DolphinCoder- StarCoder2-15B (Q80) 1.0000.9800.9400.9940.9940.9890.5120.4870.415 DeepSeek-R1-Qwen3- 8B-BF16 1.0000.9800.9400.9940.9950.9890.5570.5890.404 Maverick-7B (FP16)1.0000.9600.9600.9950.9950.9900.5460.5650.403 Mistral-7B-Instruct-v0.2 (8-bit) 0.9200.8600.6600.9830.9790.9740.3360.3160.243 mid to upper ratings with an overall mean of 3.57, suggest- ing explanations are generally adequate but variable. SMEs noted occasional misalignment between question wording and rationale, and reliance on unstated assumptions in op- tions, which we partly attribute to token limit constraints when GPT OSS 20B generates multiple items and explana- tions in a single pass. We next evaluate Code Tutor support during programming practice. Robustness and Solution Quality of Coding Tutor We evaluate the Code Tutor by measuring robustness across difficulty levels and code quality utilizing seven models (see Table 1) spanning 7B to 27B parameters and 4 bit to BF16 precision, operated inside the tutor loop. Each step allows up to five refinement attempts after sandbox feedback, and each problem allows up to three final sandbox attempts. We evaluate 150 problems (50 easy, 50 medium, 50 hard). We summarize task robustness using a Difficulty Weighted Per- formance Measure (DWPM): DWPM = 0.2A easy + 0.3A med + 0.5A hard , A d = C d 50 , d∈easy, med, hard. DWPM (0-1) Gemma 3 LLaMA 4 Scout DolphinCoder StarCoder 2 GPT 4o Mini DeepSeek R1 LLaMA 4 Maverick Mistral DWPM = 1.000 4.0-bit DWPM = 1.000 FP16/BF16 DWPM = 0.964 8.0-bit DWPM = 0.976 FP16/BF16 DWPM = 0.964 FP16/BF16 DWPM = 0.968 FP16/BF16 DWPM = 0.772 8.0-bit Overall coding robustness (DWPM) 0.00.20.40.60.81.0 CodeBLEU (token-level overlap) 0.96 0.97 0.98 0.99 1.00 CodeBERT similarity (semantic match) CodeBLEU vs CodeBERT Code Difficulty Trend line Easy Medium Hard 78151727 Model parameters (billions) Figure 6: (A) Overall coding robustness with model size and quantization of 7 student LLMs measured by DWPM. (B) CodeBLEU vs. CodeBERT similarity across problems, coloured by difficulty level. where, the weighting (e.g., 0.2, 0.3, 0.5) emphasizes harder problems, reflecting their longer and more compositional step sequences (weights set by an SME) and C d is the num- ber of problems solved correctly at difficulty d. Using identical task prompts (adapted only to each model’s native template), Fig. 6(A) shows that larger pa- rameter models (e.g., Gemma 3 27B, Llama 4 Scout 17B) achieve near ceiling DWPM, whereas quantised models such as Mistral 7B 8 bit degrade primarily due to formatting deviations that trigger sandbox failures (for example, adding non code prefaces and conclusions despite tutor warnings). To quantify the association between model capacity and DWPM, we combine parameter count and numeric precision into an effective capacity proxy: Effective Size i = params i + precision i 10 where, i is LLM, params i is the number of parameters, and precision i is the numerical precision level for that model. The division by 10 rescales precision so that it modulates but does not dominate the parameter term. Under this proxy, the models have Effective Size i values of 27.40 (Gemma 3 27B Q4), 18.60 (Llama 4 Scout 17B 16E), 16.60 (GPT 4o Mini), 15.80 (DolphinCoder StarCoder2 15B Q8), 9.60 (DeepSeek Qwen3 8B BF16), 8.60 (Maverick 7B FP16), and 7.80 (Mistral 7B 8 bit). We then compute the Spear- man rank correlation between model capacity and DWPM. Models are ranked by Effective Size i (ascending) and in- dependently ranked by DWPM (ascending), using average ranks for ties. The resulting Spearman correlation is ρ = 1− 6 P n i=1 d 2 i n (n 2 − 1) = 0.875 (n = 7), indicating a strong positive association between effective ca- pacity and difficulty weighted robustness for student models operating inside the tutor architecture. Furthermore, to characterize solution quality beyond pass or fail, we compute CodeBLEU and CodeBERT similarity and report means over all 150 problems per model (Fig. 6(B), Table 1). Across models, CodeBERT scores are con- sistently high while CodeBLEU is moderate, reflecting se- mantic correctness with lexical variation relative to the ref- erence. The small deviation in the trend line is explained by a subset of problems (9 total - 3 easy, 4 medium, and 2 hard) where models produce concise, semantically equiva- lent solutions using built in functions or alternative control structures, which the tutor appropriately accepts. These re- sults suggest that the tutor’s stepwise decomposition, itera- tive correction, and sandbox execution enable diverse LLMs to converge to runnable and logically sound code. While higher capacity yields higher DWPM, smaller models still improve through repeated iterations, implying that robust- ness is driven both by model strength and by the tutor’s structured guidance and verification. SME Judgements on Depth of Coding Tutor To assess step clarity in the coding tutor, we asked three SMEs to rate tutor-generated step sequences for 15 coding questions on a five-point Likert scale, where 1 denotes very unclear and 5 denotes very clear (Figure 7). For each SME, 123456789101112131415 SME #1 SME #2 SME #3 Questions (Easy, Medium, Hard) Step Clarity (15) Across 3 SMEs 1 5 Figure 7: Step clarity ratings from three SMEs across 15 coding tutor questions grouped by easy medium and hard we compute the mean rating as ̄x k = 1 15 P 15 j=1 x k,j , where x k,j is the rating assigned by SME k ∈ 1, 2, 3 to ques- tion j. The resulting means are 4.07, 3.47, and 4.00, in- dicating that SMEs generally find the step sequences clear and usable. We then compute the overall mean across SMEs as ̄x all = 1 3 P 3 k=1 ̄x k ≈ 3.84, which suggests that the tu- tor typically produces well-structured, instructionally useful steps. Finally, we stratify ratings by difficulty to examine ro- bustness under increasing task complexity. Mean step clarity is 4.00 for easy, 3.80 for medium, and 3.73 for hard prob- lems, showing only a modest decline as difficulty increases. SME comments are consistent with these trends: steps are usually readable, but occasionally rely on basic program- ming terminology that may be unfamiliar to absolute begin- ners. In practice, this implies learners receive a clear, ordered solution path, while instructors can further improve accessi- bility by adding brief glossaries or short reminders for key terms. Related Works and Comparison Study LLM-based teaching assistants for educational Q&A and tu- toring have been explored extensively. BarkPlug Keith et al. (2024); Neupane et al. (2024) leveraged RAG to pro- vide grounded answers about campus resources, extended to BarkPlug 2.0 Keith et al. (2025) as a multi-agent system for high DFW courses. Hicke et al. (2023) explored SFT, RAG, and DPO for AI TA course Q&A. Khanmigo Khan Academy (2025) employs Socratic tutoring, while Pensieve Discuss Yang et al. (2025) targets collaborative program- ming through synchronized editing. Jill Watson Taneja et al. (2024) and Parrot Dai and Sarukkai (2025) focus on direct response generation with retrieval grounding. Despite these advances, several gaps remain. First, most systems address either conceptual Q&A or coding support in isolation, lacking unified frameworks that integrate both with formative assessment; ACE-TA addresses this through a single agentic pipeline with intelligent routing across three specialized modules. Second, existing retrieval approaches employ generic vector stores rather than task-optimized in- dexing; ACE-TA introduces dual vector databases with task-specific chunking strategies (late chunking for Q&A fidelity, semantic-only for quiz diversity). Third, interac- tive coding support is limited, with systems either provid- ing complete solutions or minimal scaffolding; ACE-TA implements stepwise decomposition with sandboxed exe- cution and iterative feedback at the individual step level. Finally, few systems generate adaptive quizzes for higher- order assessment; ACE-TA produces Bloom-tagged MCQs with difficulty adaptation based on learner responses. For detailed comparison across pedagogical and technical di- mensions see Comparison with Existing Work in Appendix. Conclusion & Future Research Opportunities In this work, we introduced ACE-TA, an agentic teaching assistant that unifies three workflows, grounded conceptual Q&A, conceptual quiz generation, and stepwise code tutor- ing. A lightweight router dispatches learner queries to spe- cialized modules for retrieval augmented generation and ex- planation, diagnostic multiple choice assessment, and sand- boxed stepwise code construction, reducing context switch- ing while keeping outputs anchored in course materials. Em- pirically, ACE-TA retrieves relevant evidence and produces faithful explanations, achieves broad and balanced quiz cov- erage, and supports robust code completion across diverse models, with SMEs rating both conceptual depth and step clarity favorably. Overall, these results indicate that ACE- TA can provide reliable assistance that strengthens concep- tual understanding and practical coding skill. Future work will focus on two directions. First, we will evaluate ACE- TA on additional courses beyond a single Python textbook to test generalization. Second, we will conduct a classroom deployment study to measure learning impact and identify failure modes that guide targeted improvements in ground- ing and feedback quality. Acknowledgment This work was supported by the PATENT Lab (Predictive Analytics and Technology Integration Laboratory) at the Department of Computer Science, University of Alabama, Tuscaloosa. References Acharya, D. B.; Kuppan, K.; and Divya, B. 2025. Agentic ai: Autonomous intelligence for complex goals—a com- prehensive survey. IEEE Access 13:18912–18936. Dai, K., and Sarukkai, A. 2025. Parrot: An agentic class- room AI. In ICML 2025 Workshop on Collaborative and Federated Agentic Workflows. Douze, M.; Guzhva, A.; Deng, C.; Johnson, J.; Szilvasy, G.; Mazar ́ e, P.-E.; Lomeli, M.; Hosseini, L.; and J ́ egou, H. 2024. The faiss library. Google Gemini Team. 2025. Gemini 2.5: Pushing the fron- tier with advanced reasoning, multimodality, long con- text, and next generation agentic capabilities. G ̈ unther, M.; Mohr, I.; Williams, D. J.; Wang, B.; and Xiao, H. 2024. Late chunking: contextual chunk embeddings using long-context embedding models. arXiv preprint arXiv:2409.04701. Hicke, Y.; Agarwal, A.; Ma, Q.; and Denny, P.2023. Ai-ta: Towards an intelligent question-answer teach- ing assistant using open-source llms.arXiv preprint arXiv:2311.02775. Keith, J. M.; Amirlatifi, A.; Rahimi, S.; Neupane, S.; and Mittal, S. 2024. Bark plug: The chatgpt of the bagley college of engineering at mississippi state university. In 2024 ASEE Annual Conference & Exposition, number 10.18260/1-2–46635. Portland, Oregon: ASEE Confer- ences. https://peer.asee.org/46635. Keith, J. M.; Amirlatifi, A.; Mittal, S.; Neupane, S.; and TRI- PATHI, H. 2025. Barkplug 2.0 and beyond-a chatbot for assisting students in high dfw courses. In 2025 ASEE An- nual Conference & Exposition. Khan Academy. 2025. Khanmigo for learners. https: //w.khanmigo.ai/learners. Accessed: 2025- 09-23. Khurana, D.; Koli, A.; Khatter, K.; and Singh, S. 2023. Natural language processing: state of the art, current trends and challenges. Multimedia Tools and Applications 82:3713–3744. Kim, H.; Berg, B.; Karamarkovich, S.; Holsapple, M.; and Shapiro, D. 2025. Current term enrollment esitmates: Spring 2025. Technical report, National Student Clear- inghouse Research Center, Herndon, VA. Neupane, S.; Hossain, E.; Keith, J.; Tripathi, H.; Ghiasi, F.; Golilarz, N. A.; Amirlatifi, A.; Mittal, S.; and Rahimi, S. 2024. From questions to insightful answers: Building an informed chatbot for university resources. OpenAI. 2025. gpt-oss-120b & gpt-oss-20b model card. Robertson, S., and Sp ̈ arck Jones, K.1976.Relevance weighting of search terms. Journal of the American Soci- ety for Information Science 27:129–146. Sapkota, R.; Roumeliotis, K. I.; and Karkee, M. 2026. Ai agents vs. agentic ai: A conceptual taxonomy, applica- tions and challenges. Information Fusion 126:103599. Schechter Vera, H.; Dua, S.; Zhang, B.; Salz, D.; Mullins, R.; Raghuram Panyam, S.; Smoot, S.; Naim, I.; Zou, J.; Chen, F.; Cer, D.; Lisak, A.; Choi, M.; Gonzalez, L.; Sanseviero, O.; Cameron, G.; Ballantyne, I.; Black, K.; Chen, K.; Wang, W.; Li, Z.; Martins, G.; Lee, J.; Sherwood, M.; Ji, J.; Wu, R.; Zheng, J.; Singh, J.; Sharma, A.; Sreepat, D.; Jain, A.; Elarabawy, A.; Co, A.; Doumanoglou, A.; Samari, B.; Hora, B.; Potetz, B.; Kim, D.; Alfonseca, E.; Moiseev, F.; Han, F.; Palma Gomez, F.; Hern ́ andez ́ Abrego, G.; Zhang, H.; Hui, H.; Han, J.; Gill, K.; Chen, K.; Chen, K.; Shanbhogue, M.; Boratko, M.; Suganthan, P.; Duddu, S. M. K.; Mariserla, S.; Ari- afar, S.; Zhang, S.; Zhang, S.; Baumgartner, S.; Goenka, S.; Qiu, S.; Dabral, T.; Walker, T.; Rao, V.; Khawaja, W.; Zhou, W.; Ren, X.; Xia, Y.; Chen, Y.; Chen, Y.-T.; Dong, Z.; Ding, Z.; Visin, F.; Liu, G.; Zhang, J.; Kenealy, K.; Casbon, M.; Kumar, R.; Mesnard, T.; Gleicher, Z.; Brick, C.; Lacombe, O.; Roberts, A.; Sung, Y.; Hoffmann, R.; Warkentin, T.; Joulin, A.; Duerig, T.; and Seyedhosseini, M. 2025. Embeddinggemma: Powerful and lightweight text representations. Taneja, K.; Maiti, P.; Kakar, S.; Guruprasad, P.; Rao, S.; and Goel, A. K. 2024. Jill watson: A virtual teaching assistant powered by chatgpt. In Olney, A. M.; Chounta, I.-A.; Liu, Z.; Santos, O. C.; and Bittencourt, I. I., eds., Artificial In- telligence in Education, 324–337. Cham: Springer Nature Switzerland. Yang, Y.; Kim, M.; Rondinelli, M.; and Shao, K. 2025. Pen- sieve grader: An ai-powered, ready-to-use platform for ef- fortless handwritten stem grading. Appendix Dataset Description and Preparation For our experiments, we utilize Mark Lutz, Learning Python (Fifth Edition), O’Reilly Media, 2013 6 as our primary data source to evaluate ACE-TA. We curate three datasets. First, the Conceptual Q&A dataset includes 100 textbook ques- tions with their reference answers generated by Gemini 2.5 Pro (Google Gemini Team, 2025), retrieved context chunks from the book, and corresponding ACE-TA answers, so each entry couples a question, evidence, and two candidate explanations. A qualitative subset of 20 of these questions keeps the same context but stores two masked answers from ACE-TA and Gemini 2.5 Pro together with depth ratings from three SMEs and short comments on focus and clarity. Second, the Quiz Generator dataset is designed by prompt- ing the quiz module to produce 108 MCQs across 35 Python topics. For each item, we record the associated subtopic la- bel and its relevance score produced by GPT OSS 20B, with subtopics drawn from the textbook’s topic structure. A qual- itative subset of 10 items is rated by SMEs on a five point adequacy scale, with short notes on whether explanations are clear, confusing, or assume unstated prior knowledge. Finally, the Code Tutor dataset covers programming support 6 https://w.oreilly.com/library/view/lea rning-python-5th/9781449355722/ using 150 coding problems, evenly split into easy, medium, and hard. We evaluate seven student facing LLM configura- tions within the tutor loop; for each model problem pair, we log task success and the number of additional attempts re- quired. For qualitative analysis, SMEs review 15 problems and assign step clarity ratings, noting instructions that are vague or overly technical. Vector Database ACE-TA maintains two vector indices, Hybrid Vector DB and Quiz Vector DB, each optimized for a distinct retrieval objective instead of a single generic store. For conceptual Q&A, Hybrid Vector DB is built from course textbook PDFs using late (G ̈ unther et al., 2024) and semantic chunking to preserve sentence level coherence while controlling chunk length. Each chunk is indexed with a dense channel using Gemma 2 Embedding 300M (Schechter Vera et al., 2025) and FAISS (Douze et al., 2024), and a lexical channel using BM25 (Robertson and Sp ̈ arck Jones, 1976) over the same text. This hybrid design supports grounded explanations where lexical fidelity and semantic proximity are jointly re- quired. For quiz generation, Quiz Vector DB applies seman- tic chunking without late alignment to increase concept di- versity per chunk, prioritizing coverage and compositional variety over paragraph level reconstruction. Comparison with Existing Work Table 2: Feature comparison across LLM-based tutoring systems.✓ = full support,• = partial support,× = not sup- ported. FeatureKhanmigo Pensieve Discuss Jill Watson Parrot ACE-TA (Ours) Guides Instead of Giving Answers ✓• ×✓ Handles Coding Help •✓× ×✓ Handles Theory Questions ✓•✓ Generates Quizzes •× ×✓ Checks Quiz Answers •× ×✓ Uses Retrieval to Stay Grounded •×✓ Late & Semantic Chunking × ×✓ Agentic Pipeline •×•✓ Adaptivity to Student Attempts ✓•✓ One System Covers All Tasks × ×✓ Table 2 contextualizes ACE-TA against representative LLM-based tutoring systems across pedagogical and tech- nical dimensions. In guidance style, Khanmigo emphasizes Socratic questioning to elicit understanding, Pensieve Dis- cuss provides collaborative support through synchronized editing, and Jill Watson and Parrot deliver direct responses, while ACE-TA employs stepwise scaffolding that decom- poses tasks into manageable increments with iterative feed- back. For coding support, only Pensieve Discuss and ACE- TA offer dedicated programming environments; however, ACE-TA uniquely implements a generate-execute-verify loop operating at individual step granularity with sandboxed validation, whereas Pensieve Discuss focuses on collabora- tive editing with autograding at submission level. Most sys- tems handle conceptual Q&A, but ACE-TA distinguishes itself by routing queries through task-specific retrieval com- bining late chunking for semantic-lexical fidelity and BM25- FAISS hybrid search with cross-encoder reranking, contrast- ing with the generic retrieval in Jill Watson and Parrot or the absence of retrieval-grounded responses in Khanmigo and Pensieve Discuss. For formative assessment, only Khan- migo and ACE-TA support quiz generation, though ACE- TA extends this with Bloom-level tagging, distractor ratio- nales, multi-concept coverage validation, and adaptive dif- ficulty progression based on learner performance. Regard- ing technical architecture, ACE-TA maintains dual vector databases optimized for distinct retrieval objectives (ground- ing versus diversity), employs late and semantic chunking strategies absent in other systems, and integrates routing, re- trieval, quiz synthesis, and coding guidance within a unified agentic pipeline, enabling seamless transitions between con- ceptual learning, assessment, and applied problem-solving without external tool switching or manual orchestration.