Paper deep dive
MM-IssueLoc: A Controlled Benchmark for Evaluating Visual Evidence in Multimodal Repository-Level Issue Localization
Shaoxiong Zhan, Shi Hu, Boyu Feng, Hai Lin, Andrew Gong, Zhengda Zhou, Jiaying Zhou, Yunyun Hou, Hao Su, Hai-Tao Zheng
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%
Last extracted: 7/18/2026, 7:39:09 AM
Summary
The paper introduces MM-IssueLoc, a controlled benchmark designed to evaluate the role of visual evidence in repository-level issue localization. It comprises 652 real issue-PR instances across 23 programming languages, annotated with image categories, relevance scores, and file/function-level gold labels. The benchmark isolates visual evidence through paired evaluation modes (text-only, with-image, VCE, VCE+image) and compares LLM-based agents and retrieval-based systems. Results indicate that current systems remain unreliable for multimodal localization, with top performers reaching only ~39% file Acc@5 and ~34% function Acc@10. The study highlights that visual evidence provides useful but model-dependent signals and that strong performance on text-dominant benchmarks like SWE-bench does not transfer to multimodal settings.
Entities (10)
Relation Signals (8)
Tsinghua University → affiliatedwith → Shaoxiong Zhan
confidence 98% · Shaoxiong Zhan 1,2 * ... 1 Tsinghua University
JD.com → affiliatedwith → Shi Hu
confidence 98% · Shi Hu 2 * ... 2 JD.com
MM-IssueLoc → contains → 652 issue-PR instances
confidence 97% · MM-IssueLoc contains 652 issue-PR instances across 23 languages, with annotations for 7 image categories and 4 relevance levels.
MM-IssueLoc → evaluates → Visual Evidence
confidence 95% · MM-IssueLoc turns visual evidence into an explicit evaluation variable, enabling future work to test whether systems improve by using visual evidence for localization
MM-IssueLoc-VL-Embedding → isdevelopedfor → MM-IssueLoc
confidence 93% · We evaluate LLM-based and retrieval-based systems, including MM-IssueLoc-VL-Emb as a controlled multimodal retriever.
SWE-BENCH → iscomparedwith → MM-IssueLoc
confidence 92% · Cross-benchmark comparisons show that high localization scores on text-dominant SWE benchmarks do not transfer cleanly to multimodal issue localization.
Visual Content Evidence (VCE) → converts → images
confidence 91% · VCE-based diagnostics that convert images into structured textual evidence.
OpenHands → achievesbeston →
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Real repository issues routinely include visual evidence such as screenshots, error dialogs, rendered UI states, and logs, yet repository-level issue localization is evaluated mostly as a text-only task. Existing multimodal SE benchmarks evaluate end-to-end repair, entangling localization with patch synthesis and obscuring whether visual input helped, hurt, or was ignored. We introduce \textbf{MM-IssueLoc}, a controlled benchmark and evaluation protocol for repository-level localization with visual evidence. MM-IssueLoc contains 652 issue-PR instances across 23 languages, with annotations for 7 image categories and 4 relevance levels. It provides file-level and function-level gold labels, paired text-only and with-image evaluation, and VCE-based diagnostics that convert images into structured textual evidence. We evaluate LLM-based and retrieval-based systems, including MM-IssueLoc-VL-Emb as a controlled multimodal retriever. Results show that existing systems remain far from reliable multimodal repository localization: the strongest agent reaches 38.96 file Acc@5 and 22.45 function Acc@10, while the strongest retriever reaches 33.86 function Acc@10. Cross-benchmark comparisons show that high localization scores on text-dominant SWE benchmarks do not transfer cleanly to multimodal issue localization. MM-IssueLoc turns visual evidence into an explicit evaluation variable, enabling future work to test whether systems improve by using visual evidence for localization, rather than by relying on text-only cues or downstream patch-generation effects.
Tags
Links
- Source: https://arxiv.org/abs/2607.15205v1
- Canonical: https://arxiv.org/abs/2607.15205v1
Trouble viewing inline? Open PDF directly →
Full Text
76,444 characters extracted from source content.
Expand or collapse full text
M-IssueLoc: A Controlled Benchmark for Evaluating Visual Evidence in Multimodal Repository-Level Issue Localization Shaoxiong Zhan 1,2 * , Shi Hu 2 * , Boyu Feng 2 , Hai Lin 1 , Andrew Gong 2 , Zhengda Zhou 2 , Jiaying Zhou 2 , Yunyun Hou 2 , Hao Su 2 , Hai-Tao Zheng 1 † 1 Tsinghua University 2 JD.com zhansx24@mails.tsinghua.edu.cn, hushi.5@jd.com, zheng.haitao@sz.tsinghua.edu.cn Abstract Real repository issues routinely include visual evidence such as screenshots, error dialogs, rendered UI states, and logs, yet repository- level issue localization is evaluated mostly as a text-only task. Existing multimodal SE benchmarks evaluate end-to-end repair, entan- gling localization with patch synthesis and ob- scuring whether visual input helped, hurt, or was ignored. We introduce M-IssueLoc, a controlled benchmark and evaluation proto- col for repository-level localization with visual evidence. M-IssueLoc contains 652 issue- PR instances across 23 languages, with an- notations for 7 image categories and 4 rele- vance levels. It provides file-level and function- level gold labels, paired text-only and with- image evaluation, and VCE-based diagnostics that convert images into structured textual evi- dence. We evaluate LLM-based and retrieval- based systems, including M-IssueLoc-VL- Emb as a controlled multimodal retriever. Re- sults show that existing systems remain far from reliable multimodal repository localiza- tion: the strongest agent reaches 38.96 file Acc@5 and 22.45 function Acc@10, while the strongest retriever reaches 33.86 function Acc@10. Cross-benchmark comparisons show that high localization scores on text-dominant SWE benchmarks do not transfer cleanly to multimodal issue localization. M-IssueLoc turns visual evidence into an explicit evalu- ation variable, enabling future work to test whether systems improve by using visual ev- idence for localization, rather than by rely- ing on text-only cues or downstream patch- generation effects.https://github.com/ Jasaxion/M-IssueLoc-Bench 1 Introduction Repository-level issue localization takes an issue description and a repository snapshot as input, * Equal contribution. † Corresponding author. and identifies the code locations relevant to re- solving the issue (Chen et al., 2025). For auto- mated software engineering (SE) systems, local- ization is a prerequisite for issue resolving: er- rors at this step propagate into every downstream stage of patch generation. Real-world repository is- sues are natively multimodal; developers routinely attach screenshots, UI states, error dialogs, ren- dered outputs, and log excerpts alongside textual descriptions. Recent work has begun to extend SE benchmarks beyond text, e.g., SWE-bench Multi- modal (Yang et al., 2024b), which studies whether SE agents generalize to visually grounded tasks. However, no existing benchmark isolates the role of visual evidence at the localization step. Text- only benchmarks such as SWE-bench and its mul- tilingual extensions strip images entirely (Jimenez et al., 2023; Zan et al., 2025). SWE-bench Multi- modal (Yang et al., 2024b) retains images but eval- uates end-to-end patch generation, so localization is folded into repair, and whether an image helped identify the right file or function cannot be read off the final pass/fail signal. Neither setting provides fine-grained localization gold, per-image annota- tions, or paired with-image versus without-image runs. These are the ingredients needed to treat visual evidence as a first-class input to repository- level localization. We introduce M-IssueLoc, a controlled bench- mark and evaluation protocol for multimodal repos- itory level issue localization. Each instance is grounded in a real GitHub issue, its linked pull request, and the repository snapshot before the fix. M-IssueLoc contains 652 issue PR instances across 23 programming languages, with 1,050 is- sue images, file level gold labels for all instances, and function level gold labels for 343 instances. Each image is annotated with one of seven evi- dence categories, such as UI screenshot, error mes- sage, rendering result, code screenshot, log output, and data visualization, as well as a relevance score 1 arXiv:2607.15205v1 [cs.SE] 16 Jul 2026 ImageCategoryTopLanguages RelevanceScore Rule-basedConstruction &HumanReview AnnotationProvenance DifficultyBucket ChangedFiles Easy:1 Medium:2~3 Hard:>4 652File-level 343Function-level 1050AttachedImages 608OpenRepositories 202MedianRepoStars M-IssueLocEvalDataDashboard Figure 1: Dashboard of M-IssueLoc. M-IssueLoc contains 652 file-level instances and 343 function-level instances, drawn from 608 repositories across 23 programming languages, with 1,050 attached issue images. Each panel reports the distribution of one annotation dimension: programming language, image category, relevance score, annotation provenance, and difficulty bucket. indicating whether the image is harmful, neutral, helpful, or vital for localization. The benchmark also includes 55 human reviewed harmful image instances, which serve as a robustness stress test for visually plausible but localization incorrect evi- dence. M-IssueLoc is designed as an evaluation in- strument rather than as a leaderboard for a single model family. It supports four paired input modes: text-only, with-image, Visual Content Evidence (VCE), and VCE+image. These modes allow us to separate the effect of visual information from the effect of raw image conditioning. We evaluate both llm-based and retrieval-based localization sys- tems, including AgentLess, LocAgent, OpenHands, Mini-SWE-Agent, BM25, SweRank, Qwen3-VL- Embedding, and our controlled multimodal re- triever, M-IssueLoc-VL-Embedding (Xia et al., 2024; Chen et al., 2025; Wang et al., 2025; Reddy et al., 2025; Li et al., 2026). We further compare representative systems across SWE-bench Lite, SWE-bench Verified, SWE-bench Multimodal, and M-IssueLoc (Appendix B.4). These cross- benchmark results show that strong localization performance on text-dominant SWE benchmarks does not transfer cleanly to multimodal issue local- ization, which highlights the need for a benchmark that controls visual evidence explicitly. Our findings show that M-IssueLoc exposes a capability gap that is difficult to observe in text dominant benchmarks. On M-IssueLoc, the best file level result reaches 38.96 Acc@5, and the best function level agent result reaches 22.45 Acc@10. Controlled ablations further show that visual evidence can provide useful localization signal: removing images reduces M-IssueLoc- VL-Embedding by 4.44 to 4.91 File@5 points, while VCE improves several agent framework pairs by converting visual content into structured text. These results motivate M-IssueLoc as a con- trolled benchmark for evaluating when and how visual evidence contributes to repository level lo- calization. Our contributions are three-fold: 1) we intro- duce M-IssueLoc, a controlled multimodal issue localization benchmark with 652 real issue PR in- stances, 1,050 images, seven evidence categories, four relevance levels and file/function level gold la- bels; 2) we provide a controlled evaluation protocol with text only, with image, VCE, and VCE+image modes for isolating visual evidence under strict Acc@K metrics; and 3) we evaluate representative agents and retrievers, introduce M-IssueLoc-VL- Embedding as a controlled ablation model, and show that visual evidence offers useful but model dependent localization signal. 2 Related Work Repository-level SE benchmarks. SWE-bench establishe repository issue resolving as a scalable benchmark for automated SE systems (Jimenez et al., 2023). Its later variants and related multi- lingual efforts broaden the issue-resolving ecosys- tem, including multilingual coverage in Multi- 2 SWE-bench (Zan et al., 2025) and larger data- construction efforts such as SWE-smith (Yang et al., 2025). SWE-bench Multimodal is the closest prior artifact to M-IssueLoc: it asks whether SE agents generalize to visual, user-facing JavaScript tasks (Yang et al., 2024b). However, these bench- marks evaluate issue resolving as a whole. M- IssueLoc instead isolates localization, adds per- image relevance labels and harmful-image controls, and reports both file-level and function-level gold across 23 languages. Repository Search and Localization agents. Recent SE agents provide different ways for mod- els to inspect and navigate repositories. SWE- agent studies agent-computer interfaces for soft- ware engineering (Yang et al., 2024a), Agentless argues that much of end-to-end repair can be de- composed into simpler localization and patching components (Xia et al., 2024), and OpenHands exposes a production-oriented SDK for software agents (Wang et al., 2025). Beyond these general software-agent frameworks, recent work has in- creasingly focused on specialized repository search. LocAgent and CoSIL formulate localization as an agentic, iterative search process over repository structure, where the model repeatedly inspects files, functions, and contextual evidence before select- ing suspicious locations (Chen et al., 2025; Jiang et al., 2025). RepoSearcher and CodeScout instead emphasize learned repository search policies, in- cluding reinforcement-learning-based training, to improve how models navigate large codebases and identify relevant code regions (Ma et al., 2025; Sutawika et al., 2026). M-IssueLoc uses these systems as baselines but evaluates only localiza- tion outputs. This design separates localization ability from patch-generation success and from framework-specific submission failures. Code retrieval and embedding models. Re- trieval methods offer lower-latency alternatives to full agents. SWERank and SWERank+ formu- late issue localization as code ranking, including function-level and multilingual settings (Reddy et al., 2025; Gangi Reddy et al., 2025). CodeX- Embed and large-scale code representation learn- ing study general-purpose code embeddings (Liu et al., 2024; Zhang et al., 2024). Multimodal em- bedding models such as Qwen3-VL-Embedding and jina-embeddings-v4 extend dense retrieval to visual and textual inputs (Li et al., 2026; Günther et al., 2025). M-IssueLoc-VL-Embedding builds BenchmarkMultimodalRepo-levelImage relevanceFunction gold SWE-bench / VerifiedNoYes–Partial SWE-bench MultimodalYesYesNoNo Multi-SWE-benchNoYes–Patch-derived LocAgent / Loc-BenchNoYes–Yes M-IssueLocYesYesYesYes Table 1: Positioning of M-IssueLoc. The benchmark is designed to isolate visual evidence in repository-level localization, not to replace end-to-end issue-resolving benchmarks. on this line but serves a different purpose: it is a controlled retriever for testing image-presence, stage-curriculum, and hard-negative design under a fixed benchmark protocol. Its contrastive objective follows the general InfoNCE/CLIP-style retrieval paradigm (Oord et al., 2018; Radford et al., 2021). Multimodal code understanding. Recent work has introduced visual evidence into software- engineering evaluation. MMCode studies visu- ally rich programming problems (Li et al., 2024), while FailureMem and long-horizon multimodal search examine visual evidence in broader repair or search pipelines (Ma et al., 2026; Du et al., 2026). These studies motivate multimodal software rea- soning, but they do not provide a controlled bench- mark for repository-level issue localization. M- IssueLoc complements this line by isolating local- ization from repair and by evaluating how visual evidence affects file- and function-level localiza- tion. 3 Benchmark Construction M-IssueLoc evaluates whether visual evidence in real issue reports helps localize code respon- sible for repository-level failures. To make this measurable, each instance is grounded in a GitHub issue, linked pull request, and fixed repository snap- shot; supports paired evaluation with and without images; includes per-image evidence type and lo- calization relevance annotations; and provides both file-level and function-level gold labels. 3.1 Task Definition Given an issue report and a repository snapshot, the task is to rank code locations by their likelihood of being responsible for the reported failure. Each M-IssueLoc instance is defined as x i = (t i ,b i ,I i ,R i @c i ).(1) wheret i is the issue title,b i is the issue body, I i = i i,1 ,...,i i,n is a set of attached images, 3 andR i @c i denotes the repository snapshot at the base commit of the linked pull request. A system returns either an ordered list of candidate files ˆ F i = [f i,1 ,...,f i,K ],(2) for file-level localization, or an ordered list of qual- ified functions ˆ G i = [g i,1 ,...,g i,K ],(3) for function-level localization. The gold file setF ⋆ i consists of files edited by the human-vetted fixing pull request. For instances with function-level annotations, the gold function setG ⋆ i consists of edited functions that can be matched to functions in the pre-fix repository snap- shotR i @c i . Functions newly introduced by the pull request are excluded, since they are absent from the input repository and therefore cannot be localized. Function-level gold labels are provided only when tree-sitter extraction succeeds and at least one edited pre-existing function can be identi- fied. 3.2 Data Collection and Annotation M-IssueLoc is built from public GitHub repos- itories with at least 50 stars, closed and merged pull requests, non-bot authorship, deduplication, GitHub API enrichment, issue-PR linking, pre/post file retrieval, and complete repository snapshots at the base commit. The released benchmark contains 652 instances, 1,050 localized issue images, and 650 repository snapshots. Its file-level view covers all 652 instances; its function-level view covers 343 instances. Human annotation covers 450 instances by six annotators over approximately 20 hours. Each instance receives one image category among ui_screenshot, behavior_demo, er- ror_message, rendering_result, code_screenshot, data_visualization, and log_output; a relevance score in−1, 0, +1, +2; and a difficulty bucket derived from the number of changed files. The AI-assisted extension adds 202 instances through a four-stage VLM gate:relevance scoring, category classification with confidence threshold, patch-alignment verification, and a second-model consistency check.All AI rows retain anno- tation_by=ai, which enables separate sanity checks. This split is audited in the evaluation rather than treated as bookkeeping. On file Acc@5, Open- Hands Claude scores 36.00 on the human sub- set and 37.13 on the AI subset, while AgentLess Claude scores 27.39 and 28.00. The ranking is stable, suggesting that the AI-assisted extension broadens coverage without dominating the bench- mark behavior. further benchmark construction details are provided in Appendix C. 3.3 Function-Level Gold and Harmful-Image Controls Function-level labels are extracted by intersecting unified-diff line ranges with tree-sitter AST func- tion spans. Functions newly introduced by the PR are excluded from both gold and candidate pools because the task is localization in the base reposi- tory, not prediction of new code. Languages with weak grammar coverage can still be evaluated at file level. Naturally harmful images are rare. In the hu- man pool, images that actively mislead localization occur at less than 1%. M-IssueLoc therefore includes controlled harmful-image synthesis for robustness testing. The synthesis uses three strate- gies: same-repository image-text mismatch, TF- IDF-similar issues with disjoint edit files, and pro- motion of misleading comment-thread screenshots into the issue-body image slot. Each synthetic in- stance is scored along seven dimensions, including text sufficiency, surface relevance, image direction, direction deviation, misleading score, developer impact, and suggested relevance label, followed by human review. 3.4 Training Data and Controlled Retriever The companion M-IssueLoc training data is dis- joint from the benchmark at theinstance_id level and follows the right branch of the pipeline in Figure 2. It contains 13,196 file-level rows over 7,664 instances and 13,769 function-level rows over 3,999 instances. Each anchor is mul- timodal and uses at most two images. Multi- positive expansion emits one row per gold file or function, aligning training with the multi-gold evaluation protocol. For each anchor, hard nega- tives (Robinson et al., 2021) are mined from same- repository non-gold candidates ranked by Qwen3- VL-Embedding-8B, so the model contrasts the true target with plausible repository-local distractors. M-IssueLoc-VL-Embedding is initialized from Qwen3-VL-Embedding-2B or 8B. The query con- tains issue text and images; candidates are file or function strings prefixed withFile:orFunction:. 4 GithubSource PR/IssuePairs (star≥50,prclosed, Non-bot) DataCollectionPipeline (GithubCrawl) 1.Coarse Filter 2.Deduplication 4.Merge PR/Issue 3.APIFetch Diff,Patch,Images 5.ContentDownload Pre/PostFileContents 6.RepoSnapshot Atbasecommit Annotation&SynthesisProcess Human-AnnotationTrack 450cleanedinstances Rule-based harmfulimage entries (Relevance= -1) 1.TitleMismatch 2.Disjoint“diff_files” 3.ImageSwap Human- Verified M-IssueLocEvalSet (652instances) AI-drivenAnnotationTrack (202instances) A:RelevanceB:Category D:2ndVLM Judge C:PatchAlign Four-stageVLMgate •7-classtaxonomy •4-levelrelevancescore •File&FunctionLevelGold •DifficultyBuckets Evaluation Image-usetoggle: Text-only/Image M-IssueVCESide-Channel Structured-textImageOutput M-IssueLocEmbedding TrainingDataPipeline Retrieval-based Retrievers LLM-based Agents CodeLocalizationParadigm Evaluation Tools: StrictAcc@k LeakagePrevention Separationfromevalset HardNegativeMining Anchor=Issue+Image Positive=Goldfile/function Negative=In-repo negatives via cosine ranking (strong VL- Embedding) Two-stageCurriculum Stage1 ：File-level contrastivetraining (InfoNCEloss) Stage2 ：Function-level continuation, initialised from Stage 1 (InfoNCEloss) File-levelGold Function-level Gold Figure 2: End-to-end construction and evaluation flow for M-IssueLoc. The benchmark separates human and AI annotation, explicit harmful-image controls, file/function-level gold extraction, multimodal training data construction, and controlled image-conditioning evaluation modes. Letq i ,p i , andn ij denote the normalized query, pos- itive, and hard-negative embeddings. We optimize L i =− log exp(q ⊤ i p i /τ ) exp(q ⊤ i p i /τ ) + P j exp(q ⊤ i n ij /τ ) , withτ = 0.05. Training follows a file-to-function curriculum: Stage 1 fine-tunes on file-level rows, and Stage 2 continues on function-level rows. Overall, M-IssueLoc contains 652 issue in- stances, 1,050 images, and covers 24 programming languages. All instances support file-level evalu- ation, and 343 also support function-level evalua- tion. Each instance records annotation provenance, image evidence category, image relevance score, and difficulty, enabling controlled analyses by lan- guage, visual evidence type, relevance, annotation source, and search-space size. Figure 1 summa- rizes these benchmark statistics, and the released JSONL files contain the same fields used in evalua- tion, making the statistics directly reproducible. 4 Experimental Setup 4.1 Baselines We evaluate two representative paradigms for is- sue localization: retrieval-based localization and llm-based localization. These baselines cover both direct relevance ranking and interactive repository exploration, allowing us to evaluate whether vi- sual evidence improves localization under different levels of search and reasoning complexity. Retrieval-based Methods. This paradigm di- rectly ranks repository files or functions based on relevance score, including BM25 (Robertson and Zaragoza, 2009), bge-m3 (Chen et al., 2024), SWERank-small, SWERank-large (Reddy et al., 2025), Qwen3-VL-Embedding-2B/8B (Li et al., 2026), and M-IssueLoc-VL-Embedding-2B/8B. Llm-based methods explore the repository with tool use before returning localized code locations, including AgentLess (Xia et al., 2024), LocA- gent (Chen et al., 2025), OpenHands (Wang et al., 2025), and Mini-SWE-Agent (Yang et al., 2024a) with frontier and open-weight multimodal models. All methods are scored by the same evaluation har- ness. Agents are constrained to output strict JSON for file/function localization and are not allowed to generate patches or modify repositories. For multimodal benchmarks, we evaluate four in- put modes. The original mode passes issue text and raw images. The text-only mode removes images. The +VCE mode replaces images with a structured visual text block. The +VCE+image mode passes both the structured text and raw images. 4.2 Visual Context Extraction (VCE) VCE is a non-trainable image-to-text adapter. For each image, a VLM extracts structured fields such as OCR text, error signals, UI elements, user ac- tions, code hints, saliency, confidence, and notes. The fields are rendered as a compact textual block and appended to the issue body. The text-only to 5 +VCE comparison measures the value of structured visual content, while original to +VCE+image mea- sures whether raw pixels add further signal. See Appendix D for the VCE workflow and case stud- ies. 4.3 Evaluation Granularities and Metrics The harness evaluates both file-level (n = 652) and function-level (n = 343) localization. Following prior localization work (Chen et al., 2025), we use strict Acc@K as the primary metric: Acc@K = 1 N N X i=1 1 [G ⋆ i ⊆ TopK(r i )].(4) A prediction is successful only if all gold files or qualified functions appear in the top-Kranked list. This all-gold criterion is important for multi- location issues, where finding only one correct lo- cation may still fail to recover the full edit scope. Function-level results are reported atK ∈5, 10, since strict Acc@1 and Acc@3 are necessarily zero whenever an instance contains more than one gold function. We also report MRR, Recall@K, and Hit@K as supporting metrics. 5 Results and Analysis 5.1 RQ1: How Well Do Current Systems Localize Multimodal Repository Issues? RQ1 establishes the capability boundary of cur- rent systems. As shown in Table 2, multimodal repository-level issue localization remains far from solved. The strongest file-level result is obtained by OpenHands with GPT-5.2, which reaches 38.96 File@5, while OpenHands with Claude-Sonnet-4.6 achieves the best File@1 and File@3. However, even the best system still misses all correct files in the top five for more than 60% of instances. This suggests that current systems can provide use- ful candidates, but are not yet reliable enough for downstream repair-oriented workflows. The results further reveal a clear paradigm dif- ference. LLM-based agents are stronger at file- level localization, likely because they can inspect the repository and follow issue-level clues during exploration. Retrieval-based methods are more competitive at function-level localization: M- IssueLoc-VL-Emb-8B achieves the best Func@10 score of 33.86. This advantage is partly structural. Retrievers rank pre-enumerated functions that are independently encoded, so their outputs are nat- urally aligned with the evaluation space. In con- trast, LLM-based systems must generate function identifiers as text, where incomplete names, miss- ing qualifiers, or invalid output formats can reduce exact-match scores. Figure 3 further explains the aggregate results. First, performance collapses with difficulty. Open- Hands GPT-5.2 drops from 83.10 Acc@10 on easy instances to 2.84 on hard instances, while M- IssueLoc-VL-Emb-8B drops from 74.18 to 3.98, showing that multi-edit issues remain the main bottleneck. Second, language slices reveal non- uniform localization behavior. Both systems per- form better on JavaScript and related frontend lan- guages, where screenshots often provide direct UI or rendering evidence, but performance is weaker on languages such as C++ and Go, where the visual signal is less directly tied to localized code. Third, agent results are also shaped by harness reliability. With the same Claude-Sonnet-4.6 backend, Open- Hands achieves a 98.6% valid submission rate and 37.42 File@5, whereas Mini-SWE-Agent achieves 47.1% and 18.71. Thus, RQ1 shows that current systems are limited not only by model capability, but also by difficulty, language properties, and final- output reliability. 5.2 RQ2: Is Visual Evidence Useful, and Do Systems Use It Reliably? RQ2 separates visual evidence from visual use. M-IssueLoc is designed for this distinction: each image is annotated with evidence category and rel- evance, and the harmful subset introduces visually plausible but localization-incorrect cues. Therefore, the no-image ablation evaluates whether systems exploit image-conditioned localization evidence, rather than whether images are merely attached to issue reports. Cross-benchmark results further motivate this question. As reported in Appendix B.4, current systems already achieve strong localization perfor- mance on text-dominant SWE benchmarks. For example, OpenHands reaches 94.53 File@5 on SWE-bench-Lite and 90.20 File@5 on SWE-bench- Verified. However, performance drops substantially on SWE-bench-M, where the best File@5 is 43.14. This gap suggests that repository-level local- ization is relatively mature in text-only settings, but remains much less developed when issue evidence involves images. M-IssueLoc fills this diagnos- tic gap by providing controlled image relevance, 6 ParadigmModelFile@1File@3File@5Func@5Func@10 LLM-based localization OpenHandsGPT-5.223.9336.3538.9621.8722.45 OpenHandsClaude-Sonnet-4.625.6136.6637.4220.4120.41 OpenHandsKimi-K2.524.2335.2835.7420.1220.12 AgentLessKimi-K2.518.1528.8332.7412.1412.14 AgentLessGemini-3-Pro17.8727.7030.1112.9212.92 LocAgentQwen3.5-122B21.3625.3925.7012.9812.98 Mini-SWE-AgentClaude-Sonnet-4.615.1818.7118.7110.7911.08 Retrieval-based localization BM25Text6.7516.5621.0112.5419.83 bge-m3-0.5BText7.3614.8820.4018.6626.24 SWERank-small-0.1BText3.837.9810.7419.8326.24 SWERank-large-7BText10.1221.3226.6920.9527.82 Qwen3-VL-Emb-2BMultimodal11.6623.1629.2924.2030.90 Qwen3-VL-Emb-8BMultimodal12.1126.6932.0625.3632.65 M-IssueLoc-VL-Emb-2BMultimodal13.1925.6132.0625.3632.36 M-IssueLoc-VL-Emb-8BMultimodal13.96 27.6132.8222.7433.86 Table 2: Overall comparison on M-IssueLoc. File metrics are Acc@1/3/5, and function metrics are Acc@5/10. Values are percentages. Bold numbers indicate the best result in each column, and underlined numbers indicate the best retrieval-based file-level results. EasyMediumHard 0 10 20 30 40 50 60 70 80 90 Acc@10 (%) 83.10 25.86 2.84 74.18 31.94 3.98 A Difficulty collapse OpenHands GPT-5.2 M-IssueLoc-VL-Embedding-8B TypeScript Python JavaScript C++ Java Go C# 0 10 20 30 40 File Acc@5 (%) B Language sensitivity OpenHands GPT-5.2 M-IssueLoc-VL-Emb-8B 405060708090100 Valid final submission rate (%) 10 15 20 25 30 35 40 File Acc@5 (%) OpenHands harness (Claude-Sonnet-4.6) (98.6%, 37.42) Mini-SWE-Agent harness (Claude-Sonnet-4.6) (47.1%, 18.71) C Agent harness reliability Figure 3: Current capability frontier on M-IssueLoc. Agents are stronger at broad file discovery, retrievers are competitive at function ranking, hard instances collapse across paradigms, and language slices expose additional sensitivity that aggregate scores obscure. category, and harmful-image annotations. The main results also indicate that M-IssueLoc contains usable visual signal. In Table 2, text-only retrievers reach 19.83–27.82 Func@10, whereas M-IssueLoc-VL-Embedding-8B achieves 33.86 Func@10. This shows that visual evidence can be converted into fine-grained localization signal when a retriever is trained for multimodal issue lo- calization. Detailed retriever ablations are reported in Appendix B.2. Table 3 reports the controlled no-image ablation. We define ∆ img (m) = Acc@5 file (m, text-only) − Acc@5 file (m, text+image), so negative values indicate that removing images hurts. The results show uneven image use across systems. AgentLess GPT-5.2 benefits from images (∆ img = −2.66), while OpenHands changes by less than one point. M-IssueLoc-VL-Embedding is more image-sensitive: removing images reduces File@5 by 4.91 points for the 2B model and 4.44 points for the 8B model. Thus, visual evidence is useful, but current agent systems do not exploit it consistently. Fine-grained slices explain why aggregate image gains are unstable. Relevance labels do not induce monotonic accuracy changes, since harmful im- ages are not always recognized as misleading and some instances remain solvable from text alone. Category slices further show that images are most helpful when they are directly tied to observable 7 MethodModelNo-image File@5 With-image File@5 ∆ File@5 AgentLessGPT-5.226.9329.59-2.66 OpenHands GPT-5.238.6538.96-0.31 OpenHands Claude-Sonnet-4.638.3437.42+0.92 RetrieverMM-IssueLoc-VL-Embed-2B27.1532.06-4.91 RetrieverMM-IssueLoc-VL-Embed-8B28.3832.82-4.44 Table 3: With-image versus no-image ablation on M-IssueLoc.∆is no-image minus with-image file Acc@5. Negative values mean that removing images hurt the cell. software behavior, such as UI screenshots, behavior demos, rendering results, code screenshots, logs, or data visualizations. These signals often help identify relevant files, but they do not always map cleanly to edited functions. Additional relevance, category, difficulty, and annotation slices are re- ported in Appendix B.1. 5.3RQ3: Does Localization Depend on Image Content or Raw Image Format? RQ3 diagnoses whether the benefit of visual evi- dence comes from image content or from the raw image format. We use Visual Content Evidence (VCE)(fig. 6) to convert each image into structured textual evidence. The +VCE setting removes raw images and provides only structured visual text, while the +VCE+image setting provides both VCE and the original image. Thus, the gain from the original setting to +VCE indicates whether image content is useful after being textualized, and the difference between +VCE and +VCE+image mea- sures the residual value of raw pixels. Table 4 reports the VCE diagnostic results using the same three model backends under AgentLess and LocAgent. VCE improves file-level localiza- tion in several settings. For AgentLess, Claude- Sonnet-4.6 improves from 25.15 original File@5 to 28.53 with +VCE, and GPT-5.2 improves from 28.15 to 29.07. For LocAgent, Qwen3.5-122B- A10B improves substantially from 25.70 to 35.04. These results show that images contain useful lo- calization evidence and that such evidence can be consumed in structured textual form. However, raw images do not consistently add value once VCE is provided. We define the raw- pixel residual as ρ pix = Acc@5 f (VCE+img)− Acc@5 f (VCE). For AgentLess, the residuals are small:+0.22for Claude-Sonnet-4.6,+0.03for GPT-5.2, and−2.15 for Qwen3.5-122B-A10B. In contrast, LocAgent exhibits much larger variation: Claude-Sonnet-4.6 drops by 5.62 points, GPT-5.2 gains 4.74 points, and Qwen3.5-122B-A10B drops by 11.69 points. This indicates that VCE exposes useful image con- tent, while the effect of raw pixels is mediated by the framework’s visual-input integration. 6 Discussion, Limitations, and Future Work Discussion. M-IssueLoc follows prior issue- resolving benchmarks by using merged pull request diffs as localization gold labels. Although such labels are not minimal causal explanations, they provide a scalable and auditable proxy for code locations modified by human developers. We there- fore use strict all-gold Acc@K to measure whether a method recovers the full edit scope, and provide function-level labels when tree-sitter can reliably map edits to pre-existing functions. We also release per-instance predictions to support reanalysis under alternative scoring rules. Limitations. M-IssueLoc is designed for con- trolled diagnosis, and its results should be inter- preted within this scope. The harmful-image subset is primarily rule-constructed and human-reviewed, so it serves as a robustness stress test rather than an estimate of real-world frequency. VCE is a diagnos- tic protocol, not an upper bound on visual reason- ing, since it uses fixed extraction fields and a fixed downstream localization setting. Function-level la- bels may be noisy for language features that are difficult for tree-sitter to resolve, such as macros, templates, or generated code. Our retriever also uses at most two images per issue, leaving many- image issues as a future stress case. Finally, M- IssueLoc evaluates localization only, rather than end-to-end software repair or downstream patch generation. Future work. Future work can extend M- IssueLoc along three directions: collecting more 8 MethodModelOriginal+VCE+VCE+Imageρ pix AgentLessClaude-Sonnet-4.625.1528.5328.750.22 AgentLessGPT-5.228.1529.0729.100.03 AgentLessQwen3.5-122B-A10B28.0726.5324.38-2.15 LocAgentClaude-Sonnet-4.627.7330.9325.31-5.62 LocAgentGPT-5.219.1316.1220.864.74 LocAgentQwen3.5-122B-A10B25.7035.0423.35-11.69 Table 4: VCE diagnostic results on M-IssueLoc, measured by file-level Acc@5. VCE converts image content into structured text, while ρ pix measures the residual value of raw pixels after VCE has already been supplied. naturally occurring misleading images and many- image issues, improving function-level annotation with language-specific parsers or hybrid static anal- ysis, and studying whether better multimodal lo- calization improves broader software-engineering workflows such as repository understanding, de- bugging, triage, and downstream repair. 7 Conclusion M-IssueLoc shows that multimodal repository- level issue localization is a distinct and under- evaluated capability. Cross-benchmark results in- dicate that strong localization performance on text- dominant SWE benchmarks is not sufficient ev- idence of multimodal issue localization ability. By pairing real issue-PR instances with file-level and function-level gold labels, image-category and relevance annotations, and controlled text-only, with-image, VCE, and VCE+image modes, M- IssueLoc makes visual evidence an explicit evalua- tion variable. Our results show that visual evidence can provide useful localization signal, but current agents and retrievers use it unevenly. Overall, M- IssueLoc provides a controlled basis for evaluating how visual evidence affects repository-level local- ization. References Jianlv Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. 2024. Bge m3-embedding: Multi-lingual, multi-functionality, multi-granularity text embeddings through self-knowledge distillation. arXiv preprint arXiv:2402.03216, 4(5). Zhaoling Chen, Robert Tang, Gangda Deng, Fang Wu, Jialong Wu, Zhiwei Jiang, Viktor Prasanna, Arman Cohan, and Xingyao Wang. 2025. Locagent: Graph- guided llm agents for code localization. In Proceed- ings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pa- pers), pages 8697–8727. Yifan Du, Zikang Liu, Jinbiao Peng, Jie Wu, Junyi Li, Jinyang Li, Wayne Xin Zhao, and Ji-Rong Wen. 2026. Towards long-horizon agentic multimodal search. arXiv preprint arXiv:2604.12890. Revanth Gangi Reddy, Ye Liu, Wenting Zhao, JaeHyeok Doo, Tarun Suresh, Daniel Lee, Caiming Xiong, Yingbo Zhou, Semih Yavuz, and Shafiq Joty. 2025. Swerank+: Multilingual, multi-turn code ranking for software issue localization. arXiv e-prints, pages arXiv–2512. Michael Günther, Saba Sturua, Mohammad Kalim Akram, Isabelle Mohr, Andrei Ungureanu, Bo Wang, Sedigheh Eslami, Scott Martens, Maximilian Werk, Nan Wang, and 1 others. 2025. jina-embeddings-v4: Universal embeddings for multimodal multilingual retrieval. In Proceedings of the 5th Workshop on Multilingual Representation Learning (MRL 2025), pages 531–550. Zhonghao Jiang, Xiaoxue Ren, Meng Yan, Wei Jiang, Yong Li, and Zhongxin Liu. 2025. Issue localization via llm-driven iterative code graph searching. In 2025 40th IEEE/ACM International Conference on Automated Software Engineering (ASE), pages 3034– 3045. IEEE. Carlos E Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik R Narasimhan. 2023. Swe-bench: Can language mod- els resolve real-world github issues? In The twelfth international conference on learning representations. Kaixin Li, Yuchen Tian, Qisheng Hu, Ziyang Luo, Zhiy- ong Huang, and Jing Ma. 2024. Mmcode: Bench- marking multimodal large language models for code generation with visually rich programming problems. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 736–783. Mingxin Li, Yanzhao Zhang, Dingkun Long, Ke- qin Chen, Sibo Song, Shuai Bai, Zhibo Yang, Pengjun Xie, An Yang, Dayiheng Liu, and 1 oth- ers. 2026.Qwen3-vl-embedding and qwen3-vl- reranker: A unified framework for state-of-the-art multimodal retrieval and ranking. arXiv preprint arXiv:2601.04720. Ye Liu, Rui Meng, Shafiq Joty, Silvio Savarese, Caim- ing Xiong, Yingbo Zhou, and Semih Yavuz. 2024. Codexembed: A generalist embedding model family for multiligual and multi-task code retrieval. arXiv preprint arXiv:2411.12644. 9 Ruize Ma, Yilei Jiang, Shilin Zhang, Zheng Ma, Yi Feng, Vincent Ng, Zhi Wang, Xiangyu Yue, Chuanyi Li, and Lewei Lu. 2026. Failuremem: A failure-aware multimodal framework for autonomous software repair. arXiv preprint arXiv:2603.17826. Zexiong Ma, Chao Peng, Qunhong Zeng, Pengfei Gao, Yanzhen Zou, and Bing Xie. 2025. Tool-integrated reinforcement learning for repo deep search. arXiv preprint arXiv:2508.03012. Aaron van den Oord, Yazhe Li, and Oriol Vinyals. 2018. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748. Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sas- try, Amanda Askell, Pamela Mishkin, Jack Clark, and 1 others. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748–8763. PmLR. Revanth Gangi Reddy, Tarun Suresh, JaeHyeok Doo, Ye Liu, Xuan Phi Nguyen, Yingbo Zhou, Semih Yavuz, Caiming Xiong, Heng Ji, and Shafiq Joty. 2025. Swerank: Software issue localization with code ranking. arXiv preprint arXiv:2505.07849. Stephen Robertson and Hugo Zaragoza. 2009. The prob- abilistic relevance framework: BM25 and beyond, volume 4. Now Publishers Inc. Joshua David Robinson, Ching-Yao Chuang, Suvrit Sra, and Stefanie Jegelka. 2021. Contrastive learning with hard negative samples. In International Conference on Learning Representations. Lintang Sutawika, Aditya Bharat Soni, Apurva Gandhi, Taha Yassine, Sanidhya Vijayvargiya, Yuchen Li, Xuhui Zhou, Yilin Zhang, Leander Melroy Maben, Graham Neubig, and 1 others. 2026. Codescout: An effective recipe for reinforcement learning of code search agents. arXiv preprint arXiv:2603.17829. Xingyao Wang, Simon Rosenberg, Juan Michelini, Calvin Smith, Hoang Tran, Engel Nyst, Rohit Malho- tra, Xuhui Zhou, Valerie Chen, Robert Brennan, and 1 others. 2025. The openhands software agent sdk: A composable and extensible foundation for production agents. arXiv preprint arXiv:2511.03690. Chunqiu Steven Xia, Yinlin Deng, Soren Dunn, and Lingming Zhang. 2024.Agentless: Demystify- ing llm-based software engineering agents. arXiv preprint arXiv:2407.01489. John Yang, Carlos E Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, and Ofir Press. 2024a. Swe-agent: Agent-computer interfaces enable automated software engineering. Advances in Neural Information Processing Systems, 37:50528– 50652. John Yang, Carlos E Jimenez, Alex L Zhang, Kil- ian Lieret, Joyce Yang, Xindi Wu, Ori Press, Niklas Muennighoff, Gabriel Synnaeve, Karthik R Narasimhan, and 1 others. 2024b. Swe-bench multi- modal: Do ai systems generalize to visual software domains? In The Thirteenth International Confer- ence on Learning Representations. John Yang, Kilian Lieret, Carlos E Jimenez, Alexan- der Wettig, Kabir Khandpur, Yanzhe Zhang, Binyuan Hui, Ofir Press, Ludwig Schmidt, and Diyi Yang. 2025. Swe-smith: Scaling data for software engineer- ing agents. In The Thirty-ninth Annual Conference on Neural Information Processing Systems Datasets and Benchmarks Track. Daoguang Zan, Zhirong Huang, Wei Liu, Hanwu Chen, Shulin Xin, Linhao Zhang, Qi Liu, Aoyan Li, Lu Chen, Xiaojian Zhong, and 1 others. 2025. Multi- swe-bench: A multilingual benchmark for issue re- solving. In The Thirty-ninth Annual Conference on Neural Information Processing Systems Datasets and Benchmarks Track. Dejiao Zhang, Wasi Ahmad, Ming Tan, Hantian Ding, Ramesh Nallapati, Dan Roth, Xiaofei Ma, and Bing Xiang. 2024. Code representation learning at scale. arXiv preprint arXiv:2402.01935. A Training and Evaluation Setup A.1 Training Hyperparameters We report the training details of M-IssueLoc-VL- Embedding in the multimodal setting, where each input contains the issue text and attached images. Table 5 summarizes the two-stage protocol and the function-only ablation. Stage 1 uses file-level su- pervision for coarse localization, while Stage 2 con- tinues from the Stage-1 checkpoint with function- level supervision. The function-only ablation di- rectly trains on function-level data to assess the effect of file-level initialization. Table 6 lists the shared optimization settings. We use full fine-tuning with the vision encoder frozen to reduce training cost while preserving pretrained visual representations. Table 7 reports the hardware and batch configu- rations. All experiments use four NVIDIA H200 GPUs. The effective batch size equals the per- GPU batch size multiplied by gradient accumula- tion steps and the number of GPUs. A.2 Agent Evaluation Parameters We configure each agent framework following the setup reported in its original paper, and restrict every framework to localization-only output on a shared evaluation harness. 10 Table 5: Training configurations for the proposed two-stage training and function-only ablation. SettingData levelInitializationEpochsLRSeed Stage-1 FileFile-levelQwen3-VL-Embedding21× 10 −5 42 Stage-2 FunctionFunction-levelStage-1 checkpoint35× 10 −6 43 Function-only AblationFunction-levelQwen3-VL-Embedding31× 10 −5 42 Table 6: Shared optimization hyperparameters. HyperparameterValue Max sequence length8192 Contrastive temperature0.05 Warmup ratio0.05 Logging steps10 Checkpoint interval200 steps Vision encoderFrozen Training strategyFull fine-tuning Image input in M settingEnabled Image input in text-only settingDisabled Table 7: Hardware and batch configurations for different model sizes. ModelGPUsBatch/GPUAccum. StepsGlobal Batch 2B4×H20048128 8B4×H20011664 OpenHands uses a CodeAct-style agent equipped with a terminal, a file editor, and an atomicfinish(message=...)submission action. Each run is capped at 10 iterations, with a follow-up “produce final JSON now” nudge issued when no answer is emitted. Mini-SWE-Agent uses a bash-only interface, a 10-step limit, and a heredoc-to-file final submission protocol. Both frameworks share the same repository snapshots, scoring code, retry policy for transient failures, and localization-only output schema. The observed gap between OpenHands and Mini-SWE-Agent is therefore interpreted as a framework-level submission and tool-surface effect, rather than a pure model-capability difference. B Extended Results B.1 Stratified Evaluation Slices Table 8 reports the relevance slice used in RQ2. The non-monotonic pattern is visible across sys- tems: vital images are not always easier than help- ful images, and harmful images are not automati- cally detected as misleading. Table 9 reports selected category slices. Text- rich categories such as error-message and code screenshots are easier to convert into localization cues than behavior demos or visualizations, but file-level and function-level trends can diverge. SystemVitalHelpfulNeutralHarmful OpenHands GPT-5.236.1440.1621.4334.55 Qwen3-VL-Embed-8B35.1529.6621.4332.73 M-IssueLoc-VL-Emb-8B30.2032.2828.5734.55 Table 8: File Acc@5 by image relevance score. Harm- ful rows are a robustness stress test and should not be interpreted as a natural-frequency estimate. CategoryOpenHands GPT-5.2Qwen3-VL-Emb-8BMM-IssueLoc-VL-Emb-8B UI screenshot39.5531.0730.51 Behavior demo40.4040.4036.36 Rendering result40.0024.7125.88 Code screenshot36.9033.3336.90 Log output37.8827.2730.30 Data visualization32.0020.0028.00 Table 9:Representative category slices on M- IssueLoc. Values are File Acc@5 percentages. The selected categories illustrate that visual evidence is most useful when it is directly tied to UI behavior, rendering results, code screenshots, logs, or visualized outputs. Tables 10 and 11 give the language and annotation-provenance checks referenced in the main text. The human and AI-assisted subsets pre- serve the same broad ordering for representative agent baselines, while language slices reveal larger variation for retrievers. SystemTypeScript Python JavaScriptC++JavaGoC#Rust OpenHands GPT-5.238.4136.5142.50 31.11 39.53 36.36 42.42 40.74 M-IssueLoc-VL-Emb-8B31.7932.5436.67 22.22 30.23 12.12 27.27 40.74 Table 10: File Acc@5 by programming language for two representative systems. C# has no function-level gold in the released function subset. B.2 Controlled Retriever Training Ablations ThemainpaperusesMM-IssueLoc-VL- Embedding as a controlled probe of image- conditioned retrieval. Figure 4 and Tables 12 and 13 provide the supporting training ablations. These results are not the central contribution of the benchmark, but they verify that the con- trolled retriever is sensitive to curriculum and image-conditioned training choices. 11 SystemHuman File@5AI-annotation File@5 OpenHands Claude-Sonnet-4.636.0037.13 AgentLess Claude-Sonnet-4.627.3928.00 Table 11: Annotation-provenance sanity check. The AI- annotation extension does not dominate the benchmark behavior for representative agent baselines. Stage1 file Function only Full curriculum 20.0 22.5 25.0 27.5 30.0 32.5 35.0 37.5 Acc@K (%) A. Curriculum ablation 2B File@5 2B Func@10 8B File@5 8B Func@10 2B text training 2B full multimodal 8B text training 8B full multimodal 20 22 24 26 28 30 32 34 36 38 Acc@K (%) B. Image-conditioned training File@5 Func@10 Vital File@5 Figure 4: Controlled retriever training ablations. The full curriculum improves function ranking over file-only training, and image-conditioned training improves the strongest 8B function and vital-image cells. B.3 External Image Ablation on SWE-bench-M We additionally compare with-image and no-image localization on SWE-bench-M. This experiment serves as an external sanity check for RQ2, rather than primary evidence for visual contribution, be- cause SWE-bench-M does not provide controlled image relevance, category, or harmful-image anno- tations. Table 14 shows that image effects on SWE- bench-M are also mixed. Mini-SWE-Agent with Claude-Sonnet-4.6 benefits substantially from images, while OpenHands changes only slightly. Some methods even perform better without images, suggesting that current systems do not consistently exploit the visual channel. This trend is consistent with M-IssueLoc, but M-IssueLoc remains the primary benchmark for RQ2 because it explicitly controls visual evidence. B.4 Cross-benchmark Localization Results We further evaluate representative systems on SWE-bench-Lite, SWE-bench-Verified, and SWE- bench-M. This experiment is used to contextual- ize M-IssueLoc rather than as primary evidence for RQ2, since only M-IssueLoc provides con- trolled image relevance and category annotations. Since none of these three benchmarks ship with lo- calization gold labels, we follow prior work (Chen et al., 2025) and derive file- and function-level lo- calization ground truth from the repair patches re- leased with each benchmark. Specifically, we take the files and functions modified by the provided fixing patch as the gold set, which allows us to Backbone StageFile@1 File@3 File@5 Func@10 2Bstage1-file14.4226.5332.3627.41 2Bfn-only10.8921.9327.9130.03 2Bfulltrain13.1925.6132.0632.36 8Bstage1-file15.1829.1435.1220.70 8Bfn-only11.9624.0829.1429.15 8Bfulltrain13.9627.6132.8233.86 Table 12: Retriever curriculum-stage ablation. ModelFile@5Func@10Vital File@5Helpful File@5 2B text-only training30.8331.2026.2432.28 2B full multimodal32.0632.3628.7133.86 8B text-only training32.2127.1128.2233.07 8B full multimodal32.8233.8631.1933.33 Table 13: Training-data image ablation for M- IssueLoc-VL-Embedding. score the localization outputs of each system un- der the same strict Acc@K protocol used on M- IssueLoc. Table 15 shows a clear gap between text- dominant and multimodal localization. On SWE- bench-Lite and SWE-bench-Verified, current agent systems already achieve strong localization perfor- mance. For example, OpenHands reaches 94.53 File@5 on SWE-bench-Lite and 90.20 File@5 on SWE-bench-Verified. However, the same family of systems drops substantially on SWE-bench-M, where the best File@5 is 43.14. This contrast sug- gests that current systems are relatively mature for text-based SWE localization, but still struggle when localization requires interpreting multimodal issue evidence. Thus, SWE-bench-M provides additional evidence that multimodal repository- level localization remains underdeveloped, while M-IssueLoc offers the controlled annotations needed to diagnose this gap. C Benchmark Details C.1 Released Views and Statistics M-IssueLoc contains 652 canonical instances. The file-level view contains all 652 rows, and the function-level view contains 343 rows with non- empty extracted edit functions. The annotation split is 450 human and 202 AI-assisted rows. The difficulty split is 214 easy, 263 medium, and 176 hard instances. The largest language groups are TypeScript (151), Python (126), JavaScript (120), C++ (45), Java (44), Go (33), C# (33), Rust (27), C (21), and PHP (20), with 23 languages in total. 12 MethodModelNo-image File@5 With-image File@5 ∆ File@5 With-image Func@10 AgentLessClaude-Sonnet-4.630.3935.29-4.9015.38 OpenHandsClaude-Sonnet-4.645.1043.14+1.9628.85 OpenHandsGPT-5.241.1842.16-0.9832.69 Mini-SWE-AgentClaude-Sonnet-4.613.7322.55-8.8217.31 M-IssueLoc-VL-Embedding 8B29.4128.43+0.9813.46 Table 14: External image ablation on SWE-bench-M.∆is no-image minus with-image File@5, so negative values indicate that removing images hurts. Values are Acc@K percentages. Benchmark Method / ModelFile@1 File@3 File@5 Func@5 Func@10 SWE-bench-Lite AgentLess / Claude-Sonnet-4.678.4789.4291.2470.4470.80 OpenHands / Claude-Sonnet-4.687.5994.1694.1679.2079.20 OpenHands / GPT-5.283.2193.4394.5374.0974.45 Mini-SWE-Agent / Claude-Sonnet-4.672.9973.7273.7263.8763.87 SWE-bench-Verified AgentLess / Claude-Sonnet-4.669.6084.4086.8051.4552.34 OpenHands / Claude-Sonnet-4.677.8087.4088.2059.4759.69 OpenHands / GPT-5.274.4088.4090.2057.0257.68 Mini-SWE-Agent / Claude-Sonnet-4.665.6068.8068.8049.4449.44 SWE-bench-M with images AgentLess / GPT-5.214.7128.4333.3315.3817.31 OpenHands / Claude-Sonnet-4.629.4141.1843.1428.8528.85 OpenHands / GPT-5.224.5137.2542.1632.6932.69 Mini-SWE-Agent / Claude-Sonnet-4.618.6322.5522.5517.3117.31 M-IssueLoc-VL-Embedding / 8B16.6723.5328.4311.5413.46 Table 15: Cross-benchmark localization results. SWE-bench-M is evaluated with images, while SWE-bench-Lite and SWE-bench-Verified are text-only benchmarks. Values are Acc@K percentages. CategoryUIBehaviorErrorRenderingCodeLogData-viz Instances177999285846650 Table 16: Image category distribution. Categories are mutually exclusive at the instance level. RelevanceVital (+2)Helpful (+1)Neutral (0)Harmful (-1) Instances2023811455 Table 17: Per-image relevance distribution used for strat- ified evaluation. C.2 Benchmark Examples Figure 5 presents representative examples from M-IssueLoc, organized by the image categories defined in Table 16. Each example illustrates the issue context, the associated visual evidence, and the corresponding gold annotations at both file and function levels. C.3 AI Annotation Gate The AI-assisted extension uses four stages. Stage A estimates relevance and admits only +1 or +2 rows for the main augmented subset. Stage B assigns one of seven image categories with confidence at least 0.6. Stage C judges whether the proposed patch plausibly addresses the issue. Stage D repeats relevance and category with a second VLM and accepts only if relevance differs by at most one level and category matches. The accepted rows are marked with annotation_by=ai. C.4 Harmful-Image Synthesis The harmful subset uses same-repository image- text mismatches, TF-IDF-similar issues with dis- joint edit files, and comment-thread image promo- tion. Each candidate receives a seven-axis quality assessment: text sufficiency, surface relevance, im- age direction, direction deviation, misleading score, developer impact, and suggested label. Human re- view filters implausible or insufficiently misleading candidates. D Visual Content Evidence Extraction D.1 VCE Mechanism Visual Content Evidence (VCE) converts issue im- ages into structured textual evidence before local- ization. Instead of directly using the raw image 13 as model input, VCE extracts task-relevant visual content, such as error messages, UI states, code snippets, logs, rendering artifacts, and chart pat- terns, into a compact textual representation. This design provides a diagnostic side channel for sepa- rating the contribution of visual content from the effect of direct image conditioning. In our eval- uation, VCE is used to test whether localization improvements come from information contained in the image or from the model’s ability to process the image modality itself. D.2 VCE Extraction Examples Figure 7 shows representative image-to-VCE ex- traction examples across different image categories. Each example contains the original visual evidence, the extracted VCE text, and its corresponding inter- pretation for file-level and function-level localiza- tion. D.3 VCE Prompt The full prompt template is provided in Figure 8. E Prompt Templates E.1 AI-Driven Harmful Impact Prompt The full prompt template is provided in Figure 9. E.2 AgentLess Prompt The full prompt template is provided in Figure 10. E.3 LocAgent Prompt The full prompt template is provided in Figure 11. E.4 OpenHands Prompt The full prompt template is provided in Figure 12. E.5 Mini-SWE-Agent Prompt The full prompt template is provided in Figure 13. 14 Issuetitle:Landing page as per new designs Body: I have worked on the landing page , and created a pull request for the same. <image> GT file: "client/public/index.html", "client/src/Themes/Theme.js", "client/src/components/App.js", "client/src/providers/DarkThemeProvider.jsx" Image Category: Rendering Result Issuetitle:WooCommerce Related products section CSS issue Body: Reported: https://secure.helpscout.net/conversation/724266 382/166432?folderId=607541 <image> This is something I X GT file: "assets/front/css/_parts/1_0_plugin_compat.css" Category:UI Screenshot Image Issuetitle: Feature request: Dashboard Pie Charts - Display percentage instead of hours Body: Ability to change hovertext display on piechart segments from hours to a percentage, or at least include the X GT file: "ui/src/dashboard/Entry/DashboardPieChart.tsx" Category:Data Visualization Image Issuetitle: When the ContentType is set to `multipart/form-data` and query X Body: When the ContentType is set to multipart/form-data and query parameters (annotated with `@RequestParam`) are X GT file: "src/main/java/com/ly/doc/model/torna/Apis.java", "src/main/java/com/ly/doc/model/torna/ DubboInfo.java", "src/main/java/com/ly/doc/utils/TornaUtil.java" Category:Behavior Demo Image Issuetitle: Bot stop responding after some time Body: I noticed that after some time without receiving any commands, the bot has problems determining whether or not it is connected X GT file: "config/config.py", "vulkan/music/Player.py", "vulkan/music/utils.py" Category:Log Output Image Issuetitle: Feature Request: Support another case of using Promises.all() Body: Related to https://github.com/playwright- community/eslint-plugin-playwright/issues/107 It looks like I am X GT file: "src/rules/missing-playwright-await.ts", "test/spec/missing-playwright-await.spec.ts" Category:Code Screenshot Image Issuetitle:Orthogonal view feature Body: Hi, does the polyscope support an orthogonal view of the camera? I need an orthogonal view but I could not find it. I found I can adjust the FOV may lower the quality X GT function: "include/polyscope/view.h:polyscope" Image Category: Rendering Result Issuetitle:Shapes on certain layer not selectable Body: Version: KLayout 0.29.7, commit 2d4f5d822c2e8c2649387ad5c5e242b85c56ec I can‘t select shapes on `mcon.pin 78748`,X GT function: "src/edt/edt/edtServiceImpl.c:edt" Image Category: UI Screenshot Issuetitle:Feature request: support for /32 DHCP allocations Body: Hi, Thanks for iPXE, I‘m slowly getting familiar with it. On Linux it’s possible to assign a /32 v4 IP X GT function: "src/net/ipv4.c:ipv4_add_miniroute", "src/net/ipv4.c:ipv4_is_fragment", "src/net/ipv4.c:ipv4_route ( unsigned int scope_id, struct in_addr *dest )" Category: Log Output Image Issuetitle:Enhance plot Body: Improve plot_splits for time series splits. Currently the plot present some limitation. Here is an exemple with code: ``` import pandas as pd, X GT function: "mlxtend/evaluate/time_series.py:plot_split_indices" Category: Data Visualization Image Issuetitle:viewport_interaction_active function error Body: When the editor window is created but not entered by the mouse or loses focus directly from the viewport area of the editor window: `assert!(self.pointer_used, false);` X GT function: "crates/bevy_editor_pls_core/src/editor.rs:Editor.v iewport_interaction_active" Category: Behavior Demo Image Issuetitle:DataTable became blank after window resizing Body: Hello, I found a little bug when I resize my window on a dashboard, one of my enhanced table became blank. Here is the Javascript error that appear after resizing: ``` TypeError: "col.agg X GT function: "public/enhanced-table-vis- controller.js:EnhancedTableVisController" Category: Code Screenshot Image Issuetitle:`Drawer` will cause mismatch in SSR mode Body: Reproduction link [https://stackblitz.com/edit/nextjs- zxvknz?file=pages/index.js](https://stackblitz.com/edit /nextjs-zxvknz?file=pages/index.js) ### Steps to reproduce 1.X GT function: "src/Drawer.tsx:Drawer" Category: ErrorMessage Image Issuetitle: Core functionality crashes while working with FSE Body: Steps to Reproduce 1. Install a Block theme 2. Try to use the Insert Special Characters feature via FSE 3. Observe it crashes with X GT file: "src/index.js" Category:ErrorMessage Image Figure 5: Representative M-IssueLoc examples grouped by image category. 15 Screenshot-to-Text Extraction Convert screenshots into structured textual clues OCRtext UIelements Error signal User action Code hint Textualized Visual Evidence b •Button label •Error message •Function/file clue •User action ScreenshotInput Screenshot(s) Augmented Issue Text Issue input Extracted Textual Evidence + Code Localization Used as input For localization VCE(Visual Content Extraction) overview File-level Function-Level Granularity Figure 6: Overview of the VCE extraction mechanism. Issue images are converted into structured textual evidence and then used as an auxiliary diagnostic input for issue localization. ExampleofVCE Screenshot VCE+ExtractedTextualEvidence OCR text: "::FileViewProject (clear) (fill(text04040\"Hello\")\"black\") (stroke(text08040\"Hello\")\"black\") ” UI elements： "codeeditorpanel" , "canvasrenderingarea" , "menubar(FileViewProject)” Error signal: "kind": "rendering_bug", "type": "text_rendering_failure", "message": "Text rendered as outlined/hollow glyphs instead of proper filled/stroked text", "stack_hint": ["text", "fill", "stroke"] Useraction: "UsertypedfillandstroketextcommandsinRonineditorandobservedincorrectrendering" Codehint： "function_names" :[ "clear" , "fill" , "text" , "stroke" ], "file_patterns" : null , "frameworks" :[ "Ronin" ] Category : Rendering Result VCE+ExtractedTextualEvidence OCR text: “> date -u && dateWed 15 Jul 2020 06:47:51 PM UTCThu 16 Jul 2020 03:47:51 AM JST~> http ‘https://codeforces.com/enter‘HTTP/1.1 200Cache-Control: private,no-cache,no-store,max-age=0,must-revalidate Connection: keep-aliveContentEncoding: gzipContent-Type: text/html;charset=UTF-8Date: Wed, 15 Jul 2020 18:48:02 GMTExpires: -1Pragma: no-cacheServer: kittenxSet-Cookie:JSESSIONID=...Set-Cookie: ...Strict- Transport-Security: max-age=86400Transfer-Encoding: chunkedVary: Accept-EncodingX-Content-Type-Options: nosniffX-Frame-Options: sameoriginX-XSS-Protection: 1; mode=block" UI elements: "terminal output", "HTTP response headers", "HTML source code" Error signal: "kind": "http", "type": "HTTP 200 but login not working", "message": "Codeforces login endpoint returns HTTP/1.1 200 but login functionality is broken -requires CSRF token and session handling changes", "stack_hint": null User action: "User attempted HTTP request to codeforces.com/enter to debug login issue" Code hint: "function_names": ["adjustViewport"], "file_patterns": ["codeforces.com/enter"], "frameworks": ["oj-api", "online-judge-tools"] Category :Error Message Screenshot ExampleofVCE Figure 7: Representative VCE extraction examples across image categories, including error messages, UI screenshots, behavior demos, rendering results, code screenshots, data visualizations, and log outputs. 16 VCE prompt VCE+ Structured Extraction Task You are a screenshot-to-structured-text extractor for GitHub issue images. Your task is to extract structured visual evidence from each image and return a concise JSON object. System Instruction: Output only a single JSON object that conforms exactly to the schema given below. Do not include prose outside the JSON object. Use null for fields that are not applicable. Keep each string concise, preferably fewer than 25 words. Do not invent content that is not visible in the image. Input: • Issue Category:category • Issue Title:issue title • Issue Body:issue body • Issue Image:image Extraction Schema: "ocr text": <string | null>, "errorsignal": <object | null>, "uielements": <string[] | null>, "useraction": <string | null>, "codehint": <object | null>, "visual saliency": <string | null>, "confidence": <float 0--1>, "notes": <string | null> Field Definitions: • ocr text: verbatim readable text in the image. • errorsignal: structured error information, such askind, type, message, stackhint[]. • ui elements: salient UI components mentioned or visible in the image. • useraction: one-line natural-language description of the user action. • codehint: code-related hints, such asfunctionnames[], filepatterns[], frameworks[]. • visualsaliency: the part of the image that is visually highlighted or abnormal. • confidence: extractor’s self-assessed confidence score from 0 to 1. • notes: a short summary of at most 15 words. Category-Specific Extraction Focus: • uiscreenshot: focus on UI elements and user action. Usually no error signal is needed. Include visible button, label, or menu text in ocrtext. codehint may include framework or component names. • behaviordemo: focus on the user action and visual saliency. Describe what the user does step by step and list the widgets involved. • errormessage: focus on errorsignal, including error kind, type, message, and stack hints. Include the full verbatim message in ocrtext if it is short. Extract likely function or file names from the stack trace. • rendering result: focus on what is visually wrong and which widget renders the output. Include axis labels, numbers, or visible text if legible. Add component or framework names when possible. • codescreenshot: focus on transcribing the code. Populate code hint.functionnames. Include visible line numbers in notes. • datavisualization: focus on chart titles, axis labels, and abnormal series or bars. Add visualization frameworks such as matplotlib, plotly, or similar names when visible or strongly indicated. • logoutput: focus on the log text. If the log is short, transcribe it verbatim; otherwise provide a condensed summary. Fill errorsignal if an error line is present. Use codehint.filepatterns for mentioned paths. User Prompt Template: Issue category:category Issue title:issuetitle Issue body:issuebody Extraction focus for this category: categoryhint Return a JSON object following exactly the schema above. Use null for fields you cannot fill in, and never invent content not visible in the image. Output Requirement: Return only the JSON object. Do not add explanations, markdown fences, or extra text. Figure 8: VCE prompt 17 AI-driven harmful impact prompt You are a software engineering expert. Please analyze the relationship between the images and the problem described in the following GitHub issue. Issue Information: • Title:issue title • Repository:repofullname • Language:repolanguage • Body:issue body Images in the Issue: issueimages Image Source Description: This is a synthetic sample generated bysynthesis strategy. The images are taken from another issue in the same repository: imagesourceissuetitle. Ground-truth Fixed Files: difffiles Important Note: The ground-truth fixed files are provided only for evaluation. Do not treat them as information available in the original issue. Instructions: Please answer strictly according to the following seven dimensions: 1. Q1TEXTSUFFICIENT: [YES/NO] Based only on the textual issue description, can one infer the approximate location that needs to be modified? Q1 REASONING: Give a one-sentence explanation. 2. Q2SURFACERELEVANCE: [1--5] Rate the surface-level relevance between the images and the issue description. 1 means completely irrelevant, and 5 means highly relevant. Q2 REASONING: Explain in one sentence why the images appear relevant or irrelevant. 3. Q3IMAGEDIRECTION: If the images are considered, which direction, component, or module would they guide the developer toward? Q3REASONING: Give a one-sentence explanation. 4. Q4DIRECTIONDEVIATION: [1--5] Rate the deviation between the image-guided direction and the actual fixed location. 1 means the directions are consistent, and 5 means they are completely diver- gent. Q4REASONING: Explain the difference between the image-guided direction and the actual fixed files. 5. Q5MISLEADINGSCORE: [0--5] Rate the overall harmful or misleading impact of the images on locating the correct fix. 0 means no misleading impact, and 5 means severely misleading. Q5REASONING: Give a one-sentence explanation. 6. Q6DEVELOPERIMPACT: [LOW/MEDIUM/HIGH] Estimate how much additional debugging time a developer may waste if they rely on the images. Q6 REASONING: Give a one-sentence explanation. 7. Q7LABEL: [-1/0/1/2] Give the final usefulness label for the images in this issue: • -1: harmful, because the images mislead the repair direction; • 0: not helpful; • 1: helpful but not necessary; • 2: extremely helpful and necessary. Q7 REASONING: Give a one-sentence explanation. Figure 9: AI-Driven Harmful Impact Prompt 18 Agentless prompt You are a code localization expert. Given a repository structure, candidate file contents, and a bug report or feature request, identify the most likely files and functions that need to be modified. Task Overview: The localization process follows a two-stage hierarchical procedure: 1. File-level localization: identify candidate files from the repository structure and issue description. 2. Function-level localization: identify candidate functions or methods from the selected files and issue description. Stage 1: File-level Localization System Instruction: You are a code localization expert. Given a repository structure and a bug report or feature request, identify the most likely files that need to be modified. Return a JSON array of file paths, ordered by relevance from most relevant to least relevant. Return at most 10 files. Input: • Repository Structure:repo tree • Issue Description:problemstatement • Issue Images:issue imagesoptional, only used in multimodal setting User Prompt: List the files that most likely need to be changed. Return a JSON array of file paths. Expected Output Format: ["path/to/file1.py", "path/to/file2.py", "..."] Stage 2: Function-level Localization System Instruction: You are a code localization expert. Given file contents and a bug report, identify the specific functions or methods that need to be modified. Return a JSON array of function identifiers, ordered by relevance from most relevant to least relevant. Return at most 10 functions. Input: • Candidate Files:candidate files • Candidate File Contents:filecontents • Issue Description:problemstatement • Issue Images:issueimagesoptional, only used in multimodal setting User Prompt: Identify the functions or methods that need modification. Return a JSON array of identifiers in "file:function" format. Expected Output Format: ["file path:ClassName.methodname", "filepath:functionname", "..."] Important Notes: • The output must be a valid JSON array. • Do not include explanations outside the JSON array. • Rank files and functions by relevance. • Return at most 10 files in the file-level stage. • Return at most 10 functions in the function-level stage. • If images are provided, use them as auxiliary evidence together with the issue description. Figure 10: AgentLess Prompt 19 LocAgent prompt You are a code localization agent. Your task is to find the files and functions in a repository that need to be modified to address a given issue. Input: • Issue Description:problem statement • Issue Images:issueimagesoptional, only used in the multimodal set- ting • Repository: accessible through tool calls Task Instruction: Find the files and functions that need to be modified. Available Tools: You have the following tools available. At each turn, call exactly one tool by outputting a JSON object. 1. findfiles: list files matching a glob pattern. Example: "tool": "findfiles", "pattern": "**/*.py" 2. grep: search for a regex pattern in the repository. Example: "tool": "grep", "pattern": "somefunction", "include": "*.py" 3. read file: read a file, optionally with a line range. Example: "tool": "readfile", "path": "src/foo.py", "startline": 1, "endline": 100 4. submit: submit the final localization result. You must call this tool when done. Example: "tool": "submit", "files": ["path/to/file.py"], "functions": ["path/to/file.py:ClassName.method"] Guidelines: • Start by understanding the issue. • Search for relevant code using repository-level tools. • Explore broadly first, then narrow down to specific files and functions. • When confident, call submit to provide the final answer. • Return files and functions ordered by relevance, from most relevant to least relevant. Final Output Format: The final answer must be submitted through the submit tool: "tool": "submit", "files": ["..."], "functions": ["..."] Fallback Instruction: If the maximum number of tool calls is reached, submit the best current guess immediately using the following format: "tool": "submit", "files": [...], "functions": [...] Figure 11: LocAgent prompt 20 Openhands prompt Repository-Level Issue Localization Task You are a localization agent. Your only goal is to identify which source files and functions in repo/ are most likely affected by the issue below. You must not edit, patch, commit, install dependencies, or run tests. You only inspect the repository in a read-only manner and answer in JSON. Input: • Issue Title:title • Issue Body:body • Vision Block:vision blockoptional, only used when images are pro- vided • Repository: repo/ Procedure: Follow the steps exactly: 1. Run ls repo or a similar command to skim the top-level repository layout. 2. Use grep -rn or a similar command to find symbols, error messages, or strings from the issue inside repo/. 3. Read at most 2–3 files that look most relevant. 4. Stop after at most 8 total actions. Even if uncertain, you must answer. 5. Call the finish tool with the required JSON object as the value of message. 6. Place the same JSON object as the final assistant message text. Do not include prose, markdown fences, or explanations. Output JSON Format: "file level": [ "relative/path/to/mostlikelyfile.ext", "relative/path/to/next file.ext" ], "function level": [ "relative/path/to/file.ext:barefunctionname", "relative/path/to/file.ext:ClassName.methodname" ] Format Rules: • filelevel: return 1–10 file paths, ordered by descending likelihood. • functionlevel: return 1–10 function identifiers, ordered by descending like- lihood. • functionlevel may be empty only if the issue genuinely has no function-level target. • File paths must be relative to the repository root. • Use POSIX / separators. • Do not use a leading ./, trailing /, or repo/ prefix. • Each function identifier must use exactly one of the following formats: – relative/path/to/file.ext:bare functionname – relative/path/to/file.ext:ClassName.methodname • Do not include line numbers, argument lists, type hints, parentheses, decora- tors, or explanations. • Every function path should also appear in filelevel. • Do not list functions added by the fix. If a brand-new function is needed, point to the nearest existing anchor. • If unsure whether the scope is a class or module, prefer the bare function form. • Emit valid JSON: use double-quoted keys and strings, with no trailing commas and no comments. Reminders: • Prefer source files over tests, docs, generated files, or config files unless the issue clearly points to them. • Always emit the JSON. Empty answers score 0; imperfect answers can still receive partial credit. • Do not use terminal to run interactive commands, npm install, pytest, pip install, or similar commands. • Do not call task to spawn sub-agents. • Do not use browser use. • Use only terminal and fileeditor. Final Instruction: When confident, emit the final JSON object as the last message and stop. Figure 12: Openhands prompt 21 Mini-SWE-Agent prompt Repository-Level Issue Localization Task You are given a repository snapshot and an issue report. Your task is not to fix the issue. Your task is only to localize the source files and functions that most likely need to be modified. Allowed Behavior: • You may inspect the repository. • You may use shell commands to search and read code. • You may reason about the issue, stack traces, UI screenshots, logs, rendered outputs, and code structure. • You must not edit files. • You must not create patches. • You must not commit changes. • You must not run long package installation commands. • You must prefer source files over tests, docs, generated files, or config files unless the issue clearly points to them. Repository: The repository root is: repodir You are already working inside the repository root. Issue: • Title:title • Body:body • Comments Block:commentsblock • Visual Block:visualblock Final Submission: When you have enough evidence, write the final JSON answer to the file path stored in$MSWEAPREDFILE. This is a per-instance path pre-allocated by the run- ner. Do not use /tmp/something.json, because parallel agents share the host’s /tmp directory. Then issue exactly this bash action, and nothing else, to complete the task: echo COMPLETE TASKANDSUBMITFINALOUTPUT && cat "$MSWEAPREDFILE" Output JSON Schema: "filelevel": [ "path/to/file1.ext", "path/to/file2.ext" ], "functionlevel": [ "path/to/file1.ext:functionname", "path/to/file2.ext:Class.method" ] Format Rules: • filelevel: return 1–10 repository-relative file paths, ordered from most likely to least likely. • functionlevel:return 0–20 entries of the form file:function or file:Class.method. • Every file referenced in functionlevel must also appear in filelevel. • Use POSIX / separators. • Do not prepend ./ or repo/. • Do not include line numbers, parentheses, argument lists, decorators, or com- ments in the function identifiers. • Prefer source files over tests, docs, generated files, or config files unless the issue clearly points to them. Figure 13: Mini-SWE-Agent Prompt 22