Paper deep dive
DocIntent: Answerability-Guided Agentic Restoration for Real-World Document Visual Question Answering
Zihan Huang, Shihang Wu, Junle Liu, Peirong Zhang, Yongxin Shi, Xuhan Zheng, Lianwen Jin
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%
Last extracted: 9/1/2026, 5:32:52 AM
Summary
The paper introduces DocIntent, a training-free, Answerability-Guided Agentic Restoration framework designed to improve Document Visual Question Answering (VQA) by Multimodal Large Language Models (MLLMs) on real-world degraded documents. Unlike existing methods that optimize for generic image quality, DocIntent selectively invokes restoration tools (e.g., deshadowing, deblurring) only when necessary to answer specific questions, using a two-phase cyclic mechanism involving Answerability-Guided Restoration and Comparison-Based Rollback. Experiments on the WildDoc benchmark demonstrate that DocIntent consistently improves the average score and consistency of various open- and closed-source MLLMs.
Entities (8)
Relation Signals (6)
DocIntent â evaluatedon â WildDoc
confidence 95% · Extensive experiments on the WildDoc benchmark show that DocIntent consistently improves the average score
DocIntent â improves â Document Visual Question Answering
confidence 95% · DocIntent consistently improves the average score and consistency of different open- and closed-source MLLMs.
DocIntent â uses â Comparison-Based Rollback
confidence 95% · A Comparison-Based Rollback mechanism validates each restoration step and reverts it when question-relevant evidence becomes less decipherable.
DocIntent â uses â Answerability-Guided Restoration
confidence 95% · In Phase A: Answerability-Guided Restoration (AGRes), the MLLM first assesses whether the current image provides sufficient information to answer the question.
DocIntent â appliedto â Qwen3.5-9B
confidence 90% · DocIntent(Qwen3.5-9B) 85.4 +2.9...
DocIntent â appliedto â Gemini 3.1 Pro
confidence 90% · DocIntent(Gemini-3.1-Pro) 94.1 +1.3...
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Real-world degradations such as blur, shadow, distortion, and moire patterns severely impair the document question-answering capabilities of Multimodal Large Language Models (MLLMs). Applying restoration tools before Visual Question Answering (VQA) is an intuitive solution. However, existing restoration approaches remain limited, as manually designing and executing restoration strategies is labor-intensive and requires domain expertise. Agentic restoration offers new possibilities for automation, yet existing frameworks primarily target natural images and pursue perceptual quality, overlooking that restoration should serve downstream tasks rather than optimize generic image quality metrics. To this end, we explore the value of agentic restoration for real-world degraded document VQA and propose DocIntent, a training-free Answerability-Guided Agentic Restoration framework. DocIntent first assesses question answerability, then identifies task-relevant degradations and selectively invokes restoration tools. A Comparison-Based Rollback mechanism validates each restoration step and reverts it when question-relevant evidence becomes less decipherable. The entire process requires no additional pretrained degradation classifier or image quality assessment model. Extensive experiments on the WildDoc benchmark show that DocIntent consistently improves the average score and consistency of different open- and closed-source MLLMs. The code and experimental data will be publicly available.
Tags
Links
- Source: https://arxiv.org/abs/2608.29037v1
- Canonical: https://arxiv.org/abs/2608.29037v1
Trouble viewing inline? Open PDF directly â
Full Text
84,096 characters extracted from source content.
Expand or collapse full text
DocIntent: Answerability-Guided Agentic Restoration for Real-World Document Visual Question Answering Zihan Huang, Shihang Wu, Junle Liu, Peirong Zhang Yongxin Shi, Xuhan Zheng, Lianwen Jin * South China University of Technology Abstract Real-world degradations such as blur, shadow, distortion, and moirĂ© patterns severely im- pair the document question-answering capabil- ities of Multimodal Large Language Models (MLLMs). Applying restoration tools before Visual Question Answering (VQA) is an intu- itive solution. However, existing restoration approaches remain limited, as manually de- signing and executing restoration strategies is labor-intensive and requires domain expertise. Agentic restoration offers new possibilities for automation, yet existing frameworks primar- ily target natural images and pursue percep- tual quality, overlooking that restoration should serve downstream tasks rather than optimize generic image quality metrics. To this end, we explore the value of agentic restoration for real-world degraded document VQA and pro- pose DocIntent, a training-free Answerability- Guided Agentic Restoration framework. DocIn- tent first assesses question answerability, then identifies task-relevant degradations and selec- tively invokes restoration tools. A Comparison- Based Rollback mechanism validates each restoration step and reverts it when question- relevant evidence becomes less decipherable. The entire process requires no additional pre- trained degradation classifier or image qual- ity assessment model. Extensive experiments on the WildDoc benchmark show that DocIn- tent consistently improves the average score and consistency of different open- and closed- source MLLMs. The code and experimental data will be publicly available. 1 Introduction In the digital era, enterprises and organizations handle large volumes of documents, such as in- voices, contracts, and forms (Jaume et al., 2019; Park et al., 2019). These documents often contain critical information for business workflows, mak- ing efficient and accurate information extraction * Corresponding author. QïŒWhat is the area code of the caller? Answer Directly 212 User Preprocessing + Restored by User UnfriendlyïŒ DocIntentïŒOursïŒ FriendlyïŒ Answerability- Guided Analysis Tool Execution Comparison- Based Rollback QïŒWhat is the area code of the caller? QïŒWhat is the area code of the caller? 313 212 QïŒWhat is the area code of the caller? Figure 1: Comparison between existing real-world doc- ument VQA methods and the proposed DocIntent (bot- tom). The ground truth answer to the question is 212. essential. However, traditional manual process- ing is labor-intensive and time-consuming. Visual Question Answering (VQA) on document-related data addresses this challenge by enabling intelli- gent systems to understand document content and answer user-specified questions (Mathew et al., 2021, 2022), supporting more flexible document processing. Recently, Multimodal Large Language Models (MLLMs) (Team, 2025; Team et al., 2023; Hurst et al., 2024) have demonstrated remarkable po- tential in document understanding (Mathew et al., 2021, 2022; Masry et al., 2022). However, real- world document images often exhibit degradations such as blur, shadow, and distortion (Wang et al., 2025a; Zhang et al., 2024; Feng et al., 2021; Zhang et al., 2025a), which severely hinder information extraction by MLLMs. Research on benchmarks such as WildDoc (Wang et al., 2025a) reveals that even state-of-the-art MLLMs experience substan- tial performance degradation when confronted with degraded documents, as illustrated in Figure 1(top- left). A straightforward strategy is to preprocess 1 arXiv:2608.29037v1 [cs.CV] 29 Aug 2026 degraded documents with restoration tools before feeding them into MLLMs (Figure 1, top-right). However, this pipeline depends on manually diag- nosing degradation types and selecting appropriate tools, making it impractical for non-experts. Re- cently, agentic image restoration methods (Chen et al., 2024; Zhou et al., 2025; Zhu et al., 2024; Zuo et al., 2025; Zhang et al., 2026b) have achieved success in natural image restoration. However, these methods primarily focus on natural image processing and pursue the highest perceptual im- age quality rather than aligning with downstream task requirements. In document question answer- ing, different questions exhibit varying sensitivity to degradations, and not all degradations hinder question answering. Existing document restoration tools (Zhang et al., 2024; Yang et al., 2025; Zhao et al., 2025) optimize perceptual quality metrics such as SSIM (Wang et al., 2004) and PSNR. How- ever, these overall quality metrics cannot ensure local text clarity. The restoration process may in- troduce artifacts, text blurring, or distortion that impair question-answering performance. Process- ing task-irrelevant degradations unnecessarily ac- cumulates such risks. Therefore, a critical question arises: How to tailor document restoration to specific question-answering tasks? To address this question, we propose DocIntent, a training-free and user-friendly answerability- guided document image restoration framework. As illustrated in Figure 1 (bottom), DocIntent inte- grates downstream question awareness into restora- tion planning through a two-phase cyclic mecha- nism. In Phase A: Answerability-Guided Restora- tion (AGRes), the MLLM first assesses whether the current image provides sufficient information to answer the question. If restoration is needed, it identifies the degradation that most affects answer- ability and invokes the corresponding restoration tool to improve the document image. In Phase B: Comparison-Based Review, the Comparison- Based Rollback (CBRb) mechanism compares the restored image with the previous image to deter- mine whether the operation improves the condi- tions for answering. Beneficial restorations are accepted for subsequent reasoning, while harmful ones are rolled back, and the corresponding tool is disabled. Both phases are performed by the MLLM without requiring an additional pretrained degrada- tion classifier or image quality assessment model. Through this cycle, DocIntent selectively applies restoration to improve answerability while avoid- ing unnecessary or harmful preprocessing. In summary, our contributions are as follows: âą To the best of our knowledge, we are the first to explore the value of agentic restora- tion for document restoration and real-world degraded document VQA. We propose an Answerability-Guided Agentic Restoration framework that incorporates downstream question awareness into restoration planning. âą We design a two-phase cyclic mecha- nism with Answerability-Guided Restora- tion and Comparison-Based Rollback, en- abling MLLMs to selectively invoke docu- ment restoration tools and roll back harm- ful results without additional training, a pre- trained degradation classifier, or an image quality assessment model. A standardized tool interface also supports the integration of stronger restoration models in the future. âą Extensive experiments on the WildDoc (Wang et al., 2025a) benchmark demonstrate that DocIntent consistently improves modelsâ question-answering performance on degraded documents across different capability levels and achieves results close to human expert preprocessing. 2 Related work 2.1 Document Visual Question Answering Visual Question Answering (VQA) on document- related data (Mathew et al., 2021; Nandi et al., 2026) requires models to answer questions about document images by understanding the text, im- ages, and layout. Early methods (Xu et al., 2021; Huang et al., 2022; Appalaraju et al., 2024; Zhu et al., 2025) adopted OCR-based pipelines, ex- tracting textual and layout information (Zhang et al., 2025b) through OCR systems before an- swering questions. Subsequent end-to-end ap- proaches (Kim et al., 2021; Zhang and Zhang, 2025; Souibgui et al., 2025) eliminated the OCR module, directly learning document representa- tions through visual encoders. With the rise of MLLMs (Qwen Team, 2026a; Wang et al., 2025b; Team et al., 2023; Hurst et al., 2024; Anthropic, 2025; Bytedance Seed Team, 2025), general- purpose MLLMs have demonstrated strong perfor- mance in document understanding tasks. Special- ized document-understanding MLLMs (Hu et al., 2 CBRb Phase A Analyze Degradation PHASE A: Answerability-Guided Restoration PHASE B: Comparison-Based Review Answerability-Guided Analysis Tool Execution Comparison-Based RollbackïŒCBRbïŒ Available Restoration Tools QïŒWhich subject had the highest pulse rate in baseline period? Is the question answerable? Answer VS The current image contains shadow, uneven illumination, and blur.The shadow is the most obvious, resulting in a decrease in contrast. Try to use the deshadowing tool. Tool Calling The deshadow tool did not enhance the contrast and instead caused blurring. Therefore, rollback to the previous image and disable the deshadowing tool. AfterBefore Deshadowing Appearance Enhance Rollback Continue Disable Tool Subject 1. Phase A Phase B Phase B Phase A + & Rollback Continue Figure 2: Overview of the DocIntent framework with two-phase cyclic mechanism: Phase A (Answerability-Guided Restoration) and Phase B (Comparison-Based Review). The figure illustrates an example where deshadowing triggers rollback (bottom-left), while appearance enhancement succeeds and continues (bottom-right). 2025; Li et al., 2024b; Huang et al., 2024; Zhao et al., 2024; Yu et al., 2025) have also achieved sig- nificant results on standard benchmarks (Mathew et al., 2021; Masry et al., 2022; Kim et al., 2024; Mathew et al., 2022).Uni-DocRobust (Zhou et al.) improves robustness through feature restora- tion, whereas DocIntent selectively invokes image restoration tools based on question answerability. However, existing MLLMs exhibit significant performance degradation on real-world document question answering (Wang et al., 2025a). Real- world degradations severely impair MLLMsâ abil- ity to perceive and understand document content. To address this challenge, we propose DocIn- tent, a training-free answerability-guided docu- ment restoration framework. DocIntent leverages MLLMsâ inherent degradation recognition capabil- ity and employs a selective restoration strategy to enhance their document-question-answering per- formance under real-world degradation scenarios. 2.2 Agentic Image Restoration In the field of image restoration, Restore- Agent (Chen et al., 2024) pioneered the agentic image restoration paradigm, using task-specific trained models (Zamir et al., 2022) to restore images of natural scenes with multiple degrada- tions. Existing agentic image restoration frame- works (Zhu et al., 2024; Chen et al., 2024; Li et al., 2025; Zhou et al., 2025; Zuo et al., 2025; Zhang et al., 2026b) primarily focus on natural images and pursue optimal quality. AgenticIR (Zhu et al., 2024) uses a fine-tuned vision model for degra- dation recognition and quality assessment, while 4KAgent (Zuo et al., 2025) combines multiple im- age quality assessment models to plan and validate restoration results. In contrast, document image restoration is intended to serve specific downstream tasks rather than pursue the highest image quality. Different models and questions also exhibit vary- ing sensitivities to degradations. Therefore, our ap- proach does not rely on a specialized degradation classifier or image quality assessment model and requires no additional training. Instead, it enables MLLMs to adaptively assess whether the current image quality hinders answering the specific ques- tion at hand. Meanwhile, the emergence of multi- task document restoration models (Zhang et al., 3 QïŒWho is the First Vice-President? O r i g i n a l D o c I n t e n t AïŒMike Belfiglio AïŒWalter F. Wrenn, Jr. DewarpingăDeblurring QïŒWhat is value of Swanson Interests? AïŒ718 AïŒ362.0 Rotate_180ă Appearance Enhancement QïŒHow many metric tons of soybeans were produced worldwide in the 2020/2021 crop year? Deshadowing AïŒ362.05 AïŒ162.05 QïŒWhat is the suite number? AïŒ1001 AïŒ500 DemoirĂ©ing Figure 3: Cases where models fail on degraded images but succeed with DocIntent. Left two columns: Qwen3.5- 9B (Qwen Team, 2026a). Right two columns: Gemini-3.1-Pro (Google DeepMind, 2026). 2024; Yang et al., 2025; Zhao et al., 2025; Peng et al., 2024; Zhang et al., 2026a) provides strong support for the use of restoration tools. Based on these observations, we propose DocIn- tent, an answerability-guided document image restoration framework. DocIntent adopts a selec- tive restoration strategy, processing degradations that affect downstream tasks, and decides whether to accept restoration results through a comparison- based rollback mechanism, ensuring task perfor- mance while avoiding unnecessary processing over- head and risk of text quality degradation. 3 Method We first introduce the DocIntent framework, in- cluding three core modules: Answerability-Guided Analysis, Tool Execution, and Comparison-Based Rollback (Sec. 3.1). We then detail the complete two-phase cyclic workflow, demonstrating how these modules collaborate to achieve answerability- guided selective restoration while preventing qual- ity degradation (Sec. 3.2). 3.1 DocIntent Framework Question answering on degraded document images involves several practical challenges, including de- termining whether restoration is needed, selecting appropriate restoration tools, and avoiding perfor- mance degradation caused by unsuitable restora- tion. To address these issues, DocIntent is designed with three core modules: Answerability-Guided Analysis, Tool Execution, and Comparison-Based Rollback, as shown in Figure 2. Answerability-guided analysis. The MLLM first analyzes the image in conjunction with the question to determine whether the question can be answered directly from the current image. If so, it generates the answer directly without triggering restoration. Otherwise, the MLLM identifies which degradations prevent it from answering and selects the most severe one whose corresponding tool is available for restoration. Unlike traditional meth- ods (Zhu et al., 2024; Chen et al., 2024; Li et al., 2025; Zhou et al., 2025) that pursue the highest image quality, our approach triggers restoration only when degradation hinders answering. As il- lustrated in Figure 4, when degradations do not affect task-critical regions, questions can be cor- rectly answered without any restoration, making blind restoration unnecessary and potentially harm- ful. This selective restoration approach reduces computational cost while minimizing the risk of text distortion or blurring by avoiding unnecessary restoration operations. 4 Tool execution. Once the MLLM identifies degradation types that hinder question answering and selects the most severe one whose tool is avail- able, the system invokes the corresponding restora- tion tool to process the document image. We main- tain an available tool list for MLLM selection. The initial tool list contains: deblurring, dewarping, de- shadowing, appearance enhancement, demoirĂ©ing, 90° rotation, 180° rotation, and 270° rotation, which respectively target blur, warp, shadow, un- even illumination, moirĂ© pattern, and orientation degradations. We design a standardized tool in- terface to support future extensions, enabling flex- ible replacement with more powerful restoration models or integration of new tools for emerging degradation types. Notably, our core contribution is leveraging restoration tools based on task require- ments to enhance MLLMsâ question answering on degraded documents, rather than improving exist- ing restoration models. Comparison-based rollback. Existing agentic restoration frameworks typically rely on fine-tuned degradation perception models or specialized im- age quality assessment models to evaluate restora- tion results and trigger rollback (Zhu et al., 2024; Zuo et al., 2025). To avoid additional training and dependence on these specialized models, we design Comparison-Based Rollback. This mechanism pro- vides the MLLM with the current question and the images before and after restoration, allowing it to compare the readability of question-relevant evidence. When restoration distorts or blurs the relevant text and makes the question more difficult to answer, the system triggers rollback (see Fig- ure 10). We accordingly determine that the tool cannot effectively address the degradation, disable it from the available tool list, and attempt to address other degradations. When restoration improves the readability of question-relevant evidence and fa- cilitates answering, the system accepts and retains the restored result. As a training-free safeguard, this mechanism prevents harmful restoration results from accumulating across successive operations. 3.2 Workflow DocIntent achieves answerability-guided document restoration through a two-phase cyclic mecha- nism. Given an input real-world document image and question, the system performs Answerability- Guided Restoration in Phase A, followed by Comparison-Based Review in Phase B. This itera- DïŒShadowăBlur QïŒWhat is her date of birth? AïŒ9 March 1898 DïŒWarp QïŒWhat is the title in this document? AïŒHOW TO PLAN A SELF- SERVICE FOOD DONATION CENTER DïŒMoirĂ© PatternăBlur QïŒName the materials for which this procedure note is givenïŒ AïŒG14-9A and G19-33A Figure 4: Representative cases where questions can be correctly answered despite visible degradations. D denotes degradation type. tive process continues until the MLLM determines that the current image is sufficient to answer the question or that all tools capable of addressing the degradations that hinder answering have been dis- abled, as illustrated in Figure 2. The two phases coordinate to achieve selective restoration with a quality safeguard. In Phase A, the system determines whether restoration is needed. If the current image suffices for answering, the MLLM generates the answer directly. Otherwise, the system identifies the most severe degradation that hinders answering, invokes the corresponding tool, and proceeds to Phase B. In Phase B, the Comparison-Based Rollback mechanism evaluates the restoration outcome. If textual clarity improves and facilitates question answering, the system ac- cepts the restoration. Otherwise, it triggers a roll- back and disables the tool. After either outcome, the system returns to Phase A. Through this cyclic mechanism, DocIntent progressively improves doc- ument readability to facilitate question answering. The design ensures two desirable properties: (1) minimalityâonly degradations that impair answer- ability are addressed, avoiding unnecessary pro- cessing. (2) safetyâevery restoration action is validated before being committed, preventing error accumulation across successive operations. 4 Experiments 4.1 Experimental Settings Implementation details. All restoration tools are deployed on a single NVIDIA A6000 GPU: DocRes (Zhang et al., 2024) for deblurring, de- warping, deshadowing, and appearance enhance- ment, and UniDemoirĂ© (Yang et al., 2025) for demoirĂ©ing (see Appendix I for tool selection dis- cussions). Open-source MLLMs (e.g., Qwen3.5- 9B) run on 4 NVIDIA A6000 GPUs, while closed- 5 Model WildDocVQAWildChartQAWildTableVQAAverage ANLSConsistencyAcc.ConsistencyAcc.ConsistencyScoreConsistency MiniMonkey-2B (Huang et al., 2024)54.322.832.312.031.313.439.316.1 Monkey (Li et al., 2024b)31.09.922.49.823.011.725.510.5 TextHarmony (Zhao et al., 2024)37.116.021.910.214.76.524.610.9 Llava-Onevision-7B (Li et al., 2024a)52.923.749.420.233.913.345.419.1 mPLUG-DocOwl2 (Hu et al., 2025)44.219.223.311.622.14.829.911.9 MiniCPM-V4.5 (Yu et al., 2025)73.153.258.942.456.822.462.939.3 InternVL3.5-8B (Wang et al., 2025b)56.128.048.326.843.212.849.222.5 InternVL3.5-38B (Wang et al., 2025b)66.842.060.537.651.918.859.732.8 Qwen3.5-9B (Qwen Team, 2026a)82.570.851.933.271.145.268.549.7 Qwen3.5-27B (Qwen Team, 2026a)83.372.853.243.676.649.271.055.2 Qwen3.6-27B (Qwen Team, 2026b)82.972.460.346.077.853.273.757.2 Closed-source MLLMs GPT-4o (Hurst et al., 2024)63.241.631.923.259.630.851.631.9 Qwen3.5-Plus (Qwen Team, 2026a)84.477.062.552.481.559.676.163.0 Claude Sonnet 4.6 (Anthropic, 2025)63.438.450.031.659.121.657.530.5 Gemini-3.1-Pro (Google DeepMind, 2026)92.8 88.066.759.689.578.883.075.5 Doubao-2.0-Pro (Bytedance Seed Team, 2025)82.772.862.352.084.665.676.563.5 GPT-5.2 (Singh et al., 2025)66.137.658.344.864.233.662.938.7 DocIntent(Qwen3.5-9B)85.4 +2.976.4 +5.653.6 +1.742.0 +8.875.7 +4.648.4 +3.271.6 +3.155.6 +5.9 DocIntent(Gemini-3.1-Pro)94.1 +1.391.6 +3.667.7 +1.063.6 +4.091.2 +1.784.8 +6.084.3 +1.380.0 +4.5 Table 1: Evaluation on the WildDoc (Wang et al., 2025a) benchmark. "ANLS" denotes Average Normalized Levenshtein Similarity. Top results are bolded and second-best underlined. source MLLMs are accessed via official APIs. Task-specific prompts for the three WildDoc sub- tasks are in Appendix M. We retain text context (tool decisions and quality assessments) and only the previous image. Context management compar- isons and preprocessing and decoding settings are in Appendices E and F, respectively. Datasets. We adopt WildDoc (Wang et al., 2025a) as our evaluation benchmark. To the best of our knowledge, at the time of our evaluation, WildDoc was the only publicly available bench- mark specifically designed for document VQA under real-world capture degradations. It con- tains three subtasks: WildDocVQA, WildChartQA, and WildTableVQA, covering degradations such as blur, shadow, geometric distortion, moirĂ© pat- terns, uneven illumination, and rotation. Each question corresponds to images captured under four real-world conditions, enabling evaluation of question-answering performance and consistency across degradations. Metrics. Following WildDoc (Wang et al., 2025a), we use Average Normalized Levenshtein Similarity (ANLS) for WildDocVQA, Accuracy (Acc.) for WildChartQA and WildTableVQA, and Consistency (Cons.) for robustness across degrada- tion scenarios. See Appendix A for definitions. 4.2 Main Results We evaluate DocIntent on WildDoc with Qwen3.5- 9B (Qwen Team, 2026a) and Gemini-3.1- Pro (Google DeepMind, 2026) as backbone mod- els. Table 1 compares it with a range of open- and closed-source MLLMs. Among baseline models, we observe a common phenomenon: modelsâ consistency scores are sub- stantially lower than their average score, reflecting their lack of robustness when dealing with different degradation conditions. This score-consistency gap is pervasive across models of varying capabilities. For example, Claude Sonnet 4.6 achieves a 57.5% average score but only 30.5% consistency, a 27.0 percentage-point gap. Even the best-performing Gemini-3.1-Pro shows a 7.5 percentage-point gap (83.0% vs 75.5%). This widespread phenomenon indicates the fundamental challenge of maintain- ing stable performance across diverse real-world degradation scenarios. DocIntent achieves consistent improvements on both backbone models. When applied to Gemini- 3.1-Pro, the average score improves from 83.0% to 84.3%. When applied to Qwen3.5-9B, the average score improves from 68.5% to 71.6%. Notably, DocIntent demonstrates greater improvements in consistency, with gains of 4.5 points (75.5%â 80.0%) on Gemini-3.1-Pro and 5.9 points (49.7% â55.6%) on Qwen3.5-9B. These consistency improvements narrow the score-consistency gap, demonstrating that DocIntent not only improves 6 Input Pipeline MLLM-JudgeDocIntent QïŒOn which company letterhead is this written? AïŒNational Corporation for Housing Partners AïŒNational Corporation for Housing Partnerships AïŒNational Corporation for Housing Partnerships DemoirĂ©ingăDeshadowingă Appearance Enhancement DemoirĂ©ing QïŒHow much is the total income of Swanson Center for Nutrition, Inc. in working 1982 budget? AïŒ619.2 AïŒ718.9 Rotate_180ăDewarpingă Deshadowing Rotate_180 AïŒ710.9 Human Expert AïŒNational Corporation for Housing Partnerships DemoirĂ©ingă Appearance Enhancement Rotate_180ăDewarpingă Appearance Enhancement AïŒ718.9 QïŒWhat is the page number? AïŒ1 AïŒ2 DeshadowingăDewarpingă Debluring Directly Answer AïŒ7 Dewarpingă Appearance Enhancement Unanswerable Image After... Image After... Tools Calling All tools Answer All tools Answer Tools Calling Answer Tools Calling All tools Image After... Figure 5: Comparison of restoration strategies. Pipeline blindly applies all tools, MLLM-Judge applies tools to address MLLM-identified degradations, Human Expert uses manually selected tools, and DocIntent performs answerability-guided selective restoration. See Section 4.3 for detailed definitions. question-answering performance on degraded doc- uments but also enhances answer stability across diverse degradation scenarios. Figure 3 presents cases where DocIntent enables correct answers on degraded images. 4.3 Comparisons with Other Strategies Following AgenticIRâs evaluation of random tool invocation (Zhu et al., 2024), we design two restoration baselines, Pipeline and MLLM-Judge. Pipeline applies all image restoration tools sequen- tially without degradation assessment. MLLM- Judge first uses the MLLM to identify degradations in the image and then invokes the corresponding restoration tools. For both strategies, we test ran- dom and fixed tool orders. Random shuffles the tools, whereas fixed follows the sequence of orien- tation correction, dewarping, demoirĂ©ing, deblur- ring, deshadowing, and appearance enhancement. Pipeline excludes orientation correction because it requires MLLM judgment. The Human Ex- pert strategy involves 40 experts with document restoration experience who identify degradations and select tools to optimize overall document read- ability without access to the corresponding ques- tions. For Human Expert (Question-aware), ex- perts locate question-relevant evidence and select tools to improve its readability, while skipping restoration when the evidence is already clearly visible. Unlike DocIntent, all comparison strate- gies answer directly from the preprocessed images without Comparison-Based Rollback. As shown in Table 2, the Pipeline strategy blindly invokes all tools, causing severe informa- tion corruption and substantial performance degra- dation. The MLLM-Judge strategy reduces tool invocations to an average of 2.78 and 2.47 invoca- tions but exhibits performance instability. This is 7 ModelMethodAvg. ToolsAvg. ScoreAvg. Cons. Qwen3.5-9B No restoration0.0068.549.7 Pipeline-random5.0063.245.7 Pipeline-fixed5.0063.547.6 MLLM-Judge-random2.7867.748.8 MLLM-Judge-fixed2.7868.550.7 Human Expert2.3571.856.3 Human Expert (Question-aware)1.5572.257.1 DocIntent (Ours)1.6971.655.6 Gemini-3.1-Pro No restoration0.0083.075.5 Pipeline-random5.0079.068.9 Pipeline-fixed5.0079.569.3 MLLM-Judge-random2.4782.975.3 MLLM-Judge-fixed2.4783.175.2 Human Expert2.3584.079.7 Human Expert (Question-aware)1.5584.580.4 DocIntent (Ours)1.1684.3 80.0 Table 2: Comparison of different restoration strategies on WildDoc benchmark. "Avg." indicates the average results across all three subtasks. Top results are bolded and second-bestunderlined. Complete results for each subtask are provided in Appendix H. partly due to MLLM misjudgments that may trig- ger inappropriate tool usage. The fixed sequence and random ordering show only marginal differ- ences, indicating that tool sequencing is not the key limitation. Moreover, even with the compre- hensive restoration strategy planned by human ex- perts, restoration tools themselves may introduce text distortion, potentially causing information loss or readability degradation. The absence of a roll- back mechanism prevents these errors from being corrected. Both MLLM-Judge and the question- agnostic Human Expert baseline restore all identi- fied degradations, thereby accumulating risks from restoration operations, as illustrated in Figure 5. The results show that incorporating question awareness helps human experts reduce tool invo- cations while improving document VQA perfor- mance, validating the importance of integrating downstream questions into restoration planning. Without human intervention, DocIntent achieves results close to question-aware human expert pre- processing on both models, demonstrating its po- tential to automate selective document restoration. 4.4 Ablation Study We analyze the effectiveness of each core compo- nent in DocIntent through ablation experiments. As shown in Table 3, the Prompt-only setting, which enables the Answerability-Guided Prompt while disabling restoration tools and rollback, fails to improve the VQA score and slightly degrades the performance of Qwen3.5-9B. This result shows that more detailed prompting alone cannot explain DocIntentâs gains, which primarily come from the selective invocation of restoration tools through ModelAG PromptAGRes.CBRb.Avg. ScoreAvg. Cons. Qwen3.5-9B â68.549.7 ââ67.849.2 ââ69.250.5 â71.655.6 Gemini-3.1-Pro â83.075.5 ââ82.975.7 ââ83.778.4 â84.380.0 Table 3: Ablation study on DocIntent components on WildDoc benchmark."AG Prompt" denotes answerability-guided prompting. The Prompt-only set- ting enables AG Prompt while disabling AGRes and CBRb. "Avg." indicates the average results across all three subtasks. Complete results for each subtask are provided in Appendix H. Answerability-Guided Restoration (AGRes). With AGRes, the average score and consistency of Gemini-3.1-Pro improve from 83.0% and 75.5% to 83.7% and 78.4%, respectively. Adding the Comparison-Based Rollback module (CBRb) fur- ther improves them to 84.3% and 80.0%. On Qwen3.5-9B, AGRes brings modest gains, while the complete DocIntent improves the average score and consistency to 71.6% and 55.6%, respectively. These results demonstrate the effectiveness of both components. For the more capable Gemini- 3.1-Pro, AGRes provides effective restoration de- cisions and brings stable improvements, while CBRb further provides a quality safeguard. For the less capable Qwen3.5-9B, AGRes brings lim- ited gains, making CBRb more critical. CBRb promptly identifies and rolls back inappropriate restorations through quality comparison (Figure 6), bringing greater performance gains to weaker mod- els. These results validate the complete system design: answerability-guided restoration provides selective processing capability, the rollback mech- anism provides a quality safeguard, and their syn- ergy achieves stable improvements across models of different capability levels. 5 Conclusion In this paper, we demonstrate that beyond the qual- ity of restoration tools, the critical factor for docu- ment restoration in VQA is the intelligence of the orchestration strategy: determining when to restore, which tool to use, and whether to accept the results. Answerability-guided selective restoration is es- sential for performance improvement. Based on these insights, we present DocIntent, a training-free document restoration framework that achieves se- lective restoration with a quality safeguard through 8 a two-phase cyclic mechanism. Extensive experi- ments demonstrate that our method improves mod- elsâ question-answering performance on degraded documents through efficient tool utilization. Limitations While our approach efficiently utilizes restoration tools to assist MLLMs in document question an- swering on degraded images, the systemâs perfor- mance is fundamentally limited by the capabilities of the underlying restoration tools. Additionally, as a training-free agentic framework, the systemâs judgment capability depends on the intrinsic abili- ties of the underlying MLLM. Ethical Considerations This research poses no significant ethical risks. All experiments are conducted on publicly available open-source datasets, without involving any person- ally identifiable information or sensitive content. This work only restores degraded documents and does not involve content tampering or forgery, en- suring no additional ethical concerns. References Anthropic. 2025. Claude Sonnet 4.6. Accessed: 2026- 04-14. Srikar Appalaraju, Peng Tang, Qi Dong, Nishant Sankaran, Yichu Zhou, and R Manmatha. 2024. Doc- Formerv2: Local features for document understand- ing. In Proceedings of the AAAI conference on artifi- cial intelligence, volume 38, pages 709â718. Bytedance Seed Team. 2025. Seed 2.0 model card: To- wards intelligence frontier for real-world complexity. Accessed: 2026-05-09. Haoyu Chen, Wenbo Li, Jinjin Gu, Jingjing Ren, Sixi- ang Chen, Tian Ye, Renjing Pei, Kaiwen Zhou, Fen- glong Song, and Lei Zhu. 2024. RestoreAgent: Au- tonomous image restoration agent via multimodal large language models. Advances in Neural Informa- tion Processing Systems, 37:110643â110666. Giordano Cicchetti and Danilo Comminiello. 2024. NAF-DPM: A nonlinear activation-free diffusion probabilistic model for document enhancement. arXiv preprint arXiv:2404.05669. Hao Feng, Yuechen Wang, Wengang Zhou, Jiajun Deng, and Houqiang Li. 2021. DocTR: Document image transformer for geometric unwarping and illumina- tion correction. arXiv preprint arXiv:2110.12942. Google DeepMind. 2026. Gemini 3.1 pro.https: //deepmind.google/models/gemini/pro/.Ac- cessed: 2026-05-09. Anwen Hu, Haiyang Xu, Liang Zhang, Jiabo Ye, Ming Yan, Ji Zhang, Qin Jin, Fei Huang, and Jingren Zhou. 2025. mPLUG-DocOwl2: High-resolution compressing for OCR-free multi-page document un- derstanding. In Proceedings of the 63rd Annual Meet- ing of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5817â5834. Mingxin Huang, Yuliang Liu, Dingkang Liang, Lianwen Jin, and Xiang Bai. 2024. Mini-monkey: Alleviating the semantic sawtooth effect for lightweight mllms via complementary image pyramid. arXiv preprint arXiv:2408.02034. Yupan Huang, Tengchao Lv, Lei Cui, Yutong Lu, and Furu Wei. 2022. Layoutlmv3: Pre-training for doc- ument ai with unified text and image masking. In Proceedings of the 30th ACM international confer- ence on multimedia, pages 4083â4091. Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, and 1 others. 2024. GPT-4o system card. arXiv preprint arXiv:2410.21276. Guillaume Jaume, Hazim Kemal Ekenel, and Jean- Philippe Thiran. 2019. Funsd: A dataset for form understanding in noisy scanned documents. arXiv preprint arXiv:1905.13538. Geewook Kim, Teakgyu Hong, Moonbin Yim, Jinyoung Park, Jinyeong Yim, Wonseok Hwang, Sangdoo Yun, Dongyoon Han, and Seunghyun Park. 2021. Donut: Document understanding transformer without OCR. arXiv preprint arXiv:2111.15664, 7(15):2. Yoonsik Kim, Moonbin Yim, and Ka Yeon Song. 2024. TableVQA-Bench: A visual question answering benchmark on multiple table domains. arXiv preprint arXiv:2404.19205. Bingchen Li, Xin Li, Yiting Lu, and Zhibo Chen. 2025. Hybrid agents for image restoration. arXiv preprint arXiv:2503.10120. Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, and 1 others. 2024a. Llava- onevision: Easy visual task transfer. arXiv preprint arXiv:2408.03326. Zhang Li, Biao Yang, Qiang Liu, Zhiyin Ma, Shuo Zhang, Jingxu Yang, Yabo Sun, Yuliang Liu, and Xiang Bai. 2024b. Monkey: Image resolution and text label are important things for large multi-modal models. In proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 26763â26773. Ahmed Masry, Xuan Long Do, Jia Qing Tan, Shafiq Joty, and Enamul Hoque. 2022. ChartQA: A benchmark for question answering about charts with visual and logical reasoning. In Findings of the association for computational linguistics: ACL 2022, pages 2263â 2279. 9 Minesh Mathew, Viraj Bagal, RubĂšn Tito, Dimosthe- nis Karatzas, Ernest Valveny, and CV Jawahar. 2022. InfographicVQA. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vi- sion, pages 1697â1706. Minesh Mathew, Dimosthenis Karatzas, and CV Jawa- har. 2021. DocVQA: A dataset for vqa on document images. In Proceedings of the IEEE/CVF winter con- ference on applications of computer vision, pages 2200â2209. Kalyan Nandi, S Siva Sathya, and 1 others. 2026. Doc- ument visual question answering: A comprehensive analysis of methods, datasets, and challenges. In 2026 9th International Conference on Computational Intelligence in Data Science (ICCIDS), pages 1â6. IEEE. Seunghyun Park, Seung Shin, Bado Lee, Junyeop Lee, Jaeheung Surh, Minjoon Seo, and Hwalsuk Lee. 2019. Cord: a consolidated receipt dataset for post-OCR parsing. In Workshop on document intelligence at NeurIPS, volume 2019, page 5. Dezhi Peng, Zhenhua Yang, Jiaxin Zhang, Chongyu Liu, Yongxin Shi, Kai Ding, Fengjun Guo, and Lianwen Jin. 2024. UPOCR: Towards unified pixel-level OCR interface. In Forty-first International Conference on Machine Learning. Qwen Team. 2026a. Qwen3.5: Towards native multi- modal agents. Qwen Team. 2026b. Qwen3.6-27B: Flagship-level cod- ing in a 27B dense model. Aaditya Singh, Adam Fry, Adam Perelman, Adam Tart, Adi Ganesh, Ahmed El-Kishky, Aidan McLaughlin, Aiden Low, AJ Ostrow, Akhila Ananthram, and 1 others. 2025. Openai GPT-5 system card. arXiv preprint arXiv:2601.03267. Mohamed Ali Souibgui, Changkyu Choi, Andrey Barsky, Kangsoo Jung, Ernest Valveny, and Dimos- thenis Karatzas. 2025. DocVXQA: Context-aware visual explanations for document question answering. arXiv preprint arXiv:2505.07496. Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean- Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Mil- lican, and 1 others. 2023. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805. Qwen Team. 2025. Qwen2.5-vl. An-Lan Wang, Jingqun Tang, Lei Liao, Hao Feng, Qi Liu, Xiang Fei, Jinghui Lu, Han Wang, Hao Liu, Yuliang Liu, and 1 others. 2025a. WildDoc: How far are we from achieving comprehensive and robust document understanding in the wild? In Proceed- ings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 23002â23012. Weiyun Wang, Zhangwei Gao, Lixin Gu, Hengjun Pu, Long Cui, Xingguang Wei, Zhaoyang Liu, Linglin Jing, Shenglong Ye, Jie Shao, and 1 others. 2025b. In- ternvl3.5: Advancing open-source multimodal mod- els in versatility, reasoning, and efficiency. arXiv preprint arXiv:2508.18265. Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. 2004. Image quality assessment: from error visibility to structural similarity. IEEE transac- tions on image processing, 13(4):600â612. Yang Xu, Yiheng Xu, Tengchao Lv, Lei Cui, Furu Wei, Guoxin Wang, Yijuan Lu, Dinei Florencio, Cha Zhang, Wanxiang Che, and 1 others. 2021. Lay- outlmv2: Multi-modal pre-training for visually-rich document understanding. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer- ence on Natural Language Processing (Volume 1: Long Papers), pages 2579â2591. Zemin Yang, Yujing Sun, Xidong Peng, Siu Ming Yiu, and Yuexin Ma. 2025. UniDemoirĂ©: Towards uni- versal image demoirĂ©ing with data generation and synthesis. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 9354â9362. Tianyu Yu, Zefan Wang, Chongyi Wang, Fuwei Huang, Wenshuo Ma, Zhihui He, Tianchi Cai, Weize Chen, Yuxiang Huang, Yuanqian Zhao, Bokai Xu, Junbo Cui, Yingjing Xu, Liqing Ruan, Luoyuan Zhang, Hanyu Liu, Jingkun Tang, Hongyuan Liu, Qining Guo, and 15 others. 2025. MiniCPM-V 4.5: Cook- ing efficient mllms via architecture, data, and training recipe. Preprint, arXiv:2509.18154. Syed Waqas Zamir, Aditya Arora, Salman Khan, Mu- nawar Hayat, Fahad Shahbaz Khan, and Ming-Hsuan Yang. 2022. Restormer: Efficient transformer for high-resolution image restoration. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5728â5739. Jiaxin Zhang, Dezhi Peng, Chongyu Liu, Peirong Zhang, and Lianwen Jin. 2024. DocRes: A generalist model toward unifying document image restoration tasks. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, pages 15654â 15664. Jiaxin Zhang, Peirong Zhang, Huiyi Cheng, Xinhong Chen, Haowei Xu, Kai Ding, and Lianwen Jin. 2026a. DocAligner: Automating the Annotation of Pho- tographed Documents through Real-Virtual Align- ment. Pattern Recognition, 171:112191. Jinxu Zhang and Yu Zhang. 2025. DocRouter: Prompt guided vision transformer and mixture of experts connector for document understanding. Information Fusion, 122:103206. Ling Zhang, Yinghao He, Qing Zhang, Zheng Liu, Xi- aolong Zhang, and Chunxia Xiao. 2023. Document image shadow removal guided by color-aware back- ground. In Proceedings of the IEEE/CVF Conference 10 on Computer Vision and Pattern Recognition, pages 1818â1827. Peirong Zhang, Haowei Xu, Jiaxin Zhang, Xuhan Zheng, Guitao Xu, Yuyi Zhang, Junle Liu, Zhenhua Yang, Wei Zhou, and Lianwen Jin. 2025a. OCR- GenBench: A Comprehensive Benchmark for Evalu- ating OCR Generative Capabilities. arXiv preprint arXiv:2507.15085. Peirong Zhang, Jiaxin Zhang, Jiahuan Cao, Hongliang Li, and Lianwen Jin. 2025b. Smaller But Better: Unifying Layout Generation with Smaller Large Lan- guage Models. International Journal of Computer Vision, 133:3891â3917. Yisheng Zhang, Guoli Jia, Haote Hu, Shanxu Zhao, Kaikai Zhao, Long Sun, Xinwei Long, Kai Tian, Che Jiang, Zhaoxiang Liu, and 1 others. 2026b. Tir-agent: Training an explorative and efficient agent for image restoration. arXiv preprint arXiv:2603.27742. Fangmin Zhao, Weichao Zeng, Zhenhang Li, Dongbao Yang, Binbin Li, Xiaojun Bi, and Yu Zhou. 2025. Uni-DocDiff: A unified document restoration model based on diffusion. In Proceedings of the 33rd ACM International Conference on Multimedia, pages 8204â 8213. Zhen Zhao, Jingqun Tang, Binghong Wu, Chunhui Lin, Shu Wei, Hao Liu, Xin Tan, Zhizhong Zhang, Can Huang, and Yuan Xie. 2024. Harmonizing visual text comprehension and generation. Advances in Neural Information Processing Systems, 37:97499â97522. Yingjie Zhou, Jiezhang Cao, Zicheng Zhang, Farong Wen, Yanwei Jiang, Jun Jia, Xiaohong Liu, Xiongkuo Min, and Guangtao Zhai. 2025. Q-Agent: quality- driven chain-of-thought image restoration agent through robust multimodal large language model. arXiv preprint arXiv:2504.07148. Yuxuan Zhou, Baole Wei, Xingjian Hu, Haowei Chen, Yu Li, Xingyue Lin, Liangcai Gao, and Zhi Tang. Uni-docrobust: Universal plug-and-play robustness enhancement for multi-modal llms via feature restora- tion. In Forty-third International Conference on Ma- chine Learning. Kaiwen Zhu, Jinjin Gu, Zhiyuan You, Yu Qiao, and Chao Dong. 2024. An intelligent agentic system for complex image restoration problems. arXiv preprint arXiv:2410.17809. Zhaoqing Zhu, Chuwei Luo, Zirui Shao, Feiyu Gao, Hangdi Xing, Qi Zheng, and Ji Zhang. 2025. A simple yet effective layout token in large language models for document understanding. In Proceed- ings of the Computer Vision and Pattern Recognition Conference, pages 14472â14482. Yushen Zuo, Qi Zheng, Mingyang Wu, Xinrui Jiang, Renjie Li, Jian Wang, Yide Zhang, Gengchen Mai, Li- hong V Wang, James Zou, and 1 others. 2025. 4KA- gent: agentic any image to 4k super-resolution. arXiv preprint arXiv:2507.07105. 11 Appendix A Evaluation Metrics We provide detailed definitions of the evaluation metrics used in our experiments. The accuracy metric quantifies the proportion of questions where the predicted answer precisely cor- responds with any of the designated target answers for that question. For the Average Normalized Levenshtein Simi- larity (ANLS), we follow previous works (Mathew et al., 2021), which is defined as follows: ANLS = 1 N N X i=1 (max j s(a ij ,o q i ))(1) wheres(a ij ,o q i )is defined as1â NL(a ij ,o q i ) whenNL(a ij ,o q i ), the normalized Levenshtein distance, is less than a predefined thresholdÏ; oth- erwise 0. We set the thresholdÏ = 0.5, as previous works do (Mathew et al., 2021). Following WildDoc (Wang et al., 2025a), we also report the consistency score, a robustness met- ric designed to assess the resilience of models when handling the same question across images captured under various conditions. This metric calculates the document-level accuracy. A modelâs response is considered accurate only if it correctly answers the question in all four distinct scenarios presented. B Algorithm Workflow Algorithm 1 describes the complete inference work- flow of DocIntent. The functions Analyze, Select- Tool, Compare, and Answer are all implemented by the MLLM. In the Analyze function, the MLLM analyzes the current image and question to deter- mine whether it can be directly answered. In the SelectTool function, the MLLM identifies degra- dation types that hinder answering and selects the tool corresponding to the most severe degradation from the available tool list. The Execute function invokes the actual restoration tools. In the Compare function, the MLLM compares the images before and after restoration with respect to the question to determine whether the restoration improves the readability of question-relevant evidence. In the Answer function, the MLLM generates the answer to the question based on the final image. We set a maximum iteration countTfor the in- ference workflow to prevent the system from falling into excessive repetitive tool invocations in extreme Algorithm 1 DocIntent Workflow Require:Initial document imageI 0 , QuestionQ, Maximum iterations T Ensure: Answer A 1: toolsâ ALL_TOOLS 2: for t = 0 to T â 1 do 3: can_answer â Analyze(I t ,Q) 4:if can_answer then 5:return Answer(I t ,Q) 6:end if 7: toolâ SelectTool(I t ,Q,tools) 8:if tool is None then 9:return Answer(I t ,Q) 10:end if 11: I t+1 â Execute(tool,I t ) 12: is_improvedâ Compare(I t ,I t+1 ,Q) 13:if not is_improved then 14: I t+1 â I t 15: toolsâ tools\tool 16:end if 17: end for 18: return Answer(I T ,Q) cases where the comparison-based rollback mech- anism fails. In all experiments, we setT = 10, providing sufficient restoration redundancy for the system. C Additional Experiments To evaluate DocIntent across models with differ- ent capability levels, we conduct additional ex- periments on more MLLMs. As shown in Ta- ble 4, DocIntent achieves performance improve- ments across all tested models. These additional ex- periments cover the open-source model Qwen3.5- 27B (Qwen Team, 2026a) and closed-source mod- els including GPT-4o (Hurst et al., 2024), Qwen3.5- Plus (Qwen Team, 2026a), Claude Sonnet 4.6 (An- thropic, 2025), Doubao-2.0-Pro (Bytedance Seed Team, 2025), and GPT-5.2 (Singh et al., 2025), showing consistent improvements on the evaluated MLLMs. D Analysis of Tool Invocation Decisions Different MLLMs vary in their sensitivity to degra- dation types; tool invocation decisions should there- fore be analyzed in the context of the particular model and downstream task. Table 6 uses direct answering for all categories and partitions samples into the Confident subset, which the model judges 12 Model WildDocVQAWildChartQAWildTableVQAAverage ANLSConsistencyAcc.ConsistencyAcc.ConsistencyScoreConsistency Qwen3.5-27B (Qwen Team, 2026a)83.372.853.243.676.649.271.055.2 w/ DocIntent84.1 +0.874.4 +1.654.9 +1.750.8 +7.286.3 +9.758.8 +9.675.1 +4.161.3 +6.1 GPT-4o (Hurst et al., 2024)63.241.631.923.259.630.851.631.9 w/ DocIntent73.7 +10.558.4 +16.835.8 +3.934.4 +11.264.2 +4.638.4 +7.657.9 +6.343.7 +11.8 Qwen3.5-Plus (Qwen Team, 2026a)84.477.062.552.481.559.676.163.0 w/ DocIntent91.0 +6.682.4 +5.467.9 +5.459.6 +7.286.1 +4.667.6 +8.081.7 +5.669.9 +6.9 Claude Sonnet 4.6 (Anthropic, 2025)63.438.450.031.659.121.657.530.5 w/ DocIntent71.9 +8.549.6 +11.257.0 +7.039.2 +7.670.3 +11.236.4 +14.866.4 +8.941.7 +11.2 Doubao-2.0-Pro (Bytedance Seed Team, 2025)82.772.862.352.084.665.676.563.5 w/ DocIntent83.0 +0.374.8 +2.067.4 +5.158.0 +6.084.9 +0.366.4 +0.878.4 +1.966.4 +2.9 GPT-5.2 (Singh et al., 2025)66.137.658.344.864.233.662.938.7 w/ DocIntent76.7 +10.652.8 +15.262.5 +4.248.8 +4.069.4 +5.241.6 +8.069.5 +6.647.7 +9.0 Table 4: Performance of additional MLLMs w/ and w/o DocIntent on WildDoc benchmark (Wang et al., 2025a). "ANLS" denotes Average Normalized Levenshtein Similarity. ModelContext Strategy WildDocVQAWildChartQAWildTableVQAAverage ANLSConsistencyTokensAcc.ConsistencyTokensAcc.ConsistencyTokensScoreConsistencyTokens Qwen3.5-9B No Context84.675.61588040.732.82979271.543.42582665.650.623833 Full Context86.176.02796754.941.63073573.546.02961571.554.529439 Text-only (Ours)85.476.41796753.642.01961175.748.41951871.655.619032 Gemini-3.1-Pro No Context92.588.0490461.656.4532290.781.2432281.675.24849 Full Context94.092.0735266.362.8762592.485.7714784.280.27375 Text-only (Ours)94.191.6581767.763.6679191.284.8571184.380.06106 Table 5: Comparison of different context management strategies. "ANLS" denotes Average Normalized Levenshtein Similarity. Within each model, top results are bolded and second-best underlined. answerable directly, and the Forced subset, which the model judges require tool assistance. The Con- fident subset usually scores higher than the overall baseline, whereas the Forced subset scores lower, indicating that answerability judgments distinguish samples with different levels of direct-answering difficulty. To further assess the downstream utility of tool invocation, Table 7 compares Direct Answer and DocIntent on each modelâs Forced subset. DocIn- tent obtains higher scores for all models and sub- tasks, showing that Answerability-Guided Restora- tion provides consistent aggregate gains on samples that are more difficult to answer directly. Together, the two tables show that DocIntent improves down- stream VQA performance through selective restora- tion on samples judged to require tool assistance. E Context Management Strategy In agentic frameworks, context management strate- gies strongly affect system performance and effi- ciency. We explore three different context manage- ment approaches. The No Context strategy uses independent conversations for both Phase A and Phase B, retaining no historical context. The Full Context strategy retains all conversation history and intermediate restored images, providing com- plete reasoning history. The Text-only strategy, our approach, retains text context such as tool invoca- tion decisions and quality assessment results but does not retain intermediate restored images. Ta- ble 5 presents the experimental results of the three strategies. The experimental results reveal that while the No Context strategy can reduce token consump- tion, it leads to notable performance degradation. For example, Qwen3.5-9B achieves only 40.7% accuracy on WildChartQA under the No Context strategy, substantially lower than other strategies. We attribute this to the fact that historical context helps models make better judgments and complete tasks more effectively. Additionally, the No Con- text strategy may cause smaller models to misjudge during the Phase B quality comparison, leading to repeated invocations of the same tool and conse- quently increased token consumption. The Full Context strategy achieves high accuracy but incurs substantially higher token consumption. For in- stance, Qwen3.5-9Bâs average token consumption is 29439, 1.55 times that of the Text-only strategy. Furthermore, for less capable models, multiple im- ages in long contexts can cause attention dispersion issues. In contrast, the Text-only strategy achieves an excellent balance between performance and ef- 13 ModelCategoryWildDocVQAWildChartQAWildTableVQAAverage ANLSAcc.Acc.Score Qwen3.5-9B Baseline82.551.971.168.5 Confident84.6 +2.154.4 +2.583.7 +12.674.2 +5.7 Forced78.6 -3.948.8 -3.157.6 -13.561.7 -6.8 Gemini-3.1-Pro Baseline92.866.789.583.0 Confident95.6 +2.870.9 +4.291.1 +1.685.9 +2.9 Forced90.3 -2.560.7 -6.087.6 -1.979.5 -3.5 Qwen3.5-27B Baseline83.353.276.671.0 Confident90.0 +6.761.3 +8.183.2 +6.678.2 +7.2 Forced72.5 -10.841.6 -11.665.7 -10.959.9 -11.1 GPT-4o Baseline63.231.959.651.6 Confident86.3 +23.140.0 +8.179.7 +20.168.7 +17.1 Forced43.4 -19.826.7 -5.245.4 -14.238.5 -13.1 Qwen3.5-Plus Baseline84.462.581.576.1 Confident95.5 +11.165.5 +3.090.9 +9.484.0 +7.9 Forced73.4 -11.060.4 -2.174.3 -7.269.4 -6.7 Claude Sonnet 4.6 Baseline63.450.059.157.5 Confident85.8 +22.458.7 +8.777.1 +18.073.9 +16.4 Forced56.7 -6.737.7 -12.344.7 -14.446.4 -11.1 Doubao-2.0-Pro Baseline82.762.384.676.5 Confident87.0 +4.364.5 +2.287.5 +2.979.7 +3.2 Forced69.5 -13.256.1 -6.267.0 -17.664.2 -12.3 GPT-5.2 Baseline66.158.364.262.9 Confident87.3 +21.264.3 +6.078.7 +14.576.8 +13.9 Forced41.9 -24.250.2 -8.150.4 -13.847.5 -15.4 Table 6: Analysis of tool invocation decisions where all categories employ direct answering w/o DocIntent. "ANLS" denotes Average Normalized Levenshtein Sim- ilarity. ModelMethod WildDocVQAWildChartQAWildTableVQAAverage ANLSAcc.Acc.Score Qwen3.5-9B Direct Answer78.648.857.661.7 DocIntent86.9 +8.352.6 +3.867.1 +9.568.9 +7.2 Gemini-3.1-Pro Direct Answer90.360.787.679.5 DocIntent92.8 +2.563.1 +2.491.3 +3.782.4 +2.9 Qwen3.5-27B Direct Answer72.541.665.759.9 DocIntent74.6 +2.145.7 +4.191.4 +25.770.6 +10.7 GPT-4o Direct Answer43.426.745.438.5 DocIntent62.9 +19.533.1 +6.453.3 +7.949.8 +11.3 Qwen3.5-Plus Direct Answer73.460.474.369.4 DocIntent86.5 +13.169.6 +9.282.4 +8.179.5 +10.1 Claude Sonnet 4.6 Direct Answer56.737.744.746.4 DocIntent67.7 +11.054.6 +16.964.9 +20.262.4 +16.0 Doubao-2.0-Pro Direct Answer69.556.167.064.2 DocIntent70.7 +1.275.6 +19.569.1 +2.171.8 +7.6 GPT-5.2 Direct Answer41.950.250.447.5 DocIntent64.6 +22.760.1 +9.960.5 +10.161.7 +14.2 Table 7: Comparison on the Forced subset, where DocIn- tent decides to invoke restoration tools. "ANLS" denotes Average Normalized Levenshtein Similarity. ficiency, attaining the best or near-best results on most metrics while effectively controlling token consumption. Based on these observations, we chose the Text-only strategy after weighing perfor- mance and efficiency, leveraging MLLMsâ image understanding capabilities for accurate decisions based on current images while retaining text con- text to provide necessary historical experience and avoid repeated errors. F Reproducibility Details The system prompts and tool strategies were de- veloped using samples outside the evaluation sets. These samples were used only to validate the end- to-end workflow and determine answer formats, without tuning based on evaluation results. Input step1 Appearance Enhancement Try to use the appearance enhancement tool. Tool produces visual artifact and blur, rollback and try the deblurring tool. Q:What is the full form of ASBC? step2 Deblurring American Society of Biological Chemists Figure 6: Rollback mechanism demonstration. Failed restoration triggers rollback and alternative attempt. All document images are dynamically resized while preserving their original aspect ratios, with the longest edge capped at 2,048 pixels to avoid API payload limits and out-of-memory errors. For all evaluated open- and closed-source models, we settemperature=0.0. Where supported, we usetop_p=1.0,frequency_penalty=0.0, and presence_penalty=0.0. The maximum output length is set to 4,096 tokens, while all other gener- ation parameters retain their default values. All restoration tools use their official open- source checkpoints and default inference scripts. We adapt each restoration model to DocIntent through the standardized tool interface described in Section 3.1. G Cost Analysis Table 14 reports the average token consumption of different models with and without DocIntent. The additional overhead mainly arises from answerabil- ity assessment and comparison between the images before and after restoration, which support selec- tive restoration and Comparison-Based Rollback. For closed-source MLLMs, Table 10 further re- ports the exact API versions, official input/output token prices at the time of evaluation, and DocIn- tentâs average token usage and per-sample cost. For example, Gemini-3.1-Pro consumes an average of 5,033 input tokens and 1,073 output tokens, corre- sponding to approximately $0.0394 per sample. The cost also depends on each modelâs tool invo- cation strategy. Doubao-2.0-Pro (Bytedance Seed Team, 2025) invokes tools 0.69 times per sample on average, whereas GPT-4o (Hurst et al., 2024) and GPT-5.2 (Singh et al., 2025) invoke tools ap- proximately twice, indicating that model-specific decision preferences affect the final token consump- tion. Overall, DocIntent trades additional inference 14 ModelMethodAvg. Tools WildDocVQAWildChartQAWildTableVQAAverage ANLSConsistencyAcc.ConsistencyAcc.ConsistencyScoreConsistency Qwen3.5-9B No restoration0.0082.570.851.933.271.145.268.549.7 Pipeline-random5.0078.262.451.432.860.042.063.245.7 Pipeline-fixed5.0080.669.651.031.658.941.663.547.6 MLLM-Judge-random2.7882.968.850.132.070.145.667.748.8 MLLM-Judge-fixed2.7882.770.651.734.871.246.868.550.7 Human Expert2.3585.7 75.653.643.276.250.071.856.3 Human Expert (Question-aware)1.5586.376.853.543.676.950.872.257.1 DocIntent (Ours)1.6985.476.453.642.075.748.471.655.6 Gemini-3.1-Pro No restoration0.0092.888.066.759.689.578.883.075.5 Pipeline-random5.0089.175.261.454.486.477.279.068.9 Pipeline-fixed5.0089.777.261.754.087.076.879.569.3 MLLM-Judge-random2.4792.787.265.457.290.681.682.975.3 MLLM-Judge-fixed2.4792.286.466.257.690.981.683.175.2 Human Expert2.3593.589.667.465.291.084.4 84.079.7 Human Expert (Question-aware)1.5594.091.368.065.291.484.884.580.4 DocIntent (Ours)1.1694.191.667.763.691.284.884.380.0 Table 8: Complete results of restoration strategy comparison on WildDoc benchmark. "ANLS" denotes Average Normalized Levenshtein Similarity. "Avg. Tools" denotes the average number of tool invocations per sample. Top results are bolded and second-best underlined. ModelAG PromptAGRes.CBRb. WildDocVQAWildChartQAWildTableVQAAverage ANLSConsistencyAcc.ConsistencyAcc.ConsistencyScoreConsistency Qwen3.5-9B â82.570.851.933.271.145.268.549.7 ââ82.070.351.033.770.443.767.849.2 ââ84.069.652.135.671.546.469.250.5 â85.476.453.642.075.748.471.655.6 Gemini-3.1-Pro â92.888.066.759.689.578.883.075.5 ââ92.987.266.660.889.279.082.975.7 ââ94.090.866.260.890.883.683.778.4 â94.191.667.763.691.284.884.380.0 Table 9: Complete results of ablation study on DocIntent components. "AG Prompt" denotes answerability-guided prompting. The Prompt-only setting enables AG Prompt while disabling AGRes and CBRb. "ANLS" denotes Average Normalized Levenshtein Similarity. Top results are bolded and second-best underlined. cost for improvements in document VQA perfor- mance and consistency. In practical deployment, the backbone model can be selected according to the desired balance between performance and cost. H Complete Experimental Results This section provides the complete experimental results for both the strategy comparison (Table 8) and ablation study (Table 9). These tables present detailed performance metrics across all three sub- tasks of the WildDoc benchmark: WildDocVQA, WildChartQA, and WildTableVQA. I Impact of Tool Selection To analyze the impact of different tool choices on DocIntent, we conduct comparative experiments with two tool backend configurations. Group 1 employs DocRes (Zhang et al., 2024) for deblur- ring, dewarping, deshadowing, and appearance enhancement, combined with UniDemoirĂ© (Yang et al., 2025) for demoirĂ©ing. Group 2 uses NAF- DPM (Cicchetti and Comminiello, 2024) for de- blurring, BGShadowNet (Zhang et al., 2023) for deshadowing, DocRes (Zhang et al., 2024) for de- warping and appearance enhancement, and UniDe- moirĂ© (Yang et al., 2025) for demoirĂ©ing. Table 11 presents the experimental results. For Qwen3.5-9B, Group 1 achieves a 71.6% aver- age score and 55.6% consistency, while Group 2 obtains 70.9% and 54.4%. For Gemini-3.1-Pro, Group 1 achieves an 84.3% average score and 80.0% consistency, while Group 2 achieves 84.1% and 78.9%. The two configurations show similar performance levels across both models. These re- sults indicate that DocIntent achieves effective per- formance improvements regardless of tool backend choices. The answerability-guided restoration strat- egy and comparison-based rollback mechanism can effectively coordinate different restoration tool combinations. Based on overall performance, we adopt the Group 1 configuration in our main exper- iments. Notably, our contribution lies in effectively 15 ModelAPI VersionPrice ($/1M, In/Out)Avg. Tokens (In/Out)Cost/Sample ($) Gemini-3.1-Pro gemini-3.1-pro-preview4.00/18.005,033/1,0730.0394 GPT-4o gpt-4o-2024-08-062.50/10.0012,004/2,4680.0547 GPT-5.2 gpt-5.2-2025-12-111.75/14.0016,758/2,4440.0635 Claude Sonnet 4.6 claude-sonnet-4-65.00/25.0011,124/1,5040.0932 Qwen3.5-Plus qwen3.5-plus-2026-02-150.57/3.4316,814/2,2200.0172 Doubao-2.0-Pro doubao-seed-2-0-pro-2602151.37/6.866,992/9330.0160 Table 10: API versions, token usage, and per-sample inference costs of DocIntent for closed-source MLLMs. Prices are official input/output token prices at the time of evaluation. ModelTools WildDocVQAWildChartQAWildTableVQAAverage ANLSConsistencyAcc.ConsistencyAcc.ConsistencyScoreConsistency Qwen3.5-9B Group 185.476.453.642.075.748.471.655.6 Group 286.376.852.539.673.946.870.954.4 Gemini-3.1-Pro Group 194.191.667.763.691.284.884.380.0 Group 294.491.267.362.890.782.884.178.9 Table 11: Performance comparison of DocIntent with different tools. "ANLS" denotes Average Normalized Levenshtein Similarity. Better results are bolded. ModelMethodANLS Qwen3.5-9B Direct Answer95.3 DocIntent94.7 Gemini-3.1-Pro Direct Answer95.6 DocIntent95.4 Table 12: Performance on the clean DocVQA (Mathew et al., 2021) test set. Better results within each model are bolded. orchestrating restoration tools to enhance perfor- mance rather than improving existing restoration tools. J Human Expert Annotation Details We recruited 40 experts with document image pro- cessing experience to construct the Human Expert baseline. All participants are graduate students with relevant experience in document restoration. Before the annotation process, each expert received detailed task guidelines, including an introduction to degradation types in the WildDoc dataset, func- tional descriptions of all restoration tools, and sev- eral pre-annotated examples to ensure they fully understood the task requirements. All samples were evenly distributed among the 40 experts. Each sample was assigned to exactly one expert, who developed its restoration strategy. The annotation task required experts to manu- ally identify degradation types for each test sam- ple and select appropriate tools to optimize overall document readability. Importantly, experts were not provided with the associated questions during annotation, focusing solely on achieving the best overall document readability. The annotation task was completed over a span of 20 days, requiring ap- proximately 10 hours of actual workload per expert, with a total cost of $1,750. All experts participated with informed consent and understood that their an- notation data would be used for academic research purposes. For the question-aware Human Expert base- line, experts were additionally provided with the corresponding questions and instructed to locate question-relevant evidence and select tools to im- prove its readability. Experts could skip restoration when the relevant evidence was already clearly vis- ible. Any orientation correction selected by an expert was applied immediately, allowing subse- quent restoration operations to be planned based on the corrected document. K Repeated-Run Evaluation To evaluate the stability of our results, we repeated the WildDoc evaluation three times for Qwen3.5- 9B and Gemini-3.1-Pro with and without DocIn- tent. Table 13 reports the mean and standard devia- tion across the three runs. DocIntent consistently improves both the average score and consistency for both models, while the small standard devi- ations demonstrate stable performance across re- peated runs. 16 ModelMethod WildDocVQAWildChartQAWildTableVQAAverage ANLSCons.Acc.Cons.Acc.Cons.ScoreCons. Qwen3.5-9B Direct Answer 82.5± 0.1 70.8± 0.1 51.9± 0.3 33.2± 0.2 71.1± 0.1 45.2± 0.2 68.5± 0.1 49.7± 0.1 DocIntent 85.4± 0.2 76.4± 0.1 53.6± 0.4 42.0± 0.4 75.7± 0.2 48.4± 0.1 71.6± 0.3 55.6± 0.2 Gemini-3.1-Pro Direct Answer 92.8± 0.3 88.0± 0.2 66.7± 0.5 59.6± 0.6 89.5± 0.2 78.8± 0.4 83.0± 0.3 75.5± 0.5 DocIntent 94.1± 0.3 91.6± 0.2 67.7± 0.6 63.6± 0.6 91.2± 0.2 84.8± 0.2 84.3± 0.4 80.0± 0.4 Table 13: Mean and standard deviation over three runs on the WildDoc (Wang et al., 2025a) benchmark. Better results within each model are bolded. ModelAvg. ToolsWildDocVQAWildChartQAWildTableVQAAverage Qwen3.5-9B0.004943485861375313 w/ DocIntent1.6917967196111951819032 Gemini-3.1-Pro0.002019222628822376 w/ DocIntent1.165817679157116106 Qwen3.5-27B0.004839465857565084 w/ DocIntent1.4217836165231652816962 GPT-4o0.001710202325532095 w/ DocIntent2.0213894154481407614473 Qwen3.5-Plus0.003440446445304145 w/ DocIntent1.8717347186412111319034 Claude Sonnet 4.60.003227321136703369 w/ DocIntent1.8412264132761234512628 Doubao-2.0-Pro0.003342396442723859 w/ DocIntent0.696537808691527925 GPT-5.20.001867213832172407 w/ DocIntent2.0016204189062249519202 Table 14: Average token consumption per sample w/ and w/o DocIntent across different models and tasks. L Performance on Clean Documents To evaluate whether DocIntent affects question- answering performance on clean documents, we test Qwen3.5-9B and Gemini-3.1-Pro on the DocVQA (Mathew et al., 2021) test set. The results in Table 12 show that DocIntent largely preserves the modelsâ performance on clean data without causing a notable degradation. M Complete Prompt Templates We provide complete prompt templates for all three WildDoc subtasks: WildDocVQA, WildChartQA, and WildTableVQA. These include: (1) DocIntent prompts containing task instructions, answerability- guided restoration strategy, tool descriptions, and comparison-based rollback mechanism; (2) base- line prompts for direct answering in comparison experiments; (3) Phase B comparison prompt for quality assessment and rollback decisions; and (4) tool descriptions provided to the MLLM. To ensure fair comparison, DocIntent and base- line prompts for the same task share identical an- swer format rules, including extraction principles, capitalization preservation, number formatting, and unit symbol handling. The only difference lies in that DocIntent prompts include answerability- guided restoration strategy and tool descriptions, while baseline prompts contain only task instruc- tions and answer format rules without any restora- tion tools or strategy guidance. This design ensures that performance differences stem from the meth- ods themselves rather than answer format varia- tions. DocIntent prompts contain four key components: (1) task instructions defining answer format require- ments, (2) answerability-guided restoration strat- egy that explicitly instructs the MLLM to select the most severe degradation type whose tool remains available when multiple degradations coexist, (3) descriptions of all available tools covering image quality restoration and orientation correction, and (4) mandatory comparison and review mechanism after each tool execution, enforced through auto- matic message insertion and dynamic tool list ad- justment. These prompts guide the MLLM to com- plete tasks through answerability-guided restora- tion with comparison-based rollback. 17 P Prompt for DocIntent on WildDocVQA You are a Document Visual Question Answering (VQA) expert. PRIMARY TASK Answer the question about the document image as accurately as possible. Image restoration tools are available ONLY if you cannot answer confidently with the current image. Your goal is to answer the question, not to achieve perfect image quality. HOW TO SUBMIT YOUR ANSWER Call the 'answer' tool to submit your response with parameters: answer, confidence (low/medium/high), reasoning (optional). ANSWER FORMAT RULES EXTRACT ONLY - DO NOT WRITE SENTENCES Extract the requested information without adding explanatory text. Text from documents: - Preserve original capitalization, punctuation, and formatting - Keep company suffixes and special characters as shown Numbers: - Match the document format exactly - Keep currency symbols, commas, decimals, and percentages as shown Units and symbols: - Include ONLY if shown in document or explicitly asked Extract precision: - Match the level of detail requested by the question Yes/No questions: - Answer ONLY "Yes" or "No" TASK-DRIVEN RESTORATION STRATEGY Your goal is to ANSWER the question, not to fix all image problems. Only use restoration tools when degradation directly prevents you from reading the information needed to answer the question. Step 1: Identify What Information You Need - What specific information does the question ask for? - Where is that information located in the document? Step 2: Analyze Whether Degradation Affects Your Answer Focus ONLY on the region containing the information needed to answer the question. If that region is degraded and unreadable: - Identify the degradation type affecting that specific region (blur, shadow, distortion, rotation, etc.) - Select the appropriate tool to address that degradation - If multiple degradations exist, choose the most severe one that most directly prevents reading - If the most appropriate tool is no longer available (removed after quality degradation), select the next most relevant tool If that region is readable: - Answer directly, ignore degradations in other regions Step 3: Review After Each Tool Use After EVERY tool execution, you enter a COMPARISON phase where you: 1. Compare the processed image with the previous version 2. Make a decision: - Quality improved or acceptable â DO NOT call any tool - Quality worsened â Call rollback(reason="...") During comparison phase, you CANNOT call answer() or other restoration tools. WORKFLOW 1. Examine current image 2. If readable â Call answer directly; If needs improvement â Use appropriate tool 3. After each tool â Compare quality, use rollback if worse 4. Submit answer with extracted information in correct format Figure 7: Prompt for DocIntent on WildDocVQA task. 18 P Prompt for DocIntent on WildChartQA You are a Document Visual Question Answering (VQA) expert. PRIMARY TASK Answer the question about the document image as accurately as possible. Image restoration tools are available ONLY if you cannot answer confidently with the current image. Your goal is to answer the question, not to achieve perfect image quality. HOW TO SUBMIT YOUR ANSWER Call the 'answer' tool to submit your response with parameters: answer, confidence (low/medium/high), reasoning (optional). ANSWER FORMAT RULES Numbers: - Use digits only, without units or words Percentage signs: - ONLY include % if the chart itself shows the % symbol - Do NOT add % based on interpretation Units and symbols: - Give numbers WITHOUT units unless the question explicitly asks for them or the chart shows them - When in doubt about units or %, leave them out Yes/No questions: - Answer ONLY "Yes" or "No" Text extraction: - Extract exactly as shown - Preserve capitalization Time/Date questions: - Be specific and concise - Give specific year/time, not ranges unless asked Comparison questions: - Give only the direct answer without explanatory text General principle: - Prefer simpler, shorter answers - Answer ONLY what is directly asked TASK-DRIVEN RESTORATION STRATEGY Your goal is to ANSWER the question, not to fix all chart problems. Only use restoration tools when degradation directly prevents you from reading the information needed to answer the question. Step 1: Identify What Information You Need - What specific data does the question ask for? - Where is that information in the chart? Step 2: Analyze Whether Degradation Affects Your Answer Focus ONLY on the region containing the data needed to answer the question. If that region is degraded and unreadable: - Identify the degradation type affecting that specific region (blur, shadow, distortion, rotation, etc.) - Select the appropriate tool to address that degradation - If multiple degradations exist, choose the most severe one that most directly prevents reading - If the most appropriate tool is no longer available (removed after quality degradation), select the next most relevant tool If that region is readable: - Answer directly, ignore degradations in other regions Step 3: Review After Each Tool Use After EVERY tool execution, you enter a COMPARISON phase where you: 1. Compare the processed chart with the previous version 2. Make a decision: - Quality improved or acceptable â DO NOT call any tool - Quality worsened â Call rollback(reason="...") During comparison phase, you CANNOT call answer() or other restoration tools. If tools don't help much but you can still see approximate values, answer with medium/low confidence. Tools not helping does not mean "Unable to answer". WORKFLOW 1. Try to answer directly - If you can answer confidently â Call answer tool immediately 2. Use restoration if needed - ONLY use tools if image quality prevents answering 3. Submit answer - Call answer tool with your response and appropriate confidence level Figure 8: Prompt for DocIntent on WildChartQA task. 19 P Prompt for DocIntent on WildTableVQA You are a Table Visual Question Answering (VQA) expert specialized in answering questions about tables. PRIMARY TASK Answer the question about the table image as accurately as possible. Image restoration tools are available ONLY if you cannot answer confidently with the current image. Your goal is to answer the question, not to achieve perfect image quality. HOW TO SUBMIT YOUR ANSWER Call the 'answer' tool to submit your response with parameters: answer, confidence (low/medium/high), reasoning (optional). TABLE FACT VERIFICATION QUESTIONS Do NOT provide the actual values. ANSWER FORMAT RULES For fact verification: - Answer ONLY "True", "False", "Yes", or "No" Numbers: - Use digits only, without units or words For negative numbers in financial tables: - Use parentheses format for negatives - This is standard accounting notation Units and symbols: - Include ONLY if explicitly asked Text extraction: - Extract ONLY the key word or short phrase - Do NOT add explanatory text or complete sentences Text case and spelling: - Extract exactly as shown in the table - Preserve capitalization and spelling exactly TASK-DRIVEN RESTORATION STRATEGY Your goal is to ANSWER the question, not to fix all table problems. Only use restoration tools when degradation directly prevents you from reading the information needed to answer the question. Step 1: Identify What Information You Need - What specific cell(s) or row(s) does the question ask about? - Where is that information in the table? Step 2: Analyze Whether Degradation Affects Your Answer Focus ONLY on the cells/rows containing the information needed to answer the question. If those cells are degraded and unreadable: - Identify the degradation type affecting those specific cells (blur, shadow, distortion, rotation, etc.) - Select the appropriate tool to address that degradation - If multiple degradations exist, choose the most severe one that most directly prevents reading - If the most appropriate tool is no longer available (removed after quality degradation), select the next most relevant tool If those cells are readable: - Answer directly, ignore degradations in other cells Step 3: Review After Each Tool Use After EVERY tool execution, you enter a COMPARISON phase where you: 1. Compare the processed table with the previous version 2. Make a decision: - Quality improved or acceptable â DO NOT call any tool - Quality worsened â Call rollback(reason="...") During comparison phase, you CANNOT call answer() or other restoration tools. WORKFLOW 1. Examine current table 2. If readable â Call answer directly; If needs improvement â Use appropriate tool 3. After each tool â Compare quality, use rollback if worse 4. Submit answer with extracted information in correct format Figure 9: Prompt for DocIntent on WildTableVQA task. 20 P Prompt for Comparison-Based Rollback Comparison and Review PHASE Tool execution completed. Compare the quality before and after the restoration operation. BEFORE (previous image): [Image displayed] AFTER (current image): [Image displayed] QUALITY ASSESSMENT Recall what information the question asks for, then evaluate whether the restoration operation improved your ability to extract that information: Quality improved if: - The information needed to answer the question is now clearer and more readable - You can now extract the required text, numbers, or visual elements more easily - The restoration resolved the degradation that was preventing you from answering Quality worsened if: - The information needed to answer the question became less readable - New artifacts, blur, or distortion appeared that hinder extraction - The restoration made it harder to answer the question than before Evaluation principle: Focus on whether you can better answer the question with the current image. Improvements or degradations in irrelevant regions should not influence your decision. DECISION PROTOCOL If quality improved: - Call continue() to accept the restoration and proceed - The system will return to analysis phase with the improved image If quality worsened: - Call rollback(reason="...") to revert to previous state - The system will return to analysis phase with the previous image Note: Only continue() and rollback() are available during comparison. All other tools are disabled. Figure 10: Prompt for Comparison-Based Rollback. 21 P Descriptions for Tools rotate_90 Function: Rotate image clockwise by 90 degrees Use case: Correct left-rotated document orientation rotate_180 Function: Rotate image clockwise by 180 degrees Use case: Correct upside-down document orientation rotate_270 Function: Rotate image clockwise by 270 degrees Use case: Correct right-rotated document orientation dewarping Function: Correct geometric distortion and perspective skew Parameters: None Use case: Fix warped, curved, or trapezoidal document surfaces deshadowing Function: Remove shadows and dark regions Parameters: None Use case: Eliminate shadows that obscure text or content deblurring Function: Reduce motion blur and improve sharpness Parameters: None Use case: Sharpen blurry text while preserving structure appearance Function: Enhance overall visual quality Parameters: None Use case: Improve brightness, contrast, uneven illumination, and reduce stains or discoloration demoire Function: Remove moirĂ© patterns Parameters: None Use case: Eliminate wavy or rainbow-like interference patterns from screen photos rollback(reason) Function: Revert to the state before the last restoration operation Parameters: reason (required) - explanation for rollback Use case: Restore previous image when the tool degrades quality continue() Function: Accept the current restoration and proceed Parameters: None Use case: Confirm the restoration improved quality answer(answer, confidence, reasoning) Function: Submit final answer to complete the task Parameters: - answer (required) - the extracted answer - confidence (required) - certainty level â low, medium, high - reasoning (optional) - explanation of answer derivation Use case: Provide final response after analysis or restoration Figure 11: Tools descriptions. 22 P Prompt for Baseline on WildDocVQA You are a Document Visual Question Answering (VQA) expert. PRIMARY TASK Answer the question about the document image as accurately as possible. OUTPUT FORMAT You MUST output ONLY a valid JSON object in this EXACT format: "answer": "your concise answer here", "confidence": "low/medium/high", "reasoning": "brief explanation (optional)" CRITICAL RULES 1. Output ONLY JSON - no other text before or after 2. Extract text EXACTLY as it appears in the document - don't simplify or paraphrase 3. NO explanatory phrases like "The answer is", "According to the document" 4. Preserve original formatting, capitalization, and punctuation ANSWER FORMAT RULES EXTRACT ONLY - DO NOT WRITE SENTENCES Extract the requested information without adding explanatory text. Text from documents: - Preserve original capitalization, punctuation, and formatting - Keep company suffixes and special characters as shown Numbers: - Match the document format exactly - Keep currency symbols, commas, decimals, and percentages as shown Units and symbols: - Include ONLY if shown in document or explicitly asked Extract precision: - Match the level of detail requested by the question Yes/No questions: - Answer ONLY "Yes" or "No" WORKFLOW 1. Examine current image 2. Locate the specific information requested by the question 3. Extract the answer following the format rules above 4. Output JSON with answer, confidence level, and brief reasoning Figure 12: Prompt for Baseline on WildDocVQA task. 23 P Prompt for Baseline on WildChartQA You are a Document Visual Question Answering (VQA) expert. PRIMARY TASK Answer the question about the document image as accurately as possible. OUTPUT FORMAT You MUST output ONLY a valid JSON object in this EXACT format: "answer": "your concise answer here", "confidence": "low/medium/high", "reasoning": "brief explanation (optional)" CRITICAL RULES 1. Output ONLY JSON - no other text before or after 2. Answer must be the EXACT value/text from the chart 3. NO explanatory phrases like "The value is", "The chart shows" 4. NO approximations or rounding ANSWER FORMAT RULES Numbers: - Use digits only, without units or words Percentage signs: - ONLY include % if the chart itself shows the % symbol - Do NOT add % based on interpretation Units and symbols: - Give numbers WITHOUT units unless the question explicitly asks for them or the chart shows them - When in doubt about units or %, leave them out Yes/No questions: - Answer ONLY "Yes" or "No" Text extraction: - Extract exactly as shown - Preserve capitalization Time/Date questions: - Be specific and concise - Give specific year/time, not ranges unless asked Comparison questions: - Give only the direct answer without explanatory text General principle: - Prefer simpler, shorter answers - Answer ONLY what is directly asked WORKFLOW 1. Examine current chart 2. Locate the specific information requested by the question 3. Extract the answer following the format rules above 4. Output JSON with answer, confidence level, and brief reasoning Figure 13: Prompt for Baseline on WildChartQA task. 24 P Prompt for Baseline on WildTableVQA You are a Table Visual Question Answering (VQA) expert specialized in answering questions about tables. PRIMARY TASK Answer the question about the table image as accurately as possible. OUTPUT FORMAT You MUST output ONLY a valid JSON object in this EXACT format: "answer": "your concise answer here", "confidence": "low/medium/high", "reasoning": "brief explanation (optional)" CRITICAL RULES 1. Output ONLY JSON - no other text before or after 2. Answer must be the EXACT text from the table cell(s) 3. NO explanatory phrases like "The answer is", "According to the table" TABLE FACT VERIFICATION QUESTIONS Do NOT provide the actual values. ANSWER FORMAT RULES For fact verification: - Answer ONLY "True", "False", "Yes", or "No" Numbers: - Use digits only, without units or words For negative numbers in financial tables: - Use parentheses format for negatives - This is standard accounting notation Units and symbols: - Include ONLY if explicitly asked Text extraction: - Extract ONLY the key word or short phrase - Do NOT add explanatory text or complete sentences Text case and spelling: - Extract exactly as shown in the table - Preserve capitalization and spelling exactly WORKFLOW 1. Examine current table 2. Locate the specific cell(s) or row(s) requested by the question 3. Extract the answer following the format rules above 4. Output JSON with answer, confidence level, and brief reasoning Figure 14: Prompt for Baseline on WildTableVQA task. 25