Paper deep dive
SciMIF: Understanding Multimodal Instruction Following in Scientific Domains
Ye Shen, Yuting Zheng, Dun Pei, Zijian Chen, Wenlong Zhang, Qi Jia, Guangtao Zhai
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 94%
Last extracted: 8/29/2026, 3:44:21 AM
Summary
The paper introduces SciMIF, a benchmark for evaluating Multimodal Large Language Models (MLLMs) on scientific instruction following. It comprises 2,527 samples across five disciplines (chemistry, geography, biology, materials science, physics) and ten functional constraint groups. The study reveals significant performance disparities, with chemistry being particularly challenging, and finds that model scale does not guarantee improved constraint adherence, especially for fine-grained requirements.
Entities (12)
Relation Signals (12)
SciMIF → coversdiscipline → Chemistry
confidence 98% · SciMIF covers five representative disciplines: chemistry, geography, biology, materials science, and physics.
SciMIF → coversdiscipline → Geography
confidence 98% · SciMIF covers five representative disciplines: chemistry, geography, biology, materials science, and physics.
SciMIF → coversdiscipline → Biology
confidence 98% · SciMIF covers five representative disciplines: chemistry, geography, biology, materials science, and physics.
SciMIF → coversdiscipline → Materials Science
confidence 98% · SciMIF covers five representative disciplines: chemistry, geography, biology, materials science, and physics.
SciMIF → coversdiscipline → Physics
confidence 98% · SciMIF covers five representative disciplines: chemistry, geography, biology, materials science, and physics.
SciMIF → evaluates → Multimodal Large Language Models
confidence 95% · we introduce SciMIF, a novel benchmark designed to evaluate the capability of MLLMs in following complex scientific instructions.
Qwen3.5-397B-A17B → evaluatedon → SciMIF
confidence 92% · Qwen3.5-397B-A17B 72.19 39.96 73.49
GPT-5.2 → evaluatedon → SciMIF
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Understanding instruction-following capabilities in scientific domains is essential for effectively leveraging Multimodal Large Language Models (MLLMs) to advance the development of scientific fields. In this work, we introduce SciMIF, a novel benchmark designed to evaluate the capability of MLLMs in following complex scientific instructions. Specifically, based on an extensive analysis of 22 distinct tasks across 5 representative scientific disciplines, we propose a comprehensive taxonomy comprising 10 constraint groups that captures both general functional requirements and discipline-specific characteristics. Guided by this taxonomy, we develop a high-fidelity instruction injection pipeline to systematically augment existing scientific datasets. We conduct comprehensive experiments on multiple state-of-the-art closed-source and open-source MLLMs. Our findings reveal significant performance disparities across different scientific disciplines, with chemistry posing greater challenges for current MLLMs. Furthermore, we observe that increasing the model scale does not yield corresponding improvements in constraint adherence, and current models still struggle severely with fine-grained constraints and instructions requiring the deep application of disciplinary knowledge. SciMIF fills the current void in evaluating multimodal instruction adherence within scientific domains, laying a crucial foundation for future enhancements of MLLMs in rigorous scientific applications. Data and code will be released at this https URL .
Tags
Links
- Source: https://arxiv.org/abs/2608.25973v1
- Canonical: https://arxiv.org/abs/2608.25973v1
Trouble viewing inline? Open PDF directly →
Full Text
90,021 characters extracted from source content.
Expand or collapse full text
SciMIF: Understanding Multimodal Instruction Following in Scientific Domains Ye Shen Yuting Zheng Dun Pei Zijian Chen Wenlong Zhang Qi Jia Guangtao Zhai Abstract Understanding instruction-following capabilities in scientific domains is essential for effectively leveraging Multimodal Large Language Models (MLLMs) to advance the development of scientific fields. In this work, we introduce SciMIF, a novel benchmark designed to evaluate the capability of MLLMs in following complex scientific instructions. Specifically, based on an extensive analysis of 22 distinct tasks across 5 representative scientific disciplines, we propose a comprehensive taxonomy comprising 10 constraint groups that captures both general functional requirements and discipline-specific characteristics. Guided by this taxonomy, we develop a high-fidelity instruction injection pipeline to systematically augment existing scientific datasets. We conduct comprehensive experiments on multiple state-of-the-art closed-source and open-source MLLMs. Our findings reveal significant performance disparities across different scientific disciplines, with chemistry posing greater challenges for current MLLMs. Furthermore, we observe that increasing the model scale does not yield corresponding improvements in constraint adherence, and current models still struggle severely with fine-grained constraints and instructions requiring the deep application of disciplinary knowledge. SciMIF fills the current void in evaluating multimodal instruction adherence within scientific domains, laying a crucial foundation for future enhancements of MLLMs in rigorous scientific applications. Data and code will be released at https://github.com/shenye7436/SciMIF. 1Shanghai Artificial Intelligence Laboratory, 2Shanghai Jiao Tong University 1 Introduction The application of Multimodal Large Language Models (MLLMs) in scientific domains has rapidly evolved from basic, single-discipline question-answering tasks to complex paradigms such as autonomous science agents and AI-driven scientific discovery (Jiang et al. 2025; Boiko et al. 2023). Consequently, the requirements placed on these models have expanded from factual knowledge retrieval to the reliable execution of multi-step scientific actions under explicit operational requirements. Instruction following is a fundamental capability that transforms models from text-completion engines into versatile task solvers (Ouyang et al. 2022; Wang et al. 2023b). Following the terminology of CFBench (Zhang et al. 2025b), a constraint refers to an individual requirement, whereas an instruction denotes a complete query containing one or more constraints. In general domains, benchmarks such as IF-Eval (Zhou et al. 2023) and FollowBench (Jiang et al. 2024) have advanced the evaluation of general-purpose instruction following by systematically assessing whether models satisfy diverse constraints on output format, length, structure, and style. However, it remains underexplored whether current models can reliably follow scientific instructions involving domain-specific constraints. Scientific instruction following differs from its general-domain counterpart in three important aspects. First, scientific instructions are scientific knowledge-dependent. Even seemingly simple constraints involving letters, numbers, terminology, or output formats may carry domain-specific meanings that cannot be handled through surface-level text manipulation alone. For example, constraints involving the numbers of chemical bonds, functional groups, or amino-acid residues require models to accurately recognize and understand the relevant scientific entities. Second, the semantics of scientific instructions exhibit strong interdisciplinary variations. Although different disciplines may share functional constraint groups, their concrete meanings differ substantially across scientific contexts. As illustrated in Figure 1, a terminology constraint may require valid molecular nomenclature in chemistry, a hierarchical address in geography, an entity relationship in biology, or a characterization technique in materials science. Third, scientific instruction following is frequently multimodal. Scientific reasoning often depends on specialized visual inputs, including molecular structures, biological diagrams, materials microscopy images, geographical scenes, and physical plots. For instance, in the materials example in Figure 1, identifying the required characterization technique depends jointly on the textual request and the information presented in the scientific figure. These characteristics also expose a practical limitation of existing scientific evaluations. Most scientific benchmarks primarily determine whether a final answer is correct, but scientific correctness alone does not guarantee that an output is usable. A scientifically correct answer may still violate a requested unit, notation, output structure, analysis method, or solution procedure. Conversely, a model may strictly reproduce the requested form while relying on scientifically invalid reasoning. Treating these outputs as a single notion of correctness obscures two distinct model capabilities. We therefore distinguish scientific correctness, which measures whether an answer is scientifically valid, from instruction adherence, which measures whether the specified constraints are satisfied. Evaluating them separately enables failures to be attributed to insufficient scientific reasoning, inadequate instruction following, or both. Figure 1: An overview of SciMIF, illustrating its coverage of five scientific disciplines, one general constraint domain, and ten functional constraint categories for constructing diverse scientific instruction-following constraints. To address these challenges, we introduce SciMIF, a Scientific Multimodal Instruction Following benchmark for systematically evaluating models across diverse scientific questions. As illustrated in Figure 1, SciMIF covers five representative disciplines: chemistry, geography, biology, materials science, and physics. We tasked domain experts with systematically examining 22 scientific tasks, their source datasets, disciplinary conventions, and related literature, and deriving a taxonomy of ten functional constraint groups. These groups capture shared instruction-following capabilities across disciplines, while their concrete constraints are adapted to domain-specific knowledge and practices. Based on this taxonomy, we develop a scalable framework for transforming existing scientific problems into instruction-following evaluations. The framework first identifies constraints already implicit in the original tasks and separates them into independently evaluable requirements. It then injects additional compatible scientific and general constraints without changing the correct. After automatic consistency checking and human verification, the benchmark contains 2,527 samples and supports evaluation evaluation of constraint satisfaction, complete instruction fulfillment, and fine-grained adherence to decomposed requirement. Extensive experiments on representative closed-source and open-source MLLMs reveal substantial limitations in scientific instruction following. Performance varies considerably across disciplines, with chemistry and geography posing greater challenges than other domains. Increasing model scale does not consistently improve constraint adherence, indicating that parameter scaling alone is insufficient. Models also perform substantially worse on general constraints and fine-grained requirements, while scientific correctness and instruction adherence remain weakly coupled. These findings motivate future research on domain-aware instruction alignment, stronger adherence to general operational requirements, and structure-aware or tool-assisted methods for handling fine-grained scientific constraints. In summary, our main contributions are as follows: • We establish an expert-derived taxonomy of scientific constraints spanning five disciplines and ten functional groups, capturing both shared capability dimensions and discipline-specific requirements. • We propose a scalable framework for converting existing scientific tasks into instruction-following evaluations by recognizing implicit constraints and injecting compatible scientific and general constraints without changing the reference answers. • We construct SciMIF and evaluate representative MLLMs, revealing substantial disciplinary variation, difficulties with fine-grained constraints, and a clear gap between scientific correctness and instruction adherence. 2 Related Work 2.1 Scientific Reasoning Benchmarks Scientific benchmarks have evolved from general science question answering toward more specialized and challenging reasoning tasks. ScienceQA (Saikh et al. 2022) provides elementary and secondary school science questions covering multiple disciplines. Subsequently, MMMU (Yue et al. 2024) has pushed the difficulty to the expert level by introducing university-level expertise. The core of scientific tasks lies in the construction of logical chains. MathVista (Lu et al. 2023) is designed specifically for multimodal mathematical reasoning, emphasizing the model’s ability to handle complex visual elements such as geometric figures and function graphs. SciBench (Wang et al. 2023a) further focuses on complex scientific computing problems, challenging the limits of the model’s multi-step reasoning and formula calculation through in-depth coverage of disciplines such as physics and chemistry. In addition, in-depth evaluations of specific fields are also crucial. For example, in the field of geography, GeoQA (Chen et al. 2021) examines the model’s perception of spatial layout and geographical features. Although these benchmarks have substantially advanced the evaluation of scientific knowledge and reasoning, they primarily measure whether models produce correct answers, without systematically examining whether the outputs satisfy explicit scientific and operational constraints. Benchmark Scope # Grp. # Const. # Disc. # Samp. Multi- modal General-purpose instruction-following benchmarks IFEval (Zhou et al. 2023) General 9 25 – 541 ✗ FollowBench (Jiang et al. 2024) General 5 16 – 820 ✗ M-IFEngine (Ding et al. 2025) General 6 32 – 400 ✓ Scientific instruction-following benchmarks SciIF (Su et al. 2026) Scientific 3 10 4 1,244 ✗ SciMIF (Ours) Both 10 42 5 2,527 ✓ Table 1: Comparison of general-purpose and scientific instruction-following benchmarks. Scope specifies whether they evaluate general, scientific, or both types of constraints. # Grp. and # Const. denote the numbers of functional constraint groups and individual constraints; # Disc. and # Samp. denote covered scientific disciplines and evaluation samples. “–” indicates scientific discipline coverage is not applicable. 2.2 Instruction-Following Benchmarks General-purpose instruction-following benchmarks evaluate whether models comply with explicit user requirements. IFEval (Zhou et al. 2023) introduces objectively verifiable textual constraints, FollowBench (Jiang et al. 2024) evaluates constraints of varying difficulty and their combinations, and M-IFEngine (Ding et al. 2025) extends instruction-following evaluation to multimodal inputs. However, these benchmarks mainly focus on general linguistic, formatting, and structural requirements, without covering constraints grounded in scientific knowledge and disciplinary conventions. Scientific instruction-following evaluation remains limited. SciIF (Su et al. 2026) evaluates university-level scientific question answering under process-level guidance, such as prescribed reasoning steps and solution procedures. Its constraints are shared across scientific questions rather than systematically derived from the representations, conventions, and operational requirements of individual disciplines, and its evaluation remains text-only. As summarized in Table 1, SciMIF complements existing benchmarks with an expert-derived taxonomy constructed from 22 tasks across five disciplines. It covers 10 functional groups and 42 discipline-adapted constraints, combines scientific and general requirements, and supports multimodal scientific inputs. 3 SciMIF This section introduces the design of SciMIF including its taxonomy of scientific constraints, instruction-augmented sample construction procedure, and benchmark statistics. 3.1 Taxonomy of Scientific Constraints To provide a comprehensive evaluation of scientific instruction following, we construct a hierarchical taxonomy covering both general and discipline-specific constraints. In real-world scientific queries, user requirements and agent interfaces often combine general operational constraints with scientific requirements, making the two inseparable in practical use. Our taxonomy therefore considers both: general constraint types are adapted from prior instruction-following benchmarks, while the primary effort focuses on constructing discipline-specific constraints. Specifically, domain experts analyze 22 tasks from 13 scientific datasets, together with their reference answers, disciplinary conventions, evaluation protocols, and relevant literature, to identify recurring scientific requirements. Functionally similar requirements are abstracted into shared constraint groups, while their domain-specific realizations are preserved as individual constraints. This results in a two-level organization: the domain level identifies the field of the corresponding instruction, covering one general domain and five scientific domains, whereas the group level identifies the functional purpose of each constraint. Each individual constraint is therefore characterized by both a domain and a group, while each functional group may contain distinct domain-specific constraints, as shown in Figure 1. This organization enables model performance to be analyzed from both disciplinary and capability-oriented perspectives. Detailed domain-specific instantiations and complete constraint specifications are provided in the Appendix A. At the group level, we define ten functional types according to the requirements imposed on model outputs and task execution: • Procedure: Requires reasoning, analysis, or experimental operations to follow a prescribed sequence of steps. • Number: Specifies the required cardinality of output items or scientific entities, such as responses, atoms, bonds, functional groups, or sequence elements. • Method: Requires the use of a designated scientific formula, theorem, law, or analytical approach. • Unit: Requires numerical quantities to be expressed in specified units while preserving dimensional consistency. • Format: Specifies the required representation of scientific outputs, such as expressing material properties as discrete labels or continuous values, or organizing geographic locations in a prescribed hierarchical format. • Terminology: Requires valid and standardized nomenclature or specialized expressions appropriate to a scientific discipline. • Precision: Requires numerical quantities to be reported with a specified level of accuracy, such as a fixed number of decimal places or significant figures. • Letter: Specifies general letter-level requirements, such as using only lowercase or uppercase letters. • Structure: Specifies how different parts of the response should be organized, such as presenting the analysis before the final answer. • Selection: Restricts the answer to one or more valid choices from a predefined candidate set. Although these functional groups are shared across disciplines, their concrete instantiations may require substantially different scientific knowledge and recognition capabilities. For example, a terminology constraint may involve valid molecular nomenclature in chemistry, a hierarchical address in geography, an entity relationship in biology, or an appropriate characterization technique in materials science. Similarly, a number constraint may refer to atoms and chemical bonds in chemistry, sequence elements in biology, or physical quantities in physics, while a procedure constraint may specify a reaction sequence, a biological analysis pipeline, an experimental workflow, or a physics derivation. These discipline-specific instantiations allow SciMIF to analyze model performance both across scientific disciplines and along shared constraint-following capabilities. Figure 2: An overview of the data construction pipeline. 3.2 Data Construction As illustrated in Figure 2, we construct SciMIF through a structured pipeline consisting of seed preparation, constraint recognition and selection, constraint injection, and human verification. We provide more details of the construction pipeline in the Appendix B. Seed Preparation. Each seed sample from a source dataset contains a textual question q, an optional visual input I, a reference answer a, and a scientific task type t. We represent the sample as x=(q,I,a,t).x=(q,I,a,t). (1) For each sample, we determine an applicable constraint inventory CxC_x, which contains scientific constraints compatible with its discipline and task type, together with general constraints applicable to scientific tasks. Constraint Recognition. Annotators first identify the constraints already expressed or implied by the original query, which can be expressed as a set CoC_o: Co=fr(q,Cx),C_o=f_r(q,C_x), (2) The remaining constraints in Cx∖CoC_x C_o are then filtered to remove requirements that are redundant, incompatible with the task, or contradictory to existing constraints. Scientific constraints CsC_s are selected according to the disciplinary knowledge and task type, while mutually compatible general constraints CgC_g are randomly sampled by N classes from Cx∖CoC_x C_o. Constraint Injection. Scientific constraints from CsC_s should be injected to original query q as follows: qd=fd(q,Cs),q_d=f_d(q,C_s), (3) where qdq_d represents the augmented query after injecting appropriate scientific constraints. For general constraints in CgC_g, if one constraint is injected into qdq_d, the new augmented query can be expressed as: qg=fg(qd,cn∣cn⊆Cg),q_g=f_g(q_d,c_n c_n C_g), (4) where cnc_n denotes any one of each general constraint class. fg(⋅)f_g(·) denotes a double automatic validation procedure consisting of included(qg,cn)∩unchanged(qg,a)I_included(q_g,c_n) _unchanged(q_g,a), which ensures that cnc_n has been included and ground-truth answer a remains unchanged. A failed injection is retried up to k times using an alternative constraint from the same category. The resulting sample after the validation is represented as x^=(q^,I,a,t), x=( q,I,a,t), (5) where q q denotes the final augmented query. Statistic Chem. Geo. Bio. Mat. Phy. Total 518 523 493 497 496 Scientific Constraints 7 5 5 5 3 Task Types 8 4 3 4 3 Avg. Question (/tokens) 231.46 168.17 147.29 215.60 293.68 Multimodal Samples × ✓ × ✓ ✓ Table 2: Overall statistics of the constructed dataset. Human Verification. Two annotators ask for each x x and its associated constraint list meet the following two requirements: (1) Logical Coherence and Fluency, ensuring that the injected constraints are naturally integrated without semantic contradictions or grammatical errors; and (2) Constraint Fidelity, ensuring that every specified constraint is accurately and unambiguously expressed in the query. Problematic samples are manually revised, while samples that cannot be meaningfully repaired are discarded. All retained samples undergo manual verification, and 884 samples are revised during this process. Model Chemistry Geography Biology Material Physics Overall CSR ISR DRFR CSR ISR DRFR CSR ISR DRFR CSR ISR DRFR CSR ISR DRFR Closed-Source MLLMs GPT-5.2 72.87 46.33 73.99 82.30 61.57 82.40 88.87 72.82 86.58 87.79 78.07 88.93 87.97 69.56 86.94 65.67 Grok-4-Fast 65.56 33.01 67.37 83.96 53.15 83.25 80.18 45.53 79.06 89.74 75.25 89.01 82.06 58.27 81.16 53.04 Gemini-3.1-Pro-Preview 65.79 32.24 66.37 79.53 47.80 80.01 75.30 44.02 72.01 89.94 75.45 89.00 77.46 50.60 77.78 50.02 Claude-Sonnet-4.6 62.02 26.31 63.42 71.31 31.72 72.75 76.56 46.32 73.75 81.54 67.40 84.06 86.48 65.12 85.87 47.37 Open-Source MLLMs Qwen3.5-27B 68.33 32.09 70.04 79.54 45.00 80.03 83.18 49.21 80.79 89.76 69.75 89.62 85.74 60.82 85.70 51.37 Qwen3.5-35B-A3B 68.52 29.90 70.35 77.61 41.31 78.39 83.91 49.24 81.22 90.08 68.83 89.88 85.00 60.48 85.25 49.95 Qwen3.5-122B-A10B 68.08 30.62 69.65 78.40 43.27 79.16 82.13 47.18 80.67 89.77 68.70 89.50 85.89 62.30 85.90 50.41 Qwen3.5-397B-A17B 72.19 39.96 73.49 81.57 54.23 81.97 85.41 56.87 83.52 89.63 73.37 89.86 85.80 62.50 85.67 57.39 InternVL3.5-8B 62.99 31.47 63.83 73.04 44.93 73.82 79.14 57.20 77.97 79.34 59.76 80.56 84.70 63.51 83.09 51.37 InternVL3.5-14B 61.88 30.89 63.96 75.23 49.90 75.75 80.48 58.62 79.05 79.33 59.76 81.43 86.42 66.53 85.19 53.14 InternVL3.5-38B 59.96 29.73 62.36 74.85 47.80 75.48 77.32 56.80 76.18 79.26 61.17 81.35 84.52 62.70 83.28 51.64 Table 3: Performance (%) of evaluated MLLMs. Overall denotes the average ISR across the five scientific disciplines. The best score in each column is bolded separately for closed-source and open-source MLLMs. 3.3 Data Statistics Data Sources. SciMIF is constructed from 13 existing scientific datasets, covering 22 task types across five disciplines: chemistry, geography, biology, materials science, and physics. Representative sources include ChemEval (Huang et al. 2024), IMAGEO-Bench (LI et al. 2025), LAB-Bench (Laurent et al. 2024), MatCha (Lai et al. 2025), and PhysUniBench (Wang et al. 2025a). These datasets provide diverse task formulations, disciplinary conventions, and input modalities. The complete source-to-task mapping and task-level sample statistics are provided in the Appendix C. Dataset Composition. We summarize the overall statistics of the dataset in Table 2. The final benchmark contains 2,527 samples in total, spanning 5 disciplines and including ten functional constraint groups. Each sample consists of a verified question, optional visual input, and a reference answer. On average, each sample contains 211.03 tokens in the question. Among all samples, 27.50% include multimodal inputs with associated images. 4 Experiment 4.1 Setup Models. To evaluate the effectiveness of our dataset and evaluation pipeline, we conduct experiments on a diverse set of state-of-the-art large language models (LLMs), including GPT-5.2 (OpenAI 2025), Grok-4-Fast (xAI 2025), Gemini-3.1-Pro-Preview (Team et al. 2023), and Claude-Sonnet-4.6 (Anthropic 2026), which are widely recognized for their strong reasoning and instruction-following capabilities. Simultaneously, we also assess advanced open-source models, including InternVL3.5 series (Wang et al. 2025b), and Qwen3.5 series (Qwen Team 2026) to further analyze performance differences across model families. Details. For multimodal samples, images are preprocessed before generation to reduce transmission overhead during large-scale model inference while preserving essential semantic information. We set N=3N=3, and k=3k=3 to balance diversity and consistency in constraint augmentation. 4.2 Metrics To evaluate model performance, we adopt several existing metrics that measure different aspects of instruction-following performance. Specifically, we employ the Constraint Satisfaction Rate (CSR) (Zhang et al. 2025b) and the Instruction Satisfaction Rate (ISR) (Zhang et al. 2025b), as well as the Decomposed Requirements Following Ratio (DRFR) (Qin et al. 2024). Together, these metrics provide complementary perspectives on model performance, including CSR captures the averaged constraint-level satisfaction across samples, ISR evaluates instruction-level success, and DRFR measures decomposed-requirement-level compliance across the total number of constraints. Constraint verification is categorized into exact match, precision-based, and LLM-as-a-judge, with detailed protocols provided in the Appendix D. 5 Result & Analysis This section reports model performance across disciplines, constraint domains, and constraint groups. Subsequently, we analyze performance trends as the number of constraints increases and examine the relationship between answer correctness and instruction following. Due to space limitations, the main text focuses on representative models and key findings. Additional analyses of source-level variation, modality-associated differences, correctness contrasts, and representative cases are provided in the Appendix E. 5.1 Performance on Different Science Disciplines We first evaluate the instruction-following performance of MLLMs across different scientific disciplines. As summarized in Table 3, we report the CSR, ISR, and DRFR metrics for five domains to analyze how disciplinary data characteristics influence model behavior. Performance across disciplines. Models demonstrate that no single model performs well across all disciplines, showing significant variation in performance across domains. The highest ISR scores are observed in the biology and materials scientific domain, while performance drops significantly in chemistry and geography. Specifically, GPT-5.2 achieves an ISR of 72.82% in biology and 78.07% in materials science. In contrast, chemistry presents the greatest challenge, with the highest ISR for GPT-5.2 falling to 46.33%. This disparity is likely due to the nature of the tasks in each domain: biology and materials science tasks often rely on entity recognition and method restriction, which are well-represented in the training data. On the other hand, chemistry and geography require models to handle more complex structural constraints and spatial hierarchies, which are harder to capture through standard linguistic alignment. Closed-Source vs Open-Source. Closed source models maintain a consistent performance advantage over open source models across all disciplines. The leading closed-source model, GPT-5.2, achieves an overall score of 65.67%, exceeding the 57.39% attained by the leading open-source model, Qwen3.5-397B-A17B. Although the gap narrows in disciplines such as physics, where Qwen3.5-397B-A17B achieves an ISR of 62.50%, compared with 69.56% for GPT-5.2, closed-source models demonstrate greater robustness in satisfying multifaceted scientific constraints. This superiority reflects the advantages conferred by high-quality data and advanced alignment strategies in proprietary models. Impact of Model Scale. An increase in parameter size does not lead to a linear improvement in the instruction-following performance of models in scientific contexts. For the InternVL3.5 series, the 8B model yields an overall score of 51.37% which is nearly identical to the 51.64% of the 38B version. A more significant trend appears in the Qwen3.5 family where the score of the 27B model is 51.37% while the 122B version drops to 50.41%. These results suggest that the primary bottleneck for scientific instruction-following is not raw computational power but rather the quality of domain-aware alignment. Scaling up parameters without targeted optimization of scientific instruction-tuning data may cause performance saturation or an alignment tax that hinders the ability to satisfy complex disciplinary rules. Figure 3: DRFR scores (%) across different scientific constraint domains for the evaluated MLLMs. Figure 4: DRFR scores (%) of representative MLLMs between general and scientific constraint domains. Model Format Precision Selection Structure Number Letter Procedure Method Terminology Unit Closed-Source MLLMs GPT-5.2 72.18 84.69 82.24 83.41 68.62 51.30 94.19 95.91 73.59 85.61 Grok-4-Fast 82.31 77.99 82.89 77.35 58.50 53.90 78.14 94.55 72.23 80.00 Gemini-3.1-Pro-Preview 74.38 75.60 80.92 63.90 55.76 50.00 78.29 96.68 72.23 77.09 Claude-Sonnet-4.6 50.67 85.37 83.44 76.91 58.89 35.33 76.74 98.28 74.60 88.86 Open-Source MLLMs Qwen3.5-27B 71.20 88.89 83.22 85.19 46.81 50.67 80.95 97.94 76.22 85.86 Qwen3.5-35B-A3B 72.53 87.23 84.21 85.55 43.23 52.00 80.70 98.16 74.64 84.57 Qwen3.5-122B-A10B 73.23 86.75 83.01 85.62 53.56 56.29 79.20 98.38 73.73 83.52 Qwen3.5-397B-A17B 73.70 89.21 83.66 87.76 60.05 53.59 86.06 98.04 74.77 83.38 InternVL3.5-8B 50.57 77.03 75.00 78.03 57.79 37.01 90.49 88.76 71.78 88.40 InternVL3.5-14B 53.63 81.82 76.32 80.27 57.56 36.36 90.94 90.03 69.53 88.69 InternVL3.5-38B 48.76 81.34 77.63 76.46 56.88 38.31 89.49 90.55 71.11 87.52 Table 4: DRFR scores (%) of different constraint groups from evaluated MLLMs. The best score in each column is bolded separately for closed-source and open-source MLLMs. 5.2 Performance on Constraint Domains We explore performance across constraint domains to identify which requirements pose the greatest challenges to alignment. As established in the previous section, these domains include a general category and five scientific subcategories. Performance across Scientific Constraint Domains. The distribution of constraint groups varies significantly across scientific disciplines, contributing to differing levels of task complexity. As illustrated in Figure 3, chemistry is the most challenging domain, with an average DRFR score of only 68.2% and physics achieves the highest average performance with an average DRFR score of 92.%. This variance is largely driven by the distinct constraint profiles inherent to each field. Chemistry tasks predominantly feature a high concentration of fine-grained constraints, demanding specific numerical values and strict structural formats. In contrast, domains like physics tend to rely more heavily on coarse-grained constraints. These usually involve methodological guidelines, broad analytical steps, or general reasoning procedures. Consequently, the dense distribution of highly specific, minute constraints in chemistry makes its tasks inherently more complex to execute than those in domains dominated by broader instructional constraints. General vs Scientific Constraints. Models consistently perform better on scientific constraints than on general constraints. As shown in Figure 4, GPT-5.2 achieves a DRFR score of 88.74% on scientific constraints, compared with 74.65% on general constraints. A similar gap is observed for Qwen3.5-397B-A17B and other models. One possible explanation is that scientific constraints are often semantically coupled with the scientific task itself and models may therefore prefer to attend to these requirements. In contrast, general constraints primarily regulate the presentation and organization of the output and may be relatively independent of the underlying scientific reasoning process, making them more likely to be overlooked. These results highlight the need to evaluate both scientific and general constraints, as strong adherence to domain-specific requirements does not necessarily imply reliable compliance with the full instruction. 5.3 Performance across Constraint Groups We examine the performance of models across different functional groups to identify which instruction types are most manageable or challenging for current systems. As detailed in section 3.1, there are ten distinct groups and the results for the DRFR scores are summarized in Table 4. Performance Consistency Across Groups. More challenging groups lead to more significant differences in model performance. As observed, models struggle more in the number, letter, and format groups, with a large performance gap between GPT-5.2 and Qwen3.5-397B-A17B, the top-performing models in the closed-source and open-source categories, respectively. The difference in these groups can reach as high as 8% to 9%. However, in less challenging groups, such as precision, procedure, and method, the performance gap between models is much narrower, typically ranging from just 2% to 4%. This indicates that the group categorization is well-balanced and covers a range of difficulty levels, providing clear differentiation between model performance across various tasks. Analysis of the Challenge of Constraints. Models perform poorly on letter and number constraints, reflecting difficulties in both fine-grained symbolic processing and domain-specific knowledge application. GPT-5.2 achieves DRFR scores of 51.30% for letters, 68.62% for numbers, and 73.59% for terminology. Unlike generic sub-token tasks involving character counting or manipulation, our number constraints require models to interpret scientific structures and identify chemical bonds, functional groups, amino-acid residues, or sequence motifs rather than count numerical characters. Terminology constraints evaluate context-appropriate scientific term usage. These results highlight limited grounding of fine-grained symbols and quantities in scientific objects. Potential remedies include structure-aware representations, constrained decoding, and external scientific parsers or verification tools. 5.4 Relationship Between Answer Correctness and Instruction-Following We examine whether models can simultaneously produce scientifically correct answers and satisfy the specified constraints. Figure 5 reports the proportion of samples in four categories: Correct and Followed (CF), Correct but Violated (CV), Incorrect but Followed (IF), and Incorrect and Violated (IV). Across all evaluated models, the CF rate remains below 30%, indicating that jointly achieving scientific correctness and instruction adherence is still challenging. Moreover, approximately 20% of samples fall into the CV category, showing that even correct answers frequently violate one or more constraints. The IF rate also exceeds 30% for several models, suggesting that models may satisfy formatting or structural requirements without producing scientifically valid answers. These results reveal a clear gap between scientific correctness and constraint adherence, highlighting the need to improve both capabilities jointly rather than optimizing either in isolation. The verification procedures and weak association evidence are detailed in the Appendix E.2. Figure 5: Proportions of Correct Followed (CF), Correct Violated (CV), Incorrect Followed (IF), and Incorrect Violated (IV) samples across representative models. 6 Conclusion In this work, we propose SciMIF, a comprehensive benchmark for evaluating the instruction-following capabilities of MLLMs across five scientific disciplines: Chemistry, Geography, Biology, Material, and Physics. By systematically injecting 10 distinct groups of general and domain-specific constraints, SciMIF assesses a model’s ability to seamlessly integrate rigorous disciplinary knowledge with complex formatting and methodological requirements. Our extensive evaluations show that the performance of MLLMs varies drastically by science domains and models are highly vulnerable to fine-grained constraints and suffer sharp performance drops as constraint counts increase. Our analysis also uncover a scaling paradox where merely increasing parameter size does not linearly improve scientific instruction following. Based on these findings, we suggest following directions for future research. First, training and evaluation should be reoriented toward practical application capabilities in constrained environments, rather than being primarily driven by scientific knowledge injection. Besides, to mitigate limitations related to symbolic and discrete constraints, MLLM-centric agents could consider integrating external numerical computation tools and symbolic engines to guide the inference process, enabling more precise and reliable control in real-world scientific workflows. References Anthropic (2026) Anthropic Introducing claude sonnet 4.6. Note: https://w.anthropic.com/news/claude-sonnet-4-6 Cited by: §4.1. Boiko et al. (2023) D. A. Boiko, R. MacKnight, B. Kline, and G. Gomes Autonomous chemical research with large language models. Nature 624 (7992), p. 570–578. Cited by: §1. Chen et al. (2021) J. Chen, J. Tang, J. Qin, X. Liang, L. Liu, E. Xing, and L. Lin GeoQA: a geometric question answering benchmark towards multimodal numerical reasoning. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, p. 513–523. Cited by: §2.1. Ding et al. (2025) S. Ding, S. Wu, X. Zhao, Y. Zang, H. Duan, X. Dong, P. Zhang, Y. Cao, D. Lin, and J. Wang M-ifengine: towards multimodal instruction following. arXiv preprint arXiv:2504.07957. Cited by: §2.2, Table 1. Fang et al. (2024) Y. Fang, X. Liang, N. Zhang, K. Liu, R. Huang, Z. Chen, X. Fan, and H. Chen Mol-instructions: A large-scale biomolecular instruction dataset for large language models. In ICLR, External Links: Link Cited by: Table 12, Table 12, Appendix C. Huang et al. (2024) Y. Huang, R. Zhang, X. He, X. Zhi, H. Wang, X. Li, F. Xu, D. Liu, H. Liang, Y. Li, et al. ChemEval: a comprehensive multi-level chemical evaluation for large language models. arXiv preprint arXiv:2409.13989. Cited by: Table 12, Table 12, Table 12, Table 12, Table 12, Appendix C, §3.3. Jiang et al. (2025) X. Jiang, W. Wang, S. Tian, H. Wang, T. Lookman, and Y. Su Applications of natural language processing and large language models in materials discovery. npj Computational Materials 11 (1), p. 79. Cited by: §1. Jiang et al. (2024) Y. Jiang, Y. Wang, X. Zeng, W. Zhong, L. Li, F. Mi, L. Shang, X. Jiang, Q. Liu, and W. Wang FollowBench: a multi-level fine-grained constraints following benchmark for large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), L. Ku, A. Martins, and V. Srikumar (Eds.), Bangkok, Thailand, p. 4667–4688. External Links: Link Cited by: §1, §2.2, Table 1. Lai et al. (2025) Z. Lai, Y. Zheng, Z. Cai, H. Lyu, J. Yang, H. Liang, Y. Hu, and B. Wang Can multimodal llms see materials clearly? a multimodal benchmark on materials characterization. External Links: 2509.09307, Link Cited by: Table 12, Appendix C, §3.3. Laurent et al. (2024) J. M. Laurent, J. D. Janizek, M. Ruzo, M. M. Hinks, M. J. Hammerling, S. Narayanan, M. Ponnapati, A. D. White, and S. G. Rodriques LAB-Bench: measuring capabilities of language models for biology research. arXiv preprint arXiv:2407.10362. Cited by: Table 12, Appendix C, §3.3. Li et al. (2025) J. Li, J. Li, W. Wang, Y. Liu, C. Zheng, D. Zhou, X. Wei, and Q. Li Speak-to-structure: evaluating llms in open-domain natural language-driven molecule generation. External Links: 2412.14642, Link Cited by: Table 12, Table 12, Table 12, Appendix C. LI et al. (2025) L. LI, Y. Runlong, H. Qikai, L. Bowei, D. Min, Z. Yang, and J. Xiaowei IMAGEO-bench: a systematic benchmark dataset for evaluating image geolocalization ability in large language models. Zenodo. External Links: Document, Link Cited by: Table 12, Table 12, Appendix C, §3.3. Liu et al. (2025) S. Liu, H. Liu, J. Liu, L. Xiao, S. Gao, C. Lyu, Y. Gu, W. Zhang, D. F. Wong, S. Zhang, et al. Compassverifier: a unified and robust verifier for llms evaluation and outcome reward. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, p. 33454–33482. Cited by: §D.2. Lu et al. (2023) P. Lu, H. Bansal, T. Xia, J. Liu, C. Li, H. Hajishirzi, H. Cheng, K. Chang, M. Galley, and J. Gao MathVista: evaluating mathematical reasoning of foundation models in visual contexts. arXiv preprint arXiv:2310.02255. Cited by: §2.1. Niyongabo Rubungo et al. (2025) A. Niyongabo Rubungo, K. Li, J. Hattrick-Simpers, and A. Bousso Dieng LLM4Mat-Bench: benchmarking large language models for materials property prediction. Machine Learning: Science and Technology 6 (2), p. 020501. Cited by: Table 12, Appendix C. OpenAI (2025) OpenAI Introducing gpt-5.2. Note: https://openai.com/index/introducing-gpt-5-2/ Cited by: §4.1. Ouyang et al. (2022) L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems 35, p. 27730–27744. Cited by: §1. Qin et al. (2024) Y. Qin, K. Song, Y. Hu, W. Yao, S. Cho, X. Wang, X. Wu, F. Liu, P. Liu, and D. Yu InfoBench: evaluating instruction following ability in large language models. In Findings of the Association for Computational Linguistics: ACL 2024, p. 13025–13048. Cited by: §4.2. Qwen Team (2026) Qwen Team Qwen3.5: towards native multimodal agents. External Links: Link Cited by: §4.1. Saikh et al. (2022) T. Saikh, T. Ghosal, A. Mittal, A. Ekbal, and P. Bhattacharyya ScienceQA: a novel resource for question answering on scholarly articles. International Journal on Digital Libraries 23 (3), p. 289–301. Cited by: §2.1. Su et al. (2026) E. Su, J. Wu, C. Tang, L. Wang, P. Li, A. Wang, J. Zhang, Y. Wang, Y. Meng, X. Ma, et al. SciIF: benchmarking scientific instruction following towards rigorous scientific intelligence. arXiv preprint arXiv:2601.04770. Cited by: §2.2, Table 1. Team et al. (2023) G. Team, R. Anil, S. Borgeaud, J. Alayrac, J. Yu, R. Soricut, J. Schalkwyk, A. M. Dai, A. Hauth, K. Millican, et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805. Cited by: §4.1. Wang et al. (2025a) L. Wang, E. Su, J. Liu, P. Li, P. Xia, J. Xiao, W. Zhang, X. Dai, X. Chen, Y. Meng, M. Ding, L. Bai, W. Ouyang, S. Tang, A. Wang, and X. Ma PhysUniBench: an undergraduate-level physics reasoning benchmark for multimodal models. External Links: 2506.17667, Link Cited by: Table 12, Appendix C, §3.3. Wang et al. (2025b) W. Wang, Z. Gao, L. Gu, H. Pu, L. Cui, X. Wei, Z. Liu, L. Jing, S. Ye, J. Shao, et al. Internvl3. 5: advancing open-source multimodal models in versatility, reasoning, and efficiency. arXiv preprint arXiv:2508.18265. Cited by: §4.1. Wang et al. (2023a) X. Wang, Z. Hu, P. Lu, Y. Zhu, J. Zhang, S. Subramaniam, A. R. Loomba, S. Zhang, Y. Sun, and W. Wang SciBench: evaluating college-level scientific problem-solving abilities of large language models. arXiv preprint arXiv:2307.10635. Cited by: §2.1. Wang et al. (2023b) Y. Wang, Y. Kordi, S. Mishra, A. Liu, N. A. Smith, D. Khashabi, and H. Hajishirzi Self-Instruct: aligning language models with self-generated instructions. In Proceedings of the 61st annual meeting of the association for computational linguistics (volume 1: long papers), p. 13484–13508. Cited by: §1. xAI (2025) xAI Grok 4 fast. Note: https://x.ai/news/grok-4-fast/ Cited by: §4.1. Xu et al. (2025a) W. Xu, X. Zhao, Y. Zhou, X. Yue, B. Fei, F. Ling, W. Zhang, and L. Bai EarthSE: a benchmark for evaluating earth scientific exploration capability of llms. arXiv preprint arXiv:2505.17139. Cited by: Table 12, Table 12, Appendix C. Xu et al. (2025b) X. Xu, Q. Xu, T. Xiao, T. Chen, Y. Yan, J. Zhang, S. Diao, C. Yang, and Y. Wang UGPhysics: a comprehensive benchmark for undergraduate physics reasoning with large language models. arXiv preprint arXiv:2502.00334. Cited by: Table 12, Appendix C. Yue et al. (2024) X. Yue, Y. Ni, K. Zhang, T. Zheng, R. Liu, G. Zhang, S. Stevens, D. Jiang, W. Ren, Y. Sun, et al. MMMU: a massive multi-discipline multimodal understanding and reasoning benchmark for expert agi. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 9556–9567. Cited by: §2.1. Zaki et al. (2023) M. Zaki N. Krishnan et al. MaScQA: a question answering dataset for investigating materials science knowledge of large language models. arXiv preprint arXiv:2308.09115. Cited by: Table 12, Appendix C. Zhang et al. (2025a) J. Zhang, J. Gan, X. Wang, Z. Jia, C. Gu, J. Chen, Y. Zhu, M. D. Ma, D. Zhou, L. Li, et al. MatSciBench: benchmarking the reasoning ability of large language models in materials science. arXiv preprint arXiv:2510.12171. Cited by: Table 12, Appendix C. Zhang et al. (2025b) T. Zhang, C. Zhu, Y. Shen, W. Luo, Y. Zhang, H. Liang, F. Yang, M. Lin, Y. Qiao, W. Chen, et al. CFBench: a comprehensive constraints-following benchmark for llms. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 32926–32944. Cited by: §1, §4.2. Zhang et al. (2025c) X. Zhang, Y. Dong, Y. Wu, J. Huang, C. Jia, B. Fernando, M. Z. Shou, L. Zhang, and J. Liu PhysReason: a comprehensive benchmark towards physics-based reasoning. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 16593–16615. Cited by: Table 12, Appendix C. Zhou et al. (2023) J. Zhou, T. Lu, S. Mishra, S. Brahma, S. Basu, Y. Luan, D. Zhou, and L. Hou Instruction-following evaluation for large language models. arXiv preprint arXiv:2311.07911. Cited by: §1, §2.2, Table 1. Appendix A Constraint Definitions Domain Group Instruction Name Description General Format json_constraint Format the output answer strictly as a JSON object. Selection choose_from Select and output exactly one option from choices. Precision decimal_number Ensure that all numerical quantities in the output retaining the required number of decimal places. Letter all_uppercase Output format must be in all uppercase. Chemistry Terminology molecular_validity Generate valid chemical nomenclature adhering to specified systems. Procedure reaction_steps Output an orderly, multi-step chemical reaction sequence. Geography Terminology address_hierarchy Generate valid geographical addresses following spatial hierarchies. Unit unit_consistency Output numerical results with specified geographical units. Biology Method method_constraint Use specified biological formulas or laws for calculation and reasoning. Numer protein_length Output the required length of a specified biological sequence. Material Terminology characterization_technique Identify valid materials characterization techniques. Format property_type Output different formats based on the material properties. Physics Unit unit_consistency Output numerical results using specified physical units. Procedure analysis_steps_constraint Solve problems step-by-step with physical principles. Table 5: Representative Constraints Selected from Each Domain. A.1 Discipline-Specific Instantiations Although the ten functional constraint groups are shared across SciMIF, their concrete meanings and implementations vary across scientific disciplines. This variation reflects differences in scientific representations, disciplinary conventions, reasoning processes, and output requirements. Figure 6 shows the distribution of functional constraint groups within each scientific discipline. Rather than applying generic instruction templates uniformly, SciMIF instantiates each group according to the knowledge and operational requirements of the corresponding domain. General Constraints. General constraints provide cross-domain requirements that can be combined with scientific constraints from any discipline. They mainly regulate the representation and organization of model outputs, including output format, numerical precision, candidate selection, response structure, capitalization, and the required number of responses. For example, a model may be required to return a result as a JSON object, report all numerical quantities to a specified number of decimal places, select an answer from a predefined candidate set, or separate the reasoning process from the final conclusion. Although these constraints do not necessarily introduce additional scientific knowledge, they are important for ensuring that scientific outputs are unambiguous, machine-readable, and suitable for downstream automated evaluation or analysis. Chemistry. Chemistry involves specialized molecular representations, stoichiometric relationships, and ordered reaction processes. Accordingly, terminology constraints require models to generate valid chemical nomenclature or molecular representations, such as SMILES, while preserving the identity and validity of the target molecule. Number constraints regulate scientifically meaningful quantities, including the numbers of atoms, bonds, functional groups, or other molecular substructures. Procedure constraints require chemical transformations or synthesis routes to be expressed as ordered reaction steps. Method constraints may further specify the chemical principles, calculation rules, or analytical approaches that must be applied during reasoning. These instantiations evaluate whether models can satisfy explicit instructions while correctly interpreting molecular structure and chemical processes. Geography. Geography emphasizes spatial organization, hierarchical relationships, scale dependence, and the interpretation of Earth-related observations. Terminology constraints require models to produce valid geographical expressions, such as locations organized according to administrative or spatial hierarchies. Format constraints are used when models must identify a geographical scene or return a result using a predefined category or structured representation. Unit constraints regulate geographical quantities such as distance, area, elevation, or radiation measurements. Procedure and method constraints specify the analytical steps or spatial reasoning approaches that should be used to interpret maps, remote-sensing images, or other Earth science data. These constraints assess both spatial understanding and compliance with domain-specific geographical conventions. Figure 6: Distribution of the ten functional constraint groups across the five scientific disciplines in SciMIF. The proportions represent the frequency with which each group is instantiated within the corresponding discipline. Biology. Biology involves multi-level entities and processes ranging from molecular sequences to complex biological systems. Terminology constraints require models to use valid biological entity names or represent relationships among genes, proteins, molecules, and other biological entities. Number constraints specify quantities grounded in biological structures, such as protein lengths, sequence lengths, or the number of particular residues or motifs. Procedure constraints require models to organize biological analysis into a predefined sequence of operations, while method constraints require the use of specified biological formulas, analytical rules, or experimental approaches. These instantiations examine whether models can connect instruction with the structural and functional properties of biological systems. Group Constraint Name Description Metric Unit chemistry: unit_consistency Output numerical results using specified chemical stoichiometric units. Script-Based Terminology chemistry: molecular_validity Generate valid chemical nomenclature adhering to specified systems. Script-Based Number chemistry: atom_count_constraint Ensure generated molecules contain a specified number of atoms. Script-Based chemistry: atom_bond_constraint Ensure generated molecules contain a specified number of chemical bonds. Script-Based chemistry: atom_group_constraint Ensure generated molecules contain a specified number of functional groups. Script-Based Method chemistry: method_constraint Apply specified chemical formulas or laws for calculation and reasoning. Script-Based Procedure chemistry: reaction_steps Output an orderly, multi-step chemical reaction sequence. LLM-as-a-Judge Table 6: Constraint Specifications in the Chemistry Domain. Group Constraint Name Description Metric Unit geography: unit_consistency Output numerical results with specified geographical units. Script-Based Terminology geography: address_hierarchy Generate valid geographical addresses following spatial hierarchies. Script-Based Format geography: scene_option_constraint Select the answer strictly from a provided list of geographical scenes. Script-Based Method geography: method_constraint Apply specified geographical formulas or laws for calculation and reasoning. Script-Based Procedure geography: analysis_steps_constraint Conduct reasoning according to specified geographical analysis steps. LLM-as-a-Judge Table 7: Constraint Specifications in the Geography Domain. Group Constraint Name Description Metric Unit biology: unit_consistency Output numerical results using specified biological units. Script-Based Terminology biology: entity_relationship_format_validity Extract and format specified biological entity relationships accurately. Script-Based Number biology: protein_length Output the required length of a specified biological sequence. Script-Based Method biology: method_constraint Use specified biological formulas or laws for calculation and reasoning. Script-Based Procedure biology: analysis_steps_constraint Analyze biological molecules according to a specified sequential bio-analysis flow. LLM-as-a-Judge Table 8: Constraint Specifications in the Biology Domain. Group Constraint Name Description Metric Unit material: unit_consistency Output numerical results using specified materials science units. Script-Based Terminology material: characterization_technique Identify valid materials characterization techniques. Script-Based Format material: property_type Output different formats based on the material properties. Script-Based Method material: method_constraint Apply specified materials science formulas or laws for reasoning. Script-Based Procedure material: analysis_steps_constraint Conduct reasoning according to specified materials analysis steps. LLM-as-a-Judge Table 9: Constraint Specifications in the Material Domain. Group Constraint Name Description Metric Unit physics: unit_consistency Output numerical results using specified physical units. Script-Based Method physics: method_constraint Apply specified physical formulas or laws for calculation and reasoning. Script-Based Procedure physics: analysis_steps Solve problems step-by-step with physical principles. LLM-as-a-Judge Table 10: Constraint Specifications in the Physics Domain. Group Constraint Name Description Metric Precision general: decimal_number Ensure that all numerical quantities in the output retaining the required number of decimal places. Script-Based general: scientific_annotation Express all numerical quantities strictly in scientific notation. Script-Based Letter general: all_uppercase Output format must be in all uppercase. Script-Based general: all_lowercase Format the output answer strictly in all lowercase letters. Script-Based Format general: wrap_up Enclose the final output answer within a specified box format. Script-Based general: json_constraint Format the output answer strictly as a JSON object. Script-Based general: list_constraint Format the output answer strictly as a Python list. Script-Based general: tuple_constraint Format the output answer strictly as a Python tuple. Script-Based general: dictionary_constraint Format the output answer strictly as a Python dictionary. Script-Based general: markdown_constraint Format the output answer strictly using Markdown syntax. Script-Based general: html_constraint Format the output answer strictly using HTML tags. Script-Based general: xml_constraint Format the output answer strictly using XML tags. Script-Based general: csv_constraint Format the output answer as Comma-Separated Values (CSV). Script-Based Selection general: choose_from Select and output exactly one option from choices. Script-Based general: judge Output strictly “Yes” or “No” without additional explanations. Script-Based Structure general: response_structure Follow a structured framework placing final answer at a specified position. Script-Based Number general: number_response Generate a specific number (N) of distinct categorical responses. LLM-as-a-Judge Table 11: Constraint Specifications in the General Domain. Material. Material relies heavily on multimodal characterization, property representation, experimental procedures, and structure-property relationships. Terminology constraints require models to identify valid characterization techniques based on textual and visual evidence. Format constraints specify how material properties should be represented, for example as discrete categories, continuous values, or structured labels. Procedure constraints regulate the ordered analysis of experimental observations or characterization results, while method constraints require reasoning to follow designated material principles or analytical approaches. Number and unit constraints may additionally regulate material quantities such as composition, mass change, dimensional measurements, or physical properties. These constraints evaluate whether models can integrate multimodal evidence with appropriate materials knowledge and report the result in the requested form. Physics. Physics emphasizes dimensional consistency, causal deduction, and principle-based reasoning. Unit constraints require numerical quantities to be expressed using specified physical units while preserving dimensional consistency. Method constraints require the explicit application of designated physical laws, equations, or theorems during problem solving. Procedure constraints specify the sequence of modeling, derivation, calculation, and verification steps that should be followed. These instantiations assess whether models can follow explicit analytical requirements while maintaining logically valid and physically consistent derivations. A.2 Complete Constraint Specifications Table 5 presents representative constraints selected from the general domain and the five scientific disciplines. The examples illustrate how the functional constraint groups are instantiated as concrete and independently evaluable requirements in different scientific contexts. For each constraint, we report its functional group, canonical name, and operational description. The complete constraint inventory contains 42 individual constraints organized into ten functional groups. Each constraint is assigned a canonical identifier in the form domain:constraint_name, where domain specifies whether the constraint belongs to the general domain or one of the five scientific disciplines. This naming convention distinguishes individual constraints that belong to the same functional group but require different disciplinary knowledge or evaluation procedures. For example, the terminology group includes molecular validity in Chemistry, address hierarchy in Geography, biological entity relationships in Biology, and characterization techniques in Material. Each constraint specification contains four components: (1) the functional group to which the constraint belongs; (2) a canonical constraint name; (3) an operational description defining the requirement imposed on the model output; and (4) an evaluation method used to determine whether the requirement is satisfied. For constraints with deterministic output conditions, a rule-based evaluation approach is adopted, including output format, numerical precision, capitalization, units, candidate selection, and the number of specified scientific entities. Constraints requiring the assessment of open-ended reasoning processes, such as the application of a designated scientific method or adherence to a specified analytical procedure, are evaluated using an LLM-as-a-Judge protocol. The complete chemistry constraint specifications are provided in Table 6. They cover requirements involving chemical units, molecular representations, atom and bond quantities, functional groups, designated calculation methods, and ordered reaction procedures. The complete geography constraint specifications are provided in Table 7. They include geographical units, spatial and administrative hierarchies, geographical scene representations, designated spatial-analysis methods, and ordered analysis procedures. The complete biology constraint specifications are provided in Table 8. They cover biological units, entity relationships, sequence-related quantities, designated biological methods, and sequential bio-analysis procedures. The complete material constraint specifications are provided in Table 9. They include materials science units, characterization techniques, material-property representations, designated analytical methods, and ordered materials analysis procedures. The complete physics constraint specifications are provided in Table 10. They cover physical units, designated physical laws and formulas, and step-by-step derivation or analysis procedures. Finally, the complete general constraint specifications are provided in Table 11. These constraints can be combined with discipline-specific requirements and primarily regulate output format, numerical precision, candidate selection, response structure, capitalization, and the required number of output items. Together, these specifications provide an explicit mapping from the constraint taxonomy to discipline-grounded and independently verifiable requirements. They enable model performance to be evaluated both at the level of individual scientific requirements and across common instruction-following capability groups. Appendix B Detailed Data Construction Procedure This section provides the formal definitions and implementation details of the SciMIF construction pipeline described in Section Data Construction. An overview of the complete procedure is provided in Algorithm 1. B.1 Seed Representation and Applicable Constraints Each seed sample from a source dataset is represented as x=(q,I,a,t),x=(q,I,a,t), (6) where q is the original textual query, I is an optional visual input, a is the reference answer, and t denotes the scientific task type. Let C denotes the complete constraint inventory of SciMIF. For each sample x, we construct an applicable subset x=xsci∪xgen,C_x=C^sci_x ^gen_x, (7) where xsciC^sci_x contains scientific constraints compatible with the discipline and task type of x, and xgenC^gen_x contains general constraints applicable to the sample x. B.2 Existing Constraint Filtering Annotators manually identify constraints, including scientific and general, that are already explicitly stated or implicitly required by the original query. The recognized constraint set is defined as Co=fr(q,Cx),C_o=f_r(q,C_x), (8) where frf_r denotes the manual recognition process, and the set o⊆xC_o _x. Recognizing existing constraints before augmentation prevents the construction process from repeatedly injecting an equivalent requirement or introducing a new constraint that conflicts with the original task. B.3 Candidate Constraint Selection The initial candidate pool is constructed from Cx∖CoC_x C_o, in which the applicable constraints that are not already present in the query. A selection operation is then applied: (Cs,Cg)=fs(x,Cx∖Co),(C_s,C_g)=f_s (x,C_x C_o ), (9) where CsC_s represents the scientific constraints that domain experts determine are compatible with the discipline, task type t, visual input I, and reference answer a. CgC_g represents general constraints that annotators exclude constraints that duplicate existing requirements or conflict with either the original query or the selected scientific constraints. B.4 Scientific Constraint Injection The selected scientific constraints are first adapted to the original query: qd=fd(q,Cs),q_d=f_d (q,C_s ), (10) where qdq_d denotes the query after scientific constraint injection. During adaptation, the wording of each scientific constraint may be adjusted to match the terminology and conventions of the corresponding discipline. However, the required scientific operation and evaluation criterion remain unchanged. B.5 General Constraint Injection Algorithm 1 SciMIF Data Construction 1: Seed sample x=(q,I,a,t)x=(q,I,a,t), constraint inventory C, N, k 2: Augmented sample x x 3: Construct applicable constraints x=xsci∪xgenC_x=C^sci_x ^gen_x 4: Identify existing constraints o=fr(q,x)C_o=f_r(q,C_x) 5: Select compatible constraints (Cs,Cg)=fs(x,x∖o)(C_s,C_g)=f_s(x,C_x _o) 6: Inject scientific constraints qd=fd(q,Cs)q_d=f_d(q,C_s) 7: Sample at most N categories from CgC_g 8: q(0)←qdq^(0)← q_d 9: for i=1,…,Ni=1,…,N do 10: q(i)←q(i−1)q^(i)← q^(i-1) 11: for all candidate constraints cnc_n in category i do 12: Generate qg=fg(q(i−1),cn)q_g=f_g(q^(i-1),c_n) up to k times 13: if included(qg,cn)∧valid(qg,a)I_included(q_g,c_n) _valid(q_g,a) then 14: q(i)←qgq^(i)← q_g 15: break 16: end if 17: end for 18: end for 19: q^←q(N) q← q^(N) 20: x^←(q^,I,a,t) x←( q,I,a,t) 21: return x x We sample at most N mutually compatible general constraint categories from CgC_g. Let the selected constraints be Cg1,Cg2,…,CgN,\C_g^1,C_g^2,…,C_g^N\, (11) Each category contains multiple constraints, one of which is cnc_n. General constraints are manually categorized according to their compatibility, and at most one constraint is selected from the same category for a single augmented instruction. cnc_n is injected sequentially, then the candidate augmented query is generated as qg=fg(qd,cn).q_g=f_g (q_d,c_n ). (12) The candidate query is accepted only when it passes both automatic checks: included(qg,cn)∧unchanged(qg,a)=1.I_included (q_g,c_n ) _unchanged (q_g,a )=1. (13) IincludedI_included verifies that the target constraint is explicitly and unambiguously expressed in the augmented query, while IunchangedI_unchanged verifies that the original reference answer remains semantically sufficient for the augmented query. Literal equality is not required, which means equivalent numerical representations, capitalization changes, and output formatting are permitted, provided that the factual content of the answer remains correct and sufficient. We utilize DeepSeek-Chat as the verification model. If either check fails, the generation is retried up to k times. When all k attempts for cnc_n fail, another compatible constraint from the same category is selected and verified using the same procedure. If no constraint in that category passes the verification, the category is skipped rather than forcing an invalid augmentation. After sequentially processing the selected general constraints, the final augmented query and complete constraint set are defined as q q. The resulting augmented sample is x^=(q^,I,a,t). x=( q,I,a,t). (14) Source Task Num. Source Task Num. ChemEval (Huang et al. 2024) chemistry_numerical _task 70 IMAGEO-Bench (LI et al. 2025) earth_scene_option 100 ChemEval (Huang et al. 2024) chemistry_entity_option 70 LAB-Bench (Laurent et al. 2024) biology_sequence_qa 143 ChemEval (Huang et al. 2024) chemistry_reaction_steps 20 Mol-Instructions (Fang et al. 2024) biology_entity_ relationship 100 ChemEval (Huang et al. 2024) chemistry_molecular _format 70 Mol-Instructions (Fang et al. 2024) biology_analysis_steps 250 ChemEval (Huang et al. 2024) chemistry_open_ended 198 MaScQA (Zaki et al. 2023) material_numerical _problem 100 S2-TOMG-Bench-mini (Li et al. 2025) chemistry_molcustom_ bondnum 30 MatCha (Lai et al. 2025) material_characterization _technique 100 S2-TOMG-Bench-mini (Li et al. 2025) chemistry_molcustom_ functionalgroup 30 LLM4Mat-Bench (Niyongabo Rubungo et al. 2025) material_property _prediction 97 S2-TOMG-Bench-mini (Li et al. 2025) chemistry_molcustom_ atomnum 30 MatSciBench (Zhang et al. 2025a) material_reasoning _process 200 EarthSE (Xu et al. 2025a) earth_calculation_task 73 UGPhysics (Xu et al. 2025b) physics_formatted_ numerical_task 150 EarthSE (Xu et al. 2025a) earth_reasoning_steps 250 PhysReason (Zhang et al. 2025c) physics_formatted_ reasoning_process 196 IMAGEO-Bench (LI et al. 2025) earth_address_format 100 PhysUniBench (Wang et al. 2025a) physics_formatted_ open_ended 150 Table 12: Overview of task types, source datasets, and sample counts. B.6 Human Verification Automatic validation ensures that the target constraints are explicitly included in the augmented query and that the original reference answer remains valid. However, automatic rules may not fully capture linguistic naturalness, implicit contradictions, or ambiguous scientific expressions. We therefore conduct human verification on all augmented samples before including them in the final benchmark. Two annotators independently review each augmented query q q together with its associated constraint set. The verification focuses on the following two criteria. Logical Coherence and Fluency. The augmented query must remain semantically coherent and grammatically fluent after constraint injection. Each injected constraint should be naturally integrated into the original scientific problem and should not introduce contradictory requirements, ambiguous references, unnecessary repetition, or incompatibility with the original task setting. Constraint Fidelity. Every constraint must be explicitly and unambiguously expressed in q q. The augmented query must preserve the intended operational meaning and evaluation condition of each constraint, rather than merely mentioning related terminology. The annotators also verify that the injected requirements do not alter the scientific question, change the expected answer, or introduce additional conditions that cannot be satisfied by the original reference answer. Annotation Consistency. Before full-scale verification, the two annotators independently examined the same 20 randomly sampled instances to calibrate the verification criteria. They reached the same decision on all sampled instances, yielding an observed agreement of 100%. This pilot verification was used to confirm that the two criteria could be applied consistently before the remaining samples were reviewed. Verification Results. Samples that fail either criterion undergo manual revision. Annotators rewrite only the problematic portions of the augmented query while preserving the original scientific task, optional visual input, and reference answer. When a sample cannot be repaired without changing its scientific meaning or making one or more constraints invalid, it is discarded. All retained samples undergo human verification, and 884 samples are manually revised before inclusion in the final benchmark ′D . Appendix C Data Sources and Task Coverage SciMIF is constructed from 13 existing scientific datasets spanning five disciplines. The chemistry sources include S2-TOMG-Bench-mini (Li et al. 2025) and ChemEval (Huang et al. 2024); the geography sources include EarthSE (Xu et al. 2025a) and IMAGEO-Bench (LI et al. 2025); the biology sources include Mol-Instructions (Fang et al. 2024) and Lab-Bench (Laurent et al. 2024); the material sources include MaScQA (Zaki et al. 2023), MatCha (Lai et al. 2025), LLM4Mat-Bench (Niyongabo Rubungo et al. 2025), and MatSciBench (Zhang et al. 2025a); and the physics sources include UGPhysics (Xu et al. 2025b), PhysReason (Zhang et al. 2025c), and PhysUniBench (Wang et al. 2025a). Together, these datasets contribute 22 task types with diverse problem formulations, disciplinary conventions, and input modalities. Specifically, SciMIF contains eight task types in Chemistry, four in Geography, three in Biology, four in Material, and three in Physics. The complete task list, corresponding data sources, and sample statistics are presented in Table 12. This broad coverage increases the structural diversity of SciMIF and reduces its dependence on any single discipline or task design. Appendix D Evaluation Details D.1 Instruction-Following Metrics In this section, we introduce specific calculation formula of each metric as follows. CSR measures the average proportion of satisfied constraints across all instructions. Formally, it is defined as CSR=1m∑i=1m(1ni∑j=1nisi,j),CSR= 1m _i=1^m ( 1n_i _j=1^n_is_i,j ), where m denotes the total number of instructions, nin_i represents the number of constraints associated with the i-th instruction, and si,j∈0,1s_i,j∈\0,1\ indicates whether the j-th constraint in the i-th instruction is satisfied. ISR evaluates the proportion of instructions for which all associated constraints are completely satisfied. It is computed as ISR=1m∑i=1msi,ISR= 1m _i=1^ms_i, where si∈0,1s_i∈\0,1\ indicates whether all constraints in the i-th instruction are satisfied. DRFR measures the overall satisfaction of decomposed requirements across all instructions. Instead of evaluating instructions as a whole, this metric assesses requirement-level compliance through a set of scoring questions. It is defined as DRFR=∑i∑j=1miri,j′∑imi,DRFR= _i _j=1^m_ir _i,j _im_i, where mim_i denotes the number of scoring questions associated with the i-th instruction, and ri,j′r _i,j represents the result of the j-th scoring question for the i-th instruction. Model Chemistry Geography Biology Material Physics General Scientific Closed-Source MLLMs GPT-5.2 72.65 91.28 92.69 92.06 93.66 74.65 88.74 Grok-4-Fast 68.43 83.50 82.00 91.92 88.40 74.46 82.93 Gemini-3.1-Pro-Preview 63.51 89.97 80.24 93.48 88.84 66.04 83.45 Claude-Sonnet-4.6 71.38 86.27 80.11 91.41 93.65 61.14 84.74 Open-Source MLLMs Qwen3.5-27B 67.61 87.73 85.81 91.04 92.91 74.34 85.26 Qwen3.5-35B-A3B 65.69 87.95 86.19 91.05 92.51 74.41 84.88 Qwen3.5-122B-A10B 64.95 87.80 84.34 90.45 92.82 75.02 84.37 Qwen3.5-397B-A17B 69.73 90.28 87.92 90.37 92.69 76.37 86.47 InternVL3.5-8B 69.60 88.43 82.62 92.43 92.84 59.04 85.27 InternVL3.5-14B 69.13 87.41 85.65 91.80 93.03 61.96 85.61 InternVL3.5-38B 68.01 89.16 82.24 90.95 92.64 59.71 84.85 Table 13: DRFR scores (%) of different constraint domains from evaluated MLLMs. The best score in each column is bolded separately for closed-source and open-source MLLMs. D.2 Scientific Answer Correctness Evaluation We separately evaluate whether each model response correctly answers the underlying scientific problem. We utilize CompassVerifier-32B (Liu et al. 2025) to evaluate the answer correctness. The judge model is provided with the original question, the visual input when available, the reference answer, and the model response, and returns a binary correctness label. This evaluation focuses only on scientific correctness and does not penalize violations of injected constraints unless they alter the scientific meaning of the answer. The resulting labels are combined with instruction-adherence results to classify responses as Correct Followed (CF), Correct Violated (CV), Incorrect Followed (IF), or Incorrect Violated (IV). D.3 Constraint Verification Protocols We adopt 2 constraint verification protocols to judge the instruction-following capability of models. Script-Based Evaluation We employ programmatic verification when constraint compliance can be determined using explicit and reproducible criteria. This category includes numerical-format and casing checks, structured parsing of JSON, lists, tuples, dictionaries, HTML, XML, and CSV, and domain-specific validation with tools such as RDKit. Regular-expression matching is used to identify formulas, answer markers, option labels, scientific notation, and other predefined textual patterns. For constraints requiring semantic extraction, an LLM may extract a molecular representation or scientific method from the response. However, the extracted content is subsequently verified using regular expressions, structured parsers, or RDKit, and the LLM does not make the final compliance decision. The extractor model in the experiment is GPT-4.1. LLM-as-a-Judge We use an LLM as the evaluator when constraint satisfaction requires semantic interpretation that cannot be reliably determined through fixed rules. The judge compares the augmented question, constraint parameters, and model response to determine whether the required reasoning or reaction steps are covered and to calculate the proportion of matched steps. It is also used to determine whether a response contains exactly the required number of semantically distinct response categories. The judge is instructed to return a structured decision and justification, which are parsed into the final compliance score. We select GPT-4.1 as the judge model. Appendix E Additional Results and Analysis E.1 Complete results over scientific and general constraints We illustrate complete results of models over 5 scientific constraint domains and general constraint domain in Table 13 to support the conclusion in the section Performance on Constraint Domains. E.2 Analysis of the Significance of Correctness and Instruction Following Model ϕφ Jaccard p-value GPT-5.2 0.1178 0.3585 3.21×10−93.21× 10^-9 Grok-4-Fast 0.0081 0.2930 0.68 Gemini-3.1-Pro-Preview 0.1548 0.3538 7.35×10−157.35× 10^-15 Claude-Sonnet-4-6 0.2069 0.3640 4.31×10−254.31× 10^-25 Qwen3.5-397B-A17B 0.0672 0.3300 7.68×10−47.68× 10^-4 InternVL3.5-14B 0.1459 0.2704 2.33×10−132.33× 10^-13 Table 14: Within-model associations between scientific correctness and instruction following. ϕφ coefficient measures the direction and strength of the binary association, Jaccard measures the overlap between the two outcomes, and the p-values are obtained from two-sided Pearson χ2χ^2 tests of independence. To examine the within-model association between scientific correctness and instruction following, we construct a 2×22× 2 contingency table for each model. Let A denote scientific correctness and B denote instruction following. The four observed cell counts are CFCF, CVCV, IFIF , and IVIV: B=1B=0A=1CFCVA=0IFIV. array[]c|c&B=1&B=0\\ A=1&CF&CV\\ A=0&IF&IV array. We apply the two-sided Pearson χ2χ^2 test of independence separately to each model. The null hypothesis is H0:A and B are independent(ϕ=0),H_0:\ A and B are independent (φ=0), (15) whereas the alternative hypothesis is H1:A and B are not independent(ϕ≠0).H_1:\ A and B are not independent (φ≠ 0). (16) For each cell, the expected count under H0H_0 is calculated as Eij=(row totali)(column totalj)n,E_ij= (row total_i)(column total_j)n, (17) and the Pearson test statistic is χ2=∑i=12∑j=12(Oij−Eij)2Eij,χ^2= _i=1^2 _j=1^2 (O_ij-E_ij)^2E_ij, (18) where OijO_ij and EijE_ij are the observed and expected counts, respectively. Because a 2×22× 2 table has one degree of freedom, the two-sided p-value is obtained from the upper tail of a χ12χ^2_1 distribution: p=Pr(χ12≥χobs2∣H0)=1−Fχ12(χobs2).p= \! (χ^2_1≥χ^2_obs H_0 )=1-F_χ^2_1\! (χ^2_obs ). (19) Thus, a small p-value indicates that the observed discrepancy from independence would be unlikely under H0H_0. We reject H0H_0 when p<0.05p<0.05. Because six model-specific tests are conducted, we additionally apply a Bonferroni-corrected threshold of α∗=0.056=0.00833.α^*= 0.056=0.00833. (20) The direction and magnitude of association are quantified using the ϕφ coefficient: ϕ=CF⋅IV−CV⋅IF(CF+CV)(IF+IV)(CF+IF)(CV+IV).φ= CF· IV-CV· IF (CF+CV)(IF+IV)(CF+IF)(CV+IV). (21) Equivalently, for a 2×22× 2 table, |ϕ|=χ2n,|φ|= χ^2n, (22) with its sign determined by CF⋅IV−CV⋅IFCF· IV-CV· IF. The coefficient ranges from −1-1 to 11. ϕ>0φ>0 indicates that correctness and instruction following tend to occur together, ϕ<0φ<0 indicates an inverse association, and values close to zero indicate weak association. As a complementary descriptive measure, we calculate the Jaccard coefficient: J=CFCF+CV+IF.J= CFCF+CV+IF. (23) It represents the proportion of responses satisfying both criteria among those satisfying at least one. The IVIV cell is excluded because those responses satisfy neither criterion. Table 14 demonstrates that scientific correctness and instruction following are statistically related but remain distinct and only weakly coupled capabilities. Under the two-sided Pearson χ2χ^2 test of independence, five of the six models exhibit statistically significant positive associations. These five results remain significant after Bonferroni correction (α∗=0.00833α^*=0.00833), indicating that the conclusions are robust to multiple testing rather than artifacts of repeated comparisons. Grok-4-fast is the only exception, with p=0.68p=0.68 and a near-zero ϕφ coefficient (ϕ=0.0081φ=0.0081), providing no significant evidence against independence. Statistical significance, however, does not imply strong practical coupling. As shown in Table 14, the ϕφ coefficients are modest even for the statistically significant models, ranging from 0.06720.0672 to 0.20690.2069. The Jaccard coefficients show that only 27.04%27.04\%–36.40%36.40\% of responses satisfying at least one criterion satisfy both simultaneously. Taken together, the p-values quantify the evidence against independence, the positive but modest ϕφ coefficients characterize the direction and limited strength of the associations, and the Jaccard coefficients describe the limited overlap between the two desirable outcomes. These findings expose a systematic capability gap that instruction adherence is not a reliable proxy for scientific correctness, and scientifically correct answers do not necessarily satisfy the specified constraints. Consequently, optimizing either capability in isolation is insufficient, and models should be explicitly trained and evaluated for their joint attainment. E.3 Results across Source Datasets To determine whether discipline-level averages obscure variation among the heterogeneous source datasets, we further disaggregate GPT-5.2’s performance within Material. This setting holds the model and discipline fixed while comparing all four material sources in SciMIF. As shown in Table 15, DRFR varies substantially across sources, ranging from 63.92% on LLM4Mat-Bench to 96.28% on MatSciBench. In particular, LLM4Mat-Bench is 22.55 percentage points below the next-lowest source, MatCha, and 32.36 points below MatSciBench. This result shows that the aggregate materials science score does not imply uniform instruction-following ability across its constituent datasets. MaScQA Matcha LLM4Mat-Bench MatSciBench DRFR 93.19 86.47 63.92 96.28 Table 15: Per-source DRFR (%) of GPT-5.2 on materials science. E.4 Text-Only and Multimodal Performance To examine modality-associated differences in instruction-following performance, we restrict the analysis to Geography, Material, and Physics, the three disciplines in SciMIF that contain both text-only and multimodal samples. Chemistry and Biology are excluded because their samples are exclusively text-based. We use Gemini-3.1-Pro-Preview as a representative model and compare the two modalities within each discipline, thereby holding the evaluated model fixed and reducing cross-disciplinary confounding. As shown in Table 16, the multimodal subsets consistently obtain lower DRFR scores than their text-only counterparts: 64.67% versus 87.92% in Geography, 85.50% versus 90.16% in Material, and 74.49% versus 83.76% in Physics. These values correspond to absolute decreases of 23.25, 4.66, and 9.27 percentage points, respectively. The consistently lower scores provide evidence of a modality-associated degradation in instruction compliance, while the heterogeneous gaps indicate that this pattern varies across disciplinary contexts. Modality Geography Material Physics Multimodal 64.67 85.50 74.49 Text-only 87.92 90.16 83.76 Table 16: DRFR scores (%) of Gemini-3.1-Pro-Preview on multimodal and text-only subsets. E.5 Correctness Contrast To investigate how instructions influence task difficulty, we evaluated GPT-5.2 and Qwen3.5-397B-A17B on the original and constraint-augmented queries. The results in Figure 7 indicate that, in the absence of explicit instructions, the models generally achieve higher accuracy across most domains. However, when tasks include additional requirements, the models’ attention to instructions incurs an accuracy penalty, suggesting a potential degradation of capability in settings that demand strict output adherence, such as agent pipelines. This trade-off highlights the challenge of simultaneously maintaining model reliability and usability, and motivates further research into methods that enhance both scientific rigor and practical effectiveness. (a) GPT-5.2. (b) Qwen3.5-397B-A17B. Figure 7: Scientific answer correctness before and after constraint augmentation for two representative models across the five disciplines. Before denotes evaluation on the original scientific questions, while After denotes evaluation after compatible scientific and general constraints are added. The comparison shows how additional instruction-following requirements are associated with changes in scientific answer correctness. E.6 Constraint-Level Error Analysis To better understand the underlying mechanisms behind the performance disparities across different constraint types, we conduct a detailed analysis on GPT-5.2, whose DRFR scores for each constraints are shown in Figure 8. The observations clearly indicate that GPT-5.2’s performance in executing chemistry constraints is significantly lower than its performance in other disciplines. This phenomenon aligns with the conclusion that GPT-5.2 achieved its lowest score in the field of chemistry, and further corroborates its areas of weakness within the subject. Figure 8: GPT-5.2 performance across individual scientific constraints. Figure 9: A representative physics case illustrating partial constraint satisfaction. The response satisfies the required analysis procedure and response structure but violates the lowercase constraint. E.7 Representative Case Studies To provide a more concrete view of model behavior beyond aggregate scores, we analyze a representative physics example containing both scientific and general constraints. As shown in Figure 9, the instruction asks the model to solve a mechanics and electromagnetism problem while simultaneously satisfying three independently evaluated constraints. Specifically, the model is required to follow a step-by-step physics analysis procedure, organize the response according to a prescribed structure, and produce the entire output in lowercase. The response correctly derives the maximum static-friction force, allowable magnetic force, current requirement, and circuit-resistance range using relevant physical principles. It therefore satisfies the constraint named physics:analysis_steps. The final result, 1.9Ω≤r≤9.9Ω1.9\, ≤ r≤ 9.9\, , is also placed after the reasoning process, satisfying the general:response_structure constraint. However, the response violates the constraint, which is general:all_lowercase because uppercase symbols such as R, E, B, and L appear in the output. This case shows that correct scientific reasoning and structural compliance do not necessarily guarantee adherence to fine-grained symbolic requirements. It also demonstrates the value of constraint-level evaluation, which can identify the specific source of failure rather than treating the entire instruction as uniformly unsuccessful. Appendix F Data Compliance and Release SciMIF is constructed by augmenting samples from 13 existing scientific datasets. All source datasets are used exclusively for academic research and are processed in accordance with their respective licenses, terms of use, and redistribution requirements. The original scientific questions, visual inputs, and reference answers retain their original provenance, and the corresponding source datasets are cited in the main paper and documented in the Appendix. Our construction process does not alter the underlying scientific problem or its reference answer. Instead, it adds compatible scientific and general constraints to the original query and records the injected constraints as independently evaluable metadata. Each released sample will therefore include its augmented query, reference answer, constraint list, task type, discipline, and source-dataset identifier, subject to the redistribution permissions of the corresponding source dataset. For source datasets that permit redistribution, the processed samples will be included directly in the SciMIF release while preserving the required attribution and license information. For datasets or visual assets whose licenses do not permit direct redistribution, we will not republish the restricted content. Instead, we will provide source identifiers, data-processing scripts, and reconstruction instructions that allow eligible users to obtain the original data from its official source and reproduce the corresponding SciMIF samples. The public release will include the constraint taxonomy, augmented queries, constraint annotations, evaluation configurations, deterministic verification scripts, LLM-as-a-Judge prompts, and model evaluation code. We will additionally provide documentation describing the provenance and applicable license of each source dataset. This release strategy is intended to support reproducibility while respecting the ownership, attribution, and redistribution conditions of the original scientific resources.