Paper deep dive
MobiFlow: Real-World Mobile Agent Benchmarking through Trajectory Fusion
Yunfei Feng, Xi Zhao, Cheng Zhang, Dahu Feng, Daolin Cheng, Jianqi Yu, Yubin Xia, Erhu Feng
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%
Last extracted: 7/20/2026, 4:47:45 AM
Summary
The paper introduces MobiFlow, an evaluation framework for mobile GUI agents that addresses the limitations of existing benchmarks like AndroidWorld by supporting third-party applications without system-level APIs. MobiFlow utilizes a multi-trajectory fusion algorithm to construct Trajectory-Fused State Graphs (TCSG), compressing the state space while maintaining coverage of real-world tasks. The framework evaluates 20 applications across 240 tasks using comprehensive metrics including Success Rate, Completion Rate, Coverage Rate, Action Match Rate, and Time to Action, demonstrating higher alignment with human assessments compared to prior methods.
Entities (12)
Relation Signals (11)
MobiFlow â covers â 20 applications
confidence 98% ¡ MobiFlow covers 20 widely used third-party applications
MobiFlow â covers â 240 tasks
confidence 98% ¡ comprises 240 diverse real-world tasks
UI-TARS â achievessuccessrate â 60.4%
confidence 95% ¡ UI-TARS and GUI-OWL achieve success rates of 60.4% and 55.7%, respectively
GUI-Owl â achievessuccessrate â 55.7%
confidence 95% ¡ UI-TARS and GUI-OWL achieve success rates of 60.4% and 55.7%, respectively
MobiFlow â usesmethod â Trajectory-Fused State Graph
confidence 95% ¡ MobiFlow constructs state transition graphs from third-party applications... using an efficient graph-construction algorithm based on multi-trajectory fusion
MobiFlow â employsmetric â Success Rate
confidence 90% ¡ This metric considers only task outcomes... Success Rate (SR)
MobiFlow â employsmetric â Completion Rate
confidence 90% ¡ Completion Rate (CR) measures the agentâs progress toward the goal state
MobiFlow â employsmetric â Coverage Rate
confidence 90% ¡ Coverage Rate (CVR) quantifies the proportion of explored states
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Mobile agents can autonomously complete user-assigned tasks through GUI interactions. However, existing mainstream evaluation benchmarks, such as AndroidWorld, operate by connecting to a system-level Android emulator and provide evaluation signals based on the state of system resources. In real-world mobile-agent scenarios, however, many third-party applications do not expose system-level APIs to determine whether a task has succeeded, leading to a mismatch between benchmarks and real-world usage and making it difficult to evaluate model performance accurately. To address these issues, we propose MobiFlow, an evaluation framework built on tasks drawn from arbitrary third-party applications. Using an efficient graph-construction algorithm based on multi-trajectory fusion, MobiFlow can effectively compress the state space, support dynamic interaction, and better align with real-world third-party application scenarios. MobiFlow covers 20 widely used third-party applications and comprises 240 diverse real-world tasks, with enriched evaluation metrics. Compared with AndroidWorld, MobiFlow's evaluation results show higher alignment with human assessments and can guide the training of future GUI-based models under real workloads.
Tags
Links
- Source: https://arxiv.org/abs/2604.09587v1
- Canonical: https://arxiv.org/abs/2604.09587v1
Trouble viewing inline? Open PDF directly â
Full Text
67,105 characters extracted from source content.
Expand or collapse full text
MobiFlow: Real-World Mobile Agent Benchmarking through Trajectory Fusion Yunfei Feng 1 Xi Zhao 1 Cheng Zhang 1 Dahu Feng 2 Daolin Cheng 1 Jianqi Yu 3 Yubin Xia 1 Erhu Feng 1 Single Task ,Multiple Trajectories Trajectory-Fused State GraphResults Analysis SR CR TTA CVR AMR ¡ Third-party APP ¡ Diverse Scenario ¡ Multi -Solutions¡ Complex real tasks ⢠Highly Aligned With Human Test ⢠Effectively Compress StateSpace ⢠Task-Oriented Complete ⢠Efficient GraphConstructionn ⢠Comprehensive Evaluation Multi-scenario Capability ExecutionEfficiency CorrelatedCapability Figure 1. The framework of MobiFlow. It constructs state transition graphs from third-party applications and evaluates 20 applications across 240 tasks. The framework extends existing metrics to assess multiple models in terms of task completion, execution efficiency, generalization, and alignment with competency requirements. The code is available athttps://github.com/nanookfyf/ MobiBench. Our data will be released upon acceptance. Abstract Mobile agents can autonomously complete user- assigned tasks through GUI interactions. How- ever, existing mainstream evaluation benchmarks, such as AndroidWorld, operate by connecting to a system-level Android emulator and provide eval- uation signals based on the state of system re- sources. In real-world mobile-agent scenarios, however, many third-party applications do not expose system-level APIs to determine whether a task has succeeded, leading to a mismatch be- tween benchmarks and real-world usage and mak- 1 Institute of Parallel and Distributed Systems (IPADS), Shanghai Jiao Tong University 2 Department of Precision Instru- ment, Tsinghua University 3 National Innovation Institute of High-end Smart Appliances. Correspondence to: Erhu Feng <fengerhu1@sjtu.edu.cn>. Preprint. April 14, 2026. ing it difficult to evaluate model performance accurately. To address these issues, we pro- pose MobiFlow, an evaluation framework built on tasks drawn from arbitrary third-party appli- cations. Using an efficient graph-construction algorithm based on multi-trajectory fusion, Mob- iFlow can effectively compress the state space, support dynamic interaction, and better align with real-world third-party application scenarios. Mob- iFlow covers 20 widely used third-party appli- cations and comprises 240 diverse real-world tasks, with enriched evaluation metrics. Com- pared with AndroidWorld, MobiFlowâs evalua- tion results show higher alignment with human assessments and can guide the training of future GUI-based models under real workloads. 1 arXiv:2604.09587v1 [cs.AI] 28 Feb 2026 MobiFlow: Real-World Mobile Agent Benchmarking through Trajectory Fusion 1. Introduction With the advancement of artificial intelligence technology, graphic User Interface (GUI) agents (Ye et al., 2025a), driven by multimodal large models(Ma et al., 2024), are emerging as a key technology poised to transform next- generation terminal applications. These agents, guided by human instructions, automatically accomplish daily and pro- fessional tasks across diverse device environments, thereby enhancing production efficiency and improving user op- erational experience. However, how to conduct fast and accurate evaluations of GUI agentsâ actual potential based on real-world usage scenarios remains a significant chal- lenge(Rawles et al., 2024). Existing mobile-use evaluation benchmarks are primarily evaluated through online and offline methods. Online bench- mark enables interaction with agents via Android emulators and extracts evaluation signals from system-level resource states(Toyama et al., 2021; Rawles et al., 2024). However, the vast majority of third-party applications do not expose system-level interfaces, making it difficult to obtain accurate evaluation signals and consequently limiting the range of applications that can be evaluated. For the Apps that can be evaluated, the environmental factors also make experimental reproduction difficult. Moreover, for vendor applications that do provide system-level APIs, mobile agents operating through GUI interactions are of limited practical relevance. As a result, such evaluation settings inevitably exhibit a gap from real-world usage scenarios. Offline benchmarks rely on pre-collected human interac- tion trajectories and assess agents by comparing their action sequences against these reference trajectories(Zhang et al., 2024a; Xu et al., 2025a). While this approach can be ap- plied to arbitrary applications, the state space of mobile user interfaces is nearly infinite, making it difficult for offline data collection to cover all possible states. Consequently, existing offline datasets typically provide only one or a few trajectories per task. Some studies(Song et al., 2025) attempt to construct a complete interface state transition graph by traversing all interactive UI elements; however, as the graph depth increases, the state space grows expo- nentially. Consequently, existing mobile usage evaluation benchmarks still struggle to assess agent performance on real-world third-party applications comprehensively. To address the dilemma that online benchmarks fail to cover the full range of applications, while offline benchmarks fail to capture all task completion trajectories. We observe that for each task, the set of independent UI objects relevant to the task on a single screen is finite (usually< 3), and different trajectories often converge to the same node after only a few steps. Based on this insight, we propose Mob- iFlow, which compresses the state space by constructing Trajectory-Fused State Graphs, enabling effective eval- uation signals for arbitrary third-party applications while preserving complete state coverage. A ction O bservation Task: Could you order a milk tea for me? S_1 S_2 S_3 S_4 S_5 Done Interact WithRealDevices Task S tate TransitionGraph a_1 a_3 a_2 a_4 a_5 a_6 a_8 a_7 a_9 a_10 Figure 2. Modeling agent-device interactions with state transi- tion graph. Executing actions triggers state transitions. Complet- ing a task corresponds to reaching a terminal state. We efficiently collect human operation trajectories from real-world task scenarios using front-end tools 1 , including interface screenshots, actions, and annotation information. By merging nodes with identical annotation information and sharing their action transitions, we consolidate multi- ple real trajectories into a state transition graph capable of simulating realistic tasks. This construction effectively mod- els real-world interaction environments while compressing state complexity, enabling a comprehensive evaluation of mobile agentsâ practical capabilities. MobiFlow covers 20 widely used mobile applications and includes 240 real-world tasks designed to comprehensively evaluate the performance of current mainstream general- purpose models and GUI-specialized models on practical tasks(Ye et al., 2025a; Qin et al., 2025; Wang et al., 2025; Zhang et al., 2025). We abstract the agentâs task execu- tion process as state transitions on a graph (see figure2), which enables more precise evaluation(Bu et al., 2025). We can establish new metrics, such as coverage rate and com- pletion rate, to comprehensively measure model capabili- ties. In addition, we design specialized scenarios to assess specific model capabilities, including instruction-following, instruction-noise interference, and multi-application inter- ference, etc. Through the aforementioned definitions and scenario design, we show that the metrics derived from the graph structure exhibit strong interpretability. On our benchmark, UI-TARS and GUI-OWL achieve success rates of 60.4% and 55.7%, respectively, with single-action execution times of 1.75 s and 20.02 s. Compared to other benchmarks, our evaluation results align more closely with human judgment. More- over, MobiFlow has been deployed to assess the capabilities of GUI agent models on smartphones with sales volumes exceeding ten million units. In summary, our main contributions are as follows: â˘An efficient multi-trajectory fusion-based graph construction algorithm. Our algorithm effectively 1 Implementation details are provided in the AppendixA. 2 MobiFlow: Real-World Mobile Agent Benchmarking through Trajectory Fusion compresses the state space size, reduces complexity, and ensures generalization to real-world tasks. â˘Diverse evaluation metrics and test scenarios. Our method evaluates multiple dimensions, including time efficiency and coverage, and designs specialized sce- narios, such as instruction following and noise interfer- ence. â˘Effective guidance for the future development of mobile agents. We conduct comparative evaluations between general-purpose models and specialized mod- els on the evaluation set, along with an attribution anal- ysis of their performance across different scenarios. 2. Related Work This section review existing evaluation frameworks(Detailed comparisons are provided in Table1), which can be broadly categorized into two types: offline evaluation based on pre- collected human interaction trajectories and online evalua- tion based on system-level emulators. Current offline evaluation methods perform step-by-step evaluation by comparing the predicted actions against ref- erence action trajectories. AITW(Rawles et al., 2023) introduced large-scale data for training and evaluation. AITZ(Zhang et al., 2024c) refined AITWâs data, resulting in a more concise dataset. ANDROIDCONTROL(Leung et al., 2025) and AMEX(Chai et al., 2025a) focused on single- step action accuracy via element or coordinate matching. Mobile-Bench-v2(Xu et al., 2025a) covers a wide range of real-world tasks by collecting multiple possible trajectories and assessing them individually, which fails to support effec- tive interaction. ColorBench(Song et al., 2025) constructs the complete state transition space of an application, leading to an explosion of the state space. Although existing offline evaluation methods(Lee et al., 2024; Chai et al., 2025b) can cover arbitrary scenarios and conveniently provide valid evaluation signals, they still fail to effectively encompass diverse dynamic interactions. Several researchers have proposed online evaluation systems that enable interaction with agents via Android emulators and extract evaluation signals from system-level resources state. Mobile-Env(Zhang et al., 2024b) covers only 74 tasks, which limits its diversity. AndroidArena(Xing et al., 2024) and AndroidWorld(Rawles et al., 2024) expand the range of tasks; they are confined to built-in system applications whose designs often differ significantly from mainstream apps. AndroidLab (Xu et al., 2025b) is similarly constrained by its reliance on specific applications. Other works, such as SPA-Bench(Chen et al., 2024), AndroidDaily(Lee et al., 2025) and MobileWorld(Kong et al., 2025), still suffer from several limitations, including unstable environments and highly limited evaluation signals. To address the dilemma that online benchmarks fail to cover the full range of applications, while offline benchmarks fail to capture all task completion trajectories, we aim to propose an evaluation framework that ensures coverage of diverse interactions while encompassing arbitrary applications and achieving high alignment with human evaluation. 3. Formulation We provide a formal description of how a Mobile agent accomplishes a given task. LetGdenote the set of all tasks, and consider a specific taskg â G. We model the mobile device together with the application software as an observable finite-state machine: M g = (S,A,O,T ,R),(1) whereSis a finite set of states andAis a finite set of actions 2 . Each actionaâAcorresponds to a basic UI oper- ation, such as clicking, swiping, or text input.Odenotes the observation space, where the observationoâOperceived by the agent is determined by the current statesâS. The state transition functionT :S ĂAâSis assumed to be deterministic(the UI transition structure of APPs remains fixed, even though its content may vary randomly), andR represents the environment reward function. â˘A statistical analysis that motivates the task-oriented environment formulation is presented later in this pa- per. â˘The transition functionTinduces a directed graphG = (V,E ), where an edgee = (s,s Ⲡ) â Eexists if and only if there is an actionaâAsuch thatT (s,a) = s Ⲡ. Each edge is labeled with the corresponding action a. We define a Mobile agent as a decision-making entity equipped with internal reasoning and memory mechanisms: AG = (ÎŁ, Î ,H),(2) whereÎŁdenotes the reasoning space, updated according to Ď t âź ÎŁ(¡ | h tâ1 ,o t );Î denotes the decision space, from which actions are sampled viaa t âź Î (¡ | h tâ1 ,Ď t ,o t ); andHdenotes the memory space, which is updated as h t = h tâ1 âŞo t ,Ď t ,a t . â˘In practice, the reasoning and decision spaces are often integrated into a single module in many models; we present them separately here for conceptual clarity. â˘Depending on the implementation, the memory up- date process may not explicitly store all compo- nentsâobservation, reasoning state, and action. 2 Details of the action space are provided in Appendix B. 3 MobiFlow: Real-World Mobile Agent Benchmarking through Trajectory Fusion Benchmark#APP#TaskInteractiveReal AppNo DependenciesMulti-SolutionEvaluation Mobile-Bench-v2(Xu et al., 2025a)4912,856ââTrajectory-based AndroidControl(Leung et al., 2025)-14,548âââTrajectory-based MobileAgentBench(Wang et al., 2024)10010âââResult-based AndroidWorld(Rawles et al., 2024)20116âââResult-based AndroidLab(Xu et al., 2025a)9138âââResult-based SPA-Bench (Chen et al., 2024)66340âââResult-based MobiFlow20240âGraph-based Table 1. Comparison of different datasets and environments for benchmarking Mobile GUI agents. Column definitions: #Task. (number of tasks), #Apps (number of applications), Interactive (Support environmental interaction), Real App (Including any third-party apps), No Dependencies (Does not need additional software dependencies, such as an Android emulator), Multi-Solution (supports solving tasks in multiple ways), Evaluation (Evaluation Strategy) 4. MobiFlow To capture state transitions across diverse tasks, accurately evaluate agent performance under varied scenarios, and enable in-depth attribution analysis of agent capabilities, we propose the MobiFlow framework. This section is orga- nized into three parts, detailing our innovations in evaluation environment construction, metric design, and specialized scenario development, respectively. 4.1. Task Graph Construction Observations. Constructing graphs via exhaustive search leads to state space explosion, while directly building com- plete graphs incurs extremely high construction complex- ity and human labor costs. Through an analysis of the top 30 mainstream apps 3 , we find that the number of inde- pendent interactive elements on an app interface approxi- mately follows a normal distribution with a mean of 55.8(see fig3). In contrast, for a single task, the number of task- relevant elements approximately follows a Gumbel extreme value distribution, with a mean of only 1.7. This indicates that task-oriented graph construction yields a state space complexity(O(1.7 d ), wheredis action depth) that is far smaller than that of search-based approaches(O(55.8 d )). Moreover, we observe that most trajectories converge to states with similar transition structures after only a few steps. 55.831.74 Figure 3. Comparison of the Distribution of Interactive Ele- ment Counts Across Different Modeling Approaches. The left figure presents the app-based modeling approach, while the right figure illustrates the task-based modeling approach. 3 Static details in AppendixD Correctness and Completeness. Considering the require- ments of industrial deployment and the characteristics of real-world applications, we aim to construct a Task- Oriented Complete State-Transition Graph (TCSG), which includes all correct, task-relevant state transitions while excluding irrelevant and redundant states. In real- world deployments, task success rates are typically required to exceed 95%, and triggering incorrect, task-irrelevant tran- sitions is undesirable, as additional action paths can reduce both execution efficiency and success rates. Accordingly, we focus on whether an agent can sample correct actions to complete the task. For incorrect actions, we handle them by keeping the interface unchanged, transitioning to a blank screen, or presenting task-specific UI prompt interfaces. Trajectory Fusion. We observe that different trajectories tend to converge to states with highly similar transition struc- tures after very few steps, indicating the presence of many reusable state transitions. This reuse can further reduce the complexity of the state space. For a single task, we collect all directly completed trajectories and assign consis- tent labels to states that share identical transition structures across these trajectories. States with the same labels are then merged to share transition relations, thereby enabling low-complexity TCSG construction (as shown in Figure 4). For cross-application tasks, we can likewise construct via trajectory merging or multi-graph connectivity methods. S_1_1S_1_2 S_1_3 S_2_1 S_2_2S_2_3 S_2_4 S_2_5 S_3_1S_3_2 S_3_3 S_3_4 S_1_2 S_2_3 Tag1Tag2 Tag3 Tag4 Tag1 Tag2 Tag3 Tag4 Finite-State Transition GraphMultiple Operation Trajectory Sequences T1 T2 T3 Figure 4. Algorithm workflow illustration: For states with identi- cal labels (or equivalently, similar transition structures), we merge them and allow the merged state to share their transition conditions, thereby compressing the state complexity. Efficient Graph Construction. The construction of the TCSG can be mainly divided into three stages. First, tra- jectories are traversed to assign labels to states. Second, 4 MobiFlow: Real-World Mobile Agent Benchmarking through Trajectory Fusion chain-structured transitions are initialized. Finally, the tran- sition spaces of states sharing the same labels are merged via union coverage 4 . We achieve fast localization of target UI elements by parsing XML contents. More efficient label- ing is enabled through UI structural similarity matching and visual model inference. We employ efficient node represen- tations and maintain a union-find structure to support fast merging of transition spaces. With the techniques described above, the time cost of practical data collection and graph construction can be substantially reduced. For example, a task that involves posting a comment on Weibo typically requires around 15 interaction steps to complete. By col- lecting 7 trajectories, we can cover all valid task completion paths, and the entire construction process, including manual verification, can be finished in under 10 minutes. 4.2. Comprehensive Evaluation Based on the aforementioned construction algorithm, a TCSG encompassing all reasonable behavioral trajectories can be rapidly generated for a given task. As the graph is constructed from a global perspective, it enables a more com- prehensive evaluation of model behavior. In contrast, most previous evaluation methods, constrained by paradigms that rely solely on outcome signals or single-trajectory analysis, primarily focus on task success rate (SR)(Xie et al., 2024). This metric considers only task outcomes and ignores the execution process, which limits its ability to fully assess the modelâs capabilities. To more accurately evaluate task completion on the TCSG, we introduce four complementary metrics. Completion Rate (CR) measures the agentâs progress toward the goal state, while Coverage Rate (CVR) quantifies the proportion of explored states, reflecting the agentâs exploratory capabil- ity. Action Match Rate (AMR) assesses the alignment be- tween the agentâs executed actions and the instructed ac- tions, serving as an indicator of instruction compliance and safety. Time to Action (TTA) evaluates execution efficiency. Together, these metrics form a comprehensive evaluation framework spanning four dimensions: task progression, state exploration, instruction compliance, and execution efficiency. A detailed comparison and summary of these metrics is provided in Table2. 4.3. Diverse Scenario In this section, we introduce a variety of evaluation scenarios constructed based on task-state transition graphs.(see fig 5) These scenarios are designed to assess model performance from multiple perspectives, including practicality, safety, robustness, and exploratory capability. Base Scenario: Based on construction algorithms, we con- structed 160 distinct tasks spanning 20 mainstream applica- 4 The construction algorithm for the transition graph is provided in AppendixC tions. To ensure these tasks reflect real-world usage scenar- ios, we selected tasks across multiple domains, including common activities like information search, product purchas- ing, ride-hailing navigation, social chatting, and application settings. Furthermore, we resampled task descriptions to en- sure comprehensive alignment with real-world applications. This framework enables a thorough evaluation of agentsâ capabilities for handling common everyday tasks. Special Scenario 1 Instruction Following: The agent should be able to understand and adhere to human instruc- tions effectively. In this scenario, specific execution steps will be provided in the instructions. The action matching rate(AMR) will be used to evaluate the agentâs ability to follow instructions and its understanding and memory of complex, lengthy instructions. Additionally, this scenario can also assess the agentâs safety performance. Instruction Following Instruction Noisy Open Exploration 1.Click âSearchâ 2.Input âteeâ 3.Click âXYSJâ ... Hy, caan u p|z help mi ord3r a cup of milk -& tee ?lol ď§ď ###thx!! ď ď ď APP Interference Figure 5. Special scenarios including instruction following, in- struction interference, application interference, and open explo- ration (to observe whether the model can deviate from erroneous paths). Special Scenario 2 Instruction Noise Interference: In daily communication, errors such as typos or the use of emojis and special characters may occur. The agent should be able to identify the true task intent within such noisy instructions and correctly complete the task. In this sce- nario, noise will be introduced into task instructions through methods such as common typo substitutions, emoji injec- tions, and multilingual mixing. This aims to evaluate the modelâs resistance to interference and the robustness of its instruction comprehension. Special Scenario 3 APP Interference: In real-world appli- 5 MobiFlow: Real-World Mobile Agent Benchmarking through Trajectory Fusion MetricFormulaNotation ExplanationDescription Success Rate (SR) 1 N P N i=1 suc(i)N : number of tasks;suc(i) = 1ifg i is com- pleted, else 0. Indicates whether a task is fully completed, focusing only on the final outcome. Completion Rate (CR) max n d(s visited ,s goal ) d(s start ,s goal ) o s start ,s goal : initial and goal states;d(¡,¡): shortest-path distance. Measures the maximum progress achieved to- ward the goal state. Coverage Rate (CVR) P i |S visited | i P i |S| i |S visited | i : visited states in taski;|S| i : total states. Evaluates exploration by the proportion of states observed. Action Match Rate(AMR) 1 N P i P j am[i][j] |IF i | |IF i |: number of instructions;am[i][j] = 1if instruction j is satisfied. Measures instruction-following ability and safety. Time to Ac- tion (TTA) 1 N P i P j t ij |Ď i | |Ď i | : number of interactions;t ij : time to gener- ate action j. Captures execution efficiency via average ac- tion latency. Table 2. Evaluation metrics defined on the state-transition graph. Baseline Scenario: 160 Tasks across 20ApplicationsSpecial Scenario Figure 6. Task statistics for MobiFlow, covering the type, complexity, and application domain of basic-scenario tasks, as well as the quantity of special-scenario tasks. cations, software often faces various interferences, such as pop-up notifications from other apps, ad pushes, unintended touches, or interface jumps caused by system anomalies. The agent should respond safely when encountering such abnormal situations. Therefore, instances containing these interferences are collected separately to evaluate the modelâs ability to cope with specific APP disturbances. Special Scenario 4 Open Exploration: A knowledgeable agent should possess exploration and autonomous learning capabilities. To this end, we artificially constructed a set of special transition graphs. For example, based on basic scenarios, we introduce erroneous paths that do not satisfy task requirements, allowing us to observe whether the model can detect issues within these incorrect paths and reasonably backtrack to accomplish the designated task. 5. Experiments This section analyzes the evaluation results of several main- stream general and specialized models on our evaluation benchmark(see Table3). 5.1. Experimental Setup We evaluated a series of advanced models, including general- purpose models and GUI-specific models: OpenAI GPT-5, Claude-opus-4.1, Grok4, Gemini-2.5-flash, Gemini-2.5-Pro, Gemini-3-flash(OpenAI, 2025; Anthropic, 2025; Comanici et al., 2025), UI-TARS-1.5(Qin et al., 2025), AutoGLM- Phone(Liu et al., 2024), GUI-Owl(Ye et al., 2025b), and MobiMind(Zhang et al., 2025). To ensure a consistent com- parison, we adopted the same execution framework for gen- eral models, incorporating OminiParser(Lu et al., 2024) to annotate actionable UI elements to reduce the complexity of coordinate-based interface operations 5 . For GUI-specific agents, we employed their official execution frameworks. The maximum number of interaction steps allowed was set to 50. Additionally, the screen resolutions of the evaluation data were primarily 1080Ă2400 and 1200Ă2670. For specialized models, we standardized deployment and resource allocation, conducting inference deployment on NVIDIA A100-SXM4-80GB GPUs using the vLLM(Kwon et al., 2023) inference framework. For general models, we uniformly utilized the OpenRouter API for requests, with the temperature parameter consistently set to 0. 5.2. Main Results Task Completion. In the base scenario and the APP in- terference scenario, GUI agents achieve superior perfor- mance. Specifically, GUI-OWL attains the highest com- pletion rate (CR) of 60.4 in the base scenario, while UI- 5 Detailed implementation can be found in the AppendixF 6 MobiFlow: Real-World Mobile Agent Benchmarking through Trajectory Fusion Table 3. Main Results on MobiFlow.Includes general-purpose and specialized models. TTA (Time to Action), SR (Success Rate), CR (Completion Rate), AMR (Action Matching Rate), CVR (Coverage Rate). CR1 (with instruction noise), CR2 (under APP interference). Base ScenarioFollowingNoisy ScenarioOpen Exploration ModelSize(B)TTA(s)â SR(%)âCR(%)âCVR(%)AMR(%)âSR(%)âCR1(%)âCR2(%)âCR(%)âCVR(%) General Model Gemini-2.5-Flash-17.0330.741.717.435.533.335.537.753.221.1 Gemini-2.5-Pro-30.1238.752.126.955.341.656.451.759.728.1 Gemini-3.0-Flash-33.3140.259.730.256.441.664.758.46534.3 Grok4-142.21 8.312.38.414.18.38.110.017.810.2 Claude-opus-4.1-46.6843.660.334.662.258.368.670.157.937.7 GPT-5-55.4949.762.735.371.459.371.146.770.039.3 GUI Model AutoGLM-Phone9B3.8516.730.210.630.524.224.222.520.115.1 MobiMind-Mixed4B1.7653.872.931.959.942.355.158.358.635.1 UI-TARS-1.58B1.7560.476.334.463.150.063.174.565.737.1 GUI-Owl8B20.0255.777.937.565.441.664.767.467.138.1 TARS achieves the highest CR of 74.5 under APP inter- ference. However, specialized GUI models do not exhibit a comprehensive advantage over general-purpose models. In instruction-following, instruction-noise interference, and open exploration scenarios, general-purpose models (e.g., OpenAI GPT-5) demonstrate stronger capabilities. Execution Efficiency. We sampled tasks of varying com- plexity and analyzed the execution efficiency of agent in- teractions, as shown in Figure 7. The factors that affect single-action latency include network latency, model size, the number of decoding tokens, and the number of model invocations per action. Under the same model size, fewer de- coding tokens generally lead to higher efficiency. For some models, such as GUI-OWL, a single action requires multiple model calls, which degrades action execution efficiency. In contrast, MobiMind and UI-TARS achieve efficiency ad- vantages due to their lightweight architectures, single-call execution per action, and limited decoding outputs. Gen- eral models, on the other hand, suffer from higher action latency because of greater network latency, larger model sizes, and decoding outputs that vary substantially with task difficulty 6 . Performance Fluctuations. We observed that the evalu- ation results of general models exhibit greater fluctuation than those of GUI models. Furthermore, while the exe- cution paths of GUI models remain relatively consistent across multiple sampling attempts for a given task, general models demonstrate diverse paths but unstable completion outcomes, characterized by substantial variance. 7 Trajectory Coverage. Across multiple model evaluations, coverage and completion rates consistently show a positive correlation. Models with higher coverage also demonstrate better performance in open exploration scenarios. Resolution Scaling. We analyze the impact of input image 6 All experiments followed the settings in the section5.1. 7 Experiment details are provided in the AppendixG. resolution on model performance 8 . For images with a reso- lution of 1080Ă2400, the modelâs task completion rate drops significantly when the resolution is scaled below 0.2Ă. Figure 7. Model execution efficiency. GUI models use the same inference framework and operate under identical network request environments. 5.3. In-Depth Analysis We observe that general models not only exhibit greater ro- bustness against instruction noise interference compared to specialized GUI models, but also achieve a higher comple- tion rate in instruction-following capability under such con- ditions. As shown in the table 4, one possible explanation is that the presence of specific interference may heighten the modelâs attention to the task-relevant portion of the long- context instructions. General models exhibit strong generalization capabilities and outperform GUI models in exploratory tasks. However, lightweight specialized models support on-device deploy- ment, while their stable accuracy in completing a large volume of tasks further facilitates practical deployment and application in real-world scenarios. Across different scenarios, we analyze the performance 8 Details are provided in the AppendixH 7 MobiFlow: Real-World Mobile Agent Benchmarking through Trajectory Fusion Model No Instruction Noise AMR Instruction Noise AMR General Model Gemini-2.5-Flash35.5538.55+3.0 Gemini-2.5-Pro55.3456.48+1.1 Claude-sonnet-4.566.3867.38+1.0 GPT-571.3871.07-0.3 GUI specialized Model MobiMind-Mixed59.8955.11-4.7 UI-TARS-1.563.1263.12-0.0 GUI-OWL65.3764.72-0.4 Table 4. Robustness to Instruction Noise: A Comparison of General Models vs. GUI Models Figure 8. Correlation Analysis. Correlation between General Modelsâ Performance in Specific Scenarios and Different Bench- mark Capabilities. scores of general models and compare them with established benchmarks(Text&VisionArena 9 , MultiNRC(Fabbri et al., 2025), MultiChallenge(He et al., 2024), and VISTA(Zheng et al., 2023) 10 ) to investigate which specific capabilities are reflected by each scenario. Our correlation analysis results are illustrated in the figure 8. We find that each scenario exhibits relatively high cor- relations with capabilities in visual input understanding and cross-cultural contextual comprehension. In open ex- ploration and instruction-following scenarios, the correla- tion with multi-turn dialogue capability is particularly pro- nounced. Under app interference conditions, a certain nega- tive correlation with logical reasoning capability is observed. Additionally, the open exploration scenario shows a particu- larly strong correlation with causal reasoning ability. 5.4. Special issues Thinking and observation become desynchronized. Dur- ing the experiments, we observed that GUI models occa- 9 https://lmarena.ai 10 https://scale.com/leaderboard sionally perform erroneous actions without triggering the corresponding page transition. Yet, the models proceed un- der the assumption that the transition has been completed, continuing to operate as if they were in the post-transition state. Insufficient Generalization of Reasoning. We observe that some models tend to operate based on fixed reasoning patterns. For instance, when encountering unconventional tasks, most models default to searching for task-related key- words, occasionally overlooking the information inherently available on the page itself. Interface Comprehension Deviation. We have observed that models can sometimes be misled by interface informa- tion. For instance, in search scenarios, actual application recommendation algorithms often display suggested content in light colors; however, the model frequently misinterprets this as user-input text and attempts to clear it. 5.5. Trade-offs Balancing Agent Execution and System Boundaries. Granting the agent greater autonomy over certain system operations can expand its action space and enhance overall capability. For instance, combining clicking and text-input actions can prevent the issue of repeated input field activa- tion in search scenarios. However, such delegation must be carefully managed to ensure system security and stability. Balancing Generalization and Reliability. Strengthening an agentâs generalization ability leads to more diverse ex- ecution trajectories, which enriches behavioral flexibility. Yet, this often comes at the cost of reduced determinism and reliability in task completion. In real-world deployment, a deliberate and nuanced balance must be struck to maintain both adaptability and consistent performance. 6. Discussion Cross-APP Tasks. We support cross-application tasks via trajectory merging or the multi-graph connectivity method. In this paper, we focus on evaluating the intrinsic capabili- ties of GUI models, and therefore, only single-task results are reported. Reproduction. Our state graph is configured with determin- istic transitions to ensure reproducible and accurate evalua- tion of third-party applications. MCP Support. Since this paper focuses more on the evalu- ation of GUI operations for agents, support for the Model Context Protocol (MCP)(Yan et al., 2025) will be addressed in future work. 8 MobiFlow: Real-World Mobile Agent Benchmarking through Trajectory Fusion 7. Conclusion In summary, we introduce MobiFlow, a mobile-use bench- mark built on arbitrary third-party APPs. By constructing state transition graphs, MobiFlow effectively compresses the state space of multiple task-completion trajectories. Com- pared with existing benchmarks, MobiFlow more faithfully reflects the practical capabilities of agents. We equip the benchmark with a diverse set of evaluation metrics and spe- cialized evaluation scenarios to enable in-depth analysis of the modelâs specific capabilities. We analyze the per- formance of current mainstream models and identify the challenges they face. Our benchmark provides a critical so- lution for gaining a deeper understanding of the real-world capabilities of mobile agents and facilitating their deploy- ment in practical scenarios. Impact Statement This work facilitates the real-world deployment of mobile agents, which has been adopted in industry. The trajectory- fusion-based assessment approach is also applicable to the evaluation of other types of agents. This work will con- tribute to enhancing the practicality of agents in accomplish- ing various GUI tasks. References Anthropic.Claude opus 4.1.Large language model, https://w.anthropic.com, 2025. Accessed: [Insert Full Date You Accessed the Model, e.g., 6 Jan. 2026]. Bu, W., Wu, Y., Yu, Q., Gao, M., Miao, B., Zhang, Z., Pan, K., Li, Y., Li, M., Ji, W., et al. What limits virtual agent application? omnibench: A scalable multi-dimensional benchmark for essential virtual agent capabilities. arXiv preprint arXiv:2506.08933, 2025. Chai, Y., Huang, S., Niu, Y., Xiao, H., Liu, L., Wang, G., Zhang, D., Ren, S., and Li, H. AMEX: Android multi- annotation expo dataset for mobile GUI agents. In Che, W., Nabende, J., Shutova, E., and Pilehvar, M. T. (eds.), Findings of the Association for Computational Linguis- tics: ACL 2025, p. 2138â2156, Vienna, Austria, July 2025a. Association for Computational Linguistics. ISBN 979-8-89176-256-5. doi: 10.18653/v1/2025.findings-acl. 110. URLhttps://aclanthology.org/2025. findings-acl.110/. Chai, Y., Li, H., Zhang, J., Liu, L., Liu, G., Wang, G., Ren, S., Huang, S., and Li, H. A3: Android agent arena for mobile gui agents. arXiv preprint arXiv:2501.01149, 2025b. Chen, J., Yuen, D., Xie, B., Yang, Y., Chen, G., Wu, Z., Yixing, L., Zhou, X., Liu, W., Wang, S., et al. Spa-bench: A comprehensive benchmark for smartphone agent evalu- ation. In NeurIPS 2024 Workshop on Open-World Agents, 2024. Comanici, G., Bieber, E., Schaekermann, M., Pasupat, I., Sachdeva, N., Dhillon, I., Blistein, M., Ram, O., Zhang, D., Rosen, E., et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arXiv preprint arXiv:2507.06261, 2025. Fabbri, A. R., Mares, D., Flores, J., Mankikar, M., Her- nandez, E., Lee, D., Liu, B., and Xing, C. Multinrc: A challenging and native multilingual reasoning evaluation benchmark for llms. arXiv preprint arXiv:2507.17476, 2025. He, Y., Jin, D., Wang, C., Bi, C., Mandyam, K., Zhang, H., Zhu, C., Li, N., Xu, T., Lv, H., et al. Multi-if: Bench- marking llms on multi-turn and multilingual instructions following. arXiv preprint arXiv:2410.15553, 2024. Kong, Q., Zhang, X., Yang, Z., Gao, N., Liu, C., Tong, P., Cai, C., Zhou, H., Zhang, J., Chen, L., et al. Mobileworld: Benchmarking autonomous mobile agents in agent-user interactive, and mcp-augmented environments. arXiv preprint arXiv:2512.19432, 2025. Kwon, W., Li, Z., Zhuang, S., Sheng, Y., Zheng, L., Yu, C. H., Gonzalez, J. E., Zhang, H., and Stoica, I. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles, 2023. Lee, J., Min, T., An, M., Hahm, D., Lee, H., Kim, C., and Lee, K.Benchmarking mobile device control agents across diverse configurations. arXiv preprint arXiv:2404.16660, 2024. Lee, J., Min, T., An, M., Hahm, D., Lee, H., Kim, C., and Lee, K. Benchmarking mobile device control agents across diverse configurations, 2025. URLhttps:// arxiv.org/abs/2404.16660. Leung, H. F., Xi, X., and Zuo, F. Androidcontrol-curated: Revealing the true potential of gui agents through bench- mark purification. arXiv preprint arXiv:2510.18488, 2025. Liu, X., Qin, B., Liang, D., Dong, G., Lai, H., Zhang, H., Zhao, H., Iong, I. L., Sun, J., Wang, J., et al. Autoglm: Autonomous foundation agents for guis. arXiv preprint arXiv:2411.00820, 2024. Lu, Y., Yang, J., Shen, Y., and Awadallah, A. Omniparser for pure vision based gui agent, 2024. URLhttps: //arxiv.org/abs/2408.00203. 9 MobiFlow: Real-World Mobile Agent Benchmarking through Trajectory Fusion Ma, X., Zhang, Z., and Zhao, H. Coco-agent: A comprehen- sive cognitive mllm agent for smartphone gui automation. arXiv preprint arXiv:2402.11941, 2024. OpenAI. Conversation with an ai model [generative ai chat]. Available at:chat.openai.com, 2025. Accessed: [Date you accessed the tool]. Qin, Y., Ye, Y., Fang, J., Wang, H., Liang, S., Tian, S., Zhang, J., Li, J., Li, Y., Huang, S., et al. Ui-tars: Pioneer- ing automated gui interaction with native agents. arXiv preprint arXiv:2501.12326, 2025. Rawles, C., Li, A., Rodriguez, D., Riva, O., and Lillicrap, T. Androidinthewild: A large-scale dataset for android de- vice control. Advances in Neural Information Processing Systems, 36:59708â59728, 2023. Rawles, C., Clinckemaillie, S., Chang, Y., Waltz, J., Lau, G., Fair, M., Li, A., Bishop, W., Li, W., Campbell- Ajala, F., et al. Androidworld: A dynamic benchmark- ing environment for autonomous agents. arXiv preprint arXiv:2405.14573, 2024. Song, Y., Huang, H., Lin, Q., Zhao, Y., Qu, X., Wang, J., Lou, X., Liu, W., Zhang, Z., Yu, Y., et al. Col- orbench: Benchmarking mobile agents with graph- structured framework for complex long-horizon tasks. arXiv preprint arXiv:2510.14621, 2025. Toyama, D., Hamel, P., Gergely, A., Comanici, G., Glaese, A., Ahmed, Z., Jackson, T., Mourad, S., and Precup, D. Androidenv: A reinforcement learning platform for android. arXiv preprint arXiv:2105.13231, 2021. Wang, H., Zou, H., Song, H., Feng, J., Fang, J., Lu, J., Liu, L., Luo, Q., Liang, S., Huang, S., et al. Ui-tars- 2 technical report: Advancing gui agent with multi-turn reinforcement learning. arXiv preprint arXiv:2509.02544, 2025. Wang, L., Deng, Y., Zha, Y., Mao, G., Wang, Q., Min, T., Chen, W., and Chen, S. Mobileagentbench: An efficient and user-friendly benchmark for mobile llm agents. arXiv preprint arXiv:2406.08184, 2024. Xie, T., Zhang, D., Chen, J., Li, X., Zhao, S., Cao, R., Hua, T. J., Cheng, Z., Shin, D., Lei, F., et al. Osworld: Bench- marking multimodal agents for open-ended tasks in real computer environments. Advances in Neural Information Processing Systems, 37:52040â52094, 2024. Xing, M., Zhang, R., Xue, H., Chen, Q., Yang, F., and Xiao, Z. Understanding the weakness of large language model agents within a complex android environment. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, p. 6061â6072, 2024. Xu, W., Jiang, Z., Liu, Y., Gao, P., Liu, W., Luan, J., Li, Y., Liu, Y., Wang, B., and An, B. Mobile-bench-v2: A more realistic and comprehensive benchmark for vlm-based mobile agents. arXiv preprint arXiv:2505.11891, 2025a. Xu, Y., Liu, X., Sun, X., Cheng, S., Yu, H., Lai, H., Zhang, S., Zhang, D., Tang, J., and Dong, Y. Androidlab: Train- ing and systematic benchmarking of android autonomous agents. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 2144â2166, 2025b. Yan, Y., Wang, S., Du, J., Yang, Y., Shan, Y., Qiu, Q., Jia, X., Wang, X., Yuan, X., Han, X., et al. Mcpworld: A unified benchmarking testbed for api, gui, and hybrid computer use agents. arXiv preprint arXiv:2506.07672, 2025. Ye, J., Zhang, X., Xu, H., Liu, H., Wang, J., Zhu, Z., Zheng, Z., Gao, F., Cao, J., Lu, Z., Liao, J., Zheng, Q., Huang, F., Zhou, J., and Yan, M. Mobile-agent-v3: Fundamental agents for gui automation, 2025a. URL https://arxiv.org/abs/2508.15144. Ye, J., Zhang, X., Xu, H., Liu, H., Wang, J., Zhu, Z., Zheng, Z., Gao, F., Cao, J., Lu, Z., et al. Mobile-agent-v3: Fundamental agents for gui automation. arXiv preprint arXiv:2508.15144, 2025b. Zhang, C., He, S., Qian, J., Li, B., Li, L., Qin, S., Kang, Y., Ma, M., Liu, G., Lin, Q., et al. Large language model-brained gui agents: A survey. arXiv preprint arXiv:2411.18279, 2024a. Zhang, C., Feng, E., Zhao, X., Zhao, Y., Gong, W., Sun, J., Du, D., Hua, Z., Xia, Y., and Chen, H. Mobiagent: A systematic framework for customizable mobile agents. arXiv preprint arXiv:2509.00531, 2025. Zhang, D., Xu, H., Zhao, Z., Chen, L., Cao, R., and Yu, K. Mobile-env: an evaluation platform and benchmark for llm-gui interaction. arXiv preprint arXiv:2305.08144, 2024b. Zhang, J., Wu, J., Yihua, T., Liao, M., Xu, N., Xiao, X., Wei, Z., and Tang, D.Android in the zoo: Chain-of-action-thought for GUI agents. In Al- Onaizan, Y., Bansal, M., and Chen, Y.-N. (eds.), Find- ings of the Association for Computational Linguis- tics: EMNLP 2024, p. 12016â12031, Miami, Florida, USA, November 2024c. Association for Computational Linguistics.doi: 10.18653/v1/2024.findings-emnlp. 702. URLhttps://aclanthology.org/2024. findings-emnlp.702/. Zheng, L., Chiang, W.-L., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z., Li, Z., Li, D., Xing, E., et al. Judging 10 MobiFlow: Real-World Mobile Agent Benchmarking through Trajectory Fusion llm-as-a-judge with mt-bench and chatbot arena. Ad- vances in neural information processing systems, 36: 46595â46623, 2023. 11 MobiFlow: Real-World Mobile Agent Benchmarking through Trajectory Fusion A. Real-world Trajectory Collection To ensure the collected trajectories accurately reflect real-world user behavior, we have developed a lightweight action recording tool for smartphones(see fig9). During data collection, annotators or agents interact with this tool, which captures and logs every user operation before forwarding it to the device. The tool renders bounding boxes for all interactive on-screen elements based on XML files that describe UI hierarchies. If a bounding box is missing due to incomplete XML data, OmniParser is employed to regenerate the corresponding bounding box. The resulting trajectories can subsequently be used for graph construction, evaluation, and other follow-up steps. Within the graph construction algorithm, the labeling of observed pages can be performed concurrently. Following predefined labeling rules (which may be specified based on the transition structure of observed interfaces), multiple trajectories are merged into a unified state transition graph. Furthermore, to extend the state transition graph, it suffices to append new trajectories according to the same labeling principles, thereby ensuring strong extensibility. Human Operator Agent CollectEnv Mobile Devices VLM Omni Parser Vision Annotator Multiple Trajectories Figure 9. Efficient trajectory collection tools. A trajectory collection framework based on real devices and actual applications. B. Action Space This section presents the action space supported by our environment, which includes operations such as click, swipe, wait, text input, back, and more. 12 MobiFlow: Real-World Mobile Agent Benchmarking through Trajectory Fusion ActionParameterDescription GUI Operation clickx,yClick at the specified coordinates inputstringType text into the focused field swipestartx,starty,endx,endySwipe from start to end coordinates waitâWait for update doubleclickx,yDouble Click at the specified coordinates long pressx,yLong press at the specified coordinates System Operation backâNavigate to the previous screen homeâNavigate to the Home screen Task Control Operation doneâTask done failed âTask failed Table 5. Action types and their parameters in the task execution framework. C. Task Graph Construction Details This section introduces the details of the construction. The task graph primarily consists of two elements: node construction and edge construction. For node construction, we provide the following data structure, which describes the attributes of a state, including observation information (StateInfo), labeling properties (Tag), and state transition mappings(TransitionSpace). @dataclass class State: â Task Transition Graph state â State_Info # screenshots or other (e.g., text or XML) Tag # State tagging # Action-space-based interface transition hashing Transition_Space = "click": # Box[x1,y1,x2,y2] --> next state , "swipe": # (Direction,Distance) --> next state , "input": # Text --> next state , "wait": # Duration --> next state , "back": # --> prev state , "home": # --> home state , "long Press": # Box[x1,y1,x2,y2] --> next state # extend... For edge construction, we first traverse the raw trajectories to assign state-specific labels and record all observed action transitions, thereby populating the transition space of each abstract state. The labels can be manually defined based on the structural distribution of UI elements on the interface or inferred by a model; the key requirement is that states sharing the same label can reuse a common transition space. Specifically, each raw observation is mapped to an abstract state via semantic labeling or hashing, and directed edges are added according to the corresponding(u,a,v)transitions. Subsequently, nodes with identical semantic labels are merged by taking the union of their transition spaces, aggregating historical actions across equivalent states to maximize action coverage while reducing state fragmentation. 13 MobiFlow: Real-World Mobile Agent Benchmarking through Trajectory Fusion Algorithm 1 Offline Graph Construction via Trajectory Merging 1: Input: Set of raw trajectoriesD =Ď 1 ,...,Ď N , where Ď i =(o 1 ,a 1 ),..., (o T ,¡) 2: Output: State Transition GraphG = (V,E ) 3: // Core Logic Overview: 4: // 1. Abstraction: Map raw UI observations to nodes via Semantic Labels or Hashing. 5: // 2. Construction: Build directed edges based on recorded action transitions. 6: // 3. Unification: Merge action spaces of identical states to maximize coverage. 7: Initialize node setV ââ , edge mapE ââ 8: Define abstraction function ÎŚ(o)â s id 9: Phase 1: Trajectory Traversal & Topology Building 10: for each trajectory Ď âD do 11:for each transition step (o t ,a t ,o t+1 ) in Ď do 12: Step 1: State Abstraction 13:if o t has semantic label L then 14:uâ L 15:else 16:uâ HASH(o t ) 17:end if 18: v â ÎŚ(o t+1 ) 19: Step 2: Graph Update 20: V âV âŞu,v 21:if u /âE or a t /âE [u] then 22: E [u][a t ]â v 23:else 24: E [u][a t ]âE [u][a t ]âŞv 25:end if 26:end for 27: end for 28: Phase 2: Transition Space Unification 29: for each abstract state uâV do 30:if u is a labeled node then 31: T all â S sâÎŚ â1 (u) Transitions(s) 32: E [u]âE [u]âŞT all Aggregate all historical actions 33:end if 34: end for 35: returnG = (V,E ) D. Statistics on the Number of Interface Actions We employ OminiParserV2 to conduct a statistical analysis of icon elements corresponding to executable actions across vari- ous interface types in multiple mainstream applications. Duplicate elements are removed using non-maximum suppression, and element independence is ensured through intersection detection. The resulting statistics are presented in the upper part of Figure10. For multiple feasible trajectories that accomplish the same task, we further compute the average number of navigation actions per interface, with the results shown in the lower part of Figure10. 14 MobiFlow: Real-World Mobile Agent Benchmarking through Trajectory Fusion 1.7 55.8 Figure 10. This figure illustrates the statistical distribution of transition action counts on individual interfaces across different mobile applications(Due to space limitations, we present only a subset of the statistical results). 15 MobiFlow: Real-World Mobile Agent Benchmarking through Trajectory Fusion E. State transition graph visualization Based on our graph construction algorithm, which merges multiple trajectories into a task state transition graph, we randomly selected four tasks for visualization. The visualization results are presented below. Figure 11. Visualization Examples of Task State Transition Graphs, including Different Tasks from Ele.me, AutoNavi, and NetEase Cloud Music Apps. F. Execution Details of General Models System Prompt Role Definition You are a mobile phone operation AI assistant, tasked with helping the user complete the following task: "taskdescription". Input Description I will provide you with: 1. Action History: A record of all previous operations. 2. Screen Screenshot: A complete screenshot of the current phone screen. 3. layercountAnnotated Screenshots: Annotation layers of clickable elements generated based on the screen screenshot. To avoid element overlap, all clickable elements are distributed across different layers for display. The union of elements across all layers represents the complete set of clickable elements. ⢠Clickable elements are marked with red bounding boxes. â˘Each elementâs index is displayed inside the top-left corner of its red bounding box as a red-background white-digit number. Action Historyhistory 16 MobiFlow: Real-World Mobile Agent Benchmarking through Trajectory Fusion Task Requirements Please carefully analyze the current screen state and action history, then determine the most appropriate next action. Actions Available 1. Click (click) ⢠Parameters: index (integer, corresponding to the UI element index in the annotated screenshots) ⢠Parameters: targetelement (string, describing the UI element to be clicked) â˘Critical: You must carefully observe the annotated screenshots, find the red bounding box that best matches the description of thetargetelement, and use the number displayed inside the top-left corner of that box (red background, white digits) as the index. ⢠Prevent Mis-selection: The reasoning must explicitly explain why this specific red bounding box was chosen and not an adjacent one. 2. Swipe (swipe) ⢠Parameters: direction (string, must be one of: UP, DOWN, LEFT, RIGHT) â˘Critical: Direction Clarification: UP means swiping the finger upward to scroll content upward, revealing content below; DOWN means swiping downward to scroll content downward, revealing content above; LEFT means swiping left to scroll content left; RIGHT means swiping right to scroll content right. 3. double click ⢠Parameters: index (integer, corresponding to the UI element index in the annotated screenshots) 4. long press ⢠Parameters: index (integer, corresponding to the UI element index in the annotated screenshots) 5. Home ⢠No parameters, navigate to the Home screen. 6. Home ⢠No parameters, navigate to the Home screen. 7. Home ⢠No parameters, navigate to the Home screen. 8. Back (back) ⢠No parameters, indicates returning to the previous state. 9. Task Complete (done) ⢠No parameters, indicates the task is completed. Output Format Please output strictly in the following JSON format: "reasoning": "Provide a detailed explanation of your analysis and the reason for choosing this action. For click actions, it must include the following complete process: 1) Detailed description of the target element: including element content, color, shape, size, and other visual features. 2) Precise location of the target element: its specific position on the screen, using surrounding elements as reference. 3) Annotation map search process: state in which annotation map (layer 17 MobiFlow: Real-World Mobile Agent Benchmarking through Trajectory Fusion number) the matching red bounding box was found. 4) Red bounding box verification: confirm that the position, contained content, and boundaries of this red box perfectly match the target element. 5) Index reading confirmation: explicitly state the number found inside the top-left corner of the selected red bounding box. 6) Final confirmation: reiterate that this choice is correct and no adjacent element was mistakenly selected. For input actions, you must first explicitly state: 1) Whether a soft keyboard is currently displayed on the screen. 2) Whether the target input field is already activated (has a cursor or is highlighted). 3) If either of the above checks is negative, you MUST choose a click action first to activate the input field, NOT an input action.", "action": "action_name(click/swipe/input/back/done)", "parameters": "parameter_name": "parameter_value" Critical Steps for Index Selection (Mandatory Reading for Click Actions) Step 1: Precisely describe the visual features of the target element. ⢠Describe in detail the content, color, shape, and other visual features of the target element. ⢠Precisely describe the elementâs position on the screen (e.g., top third of the screen, left edge, bottom-right corner). ⢠Describe other UI elements surrounding the target element as reference points. â˘Example: âNeed to click the white input box with the text âSearchâ, located at the very top of the screen, just below the app title.â Step 2: Systematically search for the red bounding box. ⢠You MUST examine each annotation map sequentially, one by one. Do not skip any map. ⢠For each map, first observe the overall distribution of all red bounding boxes. â˘Focus on finding the red bounding box whose position and features perfectly match the description from Step 1. â˘Critical Requirement: The red bounding box must completely enclose the target element, with boundaries snugly fitting. Step 3: Multi-level verification to ensure correct selection (Most Important Step). ⢠Position Verification: Confirm the red bounding boxâs location matches the position described in Step 1 exactly. â˘Content Verification: Carefully observe whether the content inside the red bounding box is indeed the target element. ⢠Boundary Verification: The boundaries of the red bounding box should be snug against the target element, not containing excessive blank space. â˘Exclude Interference: If there are multiple similar red boxes, you MUST choose the one whose position matches most precisely. 18 MobiFlow: Real-World Mobile Agent Benchmarking through Trajectory Fusion ⢠Avoid Adjacent Selection: Absolutely DO NOT select a red box next to or near the target element. Step 4: Read the index number (Final confirmation before execution). ⢠Re-confirm that the selected red bounding box indeed encloses the correct target element. ⢠Look at the number inside the top-left corner of this red bounding box. ⢠Strict Requirement: Must be the top-left corner; the number must be clearly visible. ⢠This number is the index value to use. Step 5: Final Verification. ⢠In the reasoning, explicitly state: âThe red bounding box I selected is located at [specific position], contains [specific content], and its top-left number is [X].â ⢠If you have any uncertainty about the selection, you MUST restart from Step 1. Critical Steps for Text Input (Mandatory Reading for Input Actions) Important Prerequisite: Absolutely DO NOT use the input action when the input field is not activated! Step 1: Mandatory Check of Soft Keyboard Status (Must execute, cannot skip). â˘Must Check: Carefully observe if a soft keyboard (virtual keyboard interface) is currently displayed at the very bottom of the screen. â˘Judgment Criterion: If the bottom of the screen does NOT display a soft keyboard interface containing letter and number keys, it means no input field is activated. â˘Key Rule: You are only allowed to perform an input action when the soft keyboard is fully displayed at the bottom of the screen. ⢠Must State in Reasoning: âCheck soft keyboard status: [Displayed/Not Displayed].â Step 2: Activating the Input Field (Must execute if Step 1 check fails). ⢠Strictly Forbidden: If there is no soft keyboard or the input field is not activated, you absolutely CANNOT use the input action. ⢠Must Do: You MUST first use a click action on the target input field to activate it. â˘Must State in Reasoning: âSoft keyboard not displayed / Input field not activated. Must click to activate the input field first.â Step 3: Handling Existing Content. ⢠If the input field has default text, you may try to clear it or overwrite it directly. ⢠Choose the handling method based on the specific situation. Step 4: Execute Text Input (Only when preconditions are satisfied). ⢠After confirming the soft keyboard is displayed AND the input field is activated, you may use the input action. â˘After inputting, check that the text in the input field is correct, ensuring there are no input errors, omissions, or extra characters. ⢠In reasoning, you MUST explicitly state: âConfirmed soft keyboard is displayed and input field is activated.â Step 5: Handling Soft Keyboard After Input (Important). 19 MobiFlow: Real-World Mobile Agent Benchmarking through Trajectory Fusion ⢠Must Check After Input: Observe the key types on the soft keyboard. ⢠Criteria for Hiding Soft Keyboard: âIf the soft keyboard has action buttons like âSearchâ, âOKâ, âDoneâ, âSendâ, etc., you should click these buttons. âIf the soft keyboard only has navigation buttons like âNextâ or âEnterâ, AND the keyboard is blocking important UI elements, you should click the âdown arrowâ button (usually top-right) on the soft keyboard to hide it. ⢠Must State in Reasoning: âCheck soft keyboard key types: [Action type/Navigation type]. Is the soft keyboard blocking important elements: [Yes/No]. Decision: [Click action button / Hide soft keyboard / Keep as is].â Strictly Forbidden Input Action Patterns 1. Inputting directly without a soft keyboard. 2. Inputting directly without describing the check process in reasoning. 3. Inputting directly upon seeing an input field (must check activation status first). 4. Not considering soft keyboard obstruction after input. The Only Correct Input Action Pattern â˘Reasoning includes: âCheck soft keyboard status: Displayed. Check input field status: Activated. Confirmed text input is permissible.â ⢠Only reasoning containing this complete check process allows the use of the input action. â˘Post-Input Handling: After input, you must check soft keyboard key types and whether it blocks important elements to decide if it needs to be hidden. Important Rules 1.Position Match Priority: First determine the elementâs precise location in the original screenshot, then find the corresponding red bounding box in the annotation maps. 2.Accurate Number Reading: Theindexmust be the actual number displayed inside the top-left corner of the red bounding box (red background, white digits). 3.Avoid Mis-selecting Adjacent Elements: This is the most common mistake! Ensure the chosen red bounding box fully encloses the target element, not a nearby similar element. 4.Mandatory Adjacent Element Exclusion Check: Before selecting any index, you must explicitly explain why other red bounding boxes in the vicinity were NOT chosen. 5. Soft Keyboard Obstruction Handling: After input, if the soft keyboard is blocking important elements and there is no action button, click the top-right down arrow to hide it. 6.Multi-step Operations: For complex selections (like date ranges, time slots, cascading options), multiple consecutive actions are required. 7. Special Attention for Date Selection: ⢠On a date selection interface, you must first confirm if the currently displayed month is correct. ⢠Do not just click an identical date number; you must ensure the month matches the task requirement. ⢠If the month is wrong, you need to switch to the correct month first, then select the date. 20 MobiFlow: Real-World Mobile Agent Benchmarking through Trajectory Fusion 8. Task Completion Judgment: Use the âdoneâ action only when the specified task is indeed completed. 9.Operation Coherence: Each action should be a logical choice based on the current screen state and task goal. 10.Page Error Handling: If you encounter an incorrect page or a loading failure, you can try going back to the previous level (via a swipe gesture from the leftmost screen edge or by clicking a back button). Index Selection Examples Incorrect Example 1: ⢠reasoning: âNeed to click the search button.â ⢠Problem: No description of the elementâs specific location or visual features. Incorrect Example 2: ⢠reasoning: âNeed to click the search box, located at the top of the screen. Found the search box in the annotation map, choose number 8.â ⢠Problem: Description is too vague, lacks verification process, prone to selecting the wrong adjacent element. Correct Example: reasoning: "1) Target element detailed description: Need to click the white input box with the placeholder text âSearchâ, rectangular in shape, with a light grey border. 2) Precise location description: This search box is located at the very top of the screen, approximately 50 pixels below the status bar, occupies about 80% of the screen width, centered. 3) Annotation map search: In annotation map #2, I found a red bounding box at the central top position of the screen. 4) Red bounding box verification: This red box completely encloses the search input box, its boundaries perfectly align with the edges of the input box, and it indeed contains the white input box with the text âSearchâ. 5) Index reading: The top-left inner corner of this red box clearly shows the number â15â. 6) Final confirmation: Confirmed this box does not contain any irrelevant elements, nor is it an adjacent UI element; it is precisely the search box I intend to click." parameters: "index": 15, "target_element": "Search input box" The above constitutes the system prompt designed for task completion by general-purpose models. It defines a constrained action space, provides historical context, and includes both positive and negative examples for in-context learning. Addi- tionally, a style constraint is imposed to require the model to output its reasoning process. Since general-purpose models struggle with generating precise coordinates, we have integrated an auxiliary enhancement tool. This tool utilizes an icon recognition model to overlay bounding boxes and numerical labels directly on the interface image, enabling the model to select from these annotated elements, thereby enhancing its task completion capability. 21 MobiFlow: Real-World Mobile Agent Benchmarking through Trajectory Fusion Open XHS CLICK 52 Input ä¸ćľˇĺĽ˝çŠĺ°ćšć¨č TaskDone Search for ârecommended fun places in Shanghaiâ on Xiaohongshuand open the top-ranked post. Figure 12. Execution Example of General Models: General models leverage icon recognition models for enhancement. 22 MobiFlow: Real-World Mobile Agent Benchmarking through Trajectory Fusion G. Generalization vs. Accuracy General Model GUI Model Figure 13. The tension between generalization capability and determinism. The upper figure presents the performance scores of a general-purpose model across multiple sampling trials on randomly selected tasks. The lower figure shows the sampling results of a GUI-specialized model on the same set of tasks. We selected different general-purpose models (GPT-5, Gemini-2.5-flash) and specialized models (UI-TARS-1.5, MobiMInd), randomly chose multiple tasks, and conducted repeated evaluations under identical configurations. The results revealed that the Completion rate (CR) of general-purpose models exhibited significantly greater fluctuations than that of specialized models. The specialized models produced stable outcomes with consistent execution paths, whereas general-purpose models demonstrated more diverse approaches to task completion. 23 MobiFlow: Real-World Mobile Agent Benchmarking through Trajectory Fusion H. Resolution Scaling Figure 14. Completion rate of different models under varying resolution scaling factors. We randomly sample representative examples and evaluate them at different scaling levels based on a resolution of 1080Ă2400. We evaluate UI-TARS-1.5, MobiMind, and Gemini-2.5-Flash on the same evaluation subset by scaling images with an original resolution of 1080Ă2400 to different factors and assessing task completion performance. We observe that when the scaling factor falls below 0.2, all models exhibit a pronounced drop in completion rate. 24