Paper deep dive

HALLELUAI: A Hallucination-Aware AI System for Ultra-Realistic Image-to-Video Generation at Scale

Aniket Sakpal, Yang Jiang, Rouzbeh Davoudi, Shayan Hassantabar, Mani Najmabadi

Year: 2026Venue: arXiv preprintArea: cs.CVType: PreprintEmbeddings: 42

Intelligence

Status: not_run | Model: - | Prompt: - | Confidence: 0%

Entities (0)

No extracted entities yet.

Relation Signals (0)

No relation signals yet.

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:AI-generated video is increasingly used across marketing, product storytelling, and creative workflows, yet automated; high-precision quality control remains a major constraint to scaling production. We present HALLELUAI, an end-to-end system that moderates and regenerates image-to-video outputs to meet expert-level creative standards and deliver ultra-realistic videos with consistent end-user quality of experience (QoE) at scale. The system integrates a video moderation module that evaluates frame-level aesthetics, temporal motion fidelity, and fine-grained hallucination risks relative to the source image, with an agentic regeneration module that iteratively fixes failures through prompt refinement, controlled camera adjustments, targeted model or image switching, and structured retry strategies. The moderation logic is aligned with domain-specific creative guidelines and produces granular, machine-actionable feedback that directly drives regeneration. In human-in-the-loop evaluations with creative experts, HALLELUAI shows strong alignment and reliably outputs ultra-realistic, production-grade videos suitable for product and marketing placements at scale. This framework advances trustworthy AI generated video content by enforcing visual realism, brand safety, and strict input-image fidelity while enabling image-to-video generation at scale.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

40,204 characters extracted from source content.

Expand or collapse full text

HALLELUAI: A Hallucination-Aware AI System for Ultra-Realistic Image-to-Video Generation at Scale Aniket Sakpal Expedia Group, Email: aniket17sakpal@gmail.com Yang Jiang Expedia Group, Email: jiang.yang.james@gmail.com Rouzbeh Davoudi Expedia Group, Email: rouzbeh.davoudi@gmail.com Shayan Hassantabar Expedia Group, Email: s_hassantabar@yahoo.com Mani Najmabadi Expedia Group, Email: maninajmabadi@gmail.com Abstract AI-generated video is increasingly used across marketing, product storytelling, and creative workflows, yet automated; high-precision quality control remains a major constraint to scaling production. We present HALLELUAI, an end-to-end system that moderates and regenerates image-to-video outputs to meet expert-level creative standards and deliver ultra-realistic videos with consistent end-user quality of experience (QoE) at scale. The system integrates a video moderation module that evaluates frame-level aesthetics, temporal motion fidelity, and fine-grained hallucination risks relative to the source image, with an agentic regeneration module that iteratively fixes failures through prompt refinement, controlled camera adjustments, targeted model or image switching, and structured retry strategies. The moderation logic is aligned with domain-specific creative guidelines and produces granular, machine-actionable feedback that directly drives regeneration. In human-in-the-loop evaluations with creative experts, HALLELUAI shows strong alignment and reliably outputs ultra-realistic, production-grade videos suitable for product and marketing placements at scale. This framework advances trustworthy AI generated video content by enforcing visual realism, brand safety, and strict input-image fidelity while enabling image-to-video generation at scale. Keywords: image-to-video generation, hallucination detection, generative AI, diffusion models 1 Introduction 1.1 Motivation and Context AI-generated video (AIGV) has rapidly become central to content production across visually intensive platforms such as digital advertising, social media, and product detail pages. Advances in diffusion-based image-to-video models now enable large-scale video synthesis at negligible marginal cost, supporting rapid experimentation, personalization, and localization. For domains such as travel, real estate, and e-commerce—where visual realism directly influences trust and conversion, AIGV offers a scalable alternative to traditional, resource-intensive production pipelines. The economic incentives are substantial. Generative video systems compress production timelines from weeks to minutes and reduce per-asset costs by orders of magnitude relative to conventional workflows. These advantages position AIGV as a foundational technology for high-throughput visual storytelling. However, realizing this potential in production environments requires robust mechanisms to guarantee quality, realism, and strict fidelity to the source image. 1.2 Problem Statement – Quality Challenges in Image-to-Video Generation Despite rapid progress, image-to-video (I2V) models still produce artifacts that degrade quality of experience (QoE). Common failures include frame-level degradations (e.g., blur, noise, and brightness drift), temporal inconsistencies such as jitter and unnatural motion, and content hallucinations that introduce or alter scene elements relative to the source image [19, 26]. Hallucinations are particularly problematic because the input image serves as the ground truth. Deviations from it can misrepresent reality, reduce user trust, and create legal risks in domains such as travel and real estate [26]. Current evaluation methods do not adequately address these challenges. Distributional metrics including Inception Score (IS) [2], Fréchet Inception Distance (FID) [6], and Fréchet Video Distance (FVD) [20] assess dataset-level realism but not fidelity to a specific input image. Likewise, existing video quality assessment methods often overlook AIGV-specific artifacts such as cross-frame inconsistencies and subtle hallucinations [4, 5, 12, 24]. As a result, there is no reliable automated mechanism for detecting and mitigating these failures at scale. Recent experience with systems such as OpenAI’s Sora [18] further highlights the need for robust quality assessment and moderation frameworks for reliable I2V deployment [15, 9, 16]. 1.3 Gaps in Prior Work Despite recent advances in generative video evaluation, existing approaches remain insufficient for production-grade, image-conditioned AI-generated video. Per-asset, conditional evaluation: Metrics such as IS [2], FID [6], and FVD [20] are designed for model-level comparison rather than asset-level acceptance. They do not condition on a specific input image and therefore fail to assess source-image fidelity, temporal stability, camera-motion quality, and hallucinations. FVD can also be unstable and may mis-rank models under limited sample sizes [20]. Recent work further highlights the challenge of evaluating semantic consistency and multi-level structure in AI-generated video [10]. Actionability: Frameworks such as FVMD [12], EvalCrafter [14], VBench [7], and Video-Bench [17] improve interpretability and alignment with human judgment, while VQAScore, GenAI-Bench, and VideoScore extend VQA-based evaluation to generative content [23, 11, 13, 5]. However, these methods remain primarily evaluative: they do not enforce source-image fidelity, gate individual assets, or provide diagnostics that directly support remediation. Hallucination detection: Existing approaches focus on prompt consistency in text-to-video generation [3], diffusion-model hallucinations [1], or broader multimodal settings using uncertainty- and self-consistency-based techniques [8, 22]. Recent multimodal research likewise emphasizes the need for stronger grounding and semantic consistency [25, 21]. However, these methods do not condition on a source image or detect fine-grained temporal and structural deviations. Closed-loop deployment: Current benchmarks assess model capability but do not support per-asset gating, domain-specific compliance, or automated remediation. Consequently, the literature lacks end-to-end frameworks that combine expert-aligned evaluation, source-image fidelity, and iterative regeneration for deployment in high-trust domains such as travel and real estate [25, 21]. 1.4 Contributions This work introduces a production-oriented moderation and regeneration framework for image-to-video generation that addresses the above gaps. • Per-asset, conditional video moderation: a domain-aligned module evaluating each video relative to its input image across visual quality, temporal motion, and source-image fidelity. • Machine-actionable, expert-aligned diagnostics: structured failure taxonomies, severity assessments, and pass/fail decisions mapping failure modes to corrective actions. • Fine-grained, temporally aware hallucination detection: a layer identifying object-, structure-, and identity-level deviations, temporal inconsistencies, and artifacts relative to the source image. • Closed-loop, agentic regeneration: a coupled module translating moderation feedback into targeted actions—prompt refinement, camera-control adjustment, model switching, base-image selection—iterating until quality criteria are met. • Human-in-the-loop validation: an evaluation protocol demonstrating alignment with expert judgment and real-world deployability. 2 System Overview We propose an agentic closed-loop image-to-video generation system that unifies automated moderation with autonomous planning and targeted regeneration to enforce expert-defined creative quality and strict source-image fidelity at scale (Fig. 1). Given an input image and creative guidelines (1), the system generates an initial video (2), which is evaluated by the Video Moderation Module (3) acting as both gatekeeper and diagnostics engine. The module assesses outputs across frame-level visual quality (3a), temporal motion quality (3b), and hallucination relative to the source image (3c), producing a structured moderation report (4) with dimension-wise scores, rationales, risk indicators, and a PASS/FAIL decision enforced by a decision gate (5). For failed outputs, control is passed to the Agentic Regeneration Module (7), which converts feedback into autonomous actions. A planning agent (7a) maps failure categories and severities to targeted interventions, including prompt refinement for motion constraints (7b), camera direction or pacing adjustments (7c), base-image substitution for artifact-prone regions (7d), and alternative model selection for model-specific failures (7e). These actions drive targeted regeneration (8), after which outputs are re-evaluated by the moderation module. The loop iterates until acceptance criteria are satisfied and final approval is reached (6), or until iteration, latency, or compute limits are exceeded (Fig. 1). By coupling fine-grained diagnostics with autonomous planning and remediation, the system enables systematic quality control and scalable deployment of image-to-video generation. Figure 1: Architectural overview of the proposed closed-loop system for image-to-video generation. The framework integrates a diagnostic Video Moderation Module (3), calibrated by Creative Expert Guidelines, with an Agentic Regeneration Module (7). Candidate videos undergo iterative evaluation across frame-level, temporal, and fidelity dimensions (3a–c). Failures trigger a Planning Agent (7a) to execute targeted remediation strategies (7b–e) until the asset satisfies predefined quality thresholds or resource constraints are met. 3 Video Moderation Module The Video Moderation Module is the system’s primary quality gate and diagnostic engine, assessing image-to-video outputs before approval or regeneration. For each candidate video, it evaluates domain-aligned creative and fidelity constraints across three dimensions: frame-level visual quality, temporal motion quality, and hallucination detection relative to the input image. The evaluation yields a structured, machine-actionable report with dimension-wise scores, calibrated risk levels, localized rationales, and a unified PASS/FAIL decision—designed to directly parameterize downstream regeneration for targeted correction rather than unguided retries. The module is grounded in a hierarchical problem taxonomy (Fig. 1) categorizing common failure modes in ultra-realistic image-to-video generation, providing a shared abstraction layer that maps detected failures to corrective strategies such as prompt constraints, parameter adjustments, base-image substitution, or model switching. 3.1 Frame-Level Quality Signals: Blur, Contrast, Brightness, and Noise We compute four complementary frame-level signals over a generated video V=f1,…,fTV=\f_1,…,f_T\ and a reference input image I, with all frames and I converted to grayscale for statistics. Each signal is summarized via normalized ratios and compared against calibrated thresholds to produce a unified PASS/FAIL decision with a concise reason. Blur (Sharpness Degradation). Blur is quantified via the variance of the Laplacian: for each frame ftf_t, we compute a sharpness score ℒt=Var(∇2ft)L_t=Var(∇^2f_t), then measure the relative sharpness drop Δblur _blur by comparing the anchor ℒ1L_1 to the minimum sharpness across the video. Contrast (Low/High Contrast). Contrast is quantified via the standard deviation of grayscale intensities. With reference Cref=σ(I)C_ref=σ(I) and frame contrast Ct=σ(ft)C_t=σ(f_t), we compute normalized decrease Δ↓ _ and increase Δ↑ _ relative to reference, optionally tracking within-video spread to capture excessive variability. Brightness/Exposure (Low/High Exposure). Brightness is quantified via mean grayscale intensity. With Bref=μ(I)B_ref=μ(I) and Bt=μ(ft)B_t=μ(f_t), we compute normalized exposure decrease Δ↓B ^B_ and increase Δ↑B ^B_ from the minimum and maximum brightness relative to the reference. Noise (Stochastic Artifacts). Noise is estimated via a per-frame Laplacian-variance score NtN_t; frames are marked noisy when NtN_t exceeds a threshold τnoise _noise, and the video is flagged if the percentage of noisy frames exceeds a small tolerance (e.g., 5%5\%), capturing persistent stochastic artifacts. Algorithm 1 Unified Frame-Level Quality Detection (Blur, Contrast, Brightness, Noise) Reference image I, video V=f1,…,fTV=\f_1,…,f_T\ Thresholds τblur,τ↓,τ↑,τ↓B,τ↑B,τnoise,τp _blur, _ , _ ,τ^B_ ,τ^B_ , _noise, _p Decision d∈PASS,FAILd∈\PASS,FAIL\, reason r d←PASSd , r←r← Acceptable Quality Convert I to grayscale once and compute Cref=σ(I)C_ref=σ(I), Bref=μ(I)B_ref=μ(I) (1) Blur / Sharpness Compute ℒt=Var(∇2ft)L_t=Var(∇^2f_t) for all t Δblur←(ℒ1−mint⁡ℒt)/ℒ1 _blur←(L_1- _tL_t)/L_1 if Δblur>τblur _blur> _blur then d←FAILd ; r←r← Sharpness Degradation end if (2) Contrast Compute Ct=σ(ft)C_t=σ(f_t) for all t Δ↓←(Cref−mint⁡Ct)/Cref _ ←(C_ref- _tC_t)/C_ref Δ↑←(maxt⁡Ct−Cref)/Cref _ ←( _tC_t-C_ref)/C_ref if d=PASSd=PASS and Δ↓>τ↓ _ > _ then d←FAILd ; r←r← Low Contrast else if d=PASSd=PASS and Δ↑>τ↑ _ > _ then d←FAILd ; r←r← High Contrast end if (3) Brightness / Exposure Compute Bt=μ(ft)B_t=μ(f_t) for all t Δ↓B←(Bref−mint⁡Bt)/Bref ^B_ ←(B_ref- _tB_t)/B_ref Δ↑B←(maxt⁡Bt−Bref)/Bref ^B_ ←( _tB_t-B_ref)/B_ref if d=PASSd=PASS and Δ↓B>τ↓B ^B_ >τ^B_ then d←FAILd ; r←r← Low Exposure else if d=PASSd=PASS and Δ↑B>τ↑B ^B_ >τ^B_ then d←FAILd ; r←r← High Exposure end if (4) Noise Compute per-frame noise score NtN_t (Laplacian variance) and mark noisy frames where Nt>τnoiseN_t> _noise pnoise←1T∑t=1T[Nt>τnoise]p_noise← 1T _t=1^TI[N_t> _noise] if d=PASSd=PASS and pnoise>τp_noise> _p then d←FAILd ; r←r← Noise Artifacts end if 3.2 Temporal Motion Quality This component ensures camera motion is cinematic, intentional, and domain-appropriate. It evaluates (i) prompt alignment—whether the dominant motion direction/style matches the prompt intent—and (i) motion intensity and smoothness—whether the pace is neither stagnant nor aggressive and free of jitter. We compute dense optical flow using RAFT to obtain a per-pixel displacement field between consecutive frames, deriving robust frame-level motion statistics that classify the video as Low Motion, High Motion, or Acceptable, with additional jitter flags for high-frequency instability. 3.2.1 Unpleasant Camera Motion (Intensity & Smoothness) Module We estimate motion intensity from the RAFT flow magnitude. At each step t, RAFT produces a dense displacement field t(x)∈ℝ2u_t(x) ^2 mapping pixels from ftf_t to ft+1f_t+1. We compute the per-pixel magnitude mt(x)=∥t(x)∥2m_t(x)= _t(x) _2 and summarize motion via a robust statistic: the mean magnitude of the top-q fraction of pixels (e.g., q=15%q=15\%), Mt=Mean(Topq(mt(x)))M_t=Mean(Top_q(\m_t(x)\)), focusing on salient moving regions and reducing background-noise sensitivity. Video-level intensity is μM=Meant(Mt) _M=Mean_t(M_t) and stability σM=Stdt(Mt) _M=Std_t(M_t). The module flags Low Motion when μM _M falls below a threshold (stagnant/slow pans), High Motion when μM _M exceeds a threshold (whip-pan/aggressive moves), and Jitter when σM _M or the high-frequency variation of MtM_t exceeds a threshold (abrupt accelerations). These outcomes map directly to actionable remediation (reduce motion strength, tighten camera range, or stabilize). 3.2.2 Prompt Alignment Module This module verifies that observed camera motion matches the prompt-specified intent (e.g., pan left-to-right, tilt up, dolly-in/zoom-in). We estimate camera motion using TAPIR point tracking: salient points initialized on the first frame are tracked through the video to obtain 2D trajectories. To reduce object-motion confounds, we preferentially sample edge points (high-gradient regions) and aggregate trajectories into group-level displacement statistics. Letting pit∈ℝ2p_i^t ^2 denote tracked point i at time t, we compute net displacement Δpi=piT−pi1 p_i=p_i^T-p_i^1 and summarize motion by the dominant direction ^=Normalize(Mediani(Δpi)) v=Normalize(Median_i( p_i)). We further use structured point groups (e.g., left-edge set LP_L, top-edge set UP_U) to detect zoom/dolly: if LP_L moves left while UP_U moves up (divergence from the image center), the motion is consistent with zoom-in/dolly-in, whereas convergence indicates zoom-out/dolly-out. The observed label y y is compared against the prompt intent yprompty_prompt to yield an alignment score saligns_align, triggering a FAIL when below threshold. Algorithm 2 Temporal Motion Quality: Intensity & Smoothness + Prompt Intent Alignment (TAPIR) Video V=f1,…,fTV=\f_1,…,f_T\, prompt motion intent yprompty_prompt Top-pixel fraction q (e.g., 0.150.15) Thresholds τlow,τhigh,τjit _low, _high, _jit, alignment threshold τalign _align Decision dmotion∈PASS,FAILd_motion∈\PASS,FAIL\ and reason rmotionr_motion dmotion←PASSd_motion , rmotion←r_motion← Acceptable Motion (A) Unpleasant Camera Motion: Intensity & Smoothness (RAFT) for t=1t=1 to T−1T-1 do t←RAFT(ft,ft+1)u_t (f_t,f_t+1) mt(x)←∥t(x)∥2m_t(x)← _t(x) _2 for all pixels x Mt←Mean(Topq(mt(x)))M_t (Top_q(\m_t(x)\) ) end for μM←Meant(Mt) _M _t(M_t), σM←Stdt(Mt) _M _t(M_t) if μM<τlow _M< _low then dmotion←FAILd_motion ; rmotion←r_motion← Low Motion (Stagnant) else if μM>τhigh _M> _high then dmotion←FAILd_motion ; rmotion←r_motion← High Motion (Aggressive) else if σM>τjit _M> _jit then dmotion←FAILd_motion ; rmotion←r_motion← Jitter / Instability end if (B) Prompt Intent Alignment (TAPIR) (B1) Point Initialization & Tracking: Initialize an edge-biased point set P on the first frame f1f_1 Track points pitt=1T\p_i^t\_t=1^T using TAPIR and compute net displacements Δpi←piT−pi1 p_i← p_i^T-p_i^1 (B2) Dominant Motion Estimation: Estimate global camera translation ^←Normalize(Mediani∈(Δpi)) v \! (Median_i ( p_i) ) (B3) Zoom / Dolly Detection: Partition points into spatial edge sets (e.g., left LP_L, top UP_U) δL←Mediani∈L(Δpi) _L _i _L( p_i), δU←Mediani∈U(Δpi) _U _i _U( p_i) (B4) Motion Intent Classification: Infer observed motion label y^←ClassifyMotion(^,δL,δU) y ( v, _L, _U) (B5) Prompt Alignment Check: Compute alignment score salign←[y^=yprompt]s_align [ y=y_prompt] if dmotion=PASSd_motion=PASS and salign<τaligns_align< _align then dmotion←FAILd_motion ; rmotion←r_motion← Prompt Misalignment end if 3.3 Hallucinations We evaluate two types of hallucination in image-to-video generation: object hallucination and new-structure hallucination. Object hallucination refers to failures in preserving temporal consistency, physically plausible motion, and coherent object interactions across frames relative to the input image, leading to object drift, deformation, and other artifacts. New-structure hallucination is the synthesis of objects or scene elements absent from the source image—particularly important in high-trust domains (e.g., travel, real estate), where introducing nonexistent structures may misrepresent reality and increase legal and reputational risk. The following section details the annotation categories for each type. Figure 2: Prompt template provided to the VLLM for hallucination detection. It defines seven hallucination categories and instructs the model to identify and track objects across frames, categorize changes as hallucinations (distinguishing them from camera motion), assign a 1–10 severity score per category, and output results as JSON (0 if none detected). 3.3.1 Hallucination Categories 1. Object Hallucination • Object Fusion: Two or more objects unnaturally merge, exhibit boundary corruption, or interpenetrate. • Implausible Transformation: An object undergoes non-causal changes in shape, color, or identity unexplained by interactions or scene dynamics. • Object–Background Merge: An object blends into the background despite being expected to remain visually distinct. • Unnatural Object Movement: Object motion violates physical constraints (e.g., unsupported motion, inconsistent trajectories) or lacks plausible causation. • Implausible Disappearance: An object abruptly vanishes without occlusion, exit from view, or other reasonable explanation. • Object Splitting: A single object unnaturally divides into multiple distinct objects. • Text Hallucination: Textual elements (e.g., numbers, signs) change content, deform, or become illegible. 2. New Structure Hallucination: • Natural entry: Objects absent from the source image appear by entering the field of view with temporally continuous, physically plausible motion. • Camera-induced entry: Previously unseen objects become visible due to global camera motion (e.g., panning, tilting, zooming) rather than object motion. • Abrupt emergence: Objects not in the source image appear instantaneously without reasonable physical explanation. 3.3.2 Hallucination Detection Module As illustrated in Fig. 3, we propose a hallucination-detection module for videos generated by an Image2Video model. Given a generated video, we uniformly sample frames F=f1,…,fnF=\f_1,…,f_n\. To detect object hallucinations, we first extract objects O=o1,…,omO=\o_1,…,o_m\ from the input image using a multimodal large language model (MLLM), then provide O, the input image, and F to an MLLM (e.g., GPT-4o) to identify object-level inconsistencies across time, mapped to the predefined categories C=c1,…,cmC=\c_1,…,c_m\. For each category cic_i, the MLLM outputs a severity score Soi∈[0,10]S^i_o∈[0,10] and a brief rationale when evidence is present, following the prompt template shown in Fig. 2. The object-hallucination decision dobjectd_object aggregates the category-wise scores, thresholding each against τobjecti _object^i. To detect new-structure hallucinations, we compare each sampled frame fif_i with the input image to determine whether novel objects appear under three cases: (1) natural entry, (2) camera-induced entry, and (3) abrupt emergence. An MLLM produces a per-frame binary decision dnewid^i_new, and the video-level decision is dnew=⋃i=1ndnewid_new= _i=1^nd^i_new. Figure 3: Hallucination detection module for Image2Video generation. Uniformly sampled frames are analyzed with an MLLM to (top) detect and track input-image objects, map temporal inconsistencies to predefined object-hallucination categories, and aggregate category scores for a final decision; and (bottom) compare each sampled frame with the input image to identify new-structure hallucinations (natural entry, camera-induced entry, or abrupt emergence) 3.4 Scoring and Risk Assessment The module aggregates frame-level, temporal-consistency, and hallucination scores via domain-calibrated weights into a unified verdict, paired with a narrative summary contextualizing each issue against creative standards and Quality of Experience (QoE). A PASS/FAIL decision follows from thresholded expert-derived criteria, with a standardized risk level (No, Low, Medium, High) tied to specific failures. Outputs form a tuple of failure codes, severities, and remediation levers (e.g., reduce motion, adjust exposure, switch models), serving both as a gating mechanism and as input to the agentic regeneration module for targeted, iterative corrections. 4 Agentic Regeneration Module The Agentic Regeneration Module converts moderation findings into targeted corrective actions via a closed-loop planner–executor process. A planning agent interprets frame-level, temporal, and hallucination signals relative to the source image, selects a minimal yet effective intervention set, and issues an updated generation request; the regenerated asset is re-scored, looping until quality thresholds or operational limits are met. Inputs and Outputs. Inputs comprise the source image and metadata (e.g., campaign, channel), prior generation configuration (prompt, model, seed, motion parameters), and the structured moderation report (failure codes, severities, rationales, PASS/FAIL). Outputs comprise a next-generation plan (revised prompt, actions, parameters, chosen model or base image) and execution artifacts (regenerated video with a full action log for reproducibility). Action Policy. The agent draws on a constrained intervention library: (i) prompt refinements enforcing motion clarity, object preservation, source-image fidelity, and aesthetic stability; (i) camera/motion adjustments to reduce jitter, overshoot, or blur; (i) base-image selection when artifacts cluster around occlusions or ambiguous regions; and (iv) model switching or tuning for model-specific failures. A cost- and efficacy-aware policy prioritizes low-cost, minimally invasive actions, escalates for high-severity or repeated failures, and bundles actions when interactions are likely. Iteration and Governance. The loop (Generate → Moderate → Regenerate) terminates on PASS, budget exhaustion, or escalation to human review for persistent issues, logging all decisions for auditability. Coupling structured diagnostics with a cost-aware strategy, the module converts failing candidates into compliant assets while preserving input-image fidelity and minimizing manual oversight. 5 Tuning and Evaluation Threshold calibration and parameter tuning were performed using 58 AI-generated video clips across the three moderation components. For hallucination detection, we used GPT-4o as the underlying LLM. Prompts were iteratively refined to elicit accurate reasoning for each hallucination category. Recall, defined as the proportion of hallucination-containing videos correctly detected, was used as the primary metric for selecting the final prompts. Category-specific hallucination thresholds were then optimized to maximize sensitivity for detecting high-risk videos, as defined in Section 3.4. We then evaluate the proposed tuned system using a human-in-the-loop protocol designed to measure alignment with expert creative judgment and readiness for production deployment. The protocol consists of three stages: Shadow Mode (calibration), Pseudo Production (gating performance), and Production QA (longitudinal monitoring). 5.1 Stage 1: Shadow Mode In Shadow Mode, unseen 146 AI-generated video clips were independently evaluated by creative expert reviewers, the HALLELUAI moderation system, and industry benchmark methods, without cross-visibility of decisions. Each clip was standardized through trimming and normalizing the aspect-ratio before evaluation. 5.1.1 Creative Expert Alignment First, we evaluated the alignment of moderation systems by comparing their output to reviews from human creative experts on the video dataset. The HALLELUAI moderation system achieves 88% precision and an agreement rate of 87%. The results are summarized in Table 1 Agreement Analysis. Table 1: HALLELUAI moderation Performance Metrics Compared to Expert Decisions Metric Value Overall PASS/FAIL Agreement 86.9% Precision on AI-approved assets 88% Recall on AI-approved assets 74% Precision is the key metric, as it directly reflects the quality and reliability of videos approved by the system. The system exhibits conservative gating behavior, filtering a portion of expert-approved clips while limiting false approvals. Stage-Level Diagnostic Breakdown To analyze performance across moderation components, we decompose precision by category, as shown in Table 2. Table 2: Stage-level contribution to Precision and False Positives (FP). Percentages reflect the proportion of total observed errors attributable to each signal. Module Sub-Category Precision Frame-Level Quality Blur 100% Contrast 97% Brightness 100% Noise 100% Temporal Motion Quality Prompt Alignment 100% Motion Intensity 97% Hallucination Detection Object Hallucination 97% New Structure Hallucination 97% Text Hallucination 100% The majority of precision-impacting errors arise from fine-grained object-level hallucinations. Motion intensity thresholds contribute primarily to false negatives, indicating conservative rejection behavior. Frame-level degradations contribute minimally to disagreement. 5.1.2 Benchmark Comparison Since no open-source systems provide comprehensive evaluation for image-to-video generation, we compared our system against UGC video quality assessment (VQA) models such as DOVER [24] and COVER [4], as well as multimodal large language models (MLLMs) via zero-shot prompting (See Fig. 4 for prompt details). The MLLM prompt was designed to evaluate the three components our system addresses—frame-level quality, temporal motion quality, and hallucination—with frames uniformly sampled at 2 fps. As shown in Table 3, UGC VQA models like DOVER and COVER could not differentiate PASS from FAIL videos, as they were not designed for image-to-video generation and lack hallucination-detection metrics. The MLLM-based method showed low precision, unable to detect artifacts in AI-generated videos—demonstrating that without detailed, comprehensive prompting, MLLMs cannot reliably identify such artifacts. Figure 4: Evaluation Prompt Designed for MLLMs to Evaluate Frame-level Quality,Temporal Motion Quality and Hallucination in Image2Video Generation Table 3: Performance Comparison of Different Evaluation Methods Method Prec. Rec. F1 Acc. DOVER 0.20 0.21 0.20 0.56 COVER 0.27 0.33 0.30 0.58 Qwen3-VL-8B 0.28 1.00 0.43 0.30 NOVA 2 Lite 0.25 0.74 0.37 0.34 HALLELUAI 0.88 0.74 0.81 0.87 5.2 Stage 2: Pseudo Production (Gating Performance) In Pseudo Production mode, we evaluate the combined moderation + regeneration system end-to-end by sending only system-approved clips for human creative review. Of 1,158 system-approved clips, 1,126 were also accepted by experts, yielding 97% precision and indicating that the vast majority of surfaced clips are production-ready. This precision is not directly comparable to Shadow Mode, since outputs here have undergone iterative regeneration and refinement before evaluation—the system is not merely filtering errors but actively improving clips, driving higher agreement with expert judgment. 5.3 Stage 3: Production QA (Longitudinal Monitoring) Following calibration, ongoing monitoring employs random sampling (1–5%) of system-approved assets for expert review. Precision levels of 97% remain consistent with Pseudo Production Mode results, and regression testing ensures that threshold or regeneration-policy updates do not degrade previously validated performance. Deployed at scale, the system generated approximately 70,000+ videos, further validating its reliability. 5.4 Evaluation Summary These results demonstrate that structured moderation combined with agentic regeneration can function as a reliable production-grade quality control layer for ultra-realistic image-conditioned video generation. 6 Conclusion HALLELUAI is an expert-aligned, production-oriented system that makes image-to-video quality assurance a controllable, closed-loop process, coupling per-asset moderation—aesthetics, motion quality, and source-image–conditioned hallucination detection—with an agentic regeneration policy that converts failures into targeted fixes rather than blind retries. Human-in-the-loop validation shows strong agreement with creative experts, supporting deployment as a high-precision gate for scalable AIGV. By enforcing input-image fidelity, emitting machine-actionable diagnostics, and maintaining auditable decision trails, it bridges benchmark-style evaluation and real-world creative governance—to our knowledge, the first integrated moderation-and-regeneration framework purpose-built for ultra-realistic, image-conditioned video generation at scale. References [1] S. K. Aithal, P. Maini, Z. C. Lipton, and J. Z. Kolter (2024) Understanding hallucinations in diffusion models through mode interpolation. Advances in neural information processing systems 37, p. 134614–134644. Cited by: §1.3. [2] S. Barratt and R. Sharma (2018) A note on the inception score. arXiv preprint arXiv:1801.01973. Cited by: §1.2, §1.3. [3] Z. Chu, L. Zhang, Y. Sun, S. Xue, Z. Wang, Z. Qin, and K. Ren (2024) Sora detector: a unified hallucination detection for large text-to-video models. arXiv preprint arXiv:2405.04180. Cited by: §1.3. [4] C. He, Q. Zheng, R. Zhu, X. Zeng, Y. Fan, and Z. Tu (2024) Cover: a comprehensive video quality evaluator. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 5799–5809. Cited by: §1.2, §5.1.2. [5] X. He, D. Jiang, G. Zhang, M. Ku, A. Soni, S. Siu, H. Chen, A. Chandra, Z. Jiang, A. Arulraj, et al. (2024) Videoscore: building automatic metrics to simulate fine-grained human feedback for video generation. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, p. 2105–2123. Cited by: §1.2, §1.3. [6] M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter (2017) GANs trained by a two time-scale update rule converge to a local nash equilibrium. Advances in Neural Information Processing Systems 30. Cited by: §1.2, §1.3. [7] Z. Huang, Y. He, J. Yu, F. Zhang, C. Si, Y. Jiang, Y. Zhang, T. Wu, Q. Jin, N. Chanpaisit, et al. (2024) Vbench: comprehensive benchmark suite for video generative models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 21807–21818. Cited by: §1.3. [8] S. Kadavath et al. (2022) Language models (mostly) know what they know. arXiv preprint arXiv:2207.05221. Cited by: §1.3. [9] S. Kou, J. Jin, Z. Zhou, Y. Ma, Y. Wang, Q. Chen, P. Jiang, X. Yang, J. Zhu, K. Yu, et al. (2026) Think-then-generate: reasoning-aware text-to-image diffusion with llm encoders. arXiv preprint arXiv:2601.10332. Cited by: §1.2. [10] J. Li, H. Xu, S. Zhu, J. He, and H. Wang (2025) Multilevel semantic-aware model for ai-generated video quality assessment. In ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), p. 1–5. Cited by: §1.3. [11] Z. Lin, D. Pathak, B. Li, J. Li, X. Xia, G. Neubig, P. Zhang, and D. Ramanan (2024) Evaluating text-to-visual generation with image-to-text generation. In European Conference on Computer Vision, p. 366–384. Cited by: §1.3. [12] Liu et al. (2024) Fréchet video motion distance: a metric for evaluating motion consistency in video generation. arXiv preprint arXiv:2407.16124. Cited by: §1.2, §1.3. [13] Liu et al. (2024) GenAI-bench: evaluating and improving compositional text-to-visual generation. arXiv preprint arXiv:2406.13743. Cited by: §1.3. [14] Y. Liu, X. Cun, X. Liu, X. Wang, Y. Zhang, H. Chen, Y. Liu, T. Zeng, R. Chan, and Y. Shan (2024) Evalcrafter: benchmarking and evaluating large video generation models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 22139–22149. Cited by: §1.3. [15] W. Ma, X. Yang, L. Jiao, L. Li, X. Liu, F. Liu, P. Chen, Y. Yang, M. Ma, L. Sun, et al. (2025) Video diffusion generation: comprehensive review and open problems. Artificial Intelligence Review 58 (11), p. 338. Cited by: §1.2. [16] Y. Ma, K. Feng, Z. Hu, X. Wang, Y. Wang, M. Zheng, X. He, C. Zhu, H. Liu, Y. He, Z. Wang, Z. Li, X. Li, W. Liu, D. Xu, L. Zhang, and Q. Chen (2025) Controllable video generation: a survey. arXiv preprint arXiv:2507.16869. Cited by: §1.2. [17] M. Ning, B. Zhu, Y. Xie, B. Lin, J. Cui, L. Yuan, D. Chen, and L. Yuan (2025) Video-bench: a comprehensive benchmark and toolkit for evaluating video-based large language models. Computational Visual Media. Cited by: §1.3. [18] OpenAI (2026) Sora: openai text-to-video model. Note: Product documentation and public reports, URL: https://openai.com/research/sora Cited by: §1.2. [19] W. Ren, H. Yang, G. Zhang, C. Wei, X. Du, S. Huang, and W. Chen (2024) ConsistI2V: enhancing visual consistency for image-to-video generation. arXiv preprint arXiv:2402.04324. Cited by: §1.2. [20] T. Unterthiner et al. (2018) Towards accurate generative models of video: a new metric & challenges. In International Conference on Learning Representations Workshop, Cited by: §1.2, §1.3. [21] J. Wang, H. Duan, G. Zhai, J. Wang, and X. Min (2025) Aigv-assessor: benchmarking and evaluating the perceptual quality of text-to-video generation with lmm. In Proceedings of the Computer Vision and Pattern Recognition Conference, p. 18869–18880. Cited by: §1.3, §1.3. [22] X. Wang, J. Wei, D. Schuurmans, Q. Le, E. Chi, S. Narang, A. Chowdhery, and D. Zhou (2022) Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171. Cited by: §1.3. [23] Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli (2004) Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing 13 (4), p. 600–612. Cited by: §1.3. [24] H. Wu, E. Zhang, L. Liao, C. Chen, J. Hou, A. Wang, W. Sun, Q. Yan, and W. Lin (2023) Exploring video quality assessment on user generated contents from aesthetic and technical perspectives. In Proceedings of the IEEE/CVF international conference on computer vision, p. 20144–20154. Cited by: §1.2, §5.1.2. [25] Z. Zhang, W. Sun, L. Xinyue, J. Jia, X. Min, Z. Zhang, C. Li, Z. Chen, W. Puyi, S. Fengyu, et al. (2025) Benchmarking multi-dimensional aigc video quality assessment: a dataset and unified model. ACM Transactions on Multimedia Computing, Communications and Applications 21 (9), p. 1–24. Cited by: §1.3, §1.3. [26] M. Zhao, H. Zhu, C. Xiang, K. Zheng, C. Li, and J. Zhu (2024) Identifying and solving conditional image leakage in image-to-video diffusion model. Advances in Neural Information Processing Systems 37, p. 30300–30326. Cited by: §1.2.

Instant research discovery