Paper deep dive
Weave Mamba Fusion: Global Cross-Scale Interaction for Lightweight Face Detection
Dohun Kim, Jinmyung Jung
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Feature pyramid methods, from FPN to BiFPN, have achieved strong performance in face detection by fusing multi-scale features. However, detecting faces under unconstrained conditions, such as small scale, occlusion, and extreme pose, remains difficult, as it requires global cross-scale dependencies that local fusion cannot model. State space models such as Mamba provide global context with linear complexity by scanning features as a sequence, and therefore offer a promising direction for this problem. Nevertheless, such a scan needs the two pyramid scales combined into a single feature map, and the way they are combined determines whether cross-scale structure is preserved. Summation collapses the two scales before the scan, so the scan has no cross-scale structure to exploit, while concatenation keeps both scales but at far higher cost. To address this, we propose \textbf{Weave Mamba Fusion (WMF)}, which interleaves two adjacent pyramid scales column by column so that each step of a horizontal bidirectional SS2D scan moves from one scale to the other. With partial-channel processing and parameter-free de-weaving, WMF enables efficient cross-scale interaction while preserving feature structure. Integrating WMF into every fusion node yields \textbf{WeaveBiFPN}, the neck of our \textbf{WeaveFace} detector. On WIDER FACE, WeaveFace achieves 91.41\% mean AP with only 0.34M parameters and 1.16 GFLOPs, outperforming prior detectors under 0.5M parameters. Its largest gains are on the Hard subset, where it reaches 87.14\% AP. The code is publicly available at \url{this https URL}.
Tags
Links
- Source: https://arxiv.org/abs/2610.05865v1
- Canonical: https://arxiv.org/abs/2610.05865v1
Trouble viewing inline? Open PDF directly →
Full Text
48,744 characters extracted from source content.
Expand or collapse full text
Weave Mamba Fusion: Global Cross-Scale Interaction for Lightweight Face Detection Dohun Kim Jinmyung Jung† Affiliation: School of Data Science, The University of Suwon, Korea E-mail dohun0714@naver.com, jmjung@suwon.ac.kr Affiliation: Affiliation: Corresponding Author Abstract Feature pyramid methods, from FPN to BiFPN, have achieved strong performance in face detection by fusing multi-scale features. However, detecting faces under unconstrained conditions, such as small scale, occlusion, and extreme pose, remains difficult, as it requires global cross-scale dependencies that local fusion cannot model. State space models such as Mamba provide global context with linear complexity by scanning features as a sequence, and therefore offer a promising direction for this problem. Nevertheless, such a scan needs the two pyramid scales combined into a single feature map, and the way they are combined determines whether cross-scale structure is preserved. Summation collapses the two scales before the scan, so the scan has no cross-scale structure to exploit, while concatenation keeps both scales but at far higher cost. To address this, we propose Weave Mamba Fusion (WMF), which interleaves two adjacent pyramid scales column by column so that each step of a horizontal bidirectional S2D scan moves from one scale to the other. With partial-channel processing and parameter-free de-weaving, WMF enables efficient cross-scale interaction while preserving feature structure. Integrating WMF into every fusion node yields WeaveBiFPN, the neck of our WeaveFace detector. On WIDER FACE, WeaveFace achieves 91.41% mean AP with only 0.34M parameters and 1.16 GFLOPs, outperforming prior detectors under 0.5M parameters. Its largest gains are on the Hard subset, where it reaches 87.14% AP. The code is publicly available at https://github.com/dohun-mat/WeaveMambaFusion. Keywords: Feature Pyramid Networks, State Space Models, Face Detection, Weave Mamba Fusion. 1 Introduction Figure 1: Feature map visualization of different feature pyramid networks. Given the input image, we visualize the fused features from (a) FPN, (b) PANet, (c) BiFPN, and (d) our WeaveBiFPN. Our WeaveBiFPN separates faces from the background more clearly than the others. Face detection enables a wide range of applications such as face recognition, biometric authentication, and human–computer interaction. Despite substantial progress driven by deep convolutional networks, detection in unconstrained, real-world environments remains an open problem. Challenges such as extreme scale variation, severe occlusion, pose variation, and blur are most pronounced on the Hard subset of the WIDER FACE benchmark [31]. Reliable detection under such conditions requires combining the fine-grained spatial information of high-resolution features with the rich semantics of low-resolution ones, making multi-scale feature fusion an essential ingredient of modern detectors. FPN [15], PANet [16], and BiFPN [21] have progressively refined how information flows across pyramid levels—adding bottom-up paths and learnable fusion weights atop FPN’s original top-down design (Fig. 2(a)–(c)). Yet even with learnable weights, their cross-scale fusion provides only local interaction—summation or concatenation [2] followed by local convolutions—and cannot reach distant context. As shown in Fig. 1, the fused features therefore lack global context, a limitation most evident for small and occluded faces. Such faces provide little information on their own, so detecting them relies on surrounding semantic context. This context lies in the low-resolution pyramid level, but at positions away from the face, not at the face’s own location. A local fusion can only combine features at nearby positions and thus cannot reach this distant context. Detecting these faces therefore calls for global, cross-scale interaction in which each position can integrate distant context from the other scale. Overcoming this limitation requires a fusion module that (i) retains the spatial information from both scales, (i) enables long-range, cross-scale interaction rather than the spatially local interaction inherent to convolution, and (i) remains efficient enough to operate at every node of a high-resolution pyramid. Self-attention [23] provides all-pairs interaction—one way to achieve (i)—but its quadratic complexity becomes prohibitively expensive at the high resolutions where pyramid fusion operates. State space models (SSMs) such as Mamba [7], by contrast, offer global context at linear complexity. An SSM alone, however, is not enough. Once the two scales are collapsed by summation, the scan has no cross-scale structure to exploit. The key insight is that, because a scan is sequential, weaving the two scales lets a single scan carry information from one scale into the other at every step. Motivated by this, we propose Weave Mamba Fusion (WMF), a fusion operator that achieves global cross-scale interaction at linear cost. WMF weaves the two feature maps column by column into a single 2D map and processes it with a bidirectional horizontal scan built on S2D [36, 18]. Because adjacent columns belong to different scales, every step of the scan propagates information from one scale to the other, giving each location access to the full spatial extent of the other scale rather than a local neighborhood. To keep this efficient, we further adopt a partial-channel design inspired by partial convolution (PConv) [4], applying S2D to only a fraction of the channels and leaving the rest as an identity branch, which substantially reduces the cost of the scan with negligible loss in fusion quality. Applying WMF at every cross-scale node of both BiFPN pathways yields WeaveBiFPN, the neck of our WeaveFace detector, which attains state-of-the-art accuracy among lightweight face detectors, with its largest gain on the Hard subset where long-range cross-scale context matters most. Figure 2: Comparison of feature pyramid architectures. (a) FPN. (b) PANet. (c) BiFPN. (d) Our WeaveBiFPN. The dashed boxes enlarge the feature fusion in each design. Our WeaveBiFPN keeps the bidirectional structure of BiFPN but replaces its weighted fusion with Weave Mamba Fusion. The main contributions of this paper are summarized as follows: • We propose Weave Mamba Fusion (WMF), an SSM-based redesign of the multi-scale fusion operator for face detection. Rather than summation or concatenation followed by local convolutions, WMF weaves two feature maps into a single 2D map and processes it with a bidirectional scan, achieving global cross-scale interaction within a single operator. • We introduce a partial-channel design that applies the scan to only a fraction of the channels, substantially reducing WMF’s dominant cost while preserving cross-scale fusion. • We propose WeaveBiFPN and the resulting WeaveFace detector. On WIDER FACE, WeaveFace attains 87.14% Hard AP and 91.41% mean AP with only 0.34M parameters and 1.16 GFLOPs, making it the most accurate detector with fewer than 0.5M parameters. 2 Related Work 2.1 Face Detection Driven by advances in deep CNNs, face detection has progressed rapidly. Early work such as MTCNN [33] learns face detection and landmark localization together through cascaded multi-task learning. RetinaFace [6] adopts a multi-branch prediction head and has become a widely used baseline for face detection in unconstrained environments. DSFD [13] improves small-face detection through dual-shot detection and feature enhancement, and TinaFace [37] shows that competitive face detection accuracy can be achieved using only generic object detection techniques. These detectors [6, 13, 37, 17, 24], however, achieve such accuracy only at the cost of tens of millions of parameters and tens to hundreds of GFLOPs. Lightweight face detection has also been actively studied. SCRFD [9] improves efficiency by redistributing training samples and computation toward the smaller, harder faces. EResFD [11] shows that a ResNet with greatly reduced channels outperforms previous lightweight backbones, and FDLite [1] reduces the parameter count to only 0.24M. More recent detectors instead raise accuracy by strengthening the BiFPN neck, as in FeatherFace [12], which adds CBAM attention and deformable convolutions. 2.2 Feature Pyramid Networks FPN [15, 25, 22] introduces a multi-scale fusion architecture in which a top-down pathway passes semantic information from deeper levels to shallower ones, while lateral connections add the matching backbone features at each level. PANet [16] adds a bottom-up pathway to reinforce high-level features with low-level spatial information, and BiFPN [21] uses a bidirectional cross-scale structure with learnable fusion weights. Beyond these hand-designed pathways, AFPN [30] fuses adjacent levels progressively before incorporating distant ones. Despite their different topologies, these methods share the same fusion operator: two levels are resized to a common spatial size and merged position-wise, followed by convolutions with a local receptive field. Their cross-scale interaction is therefore confined to a local neighborhood and cannot capture long-range dependencies to distant context in the other scale. FPT [32] instead models non-local interaction via attention across pyramid levels, but its quadratic cost makes it ill-suited to the lightweight regime. We instead replace the local, position-wise fusion operator with Weave Mamba Fusion (WMF), which achieves global cross-scale interaction at linear cost. 2.3 State Space Models for Multi-Scale Fusion State space models (SSMs) have gained attention as an alternative to Transformers, modeling long-range dependencies with linear complexity in sequence length. The Structured State Space sequence model (S4) [8] established the efficiency of long-range sequence modeling, and Mamba [7] advanced it with a selective SSM (S6) whose parameters depend on the input. Vision Mamba [36] and VMamba [18] extend Mamba to the image domain; in particular, VMamba introduces the Selective Scan 2D (S2D), which combines scans along multiple directions to model global information in 2D images. Within detection, most SSM-based methods place the scan in the backbone [28, 10, 20], extracting single-scale features before any cross-scale fusion occurs. A smaller body of work instead places SSMs in the fusion pathway: Mamba-YOLO’s [28] PAFPN neck, PyramidMamba [26], and MDDCNet [14] all add SSM blocks alongside multi-scale connections, but the cross-scale combination itself remains a conventional position-wise operation, with the SSM only enhancing features before or after it. WMF instead applies the SSM as the fusion operator: the two scales are woven at alternating positions, so every step of the S2D scan moves from one scale to the other. 3 WeaveFace Figure 3: Overview of WeaveFace. The proposed WeaveFace architecture, illustrated in Fig. 3, follows a standard detection pipeline, comprising an EResNet backbone [11] and CBAM [29] refinement modules. Our core contribution lies in the WeaveBiFPN neck, which integrates our novel Weave Mamba Fusion (WMF) to enable global cross-scale feature interaction. The fused features are passed to task-specific detection heads to produce classification, bounding box, and landmark outputs. These heads are optimized using a standard multi-task loss, which effectively leverages the global cross-scale features captured by WMF to improve localization and classification accuracy. 3.1 Weave Mamba Fusion (WMF) WMF is the fusion operator applied at each cross-scale node of WeaveBiFPN. Whereas BiFPN combines the two scales by weighted summation, WMF instead weaves them into a single 2D map and processes it with a scan. It builds on the S2D [18], which extends the selective state space model of Mamba [7]—itself based on structured SSMs [8]—to 2D data. S2D scales linearly in the sequence length L but quadratically in the channel width C, giving (L⋅C2)O(L· C^2). This asymmetry motivates WMF’s two design choices: the linear cost in L makes spatial weaving inexpensive, while the quadratic cost in C motivates applying the scan to only a fraction of the channels. Concretely, WMF weaves the two same-resolution maps column by column into a single 2D map, so that every step of S2D’s horizontal scan crosses between the two scales, realizing the global cross-scale interaction. Applying S2D to only a fraction of the channels then keeps this interaction efficient. Together, these two choices let a single operator satisfy structure preservation, global cross-scale interaction, and efficiency. WMF operates on two inputs of the same spatial resolution. In the WeaveBiFPN architecture, resolution alignment between adjacent levels is a prerequisite for WMF: adjacent-level features are resized via bilinear upsampling (top-down) or strided downsampling (bottom-up) to match the target level’s dimensions. We denote the resulting feature tensors of the current and resized adjacent levels as X1,X2∈ℝB×C×H×WX_1,X_2 ^B× C× H× W. (1) Width-axis Weaving X1X_1 and X2X_2 are interleaved column-wise to form a single 2D map Z∈ℝB×C×H×2WZ ^B× C× H× 2W: Z[:,:,:,2i]=X1[:,:,:,i],Z[:,:,:,2i+1]=X2[:,:,:,i],i∈0,…,W−1.Z[:,:,:,2i]=X_1[:,:,:,i], Z[:,:,:,2i+1]=X_2[:,:,:,i], i∈\0,…,W-1\. (1) Even columns of Z thus come from X1X_1 and odd columns from X2X_2, so the two scales alternate along the width. (2) Partial-channel Split We apply layer normalization to the woven feature map Z∈ℝB×C×H×2WZ ^B× C× H× 2W. To optimize computational efficiency, we employ a partial-channel strategy that partitions the normalized features into two distinct streams. Specifically, the features are split along the channel dimension as follows: Z¯=LN(Z),(Zp,Zu)=Split(Z¯),Cp=C/N, Z=LN(Z), (Z_p,Z_u)=Split( Z), C_p=C/N, (2) where Z¯ Z is divided into a processed component Zp∈ℝB×Cp×H×2WZ_p ^B× C_p× H× 2W and an identity component Zu∈ℝB×(C−Cp)×H×2WZ_u ^B×(C-C_p)× H× 2W. Z^p=S2D(Zp),Z^u=Zu. Z_p=S2D(Z_p), Z_u=Z_u. (3) The S2D operator is applied exclusively to ZpZ_p, while ZuZ_u bypasses the scan and is passed forward via an identity mapping. This partial execution exploits channel redundancy, since only 1/N1/N of the channels are needed to capture the essential cross-scale dependencies. The two streams are then recombined via a 1×11× 1 convolution. We set N=2N=2 by default, so that the scan is applied to half of the channels while the remaining half preserve the original signal through the identity path. As ZpZ_p contains features from both scales, each horizontal scan step exchanges information across them. Because this exchange occurs along the width, a bidirectional horizontal scan (s=2s=2) suffices, allowing us to remove the two vertical scans in VMamba’s S2D. (3) De-weaving The two branches are concatenated along the channel axis, then de-woven column-wise to recover two scale-specific feature maps: Z^=Concat(Z^p,Z^u)∈ℝB×C×H×2W, Z=Concat( Z_p, Z_u) ^B× C× H× 2W, (4) Y1[:,:,:,j]=Z^[:,:,:,2j],Y2[:,:,:,j]=Z^[:,:,:,2j+1],j∈0,…,W−1. Y_1[:,:,:,j]= Z[:,:,:,2j], Y_2[:,:,:,j]= Z[:,:,:,2j+1], j∈\0,…,W-1\. (5) As the exact inverse of the weave, de-weaving restores every column to its source—even columns to X1X_1, odd columns to X2X_2—recovering the original spatial structure of both feature maps without introducing any parameters. (4) Channel Re-mixing The two outputs are concatenated along the channel axis, projected back to C channels, and passed through GELU: Y=GELU(Conv1×1([Y1;Y2])).Y=GELU\! (Conv_1× 1([Y_1;\,Y_2]) ). (6) The 1×11×1 convolution fuses Y1Y_1 and Y2Y_2, allowing the S2D-processed channels to interact with the identity-branch channels so that every channel contributes to the final representation. 3.2 Complexity Let L=H⋅WL=H· W be the number of tokens in each input. We analyze the cost of a single application of WMF, since WeaveBiFPN applies it at every cross-scale node. The cost is dominated by two operations: the S2D scan and the final 1×11×1 convolution. Applied to the full C channels, S2D’s input and output projections would cost (2L⋅C2)O(2L· C^2), where the factor of 2L2L comes from the woven width 2W2W. The partial-channel design instead processes only C/NC/N channels, reducing this to (2L⋅C2/N2)O(2L· C^2/N^2)—an N2×N^2× reduction. The final 1×11×1 convolution then maps the 2C2C concatenated channels (S2D-processed and identity branches) back to C channels at width W, costing (2L⋅C2)O(2L· C^2), where the factor of two now comes from the doubled input channels rather than the woven width. Weaving and de-weaving only rearrange elements and add no parameters, costing just (L⋅C)O(L· C). Summing these terms, the 1×11×1 convolution dominates and the total cost is (L⋅C2)O(L· C^2)—linear in the number of tokens—so each WMF call remains efficient even at the high resolutions of WIDER FACE. 3.3 WeaveBiFPN: Fully Bidirectional Pyramid with WMF The top-down pathway propagates semantic context by upsampling, which interpolates each high-resolution position from its nearest low-resolution neighbors, smoothing out fine detail and producing blurred, imprecise features (feature aliasing [15]). Because a local fusion combines features only at nearby positions, it can only partially correct this imprecision. WMF instead draws on spatially distant context to correct it more fully. We therefore apply WMF at every top-down node, fusing each level with the upsampled feature from above: Pltd=WMF(ϕ(w¯1Plin),ϕ(w¯2Up(Pl+1td))),P_l^td=WMF\! (φ( w_1\,P_l^in),\;φ( w_2\,Up(P_l+1^td)) ), (7) where PltdP_l^td denotes the intermediate feature at level l on the top-down pathway, Up(⋅)Up(·) denotes bilinear upsampling, ϕ(⋅)φ(·) is the Swish activation, and w¯1 w_1 and w¯2 w_2 are learnable scalar weights normalized as in BiFPN [21]. In the bottom-up pathway, downsampling compresses each local region of the high-resolution map into a single low-resolution location. A small face that occupies only a few pixels is thus reduced to a single position and becomes hard to distinguish from the background. Local fusion alleviates this problem only partially, as each fusion step aggregates a small neighborhood of positions. WMF addresses this limitation by aggregating context from spatially distant positions within a single fusion step. We therefore apply WMF symmetrically at every bottom-up node, fusing each level with the downsampled feature from below: Plout=WMF(ϕ(w¯1′Pltd),ϕ(w¯2′Down(Pl−1out))),P_l^out=WMF\! (φ( w_1 \,P_l^td),\;φ( w_2 \,Down(P_l-1^out)) ), (8) where PloutP_l^out denotes the output feature on the bottom-up pathway and Down(⋅)Down(·) denotes parameter-free 3×33×3 max pooling with stride 2. In BiFPN, each intermediate output node also receives a same-level skip connection. We remove this skip connection so that every node fuses exactly two inputs, as WMF requires. At the pyramid boundaries, we set P4td=P4inP_4^td=P_4^in and P2out=P2tdP_2^out=P_2^td. 3.4 Auxiliary Modules and Detection Heads To form a complete WeaveFace, we combine WeaveBiFPN with widely used, lightweight components. CBAM Refinement. We apply CBAM [29] both before and after WeaveBiFPN, each with a residual connection over the input feature map F: Fout=ReLU(F+CBAM(F))F_out=ReLU(F+CBAM(F)). CBAM sequentially combines channel and spatial attention. Pre-CBAM suppresses background noise before it propagates into the S2D scan, while Post-CBAM re-emphasizes channels and regions already salient in the fused features. Detection Head. The detection head comprises a Cascade Context Prediction Module (CCPM) followed by a ShuffleNet block [35]. To enlarge the per-scale receptive field, we build on the three-branch CCPM of EResFD [11] but replace its standard convolutions with deformable convolutions (DCNv3 [27]), whose input-adaptive sampling supplies flexible intra-scale spatial context that complements WMF’s cross-scale global fusion. The subsequent ShuffleNet block (1×1→1×1→ depthwise 3×3→1×13×3→ 1×1, preceded by channel shuffling) promotes cross-group mixing at minimal cost. Three parallel 1×11×1 heads then produce classification scores, bounding-box regressions, and landmark localizations per anchor. Predictions from all pyramid levels are concatenated and supervised by the RetinaFace multi-task loss [6]: ℒ=ℒcls+λ1ℒbox+λ2ℒldm,λ1=2,λ2=1,L=L_cls+ _1L_box+ _2L_ldm, _1=2,\;\; _2=1, (9) with softmax cross-entropy for ℒclsL_cls and smooth-L1L_1 for the box and landmark regression terms. At inference, softmax scores and non-maximum suppression yield the final detections. 4 Experimental Results and Analysis 4.1 Experimental Setup Dataset. We evaluate WeaveFace on the WIDER FACE benchmark, which comprises 32,203 images and 393,703 annotated faces across three difficulty levels: Easy, Medium, and Hard. Since WIDER FACE lacks landmark ground truth, we train the landmark regression branch using the five-point annotations provided by RetinaFace [6]. Implementation Details. During training, input images are resized to 640×640640× 640 pixels. To improve generalization, we apply random square cropping, photometric distortion (brightness, contrast, saturation, and hue), horizontal flipping, zero-padding to a square shape, and mean subtraction. The model is optimized using AdamW with an initial learning rate of ×10−31\!×\!10^-3 and a OneCycleLR schedule. We train the network for 450 epochs with a batch size of 32 on a single NVIDIA RTX A6000 GPU. To ensure a fair comparison with prior works, FLOPs are calculated at a standard VGA resolution of 640×480640× 480. At inference, we adopt multi-scale testing, where input images are resized to multiple scales and the resulting predictions are merged via non-maximum suppression (NMS). The latency reported in Table 3 is averaged over 1000 runs at 640×480640× 480 resolution with batch size 1, following 50 warmup iterations, and measured with GPU–CPU synchronization enabled. Table 1: Comparison with state-of-the-art face detectors on the WIDER FACE validation set (multi-scale AP). Fusion denotes the operator family used for multi-scale feature fusion and context aggregation. Results are from original publications or the multi-scale evaluations in [9] (‡) and [11] (§). †: self-attention baseline (ResNet-50 backbone with a DETR Transformer encoder [3]). Model Backbone Neck Fusion Easy Medium Hard Avg. (%) #Params (M) GFLOPs DSFD‡ [13] ResNet-152 FEM Conv. 96.60 95.70 90.40 94.23 120.06 259.55 TinaFace‡ [37] ResNet-50 FPN Conv. 97.00 96.30 93.40 95.57 37.98 172.95 TransEnc-R50† ResNet-50 FPN Self-attention 93.03 92.89 88.56 91.49 33.72 32.92 RetinaFace‡ [6] ResNet-50 FPN Conv. 96.70 96.10 91.40 94.73 29.50 37.59 SCRFD-10GF [9] Basic Res PANet Conv. 95.93 94.95 90.81 93.90 3.86 9.98 FaceBoxes‡ [34] - - - 85.90 81.60 55.70 74.40 1.01 0.28 SCRFD-0.5GF [9] Depth-wise Conv PANet Conv. 92.71 91.45 86.23 90.13 0.57 0.51 RetinaFace Lite‡ [6] MobileNet0.25 FPN Conv. 91.40 89.20 82.50 87.70 0.44 0.80 FDLite [1] BLite FPN Conv. 92.31 89.91 82.30 88.17 0.24 0.94 MTCNN§ [33] - - - 85.10 82.00 60.70 75.93 0.12 0.01 EResFD [11] EResNet SepFPN Sep. Conv. 89.02 87.96 80.41 85.80 0.09 0.30 Ours EResNet WeaveBiFPN S2D 94.33 92.76 87.14 91.41 0.34 1.16 Table 2: Comparison on AFW and PASCAL Faces without fine-tuning, following the AP protocol of [19]. All models are trained solely on WIDER FACE. Bold: our model; underline: best result in each column. Method AFW AP (%) PASCAL AP (%) #Params MogFace [17] 99.85 99.32 85.26M EfficientSRFace-L [24] 99.94 98.84 18.84M SCRFD-0.5GF [9] 98.60 98.54 0.57M SCRFD-1.0GF [9] 99.70 98.60 0.64M SCRFD-2.5GF [9] 99.82 98.91 0.67M FaceBoxes [34] 98.91 96.30 1.01M WeaveFace (Ours) 99.50 98.77 0.34M 4.2 Quantitative Results Table 1 compares WeaveFace with state-of-the-art face detectors on the WIDER FACE validation set. Our results demonstrate that WeaveFace achieves a superior balance between computational efficiency and detection accuracy, effectively redefining the performance frontier in the lightweight regime. With only 0.34M parameters and 1.16 GFLOPs, WeaveFace achieves the highest AP across all difficulty levels among detectors in the sub-0.5M parameter range. It outperforms FDLite [1] by 2.02%/2.85%/4.84% on Easy/Medium/Hard, and surpasses EResFD [11], which shares the same EResNet backbone, by 6.73% on Hard. WeaveFace also offers a favorable accuracy-efficiency trade-off against heavier detectors. It surpasses SCRFD-0.5GF [9] on every subset with 40%40\% fewer parameters, and trails SCRFD-10GF by only 3.67%3.67\% on Hard despite using 11×11× fewer parameters and 9×9× fewer GFLOPs. With nearly 100×100× fewer parameters and over 28×28× fewer GFLOPs, it nearly matches the self-attention-based TransEnc-R50 in mean AP (91.41% vs. 91.49%). Table 2 assesses robustness beyond WIDER FACE. Despite having the fewest parameters among all compared methods, WeaveFace attains 99.50% AP on AFW and 98.77% AP on PASCAL Faces. It outperforms SCRFD-0.5GF [9] on both datasets (99.50% vs. 98.60% on AFW, 98.77% vs. 98.54% on PASCAL) while using 40% fewer parameters, and trails the much heavier MogFace [17] (85.26M parameters) by only 0.35% on AFW. These results indicate that WMF’s global cross-scale interaction generalizes to unseen data distributions rather than overfitting to WIDER FACE-specific patterns. 4.3 Qualitative Results Figure 4: Qualitative comparison between EResFD (left) and WeaveFace (right) on the WIDER FACE Hard subset. Red boxes represent successful detections. Yellow boxes indicate faces missed by EResFD but correctly identified by WeaveFace, while green boxes highlight false positives generated by EResFD that are effectively mitigated by WeaveFace. Fig. 4 provides a qualitative comparison between EResFD and the proposed WeaveFace on challenging images from the WIDER FACE Hard subset. Both models are evaluated under identical conditions: a confidence threshold of 0.5, an NMS IoU threshold of 0.4, and single-scale inference at a fixed input resolution (short side of 1600 pixels). EResFD struggles in crowded scenes with small faces and heavy occlusion, producing both missed detections (yellow) and false positives (green). WeaveFace recovers most of these missed faces while suppressing the false positives, indicating improved robustness in dense and occluded scenes. 4.4 Ablation Study We conduct ablation experiments on the WIDER FACE [31] dataset to evaluate the effectiveness of the proposed WMF: the contribution of the weaving mechanism itself, its generality across pyramid topologies, the design of the fusion operator together with the partial-channel divisor N, and the effect of the scan count s. Unless otherwise specified, our default WeaveFace configuration applies WMF within a WeaveBiFPN, utilizing a partial-channel divisor of N=2N=2 and a scan count of s=2s=2; this configuration serves as the baseline for all ablation studies (Table 1). All variants share the same EResNet backbone, CBAM attention, and CCPM detection head, and are trained under identical settings (Sec. 4.1). To ensure a rigorous evaluation, we isolate the component under study while keeping all other factors constant. Table 3: Comparison of different fusion operators on the BiFPN topology. We evaluate our proposed WMF against standard summation and cross-attention baselines. Bold: our model; underline: best result in each column; Lat.: latency. Fusion Easy Medium Hard Avg. #Params (M) FLOPs (G) Lat. (ms) Sum (w/ sep. conv.) 92.88 90.87 84.76 89.50 0.217 0.564 15.1 Cross-attention 93.47 91.55 86.46 90.49 0.577 11.228 23.7 WMF 94.33 92.76 87.14 91.41 0.344 1.159 23.3 Effectiveness of the Weaving Mechanism We compare WMF against two standard fusion baselines on the BiFPN topology (Table 3). Simple summation with a separable convolution reaches only 89.5089.50 mean AP, degrading sharply on the Hard subset (84.7684.76). Replacing summation with cross-attention improves accuracy at every difficulty level (90.4990.49 mean AP, +0.99+0.99 over Sum), but at a steep efficiency cost: FLOPs rise by nearly 20×20× (11.22811.228G vs. 0.5640.564G) and parameters more than double (0.5770.577M vs. 0.2170.217M). Our proposed WMF attains the best accuracy on every subset, reaching 91.4191.41 mean AP (+1.91+1.91 over Sum, +0.92+0.92 over cross-attention). Notably, WMF achieves this while using only 0.3440.344M parameters — roughly 40%40\% fewer than cross-attention — and 1.1591.159G FLOPs, nearly an order of magnitude (9.7×9.7×) less compute. Despite this large FLOP reduction, WMF’s latency (23.323.3 ms) is on par with, and in fact marginally lower than, cross-attention (23.723.7 ms), reflecting that the selective scan underlying WMF is memory-bandwidth-bound rather than compute-bound [7, 5]. Table 4: Generality of WMF across pyramid topologies. Each topology is paired with its standard fusion method (summation for FPN/BiFPN, concatenation for PANet) and the proposed WMF. Bold: WMF (ours); underline: best result in each column within each topology. Topology Fusion Easy Medium Hard Avg. #Params (M) FLOPs (G) FPN Sum 91.17 89.31 83.81 88.10 0.229 0.672 WMF 91.47 89.72 84.12 88.44 0.187 0.639 PANet Concat 92.33 90.35 84.88 89.19 0.562 1.071 WMF 92.75 91.08 85.03 89.62 0.329 0.751 BiFPN Sum 92.88 90.87 84.76 89.50 0.217 0.564 WMF 94.33 92.76 87.14 91.41 0.344 1.159 Generality across Pyramids Table 4 replaces the standard fusion of FPN (summation), PANet (concatenation), and BiFPN (summation) with WMF, holding N=2N=2, s=2s=2 fixed across all three pyramids. WMF improves mean AP on every topology, and does so without added cost on FPN and PANet: mean AP rises by +0.34+0.34 on FPN and +0.43+0.43 on PANet while using 18–41% fewer parameters and 5–30% fewer FLOPs. The gain is markedly larger on BiFPN (+1.91+1.91 mean AP), though here WMF comes with more parameters (0.217M→0.344M0.217M→ 0.344M) and FLOPs (0.564G→1.159G0.564G→ 1.159G). Adding parameters to the BiFPN fusion without weaving, however, yields a far smaller gain, indicating that the improvement stems from the weaving mechanism itself rather than the added capacity. BiFPN’s bidirectional, weighted aggregation thus benefits the most from weaving, motivating our WeaveBiFPN neck. Table 5: Accuracy–efficiency trade-offs across different scale-combination schemes (summation, concatenation, and weaving) as the partial-channel divisor N increases. N determines the extent of lightweighting in S2D, where only 1/N1/N of channels are processed; thus, a larger N reduces model parameters and FLOPs at the expense of accuracy. Bold: our final configuration; underline: best result in each column. Fusion Operator N Easy Medium Hard Avg. #Params (M) FLOPs (G) Sum + S2D 1 93.27 91.54 85.40 90.07 0.409 1.152 2 91.71 89.82 83.85 88.46 0.245 0.711 4 90.18 88.13 81.20 86.50 0.193 0.566 Concat + S2D 1 95.14 93.79 88.40 92.44 1.080 2.649 2 93.49 91.57 85.81 90.29 0.510 1.350 4 93.22 91.00 84.47 89.56 0.344 0.925 Weave + S2D (ours) 1 94.37 93.05 88.10 91.84 0.509 1.963 2 94.33 92.76 87.14 91.41 0.344 1.159 4 93.17 91.57 86.20 90.31 0.292 0.870 Fusion Operator and Partial-Channel Design Table 5 varies both the fusion operator (summation, concatenation, weaving) and the partial-channel divisor N, which controls how aggressively the channels are reduced for lightweighting. At full channel width (N=1N=1), concatenation attains the highest mean AP (92.4492.44), but only by roughly doubling the parameters of weaving for a +0.60+0.60 gain. This ranking reverses as channels are reduced. Weaving is markedly more robust to channel reduction: halving the channels costs weaving only 0.430.43 AP, versus 2.152.15 for concatenation and 1.611.61 for summation, and the gap persists at the 1/41/4 ratio. The reason is structural. Summation and concatenation merge the two scales into the channel dimension before scanning, so reducing channels discards information from both scales simultaneously. Weaving instead interleaves the scales along the spatial width, so even a reduced channel subset still carries a representation of both scales. As a result, weaving outperforms concatenation at every reduced ratio while using fewer parameters and FLOPs (91.4191.41 vs. 90.2990.29 at N=2N=2). We adopt N=2N=2 as our default: it comes within 1.031.03 AP of full-channel concatenation while using only a third of its parameters and less than half of its FLOPs, making it the most favorable point on the accuracy–efficiency trade-off. Table 6: Effect of the scan count s in WeaveBiFPN, where s denotes the number of scan directions utilized by S2D. Bold: our final configuration (s=2s=2); underline: best result in each column. s Easy Medium Hard Avg. #Params (M) FLOPs (G) 1 93.87 92.07 86.51 90.82 0.323 0.991 2 94.33 92.76 87.14 91.41 0.344 1.159 4 94.01 92.52 87.06 91.20 0.386 1.496 Effect of Scan Count With N=2N=2 fixed, Table 6 varies the scan count s — the number of scan directions, from forward horizontal (s=1s=1) to fully bidirectional along both axes (s=4s=4). Adding the backward horizontal pass (s=1→2s=1→ 2) reinforces the cross-scale mixing already induced by weaving, improving mean AP. The further vertical scans (s=2→4s=2→ 4) add only intra-scale context and bring no additional gain. We therefore adopt s=2s=2, reaching 91.4191.41 mean AP at 0.3440.344M parameters and 1.1591.159G FLOPs. 5 Conclusion In this paper, we studied multi-scale feature fusion for lightweight face detection and identified the fusion operator itself as the core limitation of pyramid-based detectors: combining two scales with summation or concatenation, followed by local convolutions, only mixes features at nearby positions and misses distant cross-scale context. To overcome this, we proposed Weave Mamba Fusion (WMF), which weaves two adjacent feature maps column by column and processes them with a 2D Selective Scan, so that the scan mixes the two scales at every step and achieves global cross-scale interaction. A partial-channel design applies S2D to only 1/N1/N of the channels, cutting its dominant cost to roughly 1/N21/N^2 and letting us place WMF at every node of a high-resolution pyramid. Placing WMF in a fully bidirectional pyramid yields WeaveBiFPN, on which we build WeaveFace. On WIDER FACE, WeaveFace attains a 91.41% mean AP with only 0.34M parameters and 1.16 GFLOPs, the most accurate detector with fewer than 0.5M parameters, with its largest gains on the Hard subset. Limitations The primary trade-off of WeaveFace lies in its latency. While WeaveFace achieves superior accuracy compared to summation-based baselines, it exhibits higher inference latency (23.3 ms vs. 15.1 ms). This is primarily attributed to the memory-bandwidth-bound nature of the selective scan operation [7, 5], which limits the performance gains achievable solely through FLOPs reduction. We acknowledge that the current implementation does not fully exploit hardware parallelism. In future work, we aim to integrate the Mamba-2 [5] formulation, which reframes scanning as matrix multiplications, and further optimize inference by fusing the weaving step directly into the scan kernel to mitigate this overhead. References [1] Y. Aggarwal and P. Guha (2024) FDLite: a single stage lightweight face detector network. arXiv preprint arXiv:2406.19107. Cited by: §2.1, §4.2, Table 1. [2] A. Bochkovskiy, C. Wang, and H. M. Liao (2020) Yolov4: optimal speed and accuracy of object detection. arXiv preprint arXiv:2004.10934. Cited by: §1. [3] N. Carion, F. Massa, G. Synnaeve, N. Usunier, A. Kirillov, and S. Zagoruyko (2020) End-to-end object detection with transformers. In European conference on computer vision, p. 213–229. Cited by: Table 1, Table 1. [4] J. Chen, S. Kao, H. He, W. Zhuo, S. Wen, C. Lee, and S.-H. G. Chan (2023) Run, don’t walk: chasing higher flops for faster neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 12021–12031. Cited by: §1. [5] T. Dao and A. Gu (2024) Transformers are ssms: generalized models and efficient algorithms through structured state space duality. arXiv preprint arXiv:2405.21060. Cited by: §4.4, §5. [6] J. Deng, J. Guo, E. Ververas, I. Kotsia, and S. Zafeiriou (2020) Retinaface: single-shot multi-level face localisation in the wild. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 5203–5212. Cited by: Appendix 0.B, §2.1, §3.4, §4.1, Table 1, Table 1. [7] A. Gu and T. Dao (2023) Mamba: linear-time sequence modeling with selective state spaces. arXiv preprint arXiv:2312.00752. Cited by: §1, §2.3, §3.1, §4.4, §5. [8] A. Gu, K. Goel, and C. Ré (2021) Efficiently modeling long sequences with structured state spaces. arXiv preprint arXiv:2111.00396. Cited by: §2.3, §3.1. [9] J. Guo, J. Deng, A. Lattas, and S. Zafeiriou (2021) Sample and computation redistribution for efficient face detection. arXiv preprint arXiv:2105.04714. Cited by: §2.1, §4.2, §4.2, Table 1, Table 1, Table 1, Table 1, Table 2, Table 2, Table 2. [10] A. Hatamizadeh and J. Kautz (2025) Mambavision: a hybrid mamba-transformer vision backbone. In Proceedings of the Computer Vision and Pattern Recognition Conference, p. 25261–25270. Cited by: §2.3. [11] J. Jeong, B. Kim, J. Yu, and Y. Yoo (2024) EResFD: rediscovery of the effectiveness of standard convolution for lightweight face detection. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, p. 988–998. Cited by: §2.1, §3.4, §3, §4.2, Table 1, Table 1, Table 1. [12] D. Kim, J. Jung, and J. Kim (2025) Featherface: robust and lightweight face detection via optimal feature integration. Electronics 14 (3), p. 517. Cited by: §2.1. [13] J. Li, Y. Wang, C. Wang, Y. Tai, J. Qian, J. Yang, C. Wang, J. Li, and F. Huang (2019) DSFD: dual shot face detector. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 5060–5069. Cited by: §2.1, Table 1. [14] J. Li, Y. Shi, Z. Ruan, N. Guo, and J. Xu (2026) Beyond mamba: enhancing state-space models with deformable dilated convolutions for multi-scale traffic object detection. arXiv preprint arXiv:2604.08038. Cited by: §2.3. [15] T. Lin, P. Dollár, R. Girshick, K. He, B. Hariharan, and S. Belongie (2017) Feature pyramid networks for object detection. In Proceedings of the IEEE conference on computer vision and pattern recognition, p. 2117–2125. Cited by: §1, §2.2, §3.3. [16] S. Liu, L. Qi, H. Qin, J. Shi, and J. Jia (2018) Path aggregation network for instance segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Cited by: §1, §2.2. [17] Y. Liu, F. Wang, J. Deng, Z. Zhou, B. Sun, and H. Li (2022) Mogface: towards a deeper appreciation on face detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 4093–4102. Cited by: §2.1, §4.2, Table 2. [18] Y. Liu, Y. Tian, Y. Zhao, H. Yu, L. Xie, Y. Wang, Q. Ye, J. Jiao, and Y. Liu (2024) Vmamba: visual state space model. Advances in neural information processing systems 37, p. 103031–103063. Cited by: §1, §2.3, §3.1. [19] M. Mathias, R. Benenson, M. Pedersoli, and L. Van Gool (2014) Face detection without bells and whistles. In European conference on computer vision, p. 720–735. Cited by: Table 2, Table 2. [20] S. Peng, T. Zhang, L. Gao, X. Zhu, H. Zhang, K. Pang, and Z. Lei (2025) Wmamba: wavelet-based mamba for face forgery detection. In Proceedings of the 33rd ACM International Conference on Multimedia, p. 4768–4777. Cited by: §2.3. [21] M. Tan, R. Pang, and Q. V. Le (2020) EfficientDet: scalable and efficient object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Cited by: Appendix 0.C, §1, §2.2, §3.3. [22] X. Tang, D. K. Du, Z. He, and J. Liu (2018) Pyramidbox: a context-assisted single shot face detector. In Proceedings of the European conference on computer vision (ECCV), p. 797–813. Cited by: §2.2. [23] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin (2017) Attention is all you need. Advances in neural information processing systems 30. Cited by: §1. [24] G. Wang, J. Li, J. Xie, J. Xu, and B. Yang (2023) EfficientSRFace: an efficient network with super-resolution enhancement for accurate face detection. In Asian Conference on Pattern Recognition, p. 74–87. Cited by: §2.1, Table 2. [25] J. Wang, Y. Yuan, and G. Yu (2017) Face attention network: an effective face detector for the occluded faces. arXiv preprint arXiv:1711.07246. Cited by: §2.2. [26] L. Wang, D. Li, S. Dong, X. Meng, X. Zhang, and D. Hong (2025) PyramidMamba: rethinking pyramid feature fusion with selective space state model for semantic segmentation of remote sensing imagery. International Journal of Applied Earth Observation and Geoinformation 144, p. 104884. Cited by: §2.3. [27] W. Wang, J. Dai, Z. Chen, Z. Huang, Z. Li, X. Zhu, X. Hu, T. Lu, L. Lu, H. Li, et al. (2023) Internimage: exploring large-scale vision foundation models with deformable convolutions. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 14408–14419. Cited by: §3.4. [28] Z. Wang, C. Li, H. Xu, X. Zhu, and H. Li (2025) Mamba yolo: a simple baseline for object detection with state space model. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 39, p. 8205–8213. Cited by: §2.3. [29] S. Woo, J. Park, J. Lee, and I. S. Kweon (2018) Cbam: convolutional block attention module. In Proceedings of the European conference on computer vision (ECCV), p. 3–19. Cited by: §3.4, §3. [30] G. Yang, J. Lei, Z. Zhu, S. Cheng, Z. Feng, and R. Liang (2023) AFPN: asymptotic feature pyramid network for object detection. arXiv preprint arXiv:2306.15988. Cited by: §2.2. [31] S. Yang, P. Luo, C. Loy, and X. Tang (2016) Wider face: a face detection benchmark. In Proceedings of the IEEE conference on computer vision and pattern recognition, p. 5525–5533. Cited by: §1, §4.4. [32] D. Zhang, H. Zhang, J. Tang, M. Wang, X. Hua, and Q. Sun (2020) Feature pyramid transformer. In European conference on computer vision, p. 323–339. Cited by: §2.2. [33] K. Zhang, Z. Zhang, Z. Li, and Y. Qiao (2016) Joint face detection and alignment using multitask cascaded convolutional networks. IEEE signal processing letters 23 (10), p. 1499–1503. Cited by: §2.1, Table 1. [34] S. Zhang, X. Zhu, Z. Lei, H. Shi, X. Wang, and S. Z. Li (2017) Faceboxes: a cpu real-time face detector with high accuracy. In 2017 IEEE International Joint Conference on Biometrics (IJCB), p. 1–9. Cited by: Table 1, Table 2. [35] X. Zhang, X. Zhou, M. Lin, and J. Sun (2018) Shufflenet: an extremely efficient convolutional neural network for mobile devices. In Proceedings of the IEEE conference on computer vision and pattern recognition, p. 6848–6856. Cited by: §3.4. [36] L. Zhu, B. Liao, Q. Zhang, X. Wang, W. Liu, and X. Wang (2024) Vision mamba: efficient visual representation learning with bidirectional state space model. arXiv preprint arXiv:2401.09417. Cited by: §1, §2.3. [37] Y. Zhu, H. Cai, S. Zhang, C. Wang, and Y. Xiong (2020) Tinaface: strong but simple baseline for face detection. arXiv preprint arXiv:2011.13183. Cited by: §2.1, Table 1. Appendix 0.A Generality across Backbones To investigate whether WMF is agnostic to the choice of backbone, we fix the WeaveBiFPN neck (N=2N=2, s=2s=2), CBAM, and CCPM head, while varying only the backbone architecture. Specifically, we evaluate two lightweight models (EResNet and MobileNetV1-0.25) and two heavy models (ResNet-50 and ResNet-152). All backbones are initialized with ImageNet-pretrained weights and trained under identical settings to the main model. As shown in Table 7, MobileNetV1-0.25 achieves slightly higher accuracy than EResNet; however, it incurs 1.6×1.6× more parameters and violates the sub-0.5M budget constraint targeted in this work, making EResNet our default choice. When scaling up further, ResNet-152 more than doubles the parameter count of ResNet-50 without yielding performance gains, indicating that accuracy saturates around the capacity of ResNet-50 under our training recipe. Overall, these results demonstrate that the proposed WeaveBiFPN neck trains stably and scales consistently across backbones of varying capacities without requiring any backbone-specific tuning. Table 7: WeaveFace with different backbones on the WIDER FACE validation set. The WeaveBiFPN neck (N=2N=2, s=2s=2), CBAM, and CCPM head are fixed, and only the backbone is changed. The same fusion design generalizes from lightweight to heavy backbones. Bold: our default backbone. Backbone Easy Medium Hard Avg. #Params (M) FLOPs (G) EResNet 94.33 92.76 87.14 91.41 0.344 1.159 MobileNetV1-0.25 95.06 93.64 88.52 92.41 0.566 1.231 ResNet-50 96.29 95.44 91.20 94.31 29.756 36.275 ResNet-152 96.33 95.44 91.15 94.31 64.392 81.871 Appendix 0.B Anchor Settings As shown in Table 8, two anchors with a 1:1 aspect ratio are assigned to every spatial location of the three pyramid levels P2P_2–P4P_4, following the anchor design of RetinaFace [6]. The six anchor sizes double from 16 to 512 pixels, and the pixel sizes of the anchors remain fixed across input resolutions. For label assignment and hard negative mining, we follow the matching strategy and loss formulation of RetinaFace. Table 8: Anchor settings of WeaveFace. Feature-map sizes are given for the 640×640640× 640 training resolution. Level Stride Feature map Anchor sizes (px) P2P_2 8 80×8080× 80 16, 32 P3P_3 16 40×4040× 40 64, 128 P4P_4 32 20×2020× 20 256, 512 Appendix 0.C Generality to Generic Object Detection To verify that WMF is not limited to face detection, we integrate it into the BiFPN neck of EfficientDet [21] and evaluate on two generic object detection benchmarks, COCO val2017 and PASCAL VOC. The backbone, detection head, and training settings are kept identical to the original EfficientDet, so the only difference between the two variants is the addition of WMF. No face-specific components (e.g., CBAM, CCPM, or landmark regression) are used. Table 9: Results on COCO val2017 and PASCAL VOC datasets. Only WMF is added to the BiFPN neck of EfficientDet; all other settings are unchanged. COCO val2017 PASCAL VOC Variant AP AP50 AP75 Params (M) FLOPs (G) mAP Params (M) FLOPs (G) EfficientDet (baseline) 34.5 52.9 36.6 3.88 2.57 76.90 3.84 2.35 EfficientDet + WMF 36.2 53.6 38.2 4.25 3.21 78.63 4.21 2.99 As reported in Table 9, WMF improves the COCO AP from 34.5 to 36.2, yielding a performance gain of +1.7+1.7 points. Notably, the improvement is more pronounced at the stricter IoU threshold (+1.6+1.6 in AP75AP_75) compared to the looser one (+0.7+0.7 in AP50AP_50), suggesting that WMF primarily enhances localization precision rather than merely increasing the number of detections. Similarly, on the PASCAL VOC benchmark, WMF elevates mAP from 76.90 to 78.63 (+1.73+1.73). Across both tasks, this performance boost is achieved with a marginal overhead of approximately 0.37M parameters and 0.64–0.71 GFLOPs, where the slight parameter variation stems from the differing number of classes in the detection head. These findings demonstrate that the benefits of WMF generalize beyond face detection, successfully transferring to generic multi-class object detection.