Paper deep dive
On the Reliability of Cue Conflict and Beyond
Pum Jun Kim, Seung-Ah Lee, Seongho Park, Dongyoon Han, Jaejun Yoo
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 95%
Last extracted: 3/22/2026, 6:18:27 AM
Summary
The paper identifies critical reliability issues in the 'cue-conflict' benchmark for neural network bias diagnosis, specifically regarding stylization-based cue entanglement, imbalanced informativeness, and restricted label evaluation. It introduces REFINED-BIAS, a new framework that uses human-perceptual definitions of shape and texture, balanced cue pairs, and a ranking-based metric (MRR) evaluated over the full label space to provide more faithful and interpretable bias diagnosis.
Entities (4)
Relation Signals (2)
REFINED-BIAS → addresseslimitationsof → Cue-conflict benchmark
confidence 95% · REFINED-BIAS enables fairer cross-model comparison... resolving inconsistencies that prior cue-conflict evaluations could not reliably disambiguate.
REFINED-BIAS → usesmetric → Mean Reciprocal Rank
confidence 95% · measures cue-specific sensitivity over the full label space via a ranking-based metric
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Understanding how neural networks rely on visual cues offers a human-interpretable view of their internal decision processes. The cue-conflict benchmark has been influential in probing shape-texture preference and in motivating the insight that stronger, human-like shape bias is often associated with improved in-domain performance. However, we find that the current stylization-based instantiation can yield unstable and ambiguous bias estimates. Specifically, stylization may not reliably instantiate perceptually valid and separable cues nor control their relative informativeness, ratio-based bias can obscure absolute cue sensitivity, and restricting evaluation to preselected classes can distort model predictions by ignoring the full decision space. Together, these factors can confound preference with cue validity, cue balance, and recognizability artifacts. We introduce REFINED-BIAS, an integrated dataset and evaluation framework for reliable and interpretable shape-texture bias diagnosis. REFINED-BIAS constructs balanced, human- and model- recognizable cue pairs using explicit definitions of shape and texture, and measures cue-specific sensitivity over the full label space via a ranking-based metric, enabling fairer cross-model comparisons. Across diverse training regimes and architectures, REFINED-BIAS enables fairer cross-model comparison, more faithful diagnosis of shape and texture biases, and clearer empirical conclusions, resolving inconsistencies that prior cue-conflict evaluations could not reliably disambiguate.
Tags
Links
- Source: https://arxiv.org/abs/2603.10834v2
- Canonical: https://arxiv.org/abs/2603.10834v2
Trouble viewing inline? Open PDF directly →
Full Text
85,234 characters extracted from source content.
Expand or collapse full text
On the Reliability of Cue Conflict and Beyond Pum Jun Kim 1∗ Seung-Ah Lee 1∗ Seongho Park 2 Dongyoon Han 3† Jaejun Yoo 1† 1 Ulsan National Institute of Science and Technology 2 College of Medicine, Hanyang University 3 NAVER AI Lab pumjun.kim, seungah.lee, jaejun.yoo@unist.ac.kr risepsh@gmail.comdongyoon.han@navercorp.com Abstract Understanding how neural networks rely on visual cues offers a human- interpretable view of their internal decision processes. The cue-conflict benchmark has been influential in probing shape-texture preference and in motivating the insight that stronger, human-like shape bias is often associated with improved in-domain performance. However, we find that the current stylization-based instan- tiation can yield unstable and ambiguous bias estimates. Specifically, stylization may not reliably instantiate perceptually valid and separable cues nor control their relative informativeness, ratio-based bias can obscure absolute cue sensitivity, and restricting evaluation to preselected classes can distort model predictions by ignoring the full decision space. Together, these factors can confound prefer- ence with cue validity, cue balance, and recognizability artifacts. We introduce REFINED-BIAS, an integrated dataset and evaluation framework for reliable and interpretable shape–texture bias diagnosis.REFINED-BIASconstructs balanced, human- and model- recognizable cue pairs using explicit definitions of shape and texture, and measures cue-specific sensitivity over the full label space via a ranking- based metric, enabling fairer cross-model comparisons. Across diverse training regimes and architectures,REFINED-BIASenables fairer cross-model comparison, more faithful diagnosis of shape and texture biases, and clearer empirical con- clusions, resolving inconsistencies that prior cue-conflict evaluations could not reliably disambiguate. Our code is publicly available at REFINED-BIAS. 1 Introduction (a) Core Insight of Cue-conflict Human alignment improves model performance Human Model Texture Shape In - domain Acc (c) Conflicting Conclusions in Recent Studies Lonnqvist et al. (2025) Gavrikov and Keuper et al. (2024) In - domain Acc Shape-bias In - domain Acc Shape-bias Alignment Does the model TRULY reflect PREFERENCE? (b) Unrecognizable Cues in Cue-conflict Airplane textureCar textureBoat shapeBear shape Figure 1: Empirical instability of stylized image-based bias evaluation. (a) illustrates the core insight of cue-conflict that stronger shape bias, similar to humans, improves in-domain performance. (b) shows examples of unrecognizable cues in the cue-conflict dataset. (c) illustrates conflicting findings on this core insight of cue-conflict. Do neural networks (NNs) exhibit biases in human-like perception, and can we diagnose them in cognitively meaningful terms? Addressing these questions is important for bridging machine * Equal contributions. † Corresponding authors. Preprint. arXiv:2603.10834v2 [cs.CV] 12 Mar 2026 (a) Confounded Cue Shape 70 Shape TextureShape 5050 Shape Well balanced Imbalanced 70 Texture 30 ✓ Pick! ✓ Pick! (c) Insensitive Metric Shape Texture 8% 2% 80% 20% Accuracy + Bias # correct shape # correct texture = vs. Equally-biased? Preselected Classes (d) Restricted Classes 0.60 0.33 Ranked logits : True prediction : Meaningless predictions 0.06 0.002 ⋯ Shape Texture + Shape from Texture (b) Information Imbalance Figure 2: Limitations of the cue-conflict benchmark: Although it has offered a valuable and well- designed framework for studying shape and texture biases, we argue that several limitations warrant further attention: (a) cue entanglement caused by stylization, where shape and texture information leak into each other, (b) imbalanced cue information caused by stylization, leading to unfair predictive contributions, (c) ignoring differences in cue sensitivity, which prevents distinguishing models with genuine biases, and (d) evaluation restricted to preselected classes. and human vision, revealing how models internalize visual information and providing a basis for more human-like systems [1–6]. Accordingly, a growing line of work has developed diagnostic benchmarks [7–12] that evaluate model biases through a human-perceptual lens. Among them, the cue-conflict benchmark [7] remains the de facto standard and the most widely adopted framework for bias analysis, consistently used in recent studies [2, 4, 13–21]. The cue-conflict benchmark probes cue preference using stylized cue-conflict images that combine the shape of one class with the texture of another. It introduced a core insight that has been highly influential: a human-like strong shape bias is associated with higher in-domain performance (Fig. 1a). While this idea remains highly valuable, we find that the current stylization-based instantiation yields empirically unstable evaluations. Many cue-conflict images are difficult to interpret even at the stimulus level: the intended shape or texture cue is often weak, visually mixed, or hard to recognize for both human and machine (Fig. 1b). Such ambiguity helps explain conflicting conclusions in recent studies on whether shape or texture drives performance [15,17,22,23] (Fig. 1c), and why the benchmark can fail to reflect explicitly shape-inducing training strategies. These observations raise fundamental reliability questions for stylized cue-conflict evaluation: do cue-conflict images actually instantiate perceptually valid and separable shape and texture signals, are the two cues mixed with controlled and comparable informativeness, and are the resulting cues reliably recognizable for both humans and models? If not, measured “preference” can be confounded by cue construction artifacts rather than reflecting genuine perceptual bias. Motivated by this concern, we identify key limitations of the current cue-conflict instantiation that undermine precise bias diagnosis (Fig. 2). On the construction side, stylization operationalizes shape and texture through model-dependent features, which can yield imperfect cue separation and perceptual impurity (Fig. 2a), and provides no mechanism to control cue mixing ratios, often producing imbalanced cue informativeness (Fig. 2b). On the evaluation side, the ratio-based bias score obscures cue sensitivity, making models with vastly different absolute cue utilization appear similar (Fig. 2c), and restricting evaluation to a small subset of labels can distort model predictions, obscuring cue usage in the full decision space (Fig. 2d). To address these issues, we introduceREFINED-BIAS, a reliable framework for integrated evaluation and a disentangled benchmark of interpretable alignment of shape and texture. We define shape as globally and locally coherent geometric structure and texture as scale-consistent repetitive patterns [8, 24], and construct balanced, non-overlapping cue pairs from classes that are clearly recognizable to both humans and models. Building on this dataset, we further propose a cue-specific sensitivity framework evaluated in the full label space using a ranking-based metric, enabling preference to be interpreted alongside absolute sensitivity. REFINED-BIASrestores diagnostic faithfulness in three ways. (1) It consistently reflects the effects of shape-focused learning strategies on shape-related behavior, whereas cue-conflict exhibits only partial alignment. (2) It separates cue sensitivity from relative preference, allowing fairer cross-model comparison beyond ratio-based bias. (3) It supports more reliable conclusions about how shape and texture sensitivities jointly relate to model performance. As a result,REFINED-BIASenables clearer empirical insights, including architecture-dependent shape-texture trade-offs and the role of local-to-global mechanisms in improving shape understanding, while resolving prior conflicting conclusions that the original cue-conflict benchmark could not reliably disambiguate. 2 -0.3 -0.2 0.5 0.4 0.3 0.2 0.1 0 -0.1 Correlation -0.3 -0.2 0.5 0.4 0.3 0.2 0.1 0 -0.1 Correlation -0.3 -0.2 0.5 0.4 0.3 0.2 0.1 0 -0.1 Correlation -0.3 -0.2 0.5 0.4 0.3 0.2 0.1 0 -0.1 Correlation Shape cue correlates with local shape, but not with global shape Texture cue shows spurious correlations with pure shape stimuli ✘ 휌(shape content,silhouette) ✘ 휌(shape content,edge) 휌(stylized texture,silhouette) ✓ 휌(stylized texture,edge) ✘ InceptionV3BG - 9 BG - 17 BG - 33 ANVGG11VGG13VGG16RN18VGG19RN34RN50RN101RN152ViT - S ViT - B CMT - S Swin - B CMT - B Swin - S InceptionV3 BG - 9 BG - 17 BG - 33 AN VGG11VGG13VGG16 RN18 VGG19 RN34RN50 RN101RN152 ViT - S ViT - B CMT - S Swin - B CMT - B Swin - S InceptionV3 BG - 9 BG - 17 BG - 33 AN VGG11VGG13VGG16 RN18 VGG19 RN34RN50 RN101RN152 ViT - S ViT - B CMT - S Swin - B CMT - B Swin - S InceptionV3 BG - 9 BG - 17 BG - 33 AN VGG11VGG13VGG16 RN18 VGG19 RN34RN50 RN101RN152 ViT - S ViT - B CMT - S Swin - B CMT - B Swin - S + EdgeSilhouette + EdgeSilhouette + EdgeSilhouette (b) Correlation with Pure Shape Stimuli(a) Examples of Imperfect Cue Separation ShapeTextureShapeTextureShapeTexture Figure 3: Examples of imperfect cue separation in the cue-conflict dataset. (a) Qualitative examples of ambiguously extracted shape and texture cues. (b) Kendall’s rank correlation of class-wise model top-1 accuracy on stylized cues and pure shape stimuli. All ImageNet-1k pretrained CNN and ViT models listed in Appendix B are utilized. 2 Revisiting Cue Preference Benchmarking Before introducing our benchmark, we first revisit how cue preference has been measured in prior work and why the widely adopted cue-conflict paradigm requires careful re-examination. 2.1 The Cue-conflict Paradigm The cue-conflict benchmark [7] measures shape-texture bias using conflicting-cue images that combine the shape of one class with the texture of another through image stylization [25]. Since its introduction, it has become the de facto benchmark for studying cue preference. Importantly, however, this stylization-based design was adopted as a pragmatic solution rather than the only valid way to measure cue preference. In the original study, Geirhos et al.[7]initially considered more purified stimuli, such as sketches, to isolate visual cues more explicitly. Yet these introduced a substantial domain shift for standard CNNs. Stylization was therefore adopted as a practical workaround that preserved more natural image statistics while still inducing cue conflict. Under this implementation, shape and texture cues are operationalized through a stylization model: shape is associated with content features, and texture with style statistics such as Gram-matrix matching. Based on 16 ImageNet superclasses [26], the benchmark contains 1,280 cue-conflict images constructed from 160 shape-source images and 48 texture-source images. Not all possible combinations of shape and texture are used. Instead, a subset is selected from 7,680 (160× 48) candidate pairs, and some source images are reused more frequently than others. Shape bias is then computed asn s /(n t + n s ), wheren s andn t denote correctly predicted shape and texture labels, respectively, and similarly for texture. While useful in practice, its current instantiation introduces several limitations that complicate reliable bias interpretation. 2.2 Limitations of the Current Cue-conflict Instantiation In the following subsections, we describe several limitations of the current cue-conflict instantiation. 2.2.1 Problem 1. Stylization Undermines Cue Reliability The cue-conflict benchmark presumes that stylization yields two perceptually meaningful and separa- ble signals: shape (content) and texture (style). However, this operationalization is defined by the stylization model’s internal features rather than by human perceptual criteria, and the resulting cues need not align with the intended “pure” shape and texture. We test this assumption by comparing predictions on cue-conflict images against predictions on purified shape stimuli (silhouettes for global shape, edges for local shape). If content is a faithful shape proxy, it should correlate with both silhouette and edge performance, while style should not. Instead, as shown in Fig. 3b, content correlates mainly with edge-based signals and fails to track holistic shape, whereas stylized texture exhibits non-trivial correlations with shape proxies, indicating 3 HumanModel Accuracy on ShapeModel Accuracy on Texture Texture Classes 1.0 0.2 Accuracy 0.6 0.8 0.4 0.0 Uninformative✘Uninformative✘ Shape Classes 1.0 0.2 Accuracy 0.6 0.8 0.4 0.0 Informative ✓ Uninformative✘ Informative = Contains sufficiently distinctive cue features Distinctive Texture (b) Difficulty in Human/Model Recognition of Cues in Cue-conflict(a) Unequal Informativeness in Shape and Texture Cues Non-distinctive Texture (Shape > Texture)(Shape < Texture) + + ShapeTextureShapeTexture ++ ShapeTextureShapeTexture Figure 4: Unequal recognizability of cues in the cue-conflict dataset. (a) Qualitative examples of unequal informativeness, and (b) human and model perception trends, on shape and texture cues. Model top-1 accuracies are shown with bars indicating 95% confidence intervals. All ImageNet-1k pretrained CNN and ViT models listed in Appendix B are utilized. See §3.1 for human studies. leakage and imperfect disentanglement. Qualitative examples further suggest that shape-like structure can remain visible in texture cues (Fig. 3a). Together, these results imply that the current stylization pipeline does not consistently instantiate clean cue separation. Even if such disentanglement were approximately valid, preference measurement requires shape and texture cues to be mixed in an equal ratio (50:50) to measure preferences fairly [12, 27]. Otherwise, the resulting bias score may reflect cue imbalance rather than genuine preference. Yet, stylization offers no explicit control over the relative contribution of shape and texture, and in practice, one cue often dominates the other. As shown in Fig. 4a, a keyboard shape blended with an elephant texture contains more texture than shape, making the shape unrecognizable. Similarly, a bear shape mixed with a clock texture is not visible. These two issues jointly lead to a third problem: the resulting cue-conflict images are often difficult even for humans to recognize in terms of both source cues. If both cues were represented faithfully, humans and models should be able to identify both the underlying shape and the transferred texture with reasonably high accuracy. Instead, as shown in Fig. 3a and Fig. 4 (see also Appendix A.6 and A.7), both human and model recognition are substantially imbalanced, with some texture classes being consistently difficult to distinguish and only a small subset remaining highly recognizable. Collectively, the current construction can confound preference with cue validity, cue balance, and recognizability artifacts. 2.2.2 Problem 2. Relative Bias Obscures Cue Sensitivity Cue-conflict reports bias as a relative ratio between correct shape and texture predictions. While such a ratio can indicate directional preference, it does not capture how strongly a model actually uses either cue. For example, a model with 8% shape accuracy and 2% texture accuracy yields the same relative ratio as a model with 80% and 20%, despite the latter being far more sensitive to both cues. Thus, the relative metric conflates directional preference with absolute cue sensitivity. This limitation is particularly problematic when the metric is used to compare models or track development progress. An increase in shape-bias score does not necessarily mean that the model has become more shape- sensitive; it may simply indicate that texture sensitivity has deteriorated more severely. Therefore, relative preference can be informative only when interpreted alongside cue-specific sensitivity, rather than as a standalone measure of cue utilization. 2.2.3 Problem 3. Restricted Label Evaluation Distorts Model Predictions RankClassLogit Top-10.60 Top-20.33 Top-30.06 (a) True Model Prediction = , Preselected classes RankClassLogit Top-10.60 Top-20.33 Top-30.06 (b) Distorted Model Prediction : Groundtruth It looks like ! RestrictedLabel Evaluation Distorts Model Predictions : Top prediction It looks like a , but not in the class set So, it is a . Figure 5: Illustration of the difference between (a) true model prediction and (b) distorted model prediction. See Appendix A.10 for more details. A further limitation arises from the way predic- tions are evaluated in a restricted label space. Reliable analysis should be conducted in the full decision space in which the model was trained and performs inference. However, cue-conflict evaluation considers only a predefined subset of candidate labels, typically the shape and texture classes of the constructed image. This filtering can distort the model’s actual prediction. For example, if a model’s top-1, top-2, and top-3 predictions are “rabbit”, “cat”, and “dog”, re- 4 spectively, the true model prediction is “rabbit” (Fig. 5a). However, when the model’s output space is restricted to a predefined subset of classes (e.g., “cat” and “dog”), the original top-2 prediction (i.e., “cat”) becomes the top-1 result, thereby distorting the model’s true prediction. Such distortion can mislead bias evaluation: when the model’s distorted prediction happens to match the ground-truth label (“cat” in Fig. 5b), the model may appear to correctly rely on a given shape or texture cue, even though it did not genuinely recognize it. Consequently, restricted label evaluation can overestimate cue usage and obscure the model’s true perceptual behavior. 2.3 Prior Critiques and the Remaining Gaps Our critique is not the first to question aspects of the cue-conflict benchmark. Several recent studies [10–12,27] have already pointed out important limitations in the current paradigm. Some focused on cue imbalance and attempted to construct more controlled stimuli. For example, Tartaglini et al.[27]fill shape silhouettes with textures, but their design overlooks local shape features and still relies on relative bias metrics, limiting fair cross-model comparison. Burgert et al.[12]analyze prediction shifts under cue degradation, but do not fully account for low-level texture cues and remain constrained by ratio-based bias measures. Other studies examined the limitations of relative bias metrics or evaluated cue sensitivity through degraded or texture-suppressed inputs. Wen et al.[10] measure sensitivity changes under global shape degradation, whereas Doshi et al.[11]assess shape sensitivity using texture-suppressed global shape cues. These studies provide valuable evidence that the current conflicting-cue setup should not be treated as an unquestionable standard. At the same time, prior efforts remain largely fragmented, typically addressing only part of the problem, such as cue balance, relative bias, or shape sensitivity, while leaving other issues unresolved. In particular, no prior work provides a unified re-examination of how cues are constructed, how predictions are evaluated, and how cue-specific sensitivity should be measured in a consistent full-label setting. Our work is motivated by precisely this gap: rather than addressing a single limitation in isolation, we revisit cue preference benchmarking from the ground up and develop an integrated framework that improves cue construction, evaluation, and interpretability together. Taken together, the issues discussed in this section suggest that cue-conflict may not fully satisfy the desiderata of a reliable bias benchmark, which can lead to unintuitive evaluations in practice (§4.1 and 4.2). Existing alternatives likewise remain limited in delivering a fully intuitive and comprehensive assessment of model bias (Appendix A.8). 3 Methodology We proposeREFINED-BIAS, a new framework for accurately and reliably measuring and comparing shape and texture biases. First, we define disentangled stimuli following human perceptual standards rather than model-derived heuristics, ensuring that each cue is pure, equally informative, and clearly recognizable (§3.1). Second, we introduce a novel bias metric evaluated in the model’s true decision space, capturing both preference and sensitivity (§3.2). By unifying these refined stimuli and evaluation metrics,REFINED-BIASsatisfies the essential desiderata for a reliable bias assessment, enabling a precise and fair comparison of perceptual capabilities across models. 3.1 Shape and Texture Cue Construction What constitutes shape and texture in our benchmark? To address the Problem 1, we define shape and texture based on human perception rather than model heuristics, and generate cues that faithfully capture these characteristics, as shown in Fig. 6a (right). Texture is defined as a pattern that consistently repeats within patches of various image sizes. For example, classes like “honeycomb” and “dishrag” exhibit characteristic textures that remain recogniz- able even when divided into small patches of different sizes. Shape, on the other hand, is defined as a non-repeating geometric structure, encompassing both global and local features [8,24]. Global geometry refers to the overall structure of an object, such as its silhouette, while local geometry includes distinctive substructures not repeated across the object. For 5 (a) Qualitative Examples on Generated Cues(b) Illustration of Mean Reciprocal Rank (MRR) Mean Reciprocal Rank (MRR): ( 1.0 + 0.5 + 0.3 ) / 3 = 0.3 MRR for granular assessment of cue reliance bookclockipodtigerzebra1/1=1.0 1/3=0.3 1/2=0.5 : Groundtruth image class: Incorrect image class Mohlaet al. Source imageGeirhos et al. Shape Texture Book image : Clock image: Ipod image: Rank 1Rank 2Rank 3Rank 4Rank 5(Predicted) bookclockipodtigerzebra bookclockipodtigerzebra Pure shape of ipod Ipod shape with noise Cheetah shape leakage Shape + Texture Shape + Texture Ours Ipod shape not visible Cheetah texture not visible Pure texture of cheetah Figure 6:REFINED-BIASdataset and metric. (a) Qualitative comparison with existing cue generation methods [9,25]. (b) Computation of the mean reciprocal rank (MRR) metric. Additional qualitative examples are provided in Appendix A.2. instance, although “ipod” and “comic book” share a similar rectangular global geometry, their local structures are distinct, allowing reliable classification. Data collection. To further mitigate the Problem 1, we construct a curated dataset of 20 ImageNet- derived superclasses (see Appendix A.2 for details), comprising 10 shape-dominant (e.g., clock, hourglass) and 10 texture-dominant (e.g., strawberry, brain coral) categories, selected based on human perceptual judgments. On the other side, the limited source images used in cue-conflict may introduce image-level variations, such as resolution differences that can make patterns appear more coarse or fine. To mitigate this, we collect 300 diverse images per class, 10 from ImageNet and the rest from web sources. This results in a more balanced set of shape and texture cues compared with cue-conflict, which contains only 10 shape and 3 texture sources per class. In total, our dataset contains 6,000 high-quality images, roughly five times larger than cue-conflict. Shape and texture cues. In designing our cue dataset, we build upon the insights of Mohla et al. [9] rather than simply replicating their protocol, as the direct application of existing methods leaves data quality issues unresolved. As shown in Fig. 6a (middle), prior methods produce texture cues characterized by local and global shape leakage and reduced cue resolution, while shape cues suffer from noisy background clutter that obscures the object’s structural integrity. Furthermore, the stylization method of Geirhos et al.[7]tends to inherit these inherent data issues found in the cue-conflict benchmark (Fig. 6a, left). To avoid these issues, we design a more precise and carefully controlled cue generation pipeline. As shown in Fig. 6a (right), this pipeline ensures clearly recognizable pure shape and texture cues while preserving their full resolution. We further conduct human inspection on each generated cue image to enhance cue quality. The details on the generation pipeline are in Appendix A.1. Human Model Accuracy on ShapeModel Accuracy on Texture 1.0 0.2 Accuracy 0.6 0.8 0.4 0.0 1.0 0.2 Accuracy 0.6 0.8 0.4 0.0 Informative ✓ Informative ✓ Informative = Contains sufficiently distinctive cue features Shape ClassesTexture Classes Balanced Cue Recognition in REFINED-BIAS Informative ✓ Informative ✓ Figure 7: Human and model perception trends on shape and texture cues of theREFINED-BIAS dataset. The experimental setup in Fig. 4 is used. As shown in Fig. 7, our dataset achieves compa- rable shape and texture accuracies while substan- tially reducing class imbalance. This reflects the high quality of our stimuli, which also faithfully capture structural 3D information while remain- ing free from the grid-like artifacts in textures (see Appendix A.9). Note that the pure stimuli in Geirhos et al.[7]are small-scale, created for illustration rather than benchmarking, and not fully public. In contrast, our stimuli are sys- tematically constructed and released as a public resource for bias evaluation. Mitigating domain shift in CNNs. To mitigate the domain shift in CNNs that often hinders bias evaluation, we carefully curate a subset of classes where either shape or texture serves as the most discriminative feature for classification. For these selected categories, we generate clean, artifact-free images to ensure that the intended cues remain clearly recognizable to CNNs (as well as ViTs). Consequently,REFINED-BIASachieves significantly higher recognition performance across all ImageNet-1k pretrained CNNs (see Appendix B), with average top-1 accuracies of 46% for shape and 63% for texture. By comparison, the cue-conflict benchmark achieves 4% (shape) and 21% 6 (texture) in full model predictions, and 20% and 30% in distorted predictions. This indicates that REFINED-BIAS is significantly less susceptible to the domain shift in CNNs than cue-conflict. Our human study details. To ensure that our shape and texture cues are clearly recognizable to humans, we conducted parallel user studies onREFINED-BIASand cue-conflict. We recruited 88 participants, each completing 100 classification tasks using randomly sampled images from each benchmark. These tasks were evenly divided into shape and texture sections, where participants identified the target class of a single image for each task. Table 1: Inter-rater agreement scores. Fleiss’ Kappa (κ) measures the degree of agreement among raters in their predictions. Higher scores indicate stronger consistency in predictions and reflect how clearly cues are perceived. DatasetCue TypeFleiss’ Kappa REFINED-BIAS Shape0.9836 Texture0.7973 Cue-conflict Shape0.7276 Texture0.2937 To measure how consistently participants iden- tified these cues, we calculated inter-human agreement using Fleiss’ kappa (κ) [28]. While REFINED-BIASachieved near-perfect agree- ment for shape (κ = 0.98) and substantial agreement for texture (κ = 0.79), cue-conflict showed significantly lower consistency, partic- ularly for texture (κ = 0.29), indicating inher- ent ambiguity in its signals. These show that REFINED-BIASprovides more consistently rec- ognizable cues for human evaluators. 3.2 Model Comparisons with Redefined Bias Addressing the Problem 2 and Problem 3, we introduce a new metric that operates on the full logits and enables more fair cross-model comparisons. The core insights behind our metric are as follows: (1) existing bias metrics are often inflated because they rely directly on accuracy for cues, and (2) accuracy appears in both the numerator and denominator, which could distort the ratio. To resolve these issues, we (1) replace accuracy with a ranking-based metric, and (2) apply the metric only in the denominator. While various ranking-based metrics could be used, we employ Mean Reciprocal Rank (MRR) [29], which aligns well with our objectives. While accuracy assigns 0 to both 2nd and 100th place predictions, MRR distinguishes them with values of 1/2 and 1/100, enabling a more precise assessment of how models prioritize different cues. Redefined Bias. Specifically, our metric computes the reciprocal ranks of the correct shape and texture labels within the model’s full prediction ranking (Fig. 6b). We refer to these two components as Shape-Sens and Texture-Sens. Unlike conventional MRR, our ranking is computed over the logits: Shape-Sens = 1 N N X i=1 1 r shape,i ,Texture-Sens = 1 N N X i=1 1 r texture,i .(1) Here,Nis the total number of samples,r shape,i andr texture,i are the ranks of the correct shape and texture labels for thei-th sample in the model’s ranked predictions, respectively. The relative bias for shape and texture is defined as: Shape preference = Shape-Sens/(Shape-Sens + Texture-Sens),(2) Texture preference = Texture-Sens/(Shape-Sens + Texture-Sens).(3) In the following sections, we demonstrate that our dataset and metric effectively distinguish bias differences and identify models with genuinely strong biases. 4 Experiments Our experimental agenda is guided by two goals. First, we validate the correctness ofREFINED-BIAS, which can reliably contrast shape-texture bias across a diverse spectrum of training strategies (§4.1), aligning with our prior understanding and intuition. Building on this foundation, we further investigate how such bias varies across different model architectures (§4.2). We note that the term “bias” is used to refer to both preference and sensitivity collectively. 4.1 Validating REFINED-BIAS Benchmark The credibility of all subsequent experiments hinges on the correctness of theREFINED-BIASbench- mark; we evaluate whether the outcomes are consistent with our intuition and whether they remain 7 plausible given existing knowledge. To this end, we first evaluate the dataset itself using extensive training strategies for diverse pre-trained models compared with the cue-conflict benchmark. We then focus more on assessing the correctness of the revised sensitivity metric and examining how the resulting bias measurements correlate with ImageNet top-1 accuracy (i.e., in-domain performance). Learning strategies and hypothesis: We consider 32 ImageNet-1k pretrained models, all based on the same ResNet-50 architecture [30]. As a baseline, we use a model trained with random cropping only. Each model applies one additional training strategy on top of this baseline setup. The strategies are as follows (full details are provided in Appendix C.1, and visual examples are shown in Appendix C.4): •Shape augmentation explicitly promotes shape-based recognition by exposing models to conflicting cues and enforcing correct shape prediction. We use three models [7,31] following this strategy. • Contrastive learning implicitly learns cue invariance by aligning representations across texture variations such as blurring, encouraging reliance on stationary shape information. We use three models [32–34] for this strategy. •Texture distortion injects noise that disrupts textural information while preserving semantic structure, encouraging reliance on invariant shape features. We use five models [35–39] following this strategy. • Mixed augmentation mixes image pairs or masks regions, allowing models to learn stable shape and texture cues while reducing reliance on non-stationary ones. We use eight models from Wightman et al. [40], four of which also apply mild texture distortions [41]. •Adversarial training makes models robust to imperceptible image perturbations [17,42]. It does not directly improve shape or texture perception, so it does not necessarily affect shape or texture bias. We consider 12 models trained with varying levels of noise. REFINED-BIAS dataset reflects trends clearly. Based on the hypothesis, we evaluate whether each benchmark dataset faithfully reflects the expected effects of different learning strategies, using the preference metric. As shown in Table 2, our dataset demonstrates that shape-focused strategies consistently increase shape preference. Notably, even nuanced strategies such as mixed augmentations, which apply mild texture degradation, are accurately reflected by ours as an increased shape preference. While cue-conflict partly captures similar tendencies, many of its results are not statistically significant and show an inconsistent trend across the strategies. Table 2:t-test on the difference between the base- line and training strategies based on the prefer- ence. The red, yellow, and gray shading indicate significantlyshape reliance,texture reliance, and non-significance , respectively (α=0.05). Model Family ExpectedCue-conflictREFINED-BIAS ● Mixed Augshapep=2.72e-04p=0.002 ● Texture Distshapep=0.003p=0.010 ● Shape Augshapep=0.181p=0.018 ● Contrastiveshapep=0.684p=0.009 ● Adversarial neitherp=5.19e-05p=0.489 For adversarial training, results on our dataset show that robustness to imperceptible noise does not significantly affect model preference. In contrast, cue-conflict reports a larger increase in shape preference than shape-focused methods, which is counterintuitive since it is primarily aimed at improving adversarial robustness, not shape preference. Furthermore, consistent with recent findings [12] that the ImageNet-1k pre- trained ResNet-50 model does not strongly rely on texture cues, our dataset indicates lower re- liance on texture (0.49), while cue-conflict reports a higher texture preference (0.77). Overall, these results demonstrate that our dataset provides a more reliable reflection of model behavior. Sensitivity metric reveals models that truly utilize cues. The primary goal of our sensitivity metric is to enable fair comparisons across models by reliably distinguishing those that utilize either shape or texture cues. To this end, we evaluate whether it can reveal cross-model differences that the preference metric misses. On our dataset, the preference metric suggests that adversarial learning induces the strongest utilization of texture cues (Fig. 8c), while on the cue-conflict dataset, it indicates the strongest utilization of shape cues (Fig. 8a). As shown in Fig. 9a and Fig. 9b, our sensitivity metric demonstrates that adversarial learning does not increase the model’s utilization of either cue, whereas mixed augmentations lead models to utilize both shape and texture cues, revealing differences that the preference metric obscures. These results show that our sensitivity metric captures both cross-model differences in cue utilization and independent utilization of shape and texture cues for models that rely on both, unlike the preference metric, which fails to capture either. 8 Cue-conflict Dataset&Preference MetricREFINED-BIAS Dataset&Preference Metric Mixed AugAdversarialContrastiveShape AugTexture Dist Baseline ★ Learning strategies (a) Learning Strategy Shape preference 00.30.40.60.80.20.510.70.10.9 Texture preference (b) Model Architecture Shape preference Texture preference 00.30.40.60.80.20.510.70.10.9 ResNetInceptionV3AlexNetVGGBagNet Model architectures (c) Learning Strategy Largest texture reliance :Largest shape reliance : Shape preference Texture preference 00.30.40.60.80.20.510.70.10.9 (d) Model Architecture Shape preference Texture preference 00.30.40.60.80.20.510.70.10.9 Top - 1 Acc (ImageNet - 1k) 0.9 0.7 0.6 0.8 0.4 0.5 Top - 1 Acc (ImageNet - 1k) 0.9 0.7 0.6 0.8 0.4 0.5 0.9 0.7 0.6 0.8 0.4 0.5 Top - 1 Acc (ImageNet - 1k) Top - 1 Acc (ImageNet - 1k) 0.9 0.7 0.6 0.8 0.4 0.5 Shape preference ↑ In-domain acc ↑ 푟=0.607 (p<0.05)푟=−0.927 (p<0.05)푟=0.897 (p<0.05)푟=0.549 (p<0.05) Shape preference ↓ In-domain acc ↓ Shape preference↑ In-domain acc ↑ Shape preference↑ In-domain acc ↑ 00.30.40.60.80.20.510.70.10.900.30.40.60.80.20.510.70.10.900.30.40.60.80.20.510.70.10.900.30.40.60.80.20.510.70.10.9 # Parameters (M) 206040100150 # Parameters (M) 206040100150 SwinViTCMT Largest texture reliance :Largest shape reliance : Figure 8: Comparison of models based on preference metric and ImageNet-1k top-1 accuracy. Varying learning strategies with a fixed architecture are compared on the (a) cue-conflict dataset and (c) our dataset, respectively. Varying architectures with fixed learning strategies are compared on the (b) cue-conflict dataset and (d) our dataset, respectively. Red and yellow backgrounds indicate models with shape and texture preferences, respectively. r is the Pearson correlation coefficient. REFINED-BIAS Dataset & Sensitivity Metric Top - 1 Acc (ImageNet - 1k) 0.9 0.7 0.6 0.8 0.4 0.5 Shape-SensTexture-Sens Top - 1 Acc (ImageNet - 1k) 0.9 0.7 0.6 0.8 0.4 0.5 (a) Learning Strategy(b) Learning Strategy 푟=0.551 (p<0.05)푟=0.770 (p<0.05) Top - 1 Acc (ImageNet - 1k) 0.9 0.7 0.6 0.8 0.4 0.5 Shape-SensTexture-Sens Top - 1 Acc (ImageNet - 1k) 0.9 0.7 0.6 0.8 0.4 0.5 (c) Model Architecture(d) Model Architecture 푟=−0.473 (p<0.05)푟=0.939 (p<0.05) Mixed AugAdversarialContrastiveShape AugTexture Dist Baseline ★ Learning strategies ResNetInceptionV3AlexNetVGGBagNetSwinViTCMT Model architectures 00.30.40.60.80.20.510.70.10.900.30.40.60.80.20.510.70.10.900.30.40.60.80.20.510.70.10.900.30.40.60.80.20.510.70.10.9 # Parameters (M) 206040100150 # Parameters (M) 206040100150 Weakest shape reliance :Weakest shape reliance :Largest shape reliance:Largest shape reliance : Figure 9: Comparison of models based on the sensitivity metric and ImageNet-1k top-1 accuracy. (a) shape sensitivity and (b) texture sensitivity measured across different learning strategies with a fixed architecture. (c) shape sensitivity and (d) texture sensitivity measured across varying architectures with fixed learning strategies. r is the Pearson correlation coefficient. Balanced cue usage positively correlates with performance. To examine the relationship between model bias and in-domain performance, we employ a dataset with pure, balanced cue information and a sensitivity metric that clearly distinguishes model differences. Following Gavrikov and Keuper[17], we fix the architecture (ResNet-50) to isolate inductive bias as a confounding factor. In Fig. 9a and 9b, our benchmark reveals that higher in-domain accuracy positively correlates with the utilization of both shape and texture cues. These results align with prior studies [9,31,43], which show that jointly utilizing shape and texture cues improves in-domain performance, further confirming that our benchmark accurately captures their complementary roles. 4.2 What Becomes Visible Once Bias Is Measured Reliably Based on our dataset and metric, we provide an empirical analysis of how shape and texture biases vary across different model architectures and examine their relation to in-domain performance. See Appendix B for full model details. How ViT design influences cue utilization. The ViT architecture is inherently designed to capture broad, global context through its patch-wise self-attention [44], but struggles to effectively encode local features [45–47]. Subsequent designs like Swin and CMT address this by improving local- to-global feature aggregation: Swin via self-attention in progressively shifted windows, and CMT by combining local convolutional extraction with global self-attention. We investigate how this local-to-global understanding affects model behavior by analyzing shape and texture utilization using our benchmark and sensitivity metric. As shown in Fig. 9c, Swin and CMT exhibit higher shape sensitivity than ViT, indicating that improved local feature aggregation enhances shape perception (consistent with Shi et al.[48]for Swin). In contrast, Fig. 8b shows that the cue-conflict dataset, measured via the preference metric, fails to reveal this advantage: CMT shows little change, and Swin shows reduced shape preference. Which drives performance? Shape or texture? Recent studies [15,17,22,23] using cue-conflict benchmark have yielded conflicting conclusions about whether shape preference is more beneficial 9 for in-domain performance. Specifically, as shown in Fig. 8a, when the CNN architecture is fixed, and only the training strategy varies, higher texture preference is associated with better accuracy, consistent with [17]. Conversely, when the training strategy is fixed, and the architecture varies, the opposite trend emerges: models with stronger shape preference perform better (Fig. 8b), in line with [15,22,23]. These contradictory findings within the cue-conflict framework undermine a coherent understanding of preference and call into question whether the benchmark faithfully reflects its intended premise. In contrast, our dataset and preference metric provide consistent results across all setups (Fig. 8c and Fig. 8d), reliably identifying shape preference as a more important performance contributor. This consistency demonstrates that our benchmark not only accurately captures the intended insight of cue-conflict but also enables a more reliable evaluation of model preference. 5 Discussion DoesREFINED-BIASstill evaluate cue preference? Yes, but with greater precision. By identifying which information a model can actually leverage when provided in its pure form, we derive a more grounded preference that is not confounded by the failure to recognize competing signals. This approach restores evaluative reliability while preserving the core insight of cue-conflict, enabling a more reliable assessment of how training strategies influence preference. CouldREFINED-BIASstill be affected by domain shift? A natural concern is thatREFINED-BIAS may face the very domain-shift issue encountered in the cue-conflict setting of Geirhos et al.[7]. Indeed, stylization-based cue-conflict benchmarks emerged in part because earlier attempts could not avoid substantial distribution mismatch when directly isolating cues. Through cleaner stimulus construction and carefully selected distinctive classes,REFINED-BIASyields substantially higher top-1 accuracies, as reported in §3.1. These results suggest thatREFINED-BIASis less susceptible to severe domain shift than prior stylization-based cue-conflict benchmarks. What exactly is improved by the dataset, the metric, and their combination? Our dataset resolves issues of cue purity, recognizability, and balance. It eliminates the inherent ambiguity of stylization and provides a five-times larger, scalable pool of samples that are easier for both humans and models to interpret. Our sensitivity metric enables full-label evaluation to identify genuine cue utilization. By separating “how much a model knows” from “which cue it prefers”, it uniquely reveals models that genuinely utilize both cues. The integrated benefit arises from their combination, providing the empirical lesson that improved performance stems from dual reliance on both cues and that models with local-to-global attention consistently drive stronger shape utilization. What exactly is improved by the dataset, the metric, and their combination? Our dataset resolves issues of cue purity, recognizability, and balance. It eliminates the inherent ambiguity of stylization and provides a five-times larger, scalable pool of samples that are easier for both humans and models to interpret. Our sensitivity metric enables full-label evaluation to identify genuine cue utilization. By separating “how much a model knows” from “which cue it prefers”, it uniquely reveals models that genuinely utilize both cues. The integrated benefit arises from their combination, providing the empirical lesson that improved performance stems from dual reliance on both cues and that models with local-to-global attention consistently drive stronger shape utilization. Potential limitations. While our shape cues offer diagnostic clarity, they may not fully capture 3D geometry or viewpoint dependencies. Completely isolating texture from residual shape impressions remains an open challenge, and expanding cue classes will further broaden the scope of bias analysis. 6 Conclusion The cue-conflict benchmark has advanced our understanding of how networks use shape and texture, but its uncontrollable stylization, relative bias that obscures cue sensitivity, and strict class restrictions limit reliable bias analysis. To address these issues, we introduceREFINED-BIAS, a comprehensive framework designed for more controlled and precise bias evaluation. Our refined dataset and metric together provide a unified solution that addresses the limitations of cue-conflict and unresolved issues in recent benchmarks. We establish a more principled and dependable framework for evaluating cue-related biases in modern vision models. 10 References [1]Nicholas Baker, Hongjing Lu, Gennady Erlikhman, and Philip J Kellman. Deep convolutional networks do not classify based on global object shape. PLoS computational biology, 14(12): e1006613, 2018. [2]Katherine Hermann, Ting Chen, and Simon Kornblith. The origins and prevalence of texture bias in convolutional neural networks. Advances in Neural Information Processing Systems, 33: 19000–19015, 2020. [3] Robert Geirhos, Kantharaju Narayanappa, Benjamin Mitzkus, Tizian Thieringer, Matthias Bethge, Felix A Wichmann, and Wieland Brendel. Partial success in closing the gap between human and machine vision. Advances in Neural Information Processing Systems, 34:23885– 23899, 2021. [4]Xiaohan Ding, Xiangyu Zhang, Jungong Han, and Guiguang Ding. Scaling up your kernels to 31x31: Revisiting large kernel design in cnns. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11963–11975, 2022. [5]Paul Gavrikov, Janis Keuper, and Margret Keuper. An extended study of human-like behavior under adversarial training. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2361–2368, 2023. [6]Shahaf E Finder, Roy Amoyal, Eran Treister, and Oren Freifeld. Wavelet convolutions for large receptive fields. In European Conference on Computer Vision, pages 363–380. Springer, 2024. [7]Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A Wichmann, and Wieland Brendel. Imagenet-trained cnns are biased towards texture; increasing shape bias improves accuracy and robustness. In International Conference on Learning Representations, 2018. [8]Katherine Hermann and Andrew Lampinen. What shapes feature representations? exploring datasets, architectures, and training. Advances in Neural Information Processing Systems, 33: 9995–10006, 2020. [9]Satyam Mohla, Anshul Nasery, and Biplab Banerjee. Teaching cnns to mimic human visual cognitive process & regularise texture-shape bias. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1805–1809. IEEE, 2022. [10]Ziqi Wen, Tianqin Li, Zhi Jing, and Tai Sing Lee. Does resistance to style-transfer equal global shape bias? measuring network sensitivity to global shape configuration. arXiv preprint arXiv:2310.07555, 2023. [11]Fenil R Doshi, Thomas Fel, Talia Konkle, and George Alvarez. Visual anagrams reveal hidden differences in holistic shape processing across vision models. arXiv preprint arXiv:2507.00493, 2025. [12]Tom Burgert, Oliver Stoll, Paolo Rota, and Begüm Demir. Imagenet-trained cnns are not biased towards texture: Revisiting feature reliance through controlled suppression. arXiv preprint arXiv:2509.20234, 2025. [13]Tianyuan Zhang and Zhanxing Zhu. Interpreting adversarially trained convolutional neural networks. In International conference on machine learning, pages 7502–7511. PMLR, 2019. [14]Chaithanya Kumar Mummadi, Ranjitha Subramaniam, Robin Hutmacher, Julien Vitay, Volker Fischer, and Jan Hendrik Metzen. Does enhanced shape bias improve neural network robustness to common corruptions? arXiv preprint arXiv:2104.09789, 2021. [15] Tiago Oliveira, Tiago Marques, and Arlindo L Oliveira. Connecting metrics for shape-texture knowledge in computer vision. arXiv preprint arXiv:2301.10608, 2023. [16]Tianqin Li, Ziqi Wen, Yangfan Li, and Tai Sing Lee. Emergence of shape bias in convolutional neural networks through activation sparsity. Advances in Neural Information Processing Systems, 36:71755–71766, 2023. 11 [17]Paul Gavrikov and Janis Keuper. Can biases in imagenet models explain generalization? In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22184–22194, 2024. [18]Weijie Tu, Weijian Deng, and Tom Gedeon. Toward a holistic evaluation of robustness in clip models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025. [19] Junru Zhao, Tianqin Li, Dunhan Jiang, Shenghao Wu, Alan Ramirez, and Tai Sing Lee. Percep- tual inductive bias is what you need before contrastive learning. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 9621–9630, 2025. [20] Pablo Hernández-Cámara, Jose Manuel Jaén-Lorites, Alexandra Gómez-Villa, Jorge Vila-Tomás, Valero Laparra, and Jesus Malo. On the dynamic evolution of clip texture-shape bias and its relationship to human alignment and model robustness. arXiv preprint arXiv:2508.09814, 2025. [21]Sungjae Jeon, Youjin Kim, Seungtae Hong, and Jeong-Si Kim. L1-induced activation sparsity shifts cnn bias from texture to shape. In 2025 IEEE/IEIE International Conference on Consumer Electronics-Asia (ICCE-Asia), pages 1–4. IEEE, 2025. [22]Mehmet Aygun, Prithviraj Dhar, Zhicheng Yan, Oisin Mac Aodha, and Rakesh Ranjan. Enhanc- ing 2d representation learning with a 3d prior. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7750–7760, 2024. [23]Ben Lonnqvist, Elsa Scialom, Abdulkadir Gokce, Zehra Merchant, Michael H Herzog, and Mar- tin Schrimpf. Contour integration underlies human-like vision. arXiv preprint arXiv:2504.05253, 2025. [24]Yunhao Ge, Yao Xiao, Zhi Xu, Xingrui Wang, and Laurent Itti. Contributions of shape, texture, and color in visual recognition. In European Conference on Computer Vision, pages 369–386. Springer, 2022. [25] Leon A Gatys, Alexander S Ecker, and Matthias Bethge. Image style transfer using convolutional neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2414–2423, 2016. [26]Robert Geirhos, Carlos RM Temme, Jonas Rauber, Heiko H Schüt, Matthias Bethge, and Felix A Wichmann. Generalisation in humans and deep neural networks. Advances in Neural Information Processing Systems, 31, 2018. [27]Alexa R Tartaglini, Wai Keen Vong, and Brenden M Lake. A developmentally-inspired examination of shape versus texture bias in machines. arXiv preprint arXiv:2202.08340, 2022. [28]Joseph L Fleiss. Measuring nominal scale agreement among many raters. Psychological bulletin, 76(5):378, 1971. [29]Ellen M Voorhees et al. The trec-8 question answering track report. In Trec, volume 99, pages 77–82, 1999. [30]Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. [31]Yingwei Li, Qihang Yu, Mingxing Tan, Jieru Mei, Peng Tang, Wei Shen, Alan Yuille, and Cihang Xie. Shape-texture debiased neural network training. arXiv preprint arXiv:2010.05981, 2020. [32]Xinlei Chen, Saining Xie, and Kaiming He. An empirical study of training self-supervised vision transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9640–9649, 2021. [33]Ting Chen, Simon Kornblith, Kevin Swersky, Mohammad Norouzi, and Geoffrey E Hinton. Big self-supervised models are strong semi-supervised learners. Advances in Neural Information Processing Systems, 33:22243–22255, 2020. 12 [34]Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9650–9660, 2021. [35]Dan Hendrycks, Norman Mu, Ekin D Cubuk, Barret Zoph, Justin Gilmer, and Balaji Lakshmi- narayanan. Augmix: A simple data processing method to improve robustness and uncertainty. arXiv preprint arXiv:1912.02781, 2019. [36]Dan Hendrycks, Andy Zou, Mantas Mazeika, Leonard Tang, Bo Li, Dawn Song, and Jacob Steinhardt. Pixmix: Dreamlike pictures comprehensively improve safety measures. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16783–16792, 2022. [37] Apostolos Modas, Rahul Rade, Guillermo Ortiz-Jiménez, Seyed-Mohsen Moosavi-Dezfooli, and Pascal Frossard. Prime: A few primitives can boost robustness to common corruptions. In European Conference on Computer Vision, pages 623–640. Springer, 2022. [38] Patrick Müller, Alexander Braun, and Margret Keuper. Classification robustness to common optical aberrations. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3632–3643, 2023. [39]Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, et al. The many faces of robustness: A critical analysis of out-of-distribution generalization. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 8340–8349, 2021. [40]Ross Wightman, Hugo Touvron, and Hervé Jégou. Resnet strikes back: An improved training procedure in timm. arXiv preprint arXiv:2110.00476, 2021. [41]Ekin D Cubuk, Barret Zoph, Jonathon Shlens, and Quoc V Le. Randaugment: Practical automated data augmentation with a reduced search space. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, pages 702–703, 2020. [42]Hadi Salman, Andrew Ilyas, Logan Engstrom, Ashish Kapoor, and Aleksander Madry. Do ad- versarially robust imagenet models transfer better? Advances in Neural Information Processing Systems, 33:3533–3545, 2020. [43]Jiwen Tang, Gu Wang, Ruida Zhang, and Xiangyang Ji. Enhancing shape bias for object detection. Neurocomputing, page 132931, 2026. [44]Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020. [45] Li Yuan, Yunpeng Chen, Tao Wang, Weihao Yu, Yujun Shi, Zi-Hang Jiang, Francis EH Tay, Jiashi Feng, and Shuicheng Yan. Tokens-to-token vit: Training vision transformers from scratch on imagenet. In Proceedings of the IEEE/CVF international conference on computer vision, pages 558–567, 2021. [46] Zhiliang Peng, Wei Huang, Shanzhi Gu, Lingxi Xie, Yaowei Wang, Jianbin Jiao, and Qixiang Ye. Conformer: Local features coupling global representations for visual recognition. In Proceedings of the IEEE/CVF international conference on computer vision, pages 367–376, 2021. [47] Haiping Wu, Bin Xiao, Noel Codella, Mengchen Liu, Xiyang Dai, Lu Yuan, and Lei Zhang. Cvt: Introducing convolutions to vision transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pages 22–31, 2021. [48]Rui Shi, Tianxing Li, Liguo Zhang, and Yasushi Yamaguchi. Visualization comparison of vision transformers and convolutional neural networks. IEEE Transactions on Multimedia, 26: 2327–2339, 2023. 13 [49]Tianhe Ren, Shilong Liu, Ailing Zeng, Jing Lin, Kunchang Li, He Cao, Jiayu Chen, Xinyu Huang, Yukang Chen, Feng Yan, et al. Grounded sam: Assembling open-world models for diverse visual tasks. arXiv preprint arXiv:2401.14159, 2024. [50]Lihe Yang, Bingyi Kang, Zilong Huang, Zhen Zhao, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything v2. Advances in Neural Information Processing Systems, 37:21875– 21911, 2024. [51]Alexei A Efros and William T Freeman. Image quilting for texture synthesis and transfer. In Seminal graphics papers: pushing the boundaries, volume 2, pages 571–576, 2023. [52]André Araujo, Wade Norris, and Jack Sim. Computing receptive fields of convolutional neural networks. Distill, 2019. doi: 10.23915/distill.00021. https://distill.pub/2019/computing- receptive-fields. [53] Shiwei Liu, Tianlong Chen, Xiaohan Chen, Xuxi Chen, Qiao Xiao, Boqian Wu, Tommi Kärkkäinen, Mykola Pechenizkiy, Decebal Mocanu, and Zhangyang Wang. More convnets in the 2020s: Scaling up kernels beyond 51x51 using sparsity. arXiv preprint arXiv:2207.03620, 2022. [54] Yiyuan Zhang, Xiaohan Ding, and Xiangyu Yue. Scaling up your kernels: Large kernel design in convnets towards universal representations. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025. [55]Leichao Cui, Xiuxian Li, Min Meng, and Guangyu Jia. Large receptive field strategy and important feature extraction strategy in 3d object detection. arXiv preprint arXiv:2401.11913, 2024. [56]Wei Miao, Jiangrong Shen, Qi Xu, Timo Hamalainen, Yi Xu, and Fengyu Cong. Spikingyolox: Improved yolox object detection with fast fourier convolution and spiking neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 1465–1473, 2025. [57]Wieland Brendel and Matthias Bethge. Approximating cnns with bag-of-local-features models works surprisingly well on imagenet. arXiv preprint arXiv:1904.00760, 2019. [58] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems, 25, 2012. [59]Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014. [60]Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. Re- thinking the inception architecture for computer vision. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2818–2826, 2016. [61]Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2021. [62] Jianyuan Guo, Kai Han, Han Wu, Yehui Tang, Xinghao Chen, Yunhe Wang, and Chang Xu. Cmt: Convolutional neural networks meet vision transformers. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12175–12185, 2022. [63]Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083, 2017. 14 On the Reliability of Cue Conflict and Beyond Supplementary Material A Details on REFINED-BIAS A.1 Our Pipeline for Cue Generation To ensure each cue provides a well-recognizable representation for both humans and models, we develop a precise cue generation pipeline that isolates pure cue information from irrelevant features. Segmentation Segmentation Model (a) Object Segmentation (b) Shape and Texture Cue Construction Shape Cue Image Mask SegmentBlurred Filter Edge Human Curation Texture Cue Mask ImageSegmentPatch Selection Reorder Image + Label Human Curation Mask Patch for 32×32Patch for 4×4Patch for 8×8Patch for 16×16 ⊙ ⊙ ∗ ⊙ Figure A: Overview of dataset construction for shape and texture cues. Shape cue generation. Our shape cues repre- sent pure structural information by extracting con- tours exclusively from semantic object regions. We first perform semantic segmentation to isolate the object [49] and apply class-adjusted Gaussian blur within the mask to suppress internal texture. From this blurred region, we extract structural con- tours similar to Mohla et al.[9], rendering them as white edges on a black background. This pro- cess preserves both global and local shape fea- tures while ensuring the representation remains free from background clutters. Texture cue generation. For our texture cues, the objective is to preserve fine-grained surface patterns while systematically removing any local or global structural information. Building on the semantic segmentation [49], we crop patches of four predefined sizes exclusively from the interior of the object. This ensures that the patches do not contain any local contours or boundary informa- tion that could inadvertently represent the object’s shape. To extract pure texture, these patches are reordered using a strategy similar to that of Mohla et al.[9], eliminating the local structure within the object interior while preventing the formation of grid-like artifacts. Finally, we manually curate all generated samples, filtering out instances that fail to represent the intended shape or texture cues, contain noise, or exhibit incorrect semantic masks. A.2 Category Selection Guided by Visual Cue Distinctiveness Previous benchmarks, such as the 16 superclasses from Geirhos et al.[7], were primarily designed to assess model robustness to image degradation. Accordingly, many of the selected classes have large, rigid structures with well-defined silhouettes. These classes are highly recognizable even under strong distortions, making them well-suited for degradation studies. However, they offer little diagnostic value for texture bias analysis, as their surface patterns are often uninformative or homogeneous (e.g., metal surfaces for “boat” and “car”). This leads the texture cues to become less recognizable. To address this, we curated a new set of 20 ImageNet superclasses (10 shape-dominant, 10 texture-dominant) based on human-perceptual criteria: (1) Shape dominant categories. Shape cue categories were selected for having highly discriminative structural forms with minimal reliance on surface patterns. These include classes such as hourglass and ipod, where both local and global shape are the primary identifying features. (2) Texture dominant categories. Texture cue categories were chosen for featuring rich, class- specific surface textures, with less distinctive structural information. Examples include strawberry, cheetah, and dishrag, where the texture plays a central role in recognition. 15 Original Texture Image Pure Texture Grid Size4x48x88x88x816x1616x1632x3232x32 elephanttigercheetahzebrabroccolibrain_coralstrawberrydishrag Original Shape Image Pure Shape ipodcassetteclockmouserevolvercameracomic bookballoonhourglasssoccer_ball 4x4 porcupine 8x8 honeycomb (a) Shape cue (b) Texture cue Figure B: The shape-texture cues in REFINED-BIAS dataset and the corresponding source image. Our current set is intentionally limited to a balanced representation of shape- and texture-dominant categories to ensure both conceptual clarity and practical feasibility. While expanding the number of categories is a natural future direction, it will require careful validation to avoid introducing semantically ambiguous classes that could undermine the benchmark’s purpose. A.3 Psychophysical User Study To ensure thatREFINED-BIASprovides cues that are clearly interpretable as shape or texture from a human perspective, we conduct a user study. Our web-based survey consists of one hundred questions, evenly divided into two sections that assess human ability to classify images based on shape cues and texture cues. In each question, participants were asked to accurately identify the target class within a given set of classes. To construct the question set, we randomly sampled an equal number of images from each class for both shape and texture cues. Participants first completed the shape-related section, followed by the texture-related section. Before starting each section, participants were shown three randomly selected original images per class to familiarize themselves with the representative shape or texture cues. Following Geirhos et al.[7], we inserted pink noise intermittently between questions. Initial participants were randomly recruited from the lab environment, and they were encouraged to share the survey link with people in their personal networks, facilitating broader recruitment through informal social diffusion. The survey was fully anonymous, collected no personally identifiable or demographic information. The survey was implemented using Pavlovia, a widely used online platform for psychological experiments. For a fair comparison, we conducted a parallel user study on the cue-conflict dataset using the same web-based survey infrastructure. UnlikeREFINED-BIAS, where each image is assigned to either the shape or texture section, the cue-conflict study asked participants to label both the shape and texture class for each cue-conflict image. All procedures, such as the use of familiarization examples, pink noise insertion, and anonymous participation, remained identical. This setup ensured a consistent evaluation protocol across datasets, differing only in the nature of the labeling task. A total of 66 raters participated in the REFINED-BIAS study, and 22 raters completed the cue-conflict version. To compare how consistently participants responded to the survey across the two benchmarks, we measured inter-human agreement [28], indicating how consistently participants labeled each image. OnREFINED-BIAS, we observed near-perfect agreement for shape cues (κ = 0.98) and substantial agreement for texture cues (κ = 0.79), suggesting that the cues are reliably recognizable and consistently understood. In contrast, for the cue-conflict dataset showed substantial agreement on shape cues (κ = 0.72) but only fair agreement on texture cues (κ = 0.29), indicating that the texture cue in cue-conflict is ambiguous and less interpretable. These findings confirm that, compared to cue-conflict, REFINED-BIAS provides clearer and more interpretable cues for human evaluators. 16 Validation on the learning effect. To validate the presence of any learning effects, we have conducted a reverse order survey, in which eleven participants completed the texture task first, followed by the shape task. The results remained consistent with those of the original order, with only minor variation (average human shape accuracy:99%to96%, average texture accuracy:86%to91%). The slight decrease in shape accuracy, despite potential prior exposure to texture, and the increase in texture accuracy, despite no prior exposure to shape, both counter the presence of learning effects. Figure C: Survey example for the shape cue in REFINED-BIAS Figure D: Survey example for the texture cue in REFINED-BIAS Figure E: Survey example for the shape and texture cue in REFINED-BIAS 17 A.4 Relative Task Difficulty in REFINED-BIAS When measuring these bias withREFINED-BIAS, the relative task difficulty also should be taken into account. To examine how this is controlled compared to the cue-conflict, we compared the confidence intervals between tasks based on human performance shown in Fig. 3: • Avg. accuracy on REFINED-BIAS shape cues: 0.99 (95% CI: [0.98, 0.99]) • Avg. accuracy on REFINED-BIAS texture cues: 0.87 (95% CI: [0.86, 0.88]) • Avg. accuracy on cue-conflict shape cues: 0.78 (95% CI: [0.76, 0.80]) • Avg. accuracy on cue-conflict texture cues: 0.42 (95% CI: [0.39, 0.46]) The non-overlapping confidence intervals clearly indicate a difference in difficulty between the two tasks. Importantly, the gap between shape and texture accuracies is smaller inREFINED-BIAS(0.12) than in cue-conflict (0.36), suggesting that our dataset better controls the relative task difficulty. A.5 Bias Computation in Cue-conflict Given stylized images that contain shape and texture labels, the benchmark quantifies model’s bias toward shape or texture based on the proportion of correct predictions aligned with each cue. Specifically, the shape bias is defined as the ratio of correct shape decisions to the total number of correct decisions (number of corrects denoted by N ): Shape-bias = N correct-shape N correct-shape + N correct-texture . Similarly, the texture bias is defined analogously as: Texture-bias = N correct-texture N correct-shape + N correct-texture . A higher shape bias indicates the model relies more on shape than on texture information, whereas a higher texture bias suggests that the model depends more on texture than on shape cues. 18 A.6 Examples of Confounded Cues from Stylization EdgeSilhouette +++++ EdgeSilhouetteEdgeSilhouetteEdgeSilhouetteEdgeSilhouette Figure F: Examples of cue-conflict images where the stylized texture strongly correlates with structural information. The class markers with red and yellow backgrounds indicate the shape and texture classes, respectively. The white contours are added to highlight the structural information leaked into the texture cue. A.7 Examples of Unequally Informative Cues caused by Stylization +++++ Shape > Texture +++++ Shape < Texture Figure G: Examples of unequally mixed cues. The top row indicates the shape dominant conflicting images, while the bottom row indicate texture dominant cases. The class markers with red and yellow backgrounds indicate the shape and texture classes, respectively. 19 A.8 Experimental Comparison with Recent Bias Benchmarks To experimentally compareREFINED-BIASwith recent benchmarks proposed to address the limita- tions of cue-conflict, we evaluate whether each benchmark faithfully reflects the expected effects of various learning strategies, following the hypotheses in §4.1. As shown in Table A, REFINED-BIAS consistently reflects an increase in shape bias when shape-focused strategies are applied. In contrast, other benchmarks [12,27] fail to capture this expected shift. Even under shape augmentation methods, these benchmarks report a texture-biased result or no bias induction, contradicting the original intent of the training strategies. Note that, other recent benchmarks [10,11] were excluded from this comparison as their codebases are not publicly available. Table A:t-test on the difference between the baseline and training strategies based on the prefer- ence. The red, yellow, and gray shading indicate significantlyshape reliance,texture reliance, and non-significance , respectively (α=0.05). Model Family Expected Cue-conflict Tartaglini et al. [27] Burgert et al. [12] REFINED-BIAS ● Mixed Augshapep=2.72e-04p=0.402p=1.76e-04p=0.002 ● Texture Distshapep=0.003p=0.892p=0.821p=0.010 ● Shape Aug shapep=0.181p=0.196)p=0.025p=0.018 ● Contrastiveshapep=0.684)p=0.146p=0.188p=0.009 ● Adversarial neitherp=5.19e-05p=0.469p=2.04e-06p=0.489 A.9 Valid Shape and Texture Signals in REFINED-BIAS To verify whetherREFINED-BIAS’s shape and texture cues capture genuine information relevant to shape and texture, we compare them against reliable proxies. Specifically, we extracted depth maps to represent 3D shapes following Yang et al.[50], and generated textures free from grid artifacts following Efros and Freeman [51]. To ensure a comprehensive evaluation, we utilized all 6k source images used in constructing theREFINED-BIASdataset for this extraction process. These extracted data serve as proxies for valid shape and texture signals, allowing us to evaluate the validity of the information captured by our cues. As in Fig. A.9, the models’ accuracies on these cues positively correlate with their respective proxy accuracies, demonstrating thatREFINED-BIAScaptures valid shape/texture signals. 휌(RB ! s shape cue,shape proxy)휌(RB ! s texture cue,texture proxy) ✓ 1.00 0.75 0.50 0.25 0 -0.25 -0.50 InceptionV3 BG - 9 BG - 17 BG - 33 AN VGG11VGG13VGG16 RN18 VGG19 RN34RN50 RN101RN152 ViT - S ViT - B CMT - S Swin - B Swin - S CMT - B InceptionV3 BG - 9 BG - 17 BG - 33 AN VGG11VGG13VGG16 RN18 VGG19 RN34RN50 RN101RN152 ViT - S ViT - B CMT - S Swin - B Swin - S CMT - B 1.00 0.75 0.50 0.25 0 -0.25 -0.50 Figure H: Kandall’s rank correlation between class-wise Top-1 accuracy onREFINED-BIAS shape/texture cue and shape/texture proxies, respectively. 20 A.10 Distorted Model Prediction in Cue-conflict SoftMax operator ImageNet Trained Classifier Class Prediction (a) Cue-conflict’s Post Hoc Filtering(b) Distorted Model Predictions + Shape Texture Conflicting Cue ⋯ class 1class 2class 3 class 1,000 0.070.010.030.05 Discard 793 logits Shape Bias Texture Bias == ++ : correct shapes: correct textures Truncated Evaluation Post Hoc Filtering Full Decision Space Correct decisions 1.0 0.4 0.8 0.6 0.2 0.0 Accuracy : False positive increased by 3.4× 63 215 Shape Classes Correct decisions: False positive increased by 3.1× 529168 1.0 0.4 0.8 0.6 0.2 0.0 Accuracy Texture Classes Correct textures without filtering Correct textures with filtering Correct shapes without filtering Correct shapes with filtering Figure I: Illustration of post hoc filtering and the resulting distortion in cue-conflict benchmark. Solid and striped bars indicate the average number of correct top-1 predictions across 20 ImageNet-1k pretrained models (in Appendix B) with and without posthoc filtering, respectively. A.11 Architecture Dependent Shape and Texture Trade-off ResNetInceptionV3AlexNetVGGBagNet (a) Cue-conflict Dataset & Preference Metric Shape preference Texture preference 00.30.40.60.80.20.510.70.10.9 00.30.40.60.80.20.510.70.10.9 0.9 0.7 0.6 0.8 0.4 0.5 Top - 1 Acc (ImageNet - 1k) 0.9 0.7 0.6 0.8 0.4 0.5 Shape-SensTexture-Sens Top - 1 Acc (ImageNet - 1k) 0.9 0.7 0.6 0.8 0.4 0.5 (b) REFINED-BIAS Dataset & Sensitivity Metric (c) REFINED-BIAS Dataset & Sensitivity Metric 00.30.40.60.80.20.510.70.10.900.30.40.60.80.20.510.70.10.9 Top - 1 Acc (ImageNet - 1k) 흆RF, Texture−Sens =−ퟎ.ퟗퟏퟐ (퐩<ퟎ.ퟎퟓ) 흆RF, Shape−Sens =ퟎ.ퟗퟑퟖ퐩<ퟎ.ퟎퟓ 흆RF, Shape preference =ퟎ.ퟗퟏퟐ (퐩<ퟎ.ퟎퟓ) Receptive Field Size 92501505001k Receptive Field Size 92501505001k Receptive Field Size 92501505001k Figure J: Comparison of model architectures under a fixed learning strategy and their ImageNet-1k top-1 accuracy. (a) Model preferences are measured on the cue-conflict dataset using the preference metric. (b) Model shape sensitivities and (c) texture sensitivities are measured on theREFINED-BIAS dataset. The point size reflects the receptive field (RF) size of each model architecture, sourced from Araujo et al. [52]. ρ denotes the Spearman’s correlation coefficient. We observe an architecture-dependent shape–texture trade-off in the relationship between in-domain accuracy and cue sensitivity (Fig. 9c and d). We hypothesize that this behavior can be explained by differences in receptive field (RF) size across CNN architectures. In particular, we observe a positive correlation between RF size and shape sensitivity across CNN architectures (0.938 in Fig. Jb). This observation is consistent with prior analyses [4,53–56] suggesting that models with larger RF sizes capture broader spatial context, which may help integrate object-level structural information. Conversely, smaller RF size positively correlate with higher texture sensitivity (0.912 in Fig. Jc), which aligns with prior studies [11,57] showing that models with smaller RF size tend to rely on local image features such as textures. Taken together, these suggest that the RF size of CNNs can explain the observed shape–texture sensitivity trade-off. Although a similar trend appears in the cue-conflict benchmark (Fig. Ja), it arises only because the preference metric enforces the shape–texture trade-off by its design. Although these correlations do not strictly imply causality, they highlight the RF as a plausible underlying mechanism that partially explains the observed trade-off. 21 B Model Architectures B.1 Details of Models Architectures CNN architectures including BagNet [57], AlexNet [58], VGG [59], ResNet [30], and InceptionV3 [60] are trained with standard random resized cropping. Vision transformer variants, including ViT [44], Swin Transformer [61], and CMT [62], additionally employ mixed augmentation strategies during training. Due to these differences, comparisons in Fig. 8 and 9 are conducted within each model family. Table B: Effect of training strategies on shape/texture sensitivity (withREFINED-BIASdataset), bias (with cue-conflict dataset), and in-domain performance (i.e., top-1 accuracy on ImageNet-1k). SB and TB indicate relative shape and texture biases, respectively. RF denotes the receptive field size. The full and partial spaces denote full and partial decision spaces, respectively. The colors in the models column indicate the model family. The full and partial spaces denote full and partial decision spaces, respectively. ImageNet-1kREFINED-BIASCue-conflict Full SpaceFull SpacePartial Space Arch.ModelsRFIn-domainShape-SensTexture-SensSBTBSBTB ViT ●ViT-S [44]-0.78600.74560.66710.27270.72730.28900.7110 ● ViT-B [44]-0.78980.71940.64180.33850.66150.32180.6782 ●Swin-S [61]-0.82900.77390.75050.20650.79350.23970.7603 ● Swin-B [61]-0.83070.80470.75060.24360.75640.24940.7506 ●CMT-S [62]-0.83310.83240.60850.31850.68150.28590.7141 ● CMT-B [62]-0.84520.88680.62280.33120.66880.29880.7012 CNN ● BagNet-9 [57]90.46360.05170.71940.04980.95020.06500.9350 ●BagNet-17 [57]170.58650.09950.80170.00760.99240.04840.9516 ●BagNet-33 [57]330.64210.23860.80080.01860.98140.06220.9378 ● AlexNet [58]1670.56540.36170.56480.20000.80000.26360.7364 ●VGG-11 [59]1500.70370.45230.73580.07530.92470.10550.8945 ●VGG-13 [59]1560.71550.43370.73240.08200.91800.11520.8848 ●VGG-16 [59]2120.73370.55340.66720.09940.90060.11290.8871 ●VGG-19 [59]2680.74220.61300.61600.13730.86270.15320.8468 ●ResNet-18 [30]4350.69750.57410.56740.19130.80870.23670.7633 ●ResNet-34 [30]8990.73280.56800.56350.24380.75620.25950.7405 ●ResNet-50 [30]4270.76130.56300.56840.19440.80560.22210.7779 ●ResNet-101 [30]9710.77370.65900.53030.27000.73000.28660.7134 ● ResNet-152 [30]14510.78310.63780.51790.26120.73880.29400.7060 ● InceptionV3 [60]13110.77310.69440.52330.55830.44170.50380.4962 22 C Model Training Strategies C.1 Details of Models with Different Training Strategies Table C: Effect of training strategies on shape/texture sensitivity (withREFINED-BIASdataset), bias (with cue-conflict dataset), and in-domain performance (i.e., top-1 accuracy on ImageNet-1k). SB and TB indicate relative shape and texture biases, respectively. The full and partial space denote full and partial decision spaces, respectively. The colors in the models column indicate the model family. The full and partial spaces denote full and partial decision spaces, respectively. ImageNet-1k REFINED-BIASCue-conflict Full SpaceFull SpacePartial Space ModelsIn-domain Shape-Sens Texture-Sens SBTBSBTB ♦ Vanila ResNet-500.76130.56300.56840.1784 0.8216 0.2221 0.7779 ★ A1 [40]0.80090.83070.67400.2193 0.7807 0.2054 0.7946 nA2 [40]0.79780.75150.68800.1617 0.8383 0.1908 0.8092 rA3 [40]0.77540.77370.69270.1420 0.8580 0.1864 0.8136 ✣ B [40]0.79250.83670.59600.2014 0.7986 0.2011 0.7989 ♠C1 [40]0.79740.85320.67820.2230 0.7770 0.1979 0.8021 ▲ C2 [40]0.79900.83820.66070.2143 0.7857 0.2171 0.7829 ♣ D [40]0.79880.84570.63630.1958 0.8042 0.1957 0.8043 kV2 [40]0.80330.74670.73860.2116 0.7884 0.1874 0.8126 nAugMix [35]0.77520.72850.52780.2776 0.7224 0.3177 0.6823 bPixMix [36]0.78070.66480.59560.2796 0.7204 0.3196 0.6804 ♦ OpticsAugment [38]0.74210.62460.54930.1953 0.8047 0.2652 0.7348 ♠DeepAugment [39]0.76640.70670.52250.3372 0.6628 0.3557 0.6443 ♣ PRIME [37]0.76890.79950.52930.3222 0.6778 0.3378 0.6622 rShapeNet: SIN+IN [7]0.74580.72510.51370.3717 0.6283 0.3657 0.6343 ✣ ShapeNet: SIN+IN+FT [7]0.76710.67980.53650.1993 0.8007 0.2290 0.7710 ♠Shape Bias Augmentation [31]0.76190.68430.54630.3000 0.7000 0.3128 0.6872 ♠MoCo v3 [32]0.74580.61110.44090.2704 0.7296 0.3093 0.6907 rSimCLR v2 [33]0.74890.58690.43030.2043 0.7957 0.2528 0.7472 ✣ DINO v1 [34]0.75270.67050.53140.1136 0.8864 0.1639 0.8361 ●PGD-AT (l 2 , ε = 0.05) [42, 63]0.75570.66450.54680.1878 0.8122 0.2678 0.7322 ●PGD-AT (l 2 , ε = 0.1) [42, 63]0.74780.50700.54840.2436 0.7564 0.3198 0.6802 ●PGD-AT (l 2 , ε = 0.25) [42, 63]0.74120.57030.53110.2941 0.7059 0.3786 0.6214 ●PGD-AT (l 2 , ε = 0.5) [42, 63]0.73170.50700.54810.4680 0.5320 0.4329 0.5671 ●PGD-AT (l 2 , ε = 1) [42, 63]0.70420.53630.54060.5771 0.4229 0.5306 0.4694 ●PGD-AT (l 2 , ε = 3) [42, 63]0.62830.49180.49780.8615 0.1385 0.6883 0.3117 ● PGD-AT (l 2 , ε = 5) [42, 63]0.56130.42920.50270.9029 0.0971 0.7379 0.2621 ■PGD-AT (l ∞ , ε = 0.5) [42, 63]0.73170.60210.53830.3864 0.6136 0.4124 0.5876 ■PGD-AT (l ∞ , ε = 1) [42, 63]0.70420.55830.53840.5571 0.0229 0.4967 0.5033 ■PGD-AT (l ∞ , ε = 2) [42, 63]0.69080.52370.50860.6946 0.3054 0.5756 0.4244 ■PGD-AT (l ∞ , ε = 4) [42, 63]0.63860.49630.51440.8302 0.1698 0.6429 0.3571 ■ PGD-AT (l ∞ , ε = 8) [42, 63]0.54520.52680.50390.9087 0.0913 0.7361 0.2639 23 C.2 Confidence Intervals for Models with Different Training Strategies Table D: Confidence intervals (95%CI) of texture bias and shape bias measured usingREFINED-BIAS for models trained with different strategies. The colors in the Models column indicate the model family. The overlapping CIs indicate similar shape or texture bias, while non-overlapping CIs indicate the opposite. Shape-SensTexture-Sens ModelsLowerMeanUpperLowerMeanUpper ♦ Vanila ResNet-500.55480.56370.57240.55380.56900.5823 ★ A1 [40]0.82150.83010.84050.65320.67490.6839 nA2 [40]0.74020.75060.75930.67210.68830.7013 rA3 [40]0.72700.73640.74620.67150.69310.7085 ✣ B [40]0.82830.83570.84330.57640.59610.6149 ♠C1 [40]0.84400.85390.86580.66270.67930.6948 ▲ C2 [40]0.83000.83530.84450.64630.66240.6745 ♣ D [40]0.83800.84590.85420.62020.63760.6510 kV2 [40]0.73520.74400.75380.72490.74020.7507 ❖ AugMix [35]0.72090.72990.73910.50700.52710.5417 bPixMix [36]0.66000.66540.67400.57460.59560.6104 ♦ OpticsAugment [38]0.62090.62760.63760.52630.54750.5633 ♠DeepAugment [39]0.69800.70630.71740.51390.52420.5339 ♣ PRIME [37]0.79290.80190.81300.51240.53200.5466 rShapeNet: SIN+IN [7]0.71650.72640.73500.49680.51340.5263 ✣ ShapeNet: SIN+IN+FT [7]0.67440.68190.69310.51670.53600.5507 ♠Shape Bias Augmentation [31]0.67430.68500.69710.52830.54530.5599 ♠MoCo v3 [32]0.60420.61340.62310.42660.44290.4568 rSimCLR v2 [33]0.58140.58780.59930.41520.43380.4477 ✣ DINO v1 [34]0.66190.66840.67810.52050.53330.5474 ●PGD-AT (l 2 , ε = 0.05) [42, 63]0.65640.66810.67800.53210.54950.5653 ●PGD-AT (l 2 , ε = 0.1) [42, 63]0.50220.50880.51610.53120.54960.5625 ●PGD-AT (l 2 , ε = 0.25) [42, 63]0.56640.57430.58160.52100.53290.5428 ●PGD-AT (l 2 , ε = 0.5) [42, 63]0.50960.51490.52130.53530.54770.5604 ●PGD-AT (l 2 , ε = 1) [42, 63]0.52810.53830.54930.53210.54230.5556 ●PGD-AT (l 2 , ε = 3) [42, 63]0.48810.49550.50210.48990.50100.5143 ● PGD-AT (l 2 , ε = 5) [42, 63]0.42490.43150.44020.49470.50560.5193 ■PGD-AT (l ∞ , ε = 0.5) [42, 63]0.59920.60650.61250.52630.53940.5526 ■PGD-AT (l ∞ , ε = 1) [42, 63]0.55420.56140.57070.52490.53870.5561 ■PGD-AT (l ∞ , ε = 2) [42, 63]0.52310.52790.53700.49860.51080.5229 ■PGD-AT (l ∞ , ε = 4) [42, 63]0.49100.50130.51170.50420.51420.5277 ■ PGD-AT (l ∞ , ε = 8) [42, 63]0.52160.53120.53810.49240.50380.5160 24 C.3 Configurations of Models Trained with Mixed Augmentations Table E: Training settings and augmentation combinations of models provided by timm [40]. AGC refers to adaptive gradient clipping, CE and BCE represent cross-entropy and binary cross-entropy loss, respectively. AugmentationsA1A2A3BC1C2DV2 RandAugOOOOOOOX MixUpOOOOOOOO CutMixOOOXOOOO Random ErasingXXXOOOOO Repeated Aug.OOXXXOXO Auto Aug.X O Label SmoothingOXXOOOOO EMAXXXOXXXO Grad ClippingXXXXAGCAGCXX Optimizerlamblamblambrmspropsgdsgdadampsgd LR decaycosine cosine cosinestepcosine cosinecosinecosine LossBCEBCEBCECECECEBCECE Epoch600300100600800800600600 Weight Decay0.010.020.027e-61e-51e-50.012e-5 Stochastic Depth0.050.05X0.10.10.10.05X 25 C.4 Visual Examples for Different Learning Strategies Original Image Fine-grained RandAug (mixture) Coarse Color Jitter Fine-grained Posterize Coarse Solarize Coarse Figure K: Examples of the RandAug method used in most mixed-augmentation models. The figure shows the effects of Color Jitter, Posterize, and Solarize augmentations employed by RandAug. The boxed regions in each image highlight the degree of texture change. PixMix Confounded Original Image Fine-grained AugMix Coarse OpticsAugment Coarse DeepAugment Coarse PRIME Coarse Figure L: Examples of texture degradation methods. The boxed regions in each image highlight the degree of texture change. Original ImageFlip & CropDino v1SimCLR v2MoCo v3 Fine-grainedCoarseFine-grainedCoarseCoarse Figure M: Examples of augmentations used for each contrastive learning method. The parameters of all augmentations follow the original model settings. The boxed regions in each image highlight the degree of texture change. Original Image푙 ! : 휖=0.5푙 ! : 휖=1푙 ! : 휖=3푙 ! : 휖=5 Fine-grainedNearly sameNearly sameNearly sameNearly same Figure N: Examples of adversarial images generated by adversarial training methods under varying l 2 distance. The overall images are perceptually similar in both shape and texture compared to the original images. Original Image푙 ! : 휖=1푙 ! : 휖=2푙 ! : 휖=4푙 ! : 휖=8 Fine-grainedNearly sameNearly sameNearly sameNearly same Figure O: Examples of adversarial images generated by adversarial training methods under varying l ∞ distance. The overall images are perceptually similar in both shape and texture compared to the original images. 26