Paper deep dive
Perceptual compensation for tonal context in self-supervised speech models
James Kirby, Ioana Krehan, Michele Gubian
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 95%
Last extracted: 6/21/2026, 1:12:50 AM
Summary
This study investigates whether the wav2vec2.0 architecture exhibits perceptual compensation (PC) for Mandarin Chinese lexical tones. By comparing a purely self-supervised pre-trained (PT) model with a fine-tuned (FT) Mandarin ASR model using embedding similarities and probing classifiers, the researchers found that the PT model shows no evidence of tonal context compensation. While the FT model showed some evidence of context sensitivity in later layers, it failed to replicate human-like performance on isolated syllables. The findings suggest that supervised learning objectives may be necessary for neural speech models to achieve the abstraction of certain phonological regularities like lexical tone.
Entities (6)
Relation Signals (4)
James Kirby → authored → Perceptual compensation for tonal context in self-supervised speech models
confidence 100% · Perceptual compensation for tonal context in self-supervised speech models James Kirby
wav2vec2.0 → examines → Perceptual Compensation
confidence 100% · This study examines the extent to which the wav2vec2.0 architecture exhibits evidence of compensation for phonological context.
Mandarin Chinese → hasfeature → Lexical Tone
confidence 100% · Most lexical syllables in Mandarin Chinese bear one of four tones
AISHELL-3 → usedfortraining → wav2vec2.0
confidence 90% · extracting disyllable sequences from forty speakers from the test split of the AISHELL-3 corpus
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:This study examines the extent to which the wav2vec2.0 architecture exhibits evidence of compensation for phonological context. We conducted a pseudo-replication of a perceptional compensation experiment on Mandarin Chinese tones, and compared the embedding similarities and probing classifier outputs between a purely self-supervised pre-trained model and a model fine-tuned for Mandarin ASR. No evidence of compensation was found in the embedding similarities of the purely pre-trained model. Probing classifiers showed some evidence of compensation in addition to the expected layer-wise improvements in categorization, but failed to replicate human performance on isolated test syllables. Our findings contrast with previous reports of sensitivity to phonological structure emerging through pre-training alone, and suggest that supervised objectives may be necessary to encourage the abstraction of at least some types of phonological regularities.
Tags
Links
- Source: https://arxiv.org/abs/2606.17835v1
- Canonical: https://arxiv.org/abs/2606.17835v1
Trouble viewing inline? Open PDF directly →
Full Text
27,294 characters extracted from source content.
Expand or collapse full text
Perceptual compensation for tonal context in self-supervised speech models James Kirby ID 1 , Ioana Krehan 1 , Michele Gubian 1 1 Institute for Phonetics and Speech Processing, LMU Munich, Germany jkirby@phonetik.uni-muenchen.de,Ioana.Krehan@campus.lmu.de,m.gubian@phonetik.uni-muenchen.de Abstract This study examines the extent to which the wav2vec2.0 ar- chitecture exhibits evidence of compensation for phonologi- cal context. We conducted a pseudo-replication of a percep- tional compensation experiment on Mandarin Chinese tones, and compared the embedding similarities and probing classifier outputs between a purely self-supervised pre-trained model and a model fine-tuned for Mandarin ASR. No evidence of com- pensation was found in the embedding similarities of the purely pre-trained model. Probing classifiers showed some evidence of compensation in addition to the expected layer-wise improve- ments in categorization, but failed to replicate human perfor- mance on isolated test syllables. Our findings contrast with pre- vious reports of sensitivity to phonological structure emerging through pre-training alone, and suggest that supervised objec- tives may be necessary to encourage the abstraction of at least some types of phonological regularities. Index Terms: speech perception, self-supervised learning, lex- ical tone, Mandarin Chinese 1. Introduction This paper aims to contribute to our understanding of the role of phonological context in speech perception through computa- tional modeling. We do so by exploring how one neural speech architecture, wav2vec2.0 [1], represents lexical tones in differ- ent phonological contexts. By treating wav2vec2.0 as a compu- tational model of perceptual inference and comparing its behav- ior to that of human listeners, we gain insight into the extent to which these types of models encode phonological structure and what kind of information they use to do so. Perceptual compensation (PC) refers to the phenomenon of context-dependent variation in phonetic category perception, whereby listeners assign acoustically identical speech signals to different categories depending on the phonetic or phonologi- cal context [2]. For example, a following rounded vowel biases fricative perception towards /s/ and away from /S/, despite the fact that the anticipatory lip rounding makes the fricative acous- tically more similar to /S/ [3, 4]. Such context effects have been documented for many speech sounds, in infants [5], and even in nonhuman animals [6], leading to a debate regarding the ex- tent to which the effect can be explained by low-level phonetic mechanisms alone [7, 8]. Modern neural speech recognition systems are often based on models which have been pre-trained using self-supervised learning (SSL). In this framework, models are first pre-trained using a pretext task based on audio only, and then fine-tuned for specific tasks using supervised learning [9]. The success of the SSL paradigm has spurred much interest in understanding the kinds of linguistic information implicitly encoded in the repre- sentations learned by this class of models [10, 11, 12, 13, 14], as well as how model behavior compares with that of humans [15, 16, 17, 18]. For example, [15] show how a supervised ASR model adapts phoneme category boundaries in response to ambiguous training examples in a manner similar to that of humans, while [17] show that a wav2vec2.0 model fine-tuned for English ASR compensates for assimilation more often in phonotactically viable contexts than in non-viable ones. These findings can be interpreted as evidence that neural speech mod- els have acquired knowledge of how speech sounds are realized in different phonological contexts, at least when they have been fine-tuned for tasks involving explicit linguistic categories. Arguably more surprising are claims that purely pre-trained models also display such behavior. Like [17], [18] investigated how wav2vec2.0 encodes ambiguous speech sounds in phono- tactically viable and non-viable contexts. They observed an ef- fect of context in both pre-trained and fine-tuned models, lead- ing them to suggest that the model can implicitly learn English phonotactic structure even without a symbolic training objec- tive. Such findings are striking, because they suggest that the context encoder networks of SSL models may be implicitly cap- turing information about phonological context without being given any explicit information about phonological categories. This would be consistent with accounts of PC that are based on purely phonetic knowledge [19, 20]. In this paper, we contribute to this line of research by testing the extent to which a neural speech architecture compensates for context in the domain of lexical tone, through a pseudo- replication of a psycholinguistic study of PC in Mandarin Chi- nese [21]. We hypothesize that the contextualized speech em- beddings learned with self-supervision may be especially adept at compensating for suprasegmental contexts such as tone. We focus here on the wav2vec2.0 architecture, as it has been widely used in previous related work and because both pre-trained (PT) and fine-tuned (FT) checkpoints exist for Mandarin Chinese. We analyze how the model encodes semi-controlled psycholin- guistic stimuli through the use of probing classifiers and the study of embedding similarities. We find that while the embedding similarities of the FT model show some evidence of PC in later network layers, those of the PT model show no evidence of compensation for tonal context at any layer. Probing classifiers tested using both PT and FT embeddings show some indications of PC, but the clas- sification of isolated syllables encoded without context diverges sharply from that of human listeners. Our results therefore sug- gest caution before concluding that self-supervised pre-training objectives are sufficient for neural speech models to integrate information about phonological category representations; su- pervised learning may still be necessary to encourage the in- tegration of at least some types of phonological context. arXiv:2606.17835v1 [cs.CL] 16 Jun 2026 Figure 1: Left: F0 contours of the four lexical tones of Mandarin Chinese on sonorant syllables extracted from the AISHELL-3 corpus [23]. Right: realization of Tone 3 in the context of preceding tones. 2. Background Most lexical syllables in Mandarin Chinese bear one of four tones: high-level Tone 1 ([ba1]八 ‘eight’), high-rising Tone 2 ([ba2]拔 ‘to pull’), low-falling(-rising) Tone 3 ([ba3]把 ‘to take’) or high-falling Tone 4 ([ba4]爸 ‘father’). Like segments, the phonetic realization of tones is heavily influenced by car- ryover coarticulation. For example, although Mandarin T3 is typically realized with a low-falling contour in post-pausal po- sition (Fig. 1, left), when preceded by a tone that ends high, like T1, it is realized with a high-falling contour (Fig. 1, right) [22]. In other words, when following a T1 or T2, the onset of T3 is raised, increasing its acoustic similarity to canonical T4; similarly, in the context of a preceding T4, the onset of T4 is lowered, increasing its acoustic similarity to canonical T3. In perception, however, human listeners compensate for these coarticulatory effects [21, 22, 24, 25]. [21] demonstrated the effect of preceding tonal context on the perceptual boundary between low-falling T3 and high-falling T4 by having partici- pants classify stimuli drawn from a 14-step T4-T3 continuum using a two-alternative forced choice paradigm. Each trial con- sisted of a target stimulus either in isolation, or preceded by one of the three context syllables. Results (Fig. 2) showed a clear preference for T3 responses when the preceding tone was high-offset T1 or T2, and T4 when the preceding tone was low- offset T4. Importantly, stimuli presented in isolation (labeled ‘no-ctx’ in Fig. 2) were intermediate, suggesting that the tonal context serves to bias a general phonological representation. 3. Methods 3.1. Stimuli To investigate whether wav2vec2.0 displays a similar bias in its responses, we generated continua of Mandarin syllables with fixed segmental material but F0 contours varying between canonical T3 and T4 endpoints. Typical for psycholinguistic studies, [21] tested a small number of stimuli on a large number of participants. Since model responses to individual stimuli will be deterministic, we introduced variability by resynthesizing a large number of tonal continua with different preceding tone contexts by extracting disyllable sequences from forty speak- ers from the test split of the AISHELL-3 corpus [23], using the Montreal Forced Aligner [26] to determine syllable boundaries. As in [21], context syllables bore one of T1, T2 or T4, while target syllables bore either T3 or T4. To exclude creaky sam- ples, we analyzed the F0 of potential target audio clips and se- Figure 2: Proportion of T4 responses from human listeners in Experiment 3 of [21]. lected only those where at least 10 F0 samples were present and above the 10% quantile of F0 for the corresponding speaker. We then used Parselmouth [27, 28] to apply the duration and F0 manipulations described in [21] to each context-target di- syllable to obtain a 14-step T4-T3 target continuum preceded by either a context syllable (with T1, T2, or T4) or in isola- tion (labeled as no-context). Speaker-dependent 10% and 90% quantiles were used to determine F0 extrema for the continuum endpoints. This procedure generated around 13,700 continua (ca. 192,000 stimuli). 3.2. Model checkpoints All stimuli were then processed by two wav2vec2.0 model checkpoints: one pre-trained on 1,000 hours of untranscribed Mandarin speech only 1 , and one fine-tuned for the task of Man- darin ASR using 178 hours of transcribed speech 2 . Both models contain 7-layer CNN feature encoders followed by 12 Trans- former layers. We used the Transformers library [29] to down- load and process both checkpoints. 3.3. Analysis methods We report results of two methods for analyzing SSL model representations: embedding similarities and probing classifiers. We applied both methods to the 512-dimensional output of the convolutional feature extractor (layer 0) as well as to the 768- dimensional outputs of the Transformer (layers 1-12). 3.3.1. Embedding similarities We inspected the internal representations of both models by comparing the embedding similarities of each stimulus step with the corresponding T3 and T4 endpoints (cf. [18]). For a given stimulus X , the corresponding manipulated step-1 (T4) and step-14 (T3) endpoints based on the same source syllable were used as reference stimuli. For each stimulus, vectors were derived by averaging layer-wise embeddings over the entire syl- lable. Following [18], we then computed the similarity of X to T4 relative to T3 as: sim(X, T4) = 1− D cos (X, T4) D cos (X, T4) + D cos (X, T3) ,(1) where D cos (a, b) = 1−cos(a, b). Similarities were modeled as a continuous, [0,1] bounded beta-distributed response variable 1 https://github.com/kehanlu/mandarin-wav2vec2 2 https://huggingface.co/kehanlu/ mandarin-wav2vec2-aishell1 Figure 3: T4 embedding similarities (1) as function of T4-T3 continuum step; line thickness indicates 95% confidence bands. in a generalized additive mixed model (GAMM) using the mgcv R library [30]. Four smooth functions of step model the effect of context (T1, T2, T4 or no-ctx), where step ranges between 2 and 13 (extremes are excluded as they coincide with the references 1=T4 and 14=T3). Speaker-by-context random smooth terms were added to control for speaker variability. One GAMM was fit independently for each checkpoint and layer. 3.3.2. Probing classifiers We trained probing classifiers to predict T3/T4 tone labels from the representations extracted from each model checkpoint and layer, following [31]. As for the similarity computation, vectors were obtained by averaging embeddings over target syllables. Probing classifiers were linear, fully connected neural net- works implementing binary logistic regression with cross- entropy loss, using the Adam optimizer with learning rate 10 −3 . Classifiers were trained on sets of T3 and T4 syllables from forty speakers from the train split of the AISHELL-3 corpus. We trained on 100 T3 and 100 T4 syllables for each of 36 speak- ers (18 female) and validated on 4 speakers (2 female). Training was run for five epochs. Validation accuracy saturated between 82% in the CNN layers and up to 99% in the higher Transformer layers. Errors were moderately imbalanced in favor of false T3 recognition, the most extreme being layer 12, where 4-6 times more false T3s than false T4s were predicted. Testing was carried out on the classifier state after the last training epoch. The test set consisted of the manipulated stimuli described in Sec. 3.1. Binary results are modeled as a Bernoulli- distributed response variable in GAMMs with the same predic- tor structure described in Sec. 3.3.1 (step extremes included). 4. Results Here we only show the results for layer 0 (CNN output) and layers 4, 8, and 12; results for other layers follow similar trends. 4.1. Embedding similarities Embedding similarities for both PT and FT checkpoints are shown in Fig. 3. There is no evidence of sensitivity to context in the PT embeddings (top row). There appears to be a small effect of the addition of context (compared to the no-context condition), but not of compensation in the sense seen in [21]. The embedding similarities from the FT model (bottom row), on the other hand, do show some evidence of context sensitiv- ity: ambiguous stimuli in a T1 context have contextual repre- sentations that are more similar to T3 than to T4, indicating that fine-tuning fundamentally alters the model’s internal represen- tations. However, these shifts are quite small compared to the human responses, and again do not seem to be relative to the no- context embeddings. In addition, the contexts T2 and T4 appear to pattern together relative to T1, with T1 showing the greatest shift, which is again a qualitatively different pattern compared to [21] (Fig. 2). 4.2. Probing classifiers Fig. 4 shows the proportion of T4 responses at each continuum step for PT and FT models at four layers. A mild effect of con- text is visible at the final CNN layers, but already in the fourth Transformer layers we see an enhanced sensitivity to the con- tinuum endpoints together with sensitivity to context. This is strongest in layer 8 of the FT model (bottom row, third panel) where the response patterns most qualitatively resemble those of human participants (Fig. 2). However, in the no-context condition, the expected sig- moidal response curve fails to emerge in either model at any layer. This is most visible in the probes tested on FT embed- dings, where we observe a strong bias for T4 responses regard- less of the F0 contour of the stimulus. 5. Discussion This study explored the extent to which two wav2vec2.0 mod- els, one pre-trained and one fine-tuned, exhibited evidence of compensation for phonological context. Purely self-supervised representations showed no evidence of the baseline-centered boundary shifts that characterized perceptual compensation for tonal coarticulation in [21]. While some evidence of compen- sation was seen in the later layers of the fine-tuned model, fine- tuning induced only weak, qualitatively non-human-like shifts Figure 4: Proportion of T4 responses of probes as function of T4-T3 continuum step; line thickness indicates 95% confidence bands. that are not clearly anchored to the no-context baseline. The lack of evidence for PC in the purely self-supervised model was inconsistent with our hypothesis that Transformer- based embeddings would have an advantage at inducing con- textually specific tonal representations, as well as with previ- ous reports of phonological encoding emerging in purely self- supervised models [13, 14, 18, 32, 33]. Indeed, our most robust finding is the complete lack of context sensitivity in purely pre- trained embeddings (Fig. 3, top). To what should we attribute these differences? One pos- sibility is that the acoustic realizations of the segmental [r/l] contrast studied by both [15] and [18] are overall less variable across speakers and contexts than lexical tones, which are real- ized primarily through modulation of F0, a trajectory subject to perturbation via baseline shifts, range scaling, truncation, in- tonational trends, and local segmental perturbations. In other words, the set of acoustic dimensions that strongly affect the acoustic signal for tone may be much larger than for something like the [r/l] contrast, because F0 is jointly determined by a large number of extraneous factors. From the model’s perspec- tive, the influence of the preceding tonal context is competing with many other plausible explanations to account for the ob- served F0 trajectory. Both the embeddings of the FT model, as well as the lin- ear probes fit to the embeddings of both models, showed some evidence of PC. We attribute this to the fact that both the fine- tuning objective and the probing classifiers involving super- vised training, where the model or classifier is explicitly told of the existence of lexical tone categories. However, while the probing results showed the expected increase in accessibility of tone categories across layers, the responses to isolated sylla- bles still diverged sharply from that of human listeners. This finding highlights a potentially important difference between human and machine processing of phonological context: the representations learned by even fine-tuned models seem to rely on broader contextual conditioning, rather than supporting a context-invariant tonal category biased by local context. However, we must acknowledge some limitations of our probing results. Our probes were trained on instances of T3 and T4 extracted from embeddings made from whole utterances, en- coding rich contextual information, whereas they were tested on embeddings made from isolated syllables. Specifically for the task of single-syllable prediction, the utterance-level con- text encoded by the FT model may actually be detrimental for the specific task of syllable-level tone identification [34, 35]. While we were careful to insure our probes did not encode a fre- quency bias (Sec. 3.3.2), we nonetheless observed a strong bias for T4 when classifying isolated syllables using the FT model. We have no explanation for this at present, but speculate that this may reflect the fact that T4 is the most frequent tone in Mandarin speech [36]. Moreover, for many speakers, a canon- ical pre-pausal T3 will be realized with a final rise, which was never present in our stimuli (or those of [21]). On the other hand, our probes were more accurate than the embedding similarities when classifying our disyllabic test stimuli, even for the PT model. Although it is tempting to in- terpret high probing accuracy as indicating the model ‘encodes’ tonal context, it is more accurate to say that it indicates that the model has learned a representation in which the probed cat- egories are accessible. If the categories in question correlate with other properties in the training data, even a simple linear model may be able to find a fit to subtle correlations that do not directly reflect the property of interest [37]. In future work, we intend to supplement our analysis measures with causal in- terchange interventions [17] and task-based evaluations [18], as well as considering the effects of nonspeech analogs as in [21]. In summary, our results highlight a dissociation between contextualization and perceptual compensation. Purely self- supervised wav2vec2.0 representations showed no evidence of compensation for tonal coarticulation, and even supervised probes fit to fine-tuned representations diverged sharply from responses of human listeners on isolated syllables, suggesting that the learned representations rely on broader contextual con- ditioning rather than biasing a context-invariant baseline. Taken together, these results constrain accounts of perceptual compen- sation by indicating that, at least for tones, human-like PC pat- terns are unlikely to fall out of unsupervised contextual predic- tion alone, but instead require additional learning mechanisms that encourage the emergence of stable phonological category representations. 6. Generative AI Use Disclosure No generative AI or AI-assisted technologies were used in the research process or the preparation of this manuscript. 7. References [1] A. Baevski, H. Zhou, A. Mohamed, and M. Auli, “wav2vec 2.0: A framework for self-supervised learning of speech representa- tions,” in Proc. NIPS, 2020, p. 12 449–12 460. [2] M. Sonderegger and A. Yu, “A rational account of percep- tual compensation for coarticulation,” Proceedings of the Annual Meeting of the Cognitive Science Society, vol. 32, p. 375–380, 2010. [3] V. A. Mann and B. H. Repp, “Influence of vocalic context on per- ception of the [ R ]-[s] distinction,” Perception & Psychophysics, vol. 28, no. 3, p. 213–228, 1980. [4] H. Mitterer, “On the causes of compensation for coarticula- tion: Evidence for phonological mediation,” Perception & Psy- chophysics, vol. 68, no. 7, p. 1227–1240, 2006. [5] C. A. Fowler, C. T. Best, and G. W. McRoberts, “Young infants’ perception of liquid coarticulatory influences on following stop consonants,” Perception & Psychophysics, vol. 48, no. 6, p. 559– 570, 1990. [6] A. J. Lotto, K. R. Kluender, and L. L. Holt, “Perceptual compen- sation for coarticulation by Japanese quail,” The Journal of the Acoustical Society of America, vol. 102, no. 2, p. 1134–1140, 1997. [7] R. L. Diehl, A. J. Lotto, and L. L. Holt, “Speech perception,” Annual Review of Psychology, vol. 55, no. 1, p. 149–179, 2004. [8] A. G. Samuel, “Speech perception,” Annual Review of Psychol- ogy, vol. 62, no. 1, p. 49–72, 2011. [9] A. Mohamed, H.-y. Lee, L. Borgholt, J. D. Havtorn, J. Edin, C. Igel, K. Kirchhoff, S.-W. Li, K. Livescu, L. Maaløe et al., “Self-supervised speech representation learning: A review,” IEEE Journal of Selected Topics in Signal Processing, vol. 16, no. 6, p. 1179–1210, 2022. [10] D. Wells, H. Tang, and K. Richmond, “Phonetic analysis of self-supervised representations of English speech,” in INTER- SPEECH, 2022, p. 3583–3587. [11] A. Pasad, C.-M. Chien, S. Settle, and K. Livescu, “What do self-supervised speech models know about words?” Transac- tions of the Association for Computational Linguistics, vol. 12, p. 372–391, 2024. [12] M. Yang, R. C. M. C. Shekar, O. Kang, and J. H. L. Hansen, “What can an accent identifier learn?probing phonetic and prosodic information in a wav2vec2-based accent identification model,” in INTERSPEECH, 2023, p. 1923–1927. [13] G. Shen, M. Watkins, A. Alishahi, A. Bisazza, and G. Chrupała, “Encoding of lexical tone in self-supervised models of spoken lan- guage,” in Proc. NAACL, 2024, p. 4250–4261. [14] A. de la Fuente and D. Jurafsky, “A layer-wise analysis of Man- darin and English suprasegmentals in SSL speech models,” in IN- TERSPEECH, 2024. [15] O. Scharenborg, S. Tiesmeyer, M. Hasegawa-Johnson, and N. De- hak, “Visualizing phoneme category adaptation in Deep Neural Networks,” in INTERSPEECH, 2018, p. 1482–1486. [16] J. Millet and E. Dunbar, “Do self-supervised speech models de- velop human-like perception biases?” in Proc. ACL 60, 2022, p. 7591–7605. [17] C. Pouw, M. d. H. Kloots, A. Alishahi, and W. Zuidema, “Per- ception of phonological assimilation by neural speech recognition models,” Computational Linguistics, vol. 50, no. 4, p. 1557– 1585, 2024. [18] M. de Heer Kloots and W. Zuidema, “Human-like linguistic biases in neural speech models: Phonetic categorization and phonotactic constraints in wav2vec2.0,” in INTERSPEECH, 2024. [19] D. W. Gow, “Feature parsing: Feature cue mapping in spoken word recognition,” Perception & Psychophysics, vol. 65, no. 4, p. 575–590, 2003. [20] D. W. Gow and A. M. Im, “A cross-linguistic examination of assimilation context effects,” Journal of Memory and Language, vol. 51, no. 2, p. 279–296, 2004. [21] H. Zhang, H. Ding, and W.-S. Lee, “The influence of preceding speech and nonspeech contexts on Mandarin tone identification,” Journal of Phonetics, vol. 93, p. 101154, 2022. [22] Y. Xu, “Production and perception of coarticulated tones,” The Journal of the Acoustical Society of America, vol. 95, no. 4, p. 2240–2253, 1994. [23] Y. Shi, H. Bu, X. Xu, S. Zhang, and M. Li, “AISHELL-3: A multi-speaker Mandarin TTS corpus,” in INTERSPEECH, 2021, p. 2756–2760. [24] F. Chen and G. Peng, “Context effect in the categorical percep- tion of Mandarin tones,” Journal of Signal Processing Systems, vol. 82, no. 2, p. 253–261, 2016. [25] R. A. Fox and Y.-Y. Qi, “Context effects in the perception of lexi- cal tone,” Journal of Chinese Linguistics, vol. 18, no. 2, p. 261– 284, 1990. [26] M. McAuliffe, M. Socolof, S. Mihuc, M. Wagner, and M. Son- deregger, “Montreal Forced Aligner:Trainable Text-Speech Alignment Using Kaldi,” in INTERSPEECH, 2017. [27] Y. Jadoul, B. Thompson, and B. de Boer, “Introducing Parsel- mouth: A Python interface to Praat,” Journal of Phonetics, vol. 71, p. 1–15, 2018. [28] P. Boersma and D. Weenink, “Praat: doing phonetics by computer [Computer program],” Version 6.1.38, retrieved 2 January 2021 http://w.praat.org/, 2021. [29] T. Wolf, L. Debut, V. Sanh, J. Chaumond, C. Delangue, A. Moi, P. Cistac, T. Rault, R. Louf, M. Funtowicz, J. Davison, S. Shleifer, P. von Platen, C. Ma, Y. Jernite, J. Plu, C. Xu, T. Le Scao, S. Gug- ger, M. Drame, Q. Lhoest, and A. Rush, “Transformers: State-of- the-Art Natural Language Processing,” in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Process- ing: System Demonstrations, 2020, p. 38–45. [30] S. N. Wood, “Fast stable restricted maximum likelihood and marginal likelihood estimation of semiparametric generalized lin- ear models,” Journal of the Royal Statistical Society Series B: Sta- tistical Methodology, vol. 73, no. 1, p. 3–36, 2011. [31] D. Ma, N. Ryant, and M. Liberman, “Probing acoustic represen- tations for phonetic properties,” in ICASSP, 2021, p. 311–315. [32] M. d. H. Kloots, H. Mohebbi, C. Pouw, G. Shen, W. Zuidema, and M. Bentum, “What do self-supervised speech models know about Dutch? Analyzing advantages of language-specific pre-training,” in Proc. Interspeech 2025, 2025, p. 256–260. [33] M. Bentum, L. ten Bosch, and T. O. Lentz, “Word stress in self- supervised speech models: A cross-linguistic comparison,” in Proc. Interspeech 2025, 2025, p. 251–255. [34] S. Robertson and E. Dunbar, “Bigger is not always better: The effect of context size on speech pre-training,” arXiv:2312.01515, 2023. [35] Y. Meng, S. Goldwater, and H. Tang, “Effective context in neural speech models,” in INTERSPEECH, 2025, p. 246–250. [36] Y. Wu, M. Adda-Decker, and L. Lamel, “Mandarin lexical tones: A corpus-based study of word length, syllable position and prosodic position on duration,” in INTERSPEECH, 2020, p. 1908–1912. [37] Y. Belinkov, “Probing classifiers: Promises, shortcomings, and advances,” Computational Linguistics, vol. 48, no. 1, p. 207– 219, 2022.