Paper deep dive
When Does a Laugh Begin? Structured Annotator Disagreement in Temporal Laughter Localization
Eyal Hanania, Daniel Arkushin, Naveh Ayal, Jonathan Benvenisti, Amos Bercovich, Elie Zemmour, Sahar Froim
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%
Last extracted: 9/12/2026, 2:51:49 AM
Summary
This paper introduces SMILE-Disagreement, a multi-annotator temporal laughter localization dataset, demonstrating that annotator disagreement is structured rather than random noise. Key findings include that offsets are 1.73x noisier than onsets and chuckles are disputed 77% of the time compared to 20% for full laughs. The authors propose a disagreement-calibrated evaluation protocol using conformally calibrated tolerance bands to address the limitations of single-ground-truth evaluation.
Entities (13)
Relation Signals (6)
Chuckle â hashigherdisagreementratethan â Full Laugh
confidence 98% · far more common for chuckles than full laughs (77% vs. 20%)
Offset â hashigherdisagreementthan â Onset
confidence 98% · disagreement is 1.73 times larger at offsets than onsets
SMILE-Disagreement â derivedfrom â SMILE-Temporal
confidence 95% · Re-annotating the SMILE-Temporal benchmark... we find systematic patterns
SMILE-Disagreement â proposesevaluationmethod â Conformal Calibration
confidence 90% · We propose a disagreement-calibrated evaluation that scores predictions against the full annotator distribution using conformally calibrated tolerance bands
SMILE-Disagreement â comparedto â LEAD
confidence 85% · position it against LEAD [10], Vargas-Quiros [30], and GEBD [26]
SMILE-Disagreement â comparedto â GEBD
confidence 85% · position it against LEAD [10], Vargas-Quiros [30], and GEBD [26]
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Annotators routinely disagree on laughter boundaries and subtle chuckles, yet temporal laughter localization typically evaluates against a single reference annotation. We show that this disagreement is structured rather than random noise. Re-annotating the SMILE-Temporal benchmark (672 videos, 1,683 events) with 3-5 annotators per video (alpha = 0.757), we find systematic patterns: disagreement is 1.73 times larger at offsets than onsets, far more common for chuckles than full laughs (77% vs. 20%), and predictable from event attributes (AUC = 0.831). Evaluating against a single annotator breaks down under this structure: system scores shift by 0.246 F1 depending on the chosen ground truth, correctly ranking systems only 69.7% of the time (vs. 80% against all annotators). We propose a disagreement-calibrated evaluation that scores predictions against the full annotator distribution using conformally calibrated tolerance bands (wider at offsets, 0.727s, than onsets, 0.5s). The per-annotator annotations and analysis code are available at this https URL .
Tags
Links
- Source: https://arxiv.org/abs/2609.06646v1
- Canonical: https://arxiv.org/abs/2609.06646v1
Trouble viewing inline? Open PDF directly â
Full Text
53,925 characters extracted from source content.
Expand or collapse full text
When Does a Laugh Begin? Structured Annotator Disagreement in Temporal Laughter Localization Eyal Hanania, Daniel Arkushin, Naveh Ayal, Jonathan Benvenisti, Amos Bercovich, Elie Zemmour â , and Sahar Froim â WSC Sports Eyal.Hanania@wsc-sports.com Abstract. Annotators routinely disagree on laughter boundaries and subtle chuckles, yet temporal laughter localization typically evaluates against a single reference annotation. We show that this disagreement is structured rather than random noise. Re-annotating the SMILE-Temporal benchmark (672 videos, 1,683 events) with 3-5 annotators per video (al- pha = 0.757), we find systematic patterns: disagreement is 1.73x larger at offsets than onsets, far more common for chuckles than full laughs (77% vs. 20%), and predictable from event attributes (AUC 0.831). Evaluat- ing against a single annotator breaks down under this structure: system scores shift by 0.246 F1 depending on the chosen ground truth, cor- rectly ranking systems only 69.7% of the time (vs. 80% against all an- notators). We propose a disagreement-calibrated evaluation that scores predictions against the full annotator distribution using conformally cali- brated tolerance bands (wider at offsets, +/-0.727 s, than onsets, +/-0.5 s). The per-annotator annotations and analysis code are available at https://github.com/WSCSports/SMILE-Disagreement. Keywords: Laughter localization· Annotator disagreement· Multi- annotator datasets· Evaluation protocols· Affective behavior analysis 1 Introduction When does a laugh begin - and, harder, when does it end? Ask several trained an- notators to mark the same clip and they return different answers (Fig. 1): aligned on the sharp attack of a clear laugh, scattered across its gradual decay, and split on whether a faint chuckle is a laughter event at all. This is not an exotic corner case. Laughter is among the most frequent and most studied affective signals in real-world video, and localizing it in time, not merely detecting its presence, un- derpins applications from affective indexing to highlight generation. Yet laughter localization benchmarks, like temporal affect benchmarks generally [11, 16, 31], evaluate systems against a single reference annotation (henceforth single-GT), treating every difference between annotators as noise around one true boundary. â E. Zemmour and S. Froim contributed equally as co-senior authors. arXiv:2609.06646v1 [cs.CV] 6 Sep 2026 2E. Hanania et al. 78910 time in clip (s) P3 P2 P1 share Clear laugh: agreed start, disputed end 35363738 time in clip (s) P4 P3 P2 P1 share Trailing laugh: split on intensity & end 121314 time in clip (s) P3 P2 P1 share no laughter Faint chuckle: one person marks nothing Each row is one person marking laughter in the same clip; the strip below is the per-frame share (the soft target) full laughchuckle Fig. 1: Laughter has no single boundary. Three clips from the corpus, each with a disputed extent: a clear laugh whose end annotators place differently; a trailing laugh split on both intensity (full vs. chuckle) and offset; and a faint chuckle one annotator does not mark at all. Evaluating against a single ground truth hides this structured disagreement. This paper argues, with data, that these differences are not noise. Our the- sis: single-ground-truth evaluation misrepresents temporal laughter localization. Laughter boundaries are not point events; annotators disagree about them in structured, measurable ways; and that disagreement should be modeled rather than averaged away. To test it, we re-annotate the SMILE-Temporal bench- mark [9] with multiple independent annotators per video and release SMILE- Disagreement, to our knowledge the first publicly available, per-event, multi- annotator temporal-boundary corpus for laughter in affect video. We state this claim narrowly and position it against LEAD [10], Vargas-Quiros [30], and GEBD [26] in Sec. 2. The disagreement we find is not random. Annotators agree well overall (Krip- pendorffâs α = 0.757 [13] at 100 ms, stable across bin sizes and annotator counts), so the signal is trustworthy; but where they disagree is systematic. Two regu- larities stand out. First, offsets are 1.3â1.7Ă noisier than onsets: a laugh has a sharp attack but a gradual decay, so its end is genuinely harder to pin down than its start. Second, low-intensity âchucklesâ are contested far more often than full laughs (77% vs. 20% of events), an intensity effect that survives every control we apply: modality, duration, annotator count, choice of event matcher, and the (imperfect) reliability of the intensity label itself. This is not only about whether a laugh is marked at all: even when all annotators agree a chuckle occurred, its boundaries are about twice as spread out as a full laughâs at the same duration. Disagreement is, moreover, predictable from event attributes alone (Sec. 4.3). In short, the ambiguity is a property of the events, not annotator caprice, and averaging it away is not a harmless convenience but a modeling error. The practical casualty is evaluation. Because annotators genuinely differ, any single one is an arbitrary reference: the same systemâs frame-F1 moves by 0.246 on average depending on which annotator is chosen as ground truth, and - in controlled experiments with synthetic systems of known quality - single-GT When Does a Laugh Begin?3 ranks the genuinely better system above the worse one only 69.7% of the time, versusâŒ80% when scoring against all annotators (Sec. 5). We therefore propose and validate a disagreement-calibrated protocol: score a prediction against the whole annotator distribution, with a per-boundary tolerance band conformally calibrated to how much humans themselves disagree, wider at offsets than at onsets. In summary, our contributions are: 1. Dataset. SMILE-Disagreement: 672 videos with 3â5 independent annota- tors each, carrying 4,554 annotator marks (1,683 distinct laughter events after cross-annotator matching) with intensity and source labels, released as per-annotator distributions with all analysis code (Sec. 3). 2. Findings. A perspectivist analysis that ports the learning-with-disagreement lens from categorical NLP to temporal boundaries and shows the disagree- ment is structured: intensity-linked, offset-concentrated, robust to five arti- fact controls, and predictable from event attributes (Secs. 2, 4). 3. Evaluation. A disagreement-calibrated protocol (a soft distributional target plus a conformal human-spread band; specified in Sec. 5.2), validated to remove the reference-annotator arbitrariness that makes single-GT rankings unreliable, and exercised on a real public laughter localizer - whose errors concentrate exactly where human disagreement does (Sec. 5). 2 Related Work Human label variation and perspectivism. A decade of work has challenged the assumption that annotation converges on one correct label: disagreement re- flects genuine properties of the data rather than annotator error [2], and is repro- ducible and inherent rather than resolvable by more careful annotation [19]. The resulting perspectivist, learning-with-disagreement program [23, 29] keeps the variation and evaluates against it - via the soft label distributions of the LeWiDi benchmark [15], label-distribution learning [6], and, in vision, full human label distributions over CIFAR-10 [20]. All of this concerns categorical labels, where disagreement is a rate over classes. We port the perspectivist question to temporal boundaries in affect video, where a label is an interval and disagreement acquires geometry: it can concentrate at onsets or offsets, scale with event intensity, and widen or narrow per event. Temporal boundary ambiguity in video. Vision has long had evidence that temporal boundaries are soft: recognition accuracy depends on which tem- poral extent of an action one assumes [25], conscientious workers disagree sub- stantially on temporal extents [27] and interaction bounds [17], and diagnostic analyses identify imprecise boundary localization as a principal error source in temporal action detectors [1]. Detection [26] responds by collecting five anno- tators per video and evaluating against each separately. These efforts, however, treat the variation as noise to be managed - averaged away, regularized against, or (in GEBD [26]) attached to instantaneous cut points. Affective events extend over time, with an onset and an offset whose uncertainties differ - the systematic 4E. Hanania et al. asymmetry we quantify in Sec. 4.2 has no analogue for point events - and we release the variation itself, per event and per annotator, as the object of study. Multi-annotator affect, laughter, and sound-event corpora. Closest to our resource, the LEAD dataset [10] releases strong labels from 20 anno- tators for general sound events and analyzes how onset/offset labels vary, but stops short of a disagreement-aware evaluation. For laughter specifically, Vargas- Quiros et al. [30] compare annotation modalities with two annotators per seg- ment and aggregate their labels rather than releasing the per-event boundaries of â„3 distinct annotators. Affect corpora (RECOLA [24], SEWA [12]) ship mul- tiple raters as dimensional traces rather than discrete event boundaries, and the ABAW competition series [11] evaluates affect recognition against a single reference annotation per video. The classic laughter resources (MAHNOB [21], AMI [3]) and the detection and segmentation line built on them [4,7,18,22,28] all treat a laughâs extent as given - as do the SMILE benchmark [9] we re-annotate, the multimodal localizer trained against it [8], and its successor SMILE-Next [14], which scales laughter understanding to language models but neither localizes boundaries in time nor models annotator disagreement. Our corpus occupies the intersection these leave open (Table 2): per-event onset/offset distributions from â„3 distinct annotators, on laughter/affect video, publicly released. 3 The SMILE-Disagreement Dataset We re-annotate the videos of SMILE-Temporal [9], a temporal laughter-localization benchmark, with multiple independent annotators, each marking laughter onset and offset boundaries. To our knowledge it is the first publicly released, per- event, multi-annotator temporal-boundary corpus for laughter in affect video; Table 2 positions it against the closest prior resources. 3.1 Annotation protocol Six trained annotators labeled the videos independently, without seeing one an- otherâs labels. For each perceived laughter event an annotator marked its onset and offset and assigned three attributes: intensity (full laughter vs. chuckle); source, the dominant cue the laughter was perceived through (acoustic, visual, or both); and speaker (audience, 4,298 of the 4,554 benchmark marks, vs. in- dividual speaker, 256). Source records the dominant channel, not an exclusive one: a visual cue may still contribute where a laugh is labelled acoustic, possi- bly without the annotator consciously registering it, so the acoustic-dominant split (Sec. 3.3) reflects the primary perceived channel, not the absence of visual information. The benchmark is every video with â„3 distinct annotators (672 videos; per- video counts in Table 1), the accepted minimum for reliability analysis; the count is uneven, and Sec. 4 confirms every finding holds when videos are subsampled to three. It carries 4,554 annotator marks (one region drawn by one annotator; the full campaign collected 7,950, the rest repeats or below-threshold videos), which cross-annotator matching merges into 1,683 distinct events. When Does a Laugh Begin?5 Distinct annotators per video 3 4 5 # videos449 211 12 Table 1: Per-video annotator counts in the 672-video benchmark (â„3 distinct anno- tators; same-annotator repeats removed). 3.2 Annotator consistency A per-annotator profile (anonymized A1-A6, matching the released data) is given in the supplementary. No annotator is an outlier: frame-level agreement with the majority of the others spans 0.6-0.78 F1, productivity and existence rates are comparable, and onset/offset style biases stay within a few tenths of a second. The disagreement structure analyzed in Sec. 4 is therefore a property of the videos themselves, not of one annotatorâs labeling style. 3.3 Composition Of the 672 benchmark videos, 578 contain at least one annotated laughter event; these 578 total 4.49 hours of video (median 26.0 s, IQR 15.9â35.6 s), and after matching holds 1,683 distinct events - 2.5 per video and 6.24 per annotated minute - with laughter marked by at least one annotator covering 21.1% of recorded time (14.7% by majority vote). Median event duration is 1.83 s for full laughter vs. 1.16 s for chuckles, one reason low-intensity events are easier to miss outright (Sec. 4.3). Across all 7,950 campaign marks, 75.4% are full laughter and 24.6% chuckles. By cue, annotators judged 94.3% acoustic-dominant, 4.8% au- diovisual, and 0.9% visual-only; laughterâs most salient cue is usually its sound, so an acoustic-leaning split is expected, and this is the dominant channel rather than the only one, on what remains full audiovisual video (Sec. 3.1). The audio- visual minority is where the difficulty concentrates: visually-involved events are disagreed about more than acoustic-only ones (69.0% vs. 37.0%; 95% Wilson CI [62.0, 75.3], n=184 matched events), so the corpus still exercises the subtle, cross- modal cases an audiovisual system must resolve. Finally, 94.4% of marks are audience laughter: the source videos are edited, broadcast-style content, where laughter comes predominantly from an audience rather than a single on-screen speaker. This is a domain property, not a confound behind our findings: the off- set asymmetry replicates within both the audience and the individual-speaker subsets (Sec. 4.2). We publicly release SMILE-Disagreement - the per-annotator onset/offset annotations and metadata as a JSON file, for non-commercial research pur- poses, together with the full analysis code, at https://github.com/WSCSports/ SMILE-Disagreement. The underlying video content is part of the original SMILE dataset [9] and is not redistributed; our release adds only the multi-annotator annotation layer. 6E. Hanania et al. ResourceModality# Annotators/item Per-event onset/offset Released as distributionDomain Ours (SMILE-Disagreement) AV (acoustic-dom.)3-5 distinctâlaughter / affect video LEAD [10]audio20âgeneral sound events Vargas-Quiros [30]AV2/seg. (48 tot.)ââ (not released)laughter GEBD [26]video5cut-points (instant.)partialgeneric boundaries RECOLA / SEWA [12,24]AVmultiplecontinuous tracesâdimensional affect Table 2: Positioning against the closest multi-annotator resources. Our scoped novelty is the combination of per-event onset/offset distributions, public release,â„3 distinct an- notators, and the laughter/affect-video domain: no prior row occupies all four columns at once. 3.4 Relation to prior multi-annotator resources Multi-annotator boundary annotation exists in adjacent settings, and we claim no priority over it; Table 2 positions our corpus against the closest work. Each prior resource lacks one of our four defining properties: LEAD [10] annotates general sound events, not affect video; Vargas-Quiros et al. [30] use two an- notators per segment (48 total), aggregated and unreleased; GEBD [26] marks instantaneous cut points rather than onset/offset intervals; and RECOLA [24] / SEWA [12] provide continuous dimensional traces rather than discrete per- event boundaries. Our contribution is the intersection they leave open: pub- licly released, per-event onset/offset distributions from â„3 distinct annotators on laughter/affect video. 4 Analysis and Findings We treat the multi-annotator labels not as noise to be averaged into a single ground truth but as a distribution to be characterized; all statistics are com- puted on the 672-video benchmark of Sec. 3 (a summary table is in the supple- mentary). We first establish that the agreement signal is stable - the claim that disagreement is structured is only meaningful if it is - and then show that where annotators disagree is governed by two factors: where a boundary lies (onset vs. offset) and how intense the laughter is. 4.1 Annotators agree well, and robustly Discretizing each video into 100 ms frames and treating laughter presence as a binary label, frame-level inter-annotator reliability is Krippendorffâs α = 0.757. 1 Because frames within a video are strongly autocorrelated, we report a video- level (clustered) bootstrap rather than treating frames as independent units: α â [0.742, 0.769] (95% CI, B=1000, videos resampled with replacement). The value is well above the chance floor and not an artifact of our choices - it holds across frame sizes from 40 to 500 ms (αâ [0.756, 0.758]) and is unchanged when 1 Computed with the standard krippendorff library (nominal level, missing data for annotators who did not label a given video); an independent re-implementation agrees to four decimals. When Does a Laugh Begin?7 every video is subsampled to exactly three annotators (α = 0.757), so the uneven annotator count is not inflating it. Agreement on the existence of laughter is likewise high but not total: annotators agree on whether laughter is present in 77.2% of videos, leaving 22.8% contested. The signal is real; the structure in the disagreement, below, is therefore interpretable. 4.2 Offsets are fuzzier than onsets For events that multiple annotators localized in common, we measure the across- annotator standard deviation of each boundary. Offsets are markedly noisier (Fig. 2a): the mean offset spread (0.253 s) is about 1.73Ă the onset spread (0.146 s; one-sided Wilcoxon signed-rank p = 4.3 Ă 10 â19 , video-level boot- strap 95% CI on the greedy-matcher ratio [1.54, 1.97]). It is not an artifact of the corpusâs dominant communal audience laughter, whose decay many people could stop marking at different times: it holds within audience-labeled events (1.75Ă, bootstrap 95% CI [1.56, 1.97], n=1, 328) and, indicatively, within the small individual-speaker group (1.54Ă, CI [1.06, 2.27], n=60) - single-person laughs show the same offset fuzziness. The cause is acoustic. Measuring the RMS energy envelope around each co-marked event (half-max crossing times in a padded window, so they do not depend on the human boundaries), the central peak rises faster than it falls: the decay is 1.48Ă the attack (median; video-level bootstrap 95% CI [1.4, 1.5]; decay exceeds attack in 68% of 1, 278 events). A sharp attack but gradual decay makes the endpoint acoustically the less de- termined boundary - the direction of the annotation asymmetry. The effect is aggregate, not an event-wise doseâresponse (within events, decay length does not predict offset spread, Spearman Ï=â 0.07), so we read the offset fuzziness as a general perceptual consequence of the gradual decay. Any evaluation should therefore grant more tolerance at offsets than at onsets, a prescription we make precise in Sec. 5. 4.3 Intensity drives disagreement The second and larger factor is intensity, and it acts on both whether an event is marked and where its boundaries fall. Existence. Calling an event disagreed when not all of a videoâs annotators marked it (greedy-overlap matcher; an eventâs intensity is the majority vote of its co-markersâ labels), a chuckle is disagreed about 77% of the time versus only 20% for full laughter (Fig. 2b; video-level clustered bootstrap 95% CI on the gap [52, 61] points) - equivalently, 68% of all disagreed events are chuckles, against just 14% of unanimous ones. Low-intensity laughter is where humans disagree; unambiguous full laughter is where they agree. Five measurement artifacts could mimic this, and each fails to explain it (rates also in the supplementary summary table). Modality: restricting to acoustic- only events leaves the gap essentially unchanged. Duration: chuckles are shorter and easier to miss outright, yet the gap holds at 44â55 points within every du- ration quartile, and a logistic model with log-duration, position, and modality 8E. Hanania et al. When it starts When it ends 0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35 How far apart annotators are (seconds) 0.15s 0.25s n = 1,388 matched events Wilcoxon p = 4.32e-19 Annotators disagree 1.7Ă as much over where a laugh ends (a) All eventsAcoustic-only (confound control) 0 20 40 60 80 100 Laughs that annotators disagreed about (%) 20% 77% 19% 74% Chuckles are disagreed about 3.8Ă as often as full laughs also within acoustic-only events n=1,079n=604n=1,017n=482 full laugh chuckle (b) Fig. 2: The two headline regularities. (a) Annotators are about 1.73Ă as far apart on where a laugh ends as on where it begins: mean across-annotator standard devia- tion over commonly-marked events, 95% CIs (one-sided Wilcoxon signed-rank; greedy matcher - ratio 1.3â1.7Ă across matchers). (b) Disagreement rate by intensity (bars: rate with 95% Wilson CI, n inside each bar): a chuckle is disagreed about 77% of the time versus 20% for a full laugh; the right group repeats within acoustic-only events, so the gap is driven by intensity, not modality. as covariates still assigns a chuckle 8.59Ă the odds of disagreement. Annotator count: subsampling every video to exactly three annotators leaves the rates es- sentially unchanged (75.6% vs. 17.5%). Event matcher: IoU and midpoint match- ing give 77â82% vs. 20â25%. The intensity label itself: co-markers disagree on it for 37.2% of jointly-marked events (α = 0.344), consistent with intensity being graded rather than binary - but such shared label noise attenuates rather than inflates the gap, and restricting to intensity-unanimous events in fact widens it (90.2% vs. 23.8%). Nor is the structure specific to the dominant audience laughter: within audience-labeled events the rates are 74.4% vs. 19.9%, and the small individual-speaker group points the same way (88% vs. 41%; n=100/17, indicative only). Disagreement is thus substantially predictable from event attributes: the same logistic model reaches AUC 0.831 under 5-fold cross-validation grouped by video (0.757 without the intensity feature). Because intensity is itself an annotation rather than an acoustic measurement, we bound the resulting circularity against the signal. On all 672 benchmark videos (the audio is fully recoverable), a label- free acoustic loudness proxy (mean RMS, duration-independent) already sepa- rates full laughs from chuckles (AUC 0.727; full laughs are about twice as loud), and substituting it for the perceptual label lifts the disagreement model from 0.757 (no intensity signal at all) to 0.771 - below the 0.831 the perceptual label reaches, but clearly above the no-intensity baseline. The intensity effect there- fore has a genuine acoustic basis and is not merely an artifact of the poolâs own labels. When Does a Laugh Begin?9 Boundaries. Intensity structures not only existence but boundary spread, even on events multiple annotators agree exist. On co-marked events (greedy matcher, same spread statistic as Sec. 4.2), chuckles are fuzzier than full laughs (on- set 0.175 vs. 0.135 s, MannâWhitney p = 6.8Ă 10 â3 ; offset 0.275 vs. 0.244 s, p = 1.3Ă 10 â2 ; n = 395 vs. 993), and the pooled means understate this for the same reason as above - shorter events have tighter absolute spreads - so within duration quartiles the chuckle offset spread is 1.6â2.3Ă the full-laugh spread, in all four. The comparison is conservative: co-marked chuckles are by construction the least ambiguous chuckles. The offsetâonset asymmetry, in turn, holds within both intensities (1.58Ă for chuckles, 1.81Ă for full laughs), so the two axes are separable: intensity makes an event both likelier to be missed entirely and, when found, roughly twice as uncertain at the boundaries, while the offset concentra- tion is a property of laughterâs temporal envelope regardless of intensity. 5 Disagreement-Calibrated Evaluation The findings of Sec. 4 imply that a single annotator is a poor stand-in for âground truth.â This section makes the damage concrete, then proposes and validates an evaluation that scores a system against the distribution of human labels. The protocol is built from the human annotations alone - no trained model is required to construct or validate it (Sec. 5.4 then applies it to one). 5.1 A single ground truth is arbitrary We first quantify the arbitrariness. Treat each annotator as a âsystemâ and score it, at the frame level (100 ms), against each of the other annotators, one at a time. Two facts emerge. First, humanâhuman agreement averages a frame-F1 of about 0.681: this, not 1.0, is the realistic ceiling for any system, yet a single- ground-truth (single-GT) benchmark implicitly treats perfection as attainable. Second, and more damaging, a systemâs measured score depends heavily on which annotator is chosen as reference: for each (video, system) pair we take the maxâ min range of its frame-F1 across the available reference choices; the mean range is 0.246 F1 (Fig. 3a). Under single-GT, the reported number is partly an artifact of an arbitrary choice. 5.2 Scoring against the human distribution We instead score a prediction against the annotator distribution, with two in- gredients. (i) A soft target. Let y a i â 0, 1 be annotator aâs laughter label on 100 ms frame i, and let s i = 1 k P k a=1 y a i be the per-frame fraction of the k annotators marking laughter. A binary prediction Ëy is scored by the fuzzy Jaccard overlap (soft-IoU) sIoU(Ëy,s) = P i min(Ëy i ,s i ) P i max(Ëy i ,s i ) ,(1) 10E. Hanania et al. so a prediction earns graded credit for partially-agreed events instead of being right or wrong against one person. When the denominator vanishes (a video with no marked laughter and no prediction) we score sIoU=1: correct silence. (i) A conformally calibrated tolerance band. For each boundary type b â onset, offset, we hold out one annotator at a time and compute the noncon- formity score r = |tâ m|, where t is the held-out annotatorâs boundary and m is the median of the matched boundaries of the remaining annotators (events matched by temporal overlap; only events with â„2 co-markers, annotators who also marked the event, contribute). We pool the râs of type b across all events and held-out annotators (n=3, 815 per type) and take the conformal quantile q b = r (â(n+1)(1âα)â) , the â(n + 1)(1â α)â-th smallest score: the half-width cal- ibrated to cover a 1âα fraction of held-out human boundaries. At α=0.1 the band is±0.5 s at onsets but±0.727 s at offsets (1.45Ă wider): the metric is most lenient where humans themselves are least certain, inheriting by construction the boundary asymmetry of Sec. 4.2 rather than ignoring it. These global widths are driven by a heavy tail of ambiguous events; an adap- tive variant, which rescales each r by the local spread of the co-markersâ bound- aries, attains the same coverage with narrower per-event widths, widening only on the events where the annotators themselves disagree. Its specification, and two further calibration details, are in the supplementary. A predictionâs band score is the fraction of its matched boundaries falling within ±q b of the corresponding human median. Predicted events that overlap no human event contribute no boundaries to it: hallucinations and misses are punished by the soft-IoU term instead - a hallucinated eventâs frames enter the denominator of Eq. (1) at full weight, a missed eventâs frames enter it at their human weight s i , and neither adds to the numerator. Sec. 5.3 probes this division of labor for exploitability. Two caveats delimit what the calibration promises. First, coverage is marginal (it holds on average over the pooled residuals, not per video or annotator), and pooling treats residuals as exchangeable although they cluster within videos, within annotators, and within events (each event yields several leave-one-out residuals). The annotator axis we verify empirically: calibrating on five annota- tors and testing on the sixth yields 0.86â0.92 (mean 0.895, target 0.90) across all six annotators and both boundary types - approximately calibrated, with mild under-coverage at the extremes. That check pools events, so the other clustering axes remain assumptions, not verified properties. For a systemâs predictions the band is a human-calibrated tolerance, not a coverage guarantee: it asks whether a prediction deviates from the consensus by more than one human deviates from the rest. A modelâs residuals need not be exchangeable with human ones, and Sec. 5.4 shows they are not. Composition. The full protocol score is the unweighted mean of the frame- level soft-IoU (1) and the event-level band score; both are computed per video and macro-averaged (unweighted mean over videos), so long videos do not dom- inate. Sec. 5.3 validates each ingredient separately, the composition, and its insensitivity to the mixing weight. The supplementary illustrates the full proto- When Does a Laugh Begin?11 col on a real event; Sec. 2 situates it as, to our knowledge, the first temporal- boundary evaluation with human-calibrated tolerance (GEBD [26], the nearest precedent, scores instantaneous cut points against each annotator separately under a convention-chosen tolerance). 5.3 Validation: does it rank systems better? A good evaluation should rank a better system above a worse one, so we test exactly that. We inject synthetic âsystemsâ of known quality, the consensus lo- calization perturbed with increasing boundary jitter (four tiers, so tier order is ground-truth quality order), and ask, for each adjacent pair and over five ran- dom seeds, whether a scoring rule ranks the better system higher (Fig. 3). Three rules are compared, differing in how they use the annotators: frame-F1 against a single arbitrary annotator; frame-F1 averaged over all annotators; and soft-IoU against the annotator distribution. Single-GT gets the ranking right only 69.7% of the time; both distribution- aware rules reach âŒ80% (multi-annotator average 80.4%, soft-IoU 80.0%). The soft-IoUâvsâsingle-GT gap carries a paired-bootstrap 95% CI of 9â11 points; that bootstrap resamples pooled ranking trials independently, ignoring within- video correlation, so the true interval can only be wider. The gain decomposes cleanly: essentially all of it comes from evaluating against all annotators rather than an arbitrary one (the choice single-GT benchmarks silently make), while the soft-IoU rule matches, but does not exceed, the averaged ruleâs reliability on these synthetic systems. What the distributional form adds is therefore not ranking power: the reference-annotator degree of freedom is already removed by any rule that uses all annotators, so the formâs own contribution is graded credit for partially-agreed events and the calibrated tolerance band of Sec. 5.2, whose role the next paragraph tests. The decomposition is also robust to how the synthetic systems are generated: perturbing a random annotatorâs labels instead of the consensus gives 67.9% / 77.6% / 77.4%. The tolerance band plays a deliberately different role. Scored alone it declares 67% of adjacent-tier pairs tied: both systems sit within the human-calibrated tol- erance, and an evaluation aligned with human ambiguity should indeed refuse to rank differences smaller than human disagreement. When it does decide, it decides correctly 85% of the time; and the full protocol (soft-IoU + band) pre- serves ranking reliability (79.7%; flat at 79.5â80.0% across mixing weights of 0.25â0.75, so the unweighted mean is a convention, not a tuned parameter). Nor is the composition gamed by the obvious boundary-only play: a degenerate âsys- temâ predicting one well-placed 0.2 s sliver per video scores 0.159 under the full protocol versus 0.657 for even the worst jittered tier - its soft-IoU collapses, and even its band score is only 0.248. The same test also bounds how many annotators a temporal benchmark needs. When the soft target is restricted to k subsampled annotators on theâ„4- annotator videos (curve in the supplementary), ranking reliability climbs from 70.6% (k=1, the single-GT regime on this subset) through 75.6% (k=2) and 12E. Hanania et al. 0.00.51.0 frame-F1 vs one reference 0 200 400 600 800 score pairs mean 0.68 (human ceiling) (a) the same humans, any score Single GT All ann. (avg F1) Distrib. (soft-IoU) 0 20 40 60 80 100 better system ranked higher (%) 70% 80% 80% (b) ranking reliability 0.050.300.50 injected error (s) 0.55 0.60 0.65 0.70 0.75 0.80 score frame-F1 soft-IoU (c) both metrics track quality Fig. 3: Single-GT evaluation and its repair. (a) The cause: scoring one annotator- as-system against each single reference spreads scores across the entire [0, 1] range - the reported number depends on who grades. (b) The damage: single-GT ranks the better synthetic system above a one-tier-worse one far less reliably than either rule that uses all annotators (bars: mean; whiskers: minâmax over five seeds). (c) The validity check: both metrics decrease monotonically with injected boundary error; only ranking reliability differs. 77.4% (k=3) toward the all-annotator 78.4% on the same subset: most of the benefit comes from the second and third annotator. 5.4 On a real localizer Synthetic tiers validate a metric; a real system is what the protocol is for. No laughter-specific audiovisual localizer is publicly released: UnAV-100 local- izers [5] carry a laughter class among their hundred categories but ship no laughter-specialized checkpoint, and this corpus is precisely the resource such a model would need. We therefore run the strongest public trained laughter lo- calizer available, the audio-only segmentation model of Omine et al. [18], with its released checkpoint and inference settings, over the 578 benchmark videos with at least one annotated laugh, yielding 1,075 predicted events. (On the 94 additional videos where no annotator marked laughter it predicts only 5 events, so it almost never hallucinates.) An audio-only system is a reasonable first entry for this corpus, whose reported cue is 94.3% acoustic (Sec. 3); its errors can be read against the annotatorsâ own cue labels. The effects the protocol was built for all appear on the real system (Table 3). The swing is real: the reference-annotator choice moves the modelâs per-video frame-F1 by even more than the swing measured with annotators-as-systems, though a system operating farther below the ceiling has more room to swing, so we read the direction, not the magnitude. The band does metric work, most clearly at offsets. Under the global band the model falls short of the 0.90 human construction target at both boundary types; under the adaptive band its onset coverage sits within the human annotator-out range (itself measured under the global band, so we read the adaptive comparison as indicative only), When Does a Laugh Begin?13 Table 3: The protocol on the real localizer (n=916 matched events). The Human column is the matching reference per row: the frame-F1 ceiling and the annotator- as-system swing (row 1); the band-coverage target (0.90 by construction; the em- pirical annotator-out range 0.86â0.92 is measured under the global band); and tier ranking on synthetic tiers. Coverage is the within-band fraction per boundary type (onset/offset); the scalar band score pools both. Tier-ranking values are single- GT / multi / calibrated / full. MetricModelHuman Single-GT frame-F1 (swing/vid.)0.503 (0.308)0.681 (0.246) Multi-annotator frame-F10.501â Soft-IoU / band score0.335 / 0.604â Coverage global, ons./offs.0.829 / 0.7720.90 adaptive, ons./offs.0.865 / 0.8250.86â0.92 Miss rate, contested / unanim.59% / 31%â chuckle / full58% / 33%â Tier rank, SG/mul/cal/full (%) 57.9/62.2/63.4/62.6 69.7/80.4/80.0/79.7 and the offset shortfall persists. The 0.90 is a property of the human pool, not a guarantee the model inherits (Sec. 5.2); the bandâs job is to expose, boundary by boundary, where the model exceeds human deviation and where it does not. Human disagreement predicts machine error: the model misses existence- contested events at roughly twice the rate of unanimous ones (gap 28 points, video-clustered bootstrap 95% CI 22â33), and chuckles at nearly twice the rate of full laughs (gap 25 points, CI 19â30) - the correlated, structured failure mode Sec. 4.3 predicts, landing on the events where the single-GT convention is least defensible. The two contrasts largely overlap (most contested events are chuckles, and for an audio-only model both align with the faintest acoustics), so we count them as one confirmation, not two; but it is a direct one - the events a real system gets wrong are disproportionately the events humans themselves contest. Scope. The tier systems of Sec. 5.3 are synthetic (the standard device for val- idating a metric under a controlled quality ordering), and Sec. 5.4 confirms the headline effects on a trained system, including the prediction that real localizers fail on the events where human disagreement concentrates. The recommendation stands regardless: report temporal-localization performance against all annota- tors, ideally as a distribution with conformally calibrated tolerance, and never against a single one. Although we instantiate it on laughter, the protocol acts only on temporal boundaries and their inter-annotator spread, so it transfers to any affective behavior annotated as multi-rater event segments. The near- est fit is temporal-phase segmentation of facial expressions and action units: onsetâapexâoffset annotation is event-bounded in the same shape this corpus releases, and porting needs only multi-rater phase boundaries, with the toler- ances re-calibrated per task (the offset asymmetry we measure is a property of laughter, not of the method). Continuous dimensional traces (valenceâarousal in RECOLA [24], SEWA [12], and the ABAW series [11]) have no event bound- 14E. Hanania et al. aries for a band to calibrate on; there the perspectivist argument still applies, but through amplitude- and lag-aware mechanisms rather than a temporal tol- erance. 6 Limitations Our analysis is based on SMILE-Temporal, whose laughter is predominantly acoustically cued, preventing a meaningful study of audioâvisual disagreement. Furthermore, the conformal tolerance bands are calibrated to a single annotation pool and protocol, so their widths should not be interpreted as universal. Finally, evaluation on real systems is limited to one public audio-only laughter localizer. Extending the analysis to additional datasets, annotation pools, and multimodal localizers remains future work. 7 Conclusion Single-ground-truth evaluation misrepresents temporal laughter localization. Laugh- ter boundaries are not point events; annotators disagree about them in struc- tured, measurable ways; and that disagreement should be modeled rather than averaged away. The dataset is the enabler; the findings and the evaluation pro- tocol are the contribution. For practitioners the prescription fits in one sentence: evaluate temporal affect against all annotators - ideally the distribution, with tol- erance conformally calibrated to human spread - never against a single reference; expect about 1.45Ă the tolerance at offsets as at onsets; and treat low-intensity events as intrinsically ambiguous rather than mislabeled. We release SMILE- Disagreement as per-annotator distributions with all analysis code - including the evaluation harness a public laughter localizer already runs under (Sec. 5.4), where its misses concentrate on the events humans contest - and see two natural next steps: broadening the real-system panel across modalities, and collecting visually-driven laughter to extend the analysis beyond the acoustic channel. References 1. Alwassel, H., Caba Heilbron, F., Escorcia, V., Ghanem, B.: Diagnosing error in temporal action detectors. In: European Conference on Computer Vision (ECCV). p. 264â280 (2018). https://doi.org/10.1007/978-3-030-01219-9_16 2. Aroyo, L., Welty, C.: Truth is a lie: Crowd truth and the seven myths of human annotation. AI Magazine 36(1), 15â24 (2015). https://doi.org/10.1609/aimag. v36i1.2564 3. Carletta, J., Ashby, S., Bourban, S., Flynn, M., Guillemot, M., Hain, T., et al.: The AMI meeting corpus: A pre-announcement. In: International Workshop on Machine Learning for Multimodal Interaction (MLMI). p. 28â39 (2005). https: //doi.org/10.1007/11677482_3 When Does a Laugh Begin?15 4. Escalera, S., Puertas, E., Radeva, P., Pujol, O.: Multi-modal laughter recognition in video conversations. In: 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). p. 110â115 (2009). https: //doi.org/10.1109/CVPR.2009.5204268 5. Geng, T., Wang, T., Duan, J., Cong, R., Zheng, F.: Dense-localizing audio-visual events in untrimmed videos: A large-scale benchmark and baseline. In: CVPR. p. 22942â22951 (2023). https://doi.org/10.1109/CVPR52729.2023.02197 6. Geng, X.: Label distribution learning. IEEE Transactions on Knowledge and Data Engineering 28(7), 1734â1748 (2016). https://doi.org/10.1109/TKDE.2016. 2545658 7. Gillick, J., Deng, W., Ryokai, K., Bamman, D.: Robust laughter detection in noisy environments. In: Interspeech. p. 2481â2485 (2021). https://doi.org/10.21437/ INTERSPEECH.2021-353 8. Hanania, E., Kirsch, N., Arkushin, D., Benvenisti, J., Bercovich, A., Zemmour, E., Froim, S.: MTLLFM: Multimodal-temporal laughter localization: UR-FUNNY- Temporal and SMILE-Temporal benchmarks with an adaptive multimodal fu- sion model. arXiv preprint arXiv:2605.25409 (2026). https://doi.org/10.48550/ arXiv.2605.25409 9. Hyun, L., Sung-Bin, K., Han, S., Yu, Y., Oh, T.H.: Smile: Multimodal dataset for understanding laughter in video with language models. In: Findings of the Association for Computational Linguistics: NAACL 2024. p. 1149â1167 (2024) 10. Koga, N., Bando, Y., Imoto, K.: LEAD dataset: How can labels for sound event detection vary depending on annotators? In: Asia Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) (2024). https://doi.org/10.1109/APSIPAASC63619.2025.10848643 11. Kollias, D.: Abaw: Valence-arousal estimation, expression recognition, action unit detection & multi-task learning challenges. In: Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition Workshops. p. 2328â2336 (2022). https://doi.org/10.1109/CVPRW56347.2022.00259 12. Kossaifi, J., Walecki, R., Panagakis, Y., Shen, J., Schmitt, M., Ringeval, F., et al.: SEWA db: A rich database for audio-visual emotion and sentiment research in the wild. IEEE Transactions on Pattern Analysis and Machine Intelligence 43(3), 1022â1040 (2021). https://doi.org/10.1109/TPAMI.2019.2944808 13. Krippendorff, K.: Computing krippendorffâs alpha-reliability (2011), departmental Papers (ASC), University of Pennsylvania 14. Lee, J.M., Kim, S.B., Chang, J., Lee, H., Oh, T.H.: SMILE-next: Teaching large language models to detect, classify, and reason about laughter. arXiv preprint arXiv:2605.28084 (2026) 15. Leonardelli, E., Abercrombie, G., Almanea, D., Basile, V., Fornaciari, T., Plank, B., Rieser, V., Uma, A., Poesio, M.: SemEval-2023 task 11: Learning with disagree- ments (LeWiDi). In: Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval). p. 2304â2318 (2023). https://doi.org/10.18653/v1/ 2023.semeval-1.314 16. Li, J., Xie, J., Zhu, L., Qian, L., Tang, S., Zhang, W., Shi, H., Zhang, S., Wei, L., Tian, Q., Zhuang, Y.: Dilated context integrated network with cross-modal consensus for temporal emotion localization in videos. In: Proceedings of the 30th ACM International Conference on Multimedia (ACM M) (2022). https://doi. org/10.1145/3503161.3547886 17. Moltisanti, D., Wray, M., Mayol-Cuevas, W., Damen, D.: Trespassing the bound- aries: Labeling temporal bounds for object interactions in egocentric video. In: 16E. Hanania et al. IEEE International Conference on Computer Vision (ICCV) (2017). https://doi. org/10.1109/ICCV.2017.314 18. Omine, T., Akita, K., Tsuruno, R.: Robust laughter segmentation with automatic diverse data synthesis. In: Interspeech. p. 4748â4752 (2024). https://doi.org/ 10.21437/Interspeech.2024-1644 19. Pavlick, E., Kwiatkowski, T.: Inherent disagreements in human textual inferences. Transactions of the Association for Computational Linguistics 7, 677â694 (2019). https://doi.org/10.1162/tacl_a_00293 20. Peterson, J.C., Battleday, R.M., Griffiths, T.L., Russakovsky, O.: Human uncer- tainty makes classification more robust. In: IEEE/CVF International Conference on Computer Vision (ICCV). p. 9616â9625 (2019). https://doi.org/10.1109/ ICCV.2019.00971 21. Petridis, S., MartĂnez, B., Pantic, M.: The MAHNOB laughter database. Image and Vision Computing 31(2), 186â202 (2013). https://doi.org/10.1016/j.imavis. 2012.08.014 22. Petridis, S., Pantic, M.: Fusion of audio and visual cues for laughter detection. In: Proceedings of the 7th ACM International Conference on Image and Video Retrieval. p. 329â337 (2008). https://doi.org/10.1145/1386352.1386396 23. Plank, B.: The âproblemâ of human label variation: On ground truth in data, modeling and evaluation. In: Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP). p. 10671â10682 (2022). https://doi.org/10.18653/v1/2022.emnlp-main.731 24. Ringeval, F., Sonderegger, A., Sauer, J., Lalanne, D.: Introducing the RECOLA multimodal corpus of remote collaborative and affective interactions. In: IEEE International Conference and Workshops on Automatic Face and Gesture Recog- nition (FG) (2013). https://doi.org/10.1109/FG.2013.6553805 25. Satkin, S., Hebert, M.: Modeling the temporal extent of actions. In: European Conference on Computer Vision (ECCV). p. 536â548 (2010). https://doi.org/ 10.1007/978-3-642-15549-9_39 26. Shou, M.Z., Lei, S.W., Wang, W., Ghadiyaram, D., Feiszli, M.: Generic event boundary detection: A benchmark for event segmentation. In: IEEE/CVF Inter- national Conference on Computer Vision (ICCV). p. 8055â8064 (2021). https: //doi.org/10.1109/ICCV48922.2021.00797 27. Sigurdsson, G.A., Russakovsky, O., Farhadi, A., Laptev, I., Gupta, A.: Much ado about time: Exhaustive annotation of temporal data. In: AAAI Conference on Human Computation and Crowdsourcing (HCOMP). p. 219â228 (2016) 28. Truong, K.P., van Leeuwen, D.A.: Automatic discrimination between laughter and speech. Speech Communication 49(2), 144â158 (2007). https://doi.org/ 10.1016/j.specom.2007.01.001 29. Uma, A., Fornaciari, T., Hovy, D., Paun, S., Plank, B., Poesio, M.: Learning from disagreement: A survey. Journal of Artificial Intelligence Research 72, 1385â1470 (2021). https://doi.org/10.1613/jair.1.12752 30. Vargas Quiros, J., Cabrera Quiros, L., Oertel, C., Hung, H.: Impact of annotation modality on label quality and model performance in the automatic assessment of laughter in-the-wild. IEEE Transactions on Affective Computing 15(2), 519â534 (2024). https://doi.org/10.1109/TAFFC.2023.3269003 31. Zhang, Z., Wang, L., Yang, J.: Weakly supervised video emotion detection and prediction via cross-modal temporal erasing network. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). p. 18888â18897 (2023) When Does a Laugh Begin?17 Supplementary Material Qualitative gallery Figure 4 shows nine real clips spanning the disagreement spectrum the paper ana- lyzes, in the same visual language as Fig. 1 of the main text: clear full laughs draw near-identical intervals from every annotator; boundary-disagreement cases show the same event with widely varying extents (typically at the offset); existence- contested cases - overwhelmingly chuckles - split annotators on whether a laugh- ter event is present at all. 9.509.7510.0010.2510.5010.7511.0011.25 Unanimous full laughs 345644.044.545.045.546.0 2.02.53.03.54.04.55.05.5 Boundary disagreement 6.57.07.58.08.59.09.510.057585960 33.033.534.034.535.0 Existence contested 272829303132373839404142 The disagreement spectrum: one row per annotator, seconds on the x-axis (blue = full laugh, orange = chuckle, dashed = marked nothing) Fig. 4: The disagreement spectrum on nine real clips (deterministically selected; one row per annotator; blue = full laugh, orange = chuckle, dashed line = annotator marked no laughter). Top: unanimous full laughs. Middle: agreed events, disagreed bound- aries. Bottom: existence-contested clips - note they are dominated by chuckles, the paperâs headline finding. 18E. Hanania et al. Annotator Videos Ev./vid. Exist.% Chk.% Aud.% Onset bias (s) Offset bias (s) F1 vs. peers A15782.37902196-0.07+0.040.68 A23202.52843092-0.00-0.040.71 A33482.56893592+0.06+0.020.71 A42122.08823292+0.09-0.030.60 A5631.92752695+0.10+0.230.78 A64212.19883096-0.02-0.050.70 Table 4: Per-annotator profile on videos with at least one annotated laugh (anonymized; existence rates are thus conditional on laughter being present). Signed biases are vs. the median of the other annotators on events marked in common; F1 is frame-level (100 ms) against the majority of the others; Chk.%/Aud.% are the chuckle and audience shares of the annotatorâs events (content composition is comparable across annotators). No annotator is an outlier: the findings are a property of the videos, not of one annotatorâs style. Per-annotator consistency profile Table 4 profiles each annotator (anonymized A1âA6, matching the released data), supplementing Sec. 3.2 of the main paper: no annotator is an outlier, so the disagreement structure the paper analyzes is a property of the videos, not of one annotatorâs labeling style. Annotator-budget curve Figure 5 plots the ranking-reliability-vs-annotator-count result quoted in Sec. 5.3 of the main paper. When Does a Laugh Begin?19 123all (45) annotators used to build the soft target 65 70 75 80 Better system ranked higher (%) multi-GT rule, full benchmark (80%) 71% 76% 77% 78% = single-GT regime The 2nd and 3rd annotator buy most of the reliability Fig. 5: Ranking reliability of the calibrated metric vs. the number of annotators used to build the soft target (â„4-annotator videos; whiskers: minâmax over seeds). The second and third annotator buy most of the reliability - practical guidance for annotation- budget planning. 20E. Hanania et al. Summary of findings Table 5 collects the main-paper findings (Sec. 4) and their robustness checks in one place; every value also appears in the Sec. 4 prose. MeasureValueTest Frame agreement α (100 ms)0.757 [0.742, 0.769]- across 40â500 ms bins0.756â0.758stable subsampled to 3 annotators0.757stable Existence unanimous / contested77.2% / 22.8%- Onset s.d. (matched events)0.146 s p=4.3Ă 10 â19 Offset s.d. (matched events)0.253 s Offset/onset spread ratio1.3â1.7Ărange over 3 matchers within audience / indiv.-speaker events 1.75Ă / 1.54Ă(confound ctrl.) acoustic decay/attack (RMS envelope)1.48ĂCI [1.4, 1.5] Disagreement rate: chuckle vs. full77% vs. 20%gap [52, 61] pts holds within acoustic-only eventsyes(confound ctrl.) holds within duration quartilesgap 44â55 pts(confound ctrl.) at the 3-annotator floor75.6% vs. 17.5%stable across event matchers77â82% vs. 20â25%stable intensity-unanimous events only90.2% vs. 23.8%stable Boundary spread: chuckle vs. full (offset) 0.275 vs. 0.244 s 1.6â2.3Ă per duration quartile Table 5: Summary of findings on the 672-video benchmark. Brackets are video-level clustered bootstrap 95% CIs (B=1000); the onset/offset test is a one-sided Wilcoxon signed-rank test over matched events. Protocol illustration Figure 6 illustrates the disagreement-calibrated protocol (Sec. 5.2 of the main paper) on a single real event. Conformal band: specification details Three details supplement the bandâs definition in Sec. 5.2 of the main paper. Calibration/test k-mismatch. Calibration holds out one annotator (median over kâ1) while a prediction is tested against the median of all k - the more stable of the two, so the applied band errs conservative. Per-co-marker-count strata. The pooled quantile is not an artifact of the thinnest events. Splitting the leave-one-annotator-out calibration pool by co- marker count: with k=2 co-markers (n=2, 162 residuals per boundary type) the 90%-coverage half-widths are ±0.535 s (onset) and ±0.712 s (offset); with kâ„3 (n=1, 653) they are ±0.467 s and ±0.734 s - close to the pooled ±0.5/±0.727 s, so the pooled quantile is not driven by the thin-calibration events. When Does a Laugh Begin?21 10.010.511.011.512.012.513.013.5 time in clip (seconds) Person 1 Person 2 Person 3 Person 4 system onset band ±0.50s offset band ±0.73s prediction inside both bands: wrong by less than humans disagree among themselves Scoring against the human distribution, on one real event soft target: share marking laughter soft-IoU = 0.62 (this window) Fig. 6: The protocol on a real event from the corpus. Four annotators mark the same laugh (top rows; blue = full laugh, orange = chuckle); their per-frame fraction is the soft target (shaded strip); the conformally calibrated tolerance bands (teal, drawn to scale) are exactly as wide as human disagreement -±0.5 s at the onset but±0.727 s at the offset. The example prediction (black outline; its soft-IoU restricted to the displayed window, matching the on-figure label) deviates from the median annotator yet stays within both bands: it is wrong only by less than humans disagree among themselves. Adaptive variant. The adaptive band normalizes each nonconformity score r by the local spread Ï e of the co-markersâ boundaries (floored at 0.05 s), giving a per-event half-width Ìq b Ï e with the same 90% coverage and a median width of ±0.368 s at onsets and ±0.563 s at offsets (vs. the global ±0.5/±0.727 s) - tight where annotators agree, wide only where they do not.