Paper deep dive
When Does Exercise-Specific Joint Selection Help? An Audit of Evaluation and Control Design
Haotian Chen, Jingkun Yu, Yuning Zhang, Bowen Ye
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%
Last extracted: 10/3/2026, 2:21:33 AM
Summary
This paper audits the claim that exercise-specific joint selection improves skeleton-based correctness classification. Using 1,057 repetitions from ten subjects in the REHAB24-6 dataset, the authors demonstrate that reported performance gains are highly sensitive to evaluation metrics (pooled vs. within-person AUROC), control designs (dimension-matched vs. structure-matched random subsets), and classifier choice. While some models like RBF-SVM show positive within-person gains, many apparent benefits disappear when structurally appropriate controls are used, suggesting that joint-selection claims require rigorous estimands and controls rather than establishing a general clinical benefit.
Entities (8)
Relation Signals (5)
REHAB24-6 → contains → Skeleton Sequence
confidence 95% · REHAB24-6 contains 30-FPS motion-capture skeletons and exercise-correctness labels
Joint Selection → isconfoundedby → Evaluation Design
confidence 92% · This exploratory audit shows why joint-selection claims require explicit estimands and structurally appropriate controls
RBF-SVM → demonstrates → Positive Gain
confidence 90% · RBF-SVM retains a positive within-person gain
Logistic Regression → demonstrates → Negative Gain
confidence 90% · logistic regression ... have negative point gains under that estimand
Joint Selection → affects → AUROC
confidence 85% · The manual-subset kNN gain changes from 0.055 for pooled out-of-fold AUROC to 0.020 for equal-weight within-person AUROC
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Exercise-specific joint selection can improve skeleton-based correctness classification, but what does that gain establish? We audit 1,057 repetitions from ten REHAB24-6 subjects, separating evaluation aggregation, subset structure, and temporal representation. The manual-subset kNN gain changes from 0.055 for pooled out-of-fold AUROC to 0.020 for equal-weight within-person AUROC; both paired intervals include zero. Among 1,000 dimension-matched random maps, 14 match or exceed the manual pooled result, versus 145 when bilateral structure and trunk inclusion are also matched. RBF-SVM retains a positive within-person gain, whereas logistic regression and a random-convolution comparator have negative point gains under that estimand. Sequence-order and paired-seed controls further qualify the interpretation. This exploratory audit shows why joint-selection claims require explicit estimands and structurally appropriate controls; it does not establish a new algorithm or clinical benefit.
Tags
Links
- Source: https://arxiv.org/abs/2610.01188v1
- Canonical: https://arxiv.org/abs/2610.01188v1
Trouble viewing inline? Open PDF directly →
Full Text
18,065 characters extracted from source content.
Expand or collapse full text
WHEN DOES EXERCISE-SPECIFIC JOINT SELECTION HELP? AN AUDIT OF EVALUATION AND CONTROL DESIGN Haotian Chen 1,† , Jingkun Yu 3,† , Yuning Zhang 1 , Bowen Ye 2,∗ 1 School of Cyber Science and Technology, University of Science and Technology of China 2 School of Education, Shanghai Jiao Tong University 3 SWJTU-Leeds Joint School, Southwest Jiaotong University † Equal contribution; ∗ corresponding author ABSTRACT Exercise-specific joint selection can improve skeleton-based correctness classification, but what does that gain establish? We audit 1,057 repetitions from ten REHAB24-6 subjects, separating evaluation aggregation, subset structure, and tem- poral representation. The manual-subset kNN gain changes from 0.055 for pooled out-of-fold AUROC to 0.020 for equal- weight within-person AUROC; both paired intervals include zero. Among 1,000 dimension-matched random maps, 14 match or exceed the manual pooled result, versus 145 when bilateral structure and trunk inclusion are also matched. RBF- SVM retains a positive within-person gain, whereas logistic regression and a random-convolution comparator have neg- ative point gains under that estimand. Sequence-order and paired-seed controls further qualify the interpretation. This exploratory audit shows why joint-selection claims require explicit estimands and structurally appropriate controls; it does not establish a new algorithm or clinical benefit. Index Terms— skeleton sequence, exercise correctness, feature selection, subject-disjoint evaluation, AUROC 1. INTRODUCTION Selecting exercise-relevant joints is an intuitive way to sim- plify motion assessment. Yet a higher score need not isolate the value of exercise knowledge: feature scale, subset size, bilateral structure, classifier choice, and the definition of the evaluation metric can all affect the comparison. Small subject cohorts make these distinctions particularly important. Recent action-quality assessment systems exploit tempo- ral structure: MCoRe aligns stages for relative-score regres- sion on FineDiving [1], whose procedure annotations support detailed action analysis [2]. Rehabilitation assessment differs in input and label semantics. ExerciseLLM uses exercise- specific geometric measurements [4], and Rehab-Pile stan- dardizes skeleton-based assessment across datasets and deep architectures [5]. Thus neither geometric features nor task- specific joint subsets are themselves new. We do not compare published scores across incompatible tasks or splits. Our question is narrower: which parts of an apparent joint-subset benefit survive changes to evaluation and con- trol design? We contribute an exploratory audit combining (i) pooled and within-person ranking estimands with matched- row and pair-weighting checks, (i) dimension- and structure- matched random subsets, and (i) temporal, classifier, and seed controls under one subject-disjoint protocol. The study evaluates offline skeleton representations, not the interactive prototype that originally motivated it. 2. DATA AND AUDIT PROTOCOL 2.1. Cohort and separation of subjects REHAB24-6 contains 30-FPS motion-capture skeletons and exercise-correctness labels from ten healthy adults [3]. Ex- cluding 15 repetitions flagged as erroneous MoCap leaves 1,057 repetitions (558 correct and 499 incorrect). The six exercises are arm abduction, arm VW, table push-ups, leg ab- duction, lunge, and squats, with 178, 208, 96, 210, 174, and 191 repetitions. We use the first three coordinates of 26 joints and interpret segment endpoints as one-based and inclusive after boundary checks. Ten outer folds each reserve one person for test, the next cyclic person for validation, and eight for training. A person’s repetitions never cross these partitions. Scaling and models use training data only; model settings maximize validation macro AUROC over evaluable exercises, with the first grid entry breaking ties. No test label selects a setting. This sepa- ration does not remove research-selection bias: the cohort and earlier results were already examined before the additional audit protocol was frozen on September 23, 2026. 2.2. Representations and matched controls Twelve unsigned shoulder-girdle, elbow, hip, knee, ankle, and trunk angles are divided by π and resampled linearly to 40 frames. Per exercise and fold, training frames determine each angle’s mean and standard deviation (floor 10 −3 ). The all-angle condition uses 12 features; a fixed manual map arXiv:2610.01188v1 [cs.AI] 1 Oct 2026 a Skeleton inputb Representation controlsc Model coveraged AUROC estimands REHAB24-6 10 adults, 6 tasks 1,057 repetitions 26 joints, 30 FPS 12 angles / 40 frames All / manual All 12 Manual 4–8 Matched references Dimension-matched1,000 Structure-matched1,000 Train-selected subset Temporal controls OrderedSortedShuffledStats All / manual Ref.Time kNN● LR●–● RBF-SVM●– RF●– RandConv●– OOF scores Pooled Ranks across people Within-person Ranks within person Equal person weights Bootstrap × 5,000 10 subject folds 8 train Scale + fit 1 validation Choose settings 1 test Score only Fixed OOF, 95% CI Fig. 1. Subject-disjoint representation audit. Dots in (c) mark evaluated combinations; “Ref.” includes dimension-matched, structure-matched, and training-selected subsets. In (d), marker shapes identify people and filled/open markers indicate cor- rect/incorrect repetitions; ranks are schematic. Both AUROC estimands average exercises equally. Paired subject bootstrap resamples fixed OOF predictions, not model fits. uses four to eight, based on exercise descriptions rather than clinician error annotations. The maps use zero-based angle indices: 1, 3, 10, 11 for Ex1; 0, 1, 2, 3, 10, 11 for Ex2; 2, 3, 4, 5, 6, 7, 10, 11 for Ex3; 4, 5, 6, 7, 10, 11 for Ex4; and 4, 5, 6, 7, 8, 9, 10, 11 for Ex5 and Ex6. Indices 0–9 form five bilateral pairs; 10–11 are trunk features. Using seed 20260917, we sample 1,000 dimension- matched maps, drawing each exercise’s subset uniformly without replacement within a map. A further 1,000 maps (seed 20260923) preserve the manual number of complete bilateral pairs and unilateral selections, and include both trunk features. We sample uniformly from each exercise’s eligible subsets: 40 choices for Ex1 and 10 for each other exercise. These controls match structure and dimension, not clinical relevance. We report counts b at least as good as manual, empirical fractions b/1000, and smoothed fractions (b + 1)/1001, none a permutation-test p-value. A histori- cal training-only inner-LOSO single-angle selector provides another equal-dimension kNN control. Sequence controls retain 40 values per angle but either sort each angle independently or independently shuffle whole frames within each repetition. The former also removes cross- angle temporal correspondence; the latter preserves instanta- neous joint relationships. A lower-dimensional control uses mean, standard deviation, minimum, and maximum per an- gle. Transformations and the label-independent shuffle seed are fixed before the new runs. None is a pure intervention on a single clinical error mechanism. 2.3. Conventional model families Fixed-k kNN groups training examples by exercise and sub- type. With k = 3, its score is mean distance to the near- est incorrect examples minus mean distance to the near- est correct examples; distance is mean absolute difference over the selected representation. Higher scores mean cor- rect execution. Separate per-exercise LR, RBF-SVM, and 200-tree RF models use class-balanced training. Validation grids are C ∈ .01,.1, 1, 10 for LR; C ∈ .1, 1, 10 and γ ∈scale,.002,.01 for SVM; and depth∈unlimited, 8 with leaf size ∈ 1, 3 for RF. All/manual conditions re- ceive equal search budgets. We additionally repeat RF with three paired seeds, reusing the selected validation fit without reseeding. A local ROCKET-style comparator [6] applies 1,000 fixed random multivariate convolutions of lengths 7, 9, or 11, with random dilation, channel subsets, bias, and padding. Maximum and proportion-positive outputs yield 2,000 features; training-only feature standardization pre- cedes a class-balanced ridge classifier. Validation selects α ∈.01,.1, 1, 10, 100. All/manual inputs share the kernel- generation seed and budget. This is a local comparator, not an official ROCKET reproduction or a proposed method. 3. WHAT DOES AUROC AGGREGATE? Let A es be AUROC within exercise e and test person s, using half credit for score ties. We preserve the historical pooled metric A P , which first concatenates out-of-fold scores within each exercise, and contrast it with A W = 1 6 6 X e=1 1 |I e | X s∈I e A es ,(1) where I e contains people with both labels for exercise e. Both metrics average the six exercise-level values equally. Pooled ranking includes comparisons across people and fitted mod- els; within-person ranking does not require their score origins to align. Neither is universally the correct deployment metric. Of 60 possible person–exercise cells, 51 have both labels, seven have no repetitions, and two contain only incorrect rep- etitions. Thus A W uses 1,026 repetitions, with 9, 9, 8, 9, 7, and 9 evaluable people across exercises. We separately re- compute pooled AUROC on these same eligible rows, and compute a within-person variant weighted by each person’s positive–negative pair count. This separates row eligibility, cross-person comparisons, and weighting rather than attribut- ing every metric change to score comparability. Intervals use 5,000 paired subject-cluster bootstrap draws (seed 20260923) from fixed predictions, keeping all exercises from a sampled person together. Within-person means pre- serve bootstrap multiplicities. Intervals describe subject com- position conditional on fitted models; they omit refitting and research-selection uncertainty. One draw lacks a class in an exercise and is excluded; 4,999 complete draws remain. A 98.75% interval is an additional four-family sensitivity check, not a confirmatory guarantee over all analyses. An interval containing zero does not establish no effect or equivalence. 4. RESULTS 4.1. Gains depend on both learner and estimand Table 1 preserves the original pooled comparison alongside the within-person result. The kNN gain changes from .055 to .020, LR from .027 to −.007, SVM from .057 to .045, and RF from .044 to .011. The local random-convolution model reaches .710/.728 pooled and .764/.743 within-person AUROC for all/manual inputs; neither of its gain intervals excludes zero. These point reversals concern estimands, not proof that either input is better. The positive SVM evidence must also be retained: its within-person gain is .045 with 95% interval [.013,.080] and four-contrast sensitivity interval [.004,.090]. This supports a model-specific exploratory association, not a general joint- selection rule. The training-selected kNN subset attains .644 pooled and .661 within-person AUROC; no test-selected map replaces it. Table 2 shows why row matching matters. LR’s gain is already reduced to .005 after restricting pooled evaluation to eligible rows. For kNN, the matched-row pooled gain is .062, versus .021 under within-person pair weighting and .020 with 0 50 100 (a) PooledManual 0.651 1.4% 14.5% 0.450.500.550.600.650.700.75 Macro AUROC threshold 0 50 100 (b) Within-personManual 0.677 5.1% 18.8% Random maps at or above threshold (%) Dimension-matchedStructure-matched Fig. 2. Random-map references for kNN. Curves show the percentage of 1,000 maps at or above each AUROC threshold (six-exercise macro average). Dotted lines mark manual; an- notations are descriptive percentages, not p-values. equal people. Across exercises, 88.6–90.4% of the origi- nal pooled positive–negative pairs compare different people. Adding a fixed label-independent offset of ten times the fold index to every score leaves all within-person AUROCs un- changed but changes pooled AUROCs. This is an invariance diagnostic, not a proposed calibration or evidence of leakage. 4.2. The random reference set changes the conclusion For pooled kNN, the dimension-matched reference mean is .587, with 14/1,000 maps at least as good as manual (.651). Structure matching raises the mean to .622 and that count to 145/1,000 (Fig. 2). For within-person AUROC, the cor- responding counts are 51 and 188. The pooled empirical tail fraction changes from 1.4% to 14.5% (smoothed: .015 to .146) when the comparator preserves bilateral and trunk structure. Both distributions are legitimate descriptive refer- ences but test different notions of an arbitrary subset. Neither establishes the causal value of exercise-specific knowledge. 4.3. Temporal and seed controls constrain mechanisms Table 3 reports every scheduled sequence control. Sorted LR manual input reaches .760 within-person AUROC, versus .718 for ordered input; summary kNN manual input reaches .671 versus .677 for ordered input. Thus performance can- not simply be credited to detailed phase order. Shuffling re- duces the ordered-input point score in all four kNN/LR input conditions, but it also changes alignment and representation statistics; it does not identify a specific clinical timing error. Table 1. All/manual AUROC and paired gains. Pooled uses all 1,057 repetitions; within-person uses 51 evaluable cells (1,026 repetitions). The original RF row is retained; paired-seed RF is a separate sensitivity analysis. All intervals use 4,999 valid draws from 5,000 fixed-prediction subject resamples. Pooled out-of-fold AUROCWithin-person AUROC ModelAll / manual Gain Paired 95% CI All / manualGainPaired 95% CI kNN (k = 3).596 / .651.055[-.005,.113].657 / .677.020[-.068,.102] LR.664 / .691.027[-.058,.097].725 / .718-.007[-.061,.043] RBF-SVM.627 / .684.057[.005,.121].689 / .734.045[.013,.080] RF (historical seed).597 / .641.044[-.003,.087].677 / .688.011[-.029,.046] Random convolution.710 / .728.017[-.018,.053].764 / .743-.021[-.049,.012] Table 2. Decomposing manual-minus-all gains. The final three rows use the same eligible observations; only compar- isons and weights differ. Gain estimandkNNLR SVMRF Pooled: all rows.055 .027.057 .044 Pooled: eligible rows.062 .005.038 .039 Within: pair weighted .021 -.018.053 .013 Within: equal people.020 -.007.045 .011 Table 3. Within-person AUROC (all/manual) for sequence controls. Sorted and shuffled inputs match ordered dimen- sionality; summaries use four statistics per angle. RepresentationkNNLR Ordered (40 frames).657 / .677 .725 / .718 Sorted per angle (40).621 / .657 .711 / .760 Shuffled frames (40).592 / .626 .613 / .583 Mean / SD / min / max .629 / .671 .703 / .728 Across three paired RF seeds, within-person manual gains range from .018 to .031, and every paired 95% interval in- cludes zero. Median within-cell score rank correlations across seeds are .908 (all) and .932 (manual). Variation in selected hyperparameters alone would not prove prediction instability; these score-level checks address a different question. Earlier geometric diagnostics remain relevant: standard- ized Ex2 manual kNN is .586 pooled versus .426 before stan- dardization, and .618 with post hoc torso-anchored shoulder proxies. Reflecting all 65 source skeleton records about one global axis leaves the unsigned angles exactly unchanged. This demonstrates lost directional information, but without direction- or phase-specific error annotations it cannot explain individual failures. 5. LIMITATIONS AND CONCLUSION This is a repeatedly explored, ten-person healthy cohort with simulated incorrect exercise, known exercise identity, and MoCap-derived 3D inputs. It does not test RGB pose es- timation, patients, feedback efficacy, or mobile runtime. Single-class cells limit within-person coverage. Bootstrap intervals are conditional on saved fits; random seeds are not new participants.External data acquisition remained un- successful, so no independent replication is claimed. A local random-convolution baseline does not substitute for a verified reproduction of the nearest rehabilitation models. All comparisons retain finite scores for 1,057 paired repetition IDs. Saved model metrics and all 2,001 man- ual/random map score sets are independently recomputed with scalar/sklearn checks; convolution outputs are tested against a slow reference. These establish implementation consistency, not scientific validity. The supported conclusion is that joint-selection benefits are conditional on the learner, ranking estimand, and reference-set design. Independent co- horts with a prospectively chosen deployment estimand are needed before asserting a general benefit. 6. REFERENCES [1] Q. An, M. Qi, and H. Ma, “Multi-stage contrastive re- gression for action quality assessment,” in Proc. ICASSP, 2024, p. 4110–4114. [2] J. Xu et al., “FineDiving: A fine-grained dataset for procedure-aware action quality assessment,” in Proc. CVPR, 2022. [3] A. Cernek,J. Sedmidubsky,and P. Budikova, “REHAB24-6:Physical therapy dataset for analyz- ing pose estimation methods,” in Proc. SISAP, 2024. [4] J. Tang, A. Abedi, T. J. F. Colella, and S. S. Khan, “Reha- bilitation exercise quality assessment and feedback gen- eration using large language models with prompt engi- neering,” arXiv:2505.18412, 2025. [5] A. Ismail-Fawaz, M. Devanne, S. Berretti, J. Weber, and G. Forestier, “A standardized benchmark for skeleton- based rehabilitation assessment using deep learning,” in Proc. IEEE FG, 2026. [6] A. Dempster, F. Petitjean, and G. I. Webb, “ROCKET: Exceptionally fast and accurate time series classification using random convolutional kernels,” Data Mining and Knowledge Discovery, vol. 34, p. 1454–1495, 2020. 7. COMPLIANCE WITH ETHICAL STANDARDS This study retrospectively analyzes the public REHAB24-6 skeleton sequences and labels [3]. The dataset is released un- der C BY-NC 4.0, and its documentation reports participant consent to public release. We recruited no participants and collected no new human-subject data. No institutional ethics approval or waiver was obtained for this secondary analysis.