Paper deep dive
FedPref: Federated Preference Learning for Structured Radiology Report Extraction
Flint Xiaofeng Fan, Cheston Tan, Yew-Soon Ong, Roger Wattenhofer
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%
Last extracted: 8/19/2026, 3:59:16 AM
Summary
The paper introduces FedPref, a federated preference learning framework for extracting structured findings and locations from radiology reports. It addresses the challenge of uneven data distribution across institutions by using frozen public language models as teachers to propose alternative JSON extractions, which local sites rank based on their annotations. Sites then train compact Qwen3-8B adapters using Direct Preference Optimization (DPO) and share only LoRA updates via Federated Averaging. FedPref improves client-mean and worst-site F1 scores compared to isolated training, particularly benefiting smaller sites, while preserving data privacy by never sharing raw reports or annotations.
Entities (12)
Relation Signals (11)
FedPref → finetunes → Qwen3-8b
confidence 95% · sites collaboratively train compact Qwen3-8B adapters
FedPref → solves → Radiology Report Extraction
confidence 95% · FedPref: Federated Preference Learning for Structured Radiology Report Extraction
FedPref → uses → Federated Averaging
confidence 95% · pair-count-weighted Federated Averaging (FedAvg) combines into a shared adapter
FedPref → uses → LoRA
confidence 95% · sites collaboratively train compact Qwen3-8B adapters while sharing only model updates... only LoRA states cross the boundary
FedPref → uses → DPO
confidence 95% · fixed-reference Direct Preference Optimization (DPO) [10]... DPO then refines which valid structures the model favors
FedPref → evaluatedon → MIMIC-CXR
confidence 90% · centralized benchmark built from MIMIC-CXR and Chest ImaGenome
FedPref → evaluatedon → Chest ImaGenome
confidence 90% · centralized benchmark built from MIMIC-CXR and Chest ImaGenome
Qwen2.5-14B-Instruct → isteacherfor → FedPref
confidence 90% · Four frozen teachers... Qwen2.5-14B-Instruct... each generate one completion
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Radiology reports describe findings and locations in free text, but downstream search and analysis require these relations in a fixed schema. Learning this extraction requires labels that are unevenly distributed across institutions: smaller hospitals have less local evidence, and pooling data may be infeasible. We introduce FedPref: frozen public language models propose alternative JSON extractions, local annotations rank them, and sites collaboratively train compact Qwen3-8B adapters while sharing only model updates. A heterogeneous teacher pool provides cross-model contrast when repeated single-model samples collapse. On development data from six simulated hospitals with unequal data volume and disease prevalence, FedPref improves client-mean F1 by 2.49 points and worst-site F1 by 9.10 points compared with training each site in isolation, with the largest gains at the sites holding the least data. Central training on the pooled preference-pair union is 2.66 points higher on client-mean F1. On a locked, 400-report manually validated gold test set, FedPref reaches 68.68 F1 and pooled training 71.67, preserving that same ordering. FedPref thus lets institutions with unequal, unpooled data benefit from collaboration without ever sharing reports or annotations.
Tags
Links
- Source: https://arxiv.org/abs/2608.16971v1
- Canonical: https://arxiv.org/abs/2608.16971v1
Trouble viewing inline? Open PDF directly →
Full Text
35,772 characters extracted from source content.
Expand or collapse full text
**footnotetext: Preprint. Accepted at the 2nd Emerging LLM/LMM Applications in Medical Imaging (ELAMI 2026), held in conjunction with MICCAI 2026. To appear in the Springer proceedings. FedPref: Federated Preference Learning for Structured Radiology Report Extraction Flint Xiaofeng Fan Affiliation: ETH Zurich, Zurich, Switzerland E-mail xiafan,wattenhofer@ethz.ch Affiliation: Agency for Science, Technology and Research (A*STAR), Singapore, Singapore E-mail fanx,cheston-tan,ong_yew_soon@a-star.edu.sg Cheston Tan Affiliation: Agency for Science, Technology and Research (A*STAR), Singapore, Singapore E-mail fanx,cheston-tan,ong_yew_soon@a-star.edu.sg Yew-Soon Ong Affiliation: Agency for Science, Technology and Research (A*STAR), Singapore, Singapore E-mail fanx,cheston-tan,ong_yew_soon@a-star.edu.sg Roger Wattenhofer Affiliation: ETH Zurich, Zurich, Switzerland E-mail xiafan,wattenhofer@ethz.ch Abstract Radiology reports describe findings and locations in free text, but downstream search and analysis require these relations in a fixed schema. Learning this extraction requires labels that are unevenly distributed across institutions: smaller hospitals have less local evidence, and pooling data may be infeasible. We introduce FedPref: frozen public language models propose alternative JSON extractions, local annotations rank them, and sites collaboratively train compact Qwen3-8B adapters while sharing only model updates. A heterogeneous teacher pool provides cross-model contrast when repeated single-model samples collapse. On development data from six simulated hospitals with unequal data volume and disease prevalence, FedPref improves client-mean F1 by 2.49 points and worst-site F1 by 9.10 points compared with training each site in isolation, with the largest gains at the sites holding the least data. Central training on the pooled preference-pair union is 2.66 points higher on client-mean F1. On a locked, 400-report manually validated gold test set, FedPref reaches 68.68 F1 and pooled training 71.67, preserving that same ordering. FedPref thus lets institutions with unequal, unpooled data benefit from collaboration without ever sharing reports or annotations. Keywords: Federated learning Preference learning Radiology reports Structured extraction 1 Introduction Radiology reports encode two linked facts: which findings are present and where they occur. A report describing a left pleural effusion should yield the disease and its location; assigning the disease to the wrong anatomy remains an extraction error. Structuring these relations makes reports queryable and reusable as supervision. We study nine diseases and their locations in the centralized benchmark built from MIMIC-CXR and Chest ImaGenome [6, 14, 12]. Institutions observe unequal numbers and combinations of cases. Centralized training broadens coverage by pooling data; isolated training preserves institutional boundaries but leaves smaller hospitals with less evidence. Federated learning offers a third path by aggregating locally optimized model states [9, 11, 4, 1]. The schema and annotation rule stay fixed while sites differ in patient volume and disease prevalence. Figure 1: FedPref intuition: pooled versus federated preference learning. At each site, frozen teachers propose alternative structured extractions and the local structured annotation ranks eligible candidates into chosen c+c^+ and rejected c−c^-. (a) Central pools the resulting preference pairs across sites and trains one shared adapter. (b) FedPref keeps reports, annotations, candidates, and preference pairs at their sites; each site performs local preference training, and only LoRA states are exchanged for pair-count-weighted FedAvg to form the global adapter. Local preference training denotes the chosen-response SFT initialization followed by fixed-reference DPO, detailed in Sec. 3. FedPref turns structured annotations into a local preference interface (Fig. 1). For each report, four frozen teachers propose alternative JSON extractions; the site projects them into a common schema and uses its structured annotation to form a chosen response c+c^+ and rejected response c−c^-. Central provides the pooled counterpart by training on the union of retained preference pairs. FedPref instead keeps reports, annotations, candidates, and preference pairs local: sites train adapters locally and exchange only LoRA states, which pair-count-weighted Federated Averaging (FedAvg) combines into a shared adapter. Preference learning first requires rankable candidate contrast. We therefore run a separate feasibility diagnostic, outside the operational training pipeline, that tests repeated self-sampling from the Qwen3-8B target model. These samples often collapse to the same structured output, whereas heterogeneous cross-model candidates provide substantially more contrast (Sec. 5). This motivates the operational four-teacher pool and gives our first question: whether cross-model candidates supply useful local preference supervision. Each arm begins with one chosen-response supervised fine-tuning (SFT) epoch, followed by fixed-reference Direct Preference Optimization (DPO) [10]. Our second question is whether aggregating local preference updates improves client-mean and worst-site performance over isolated training, and how much of the performance attainable under pooled training it retains. We therefore evaluate three matched-exposure arms: Local trains each site in isolation, FedPref aggregates site adapters, and Central trains on the exact preference-pair union. On development data, FedPref improves client-mean and worst-site extraction over Local, with the largest gains at the two smallest sites, while Central measures the observed pooled–federated gap. A locked manually validated gold test checks generalization beyond automatically derived labels. 2 Related Work MIMIC-CXR and Chest ImaGenome provide report-level and anatomy-linked supervision, while Sabour et al. formulate the centralized structured-extraction task used here [6, 14, 12]. Federated learning coordinates models without centralizing raw institutional records, making it attractive for medical data silos [9, 11]. Data locality alone does not guarantee privacy: formal protection requires a threat model and mechanisms such as secure aggregation or differential privacy [3]. FedAvg combines local states, while LoRA reduces the number of trainable parameters communicated [9, 5]. Factorwise LoRA averaging can nevertheless introduce an optimization gap [13], and unequal client utility motivates reporting both average and lower-tail performance [7]. DPO learns directly from response rankings, and federated variants distribute feedback-driven, personalized, or medical-dialogue alignment [10, 2, 8]. FedPref connects these strands by using local disease-location annotations to rank heterogeneous teacher outputs and aggregating only the resulting LoRA states. 3 Method Overview. Site k holds report-annotation pairs k=(xi,yi)S_k=\(x_i,y_i)\. For report xix_i, teachers propose structures and annotation yiy_i designates the best and worst eligible outputs as chosen response ci+c_i^+ and rejected response ci−c_i^-. Supervised fine-tuning (SFT) on chosen responses establishes the schema and fixed reference; DPO then refines which valid structures the model favors. FedPref aggregates LoRA states, Central trains on the pair union, and Local trains independently. Local preference construction. Four frozen teachers, Qwen3-14B, Qwen2.5-14B-Instruct, Llama-3.1-8B-Instruct, and Mistral-7B-Instruct-v0.3, each generate one completion; the Qwen3-8B target is excluded [15]. Teachers receive the same task messages through native chat templates and use deterministic greedy decoding with reasoning disabled. Each completion is projected into the ontology and canonical JSON; failed projections are discarded. Eligible candidates are scored by s(c,y)=0.70F1loc+dis(c,y)+0.25F1dis(c,y)+0.05[c valid].s(c,y)=0.70F_1^loc+dis(c,y)+0.25F_1^dis(c,y)+0.051[c valid]. (1) For an empty target set, we replace F1 by 1/(1+m)1/(1+m) for m predicted positives. The larger weight on Location+Disease F1 reflects the primary task: a disease assigned to the wrong anatomy should not be preferred. Because only successfully projected candidates are eligible, the validity term records schema compliance without changing their ordering. The highest- and lowest-scoring candidates form (ci+,ci−)(c_i^+,c_i^-) when their structures differ, their margin is at least 0.100.10, and ci+c_i^+ has higher F1loc+disF_1^loc+dis. The annotation determines ordering but never enters a teacher prompt or candidate. Fixed-reference preference optimization. Round 0 performs one chosen-response SFT epoch. Central trains on the exact pair union, Local retains one root per site, and pair-weighted aggregation of those same local roots initializes FedPref. In DPO rounds t=1,2,3t=1,2,3, each policy continues from its preceding checkpoint while its round-0 root remains the fixed reference πref _ref. Writing hθ(c,x)=logπθ(c∣x)−logπref(c∣x)h_θ(c,x)= _θ(c x)- _ref(c x), site k minimizes ℒk(θ)=−klogσβ[hθ(c+,x)−hθ(c−,x)].L_k(θ)=-E_D_k σ\β[h_θ(c^+,x)-h_θ(c^-,x)]\. (2) The objective raises the chosen response’s relative likelihood while the fixed SFT reference keeps that change comparable across rounds. Aggregation and matched controls. At round t, every FedPref site receives the preceding global LoRA state ϕ(t−1)φ^(t-1) and trains for one local epoch. With nk=|k|n_k=|D_k|, the server computes ϕ(t)=∑knk∑jnjϕk(t).φ^(t)= _k n_k _jn_j _k^(t). (3) Pair weighting makes each site’s influence proportional to its retained training evidence. Central trains one adapter for three epochs on ∪k _kD_k; Local trains six isolated adapters for three local epochs, matching pair exposure across arms. In FedPref, reports, annotations, candidates, and preference pairs remain at their simulated sites; only LoRA states cross the boundary. 4 Experimental Setup Task and cohorts. We align MIMIC-CXR reports with Chest ImaGenome annotations [6, 14]. Automatically derived labels provide 4,800 training and 600 development reports; the manually validated Chest ImaGenome gold standard provides 100 diagnostic and 400 locked-test reports. The schema contains nine diseases and 18 locations, 15 with positive support. All cohorts are patient-disjoint; gold-standard patients were excluded before constructing the automatically labeled cohorts. Code and reproduction instructions are available at https://github.com/flint-xf-fan/FedPref. Simulated federation. We partition the data into six sites spanning 384 to 1,344 training reports and 297 to 717 retained pairs (Table 2). A capacity-constrained multilabel Dirichlet allocator (α=0.5α=0.5) varies disease prevalence while preserving site sizes. Prevalence divergence exceeds a capacity-matched IID allocation: 8.83 versus 1.89 points for training pairs and 10.14 versus 4.14 for development (both p=0.0005p=0.0005). The federation therefore combines unequal evidence with verified case-mix heterogeneity. Preference data and optimization. The teachers produce 19,200 training and 2,400 development responses, yielding 2,961 training and 376 development pairs. All arms adapt Qwen3-8B with LoRA rank 16 and α=32α=32. SFT uses learning rate 2×10−42× 10^-4; DPO uses 5×10−65× 10^-6, β=0.1β=0.1, BF16, cosine scheduling, and length 2,048. Each of three seeds independently trains the SFT root and all three DPO rounds, with three DPO exposures per pair in every arm. Evaluation. Predictions are canonically projected before scoring; projection success records whether they contain a valid projectable JSON object. The primary endpoint is micro-F1 over positive disease-location labels (Location+Disease F1), which requires both finding and anatomy to be correct. Disease F1, projection success, false positives on reports with no targets, and held-out preference accuracy are secondary. Global F1 pools reports and weights larger sites more; client mean weights sites equally; worst-site F1 records the minimum. Central and FedPref use one shared policy, while home-routed Local sends each site’s reports to its adapter. A paired patient-cluster bootstrap computes mean and worst-site effects over 10,000 draws and averages them across three seeds. Before gold-standard access, FedPref client-mean F1 selects one common round, breaking ties by worst-site F1 and then earlier round. Selected Central and FedPref policies are evaluated once on the locked gold-standard annotations. 5 Results Cross-model candidates provide preference contrast. We measure candidate contrast by oracle headroom, the F1 gain from selecting the best candidate per report over the target baseline, and usable-pair coverage, the fraction of reports satisfying the pair margin. Across 100 automatically labeled reports, repeated Qwen3-8B sampling produced 71 identical parsed sets, 3.05 points of headroom, and 20.0% coverage. A separate five-model diagnostic (four teachers plus the target) on 100 gold-standard reports produced 14.30 points and 76.0%; its oracle exceeded the best individual model by 6.21 points (95% CI, 3.70 to 9.01). The first diagnostic identifies target-model collapse; the second establishes cross-model contrast and motivates excluding the target from the operational four-teacher pool. Federated aggregation improves isolated sites. FedPref improves across communication rounds (Fig. 2). Its client-mean Location+Disease F1 increases from 54.93±0.4754.93± 0.47 to 69.11±0.1769.11± 0.17, while worst-site F1 increases from 53.67±0.1853.67± 0.18 to 68.12±0.2268.12± 0.22. It overtakes Local on client mean at round 2, and round 3 maximizes the prespecified FedPref client-mean selection criterion. At the selected round (Table 1), FedPref reaches 69.04±0.2269.04± 0.22 global, 69.11±0.1769.11± 0.17 client-mean, and 68.12±0.2268.12± 0.22 worst-site F1. Home-routed Local reaches 66.62±0.3266.62± 0.32 and 59.02±1.4559.02± 1.45 on client mean and worst site. FedPref therefore improves the client mean by +2.49+2.49 points (95% CI, +1.24+1.24 to +3.79+3.79) and the worst site by +9.10+9.10 points (95% CI, +1.01+1.01 to +11.87+11.87). Central reaches 71.77±0.6771.77± 0.67, 71.77±0.6571.77± 0.65, and 70.03±1.1570.03± 1.15; Central exceeds FedPref by 2.66 points on client mean. FedPref and Local attain similar held-out preference accuracy, 91.49±0.2791.49± 0.27 and 91.67±0.8591.67± 0.85, despite their different extraction F1. Preference accuracy therefore does not track the extraction gap between the two systems. FedPref achieves 99.89±0.10%99.89± 0.10\% projection success and has a 3.42±0.00%3.42± 0.00\% false-positive rate on reports with no target findings. Table 1: Three-seed development performance at selected round 3. Values are mean ± sample standard deviation; paired effects are reported in text with patient-cluster confidence intervals. System Global F1loc+disF_1^loc+dis Mean F1loc+disF_1^loc+dis Worst F1loc+disF_1^loc+dis Pref. acc. Central 71.77±0.6771.77± 0.67 71.77±0.6571.77± 0.65 70.03±1.1570.03± 1.15 94.95±0.2794.95± 0.27 FedPref 69.04±0.2269.04± 0.22 69.11±0.1769.11± 0.17 68.12±0.2268.12± 0.22 91.49±0.2791.49± 0.27 Local 67.32±0.2867.32± 0.28 66.62±0.3266.62± 0.32 59.02±1.4559.02± 1.45 91.67±0.8591.67± 0.85 Location+Disease F1 (%)Client mean506070750123RoundWorst site506070750123RoundCentralFedPrefLocal Figure 2: Performance across communication rounds under matched preference-pair exposure. Error bars show sample standard deviation over three seeds. FedPref overtakes home-routed Local on client mean at round 2 and retains a nine-point worst-site advantage at the selected third round. Benefits concentrate at smaller sites. FedPref improves sites S1, S3, S4, and S5 (Table 2). The largest gains, +9.46+9.46 and +4.47+4.47 points, occur at S4 and S5, the two smallest training cohorts; S0 and S2 favor Local. In this partition, gains concentrate at sites with less local evidence. Table 2: Per-site support (training reports / retained pairs) and selected-round FedPref–Local effect. Site S0 S1 S2 S3 S4 S5 Support 1,344/717 1,056/591 864/556 672/486 480/297 384/314 ΔF1loc+dis F_1^loc+dis (pt) −1.87-1.87 +0.87+0.87 −1.72-1.72 +3.73+3.73 +9.46+9.46 +4.47+4.47 Locked gold-standard generalization. On the 400-report locked gold test, Central records Location+Disease F1 of 71.67±0.2371.67± 0.23, Disease F1 of 90.12±0.1790.12± 0.17, projection success of 99.58±0.29%99.58± 0.29\%, and a false-positive rate of 7.58±0.00%7.58± 0.00\%. FedPref records 68.68±0.3568.68± 0.35, 86.36±0.1686.36± 0.16, 100.00±0.00%100.00± 0.00\%, and 8.33±0.76%8.33± 0.76\%, respectively. The primary-endpoint comparison yields +2.98+2.98 points for Central minus FedPref (95% patient-cluster CI, +1.53+1.53 to +4.54+4.54). Scores are mean ± sample standard deviation over three seeds, and the interval uses 10,000 patient-cluster resamples conditional on those seeds. Outputs from both policies project successfully for nearly every report and generalize to the gold-standard labels while preserving the Central–FedPref extraction ordering; the home-routed development results above provide the site-level evidence. 6 Discussion and Conclusion FedPref reframes structured clinical annotations as a local preference interface. Each institution uses its existing annotations to rank candidate structures under a shared extraction rule, keeping reports and task-specific judgments under local control. Cross-model candidates supply local supervision that adapter aggregation carries into a shared model. This separation accommodates institutions that share a target schema but differ in data volume, case mix, and governance, and suggests a general strategy for federated adaptation of schema-constrained language models. The boundaries of the present evidence and the requirements for clinical deployment are detailed in Supplementary Sec. A.1. In this controlled federation, aggregating adapters trained from local preference signals improves extraction most at the sites with the least evidence. Central training’s remaining advantage measures the observed pooled–federated gap under matched preference evidence and exposure, and a locked gold test preserves the same ordering. FedPref thus offers a practical middle ground between isolated and pooled training for structured radiology extraction. Disclosure of Interests. The authors have no competing interests to declare. References [1] Z. Dai, F. X. Fan, C. Tan, T. N. Hoang, B. K. H. Low, and P. Jaillet (2024) Federated sequential decision making: bayesian optimization, reinforcement learning, and beyond. In Federated Learning, p. 257–279. Cited by: §1. [2] F. X. Fan, C. Tan, Y. Ong, R. Wattenhofer, and W. Ooi (2025) FedRLHF: a convergence-guaranteed federated framework for privacy-preserving and personalized RLHF. In Proceedings of the 24th International Conference on Autonomous Agents and Multiagent Systems (AAMAS), External Links: 2412.15538 Cited by: §2. [3] F. X. Fan, C. Tan, R. Wattenhofer, and Y. Ong (2025) Position paper: rethinking privacy in rl for sequential decision-making in the age of llms. In 2025 International Joint Conference on Neural Networks (IJCNN), p. 1–8. Cited by: §2. [4] X. Fan, Y. Ma, Z. Dai, W. Jing, C. Tan, and B. K. H. Low (2021) Fault-tolerant federated reinforcement learning with theoretical guarantee. In Advances in Neural Information Processing Systems, p. 1007–1021. Cited by: §1. [5] E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen (2022) LoRA: low-rank adaptation of large language models. In International Conference on Learning Representations, Cited by: §2. [6] A. E. Johnson, T. J. Pollard, S. J. Berkowitz, N. R. Greenbaum, M. P. Lungren, C. Deng, R. G. Mark, and S. Horng (2019) MIMIC-CXR, a de-identified publicly available database of chest radiographs with free-text reports. Scientific data. Cited by: §1, §2, §4. [7] T. Li, M. Sanjabi, A. Beirami, and V. Smith (2020) Fair resource allocation in federated learning. In International Conference on Learning Representations, External Links: 1905.10497 Cited by: §2. [8] T. Ma, X. Luo, R. Tan, and H. Gao (2025) Privacy and fairness-guaranteed federated preference optimization for large language models in internet of medical things. IEEE Transactions on Consumer Electronics. Cited by: §2. [9] B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. Agüera y Arcas (2017) Communication-efficient learning of deep networks from decentralized data. In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, Proceedings of Machine Learning Research, Vol. 54, p. 1273–1282. Cited by: §1, §2. [10] R. Rafailov, A. Sharma, E. Mitchell, C. D. Manning, S. Ermon, and C. Finn (2023) Direct preference optimization: your language model is secretly a reward model. In Advances in neural information processing systems, Cited by: §1, §2. [11] N. Rieke, J. Hancox, W. Li, F. Milletari, H. R. Roth, S. Albarqouni, S. Bakas, M. N. Galtier, B. A. Landman, K. Maier-Hein, et al. (2020) The future of digital health with federated learning. NPJ digital medicine 3 (1), p. 119. Cited by: §1, §2. [12] A. Sabour, K. Chu, M. E. Dehaghani, and M. Moradi (2025) Mind the evaluation gap: large language models for structured data extraction from radiology reports. In International Workshop on Emerging LLM/LMM Applications in Medical Imaging, p. 19–27. Cited by: §1, §2. [13] R. Singhal, K. Ponkshe, and P. Vepakomma (2024) FedEx-LoRA: exact aggregation for federated and efficient fine-tuning of foundation models. arXiv preprint arXiv:2410.09432. Cited by: §2. [14] J. T. Wu, N. N. Agu, I. Lourentzou, A. Sharma, J. A. Paguio, J. S. Yao, E. C. Dee, W. Mitchell, S. Kashyap, A. Giovannini, L. A. Celi, and M. Moradi (2021) Chest ImaGenome dataset for clinical reasoning. In NeurIPS DB Track, External Links: 2108.00316 Cited by: §1, §2, §4. [15] A. Yang, A. Li, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Gao, C. Huang, C. Lv, et al. (2025) Qwen3 technical report. arXiv preprint arXiv:2505.09388. Cited by: §3. Appendix 0.A Scientific Supplement This supplement records the task contract, cohort construction, candidate projection, federated optimization, and supporting results used in the main paper. All tables report frozen experiment artifacts; no additional model selection was performed for the supplement. 0.A.1 Scope and Limitations FedPref is evaluated in a controlled six-site simulation derived from one public dataset. The design varies site capacity and disease prevalence while holding the ontology and annotation rule fixed. The evidence therefore does not cover institution-specific reporting styles, annotation policies, clinical workflows, or external domain shift; these questions require genuinely independent institutional cohorts. The locked test contains eight reports in three normalized-text duplicate groups; one four-report group has inconsistent targets, introducing a small source of evaluation ambiguity. The Central, FedPref, and Local arms use the same retained preference-pair corpus and matched per-pair training exposure. FedPref and Local use the same client shards and share the same local chosen-response SFT roots; introducing cross-site aggregation gives FedPref its aggregated initialization and subsequent federated DPO updates. Their comparison therefore estimates the value of cross-site aggregation within this training pipeline. Central instead optimizes directly on the exact six-shard pair union and provides the pooled reference; Central–FedPref measures the observed pooled–federated gap under matched preference evidence and exposure, rather than an aggregation-only effect. The study does not systematically vary teacher composition, scoring weights, pair margin, DPO strength, or round count, nor does it compare alternative preference objectives or a matched canonical-annotation SFT baseline. The candidate-pool diagnostics establish available response contrast, but their distinct cohorts and candidate inventories preclude attributing downstream gains to teacher heterogeneity alone. FedPref applies pair-count-weighted, factor-wise averaging to complete LoRA states. Exact low-rank aggregation methods such as FedEx-LoRA are not compared; factor-wise averaging need not equal averaging the clients’ effective updates. The protocol keeps reports and annotations local but provides no formal privacy guarantee: communicated updates are not protected by secure aggregation or differential privacy, and leakage attacks were not evaluated. Finally, the six-client, all-participation, three-round study reports 7.81 GiB of cumulative adapter traffic but does not establish runtime or network scaling under larger or partially participating federations. Direct comparisons with other federated preference methods remain future work. 0.A.2 Task Contract and Cohort Support Output contract. The model receives a radiology report and returns exactly one JSON object with two keys: diseases, a list of supported diseases, and findings, a list of disease–location objects. The semantic prompt applied the following rules to every target and teacher model: 1. emit only the two required keys and no prose or Markdown; 2. include positive, explicitly proposed, or hedged findings, while excluding findings that are only negated or absent; 3. represent each localized finding as exactly one disease and one location from the frozen vocabularies in Table 3; 4. include every localized disease in the disease list, while permitting a supported disease without a location when the report provides none; 5. expand each right- or left-sided subregion to its corresponding whole-lung parent, without inferring a subregion from a whole-lung statement; 6. deduplicate and sort diseases and findings in canonical vocabulary order; and 7. emit empty lists when the report supports no target finding. The system message instructed the model to perform careful radiology extraction, return one JSON object, use only the supplied vocabulary, and avoid unsupported inference. Table 3: Frozen output vocabularies in canonical order. Diseases Locations lung opacity; pleural effusion; atelectasis; enlarged cardiac silhouette; pulmonary edema/hazy opacity; pneumothorax; consolidation; fluid overload/heart failure; pneumonia right lung; right apical zone; right upper lung zone; right mid lung zone; right lower lung zone; right hilar structures; right costophrenic angle; left lung; left apical zone; left upper lung zone; left mid lung zone; left lower lung zone; left hilar structures; left costophrenic angle; mediastinum; upper mediastinum; cardiac silhouette; trachea Cohort integrity. Table 4 reports location support for the patient-disjoint cohorts defined in Sec. 4 of the main paper; gold-standard patients were excluded from the automatically labeled training and development data. Mediastinum, upper mediastinum, and trachea have zero positive support because none of the nine target diseases localizes there in these annotations; primary Location+Disease evaluation is therefore supported over 15 regions. Table 4: Report-level positive support by location. Training and development use automatically derived labels; diagnostic and test use gold-standard labels. Location Training (n=4,800n=4,800) Dev. (n=600n=600) Diagnostic (n=100n=100) Test (n=400n=400) Right lung 2,162 289 53 203 Right apical zone 173 26 3 15 Right upper lung zone 163 26 3 19 Right mid lung zone 416 57 8 29 Right lower lung zone 1,149 148 35 108 Right hilar structures 875 120 23 102 Right costophrenic angle 628 80 15 62 Left lung 2,263 304 54 206 Left apical zone 162 22 2 13 Left upper lung zone 128 14 3 10 Left mid lung zone 421 47 6 35 Left lower lung zone 1,362 168 32 122 Left hilar structures 847 120 26 97 Left costophrenic angle 703 92 14 62 Mediastinum 0 0 0 0 Upper mediastinum 0 0 0 0 Cardiac silhouette 950 111 27 101 Trachea 0 0 0 0 0.A.3 Candidate Generation, Projection, and Pair Construction Generation. Each frozen teacher generated one completion per report through its native chat template. Decoding was greedy with a 768-token output limit, BF16 weights, and no quantization or retrieval augmentation. Thinking was explicitly disabled for Qwen3 models. Exact model revisions are recorded in the public reproduction package11 1 https://github.com/flint-xf-fan/FedPref.. Label-independent projection. Projection operated on teacher text without opening the reference annotation. It extracted the first brace-balanced JSON object, rejected duplicate JSON keys and nonstandard constants, and required both top-level lists. Extra top-level fields were dropped and audited. Unknown vocabulary items and malformed finding entries were dropped without synonym or fuzzy matching. Recognized finding diseases were inserted into the disease list; subregions received their whole-lung parents; and the result was deduplicated and canonically sorted. An output with no valid object received the canonical empty fallback and was ineligible for oracle or pair construction. Table 5 reports the resulting audit. Table 5: Teacher-output projection audit. “Strict” denotes a valid raw JSON object before projection; “eligible” denotes successful canonical projection. Training (4,8004,800 outputs per teacher) Development (600600 per teacher) Teacher Strict Eligible Discarded Chosen Rejected Strict Eligible Discarded Llama-3.1-8B 3,650 4,771 29 815 1,219 440 598 2 Mistral-7B 3,375 4,672 128 606 1,350 414 583 17 Qwen2.5-14B 4,715 4,799 1 1,061 244 593 600 0 Qwen3-14B 4,697 4,800 0 479 148 590 600 0 Preference-pair eligibility. For each report, the highest- and lowest-scoring projected teachers formed at most one pair when their canonical structures differed, the composite-score margin was at least 0.10, and the chosen candidate had strictly higher Location+Disease F1. The training corpus retained 2,961 of 4,800 reports (61.69%); the development corpus retained 376 of 600 (62.67%). Their mean score margins were 0.426 and 0.414, with medians 0.400 and 0.382. 0.A.4 Federation Construction and Non-IID Audit Capacity-constrained multilabel allocation. All reports were assigned to sites before pair scoring. Site capacities were fixed at 28%, 22%, 18%, 14%, 10%, and 8% of each cohort. A multilabel Dirichlet profile with α=0.5α=0.5 and skew weight 0.7 allocated intact report rows. Development reused the label profiles derived from training, with independent draw seeds 20260713 and 20260714. The realized site support is reported in Table 2 of the main paper. Heterogeneity statistic. For site k, let nkn_k be cohort size, pkℓp_k the prevalence of label ℓ , and pℓp_ its global prevalence. We used nine disease indicators plus a normal-report indicator (L=10L=10) and measured D=1NL∑knk∑ℓ=1L(pkℓ−pℓ)2,N=∑knk.D= 1NL _kn_k _ =1^L(p_k -p_ )^2, N= _kn_k. (4) The null distribution came from 1,999 capacity-preserving permutations of intact multilabel report rows. The reported Monte Carlo probability uses the plus-one correction. This audit was outcome-blind and completed before training. Table 6 reports all four cohorts; retained training pairs and development reports are the training and evaluation cohorts used for the primary study. Table 6: Realized case-mix divergence against capacity-matched IID allocations. Values of D and the null 95th percentile are percentage points. Cohort N Observed D Null 95th Ratio p Training reports 4,800 9.53 1.49 6.41 0.0005 Retained training pairs 2,961 8.83 1.89 4.67 0.0005 Development reports 600 10.14 4.14 2.45 0.0005 Development retained pairs 376 9.53 5.31 1.80 0.0005 Table 7 reports disease prevalence by site among retained training pairs. Table 7: Disease prevalence (%) by site among retained training pairs. A report may contain multiple diseases, so columns need not sum to 100. Label S0 S1 S2 S3 S4 S5 Global Normal 34.3 43.1 23.6 7.2 24.2 6.7 25.7 Atelectasis 24.5 35.5 41.2 48.8 26.6 43.0 36.0 Consolidation 4.5 4.1 8.5 10.7 11.1 6.4 7.0 Enlarged cardiac silhouette 13.0 18.3 23.0 35.8 41.4 62.7 27.8 Fluid overload/heart failure 0.8 2.7 2.5 3.1 3.0 5.4 2.6 Lung opacity 61.5 50.8 73.4 89.5 62.6 72.3 67.4 Pleural effusion 15.8 15.6 21.4 35.2 29.0 27.1 22.5 Pneumonia 14.5 9.1 19.1 24.9 19.2 23.2 17.4 Pneumothorax 2.5 1.4 1.6 3.3 1.0 3.8 2.2 Pulmonary edema/hazy opacity 5.2 7.6 16.5 23.3 8.4 17.8 12.4 0.A.5 Optimization Details Adapter and training settings. All arms used LoRA rank 16 with α=32α=32, zero dropout, and no trainable bias, targeting the q, k, v, o, gate, up, and down projections. BF16 training used length 2,048, fused AdamW, cosine decay, 0.03 warmup, zero weight decay, gradient checkpointing, no packing or shuffling, and fail-closed truncation. Physical batch size was one, with gradient accumulation 24 for Central and four per client. SFT used one epoch at 2×10−42× 10^-4; each of three DPO rounds used one epoch at 5×10−65× 10^-6, β=0.1β=0.1. Seeds were 20260713, 20260714, and 20260715. Matched exposure and checkpoint lineage. Every pair received three DPO exposures in Central, FedPref, and Local. Round 1 continued from the corresponding round-0 SFT root; rounds 2 and 3 continued from the preceding policy; FedPref clients received the previous global policy. The SFT root remained the fixed DPO reference. Central opened the exact six-shard union, while FedPref and Local used the same local shards and differed only by aggregation. The server averaged complete LoRA states in float32 with pair-count weights. 0.A.6 Supporting Results Development trajectories. Table 8 gives all prespecified communication rounds. FedPref’s client mean passes Local at round 2, while its worst-site advantage widens from 0.36 points at round 0 to 9.10 points at round 3. The monotonic FedPref trajectory supports the interpretation that cross-site evidence accumulates through aggregation. Table 8: Development trajectories. Values are mean ± sample standard deviation over three seeds; all metrics are percentages. Round System Client-mean F1loc+disF_1^loc+dis Worst-site F1loc+disF_1^loc+dis Preference accuracy 0 Central 57.85±0.1457.85± 0.14 55.72±0.1155.72± 0.11 86.70±0.2786.70± 0.27 FedPref 54.93±0.4754.93± 0.47 53.67±0.1853.67± 0.18 82.00±0.4182.00± 0.41 Local 55.73±0.3755.73± 0.37 53.32±1.3053.32± 1.30 83.95±0.1583.95± 0.15 1 Central 69.34±0.0769.34± 0.07 67.45±0.3767.45± 0.37 90.51±0.1590.51± 0.15 FedPref 61.24±0.3061.24± 0.30 59.18±0.4759.18± 0.47 86.44±0.2786.44± 0.27 Local 62.29±0.1362.29± 0.13 53.80±0.0653.80± 0.06 87.85±0.3187.85± 0.31 2 Central 71.21±0.1171.21± 0.11 69.49±0.3769.49± 0.37 94.15±0.0094.15± 0.00 FedPref 66.56±0.5266.56± 0.52 65.03±0.4765.03± 0.47 89.63±0.0089.63± 0.00 Local 65.55±0.2665.55± 0.26 54.84±0.3554.84± 0.35 89.98±0.3189.98± 0.31 3 Central 71.77±0.6571.77± 0.65 70.03±1.1570.03± 1.15 94.95±0.2794.95± 0.27 FedPref 69.11±0.1769.11± 0.17 68.12±0.2268.12± 0.22 91.49±0.2791.49± 0.27 Local 66.62±0.3266.62± 0.32 59.02±1.4559.02± 1.45 91.67±0.8591.67± 0.85