Paper deep dive
Aligning with Lived Experience: Heterogeneous Benefits of Fine Tuning in Mental Health Support Generation
Mohit Chandra, Nabin Kim, Eli Min, Aamogh Sawant, Tanmay Sutar, Munmun De Choudhury
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%
Last extracted: 9/21/2026, 3:54:04 AM
Summary
This paper introduces the COPES dataset and a three-axis evaluation framework to assess Large Language Models' (LLMs) alignment with community perspectives in mental health peer support. The study evaluates zero-shot and post-trained (SFT and DPO) models, finding that post-training significantly improves Strategy Alignment and Emotion & Tone alignment, particularly for general-purpose models. However, improvements are heterogeneous across subreddits and coping strategies, with post-training inducing distributional shifts that favor problem-focused recommendations over emotion-focused ones.
Entities (12)
Relation Signals (10)
Gemma 4 E4B → evaluatedon → COPES
confidence 95% · Evaluating zero-shot and post-trained (SFT and DPO) models... Gemma-4-E4B
Qwen 3-4B Instruct → evaluatedon → COPES
confidence 95% · Evaluating zero-shot and post-trained (SFT and DPO) models... Qwen 3-4B Instruct
MediPhi-Instruct → evaluatedon → COPES
confidence 95% · Evaluating zero-shot and post-trained (SFT and DPO) models... MediPhi-Instruct
Reddit → sourceof → COPES
confidence 95% · we collected Reddit submissions and comments... to capture authentic peer-support interactions
COPES → usedfortraining → SFT
confidence 95% · post-training on COPES significantly improves Strategy Alignment
COPES → usedfortraining → DPO
confidence 95% · post-training on COPES significantly improves Strategy Alignment
DPO → improves → Strategy Alignment
confidence 90% · post-training on COPES significantly improves Strategy Alignment
SFT → improves → Strategy Alignment
confidence 90% · post-training on COPES significantly improves Strategy Alignment (>50% for general-purpose models)
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:As access to professional mental healthcare remains limited, many individuals turn to online platforms such as Reddit to seek peer support situated within human lived experience. However, a significant portion of such queries go unanswered, presenting an opportunity for using Large Language Models (LLMs) to fill this gap. While LLMs have demonstrated strong performance on clinical benchmarks, their ability to generate lived-experience informed and community-aligned peer support is underexplored. Addressing this gap, we introduce the COmmunity-centered Peer Engaged Support (COPES) dataset and a three-axis evaluation framework to assess LLM alignment with community perspectives to mental health support seeking queries. Evaluating zero-shot and post-trained (SFT and DPO) models, we show that post-training on COPES significantly improves Strategy Alignment (>50% for general-purpose models) and alignment in Emotion & Tone. However, we also observe that such improvements are heterogeneous and alignment improvements vary significantly across subreddits and requested coping strategies. Furthermore, post-training induces distributional shifts, heavily favoring problem-focused recommendations while suppressing emotion-focused strategies. Together, this work shows that while curating community-driven data improves the alignment of LLM responses, model performance remains disparate across distinct sub-communities and specific mental health needs.
Tags
Links
- Source: https://arxiv.org/abs/2609.21075v1
- Canonical: https://arxiv.org/abs/2609.21075v1
Trouble viewing inline? Open PDF directly →
Full Text
86,029 characters extracted from source content.
Expand or collapse full text
Aligning with Lived Experience: Heterogeneous Benefits of Fine Tuning in Mental Health Support Generation Mohit Chandra Affiliation: Georgia Institute of Technology Email: mchandra9@gatech.edu Nabin Kim Affiliation: Georgia Institute of Technology Email: nkim325@gatech.edu Eli Min Affiliation: Georgia Institute of Technology Email: elmmo@gatech.edu Aamogh Sawant Affiliation: Georgia Institute of Technology Email: asawant43@gatech.edu Tanmay Sutar †thanks: Work completed while at the Georgia Institute of Technology. Affiliation: Magic Hour Email: tanmay.sutar@outlook.com Munmun De Choudhury Affiliation: Georgia Institute of Technology Email: munmun.choudhury@c.gatech.edu Abstract As access to professional mental healthcare remains limited, many individuals turn to online platforms such as Reddit to seek peer support situated within human lived experience. However, a significant portion of such queries go unanswered, presenting an opportunity for using Large Language Models (LLMs) to fill this gap. While LLMs have demonstrated strong performance on clinical benchmarks, their ability to generate lived-experience informed and community-aligned peer support is underexplored. Addressing this gap, we introduce the COmmunity-centered Peer Engaged Support (COPES) dataset and a three-axis evaluation framework to assess LLM alignment with community perspectives to mental health support seeking queries. Evaluating zero-shot and post-trained (SFT and DPO) models, we show that post-training on COPES significantly improves Strategy Alignment (>50% for general-purpose models) and alignment in Emotion & Tone. However, we also observe that such improvements are heterogeneous and alignment improvements vary significantly across subreddits and requested coping strategies. Furthermore, post-training induces distributional shifts, heavily favoring problem-focused recommendations while suppressing emotion-focused strategies. Together, this work shows that while curating community-driven data improves the alignment of LLM responses, model performance remains disparate across distinct sub-communities and specific mental health needs. 1 Introduction Mental health-related conditions are increasingly prevalent in the U.S. and worldwide. It is estimated that 23.1% of the U.S. adult population struggles with some form of mental health condition of Mental Health (2024). However, access to adequate care remains inaccessible, with around 48% of U.S. adults with diagnosed conditions reporting that they were unable to obtain treatment due to affordability concerns or lack of available treatment options SAMHSA (2025). Faced with the limitations of the healthcare system, many individuals turn to online communities to seek accessible, lived-experience-based peer support to supplement or replace professional help Naslund et al. (2020); Bucci et al. (2019); Kazdin and Rabbitt (2013); De Choudhury et al. (2014). However, past research has shown that almost 35% of posts on health-related subreddits go unanswered Guimarães et al. (2021). Recent advancements in LLMs have positioned these systems as a tool for democratizing access to healthcare-related information and diagnosis Reddy (2026), with over 230 million individuals accessing ChatGPT alone for health and wellness related queries each week OpenAI (2026). LLMs have also demonstrated the ability to answer informational queries on health-related topics Yang et al. (2023); Singhal et al. (2025), and recent works have shown that LLM support on mental health related queries improves through post-training on base models Alghamdi et al. (2025); Zhang et al. (2025). Despite these advancements, LLMs continue to struggle in scenarios that involve knowledge about human lived experiences Schröder et al. (2025); Chandra et al. (2025a); Chandra et al. (2025c) and adjusting to diverse perspectives Park et al. (2024). This is a critical limitation when considering the usage of LLMs for providing mental health related peer support. Within this context, an effective response should be grounded in human lived experience while following community norms and expectations. Failing to align with these community-centered expectations can cause harm to both the support-seeker and the broader community. It also remains unclear if models (zero-shot or post-trained) inadvertently provide helpful and actionable advice for certain communities within Reddit inequitably.Moreover, there is need to understand the extent to which post-training alters the types of coping strategies recommended, and whether these shifts vary based on the target community and the underlying issues discussed in the post. To fill this gap, we present the following research questions: RQ1: How can we steer LLMs to generate community-centered responses for mental health queries, and how well do these models align with lived-experience perspectives? RQ2: How does steering LLMs toward community-centered support affect their performance across: (a) posts expressing or seeking specific coping strategies? (b) diverse mental health communities on Reddit? To address these research questions, we present the COmmunity-centered Peer Engaged Support (COPES) dataset, and an evaluation framework consisting of three axes: (1) Support Strategy Alignment, (2) Strategy Congruence, and (3) Response Emotion & Tone for assessing LLM response alignment. Furthermore, we analyze models post-trained on COPES using Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO) under this evaluation framework and compare performance to zero-shot baselines. We further examine performance based on the structure and content of advice provided in LLM responses and how these vary across subreddit categories. Our findings show that post-training LLMs on the COPES dataset allows LLMs to generate recommendations grounded in lived experiences. We observed that post-training configurations (SFT and SFT+DPO) led to significant improvement in Strategy Alignment by over 50% for general-purpose models (Qwen 3-4B Instruct, Gemma-4-E4B) and by 4.54% for MediPhi-Instruct (SFT). However, improvements varied depending on context and models involved. Moreover, post-training led to better Emotion & Tone Alignment for Gemma and MediPhi compared to their zero-shot baseline, while subreddits discussing topics around Psychosis & Anxiety and Coping & Therapy yielded better average strategy alignment across models in general. For RQ2, posts related to meaning-focused coping strategies exhibited the highest gains in Strategy Alignment (Gemma +74.27%, Qwen +69.44%), while meaning-focused posts showed the highest improvements in Emotion & Tone alignment for Gemma. Finally, post-training adjusted the coping strategies recommended by each model. Under zero-shot baseline, each model primarily used self-soothing and reframing as the uniform approach to all mental health queries. After post-training, each model developed preferred coping strategies based on the type of challenge faced. In some cases, such as posts around trauma, all three models converged on their preferred coping strategies for responses. Overall, this work presents a lived-experience centered framework for curating data and post-training models for mental health peer support task. While we observed improvements across evaluation axes, post-trained model performance remains heterogeneous, varying across both the type of support requested and the specific subreddit types. 2 Data Collection and Curation Figure 1: Overview of the dataset construction pipeline, including Reddit thread collection, quality filtering and LLM-based peer-support/coping strategy labeling, and preferred/rejected response generation. Detailed criteria and prompts are provided in Section 2. In this section, we outline the dataset creation process in three stages: data collection (Section 2.1); pre-processing (Section 2.2); and post response generation (Section 2.3). 2.1 Reddit Thread Data Collection We designed a multi-stage data collection and filtering pipeline (see Figure 1) to capture authentic peer-support interactions in online mental health support communities. Using the Pushshift dataset Baumgartner et al. (2020), we collected Reddit submissions and comments between 2011-2021 from mental health subreddits identified by Sharma and De Choudhury (2018) (see Table 4). Following the taxonomy proposed by these authors, we grouped the subreddits into five community type categories: (1) Trauma & Abuse (C1C_1), (2) Psychosis & Anxiety (C2C_2), (3) Compulsive Disorders (C3C_3), (4) Coping & Therapy (C4C_4), and (5) Mood Disorders (C5C_5). We additionally used a keyword-based filter to retain posts containing at least one term from a curated set of mental health and support-seeking keywords within post text or title (see Table 5). To improve data quality, we applied a series of filtering steps. First, we removed structurally invalid content, including posts that were deleted, removed, or duplicates. Second, we excluded non-English posts and removed non-human content by removing posts connected to known bot accounts (e.g., AutoModerator) using regex-based patterns such as “I’m a bot” or “This action was performed automatically.” Third, we excluded potentially low-quality or unhelpful interactions, including NSFW content, comments shorter than five words, controversial comments (controversiality = 1), and collapsed comments. Finally, because our response-generation procedure synthesizes peer support from multiple community comments, we required posts to have at least three non-bot comments. We also retained comments with a net upvote score of ≥1≥ 1, treating positive community feedback as a weak endorsement signal. After filtering, 5,536 posts remained. 2.2 Automated Data Labeling and Filtering After initial data collection and filtering (Figure 1(a) and (b)), we used an LLM-as-a-judge methodology to identify posts seeking non-clinical peer support. We used GPT-4.1 OpenAI (2025) as the annotation model using deterministic decoding (temperature = 0.0, top-p = 1.0). We structured the automated labeling task using a two-tiered hierarchical annotation consisting of (1) a binary peer-support classification and (2) a multi-class categorization of coping strategies. Binary Peer-Support Classification: Within this task, we labeled a post as peer support “yes” if it explicitly or implicitly sought emotional encouragement, shared experiences, reassurance, or coping advice, and “no” if it was purely informational, technical, or non-interactive (prompt in Table 6). Coping Strategy Classification: To characterize the different ways Reddit users express the need for peer support, we took inspiration from established psychological coping frameworks Folkman (1997); Folkman and Moskowitz (2000); Folkman (2013); Thoits (2011) and categorized support types into four broad categories: (1) problem-focused, (2) emotion-focused, (3) meaning-focused, and (4) social coping. Problem-focused coping focuses on taking practical action to change external factors causing stress, while meaning-focused coping focuses on changing internal factors to reduce stress by finding personal growth in the situation. Social coping involves turning to others for support and advice, and emotion-focused coping means engaging in self-soothing behaviors to reduce distress. For each Reddit post seeking peer support, we modeled each coping strategy category as an independent binary decision. We chose this approach over a single multi-class label to capture how multiple coping strategies often co-occur within a single post (prompt in Table 7). Additionally, to ensure robustness, we complemented automated labeling with human evaluation on a sampled subset of the data in Appendix B.2. After filtering out posts with missing information, we had 4,455 peer-support seeking Reddit posts with coping strategy classification labels. We further split the dataset into Train, Validation, and Test in the ratio of 70:15:15. Table 1 provides statistics and distribution of support strategy labels. Category Train Val Test Total Examples 3118 669 668 Problem Focused 1151 224 233 Emotion Focused 2224 474 466 Meaning Focused 760 148 138 Social Coping 2115 451 472 Table 1: COPES dataset distribution of categories across data splits. Dataset was split in the ratio 70:15:15 for Train:Validation:Test. 2.3 Dataset Response Generation For each labeled post, we created a preferred response and a rejected response, which together formed the preference pair used for post-training. Preferred Response Generation (Ground-Truth): We constructed the preferred response by summarizing the peer support provided in the Reddit comments for each post. We used GPT-4.1 as an LLM-based summarizer to combine the associated comments for each post into a single coherent response. The prompt also asked the model to remove redundant content, avoid generic openings, and maintain a natural peer-support tone (see Table 8). Rejected Response Generation: To construct preference pairs for Direct Preference Optimization (DPO) Rafailov et al. (2023), we also created a non-preferred response (rejected response) for every post in the dataset. Our initial experiments indicated that summarizing downvoted comments for non-preferred responses was unreliable, as these often reflected engagement rather than relevance. Hence, we generated rejected response variants using GPT-4.1 to synthesize suboptimal user responses from the original Reddit threads (prompt in Table 10). To encourage semantic and stylistic diversity among rejected responses, we defined a theoretically situated taxonomy of 11 negative support strategies (see Table 9), including emotional invalidation, misguided positivity, and social undermining, grounded in prior work on harmful social support and communication Zielinski and Veilleux (2018); Vinokur and Van Ryn (1993); Revenson et al. (1991); Almajed and ALdayel (2026); Feng and Magen (2016); Vangelisti et al. (1990); Rozin (1999); Pargament et al. (2003). 3 Model Selection and Evaluation Axes In this section, we present our evaluation and post-training methodology for LLMs to provide peer-like, long-form support and advice. We used two general-purpose models: (1) Qwen 3-4B Instruct Yang et al. (2025), (2) Google Gemma-4-E4B Google DeepMind (2026), and one model specific to the medical domain, (3) Microsoft MediPhi-Instruct (3.8B) Corbeil et al. (2025). We selected these models to compare general-purpose and medical-domain instruction-tuned models under a comparable small-model setting. Additionally, their accessible weights allowed us to apply the same SFT and DPO pipeline while avoiding dependence on proprietary model APIs. For the evaluation criteria we used three metrics: (1) Support Strategy Alignment, (2) Support Strategy Congruence, and (3) Emotion & Tone Alignment. Support Strategy Alignment: Reddit users frequently seek peer support grounded in lived human experience Bearne (2026). We therefore focus on whether generated responses preserve coping advice reflected in ground-truth responses. We operationalized support strategy alignment as a proxy measure of whether LLM-generated responses preserve coping recommendations reflected in ground-truth preferred responses, not as a direct measure of overall peer-support quality. Given the long-form nature of both ground-truth and LLM-generated responses (prompt in Table 11), assessing their alignment required us to disentangle the semantic content from the stylistic features. Hence, we extracted the recommendations present in the ground truth and the LLM-generated responses. We treated each extracted recommendation as an atomic support strategy. Then, we used vector embeddings to match the LLM-generated support strategies with the ground truth strategies. We describe both in detail below. Strategy Alignment Strategy Congruence Emotion & Tone Alignment Model Zero-Shot SFT SFT+DPO Zero-Shot SFT SFT+DPO Zero-Shot SFT SFT+DPO Qwen 3-4B Instruct 0.30 ± 0.284 0.47 ± 0.320 (+56.52%)*** 0.48 ± 0.318 (+60.32%)*** 0.64 ± 0.050 0.65 ± 0.059 (+1.71%)*** 0.65 ± 0.058 (+2.06%)*** 2.73 ± 4.296 2.80 ± 4.071 (-2.33%) 2.78 ± 4.147 (-1.63%) Gemma-4-E4B 0.32 ± 0.284 0.50 ± 0.318 (+55.06%)*** 0.50 ± 0.317 (+54.59%)*** 0.62 ± 0.055 0.65 ± 0.056 (+3.63%)*** 0.65 ± 0.060 (+3.40%)*** 3.23 ± 4.640 2.79 ± 4.161 (+13.51%)*** 2.94 ± 4.729 (+9.04%)*** MediPhi-Instruct 0.46 ± 0.345 0.48 ± 0.320 (+4.54%) 0.47 ± 0.326 (+3.03%) 0.65 ± 0.058 0.65 ± 0.061 (-0.92%)*** 0.65 ± 0.057 (-0.93%)** 2.76 ± 4.951 2.65 ± 4.205 (+3.78%) 2.73 ± 4.142 (+1.06%) Table 2: Evaluation of response alignment for LLMs across different training configurations. The table compares Strategy Alignment, Strategy Congruence, and Emotion & Tone Alignment for baseline (Zero-Shot), SFT, and SFT+DPO configurations. Values represent mean scores ± standard deviation. Percentage values in parentheses indicate relative improvement over the Zero-Shot baseline, with * denoting statistical significance. For Strategy Alignment and Emotion & Tone alignment we used Wilcoxon Signed-Rank Test for checking significance of the results. For Strategy Congruence we used Mann-Whitney U Test. Drawing on the definition provided by Min et al. (2023), we framed atomic support strategies as short, self-contained recommendations that express one coping suggestion at a time. We used the Qwen3-30B-A3B-Instruct-2507 model Yang et al. (2025) and the prompting methodology described by Chandra et al. (2025c) for the strategy extraction process (prompt in Table 12). To reduce redundancy among the extracted atomic strategies, we combined strategies based on the actors and actions involved (prompt in Table 13). We grouped the combined strategies into the four coping categories introduced in Section 2.2: Emotion-Focused Coping, Problem-Focused Coping, Meaning-Focused Coping, and Social Coping Folkman and Moskowitz (2004); Algorani and Gupta (2023). We validated the extraction and combination steps using human evaluation over a random sample of 30 responses, which included 187 extracted strategies and 100 combined groups. Two annotators independently labeled these results and achieved an inter-rater agreement score of 0.90 using Cohen’s Kappa. Examples of extracted strategies and their combined groups are available in Table 14 in the Appendix. To evaluate strategy alignment between the LLM-generated and ground truth responses, we constructed a vector database containing embeddings of Reddit post text, title, and extracted support strategies from the ground truth answer using Qwen3-Embedding-4B model. Each entry also contained information about the respective subreddit category. For each atomic support strategy (sks_k) extracted from an LLM response (LtL_t) corresponding to a post text within the test set (PtP_t), we first used the thematic similarity within subreddits in the same category (CiC_i) by isolating all posts in the same sub-category as PtP_t. We then compute the cosine similarity between the embedding of the test set post (PtP_t) and the embeddings of all posts in this filtered subset, retaining candidates that meet a specific similarity score threshold ≥0.8≥ 0.8 and capping the results at the top 10 most similar posts to ensure maximum relevance. Finally, for each extracted strategy sks_k from the LLM response (LtL_t), we compare its embedding with the strategy embeddings of the retrieved candidate posts’ Ground-Truth answers. A strategy is considered “aligned” if it achieves the required cosine similarity threshold ≥0.8≥ 0.8 with one or more strategies in the candidate set. Support Strategy Congruence: In addition to support strategy alignment, we analyzed how semantically coherent the extracted strategies are within each LLM-generated response. For a given LLM response LtL_t containing a set of strategy embeddings s1,s2,…,sn\s_1,s_2,…,s_n\, we define congruence as the mean pairwise cosine similarity among the extracted strategy embeddings. Higher congruence indicates that a response provides a more semantically coherent set of recommendations. However, a marginal reduction in congruence is not necessarily detrimental for the task of providing peer support, where diverse perspectives are often valued. Emotion and Tone Alignment: Previous work has highlighted the importance of emotional alignment between the support provider and support seeker to drive meaningful change Atzil-Slonim et al. (2018). Inspired by prior work, we used Empath Fast et al. (2016) to measure emotion and tone alignment between LLM-generated responses and ground truth along 8 relevant emotional and tonal categories Chandra et al. (2025c). We analyzed the distribution of these categories and measured alignment using Kullback-Leibler (KL) divergence. 4 RQ1: Peer-Support Response Generation Table 2 reports Strategy Alignment, Strategy Congruence, and Emotion & Tone Alignment across zero-shot, SFT, and SFT+DPO configurations for all three models. 4.1 Evaluation of Zero-shot and Post-Trained Models on COPES Strategy Alignment: As shown in Table 2, strategy alignment proved to be a challenging objective across all models (zero-shot or post-trained). The maximum strategy alignment was achieved for Gemma-4-E4B under the SFT method (50%50\%). However, post-training using SFT and SFT+DPO resulted in improvement over the zero-shot baseline across all models. Specifically, we observed the largest gain in alignment for Qwen 3-4B with SFT (and SFT+DPO) increasing alignment by 56.52%56.52\% (60.32%60.32\%). In contrast, MediPhi-Instruct showed lower improvement of 4.54%4.54\% for SFT and 3.03%3.03\% for SFT+DPO. These results reinforce the challenges of Personalized Alignment Guan et al. (2025), especially for tasks involving nuanced aspects of human social and emotional scenarios such as seeking peer support. Although SFT and SFT+DPO produced large relative gains, the absolute strategy alignment scores remained modest, suggesting that post-training only partially captures the community context underlying Reddit peer support. This may reflect the heterogeneity of peer-support interactions, where responses can involve advice, validation, disclosure, and shared lived experience rather than a single discrete coping strategy. This limitation echoes broader work on prediction in complex social systems, where performance can be constrained by the inherent variability of social behavior as well as by data or model quality (Martin et al., 2016). Strategy Congruence: We observed modest improvements for the general-purpose models, with Gemma-4-E4B achieving improvement of 3.63%3.63\% and 3.40%3.40\% using SFT and SFT+DPO respectively. In contrast, MediPhi-Instruct showed a decrease of 0.93%0.93\% (SFT+DPO) and 0.92%0.92\% (SFT) compared to the baseline zero-shot setting. A plausible explanation for this observation could be the ceiling effect related to thematic consistency of recommendations. MediPhi-Instruct training on medical corpora likely established a robust zero-shot baseline for peer-support tasks. Hence, this high initial baseline left limited room for substantial post-training gain in strategy congruence. Another plausible explanation is that diverse peer-support perspectives may be inconsistent with standard clinical guidelines found in biomedical literature used for training medical LLMs. Figure 2: Relative changes in Strategy Alignment, Strategy Congruence, and Emotion & Tone metrics for Qwen-3 4B following SFT and SFT+DPO, compared to zero-shot baselines across Reddit post coping strategy labels. Results for Gemma, Mediphi and Qwen are presented in Appendix Figure 6. Emotion and Tone Alignment: Post-training generally reduced the KL divergence of the category distribution for LLM responses to the preferred responses distribution (with exception of Qwen 3-4B Instruct). While we observed no significant difference in the distribution of categories within LLM responses with the distribution within ground truth under any setting, we observed that Gemma model produced notable more aligned responses with the ground truth when compared to the zero shot baseline (+13.51% for SFT and 9.04% for SFT+DPO). 4.2 Performance Disparity for Different Types of Subreddits Gemma-4 E4B Qwen 3-4B Instruct MediPhi-Instruct Instruct Cat. SFT SFT+ DPO SFT SFT+ DPO SFT SFT+ DPO C1 89.98% 102.84% 16.03% 20.30% 28.44% 19.53% C2 68.97% 77.34% 56.64% 63.75% 18.89% 20.47% C3 20.48% 11.45% 340.00% 397.78% -0.15% -17.02% C4 54.29% 43.29% 84.65% 99.34% 9.68% 10.35% C5 46.56% 43.82% 52.41% 51.48% -4.93% -7.53% Table 3: Relative improvement (%) in strategy alignment for post-trained model variants compared to each model’s zero-shot baseline across subreddit categories (C1…C5C_1… C_5; Ref: 2.1). Colors indicate direction of change (teal for positive, orange for negative). We further evaluated whether post-training led to varied improvements across the five subreddit communities (C1…C5C_1… C_5). Table 3 shows that post-training generally improved the strategy alignment scores across all models and categories except in the case of MediPhi-Instruct for C5C_5 and C3C_3 subreddit categories. On average, the relative improvement after post-training (compared to baseline) was most pronounced for subreddits related to C2C_2 (Psychosis & Anxiety) and C4C_4 (Coping & Therapy) communities (excluding Qwen 3 performance for C3C_3). On the other hand, when treating the unusually large Qwen gains for C3 as an outlier, we observed the least average performance boost across the model families for C5C_5 (Mood Disorders). Within C1C_1 (Trauma & Abuse), smaller and more model-dependent gains may reflect the fact that trauma-related support often foregrounds safety, validation, disclosure, and lived experience, which are less likely to appear as explicit, extractable coping strategies. 5 RQ2: Impact of Post-Training on Response Generation for Varied Coping Strategies For the second research question, we analyzed how post-training effects vary across coping strategies and subreddit communities through three complementary analyses. First, we used the coping strategy labels associated with each test-set post (Section 2.2) to analyze the heterogeneity in model performance across the four coping strategies under SFT and SFT+DPO relative to the zero-shot baseline. Second, we examined how post-training changed the distribution of generated coping strategy types within the LLM response. Additionally, we also measured broader topical alignment between LLM generated responses and the ground-truth responses using a Hierarchical Dirichlet Process (HDP) model. Specifically, we adopted the simHDPsim_HDP metric from Jin et al. (2024). For each model and training configuration, we fitted an HDP model to the generated responses and computed the cosine similarity between each output and its reference in the same topic space. 5.1 Performance Heterogeneity for Varied Coping Strategy Types Across all models and coping strategy categories, SFT and SFT+DPO based post-training improved Strategy Alignment compared to the zero-shot baseline (except for Mediphi for emotion focused coping). For SFT+DPO setting, general-purpose models achieved substantially larger relative gains than MediPhi-Instruct across categories, with Gemma-4-E4B and Qwen 3-4B (Figure 2) improving by +60.35% and +59.13% on average across all coping categories, respectively, compared to +7.02% for MediPhi-Instruct. However, the gain in performance was unevenly distributed across the coping strategy categories. For both Gemma and Qwen, posts annotated with meaning-focused coping showed the highest average gains across post-training methods (+74.27% and +69.44%, respectively). This pattern may indicate that post-training better captures meaning-focused support. While we observed similar gain in Strategy Congruence scores for Gemma and Qwen models across coping strategy categories, MediPhi-Instruct showed a marginal decrease (<<2.5%) in most categories, except for problem-focused coping under the SFT setting. For Emotion & Tone alignment, we observed that SFT configuration led to the largest improvements for meaning-focused coping posts for Gemma (+19.33%). On the other hand, MediPhi-Instruct exhibited the greatest increased in emotion and tone alignment for problem-focused coping posts (+12.68% under SFT and +17.07% under SFT+DPO). 5.2 Change in Coping Strategy Type Distribution For the zero-shot setting, general-purpose model responses focused on suggesting self-soothing and reframing coping strategies. Emotion-focused approaches appeared in an average of 35.3% of responses for Qwen and up to 55.1% for Gemma. In contrast, MediPhi generally maintained a uniform approach to each subreddit category, suggesting that almost every issue was receiving at least one recommendation from each coping strategy. After post-training, each model developed a more contextual approach to the issues presented within the Reddit posts (see Figure 5). For Trauma & Abuse related posts (C1C_1), all three models converged on emotion-focused coping as the most frequently used strategy, shifting by +2.8% for Qwen, +15.1% for MediPhi, and -20.8% for Gemma under SFT compared to their zero-shot baselines. This was followed by problem-focused coping as the second most frequently recommended strategy. However, this alignment on coping strategies did not always happen; models contained some discrepancies in their preferred strategies. Noteworthy examples were in social coping and problem-focused coping, which were emphasized in some cases and used infrequently in others. Social coping was used infrequently for C1C_1 category posts, with the largest drop being MediPhi-Instruct (-15.4% SFT+DPO), but it was used more frequently for Psychosis & Anxiety (C2C_2) category posts at 24.5% for MediPhi (w.r.t. the zero-shot setting). Gemma generally did not use social coping approaches for C5C_5 (Mood Disorders) even though other models do not make this strong distinction (see Figure 3). Figure 3: Distribution of generated coping strategy types across subreddit community categories for Gemma-4 E4B. Post-training changes the type of advice models produce, with general-purpose models shifting toward more problem-focused recommendations. Results for Gemma, Mediphi and Qwen are presented in Appendix Figure 5. 5.3 Changes in Response Topical Theme As shown in Figure 4. After SFT, models showed mixed changes in topical alignment compared to their zero-shot baselines. Gemma-4 E4B mostly preserved topical alignment, with changes ranging from −3.4%-3.4\% for C1C_1 to +9.6%+9.6\% for C2C_2. MediPhi-Instruct showed a similar pattern, with changes ranging from −4.8%-4.8\% for C5C_5 to +2%+2\% for C2C_2. Both models in general preserved category-specific content while integrating peer-support language, such as validation, reassurance, and emotional grounding. In contrast, Qwen-3 4B’s topical alignment declined across all five categories, with drops from −2.4%-2.4\% for C3C_3 to −13.3%-13.3\% for C2C_2. This observed shift was due to responses shifting toward broader supportive themes at the expense of reference-specific content, especially around anxiety and psychosis-related symptoms in C2C_2. Under SFT+DPO, all three models showed lower topical alignment than their zero-shot baselines. Gemma-4B declined across every category, with the largest drop in C1C_1 (−32.5%-32.5\%). Similarly, Qwen-3 4B exhibited reduction in alignment, ranging from −7.2%-7.2\% for C4C_4 to −16.2%-16.2\% for C3C_3. Compared to the baseline, SFT+DPO introduced a stronger and more consistent topic shift across models. Similar to SFT, responses focused on broader peer support related topics. Overall, this analysis shows that this shift induced due to post-training is highly dependent on the target subreddit category. 6 Related Work NLP for Mental Health Related Tasks Use of language models for mental health-related tasks has recently gained traction, from identifying empathic conversations Sharma et al. (2020), to generating emotional support dialogues with predefined support strategies Liu et al. (2021), or responses based on community response signals and support seeker’s response Alghamdi et al. (2025). With the rise of LLMs, benchmarking and evaluation have become one of the most-studied aspects. Previous studies have assessed LLMs for mental health counseling compentancy Nguyen et al. (2025), multi-turn sensemaking Chandra et al. (2025b), adherence to motivational interviewing theory Gabriel et al. (2024), medical examination questions Singhal et al. (2025), and alignment with experts on psychiatric medication ADRs Chandra et al. (2025c). To improve LLM performance, researchers have also used few-shot prompting for mental health condition detection Yang et al. (2023), RAG for mental health text analysis Kermani et al. (2025), and preference optimization for emotional support generation Zhang et al. (2025). However, past works have understudied model’s ability to preserve community-specific coping strategies Guan et al. (2025). Our work addresses this gap. Social media based peer support and coping Online social media platforms especially Reddit serve as an important avenue for individuals to talk about mental health because it supports pseudonymous self-disclosure, access to community, and large-scale mental health discourse De Choudhury and De (2014); Cohan et al. (2018); Morini et al. (2025). Past works have examined the relationship between linguistic accommodation and support received Sharma and De Choudhury (2018), link between receiving early social support and providing support to others Chen and Xu (2021) impact of language of received support on suicidal ideation risk De Choudhury and Kiciman (2017), and variation in informational and emotional support provided by users across community structures (Kim et al., 2023). Other works have also focused on characterizing effective support includes complex language factors such as diversity, adaptability, and style Saha and Sharma (2020). Our work builds on this literature by using Reddit peer interactions as a community-grounded reference for evaluating whether LLM-generated responses align with human lived-experiences. 7 Conclusion In this work, we introduced COPES, a community-centered dataset for mental health-related peer-support tasks, and a multi-axis offline evaluation framework to examine LLM-generated peer-support response alignment with community grounded responses. Using COPES and our evaluation framework, we evaluated two general-purpose and one medical LLM across zero-shot, SFT, and SFT+DPO settings. Our findings show that post-training on COPES resulted in a significant increase in support strategy alignment compared to the baseline zero-shot version (>50%>50\%) for general-purpose models. Additionally, post-training also shifted model behavior from recommending uniform self-soothing and reframing coping strategies to context-specific coping strategies. However, the benefits from post-training remain heterogeneous, varying across model architectures, the specific community context, and the nature of the support requested. 8 Limitations While our work presents a novel dataset, a methodology for evaluating community-centeredness of LLM responses and novel insights on the impact of post-training, it is important to acknowledge the limitations of this work. First, the COPES dataset focuses on non-clinical mental health support-seeking behavior which does not capture the full range of clinical scenarios. To construct COPES at scale, we used LLM-as-a-judge methodology to label posts. While we conducted human evaluation to validate LLM labels, reliance on automated labeling remains a limitation. Our findings should be interpreted within the scope of the dataset construction procedure. COPES is built from answered Reddit threads collected between 2011 and 2021 from a fixed set of mental health subreddits. Because our method requires multiple positively received comments, COPES reflects well-engaged, community-endorsed peer-support exchanges rather than unanswered-post deployment. As a result, our findings may not fully generalize to unanswered posts, post-2021 community norms, or unseen mental health communities, and may over-represent threads that were more visible or socially supported. Furthermore, we used LLMs to generate the negative responses due to the infeasibility of manually writing long-form responses for over 4,400 Reddit posts. Second, we also acknowledge the limitations in our evaluation methodology. To facilitate evaluation at scale, we measured strategy alignment using an embedding similarity approach and kept a strict threshold to reduce false positives. However, this conservative approach may sometimes miss forms of support that primarily rely on narrative disclosures or emotional validation. We also acknowledge the limitation of evaluating only small open-weight models. This constraint was dictated by computational and budget constraints, as well as our requirement for direct access to model weights for post-training rather than reliance on proprietary model APIs. Finally, we acknowledge that our study is limited to two post-training methods (SFT and DPO). Using reward-based preference tuning methods required defining custom reward signals which fell outside the scope of this paper. Despite these limitations, our work provides a community-centered dataset, an evaluation framework for studying support-strategy alignment in LLM-generated peer support responses, and a reproducible post-training pipeline for future research. 9 Ethical considerations For the creation of the COPES dataset, we used publicly available archival Reddit data from the Pushshift dataset Baumgartner et al. (2020). Because our work involved retrospective analysis of public posts without direct interaction with Reddit users, the Institutional Review Board (IRB) classified it as non-human subjects research and exempted it from IRB approval. The datset is intended to be used for research purposes. We also checked the dataset for personally identifiable information using the Microsoft Presidio library, focusing on entities detected with high confidence (threshold >0.7>0.7). This analysis did not identify sensitive personal identifiers, detected entities were limited to generic email addresses and helpline phone numbers. To reduce privacy risks, we will release COPES using Reddit post IDs rather than raw Reddit text. Finally, all human evaluations used to validate the LLM-generated labels and the broader evaluation pipeline were conducted by the authors, and no external participants were recruited. COPES should be treated as a research benchmark rather than evidence of readiness for real-world mental health support deployment. Better performance on COPES should not be interpreted as evidence of a model’s safety or efficacy in providing peer support. LLM-generated responses may still provide inappropriate advice, miss signs of crisis, overgeneralize from community patterns, or fail to respond to individual needs. Deploying LLMs for real-world mental health applications would require additional safeguards, crisis escalation protocols, human oversight, and evaluation with affected communities. Finally, post-training models to produce lived-experience-informed responses introduces an important trade-off. Online communities can provide valuable peer support grounded in lived experience, but they may also propagate stigma, harmful advice, or norms that are not clinically appropriate. By synthesizing preferred responses from Reddit comments, we aimed to study coping-strategy alignment rather than evaluate the clinical efficacy of community-generated advice. A remaining challenge is how to balance community-grounded alignment with safety, clinical appropriateness, and the needs of affected users. 10 Acknowledgments Chandra and De Choudhury were partly supported through awards from Microsoft and Google. Min was supported through an NSF Graduate Fellowship. References Alghamdi et al. (2025) Z. Alghamdi, T. Kumarage, G. Agrawal, M. Karami, I. Almuteb, and H. Liu RedditESS: a mental health social support interaction dataset – understanding effective social support to refine ai-driven support tools. External Links: 2503.21888, Link Cited by: §1, §6. Algorani and Gupta (2023) E. B. Algorani and V. Gupta Coping mechanisms. Note: StatPearls [Internet]. Treasure Island (FL): StatPearls PublishingUpdated April 24, 2023 External Links: Link Cited by: §3. Almajed and ALdayel (2026) L. Almajed and A. ALdayel Incongruent positivity: when miscalibrated positivity undermines online supportive conversations. External Links: 2509.10184, Link Cited by: §2.3. Atzil-Slonim et al. (2018) D. Atzil-Slonim, E. Bar-Kalifa, H. Fisher, T. Peri, W. Lutz, J. Rubel, and E. Rafaeli Emotional congruence between clients and therapists and its effect on treatment outcome. Journal of Counseling Psychology 65 (1), p. 51–64. External Links: Document Cited by: §3. Baumgartner et al. (2020) J. Baumgartner, S. Zannettou, B. Keegan, M. Squire, and J. Blackburn The pushshift reddit dataset. Proceedings of the International AAAI Conference on Web and Social Media 14 (1), p. 830–839. External Links: Link, Document Cited by: §2.1, §9. Bearne (2026) S. Bearne Reddit’s Human Content Wins Amid the AI Flood. Note: https://w.bbc.com/news/articles/c5y4zl0w062o[Accessed 07-05-2026] Cited by: §3. Bucci et al. (2019) S. Bucci, M. Schwannauer, and N. Berry The digital revolution and its impact on mental health care. Psychology and Psychotherapy: Theory, Research and Practice 92 (2), p. 277–297. External Links: Document, Link, https://bpspsychub.onlinelibrary.wiley.com/doi/pdf/10.1111/papt.12222 Cited by: §1. Chandra et al. (2025a) M. Chandra, S. Naik, D. Ford, E. Okoli, M. De Choudhury, M. Ershadi, G. Ramos, J. Hernandez, A. Bhattacharjee, S. Warreth, and J. Suh From lived experience to insight: unpacking the psychological risks of using ai conversational agents. In Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’25, New York, NY, USA, p. 975–1004. External Links: ISBN 9798400714825, Link, Document Cited by: §1. Chandra et al. (2025b) M. Chandra, S. Sriraman, H. S. Khanuja, Y. Jin, and M. D. Choudhury Reasoning is not all you need: examining llms for multi-turn mental health conversations. External Links: 2505.20201, Link Cited by: §6. Chandra et al. (2025c) M. Chandra, S. Sriraman, G. Verma, H. S. Khanuja, J. S. Campayo, Z. Li, M. L. Birnbaum, and M. De Choudhury Lived experience not found: LLMs struggle to align with experts on addressing adverse drug reactions from psychiatric medication use. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), L. Chiruzzo, A. Ritter, and L. Wang (Eds.), Albuquerque, New Mexico, p. 11083–11113. External Links: Link, Document, ISBN 979-8-89176-189-6 Cited by: §1, §3, §3, §6. Chen and Xu (2021) Y. Chen and Y. Xu Social support is contagious: exploring the effect of social support in online mental health communities. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems, CHI EA ’21, New York, NY, USA. External Links: ISBN 9781450380959, Link, Document Cited by: §6. Cohan et al. (2018) A. Cohan, B. Desmet, A. Yates, L. Soldaini, S. MacAvaney, and N. Goharian SMHD: a large-scale resource for exploring online language usage for multiple mental health conditions. In Proceedings of the 27th International Conference on Computational Linguistics, E. M. Bender, L. Derczynski, and P. Isabelle (Eds.), Santa Fe, New Mexico, USA, p. 1485–1497. External Links: Link Cited by: Table 5, §6. Corbeil et al. (2025) J. Corbeil, A. Dada, J. Attendu, A. Ben Abacha, A. Sordoni, L. Caccia, F. Beaulieu, T. Lin, J. Kleesiek, and P. Vozila A modular approach for clinical SLMs driven by synthetic data with pre-instruction tuning, model merging, and clinical-tasks alignment. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), W. Che, J. Nabende, E. Shutova, and M. T. Pilehvar (Eds.), Vienna, Austria, p. 19352–19374. External Links: Link, Document, ISBN 979-8-89176-251-0 Cited by: §3. De Choudhury and De (2014) M. De Choudhury and S. De Mental health discourse on reddit: self-disclosure, social support, and anonymity. Proceedings of the International AAAI Conference on Web and Social Media 8 (1), p. 71–80. External Links: Link, Document Cited by: §6. De Choudhury and Kiciman (2017) M. De Choudhury and E. Kiciman The language of social support in social media and its effect on suicidal ideation risk. Proceedings of the International AAAI Conference on Web and Social Media 11 (1), p. 32–41. External Links: Link, Document Cited by: §6. De Choudhury et al. (2014) M. De Choudhury, M. R. Morris, and R. W. White Seeking and sharing health information online: comparing search engines and social media. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’14, New York, NY, USA, p. 1365–1376. External Links: ISBN 9781450324731, Link, Document Cited by: §1. Fast et al. (2016) E. Fast, B. Chen, and M. S. Bernstein Empath: understanding topic signals in large-scale text. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, CHI ’16, New York, NY, USA, p. 4647–4657. External Links: ISBN 9781450333627, Link, Document Cited by: §3. Feng and Magen (2016) B. Feng and E. Magen Relationship closeness predicts unsolicited advice giving in supportive interactions. Journal of Social and Personal Relationships 33 (6), p. 751–767. External Links: Document Cited by: §2.3. Folkman and Moskowitz (2000) S. Folkman and J. T. Moskowitz Positive affect and the other side of coping. Am Psychol 55 (6), p. 647–654 (en). Cited by: §2.2. Folkman and Moskowitz (2004) S. Folkman and J. T. Moskowitz Coping: pitfalls and promise. Annual Review of Psychology 55, p. 745–774. External Links: Document, Link Cited by: §3. Folkman (1997) S. Folkman Positive psychological states and coping with severe stress. Social Science & Medicine 45 (8), p. 1207–1221. External Links: ISSN 0277-9536, Document, Link Cited by: §2.2. Folkman (2013) S. Folkman Stress: appraisal and coping. In Encyclopedia of Behavioral Medicine, M. D. Gellman and J. R. Turner (Eds.), p. 1913–1915. External Links: ISBN 978-1-4419-1005-9, Document, Link Cited by: §2.2. Gabriel et al. (2024) S. Gabriel, I. Puri, X. Xu, M. Malgaroli, and M. Ghassemi Can AI relate: testing large language model response for mental health support. In Findings of the Association for Computational Linguistics: EMNLP 2024, Y. Al-Onaizan, M. Bansal, and Y. Chen (Eds.), Miami, Florida, USA, p. 2206–2221. External Links: Link, Document Cited by: §6. Google DeepMind (2026) Google DeepMind Gemma 4. Note: https://deepmind.google/models/gemma/gemma-4/Accessed: 2026-05-06 Cited by: §3. Guan et al. (2025) J. Guan, J. Wu, J. Li, C. Cheng, and W. Wu A survey on personalized Alignment—The missing piece for large language models in real-world applications. In Findings of the Association for Computational Linguistics: ACL 2025, W. Che, J. Nabende, E. Shutova, and M. T. Pilehvar (Eds.), Vienna, Austria, p. 5313–5333. External Links: Link, Document, ISBN 979-8-89176-256-5 Cited by: §4.1, §6. Guimarães et al. (2021) A. Guimarães, E. Terolli, and G. Weikum Comparing health forums: user engagement, salient entities, medical detail. In Companion Publication of the 2021 Conference on Computer Supported Cooperative Work and Social Computing, CSCW ’21 Companion, New York, NY, USA, p. 57–61. External Links: ISBN 9781450384797, Link, Document Cited by: §1. Jin et al. (2024) Y. Jin, M. Chandra, G. Verma, Y. Hu, M. De Choudhury, and S. Kumar Better to ask in english: cross-lingual evaluation of large language models for healthcare queries. In Proceedings of the ACM Web Conference 2024, W ’24, New York, NY, USA, p. 2627–2638. External Links: ISBN 9798400701719, Link, Document Cited by: §5. Kazdin and Rabbitt (2013) A. E. Kazdin and S. M. Rabbitt Novel models for delivering mental health services and reducing the burdens of mental illness. Clinical Psychological Science 1 (2), p. 170–191. External Links: Document, Link, https://doi.org/10.1177/2167702612463566 Cited by: §1. Kermani et al. (2025) A. Kermani, V. Perez-Rosas, and V. Metsis A systematic evaluation of LLM strategies for mental health text analysis: fine-tuning vs. prompt engineering vs. RAG. In Proceedings of the 10th Workshop on Computational Linguistics and Clinical Psychology (CLPsych 2025), A. Zirikly, A. Yates, B. Desmet, M. Ireland, S. Bedrick, S. MacAvaney, K. Bar, and Y. Ophir (Eds.), Albuquerque, New Mexico, p. 172–180. External Links: Link, Document, ISBN 979-8-89176-226-8 Cited by: §6. Kim et al. (2023) M. Kim, K. Saha, M. De Choudhury, and D. Choi Supporters first: understanding online social support on mental health from a supporter perspective. Proc. ACM Hum.-Comput. Interact. 7 (CSCW1). External Links: Link, Document Cited by: Table 5, §6. Liu et al. (2021) S. Liu, C. Zheng, O. Demasi, S. Sabour, Y. Li, Z. Yu, Y. Jiang, and M. Huang Towards emotional support dialog systems. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), C. Zong, F. Xia, W. Li, and R. Navigli (Eds.), Online, p. 3469–3483. External Links: Link, Document Cited by: §6. Martin et al. (2016) T. Martin, J. M. Hofman, A. Sharma, A. Anderson, and D. J. Watts Exploring limits to prediction in complex social systems. In Proceedings of the 25th International Conference on World Wide Web, p. 683–694. External Links: Document Cited by: §4.1. Min et al. (2023) S. Min, K. Krishna, X. Lyu, M. Lewis, W. Yih, P. Koh, M. Iyyer, L. Zettlemoyer, and H. Hajishirzi FActScore: fine-grained atomic evaluation of factual precision in long form text generation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, H. Bouamor, J. Pino, and K. Bali (Eds.), Singapore, p. 12076–12100. External Links: Link, Document Cited by: §3. Morini et al. (2025) V. Morini, M. Sansoni, G. Rossetti, D. Pedreschi, and C. Castillo Participant behavior and community response in online mental health communities: insights from reddit. Computers in Human Behavior 165, p. 108544. External Links: Document Cited by: §6. Naslund et al. (2020) J. A. Naslund, A. Bondre, J. Torous, and K. A. Aschbrenner Social media and mental health: benefits, risks, and opportunities for research and practice. Journal of Technology in Behavioral Science 5 (3), p. 245–257. External Links: ISSN 2366-5963, Document, Link Cited by: §1. Nguyen et al. (2025) V. C. Nguyen, M. Taher, D. Hong, V. K. Possobom, V. T. Gopalakrishnan, E. Raj, Z. Li, H. J. Soled, M. L. Birnbaum, S. Kumar, and M. De Choudhury Do large language models align with core mental health counseling competencies?. In Findings of the Association for Computational Linguistics: NAACL 2025, L. Chiruzzo, A. Ritter, and L. Wang (Eds.), Albuquerque, New Mexico, p. 7503–7526. External Links: Link, Document, ISBN 979-8-89176-195-7 Cited by: §6. of Mental Health (2024) N. I. of Mental Health Mental Illness — nimh.nih.gov. Note: https://w.nimh.nih.gov/health/statistics/mental-illness#part_2539[Accessed 22-05-2026] Cited by: §1. OpenAI (2025) OpenAI Introducing GPT-4.1 in the api. Note: Accessed: 2026-04-27 External Links: Link Cited by: §2.2. OpenAI (2026) OpenAI Introducing ChatGPT Health — openai.com. Note: https://openai.com/index/introducing-chatgpt-health/[Accessed 23-05-2026] Cited by: §1. Pargament et al. (2003) K. I. Pargament, B. J. Zinnbauer, A. B. Scott, E. M. Butter, J. Zerowin, and P. Stanik Red flags and religious coping: identifying some religious warning signs among people in crisis. Journal of Clinical Psychology 59 (12), p. 1335–1348. External Links: Document Cited by: §2.3. Park et al. (2024) P. S. Park, P. Schoenegger, and C. Zhu Diminished diversity-of-thought in a standard large language model. Behavior Research Methods 56 (6), p. 5754–5770. External Links: ISSN 1554-3528, Document, Link Cited by: §1. Rafailov et al. (2023) R. Rafailov, A. Sharma, E. Mitchell, S. Ermon, C. D. Manning, and C. Finn Direct preference optimization: your language model is secretly a reward model. In Proceedings of the 37th International Conference on Neural Information Processing Systems, NIPS ’23, Red Hook, NY, USA. Cited by: §2.3. Reddy (2026) S. Reddy Foundation for artificial intelligence-driven democratization of healthcare access. Learning Health Systems 10 (3), p. e70093. External Links: Document, Link, https://onlinelibrary.wiley.com/doi/pdf/10.1002/lrh2.70093 Cited by: §1. Revenson et al. (1991) T. A. Revenson, K. M. Schiaffino, S. D. Majerovitz, and A. Gibofsky Social support as a double-edged sword: the relation of positive and problematic support to depression among rheumatoid arthritis patients. Social Science & Medicine 33 (7), p. 807–813. External Links: Document Cited by: §2.3. Rozin (1999) P. Rozin The process of moralization. Psychological Science 10 (3), p. 218–221. External Links: Document Cited by: §2.3. Saha and Sharma (2020) K. Saha and A. Sharma Causal factors of effective psychosocial outcomes in online mental health communities. Proceedings of the International AAAI Conference on Web and Social Media 14 (1), p. 590–601. External Links: Link, Document Cited by: §6. SAMHSA (2025) SAMHSA Key substance use and mental health indicators in the united states: results from the 2024 national survey on drug use and health. Note: https://w.samhsa.gov/data/sites/default/files/reports/rpt56287/2024-nsduh-annual-national-report.pdf[Accessed 23-05-2026] Cited by: §1. Schröder et al. (2025) S. Schröder, T. Morgenroth, U. Kuhl, V. Vaquet, and B. Paaßen Large language models do not simulate human psychology. External Links: 2508.06950, Link Cited by: §1. Sharma et al. (2020) A. Sharma, A. Miner, D. Atkins, and T. Althoff A computational approach to understanding empathy expressed in text-based mental health support. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), B. Webber, T. Cohn, Y. He, and Y. Liu (Eds.), Online, p. 5263–5276. External Links: Link, Document Cited by: §6. Sharma and De Choudhury (2018) E. Sharma and M. De Choudhury Mental health support and its relationship to linguistic accommodation in online communities. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, CHI ’18, New York, NY, USA, p. 1–13. External Links: ISBN 9781450356206, Link, Document Cited by: §2.1, §6. Singhal et al. (2025) K. Singhal, T. Tu, J. Gottweis, R. Sayres, E. Wulczyn, M. Amin, L. Hou, K. Clark, S. R. Pfohl, H. Cole-Lewis, D. Neal, Q. M. Rashid, M. Schaekermann, A. Wang, D. Dash, J. H. Chen, N. H. Shah, S. Lachgar, P. A. Mansfield, S. Prakash, B. Green, E. Dominowska, B. Agüera y Arcas, N. Tomašev, Y. Liu, R. Wong, C. Semturs, S. S. Mahdavi, J. K. Barral, D. R. Webster, G. S. Corrado, Y. Matias, S. Azizi, A. Karthikesalingam, and V. Natarajan Toward expert-level medical question answering with large language models. Nature Medicine 31 (3), p. 943–950. External Links: ISSN 1546-170X, Document, Link Cited by: §1, §6. Thoits (2011) P. A. Thoits Mechanisms linking social ties and support to physical and mental health. Journal of health and social behavior 52 (2), p. 145–161. External Links: Document Cited by: §2.2. Vangelisti et al. (1990) A. L. Vangelisti, M. L. Knapp, and J. A. Daly Conversational narcissism. Communication Monographs 57 (4), p. 251–274. External Links: Document Cited by: §2.3. Vinokur and Van Ryn (1993) A. D. Vinokur and M. Van Ryn Social support and undermining in close relationships: their independent effects on the mental health of unemployed persons. Journal of Personality and Social Psychology 65 (2), p. 350–359. External Links: Document Cited by: §2.3. World Health Organization (2025) World Health Organization Mental disorders. Note: https://w.who.int/news-room/fact-sheets/detail/mental-disordersAccessed: 2026-04-23 Cited by: Table 5. Yang et al. (2025) A. Yang, A. Li, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Gao, C. Huang, C. Lv, C. Zheng, D. Liu, F. Zhou, F. Huang, F. Hu, H. Ge, H. Wei, H. Lin, J. Tang, J. Yang, J. Tu, J. Zhang, J. Yang, J. Yang, J. Zhou, J. Zhou, J. Lin, K. Dang, K. Bao, K. Yang, L. Yu, L. Deng, M. Li, M. Xue, M. Li, P. Zhang, P. Wang, Q. Zhu, R. Men, R. Gao, S. Liu, S. Luo, T. Li, T. Tang, W. Yin, X. Ren, X. Wang, X. Zhang, X. Ren, Y. Fan, Y. Su, Y. Zhang, Y. Zhang, Y. Wan, Y. Liu, Z. Wang, Z. Cui, Z. Zhang, Z. Zhou, and Z. Qiu Qwen3 technical report. External Links: 2505.09388, Link Cited by: Appendix A, §3, §3. Yang et al. (2023) K. Yang, S. Ji, T. Zhang, Q. Xie, Z. Kuang, and S. Ananiadou Towards interpretable mental health analysis with large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, H. Bouamor, J. Pino, and K. Bali (Eds.), Singapore, p. 6056–6077. External Links: Link, Document Cited by: §1, §6. Zhang et al. (2025) C. Zhang, X. Shi, X. Zhang, Y. Zhu, Y. Yang, and Y. Luo DecoupledESC: enhancing emotional support generation via strategy-response decoupled preference optimization. In Findings of the Association for Computational Linguistics: EMNLP 2025, C. Christodoulopoulos, T. Chakraborty, C. Rose, and V. Peng (Eds.), Suzhou, China, p. 22189–22215. External Links: Link, Document, ISBN 979-8-89176-335-7 Cited by: §1, §6. Zielinski and Veilleux (2018) M. J. Zielinski and J. C. Veilleux The perceived invalidation of emotion scale (pies): development and psychometric properties of a novel measure of current emotion invalidation. Psychological Assessment 30 (11), p. 1454–1467. External Links: Document Cited by: §2.3. Subreddits r/abuse, r/adultsurvivors, r/aftersilence, r/Anger, r/bullying, r/CPTSD, r/domesticviolence, r/emotionalabuse, r/ptsd, r/PTSDCombat, r/rapecounseling, r/StopSelfHarm, r/survivorsofabuse, r/traumatoolbox, r/Agoraphobia, r/Anxiety, r/BipolarReddit, r/BipolarSOs, r/BPD, r/dpdr, r/psychoticreddit, r/MaladaptiveDreaming, r/Psychosis, r/PanicParty, r/schizophrenia, r/socialanxiety, r/compulsions, r/CompulsiveSkinPicking, r/OCD, r/Trichsters, r/CupsOfTea, r/BackOnYourFeet, r/Existential_crisis, r/getting_over_it, r/GriefSupport, r/helpmecope, r/hardshipmates, r/HereToHelp, r/iflostbetter, r/LostALovedOne, r/offmychest, r/MMFB, r/Miscarriage, r/reasonstolive, r/SuicideBereavement, r/therapy, r/depression, r/depressed, r/ForeverAlone, r/GFD, r/lonely, r/mentalhealth, r/Radical_Mental_Health, r/SuicideWatch Table 4: List of 55 mental health–related subreddits used in this study. Keyword Type Keyword List Mental Disorder Terms Anxiety disorder, Generalised anxiety disorder, Panic disorder, Social anxiety disorder, Separation anxiety disorder, Depression, Depressive episode, Bipolar disorder, Manic episode, Post-Traumatic Stress Disorder, PTSD, Schizophrenia, Delusions, Hallucinations, Disorganised thinking, Disorganised behaviour, Eating disorder, Anorexia nervosa, Bulimia nervosa, Disruptive behaviour disorder, Dissocial disorder, Conduct disorder, Oppositional defiant disorder, ODD, Neurodevelopmental disorder, Autism spectrum disorder, ASD, Attention-deficit/hyperactivity disorder, ADHD, Disorders of intellectual development, Intellectual disability Topic Keywords advice, today, need, therapist, end, never, daydream, anything, birthday, read, ocd, hate, job, question, deal, think, new, looking, interview, diagnosed, feeling, normal, tell, therapy, guy, head, thought, suicidal, better, anxiety, social, intrusive, life, symptom, tired, relationship, worst, physical, friend, scared, depressed, kill, best, love, disorder, losing, done, lonely, one, talk, take, could, wish, person, go, care, depression, bad, attack, panic, this, heart, due, weird, cause, anybody, call, work, anxious, getting, alone, wrong, school, family, back, always, mental, health, stop, fuck, even, illness, sure, compulsion, much, time, day, year, first, every, daydreaming, right, sleep, week Table 5: Unified keyword set used for filtering and analysis. Keyword filtering is widely used in mental health dataset construction to improve precision Cohan et al. (2018). Our keyword list combines (1) mental disorder terminology derived from public health definitions World Health Organization (2025) and (2) support-seeking expressions observed in online discourse Kim et al. (2023). We further expand the vocabulary with lexical variations (e.g., PTSD, anxiety, depress). Type Prompt System Prompt You are a clinical psychologist specializing in digital mental health and social support systems. Your task is to identify whether a Reddit post expresses a request for non-medical peer support. Definition: Within non-medical peer support, people who share common experiences or face similar challenges come together as equals to give and receive help based on the knowledge that comes through shared experience. It focuses on validation, normalization, and shared coping rather than technical solutions or clinical diagnosis. You will be given a Reddit post text (REDDIT_POST_TEXT) and Reddit post title (REDDIT_POST_TITLE) and your task is to determine whether the post is requesting non-medical, emotional peer support. Some examples are: • Seeking emotional encouragement or reassurance. • Asking for shared experiences to overcome a problem. • Asking for connecting to people, resources, needed services, and community. • Asking for steps/help for overcoming difficult situations. • Turning to the community for finding personal growth, purpose, or a new perspective in the situation. Do NOT classify as peer support if the post is: • A technical or troubleshooting question • A medical or disease related query • A venting or reflection without inviting interaction/reflection • A game strategy or product-related coordination Examples: Example 1: Post: “I’ve been feeling numb ever since my mom passed. How do you even begin to move on?” Label: YES Reasoning: The user asks for advice on how to cope with grief—clearly seeking emotional guidance. Example 2: Post: “Can you turn autosaves off? I accidentally ruined my build.” Label: NO Reasoning: This is a technical question about game mechanics, not a request for emotional support. Example 3: Post: “Does anyone else feel like they’re just pretending to be okay all the time?” Label: YES Reasoning: The user is asking for shared experiences to feel less alone, satisfying the peer support criteria. Example 4: Post: “Just hit 10k steps today. Feeling good!” Label: NO Reasoning: This is a self-reflective statement celebrating an achievement, not seeking support. Example 5: Post: “I always binge uncontrollably during my period. I beat myself up and get depressed. Anyone else? What helps you stop?” Label: YES Reasoning: The post contains both a request for shared experiences and a question about coping strategies. Example 6: Post: “I’m looking for anyone available to help sweep through the rest of my storm shield defenses.” Label: NO Reasoning: This is a game coordination request, not an emotional or social support need. Instructions: You must respond strictly using this JSON format: "reasoning": "your reasoning (1--2 sentences)", "answer": "YES" or "NO" Do not include any explanation outside of the JSON block. User Prompt REDDIT_POST_TITLE: <post title> REDDIT_POST_TEXT: <post text> Classify whether the post is requesting non-medical emotional peer support. Return the output strictly in JSON format as specified above. Table 6: Prompt used for binary peer-support classification. Type Prompt System Prompt You are a clinical psychologist identifying how coping strategies fall under four categories: Problem-focused, Emotion-focused, Meaning-focused, and Social coping. You will be given: • Reddit post text (REDDIT_POST_TEXT) • Four coping strategies with names, definitions, and examples. Your task: Step 1. Carefully read the text and decide if the text is seeking support of any kind. If it is not, skip to step 3 and set all binary classification to zero. Step 2. Ignore any text associated with an explicit “edit” or “thanks” portion of the text. Step 3. With the remaining text, determine coping strategies the user employs in the text. Only categorize if it is truly a good fit. Consider actions, tone, and behaviors. Step 4. Return a SINGLE JSON object in this exact format: "is_problem_focused": 0 or 1, "problem_focused_reasoning": "short reasoning for Problem-focused classification", "is_emotion_focused": 0 or 1, "emotion_focused_reasoning": "short reasoning for Emotion-focused classification", "is_meaning_focused": 0 or 1, "meaning_focused_reasoning": "short reasoning for Meaning-focused classification", "is_social_focused": 0 or 1, "social_focused_reasoning": "short reasoning for Social coping classification" Do NOT include any text outside the JSON block. User Prompt — Coping Strategy 1 — COPING_STRATEGY_NAME: Problem-focused COPING_STRATEGY_DEFINITION: Taking positive, practical action in order to change the stressful situation itself. EXAMPLES: - “I made a list of things I can control about my job search and started applying to two places a day.” - “I’m still not sure what to do but I started seeing a therapist last week” - “I was panicking about the deadline, so I emailed my professor to ask for an extension.” - “Setting boundaries” - “Clear communication about problems” — Coping Strategy 2 — COPING_STRATEGY_NAME: Emotion-focused COPING_STRATEGY_DEFINITION: Doing self-soothing behaviors to reduce distress associated with the problem. EXAMPLES: - “I put on music and cleaned my room to feel less overwhelmed.” - “I wrote out everything I was feeling in my notes app instead of bottling it up.” - “I binge-watched a comfort show to distract myself for a while.” - “Venting about the situation” - “I went for a long walk after the argument just to cool off.” — Coping Strategy 3 — COPING_STRATEGY_NAME: Meaning-focused COPING_STRATEGY_DEFINITION: Finding personal growth and perspective in the situation, changing internally rather than externally. EXAMPLES: - “Focus on the positive” - “Challenge previously held beliefs that are no longer adaptive” - “I’m going to accept what I can’t change” - “I’m focusing on what I still have instead of just what I lost.” - “I realized losing that job pushed me to finally go after something I actually care about.” — Coping Strategy 4 — COPING_STRATEGY_NAME: Social coping COPING_STRATEGY_DEFINITION: Explicitly asking for advice or mentions looking to others for help. Simply posting on Reddit is not sufficient to be social coping. EXAMPLES: - “My teacher asked me if anything was going on, so I spent some time telling her about my frustrations.” - “Do you have any tips on how to handle stress at work?” - “I went out with my friends to feel better.” - “I posted about it in a support subreddit and got some really good advice.” REDDIT_POST_TEXT: <post text> Table 7: Prompt used for coping strategy classification with combined definition-based labels. Type Prompt System Prompt You are a Reddit user who provides helpful, empathetic and actionable advice by replying to a post. Given the post title (POST_TITLE), post text (POST_TEXT), and the comments made by other users (REDDIT_COMMENTS), your task is to synthesize the best advice/perspectives from the comments into a single coherent response. Data Source Rules: • Prioritize high-value content: Focus on good and actionable ideas from all comments especially those sharing their lived experiences. • Strict sourcing: Do NOT invent advice. Every suggestion must be traceable to one of the comments. • Hidden context: You must NEVER reveal you are summarizing a thread. • BANNED PHRASES: “commenters”, “others said”, “people here”, “in this thread”, “the consensus is”. • Instead, own the knowledge: “It seems like…”, “A common trick is…”, “Have you tried…”, or “I’ve found that…” (if framing a comment’s experience as general knowledge). • You are allowed to used the word “I” in your response. You are allowed to use anthropomorphic language. Variability & Style Protocols (Critical): • DYNAMIC OPENERS (Prevent Repetition): – Use the following dynamic opener: <dynamic_opener> – Do not use the exact example, but follow the intent and style of the dynamic opener category. • AVOID LLM-SPEAK: – Using Emojis: – Using hashtags or em-dash Natural Attribution: • Do not always use “I heard from someone…”. Mix it up: – Direct suggestion: “Maybe try X…” – General wisdom: “It’s surprisingly common to feel X…” – Casual observation: “It makes sense that X would happen because…” Tone: • Casual, grounded, and specific. • Be empathetic and understanding. • Avoid “therapy speak” (e.g., “valid”, “healing journey”, “hold space”). • Be brief. 200-300 words max unless the topic is complex. Structural Variance: • Do not always follow the formula [Validate → Story → Advice → Close]. • Sometimes give the advice first. • Sometimes just share the perspective without a concrete “step”. • End abruptly sometimes, like a real text/comment (e.g., “Hope that helps,” or just stopping after the advice). Recheck Before Final Output Generation: • You have incorporated the style and intent of the dynamic opener into your response. • You have incorporated all good and actionable advice from all the comments. • You have not invented any advice. Every suggestion must be traceable to one of the comments. • The answer has an empathetic tone. User Prompt REDDIT_POST_TITLE: <post_title> REDDIT_POST_TEXT: <post_text> REDDIT_COMMENTS: REDDIT_COMMENT_1: <comment_1> REDDIT_COMMENT_2: <comment_2> … Return the synthesized advice/perspectives from the comments into a single coherent response. Return the response as a string. Table 8: Prompt used for synthesizing community-grounded preferred responses (ground-truth). Negative Strategy Definition Example Emotion Invalidation Responses that dismiss, negate, or judge the user’s emotional experience as incorrect or inappropriate. “You’re overreacting. It’s not that serious.” Problematic Control (Overdirective Support) Attempts to control or direct the user’s coping behavior rather than supporting autonomy. “Just do this and stop thinking about it.” Incongruent / Misguided Positivity Positive responses that fail to acknowledge distress and instead replace it with superficial reassurance. “Everything happens for a reason, just stay positive.” Social Undermining Responses that criticize, discourage, or obstruct the user’s coping efforts. “This is your fault. You should’ve known better.” Unsolicited Advice Advice given without acknowledging the user’s needs or emotional state, often perceived as intrusive. “You should just quit your job and move on.” Emotional Displacement (Self-Focus) Shifting the conversation away from the user’s distress toward the responder’s own experiences. “That reminds me of when I went through something way worse…” Moralization Framing the user’s feelings or situation as morally wrong or unacceptable. “You shouldn’t feel this way—it’s selfish.” Autonomy Invalidation Implying the user lacks competence or ability to handle their situation. “You clearly don’t know what you’re doing.” Spiritual Bypassing Using spiritual or philosophical beliefs to dismiss or avoid emotional processing. “This happened because the universe is teaching you a lesson.” Trivialization / Minimization Downplaying the severity of the user’s experience or distress. “Other people have it way worse than you.” Harmful Language / Stereotyping Use of stigmatizing, generalized, or harmful language toward the user. “People like you are always like this.” Table 9: Taxonomy of negative support strategies used for generating and classifying harmful responses. Type Prompt System Prompt You are a Reddit user who provides unhelpful, illogical, and non-actionable advice/answers. Given the post title (POST_TITLE), post text (POST_TEXT), the comments made by other users (REDDIT_COMMENTS), negative category name (NEGATIVE_CATEGORY_NAME), negative category definition (NEGATIVE_CATEGORY_DEFINITION), and negative strategies examples (NEGATIVE_STRATEGIES_EXAMPLES), your task is to synthesize the best unhelpful and non-actionable advice. Below are the rules for the response: STEP 1: CHOOSE THE BEST NEGATIVE STRATEGY • Keeping the context of the reddit post title (POST_TITLE), post text (POST_TEXT), the negative category name (NEGATIVE_CATEGORY_NAME), and the negative category definition (NEGATIVE_CATEGORY_DEFINITION), choose the best negative strategy from the list of negative strategies (NEGATIVE_STRATEGIES_EXAMPLES). • Here the best negative strategy is the one that would provide the most unhelpful and non-actionable advice. • This negative strategy is the one you will use to generate the negative answer (CHOSEN_NEGATIVE_STRATEGY). Additionally, you will provide a short reasoning for choosing the best negative strategy (CHOSEN_NEGATIVE_STRATEGY_REASONING). STEP 2: GENERATE THE NEGATIVE ANSWER • Generate the negative answer (NEGATIVE_ANSWER) using the chosen negative strategy (CHOSEN_NEGATIVE_STRATEGY) while keeping the post title (POST_TITLE), post text (POST_TEXT) in context. • The negative answer should be unhelpful and non-actionable. • The answer should provide at least three unhelpful and non-actionable suggestions. STEP 3: REVIEW THE NEGATIVE ANSWER • Review the negative answer (NEGATIVE_ANSWER) and make sure it is unhelpful and non-actionable. • Make sure it is not similar to any of the comments (REDDIT_COMMENTS). GENERAL ANSWER STYLE GUIDELINES VARIABLE OPENER • Use the following answer opener: <dynamic_opener> • Do not include the category name or examples in the answer. • Do not use the exact same examples as the ones provided in the negative strategies. Change the examples to make it more unique. VARIABILITY & STYLE PROTOCOLS (CRITICAL) • AVOID LLM-SPEAK: – Using Emojis: – Using hashtags or em-dash – Avoid using cliche words like “honestly”, “to be honest”, “frankly”, etc. TONE: • Casual, and specific. • Answer should be around 250 words. User Prompt REDDIT_POST_TITLE: <post_title> REDDIT_POST_TEXT: <post_text> REDDIT_COMMENTS: REDDIT_COMMENT_1: <comment_1> REDDIT_COMMENT_2: <comment_2> … NEGATIVE_CATEGORY_NAME: <category_name> NEGATIVE_CATEGORY_DEFINITION: <category_definition> NEGATIVE_STRATEGIES_EXAMPLES: <list_of_strategies> Generate the negative answer following the instructions above. Table 10: Prompt used for generating rejected responses from negative support strategies. Type Prompt System Prompt You are a supportive Reddit user who provides empathetic and actionable advice. Given a Reddit post title (REDDIT_POST_TITLE) and reddit post text (REDDIT_POST_TEXT), write a helpful response that: 1. Shows empathy: Acknowledge the poster’s feelings and validate their experience (e.g., “I’m sorry you’re going through this”, “That sounds really tough”). 2. Is actionable: Offer concrete, practical and actionable suggestions within your response. 3. Uses a warm, casual tone: Use a warm, casual tone like a supportive person. Do not use therapy-speak or jargon language. 4. Stays brief: Generate a response between 200-300 words. 5. Avoids: Avoid using emojis, hashtags, “I’m not a doctor”, generic platitudes, and other non-human like language. 6. Anthropomorphizes: You are allowed to use anthropomorphic language to answer the post. Write your response as if you are directly replying to the post. Write your response as if you’re directly replying to the post. User Prompt REDDIT POST TITLE: <post_title> REDDIT POST TEXT: <post_text> Write an empathetic and actionable response to this post. Table 11: Prompt used for generating empathetic and actionable Reddit responses. Type Prompt System Prompt An atomic strategy/suggestion is defined as a self-contained piece of advice/suggestion that can either be helpful or harmful to the person who is reading it. Instructions: - You are given a RESPONSE from a reddit user. Your task is to extract a list of atomic support strategies from the RESPONSE. - An atomic support strategy/suggestion must contain an action verb and provide a single piece of advice. - An atomic support strategy/suggestion should be extracted from a statement in the RESPONSE and not from a question. - Each atomic support strategy/suggestion should carry an entirely different piece of advice, and should be independent of other atomic support strategies/suggestions in the list. - As mentioned before, the atomic strategies/suggestions can be either helpful or harmful. You should only output the atomic support strategies as a list. Do not include any introductory text, concluding remarks, or other formatting. User Prompt RESPONSE: <sample user post> Extract the atomic strategies from the answer. Return the list of atomic strategies as a list of strings. For example: <sample list of atomic strategies> FORMAT: - Always return the list of strategies. The list should start with a [ and end with a ]. - Have one strategy per line. WARNING: - Do not include any introductory text, concluding remarks, or other formatting. - Do not include any other text in the response. - Do not include special characters like “-”, “*”, “|”, etc. in the response. Table 12: Prompt used for extracting atomic support strategies from generated responses. Type Prompt System Prompt You are a peer trying to provide general coping approaches based on a list of provided strategies. Instructions: Combine strategies by logically grouping ones that are similar based on the following groupings: - Problem-focused: going to therapy, trying medication, seeking interventions and external changes - Social-focused: reaching out to friends, family, and peers - Meaning-focused: self-reflection, meditation, changing perspectives, seeking internal change - Emotion-focused: self-soothing behaviors WARNING: - Use your best discretion to decide when strategies should be categorized more granularly based on factors such as the kind of actors and actions involved. - Do NOT alter the wording of any of the harm reduction strategies, only group them as multiple sentences in a single combined strategy. - Output the resulting strategies strictly as a JSON object with no other formatting. Here is the format of the JSON object: <JSON object displaying formatting for response> Consider the following examples: INPUT_LIST: <sample list of atomic strategies> OUTPUT: <sample list of combined atomic strategies> User Prompt INPUT_LIST: <list of atomic strategies> OUTPUT: Table 13: Prompt used for combining atomic support strategies into broader strategy groups. Atomic coping strategies Combined coping strategies 1. Keep a journal where you write down your thoughts and feelings. 2. Write down patterns or triggers that you notice in your journal. 3. Seek a therapist who specializes in anxiety and OCD. 4. Share your experiences with friends who understand. 5. Connect with family members for emotional support. 6. Join a support group to talk with others who have similar experiences. 7. Ask for help in finding the right tools to manage your feelings. 1. Keep a journal where you write down your thoughts and feelings. Write down patterns or triggers that you notice in your journal. 2. Seek a therapist who specializes in anxiety and OCD. 3. Share your experiences with friends who understand. Connect with family members for emotional support. Join a support group to talk with others who have similar experiences. Ask for help in finding the right tools to manage your feelings. 1. Look into pet sitters or a local pet care service for the week. 2. Ask a vet for advice on managing medication and care for your special-needs foster kitten during your absence. 3. Ask for help when needed. 4. Take care of yourself as necessary. 1. Look into pet sitters or a local pet care service for the week. 2. Ask a vet for advice on managing medication and care for your special-needs foster kitten during your absence. 3. Take care of yourself and ask for help when needed. Table 14: Examples of how identified coping strategies were combined into groups by GPT-4.1 based on approach to coping and actors involved. Figure 4: HDPsim_HDP % relative change vs. baseline across subreddit community categories for Gemma-4B and Qwen-3.4B. Appendix A Model Details and Hyperparameter Settings All three models (Qwen3-4B Instruct (4 Billion parameters), Gemma-4-E4B (4.5 Billion effective parameters), MediPhi-4B Instruct (3.8 Billion parameters) were post-trained through an identical two-stage pipeline and hyper-parameters. The first step involved SFT, followed by DPO (as described in Section 3). For post-training we used NVIDIA H200 GPUs using transformers, trl (SFTTrainer/DPOTrainer) and peft (LoRA). Table 15, and 16 present the hyper-parameters used for SFT and DPO post-training. Hyperparameter Value Optimisation Gradient clipping (max grad norm) 1.0 LR schedule Linear decay with warmup Warmup ratio 0.10 Peak learning rate 2×10−52×10^-5 Number of epochs 3 Per-device train batch size 2 Gradient accumulation steps 4 GPUs 2 Effective batch size 16 Max sequence length 2048 tokens Precision bf16 mixed precision Gradient checkpointing Enabled LoRA adapter Rank r 64 α 128 Dropout 0.05 Target modules q,k,v,o,gate,up,down_proj\q,k,v,o,gate,up,down\\_proj Task type Causal LM Loss masking Loss target Completion-only (TRL auto-masking) Table 15: Supervised fine-tuning (SFT) hyperparameters, identical across Qwen3-4B, Gemma-4-E4B and MediPhi-Instruct. Hyperparameter Value Starting checkpoint Merged SFT checkpoint Reference model Same SFT checkpoint (frozen) Weight decay 0.0 Gradient clipping 1.0 LR schedule Linear with warmup Warmup ratio 0.10 Peak learning rate 2×10−62×10^-6 Number of epochs 3 Per-device train batch size 2 Gradient accumulation steps 4 Effective batch size 16 Max sequence length 2048 tokens Precision bf16 mixed precision Gradient checkpointing Enabled DPO β (KL temperature) 0.10 Label smoothing (cDPO) 0.10 LoRA (r,α,dropout)(r,α,dropout) (64,128,0.10)(64,128,0.10) LoRA target modules Same as SFT (Table 15) Evaluation strategy every 10 steps Save strategy every 10 steps (keep best 3) Best-model criterion lowest eval_loss Early stopping (patience, Δ ) 55, 10−410^-4 on eval_loss Table 16: Direct Preference Optimization (DPO) hyperparameters. DPO starts from the merged SFT checkpoint produced by the configuration in Table 15. Response generation at evaluation time: For each model (Zero-Shot, SFT, SFT+DPO), we generated a response using greedy decoding and max new token=800. Atomic strategy extraction and combination: Atomic strategies were extracted and then combined into the four coping groups using Qwen3-30B-A3B-Instruct-2507 model Yang et al. (2025) with temperature=0.0temperature=0.0, top_p=1.0top\_p=1.0, max_new_tokens=2048max\_new\_tokens=2048. Hyper-parameters for other libraries: We used ‘en_core_web_sm‘ model from spacy for emotion and & tone alignment evaluation. Appendix B Human Evaluation Task Results B.1 Human Evaluation of Peer-Support Labels We conducted a manual review to assess the quality of the LLM-generated peer-support labels. Annotators reviewed 100 Reddit posts and compared the LLM-generated peer-support label with their own judgment. We report human–LLM agreement rather than inter-annotator agreement, since annotators reviewed separate subsets of posts. Overall, 9 out of 100 reviewed posts showed a human–LLM disagreement, corresponding to a 91.0% agreement rate. Annotator Posts Reviewed Disagreements Disagreement Rate Annotator A 34 4 11.8% Annotator B 33 3 9.1% Annotator C 33 2 6.1% Total 100 9 9.0% Table 17: Manual review results for the peer-support classification task. Disagreement is reported at the post level. Annotator names are anonymized. B.2 Human Evaluation of Coping Strategy Labels We additionally compared the LLM-generated coping strategy labels against human annotations. To reduce potential bias, annotators reviewed posts that they had not labeled in the previous annotation round. Because each post can receive multiple coping strategy labels, we report both post-level disagreement and category-level label changes. Overall, 9 out of 100 samples showed at least one human–LLM disagreement. Since some samples involved more than one coping strategy label change, these 9 samples correspond to 11 category-level label changes. Appendix C Information About Use of AI Assistants We used AI assistants for text rephrasing and coding related tasks. For text rephrasing, the usage was limited to correcting grammatical mistakes and choice of words. For coding related task, AI assistants were used to improve human-written code, finding and fixing bugs and README generation. Figure 5: Distribution of generated coping strategy types across subreddit community categories for all three models under zero-shot, SFT, and SFT+DPO configurations. Figure 6: Relative changes in Strategy Alignment, Strategy Congruence, and Emotion & Tone metrics for Qwen-3 4B, Gemma 4 E4B, and MediPhi-Instruct following SFT and SFT+DPO, compared to zero-shot baselines across Reddit post coping strategy labels.