Paper deep dive
ValueGraph: Value-Signal Guided Graph Pre-training for Contextualized User Representation
Yitong Han, Wei Gao, Yi Zhao, Prasanta Bhattacharya, Fengzhu Zeng, Mohammad Amanlou
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 89%
Last extracted: 9/2/2026, 3:53:29 AM
Summary
The paper introduces ValueGraph, a graph pre-training framework for contextualized user representation that incorporates automatically inferred moral-value signals as noisy auxiliary supervision. Using MoralBERT to derive 10-dimensional value profiles from user discourse, ValueGraph employs masked graph autoencoding on post-reply graphs followed by value-guided contrastive learning and clustering objectives. Experiments on stance detection and Twitter bot detection demonstrate consistent performance gains over text-based, graph-based, and LLM baselines, validating value-signal guidance as an effective inductive bias for socially informed user modeling.
Entities (11)
Relation Signals (10)
ValueGraph → evaluatedon → Twitter Bot Detection
confidence 95% · Experiments on stance detection and twitter bot detection show consistent gains
ValueGraph → evaluatedon → Stance Detection
confidence 95% · Experiments on stance detection and twitter bot detection show consistent gains
ValueGraph → uses → MoralBERT
confidence 95% · we derive 10-dimensional user-level moral feature vectors by aggregating post-level moral scores output from MoralBERT
Moral Foundations Theory → basisfor → ValueGraph
confidence 90% · we adopt Moral Foundations Theory (MFT) (Haidt et al. 2007) as our theoretical foundation.
ValueGraph → trainedon → Reddit
confidence 90% · We construct a large-scale pre-training corpus from social media conversations collected from Reddit and Twitter.
ValueGraph → trainedon → Twitter
confidence 90% · We construct a large-scale pre-training corpus from social media conversations collected from Reddit and Twitter.
ValueGraph → uses → ModernBERT
confidence 90% · Each post is encoded using ModernBERT... chosen for robustness to noisy social media text.
ValueGraph → uses → GraphMAE2
confidence 90% · Pre-training follows the masked autoencoding paradigm of GraphMAE2
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Value signals are aggregated user-level moral representations that capture users' inferred value-related tendencies from their online discourse. User behavior on social media is shaped not only by what users say or whom they interact with, but also by the value signal through which they express attitudes. Existing user representation methods largely miss this value-relevant dimension. We propose ValueGraph, a graph pre-training framework that uses automatically inferred moral-value signals as noisy auxiliary signals for contextualized user representation. From post-reply graphs, ValueGraph learns semantic and structural representations and further aligns users through relative value similarity with contrastive and clustering objectives. Rather than treating inferred values as gold psychological labels, ValueGraph uses them as soft constraints for representation learning. Experiments on stance detection and twitter bot detection show consistent gains over strong text-based, graph-based, and text-only LLM baselines, highlighting value-signal guidance as a useful inductive bias for socially informed user modeling.
Tags
Links
- Source: https://arxiv.org/abs/2609.00057v1
- Canonical: https://arxiv.org/abs/2609.00057v1
Trouble viewing inline? Open PDF directly →
Full Text
73,978 characters extracted from source content.
Expand or collapse full text
ValueGraph: Value-Signal Guided Graph Pre-training for Contextualized User Representation Yitong Han Wei Gao Yi Zhao Prasanta Bhattacharya Fengzhu Zeng Mohammad Amanlou Abstract Value signals are aggregated user-level moral representations that capture users’ inferred value-related tendencies from their online discourse. User behavior on social media is shaped not only by what users say or whom they interact with, but also by the value signal through which they express attitudes. Existing user representation methods largely miss this value-relevant dimension. We propose ValueGraph, a graph pre-training framework that uses automatically inferred moral-value signals as noisy auxiliary signals for contextualized user representation. From post-reply graphs, ValueGraph learns semantic and structural representations and further aligns users through relative value similarity with contrastive and clustering objectives. Rather than treating inferred values as gold psychological labels, ValueGraph uses them as soft constraints for representation learning. Experiments on stance detection and twitter bot detection show consistent gains over strong text-based, graph-based, and text-only LLM baselines, highlighting value-signal guidance as a useful inductive bias for socially informed user modeling.11 1 Code is released at https://github.com/HanYiton/ValueGraph 1School of Computing and Information Systems, Singapore Management University, Singapore 2Institute of Advanced Intelligence and Computing, A*STAR, Singapore ythan,weigao,yizhao,fzzeng,mohammada@smu.edu.sg, prasanta_bhattacharya@a-star.edu.sg Introduction Understanding user behavior is central to social media analytics, recommendation systems, and personalized AI. User embedding models map user content and interactions to dense representations for downstream tasks such as profiling, recommendation, stance detection, and social influence analysis, and must generalize to unseen users and evolving social contexts (Pan and Ding 2019). Most existing methods learn user representations from text (Benton et al. 2016; Ding et al. 2017), visual content (Do et al. 2018), or network structure (Rahimi et al. 2015; Pick et al. 2022), and optimize them with self-supervised or graph-based objectives (Perozzi et al. 2014; Sun et al. 2020b; Grover and Leskovec 2016; Donnat et al. 2018). These objectives often assume that users with similar content exposure or graph neighborhoods should have similar embeddings. However, similarity in interaction does not necessarily indicate similarity in the underlying factors that drive user behaviors. Users may engage with the same event because of shared exposure, controversy, or platform dynamics while expressing substantially different motivations and attitudes. When social contexts shift, such surface-level similarity can become brittle (Zhao et al. 2021; Hafidi et al. 2020; Hassani and Khasahmadi 2020; Sun et al. 2020a). For example, news recommendation systems that rely on interaction-based similarity may mistake shared exposure or controversy-driven interactions for shared preferences or values. Users who engage with the same event may hold opposing attitudes, but such spurious behavioral associations can cause the system to repeatedly recommend similar content to these users, reinforcing homogeneous information exposure and potentially amplifying polarization or extremist dynamics (Ribeiro et al. 2020). These limitations suggest that effective user representations should capture not only observable behaviors, such as what users say or where they interact, but also latent factors underlying these behaviors. To address these challenges, we adopt Moral Foundations Theory (MFT) (Haidt et al. 2007) as our theoretical foundation. MFT provides a framework for understanding moral judgments reflected in social discourse and defines interpretable moral dimensions that have been widely studied in computational social science (Graham et al. 2013). Building upon recent advances in text-based moral value prediction, we derive 10-dimensional user-level moral feature vectors by aggregating post-level moral scores output from MoralBERT (Preniqi et al. 2024), and refer to these vectors as value signals. Without explicitly modeling such value signals, conventional embedding models may struggle to distinguish users who exhibit similar interaction patterns but hold different stances and behavioral tendencies. This motivates us to leverage value signals as an auxiliary supervisory cue to learn more robust user representations. We therefore propose ValueGraph, a contextualized user representation framework that incorporates inferred value signals to regularize graph pre-training. In this work, contextualized refers to representations learned by jointly modeling users’ textual content and their interaction context in conversation graphs. Concretely, ValueGraph operates on a post-reply graph and learns user representations in two stages. First, it uses masked graph autoencoding to learn semantic and structural post representations from conversation graphs. Second, it aggregates post embeddings into user embeddings and constructs positive and negative user pairs according to inferred value similarity, which is derived from psychology and computational ethics literature (Preniqi et al. 2024; Nguyen et al. 2024; Guo et al. 2023). A user-level contrastive loss aligns users with similar inferred value profiles, while a clustering loss encourages coherent and separated regions in the embedding space. These objectives guide the encoder to represent what users engage with alongside the value signals associated with their behavior. We evaluate ValueGraph on stance detection and Twitter bot detection. Results show consistent gains over graph pre-training methods, Pre-trained Language Model (PLM) encoders, inductive GNNs, and text-only LLM baselines, while ablations confirm that the improvements come from jointly modeling semantic, structural, and value-signal information. Our main contributions are summarized as follows: • We propose ValueGraph, a value-signal guided graph pre-training framework that leverages inferred value signals as noisy auxiliary supervision for contextualized user representation learning. • We design a hierarchical objective that combines masked graph autoencoding with contrastive learning and clustering to jointly capture semantic, structural, and value information. • We provide a mechanism-level analysis of the proposed objectives, characterizing value-signal similarity preservation and cluster compactness/separation in the learned embedding space. • Experiments show that ValueGraph achieves the best performance among compared user representation methods on both stance detection and bot detection tasks, validating the effectiveness of incorporating value signals into graph-based user representation learning. Related Work User Representation. Social media user representations map high-dimensional user features into dense embeddings (Pan and Ding 2019). Prior work learns such representations from text (Benton et al. 2016; Ding et al. 2017), visual content (Do et al. 2018), network structure (Rahimi et al. 2015), or multi-view fusion (Zhang et al. 2017; Zhang et al. 2018; Ribeiro et al. 2018). These methods are effective, but rarely use value-relevant moral signals as explicit inductive bias. Graph Pre-training. GNN pre-training improves representation generalization by exploiting graph structure and neighborhood similarity (Perozzi et al. 2014; Sun et al. 2020b; Grover and Leskovec 2016; Donnat et al. 2018; Zhang et al. 2019; Tang et al. 2015; Zhao et al. 2021). Later work explores similar-domain and cross-domain pre-training to improve transferability (Hafidi et al. 2020; Hassani and Khasahmadi 2020; Sun et al. 2020a; Zhu et al. 2020; Hu et al. 2020a; You et al. 2020; Hu et al. 2020b; Lu et al. 2021; Qiu et al. 2020). However, existing graph pre-training methods rarely model value signal or motivational dimensions in user behavior. AI and Human Values. Recent work detects moral-values in text using transformer-based models (Preniqi et al. 2024; Nguyen et al. 2024; Guo et al. 2023; Zangari et al. 2025), while value-alignment research focuses on instruction following and preference-based reward modeling (Ouyang et al. 2022; Christiano et al. 2017). Instead of focusing on text-level moral-value detection or language-model alignment, we use automatically inferred value signals as auxiliary signals for graph-based user representation learning. Problem Formulation Figure 1: The framework of ValueGraph. In Stage 1, we construct a post-reply graph and perform GraphMAE2-based unsupervised pre-training to learn contextualized post representations by jointly modeling semantic and structural information. The resulting post embeddings are aggregated into user representations. In Stage 2, we introduce moral value signals extracted by MoralBERT and optimize user representations through value-guided contrastive learning and clustering objectives. The learned user representations are then used for downstream fine-tuning on stance detection and Twitter bot detection. We are given a dataset =ii=1nG=\G_i\_i=1^n of social conversation graphs. Each graph i=(i,ℰi)G_i=(V_i,E_i) represents a conversation thread, where iV_i is the set of posts and ℰiE_i is the set of directed reply edges between posts22 2 We focus on post-reply relations, since follow/friendship relations are often platform-dependent and unavailable across datasets. This design enables a more general evaluation of value-signal guided graph pre-training.. Let U denote all users who authored posts in G. Since each post is authored by exactly one user, we define an authorship mapping a:i→a:V_i , where u=a(v)u=a(v) indicates that post v is authored by user u. Let =⋃i=1niV= _i=1^nV_i be the set of all posts. We define ϕ:→2φ:U→ 2^V as a post aggregation function, where ϕ(u)=v∈∣a(v)=uφ(u)=\v a(v)=u\ is the set of posts authored by u. The goal of user representation learning is to learn a parameterized encoder fθ:u→zu∈ℝdf_θ:u→ z_u ^d that maps each user u∈u to a compact embedding zuz_u. The embedding zuz_u is generated by aggregating semantic information from the user’s posts ϕ(u)φ(u) and structural context from post-post reply relations. The resulting representations are expected to capture semantic, relational, and value-relevant behavioral cues that support downstream behavior and content understanding (Pan and Ding 2019), such as stance detection and Twitter bot detection (Section Experiments and Results). Methodology Our ValueGraph consists of two stages: Foundation GNN Pre-training and Value-Guided Pre-training. In the first stage, we pre-train a foundation GNN on the post-reply graph with a masked autoencoding paradigm to obtain noise-resilient representations. In the second stage, we refine these representations using value signals derived from MoralBERT. Figure 1 shows an overview of our ValueGraph framework. Stage 1: Foundation GNN Pre-training. We first perform unsupervised pre-training of a foundation GNN on the constructed graph dataset. Each post is encoded using ModernBERT (Warner et al. 2025), chosen for robustness to noisy social media text. Reply interactions define graph edges, enabling multi-hop message passing to capture local and global conversational patterns. Pre-training follows the masked autoencoding paradigm of GraphMAE2 (Hou et al. 2023). Node features are masked before encoding, and the model reconstructs the original features. This objective encourages noise-resilient representations that serves as a strong initialization for the subsequent value-guided pre-training. Stage 2: Value-Guided Pre-training. In this stage, we refine the pre-trained representations with a hierarchical contrastive learning framework. Specifically, we infer a value profile for each user, form contrastive user pairs between users who have similar and dissimilar values, and combine a user-level contrastive objective ℒuserL_user with a clustering objective ℒclsL_cls, which fully exploit the inter-user relations encoded in value similarity and remain robust to the noise in the signal. Human-Value Grounding for Training Data. Human values play a central role in shaping social interactions, moral reasoning, and ideological expression. Several frameworks model human values, including Hofstede’s Cultural Dimensions Theory (Hofstede 2001), MFT (Haidt et al. 2007), and Schwartz’s Theory of Basic Human Values (Schwartz et al. 2012). These frameworks have been widely adopted to analyze value-driven language and behavior. We adopt MFT as our primary value framework for three reasons. First, it captures moral judgments expressed in everyday language, making it well suited for social media analysis (Johnson and Goldwasser 2018; Mooijman et al. 2018). Second, its effectiveness in extracting moral perspectives from text has been empirically validated (Lin et al. 2018; Mokhberian et al. 2020). Third, it provides structured and interpretable moral dimensions aligned with our objective of modeling value-related signals (Araque et al. 2020; Zhang et al. 2025). More details about MFT are provided in Appendix: Moral Foundations Background. To obtain value signals for pre-training, we employ MoralBERT (Trager et al. 2022), a fine-tuned language model for capturing moral-values in social discussions33 3 https://github.com/vjosapreniqi/MoralBERT. Specifically, we use ten independently fine-tuned classifiers, each corresponding to one of ten moral foundations defined by MFT, i.e., =C=\care, harm, fairness, cheating, loyalty, betrayal, authority, subversion, purity, degradation\. Given a post v, each classifier produces a probability score MoralBERT(v,c)∈[0,1]MoralBERT(v,c)∈[0,1] for each foundation c∈c . To obtain user-level signal, we aggregate scores across posts authored by a user: score(u,c)=1|ϕ(u)|∑v∈ϕ(u)MoralBERT(v,c).score(u,c)= 1|φ(u)| _v∈φ(u)MoralBERT(v,c). (1) The resulting 10-D vector is not treated as gold user values; it serves as a noisy but more robust moral-framing signal, whose usefulness comes from relative differences across users rather than exact post-level predictions. Contrastive User Pairs Construction. Given inferred value vectors derived from MFT, we sample user pairs (u,u~)∣u,u~∈andu≠u~\(u, u) u, u ~and~u≠ u\ according to a similarity function sim(h(u),h(u~))sim(h(u),h( u)). We adopt a symmetric percentile-based strategy, where the P-th percentile (e.g., P=90%P=90\%) defines positive pairs and the (1−P)%(1-P)\%-th percentile defines negative pairs: θ+ _+ =ScoreAtP(sim(h(u),h(u~))), =ScoreAt_P (\sim(h(u),h( u))\ ), (2) θ− _- =ScoreAt(1−P)(sim(h(u),h(u~))). =ScoreAt_(1-P) (\sim(h(u),h( u))\ ). For each user u, we define positive and negative sets as S(u)S(u) and D(u)D(u). Assuming an approximately symmetric similarity distribution (see Appendix: Distribution of Sampled User Similarities), this strategy yields balanced positive and negative sets. If either set is underpopulated for a given user, we iteratively relax the threshold to the median similarity until a minimum set size is reached. Similarity Computation. User similarity is computed from the Euclidean distance between inferred value vectors: d(u,u~)=‖h(u)−h(u~)‖2,d(u, u)=\|h(u)-h( u)\|_2, (3) which is converted into a similarity score through a Gaussian radial basis function kernel (Li et al. 2021): sim(h(u),h(u~))=exp(−d(u,u~)22σ2),sim(h(u),h( u))= (- d(u, u)^22σ^2 ), (4) where σ is set to the median of the sampled distances. Loss Functions. We treat the value vectors from text as a noisy auxiliary signal that defines relative constraints among users. This places a twofold requirement on our training objective: it should both fully exploit the inter-user relations encoded in value similarity and remain robust to the noise in the signal. Based on this, we design two complementary losses. The full training is illustrated in Appendix: Algorithm. (1) User-level contrastive loss ℒuserL_user: For each seed user u∈Useedu∈ U_seed, we encourage its embedding to be close to value-similar u~∈S(u) u∈ S(u) and distant from value-dissimilar users u^∈D(u) u∈ D(u). For each positive pair (u,u~)(u, u), we apply the InfoNCE loss: ℓu,u~=−logecos(u,u~)/τecos(u,u~)/τ+∑u^∈D(u)ecos(u,u^)/τ, _u, u=- e (u, u)/τe (u, u)/τ+ _ u∈ D(u)e (u, u)/τ, (5) where τ>0τ>0 is a temperature and cos(x,y) (x,y) denotes cosine similarity. The overall contrastive loss is: ℒuser=1Npos∑u∈Useed∑u~∈S(u)ℓu,u~,L_user= 1N_pos _u∈ U_seed _ u∈ S(u) _u, u, (6) where Npos=∑u∈Useed|S(u)|.N_pos= _u∈ U_seed|S(u)|. (7) This loss constructs positive and negative pairs according to the inferred value similarity and aligns user representations at the semantic level, pulling users with similar inferred value-related textual patterns together and pushing users with dissimilar values apart. (2) Clustering loss ℒclsL_cls: To regularize the global structure of the user embedding space, we periodically perform K-means clustering on the user embeddings zuz_u every X epochs. Such periodic cluster assignment follows the alternating optimization paradigm commonly adopted in deep clustering methods, where cluster assignments are periodically updated while the encoder progressively refines representations (Caron et al. 2018). In our framework, clustering serves as a global embedding-space regularizer rather than the primary learning objective. Let cic_i denote the centroid of cluster i and ℓu _u be the cluster assignment of user u. Following maximum-margin clustering, we first define a compactness term that encourages each embedding to stay close to its assigned centroid: ℒcompact=1|Useed+|∑u∥zu−cℓu∥22.L_compact= 1|U^+_seed| _u z_u-c_ _u _2^2. (8) We further define a margin term that enforces a minimum distance m between distinct cluster centroids: ℒmargin=∑0≤p<q≤K−1[max(0,m−∥cp−cq∥2)]2.L_margin= _0≤ p<q≤ K-1 [ (0,\,m- c_p-c_q _2) ]^2. (9) Combining the two, the clustering loss is defined as: ℒcls=ℒcompact+β2K(K−1)ℒmargin,L_cls=L_compact+β\, 2K(K-1)\,L_margin, (10) where β weights the margin term. For epochs without clustering, ℒclsL_cls is set to zero. This loss imposes a global geometric constraint on the user embeddings, with the compactness term encouraging intra-cluster cohesion and the margin term encouraging inter-cluster separation. (3) The overall loss ℒL: Each loss on its own covers only half of the training objective. ℒuserL_user is a purely pairwise constraint that only specifies which users should be close to which, without constraining the global geometry of the embedding space. ℒclsL_cls, in contrast, is a self-reinforcing objective that clusters the current embeddings and pulls them toward centroids: it amplifies whatever structure already exists in the representations, without distinguishing whether that structure reflects genuine user differences. Consequently, using either loss in isolation could be unstable. We therefore design the final objective as a combination of the two: ℒ=ℒuser+λℒcls,L=L_user+ _cls, (11) where λ controls the contribution of the clustering term. The two losses act as mutual regularizers, in which ℒuserL_user provides ℒclsL_cls with a meaningful clustering axis aligned along value-relevant dimensions, while ℒclsL_cls provides ℒuserL_user with global stabilization and denoising by aggregating large numbers of users into groups. Representations that simultaneously satisfy local (pairwise) value consistency and global (group) structural consistency are preserved, while noisy structure is filtered out. Appendix: Training Loss and Hyperparameter Tuning gives training loss curves and hyperparameter tuning details. Theoretical Analysis We provide a mechanism-level characterization of how the training objectives shape the embedding space, which formalizes how value-signal contrastive learning pulls users with similar inferred profiles closer, while clustering promotes compact and separated user groups. Full Proofs are provided in Appendix: Full Proof of Theorem 1 and 2. Theorem 1 (Value-Signal Similarity Preservation). Let h(u)∈ℝ10h(u) ^10 denote the inferred value vector of user u, and let zu∈ℝdz_u ^d be the user embedding learned by ValueGraph. Suppose ℒuserL_user converges under fixed positive and negative sets constructed from h(⋅)h(·). For sampled user pairs, the learned embedding similarity is encouraged to preserve the ordering induced by value-signal similarity: sim(h(u),h(u~))↑⇒sim(zu,zu~)↑.sim (h(u),h( u) ) (z_u,z_ u ) . Theorem 2 (Cluster Compactness and Separation). Let user embeddings zuu∈\z_u\_u be optimized with ℒclsL_cls, and let ckk=1K\c_k\_k=1^K denote the resulting cluster centroids. The compactness term minimizes ‖zu−cℓu‖2\|z_u-c_ _u\|_2 for users assigned to cluster ℓu _u, while the margin term penalizes centroid pairs with distance below m and therefore encourages inter-cluster separation. Theorems 1 and 2 clarify the optimization behavior: ValueGraph uses inferred value signals to organize user embeddings, while graph pre-training supplies semantic and relational context. Experiments and Results Pre-training Corpus We construct a large-scale pre-training corpus from social media conversations collected from Reddit and Twitter. Reddit dataset44 4 https://convokit.cornell.edu/documentation/subreddit.html provides diverse community discussions across subreddits, where we follow Trager et al. (2022) to select communities reflecting diverse moral concerns. Twitter datasets include widely used rumor detection benchmarks, including PHEME (Zubiaga et al. 2016), Twitter16 (Ma et al. 2016), BEARD (Zeng and Gao 2022), and Twitter-Covid (Lin et al. 2022), which contain conversations involving rumors, controversies, and value-related debates. These datasets provide rich textual and reply-based interaction signals for learning contextualized user representations. After preprocessing, the combined corpus consists of 461,198 conversation graphs with approximately 13.6 million nodes and 39.9 million edges. LLM Graph Pre-training PLM GNN ValueGraph Model Dataset Metric GPT-5.4 GraphCL GraphMAE2 SimCSE ModernBERT Bertweet GCN GAT GTN GLAN MT_CSD Acc. 0.58 0.41 0.60 0.60 0.56 0.58 0.40 0.42 0.57 0.63∗ MacF1 0.56 0.38 0.55 0.56 0.53 0.51 0.35 0.33 0.56 0.58^~ BrLSTM RumourEval19 Acc. 0.73 0.59 0.67 0.74 0.74 0.65 0.61 0.31 0.71 0.77∗ MacF1 0.47 0.38 0.47 0.53 0.59 0.56 0.25 0.18 0.45 0.71∗ Table 1: Stance detection results on MT_CSD and RumourEval19. ∗ indicates statistical significance at 99% confidence level compared with the second-best performance based on two-tailed paired Student’s t-test. Stance Detection Stance detection classifies users’ attitudes toward a target as support, oppose, or neutral. Because socio-political stances are often associated with value signal and ethical orientation (Al-Khatib et al. 2020; Durmus and Cardie 2018), ValueGraph provides value signal that serve as a useful inductive bias for stance inference. We evaluate ValueGraph on two benchmarks, MT_CSD (Niu et al. 2024) and RumourEval19 (Gorrell et al. 2019). We integrate our post-level embeddings into two established stance models. Details are shown in Appendix: Stance Detection. (1) On MT_CSD, we adopt the graph-prompt framework of Zhao et al. (2024), which performs event-aware prompt tuning over conversation graphs. We replace its GNN node embeddings with our post embeddings and use GLAN (Yuan et al. 2019) for prediction. (2) On RumourEval19, we use BrLSTM (Yue et al. 2022), which models tree-structured reply dependencies, and replace its initial token-level inputs with precomputed post embeddings. This plug-in strategy provides contextual and relational representations without modifying downstream architectures or adding supervision. Baselines. We compare against four categories of strong baseline encoders. (1) Graph-based pre-training: GraphCL (Hafidi et al. 2020), which applies contrastive learning over augmented graph views, and GraphMAE2 (Hou et al. 2023), a masked graph autoencoder. Both are pre-trained on the same datasets using an identical graph encoder and subsequently fine-tuned for stance classification. (2) Pre-trained language models: BERTweet (Nguyen et al. 2020), SimCSE (Gao et al. 2021), and ModernBERT (Warner et al. 2025). All these pre-trained language models are fine-tuned end-to-end on the stance detection task. (3) Inductive GNNs: GCN (Kipf and Welling 2016), GTN (Yun et al. 2019), and GAT (Velickovic et al. 2017), trained from scratch on the downstream task to test whether graph structure alone, without graph pre-training or value-signal guidance, is sufficient. (4) Text-only LLM: GPT-5.4, prompted with the target, available user posts, and thread text with no graph serialization.55 5 GPT-5.4 does not receive explicit graph; structure-aware graph serialization for LLMs is nontrivial and left to future work. For fair comparison, trainable models use identical data splits, optimizer configurations, and early-stopping criteria based on validation loss. GPT-5.4 is evaluated on the same test splits with fixed prompts and deterministic decoding. Fixed hyperparameters and prompt templates are provided in Appendix: Hyperparameter Settings and GPT-5.4 Prompt Templates. Evaluation Metrics. We report accuracy (acc) and macro F1 (macF1). While accuracy provides an overall measure, macF1 is more informative under class imbalance, which is prevalent in stance detection datasets. Results. As shown in Table 1, ValueGraph consistently outperforms the non-LLM and LLM baselines on both benchmarks; GPT-5.4 performs comparably with the non-LLM baselines. On MT_CSD, ValueGraph attains 63% acc and 58% macF1, yielding relative gains of 5% in acc and 5.5% in macF1 over the strongest pre-trained baseline (GraphMAE2, 60% acc and 55% macF1). On RumourEval19, ValueGraph achieves 77% acc and 71% macF1, corresponding to a relative macF1 improvement of 20.3% over the strongest PLM baseline (ModernBERT, 59%) and 57.8% over the best graph-based baseline (GTN, 45%). Twitter Bot Detection Method Setting Type Accuracy MacF1 Precision Recall MCC GPT-5.4 Text-only T 56.9(±0.1)(± 0.1) 52.1(±0.1)(± 0.1) 57.1(±0.1)(± 0.1) 54.9(±0.1)(± 0.1) 11.8(±0.1)(± 0.1) RoBERTa Baseline T 50.7(±0.2)(± 0.2) 54.8(±0.5)(± 0.5) 48.8(±0.1)(± 0.1) 62.5(±1.2)(± 1.2) – ValueGraph T 56.9(±0.6)(± 0.6)∗ 64.3(±0.4)(± 0.4)∗ 53.3(±0.6)(± 0.6)∗ 81.0(±2.3)(± 2.3)∗ – Baseline UT 63.1(±0.5)(± 0.5) 66.2(±0.5)(± 0.5) 58.9(±0.5)(± 0.5) 75.6(±1.4)(± 1.4) – ValueGraph UT 65.9(±0.3)(± 0.3)∗ 68.9(±2.4)(± 2.4)∗ 61.2(±0.6)(± 0.6) 78.7(±1.8)(± 1.8) ∗ – BotGCN (T5) Baseline TG 56.8(±6.0)(± 6.0) 64.4(±1.3)(± 1.3) 53.7(±5.1)(± 5.1) 82.1(±8.1)(± 8.1) 17.9(±9.7)(± 9.7) ValueGraph TG 70.2(±1.3)(± 1.3)∗ 67.9(±0.3)(± 0.3)∗ 69.8(±3.3)(± 3.3)∗ 66.4(±3.2)(± 3.2)∗ 40.3(±2.5)(± 2.5)∗ Baseline FTUG 70.5(±0.6)(± 0.6) 70.4(±0.4)(± 0.4) 67.1(±1.2)(± 1.2) 74.2(±1.5)(± 1.5) 41.3(±1.1)(± 1.1) ValueGraph FTUG 71.1(±0.7)(± 0.7)∗ 71.1(±0.3)(± 0.3) 68.9(±2.7)(± 2.7)∗ 72.2(±5.0)(± 5.0)∗ 42.5(±1.1)(± 1.1) BotGAT (T5) Baseline TG 47.4(±0.0)(± 0.0) 64.3(±0.0)(± 0.0) 47.5(±0.0)(± 0.0) 99.8(±0.2)(± 0.2) –0.6(±0.9)(± 0.9) ValueGraph TG 73.1(±0.3)(± 0.3)∗ 66.0(±0.6)(± 0.6)∗ 82.5(±2.2)(± 2.2)∗ 55.1(±1.8)(± 1.8)∗ 47.8(±1.2)(± 1.2)∗ Baseline FTUG 73.2(±2.6)(± 2.6) 70.9(±2.0)(± 2.0) 74.6(±8.2)(± 8.2) 69.3(±10.0)(± 10.0) 47.3(±4.7)(± 4.7) ValueGraph FTUG 74.7(±0.8)(± 0.8)∗ 73.0(±0.3)(± 0.3)∗ 74.2(±3.0)(± 3.0) 72.1(±3.3)(± 3.3) 49.3(±1.6)(± 1.6)∗ BotRGCN (T5) Baseline TG 69.2(±1.5)(± 1.5) 69.1(±0.5)(± 0.5) 66.2(±3.4)(± 3.4) 72.6(±4.3)(± 4.3) 38.9(±2.4)(± 2.4) ValueGraph TG 70.3(±1.1)(± 1.1) 69.1(±0.5)(± 0.5)∗ 68.5(±2.9)(± 2.9)∗ 70.0(±3.5)(± 3.5)∗ 40.7(±2.0)(± 2.0) Baseline FTUG 71.0(±0.8)(± 0.8) 73.5(±0.5)(± 0.5) 65.0(±1.1)(± 1.1) 84.8(±1.7)(± 1.7) 44.7(±1.2)(± 1.2) ValueGraph FTUG 73.0(±0.8)(± 0.8)∗ 74.2(±0.2)(± 0.2) 68.0(±1.7)(± 1.7)∗ 82.0(±2.2)(± 2.2)∗ 47.0(±1.1)(± 1.1) Table 2: Comparison of text-only GPT-5.4, baseline embeddings, and ValueGraph embeddings for twitter bot detection. * and ** indicate statistical significance at 95% and 99% confidence levels, respectively, based on two-tailed paired Student’s t-test between ValueGraph and the corresponding baseline embedding. Detecting twitter bots is critical for maintaining the integrity of online discourse. However, most existing detectors rely primarily on surface-level textual features or network structure. We evaluate whether replacing or augmenting such cues with ValueGraph user embeddings guided by inferred value signals can improve bot detection performance across strong baseline models. We use TwiBot-22 (Feng et al. 2022) dataset for evaluation (Dataset details are in Appendix: Twitter Bot Detection). Baselines. We select high-performing models from the benchmark study by Feng et al. (2022), covering different data modalities. We include RoBERTa (Liu et al. 2019) as a text-based baseline, together with BotRGCN (Feng et al. 2021), BotGAT (Lei et al. 2022), and BotGCN (Feng et al. 2021), which use T5 encoder (Raffel et al. 2020) to encode text. We also add a GPT-5.4 baseline using sampled user posts and profile text with no graph serialization. Each non-LLM model is evaluated under configurations using F (user metadata or engineered features), T (posts), U (profile description), and G (network structure). We examine whether substituting T/G-based representations with ValueGraph embeddings improves performance. Setup and Metrics. All trainable encoders retain their original architectures and default settings. We perform grid search over hyperparameters (see Appendix: Hyperparameter Settings), run each experiment five times with different random seeds, and report Accuracy, MacF1, Precision, Recall, and Matthews Correlation Coefficient (MCC). Results. Table 2 shows that ValueGraph-based embeddings consistently outperform non-LLM baseline representations across models. For instance, RoBERTa in the text-only configuration improves macF1 from 54.8% to 64.3%, with recall increasing from 62.5% to 81.0%. While GPT-5.4 achieves acc generally on par with ValueGraph in the text-only setting, its macF1 is substantially lower. These gains indicate that ValueGraph captures complementary semantic and structural cues for bot-human discrimination. While the improvements are statistically significant in text+network settings, significance tends to decrease when full multimodal inputs are available, particularly for macF1 in FTUG configurations, where strong profile and graph features already dominate the decision boundary. Nevertheless, because the ValueGraph encoder is not trained with bot labels or downstream fine-tuning, these gains indicate that value-signal guidance adds useful information even in feature-rich settings. (a) ValueGraph Embeddings (b) RoBERTa Embeddings (c) T5 Embeddings (d) GraphMAE2 Embeddings Figure 2: t-SNE visualization of user embeddings for Twitter bot detection. Red and grey dots represent ground-truth human and bot users, respectively. Ablation Study Task Model/data Configuration Accuracy MacF1 Stance GLAN (MT_CSD) Only Stage 1 41.2 40.7 Stage 1 + ℒclsL_cls 50.4 34.3 Stage 1 + ℒuserL_user 45.3 32.6 Full 63.0 58.0 BrLSTM (RumourEval19) Only Stage 1 53.7 55.8 Stage 1 + ℒclsL_cls 62.7 52.8 Stage 1 + ℒuserL_user 53.9 45.7 Full 77.1 71.2 Bot BotGAT (TwiBot-22) Only Stage 1 73.1 70.3 Stage 1 + ℒclsL_cls 72.9 72.6 Stage 1 + ℒuserL_user 72.6 72.0 Full 74.7 73.0 Table 3: Ablation results of ValueGraph embeddings under different training settings across stance detection and Twitter bot detection tasks. For Twitter bot detection, BotGAT is evaluated in the FTUG setting. To assess the contribution of value-signal information, we conduct controlled ablation experiments. Since Stage 1 pre-training is essential for capturing reply structure and producing stable representations, it is retained in all settings. We evaluate four variants: (1) Stage 1 pre-training only; (2) Stage 1 ++ ℒclsL_cls; (3) Stage 1 ++ ℒuserL_user; and (4) the full model. Table 3 shows that neither ℒclsL_cls nor ℒuserL_user alone yields consistent improvements, which supports our conjecture in the Loss Function section. Notably, a single loss can raise accuracy while lowering macF1. For example, on GLAN (MT_CSD), adding ℒclsL_cls improves accuracy from 41.2%41.2\% to 50.4%50.4\% but drops macF1 from 40.7%40.7\% to 34.3%34.3\%. In contrast, the full ValueGraph model, which jointly integrates structural pre-training, clustering supervision, and value-signal guidance, achieves consistent improvements across all tasks. These results validate the complete ValueGraph design (see Appendix: Supplementary Results of the Ablation Experiments for full results). User Clustering Analysis We use t-SNE (see Appendix: t-SNE Setting for parameter setup) to visualize how different embedding models organize users for Twitter bot detection. As shown in Figure 2, ValueGraph yields clearer bot-human separation than the compared text and graph encoders. This visualization provides dataset-specific qualitative evidence of improved representation separability. A content-level comparison helps explain this separation. In the evaluated dataset, some bot clusters contain repetitive and polarized value signal. For example, the bot account (author_id anonymized) posted an accusatory narrative: “Paul Vickers died suddenly 3 months after I exposed his #Establishment lies … contributed to his death … #FactCheck …”, relying on blame attribution, repeated hashtags, and a one-sided moral stance. In contrast, a human account (author_id anonymized) wrote: “Is that what the point is? Being angry with the one we love is not the same as unloving them, is it?”, showing more dialogic and context-sensitive expression. These examples suggest that ValueGraph captures value signals useful for bot detection. Conclusion We presented ValueGraph, a value-signal guided graph pre-training framework for contextualized user representation. Grounded in MFT, ValueGraph combines semantic and structural graph pre-training, contrastive learning over inferred value similarity, and clustering to capture textual, relational, and value-relevant behavioral cues. Experiments on stance detection and Twitter bot detection show consistent gains over strong baselines. These results suggest that noisy but theory-informed value signals provide useful auxiliary guidance for socially informed user modeling without predicting users’ true values. Ethics Statement This work uses publicly available social-media datasets under their original access conditions and licenses. We do not infer users’ true psychological values; MoralBERT-derived vectors are used only as noisy aggregate signals for representation learning. Results should be interpreted as dataset-specific and should not be used for individual-level profiling or decision-making without appropriate safeguards. We will release code and processing scripts but will not redistribute raw social-media content or personally identifiable information. The released artifacts are intended for research purposes only. Acknowledgment This research is supported by the SMU-A*STAR Joint Lab in Social and Human-Centered Computing (SMU grant no.: SAJL-2022-CSS02, SAJL-2022-CSS003). This research is supported by A*STAR (C232918004, C232918005). References Al-Khatib et al. (2020) K. Al-Khatib, M. Völske, S. Syed, N. Kolyada, and B. Stein Exploiting personal characteristics of debaters for predicting persuasiveness. In ACL, Cited by: Stance Detection. Araque et al. (2020) O. Araque, L. Gatti, and K. Kalimeri MoralStrength: exploiting a moral lexicon and embedding similarity for moral foundations prediction. Knowledge-Based Systems 191, p. 105184. Cited by: Human-Value Grounding for Training Data.. Benton et al. (2016) A. Benton, R. Arora, and M. Dredze Learning multiview embeddings of Twitter users. In ACL, Cited by: Introduction, User Representation.. Caron et al. (2018) M. Caron, P. Bojanowski, A. Joulin, and M. Douze Deep clustering for unsupervised learning of visual features. In Proceedings of the European Conference on Computer Vision (ECCV), Cited by: Loss Functions.. Christiano et al. (2017) P. F. Christiano, J. Leike, T. Brown, M. Martic, S. Legg, and D. Amodei Deep reinforcement learning from human preferences. In NIPS, Cited by: AI and Human Values.. Ding et al. (2017) T. Ding, W. K. Bickel, and S. Pan Multi-view unsupervised user feature embedding for social media-based substance use prediction. In EMNLP, Cited by: Introduction, User Representation.. Do et al. (2018) T. H. Do, D. M. Nguyen, E. Tsiligianni, B. Cornelis, and N. Deligiannis Twitter user geolocation using deep multiview learning. In ICASSP, Cited by: Introduction, User Representation.. Donnat et al. (2018) C. Donnat, M. Zitnik, D. Hallac, and J. Leskovec Learning structural node embeddings via diffusion wavelets. In SIGKDD, Cited by: Introduction, Graph Pre-training.. Durmus and Cardie (2018) E. Durmus and C. Cardie Exploring the role of prior beliefs for argument persuasion. In NAACL, Cited by: Stance Detection. Feng et al. (2022) S. Feng, Z. Tan, H. Wan, N. Wang, Z. Chen, B. Zhang, Q. Zheng, W. Zhang, Z. Lei, S. Yang, X. Feng, Q. Zhang, H. Wang, Y. Liu, Y. Bai, H. Wang, Z. Cai, Y. Wang, L. Zheng, Z. Ma, J. Li, and M. Luo TwiBot-22: towards graph-based twitter bot detection. In NeurIPS, Cited by: Appendix F, Baselines., Twitter Bot Detection. Feng et al. (2021) S. Feng, H. Wan, N. Wang, and M. Luo BotRGCN: twitter bot detection with relational graph convolutional networks. In ASONAM, Cited by: Appendix F, Baselines.. Gao et al. (2021) T. Gao, X. Yao, and D. Chen SimCSE: simple contrastive learning of sentence embeddings. In EMNLP, Cited by: Baselines.. Gorrell et al. (2019) G. Gorrell, E. Kochkina, M. Liakata, A. Aker, A. Zubiaga, K. Bontcheva, and L. Derczynski SemEval-2019 task 7: RumourEval, determining rumour veracity and support for rumours. In SemEval, Cited by: Appendix F, Stance Detection. Graham et al. (2013) J. Graham, J. Haidt, S. Koleva, M. Motyl, R. Iyer, S. P. Wojcik, and P. H. Ditto Chapter two - moral foundations theory: the pragmatic validity of moral pluralism. P. Devine and A. Plant (Eds.), Advances in Experimental Social Psychology, Vol. 47, p. 55–130. External Links: ISSN 0065-2601, Document, Link Cited by: Introduction. Graham et al. (2009) J. Graham, J. Haidt, and B. Nosek Liberals and conservatives rely on different sets of moral foundations. Journal of personality and social psychology 96, p. 1029–46. Cited by: Appendix C. Grover and Leskovec (2016) A. Grover and J. Leskovec Node2vec: scalable feature learning for networks. In SIGKDD, Cited by: Introduction, Graph Pre-training.. Guo et al. (2023) S. Guo, N. Mokhberian, and K. Lerman A data fusion framework for multi-domain morality learning. In ICWSM, Cited by: Introduction, AI and Human Values.. Hafidi et al. (2020) H. Hafidi, M. Ghogho, P. Ciblat, and A. Swami GraphCL: contrastive self-supervised learning of graph representations. ArXiv abs/2007.08025. Cited by: Introduction, Graph Pre-training., Baselines.. Haidt et al. (2007) J. Haidt C. Joseph et al. The moral mind: how five sets of innate intuitions guide the development of many culture-specific virtues, and perhaps even modules. The innate mind 3, p. 367–391. Cited by: Introduction, Human-Value Grounding for Training Data.. Hassani and Khasahmadi (2020) K. Hassani and A. H. Khasahmadi Contrastive multi-view representation learning on graphs. In ICML, Cited by: Introduction, Graph Pre-training.. Hofstede (2001) G. Hofstede Culture’s consequences: comparing values, behaviors, institutions and organizations across nations. Sage Publications. Cited by: Human-Value Grounding for Training Data.. Hou et al. (2023) Z. Hou, Y. He, Y. Cen, X. Liu, Y. Dong, E. Kharlamov, and J. Tang GraphMAE2: a decoding-enhanced masked self-supervised graph learner. In W, Cited by: Stage 1: Foundation GNN Pre-training., Baselines.. Hu et al. (2020a) W. Hu, B. Liu, J. Gomes, M. Zitnik, P. Liang, V. Pande, and J. Leskovec Strategies for pre-training graph neural networks. In ICLR, Cited by: Graph Pre-training.. Hu et al. (2020b) Z. Hu, Y. Dong, K. Wang, K. Chang, and Y. Sun GPT-gnn: generative pre-training of graph neural networks. In SIGKDD, Cited by: Graph Pre-training.. Johnson and Goldwasser (2018) K. Johnson and D. Goldwasser Classification of moral foundations in microblog political discourse. In ACL, Cited by: Human-Value Grounding for Training Data.. Kipf and Welling (2016) T. N. Kipf and M. Welling Semi-supervised classification with graph convolutional networks. arXiv:1609.02907. Cited by: Baselines.. Lei et al. (2022) Z. Lei, H. Wan, W. Zhang, S. Feng, Z. Chen, J. Li, Q. Zheng, and M. Luo BIC: twitter bot detection with text-graph interaction and semantic consistency. arXiv:2208.08320. External Links: Link Cited by: Appendix F, Baselines.. Li et al. (2021) Y. Li, R. Pogodin, D. J. Sutherland, and A. Gretton Self-supervised learning with kernel dependence maximization. In NeurIPS, Cited by: Similarity Computation.. Lin et al. (2022) H. Lin, J. Ma, L. Chen, Z. Yang, M. Cheng, and C. Guang Detect rumors in microblog posts for low-resource domains via adversarial contrastive learning. In Findings of NAACL, Cited by: Appendix F, Table 5, Pre-training Corpus. Lin et al. (2018) Y. Lin, J. Hoover, G. Portillo-Wightman, C. Park, M. Dehghani, and H. Ji Acquiring background knowledge to improve moral value prediction. In ASONAM, Cited by: Human-Value Grounding for Training Data.. Liu et al. (2019) Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov Roberta: a robustly optimized bert pretraining approach. arXiv:1907.11692. Cited by: Baselines.. Lu et al. (2021) Y. Lu, X. Jiang, Y. Fang, and C. Shi Learning to pre-train graph neural networks. In AAAI, Cited by: Graph Pre-training.. Ma et al. (2016) J. Ma, W. Gao, P. Mitra, S. Kwon, B. J. Jansen, K. Wong, and M. Cha Detecting rumors from microblogs with recurrent neural networks. In IJCAI, Cited by: Appendix F, Table 5, Pre-training Corpus. Mokhberian et al. (2020) N. Mokhberian, A. Abeliuk, P. Cummings, and K. Lerman Moral framing and ideological bias of news. In SocInfo, Cited by: Human-Value Grounding for Training Data.. Mooijman et al. (2018) M. Mooijman, J. Hoover, Y. Lin, H. Ji, and M. Dehghani Moralization in social networks and the emergence of violence during protests. Nature human behaviour 2 (6), p. 389–396. Cited by: Human-Value Grounding for Training Data.. Nguyen et al. (2020) D. Q. Nguyen, T. Vu, and A. Tuan Nguyen BERTweet: a pre-trained language model for English tweets. In EMNLP, Cited by: Baselines.. Nguyen et al. (2024) T. D. Nguyen, Z. Chen, N. G. Carroll, A. Tran, C. Klein, and L. Xie Measuring moral dimensions in social media with mformer. In ICWSM, Cited by: Introduction, AI and Human Values.. Niu et al. (2024) F. Niu, M. Yang, A. Li, B. Zhang, X. Peng, and B. Zhang A challenge dataset and effective models for conversational stance detection. In LREC-COLING, Cited by: Appendix F, Stance Detection. Ouyang et al. (2022) L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. L. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, J. Schulman, J. Hilton, F. Kelton, L. Miller, M. Simens, A. Askell, P. Welinder, P. Christiano, J. Leike, and R. Lowe Training language models to follow instructions with human feedback. In NeurIPS, Cited by: AI and Human Values.. Pan and Ding (2019) S. Pan and T. Ding Social media-based user embedding: a literature review. In IJCAI, Cited by: Introduction, User Representation., Problem Formulation. Perozzi et al. (2014) B. Perozzi, R. Al-Rfou, and S. Skiena DeepWalk: online learning of social representations. In SIGKDD, Cited by: Introduction, Graph Pre-training.. Pick et al. (2022) R. K. Pick, V. Kozhukhov, D. Vilenchik, and O. Tsur Stem: unsupervised structural embedding for stance detection. In AAAI, Cited by: Introduction. Preniqi et al. (2024) V. Preniqi, I. Ghinassi, J. Ive, C. Saitis, and K. Kalimeri MoralBERT: a fine-tuned language model for capturing moral values in social discussions. In GoodIT, Cited by: Introduction, Introduction, AI and Human Values.. Qiu et al. (2020) J. Qiu, Q. Chen, Y. Dong, J. Zhang, H. Yang, M. Ding, K. Wang, and J. Tang GCC: graph contrastive coding for graph neural network pre-training. In SIGKDD, Cited by: Graph Pre-training.. Raffel et al. (2020) C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P. J. Liu Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res. 21 (1). Cited by: Baselines.. Rahimi et al. (2015) A. Rahimi, T. Cohn, and T. Baldwin Twitter user geolocation using a unified text and network prediction model. In ACL-IJCNLP, Cited by: Introduction, User Representation.. Ribeiro et al. (2018) M. Ribeiro, P. Calais, Y. Santos, V. Almeida, and W. Meira Jr Characterizing and detecting hateful users on twitter. In ICWSM, Cited by: User Representation.. Ribeiro et al. (2020) M. H. Ribeiro, R. Ottoni, R. West, V. A. F. Almeida, and W. Meira Auditing radicalization pathways on youtube. In FAT, Cited by: Introduction. Schwartz et al. (2012) S. H. Schwartz, J. Cieciuch, M. Vecchione, E. Davidov, R. Fischer, C. Beierlein, A. Ramos, M. Verkasalo, J. Lönnqvist, K. Demirutku, et al. Refining the theory of basic individual values. Journal of Personality and Social Psychology 103 (4), p. 663. Cited by: Human-Value Grounding for Training Data.. Sun et al. (2020a) F. Sun, J. Hoffmann, and J. Tang InfoGraph: unsupervised and semi-supervised graph-level representation learning via mutual information maximization. In ICLR, Cited by: Introduction, Graph Pre-training.. Sun et al. (2020b) K. Sun, Z. Zhu, and Z. Lin Multi-stage self-supervised learning for graph convolutional networks. In AAAI, Cited by: Introduction, Graph Pre-training.. Tang et al. (2015) J. Tang, M. Qu, M. Wang, M. Zhang, J. Yan, and Q. Mei LINE: large-scale information network embedding. In W, Cited by: Graph Pre-training.. Trager et al. (2022) J. Trager, A. S. Ziabari, A. M. Davani, P. Golazizian, F. Karimi-Malekabadi, A. Omrani, Z. Li, B. Kennedy, N. K. Reimer, M. Reyes, et al. The moral foundations reddit corpus. arXiv:2208.05545. Cited by: Appendix F, Human-Value Grounding for Training Data., Pre-training Corpus. Velickovic et al. (2017) P. Velickovic, G. Cucurull, A. Casanova, A. Romero, P. Lio, Y. Bengio, et al. Graph attention networks. stat 1050 (20), p. 10–48550. Cited by: Baselines.. Warner et al. (2025) B. Warner, A. Chaffin, B. Clavié, O. Weller, O. Hallström, S. Taghadouini, A. Gallagher, R. Biswas, F. Ladhak, T. Aarsen, G. T. Adams, J. Howard, and I. Poli Smarter, better, faster, longer: a modern bidirectional encoder for fast, memory efficient, and long context finetuning and inference. In ACL, Cited by: Stage 1: Foundation GNN Pre-training., Baselines.. You et al. (2020) Y. You, T. Chen, Y. Sui, T. Chen, Z. Wang, and Y. Shen Graph contrastive learning with augmentations. In NeurIPS, Cited by: Graph Pre-training.. Yuan et al. (2019) C. Yuan, Q. Ma, W. Zhou, J. Han, and S. Hu Jointly embedding the local and global relations of heterogeneous graph for rumor detection. External Links: 1909.04465, Link Cited by: Stance Detection. Yue et al. (2022) H. Yue, Y. He, K. Shu, and H. Liu Multi-target stance detection with bi-directional recursive encoding and classification. In IJCAI, Cited by: Stance Detection. Yun et al. (2019) S. Yun, M. Jeong, R. Kim, J. Kang, and H. J. Kim Graph transformer networks. In NeurIPS, Cited by: Baselines.. Zangari et al. (2025) L. Zangari, C. M. Greco, D. Picca, and A. Tagarelli ME2-bert: are events and emotions what you need for moral foundation prediction?. In COLING, Cited by: AI and Human Values.. Zeng and Gao (2022) F. Zeng and W. Gao Early rumor detection using neural Hawkes process with a new benchmark dataset. In NAACL, Cited by: Appendix F, Table 5, Pre-training Corpus. Zhang et al. (2017) D. Zhang, J. Yin, X. Zhu, and C. Zhang User profile preserving social network embedding. In IJCAI, Cited by: User Representation.. Zhang et al. (2025) H. Zhang, Q. Nguyen, P. Bhattacharya, W. Gao, L. Z. Wong, B. S. Loh, J. J. P. Simons, and J. An Enhancing stance classification on social media using quantified moral foundations. In ASONAM, Cited by: Human-Value Grounding for Training Data.. Zhang et al. (2019) J. Zhang, Y. Dong, Y. Wang, J. Tang, and M. Ding ProNE: fast and scalable network representation learning. In IJCAI, Cited by: Graph Pre-training.. Zhang et al. (2018) W. Zhang, W. Wang, J. Wang, and H. Zha User-guided hierarchical attention network for multi-modal social image popularity prediction. In W, Cited by: User Representation.. Zhao et al. (2021) T. Zhao, Y. Liu, L. Neves, O. J. Woodford, M. Jiang, and N. Shah Data augmentation for graph neural networks. In AAAI, Cited by: Introduction, Graph Pre-training.. Zhao et al. (2024) Y. Zhao, Z. Xue, J. Zhang, F. Wei, W. Che, and T. Liu Graph prompt learning for stance detection. arXiv:2403.11145. Cited by: Stance Detection. Zhu et al. (2020) Y. Zhu, Y. Xu, F. Yu, Q. Liu, S. Wu, and L. Wang Deep graph contrastive representation learning. ArXiv abs/2006.04131. Cited by: Graph Pre-training.. Zubiaga et al. (2016) A. Zubiaga, A. Caba Heilbron, M. Liakata, R. Procter, P. Tolmie, and K. Bontcheva PHEME: a dataset for fine-grained stance detection. arXiv:1610.07363. Cited by: Appendix F, Table 5, Pre-training Corpus. Appendix Appendix A Full Proof of Theorem 1 Proof. The gradient of the InfoNCE loss with respect to the embedding zuz_u takes the form ∇zuℓu,u~∝1τ(zu~−∑u^∈(u)pu^zu^), _z_u _u, u 1τ (z_ u- _ u (u)p_ uz_ u ), (12) where, for simplicity, all embeddings are assumed to be normalized so that cosine similarity can be treated as a dot product. Define A A =ecos(zu,zu~)/τ, =e (z_u,z_ u)/τ, (13) B B =ecos(zu,zu~)/τ+∑u^∈(u)ecos(zu,zu^)/τ. =e (z_u,z_ u)/τ+ _ u (u)e (z_u,z_ u)/τ. Then the loss can be written as ℓu,u~ _u, u =−logAB=−logA+logB =- AB=- A+ B (14) =−cos(zu,zu~)τ+logB. =- (z_u,z_ u)τ+ B. Differentiating the first term gives ∇zu[−cos(zu,zu~)τ]=−1τzu~. _z_u [- (z_u,z_ u)τ ]=- 1τz_ u. (15) Differentiating the second term gives ∇zuB=1τecos(zu,zu~)/τzu~+∑u^∈(u)1τecos(zu,zu^)/τzu^. _z_uB= 1τe (z_u,z_ u)/τz_ u+ _ u (u) 1τe (z_u,z_ u)/τz_ u. (16) Define the softmax probabilities as pu~ p_ u =ecos(zu,zu~)/τB, = e (z_u,z_ u)/τB, (17) pu p_ u =ecos(zu,zu^)/τB,u^∈(u). = e (z_u,z_ u)/τB, u (u). The gradient of the second term is therefore ∇zulogB=1τ(pu~zu~+∑u^∈(u)pu^zu^), _z_u B= 1τ (p_ uz_ u+ _ u (u)p_ uz_ u ), (18) and the full gradient becomes ∇zuℓu,u~=1τ((pu~−1)zu~+∑u^∈(u)pu^zu^). _z_u _u, u= 1τ ((p_ u-1)z_ u+ _ u (u)p_ uz_ u ). (19) By construction, positive pairs have high inferred value-signal similarity, while negative pairs have low inferred value-signal similarity. The InfoNCE objective therefore pulls zuz_u toward embeddings of value-similar users and pushes it away from value-dissimilar users. Under gradient descent, the update for zuz_u is zu(t+1)=zu(t)+ητ(zu~−∑u^∈(u)pu^zu^).z_u^(t+1)=z_u^(t)+ ητ (z_ u- _ u (u)p_ uz_ u ). (20) When h(u)h(u) and h(u~)h( u) are close, the positive pair dominates the softmax, so the update primarily pulls zuz_u toward zu~z_ u. This yields cos(zu,zu~)↑whencos(h(u),h(u~))↑. (z_u,z_ u ) (h(u),h( u) ) . (21) Thus, the learned embedding similarity is encouraged to preserve the ordering induced by inferred value-signal similarity. ∎ Appendix B Full Proof of Theorem 2 Proof. The compactness loss ℒcompact=1||∑u∈‖zu−cℓu‖22L_compact= 1|U| _u \|z_u-c_ _u\|_2^2 (22) minimizes the squared Euclidean distance between each user embedding zuz_u and its assigned centroid cℓuc_ _u. If zuz_u deviates from cℓuc_ _u, the loss increases; optimization therefore pulls embeddings toward their assigned centroids and encourages intra-cluster compactness. The margin loss is ℒmargin=∑p<q[max(0,m−‖cp−cq‖2)]2.L_margin= _p<q [ (0,m-\|c_p-c_q\|_2 ) ]^2. (23) For any pair of distinct centroids cpc_p and cqc_q, if ‖cp−cq‖2≥m\|c_p-c_q\|_2≥ m, the corresponding loss term is zero and no further separation force is applied. If ‖cp−cq‖2<m\|c_p-c_q\|_2<m, the pair contributes ℓpq=(m−‖cp−cq‖2)2. _pq= (m-\|c_p-c_q\|_2 )^2. (24) Let d=‖cp−cq‖2d=\|c_p-c_q\|_2. The gradient with respect to cpc_p is ∇cpℓpq=−2(m−d)cp−cqd, _c_p _pq=-2 (m-d ) c_p-c_qd, (25) with a symmetric expression for cqc_q. Thus, when d<md<m, the gradient pushes the centroids apart until their distance reaches the margin. The total loss is ℒ=ℒuser+λℒcls,L=L_user+ _cls, (26) where ℒclsL_cls includes both compactness and margin terms. Therefore, minimizing ℒclsL_cls jointly encourages embeddings within the same cluster to remain close while pushing different cluster centroids apart. This establishes the intended compactness and separation properties. ∎ Appendix C Moral Foundations Background Moral Foundations Theory (MFT): To assess individuals’ moral foundations, Graham et al. (2009) developed survey-based questions using factor analysis. We summarize the core moral foundations below: • Care/Harm: This foundation reflects the tendency to form emotional bonds and experience distress at others’ suffering. It emphasizes kindness, compassion, and nurturance, while discouraging harm. • Fairness/Cheating: Rooted in reciprocal altruism, this foundation promotes justice, equity, proportionality, and autonomy. • Loyalty/Betrayal: This foundation reflects loyalty to one’s in-group or community. It supports patriotism and self-sacrifice, but in extreme cases can lead to nepotism or favoritism. • Authority/Subversion: This foundation reflects respect for leadership, hierarchy, tradition, and established social structures. • Sanctity/Degradation: This foundation reflects the inclination to uphold purity and avoid contamination. It is often tied to religious and cultural ideals of moral elevation. Appendix D Distribution of Sampled User Similarities Figure 3: Distribution of RBF-based similarities for 100,000 randomly sampled user pairs. As shown in Figure 3, similarity scores are concentrated at lower values, with a long tail toward higher similarities. This indicates that most user pairs in the inferred value-profile space have weak affinity, while a small subset shows strong alignment. Based on this distribution, we use percentile-based sampling, selecting positive pairs from the 99.5-th percentile and negative pairs from the 0.5-th percentile. This strategy yields discriminative contrastive examples, reduces ambiguity from moderately similar pairs, and preserves enough samples for stable training. Appendix E Training Loss and Hyperparameter Tuning To optimize ValueGraph, we conduct a grid search over key hyperparameters in the loss formulation. The objective combines the user-level contrastive loss ℒuserL_user and the clustering loss ℒclsL_cls. Final hyperparameters are selected based on convergence behavior and downstream validation performance. The tuned hyperparameters are: • Temperature τ: Controls the sharpness of similarity distributions in the InfoNCE loss. Smaller τ emphasizes hard negatives, while larger τ produces smoother gradients and more stable optimization. • Margin coefficient β: Scales the margin term ℒmarginL_margin in the clustering loss. Larger β enforces stronger inter-cluster separation, while smaller β allows more flexible cluster boundaries. • Number of augmented users k: The number of users sampled per seed user during contrastive training. Larger k increases training diversity but also computational cost. • Number of clusters K: Controls the granularity of user grouping in the embedding space. • Margin m: Specifies the minimum distance encouraged between cluster centroids. We report convergence plots for representative configurations in Figure 4. The final configuration is: temperature τ=0.07τ=0.07, margin coefficient β=0.5β=0.5, augmented users k=10k=10, clusters K=5K=5, clustering weight λ=0.05λ=0.05, and margin m=1m=1. (a) K=20K=20, k=10k=10 (b) K=10K=10, k=10k=10 (c) K=5K=5, k=20k=20 Figure 4: Hyperparameter configurations used in ValueGraph training. All settings use temperature τ=0.07τ=0.07, margin coefficient β=0.5β=0.5, margin m=1m=1, and clustering loss weight λcluster=0.05 _cluster=0.05. The three configurations vary in the number of augmented users k and the number of clusters K. Appendix F Evaluation Tasks and Datasets Details of Pre-training Datasets We provide additional statistics of the pre-training datasets, including source domains, topic distributions, and graph statistics. Reddit Corpus. Our Reddit corpus is constructed from the Reddit Corpus provided by ConvoKit66 6 https://convokit.cornell.edu/documentation/subreddit.html, following the dataset construction procedure of Trager et al. (2022). It contains conversations collected from multiple subreddits representing diverse social communities and discussion topics. The selected subreddits cover political discussions, interpersonal relationships, social issues, and general-interest communities. We retain conversation threads with at least 50 comments to ensure sufficient interaction structures. After preprocessing, including removing inaccessible comments, filtering isolated nodes, and preserving reply-based interactions, the Reddit corpus contains approximately 9.6 million comments. The detailed distribution across subreddits is shown in Table 4. Subreddit Comments Politics 5,626,191 Neoliberal 2,315,326 Relationship Advice 861,704 Confession 477,353 Conservative 165,182 Worldnews 100,194 AmItheAsshole 83,624 Geopolitics 41,700 Antiwork 452 Nostalgia 378 Total 9,672,104 Table 4: Statistics of the Reddit pre-training corpus. Twitter Corpus. Our Twitter corpus combines several publicly available social media conversation datasets, including PHEME (Zubiaga et al. 2016), Twitter16 (Ma et al. 2016), BEARD (Zeng and Gao 2022), and Twitter-Covid (Lin et al. 2022). These datasets cover diverse scenarios such as rumor propagation, public health discussions, and event-driven social interactions. During preprocessing, we remove isolated nodes without parent or child interactions and preserve only users participating in conversational structures. The final Twitter corpus contains over 5 million posts. The detailed statistics of each source dataset are summarized in Table 5. Dataset Posts BEARD (Zeng and Gao 2022) 3,393,578 Twitter-Covid (Lin et al. 2022) 474,710 PHEME (Zubiaga et al. 2016) 59,387 Twitter16 (Ma et al. 2016) 1,101,985 Total 5,029,660 Table 5: Statistics of the Twitter pre-training corpus. Stance Detection MT_CSD (Niu et al. 2024) is a benchmark for Multi-Target Conversational Stance Detection, constructed from authentic Reddit interactions. It contains 15,876 human-annotated instances over extended conversation threads, with approximately 76% involving more than three reply turns. The dataset includes Reddit posts, popularity metrics, and discussions centered on targets such as Tesla, SpaceX, Donald Trump, Joe Biden, and Bitcoin. RumourEval19 (Gorrell et al. 2019) Task A is a Twitter benchmark for rumor stance classification in conversational threads. It contains 15,874 tweets and provides tweet IDs and reply-to IDs, enabling reconstruction of tree- or graph-structured conversations. Dataset statistics are shown in Tables 6 and 7. Split Label Count Train Against 3,349 Favor 2,383 None 5,441 Valid Against 710 Favor 481 None 1,115 Test Against 728 Favor 470 None 1,197 Table 6: Statistics for MT_CSD dataset Split Label Count Train Comment 2,734 Deny 333 Query 330 Support 841 Dev Comment 173 Deny 11 Query 28 Support 69 Table 7: Statistics for RumourEval19 Twitter Bot Detection TwiBot-22 (Feng et al. 2022) is a graph-based bot detection benchmark and one of the largest annotated Twitter account datasets. It models a heterogeneous graph with user, tweet, and hashtag nodes connected by multiple edge types. We select representative baselines from TwiBot-22 and categorize them by modality: F (user metadata), T (tweet and description content), and G (Twitter network structure). For BotRGCN (Feng et al. 2021), BotGAT (Lei et al. 2022), and BotGCN (Feng et al. 2021), we concatenate our 512-dimensional GNN-generated user embeddings with the original user features, including numerical properties (num_prop5), categorical properties (cat_prop3), tweets (tweet), and user descriptions (des). Appendix G Additional Experimental Details Algorithm Algorithm 1 summarizes the training procedure of ValueGraph. At each epoch, we first sample seed users and construct an augmented user batch by retrieving similar and dissimilar users according to their inferred value profiles. The corresponding posts are encoded by the graph encoder, and user embeddings are obtained by aggregating post representations. The value-guided contrastive loss ℒuserL_user is then computed to align users with similar value profiles while separating dissimilar ones. To further regularize the global structure of the embedding space, we periodically perform K-means clustering every X epochs and compute the clustering loss ℒclsL_cls. The final objective combines both losses to update the encoder parameters. Algorithm 1 ValueGraph Training 1: Graph =(,,ℰ)G=(U,V,E); foundation encoder f; similar-user sets S; dissimilar-user sets D; augmented users per seed k; temperature τ; loss weight λ; number of clusters K; epochs E; clustering interval X 2: Trained encoder f 3: for e=1e=1 to E do 4: Sample seed users Useed⊂U_seed 5: Construct augmented user batch Useed+←⋃u∈Useed(u~∼(u)∪u^∼(u))U^+_seed← _u∈ U_seed ( u k S(u)∪ u k D(u) ) 6: Collect posts V←⋃u∈Useed+ϕ(u)V← _u∈ U^+_seedφ(u) 7: Encode posts zv←f(v),∀v∈Vz_v← f(v),\ ∀ v∈ V 8: Compute user embeddings zu←MEAN(zv∣v∈ϕ(u)),∀u∈Useed+z_u (\z_v v∈φ(u)\ ), ∀ u∈ U^+_seed 9: Compute ℒuserL_user using Eq. 6 10: if emodX=0e X=0 or e=1e=1 then 11: Run K-means on zu∣u∈Useed+\z_u u∈ U^+_seed\ 12: Compute ℒclsL_cls using Eq. 10 13: else 14: ℒcls←0L_cls← 0 15: end if 16: ℒ←ℒuser+λℒclsL _user+ _cls 17: Update f by gradient descent on ℒL 18: end for 19: return f Hyperparameter Settings We report the hyperparameter configurations used in our experiments. Tables 8 and 9 and summarize the settings for stance detection and Twitter bot detection, respectively. Hyperparameters are tuned for each model and dataset. Additional search may further improve performance for specific embedding configurations. Model Hidden Dim Learning Rate Weight Decay Dropout Batch Size Max Epoch GLAN (MT_CSD) 512 1e-5 1e-4 0.2 32 100 BrLSTM (RumourEval19) 300 1e-4 1e-4 0.5 16 100 Table 8: Hyperparameter settings for stance detection models. Model Hidden Dim Learning Rate Weight Decay Dropout Batch Size Max Epoch RoBERTa (T) 128 1e-4 1e-5 0.5 96 60 RoBERTa (T,U) 128 1e-4 1e-5 0.5 96 60 BotRGCN (T,G) 256 5e-4 5e-5 0.5 128 300 BotRGCN (F,T,U,G) 128 5e-4 1e-3 0.3 128 300 GAT (T,G) 96 2e-4 5e-5 0.5 128 300 GAT (F,T,U,G) 96 2e-4 5e-5 0.5 256 300 GCN (T,G) 256 2e-4 1e-4 0.3 128 300 GCN (F,T,U,G) 256 5e-4 1e-3 0.3 128 300 Table 9: Hyperparameter settings for Twitter bot detection models. Supplementary Results of the Ablation Experiments We report supplementary ablation results in Table 10, which are omitted from the main text due to space limitations. Table 10 evaluates ValueGraph under different training configurations. Using only Stage 1 yields limited performance across tasks. Adding the clustering loss ℒclsL_cls generally improves task-aware discrimination, while incorporating the user-level consistency loss ℒuserL_user further helps capture user interaction structure and value-aware relational information. Overall, the full model achieves the strongest and most stable performance across models and tasks, supporting the effectiveness and generalizability of ValueGraph. Task Model/dataset Configuration Accuracy F1-score Stance Detection GLAN (MT_CSD) Only Stage 1 41.2 40.7 Stage 1 + ℒclsL_cls 50.4 34.3 Stage 1 + ℒuserL_user 45.3 32.6 Full 63.0 58.0 BrLSTM (RumourEval19) Only Stage 1 53.7 55.8 Stage 1 + ℒclsL_cls 62.7 52.8 Stage 1 + ℒuserL_user 53.9 45.7 Full 77.1 71.2 Twitter Bot Detection RoBERTa (Twibot-22) Only Stage 1 63.6 68.9 Stage 1 + ℒclsL_cls 64.9 67.8 Stage 1 + ℒuserL_user 64.7 67.5 Full 65.9 68.9 BotGCN (Twibot-22) Only Stage 1 70.3 71.0 Stage 1 + ℒclsL_cls 71.2 71.8 Stage 1 + ℒuserL_user 70.9 71.7 Full 71.1 71.1 BotRGCN (Twibot-22) Only Stage 1 71.6 73.3 Stage 1 + ℒclsL_cls 72.8 73.4 Stage 1 + ℒuserL_user 72.5 73.1 Full 73.0 74.2 BotGAT (Twibot-22) Only Stage 1 73.1 70.3 Stage 1 + ℒclsL_cls 72.9 72.6 Stage 1 + ℒuserL_user 72.6 72.0 Full 74.7 73.0 Table 10: Ablation results under different training configurations across two tasks: stance detection and Twitter bot detection. For Twitter bot detection, RoBERTa is evaluated in the UT setting, while BotGAT, BotGCN, and BotRGCN are evaluated in the FTUG setting. GPU Hours All models are run with fixed random seeds for reproducibility. Training and evaluation are conducted on a compute cluster with two NVIDIA H100 GPUs. The full suite of experiments consumes approximately 1,000 GPU hours, including two-stage ValueGraph pre-training and downstream evaluations under both baseline and ValueGraph configurations. ValueGraph training consists of Stage 1, which uses textual and network signals to learn initial representations, and Stage 2, which incorporates inferred value signals to produce final user embeddings. These embeddings replace the corresponding conventional features in downstream evaluations. t-SNE Setting For consistency across models, we apply t-SNE to project high-dimensional embeddings into two dimensions, using n_components=2, perplexity=30, n_iter=1000, and random_state=42. This configuration preserves local neighborhood structure while ensuring reproducibility. GPT-5.4 Prompt Templates We include GPT-5.4 as a text-only LLM baseline for stance detection and Twitter bot detection. GPT-5.4 receives only textual inputs, including the target, user posts, thread text, or profile text when available; it does not receive explicit reply-graph adjacency or message-passing structure. We use deterministic decoding with temperature set to 0. The prompts used for the two tasks are shown below. Stance Detection Prompt. For stance detection, the model is given the target, the source post or thread context, and the user’s available posts. It is asked to predict one of the task labels. In our default setting (text_mode=leaf), MT-CSD uses the discussion topic together with the leaf comment, and RumourEval19 uses the reply text only. Task: Determine the user’s stance toward the given target. Target: target Conversation context: thread_text User posts: user_posts Choose exactly one label from: Favor, Against, None. Return only the label. In practice, for MT-CSD we instantiate target with the discussion topic and use the leaf comment as the textual input; for RumourEval19, we use the reply text as the input and omit separate thread or user-history fields in the default configuration. The implemented MT-CSD prompt is: You are annotating stance in a social-media discussion about the topic: topic. Classify the STANCE of the following comment toward the discussion target. Use exactly one label: - favor: supports/agrees with the target stance - against: opposes/disagrees with the target stance - none: neutral, unrelated, or unclear stance Comment: text Reply with exactly one word: favor, against, or none. For RumourEval19, we replace the label set with the dataset-specific labels: Choose exactly one label from: Support, Deny, Query, Comment. Return only the label. The implemented RumourEval19 prompt is: You are annotating stance toward a rumor in social media. Classify the STANCE of the following reply (SDQC scheme). Use exactly one label: - support: supports/agrees the rumor is true - deny: refutes or disagrees with the rumor - query: asks for evidence or clarification - comment: neutral or unrelated to rumor veracity Reply text: text Reply with exactly one word: support, deny, query, or comment. Twitter Bot Detection Prompt. For bot detection, the model is given the user’s profile text and sampled posts. It is asked to classify the account as human or bot. In our TwiBot experiments, profile text is not available; we therefore provide only the user’s test-split posts, aggregated at the account level. Task: Determine whether the following Twitter account is operated by a human or a bot. User profile: profile_text User posts: sampled_posts Choose exactly one label from: Human, Bot. Return only the label. In practice, we set profile_text to empty and construct sampled_posts by concatenating all available tweets from the same user in the test split, separated by --- . The implemented prompt is: You are detecting whether a Twitter/X account is operated by a human or a bot. Read the following tweets posted by ONE account (may be truncated). Classify the account type. Use exactly one label: - human: likely a real person - bot: likely automated, spam, or bot-like Tweets from this account: text Reply with exactly one word: human or bot. For all prompts, long inputs are truncated to fit the model context window while preserving the target, profile text, source post, and the most recent or most relevant user posts. In our implementation, the truncation limit is 12,000 characters. No examples from the test set are included in the prompt.