Instant research discovery
Search and browse arXiv CS/AI/ML papers, enriched with AI-generated insights.
Generate novel research ideas grounded in real arXiv papers with Brainstorm.
PreviousNext
| Paper | Published | Area | Tags | Intel | Citations |
|---|---|---|---|---|---|
Search and browse arXiv CS/AI/ML papers, enriched with AI-generated insights.
Generate novel research ideas grounded in real arXiv papers with Brainstorm.
| Paper | Published | Area | Tags | Intel | Citations |
|---|---|---|---|---|---|
Search and browse arXiv CS/AI/ML papers, enriched with AI-generated insights.
Generate novel research ideas grounded in real arXiv papers with Brainstorm.
| Paper | Published | Area | Tags | Intel | Citations |
|---|---|---|---|---|---|
| Paper | Published | Area | Tags | Intel | Citations |
|---|---|---|---|---|---|
| Equilibrate RLHF: Towards Balancing Helpfulness-Safety Trade-off in Large Language Models Bo Zheng, Jiaheng Liu, Wenbo Su, Xiangyu Yue Published: 2025-02-17Area: Alignment TrainingCitations: 18 Tags: ai-safety, alignment-training, empirical | 2025-02-17 | Alignment Training | ai-safety, alignment-training, empirical | E6 / R3 (95%) | 18 |
| Safe at the Margins: A General Approach to Safety Alignment in Low-Resource English Languages - A Singlish Case Study Goh Jiayi, Isaac Lim, Jessica Foo, Shaun Khoo Published: 2025-02-18Area: Alignment TrainingCitations: 4 Tags: ai-safety, alignment-training, empirical | 2025-02-18 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (96%) | 4 |
| SEA: Low-Resource Safety Alignment for Multimodal Large Language Models via Synthetic Embeddings Cen Chen, Hao Peng, Huiping Zhuang, Shien-Ming Wu Published: 2025-02-18Area: Multimodal SafetyCitations: 5 Tags: ai-safety, alignment-training, empirical, multimodal-safety | 2025-02-18 | Multimodal Safety | ai-safety, alignment-training, empirical, multimodal-safety | E5 / R3 (97%) | 5 |
| A Mousetrap: Fooling Large Reasoning Models for Jailbreak with Chain of Iterative Chaos Liang Liu, Lujundong Li, Ruofan Wang, Xuan Tong Published: 2025-02-19Area: Adversarial RobustnessCitations: 27 Tags: adversarial-robustness, ai-safety, alignment-training, empirical | 2025-02-19 | Adversarial Robustness | adversarial-robustness, ai-safety, alignment-training, empirical | E5 / R3 (96%) | 27 |
| Attention Eclipse: Manipulating Attention to Bypass LLM Safety-Alignment Ihsen Alouani, Md Abdullah Al Mamun, Nael Abu-Ghazaleh, Pedram Zaree Published: 2025-02-21Area: Adversarial RobustnessCitations: 3 Tags: adversarial-robustness, ai-safety, alignment-training, empirical | 2025-02-21 | Adversarial Robustness | adversarial-robustness, ai-safety, alignment-training, empirical | E5 / R3 (95%) | 3 |
| C3AI: Crafting and Evaluating Constitutions for Constitutional AI Daniele Quercia, Edyta Bogucka, Ke Zhou, Yara Kyrychenko Published: 2025-02-21Area: Alignment TrainingCitations: 15 Tags: ai-safety, alignment-training, empirical | 2025-02-21 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (97%) | 15 |
| SAE-V: Interpreting Multimodal Models for Enhanced Alignment Changye Li, Hantao Lou, Jiaming Ji, Yaodong Yang Published: 2025-02-22Area: Mechanistic Interp.Citations: 8 Tags: ai-safety, alignment-training, empirical, mechanistic-interp | 2025-02-22 | Mechanistic Interp. | ai-safety, alignment-training, empirical, mechanistic-interp | E4 / R3 (96%) | 8 |
| Emergent Misalignment: Narrow Finetuning Can Produce Broadly Misaligned LLMs Anna Sztyber-Betley, Daniel Tan, Jan Betley, Mart脙颅n Soto Published: 2025-02-24Area: Deception & FailureCitations: 113 Tags: ai-safety, alignment-training, deception-failure, empirical | 2025-02-24 | Deception & Failure | ai-safety, alignment-training, deception-failure, empirical | E5 / R3 (95%) | 113 |
| One-shot Optimized Steering Vectors Mediate Safety-relevant Behaviors in LLMs Arman Cohan, Jacob Dunefsky Published: 2025-02-26Area: Representation AnalysisCitations: 9 Tags: adversarial-robustness, ai-safety, alignment-training, empirical, representation-analysis | 2025-02-26 | Representation Analysis | adversarial-robustness, ai-safety, alignment-training, empirical, representation-analysis | E6 / R3 (94%) | 9 |
| Reward Shaping to Mitigate Reward Hacking in RLHF Chengyuan Yao, Heng Wang, Jiayi Fu, Qi Han Published: 2025-02-26Area: Alignment TrainingCitations: 54 Tags: ai-safety, alignment-training, empirical | 2025-02-26 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (97%) | 54 |
| The Rise of Darkness: Safety-Utility Trade-Offs in Role-Playing Dialogue Agents Bo Wang, Jie Liu, Kehai Chen, Min Zhang Published: 2025-02-28Area: Alignment TrainingCitations: 3 Tags: ai-safety, alignment-training, empirical | 2025-02-28 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (93%) | 3 |
| Safety Tax: Safety Alignment Makes Your Large Reasoning Models Less Reasonable Fatih Ilhan, Ling Liu, Selim Furkan Tekin, Sihao Hu Published: 2025-03-01Area: Alignment TrainingCitations: 80 Tags: ai-safety, alignment-training, empirical | 2025-03-01 | Alignment Training | ai-safety, alignment-training, empirical | E6 / R4 (96%) | 80 |
| LLM Misalignment via Adversarial RLHF Platforms Ali Naseh, Erfan Entezami Published: 2025-03-04Area: Alignment TrainingCitations: 3 Tags: adversarial-robustness, ai-safety, alignment-training, empirical | 2025-03-04 | Alignment Training | adversarial-robustness, ai-safety, alignment-training, empirical | E6 / R4 (94%) | 3 |
| Improving LLM Safety Alignment with Dual-Objective Optimization David Huang, Dawn Song, Licong Lin, Song Mei Published: 2025-03-05Area: Alignment TrainingCitations: 22 Tags: adversarial-robustness, ai-safety, alignment-training, empirical | 2025-03-05 | Alignment Training | adversarial-robustness, ai-safety, alignment-training, empirical | E6 / R3 (96%) | 22 |
| SafeVLA: Towards Safety Alignment of Vision-Language-Action Model via Safe Reinforcement Learning Borong Zhang, Jiaming Ji, Josef Dai, Yaodong Yang Published: 2025-03-05Area: Agent SafetyCitations: 9 Tags: agent-safety, ai-safety, alignment-training, empirical | 2025-03-05 | Agent Safety | agent-safety, ai-safety, alignment-training, empirical | E5 / R3 (98%) | 9 |
| The MASK Benchmark: Disentangling Honesty From Accuracy in AI Systems Adam Khoja, Alice Gatti, Arunim Agarwal, Brad Kenstler Published: 2025-03-05Area: Safety EvaluationCitations: 26 Tags: ai-safety, alignment-training, benchmark, safety-evaluation | 2025-03-05 | Safety Evaluation | ai-safety, alignment-training, benchmark, safety-evaluation | E5 / R3 (97%) | 26 |
| Adding Alignment Control to Language Models Rui Wang, Weinan Zhang, Wenhong Zhu Published: 2025-03-06Area: Alignment TrainingCitations: - Tags: ai-safety, alignment-training, empirical | 2025-03-06 | Alignment Training | ai-safety, alignment-training, empirical | E8 / R6 (94%) | - |
| Safety is Not Only About Refusal: Reasoning-Enhanced Fine-tuning for Interpretable LLM Safety Ding Zhao, Miao Li, William Han, Yihang Yao Published: 2025-03-06Area: Alignment TrainingCitations: 21 Tags: adversarial-robustness, ai-safety, alignment-training, empirical | 2025-03-06 | Alignment Training | adversarial-robustness, ai-safety, alignment-training, empirical | E5 / R3 (96%) | 21 |
| Research Superalignment Should Advance Now with Alternating Competence and Conformity Optimization HyunJin Kim, James Evans, Jing Yao, JinYeong Bak Published: 2025-03-08Area: Scalable OversightCitations: - Tags: ai-safety, alignment-training, position, scalable-oversight | 2025-03-08 | Scalable Oversight | ai-safety, alignment-training, position, scalable-oversight | E6 / R3 (93%) | - |
| Mitigating Preference Hacking in Policy Optimization with Pessimism Adam Fisch, Alekh Agarwal, Christoph Dann, Dhawal Gupta Published: 2025-03-10Area: Alignment TrainingCitations: 3 Tags: ai-safety, alignment-training, theoretical | 2025-03-10 | Alignment Training | ai-safety, alignment-training, theoretical | E6 / R3 (95%) | 3 |
| Efficient Safety Alignment of Large Language Models via Preference Re-ranking and Representation-based Reward Modeling Kehai Chen, Liqiang Nie, Min Zhang, Qiyuan Deng Published: 2025-03-13Area: Alignment TrainingCitations: 2 Tags: ai-safety, alignment-training, empirical | 2025-03-13 | Alignment Training | ai-safety, alignment-training, empirical | E6 / R3 (95%) | 2 |
| SafeMERGE: Preserving Safety Alignment in Fine-Tuned Large Language Models via Selective Layer-Wise Model Merging Aladin Djuhera, Farhan Ahmed, Holger Boche, Swanand Ravindra Kadhe Published: 2025-03-21Area: Alignment TrainingCitations: 21 Tags: ai-safety, alignment-training, empirical | 2025-03-21 | Alignment Training | ai-safety, alignment-training, empirical | E7 / R3 (97%) | 21 |
| Safe RLHF-V: Safe Reinforcement Learning from Human Feedback in Multimodal Large Language Models Boyuan Chen, Chi-Min Chan, Conghui Zhang, Donghai Hong Published: 2025-03-22Area: Alignment TrainingCitations: 9 Tags: ai-safety, alignment-training, empirical | 2025-03-22 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R4 (96%) | 9 |
| Mitigating Reward Over-Optimization in RLHF via Behavior-Supported Regularization Gang Pan, Juntao Dai, Qian Zheng, Taiye Chen Published: 2025-03-23Area: Alignment TrainingCitations: 6 Tags: ai-safety, alignment-training, empirical | 2025-03-23 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (95%) | 6 |
| ENCORE: Entropy-guided Reward Composition for Multi-head Safety Reward Models Eric Hanchen Jiang, Jingxuan Fan, Mingye Gao, Xiaomin Li Published: 2025-03-26Area: Alignment TrainingCitations: 5 Tags: ai-safety, alignment-training, empirical | 2025-03-26 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (95%) | 5 |
| Misaligned Roles, Misplaced Images: Structural Input Perturbations Expose Multimodal Alignment Blind Spots Erfan Shayegani, G M Shahariar, Lei Yu, Nael Abu-Ghazaleh Published: 2025-04-01Area: Multimodal SafetyCitations: - Tags: ai-safety, alignment-training, empirical, multimodal-safety | 2025-04-01 | Multimodal Safety | ai-safety, alignment-training, empirical, multimodal-safety | E5 / R3 (96%) | - |
| An Approach to Technical AGI Safety and Security Alexander Matt Turner, Alex Irpan, Allan Dafoe, Anca Dragan Published: 2025-04-02Area: Surveys & ReviewsCitations: 35 Tags: ai-safety, alignment-training, position, surveys-reviews | 2025-04-02 | Surveys & Reviews | ai-safety, alignment-training, position, surveys-reviews | E5 / R3 (96%) | 35 |
| STAR-1: Safer Alignment of Reasoning LLMs with 1K Data Bhavya Kailkhura, Brian R. Bartoldson, Cihang Xie, Haoqin Tu Published: 2025-04-02Area: Alignment TrainingCitations: 41 Tags: ai-safety, alignment-training, dataset | 2025-04-02 | Alignment Training | ai-safety, alignment-training, dataset | E5 / R3 (95%) | 41 |
| More is Less: The Pitfalls of Multi-Model Synthetic Preference Data in DPO Safety Alignment Ananth Grama, Bolian Li, David Cho, Junyuan Hong Published: 2025-04-03Area: Alignment TrainingCitations: 5 Tags: adversarial-robustness, ai-safety, alignment-training, empirical | 2025-04-03 | Alignment Training | adversarial-robustness, ai-safety, alignment-training, empirical | E5 / R3 (94%) | 5 |
| The Hidden Space of Safety: Understanding Preference-Tuned LLMs in Multilingual context Manasa Bharadwaj, Nikhil Verma Published: 2025-04-03Area: Safety EvaluationCitations: 4 Tags: ai-safety, alignment-training, empirical, safety-evaluation | 2025-04-03 | Safety Evaluation | ai-safety, alignment-training, empirical, safety-evaluation | E8 / R3 (96%) | 4 |