Instant research discovery
Search and browse arXiv CS/AI/ML papers, enriched with AI-generated insights.
Generate novel research ideas grounded in real arXiv papers with Brainstorm.
PreviousNext
| Paper | Published | Area | Tags | Intel | Citations |
|---|---|---|---|---|---|
Search and browse arXiv CS/AI/ML papers, enriched with AI-generated insights.
Generate novel research ideas grounded in real arXiv papers with Brainstorm.
| Paper | Published | Area | Tags | Intel | Citations |
|---|---|---|---|---|---|
Search and browse arXiv CS/AI/ML papers, enriched with AI-generated insights.
Generate novel research ideas grounded in real arXiv papers with Brainstorm.
| Paper | Published | Area | Tags | Intel | Citations |
|---|---|---|---|---|---|
| Paper | Published | Area | Tags | Intel | Citations |
|---|---|---|---|---|---|
| Developing Safe and Responsible Large Language Models - A Comprehensive Framework Deepak John Reji, Fatemeh Tavakoli, Oluwanifemi Bamgbose, Shaina Raza Published: 2024-04-01Area: Alignment TrainingCitations: 5 Tags: ai-safety, alignment-training, empirical | 2024-04-01 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (95%) | 5 |
| Regularized Best-of-N Sampling to Mitigate Reward Hacking for Language Model Alignment Kaito Ariu, Kenshi Abe, Tetsuro Morimura, Yuu Jinnai Published: 2024-04-01Area: Deception & FailureCitations: 11 Tags: ai-safety, alignment-training, deception-failure, empirical | 2024-04-01 | Deception & Failure | ai-safety, alignment-training, deception-failure, empirical | E6 / R3 (97%) | 11 |
| What's in Your "Safe" Data? Identifying Benign Data that Breaks Safety Luxi He, Mengzhou Xia, Peter Henderson Published: 2024-04-01Area: Adversarial RobustnessCitations: 95 Tags: adversarial-robustness, ai-safety, alignment-training, empirical | 2024-04-01 | Adversarial Robustness | adversarial-robustness, ai-safety, alignment-training, empirical | E5 / R3 (94%) | 95 |
| Rejection Improves Reliability: Training LLMs to Refuse Unknown Questions Using RL from Knowledge Feedback Da Ma, Hongshen Xu, Kai Yu, Lu Chen Published: 2024-03-27Area: Alignment TrainingCitations: 63 Tags: ai-safety, alignment-training, empirical | 2024-03-27 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (96%) | 63 |
| Understanding the Learning Dynamics of Alignment with Human Feedback Shawn Im, Yixuan Li Published: 2024-03-27Area: Alignment TrainingCitations: 18 Tags: ai-safety, alignment-training, theoretical | 2024-03-27 | Alignment Training | ai-safety, alignment-training, theoretical | E5 / R3 (95%) | 18 |
| Scaling Data Diversity for Fine-Tuning Language Models in Human Alignment Bowen Yu, Feifan Song, Fei Huang, Haiyang Yu Published: 2024-03-17Area: Alignment TrainingCitations: 25 Tags: ai-safety, alignment-training, empirical | 2024-03-17 | Alignment Training | ai-safety, alignment-training, empirical | E7 / R4 (94%) | 25 |
| Easy-to-Hard Generalization: Scalable Alignment Beyond Human Supervision Chuang Gan, Longhui Yu, Sean Welleck, Weiyang Liu Published: 2024-03-14Area: Scalable OversightCitations: 103 Tags: ai-safety, alignment-training, empirical, scalable-oversight | 2024-03-14 | Scalable Oversight | ai-safety, alignment-training, empirical, scalable-oversight | E5 / R3 (95%) | 103 |
| Images are Achilles' Heel of Alignment: Exploiting Visual Vulnerabilities for Jailbreaking Multimodal Large Language Models Hangyu Guo, Ji-Rong Wen, Kun Zhou, Wayne Xin Zhao Published: 2024-03-14Area: Multimodal SafetyCitations: 101 Tags: adversarial-robustness, ai-safety, alignment-training, empirical, multimodal-safety | 2024-03-14 | Multimodal Safety | adversarial-robustness, ai-safety, alignment-training, empirical, multimodal-safety | E5 / R3 (94%) | 101 |
| Distract Large Language Models for Automatic Jailbreak Attack Guanhua Chen, Yan Yang, Yun Chen, Zeguan Xiao Published: 2024-03-13Area: Adversarial RobustnessCitations: 46 Tags: adversarial-robustness, ai-safety, alignment-training, empirical | 2024-03-13 | Adversarial Robustness | adversarial-robustness, ai-safety, alignment-training, empirical | E5 / R3 (96%) | 46 |
| Overcoming Reward Overoptimization via Adversarial Policy Optimization with Lightweight Uncertainty Estimation Hongning Wang, Jean-Fran莽ois Ton, Wei Shen, Xiaoying Zhang Published: 2024-03-08Area: Alignment TrainingCitations: 23 Tags: adversarial-robustness, ai-safety, alignment-training, empirical | 2024-03-08 | Alignment Training | adversarial-robustness, ai-safety, alignment-training, empirical | E5 / R3 (96%) | 23 |
| Enhancing LLM Safety via Constrained Direct Preference Optimization Xiaolin Sun, Zixuan Liu, Zizhan Zheng Published: 2024-03-04Area: Alignment TrainingCitations: 41 Tags: ai-safety, alignment-training, empirical | 2024-03-04 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (95%) | 41 |
| Exploring Multilingual Concepts of Human Values in Large Language Models: Is Value Alignment Consistent, Transferable and Controllable across Languages? Deyi Xiong, Shaoyang Xu, Weilong Dong, Xinwei Wu Published: 2024-02-28Area: Alignment TrainingCitations: 18 Tags: ai-safety, alignment-training, empirical | 2024-02-28 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (95%) | 18 |
| Keeping LLMs Aligned After Fine-tuning: The Crucial Role of Prompt Templates Anirudh Goyal, Dingli Yu, Haoyu Zhao, Kaifeng Lyu Published: 2024-02-28Area: Alignment TrainingCitations: 92 Tags: ai-safety, alignment-training, empirical | 2024-02-28 | Alignment Training | ai-safety, alignment-training, empirical | E6 / R3 (96%) | 92 |
| SoFA: Shielded On-the-fly Alignment via Priority Rule Following Bowen Yu, Haiyang Yu, Hongyu Lin, Le Sun Published: 2024-02-27Area: Alignment TrainingCitations: 18 Tags: ai-safety, alignment-training, empirical | 2024-02-27 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (96%) | 18 |
| Mitigating Fine-tuning based Jailbreak Attack with Backdoor Enhanced Safety Alignment Bo Li, Chaowei Xiao, Jiazhao Li, Jiongxiao Wang Published: 2024-02-22Area: Adversarial RobustnessCitations: 30 Tags: adversarial-robustness, ai-safety, alignment-training, empirical | 2024-02-22 | Adversarial Robustness | adversarial-robustness, ai-safety, alignment-training, empirical | E5 / R3 (95%) | 30 |
| Smaug: Fixing Failure Modes of Preference Optimisation with DPO-Positive Arka Pal, Colin White, Deep Karkhanis, Manley Roberts Published: 2024-02-20Area: Alignment TrainingCitations: 219 Tags: ai-safety, alignment-training, empirical | 2024-02-20 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (97%) | 219 |
| ArtPrompt: ASCII Art-based Jailbreak Attacks against Aligned LLMs Bhaskar Ramasubramanian, Bo Li, Fengqing Jiang, Luyao Niu Published: 2024-02-19Area: Adversarial RobustnessCitations: 215 Tags: adversarial-robustness, ai-safety, alignment-training, empirical | 2024-02-19 | Adversarial Robustness | adversarial-robustness, ai-safety, alignment-training, empirical | E5 / R3 (98%) | 215 |
| Emulated Disalignment: Safety Alignment for Large Language Models May Backfire! Chao Yang, Jiaheng Liu, Jie Liu, Wanli Ouyang Published: 2024-02-19Area: Adversarial RobustnessCitations: 35 Tags: adversarial-robustness, ai-safety, alignment-training, empirical | 2024-02-19 | Adversarial Robustness | adversarial-robustness, ai-safety, alignment-training, empirical | E5 / R3 (96%) | 35 |
| Towards Uncovering How Large Language Model Works: An Explainability Perspective Fan Yang, Haiyan Zhao, Himabindu Lakkaraju, Mengnan Du Published: 2024-02-16Area: Surveys & ReviewsCitations: 26 Tags: ai-safety, alignment-training, interpretability, survey, surveys-reviews | 2024-02-16 | Surveys & Reviews | ai-safety, alignment-training, interpretability, survey, surveys-reviews | E5 / R3 (93%) | 26 |
| Recovering the Pre-Fine-Tuning Weights of Generative Models Eliahu Horwitz, Jonathan Kahana, Yedid Hoshen Published: 2024-02-15Area: Adversarial RobustnessCitations: 13 Tags: adversarial-robustness, ai-safety, alignment-training, empirical | 2024-02-15 | Adversarial Robustness | adversarial-robustness, ai-safety, alignment-training, empirical | E5 / R3 (95%) | 13 |
| Reward Generalization in RLHF: A Topological Perspective Dong Yan, Fanzhi Zeng, Han Yang, Jiaming Ji Published: 2024-02-15Area: Alignment TrainingCitations: 7 Tags: ai-safety, alignment-training, theoretical | 2024-02-15 | Alignment Training | ai-safety, alignment-training, theoretical | E5 / R3 (95%) | 7 |
| Attacks, Defenses and Evaluations for LLM Conversation Safety: A Survey Chao Yang, Jing Shao, Yu Qiao, Zhanhui Zhou Published: 2024-02-14Area: Surveys & ReviewsCitations: 140 Tags: ai-safety, alignment-training, safety-evaluation, survey, surveys-reviews | 2024-02-14 | Surveys & Reviews | ai-safety, alignment-training, safety-evaluation, survey, surveys-reviews | E5 / R3 (98%) | 140 |
| InfoRM: Mitigating Reward Hacking in RLHF via Information-Theoretic Reward Modeling Dacheng Tao, Lefei Zhang, Liang Ding, Rong Bao Published: 2024-02-14Area: Alignment TrainingCitations: 63 Tags: ai-safety, alignment-training, empirical | 2024-02-14 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (96%) | 63 |
| Soft Prompt Threats: Attacking Safety Alignment and Unlearning in Open-Source LLMs through the Embedding Space David Dobre, Gauthier Gidel, Leo Schwinn, Sophie Xhonneux Published: 2024-02-14Area: Adversarial RobustnessCitations: 83 Tags: adversarial-robustness, ai-safety, alignment-training, empirical | 2024-02-14 | Adversarial Robustness | adversarial-robustness, ai-safety, alignment-training, empirical | E5 / R3 (95%) | 83 |
| Rethinking Machine Unlearning for Large Language Models Chris Yuhao Liu, Hang Li, Jinghan Jia, Kush R. Varshney Published: 2024-02-13Area: Model EditingCitations: 227 Tags: ai-safety, alignment-training, model-editing, safety-evaluation, survey | 2024-02-13 | Model Editing | ai-safety, alignment-training, model-editing, safety-evaluation, survey | E5 / R3 (95%) | 227 |
| ODIN: Disentangled Reward Mitigates Hacking in RLHF Bryan Catanzaro, Chen Zhu, Davit Soselia, Heng Huang Published: 2024-02-11Area: Alignment TrainingCitations: 111 Tags: ai-safety, alignment-training, empirical | 2024-02-11 | Alignment Training | ai-safety, alignment-training, empirical | E6 / R4 (95%) | 111 |
| Self-Alignment of Large Language Models via Monopolylogue-based Social Scene Simulation Bolun Zhang, Rui Ye, Shuo Tang, Siheng Chen Published: 2024-02-08Area: Alignment TrainingCitations: 49 Tags: ai-safety, alignment-training, empirical | 2024-02-08 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R4 (94%) | 49 |
| A Roadmap to Pluralistic Alignment Andre Ye, Christopher Michael Rytting, Jared Moore, Jillian Fisher Published: 2024-02-07Area: Alignment TrainingCitations: 161 Tags: ai-safety, alignment-training, survey | 2024-02-07 | Alignment Training | ai-safety, alignment-training, survey | E5 / R3 (94%) | 161 |
| Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modifications Boyi Wei, Kaixuan Huang, Mengdi Wang, Mengzhou Xia Published: 2024-02-07Area: Adversarial RobustnessCitations: 188 Tags: adversarial-robustness, ai-safety, alignment-training, empirical | 2024-02-07 | Adversarial Robustness | adversarial-robustness, ai-safety, alignment-training, empirical | E5 / R3 (96%) | 188 |
| Direct Language Model Alignment from Online AI Feedback Alexandre Rame, Biao Zhang, Bilal Piot, Felipe Llinares Published: 2024-02-07Area: Alignment TrainingCitations: 226 Tags: ai-safety, alignment-training, empirical | 2024-02-07 | Alignment Training | ai-safety, alignment-training, empirical | E6 / R4 (97%) | 226 |