Instant research discovery
Search and browse arXiv CS/AI/ML papers, enriched with AI-generated insights.
Generate novel research ideas grounded in real arXiv papers with Brainstorm.
PreviousNext
| Paper | Published | Area | Tags | Intel | Citations |
|---|---|---|---|---|---|
Search and browse arXiv CS/AI/ML papers, enriched with AI-generated insights.
Generate novel research ideas grounded in real arXiv papers with Brainstorm.
| Paper | Published | Area | Tags | Intel | Citations |
|---|---|---|---|---|---|
Search and browse arXiv CS/AI/ML papers, enriched with AI-generated insights.
Generate novel research ideas grounded in real arXiv papers with Brainstorm.
| Paper | Published | Area | Tags | Intel | Citations |
|---|---|---|---|---|---|
| Paper | Published | Area | Tags | Intel | Citations |
|---|---|---|---|---|---|
| Towards Uncovering How Large Language Model Works: An Explainability Perspective Fan Yang, Haiyan Zhao, Himabindu Lakkaraju, Mengnan Du Published: 2024-02-16Area: Surveys & ReviewsCitations: 26 Tags: ai-safety, alignment-training, interpretability, survey, surveys-reviews | 2024-02-16 | Surveys & Reviews | ai-safety, alignment-training, interpretability, survey, surveys-reviews | E5 / R3 (93%) | 26 |
| ArtPrompt: ASCII Art-based Jailbreak Attacks against Aligned LLMs Bhaskar Ramasubramanian, Bo Li, Fengqing Jiang, Luyao Niu Published: 2024-02-19Area: Adversarial RobustnessCitations: 215 Tags: adversarial-robustness, ai-safety, alignment-training, empirical | 2024-02-19 | Adversarial Robustness | adversarial-robustness, ai-safety, alignment-training, empirical | E5 / R3 (98%) | 215 |
| Emulated Disalignment: Safety Alignment for Large Language Models May Backfire! Chao Yang, Jiaheng Liu, Jie Liu, Wanli Ouyang Published: 2024-02-19Area: Adversarial RobustnessCitations: 35 Tags: adversarial-robustness, ai-safety, alignment-training, empirical | 2024-02-19 | Adversarial Robustness | adversarial-robustness, ai-safety, alignment-training, empirical | E5 / R3 (96%) | 35 |
| Smaug: Fixing Failure Modes of Preference Optimisation with DPO-Positive Arka Pal, Colin White, Deep Karkhanis, Manley Roberts Published: 2024-02-20Area: Alignment TrainingCitations: 219 Tags: ai-safety, alignment-training, empirical | 2024-02-20 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (97%) | 219 |
| Mitigating Fine-tuning based Jailbreak Attack with Backdoor Enhanced Safety Alignment Bo Li, Chaowei Xiao, Jiazhao Li, Jiongxiao Wang Published: 2024-02-22Area: Adversarial RobustnessCitations: 30 Tags: adversarial-robustness, ai-safety, alignment-training, empirical | 2024-02-22 | Adversarial Robustness | adversarial-robustness, ai-safety, alignment-training, empirical | E5 / R3 (95%) | 30 |
| SoFA: Shielded On-the-fly Alignment via Priority Rule Following Bowen Yu, Haiyang Yu, Hongyu Lin, Le Sun Published: 2024-02-27Area: Alignment TrainingCitations: 18 Tags: ai-safety, alignment-training, empirical | 2024-02-27 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (96%) | 18 |
| Exploring Multilingual Concepts of Human Values in Large Language Models: Is Value Alignment Consistent, Transferable and Controllable across Languages? Deyi Xiong, Shaoyang Xu, Weilong Dong, Xinwei Wu Published: 2024-02-28Area: Alignment TrainingCitations: 18 Tags: ai-safety, alignment-training, empirical | 2024-02-28 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (95%) | 18 |
| Keeping LLMs Aligned After Fine-tuning: The Crucial Role of Prompt Templates Anirudh Goyal, Dingli Yu, Haoyu Zhao, Kaifeng Lyu Published: 2024-02-28Area: Alignment TrainingCitations: 92 Tags: ai-safety, alignment-training, empirical | 2024-02-28 | Alignment Training | ai-safety, alignment-training, empirical | E6 / R3 (96%) | 92 |
| Enhancing LLM Safety via Constrained Direct Preference Optimization Xiaolin Sun, Zixuan Liu, Zizhan Zheng Published: 2024-03-04Area: Alignment TrainingCitations: 41 Tags: ai-safety, alignment-training, empirical | 2024-03-04 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (95%) | 41 |
| Overcoming Reward Overoptimization via Adversarial Policy Optimization with Lightweight Uncertainty Estimation Hongning Wang, Jean-Fran莽ois Ton, Wei Shen, Xiaoying Zhang Published: 2024-03-08Area: Alignment TrainingCitations: 23 Tags: adversarial-robustness, ai-safety, alignment-training, empirical | 2024-03-08 | Alignment Training | adversarial-robustness, ai-safety, alignment-training, empirical | E5 / R3 (96%) | 23 |
| Distract Large Language Models for Automatic Jailbreak Attack Guanhua Chen, Yan Yang, Yun Chen, Zeguan Xiao Published: 2024-03-13Area: Adversarial RobustnessCitations: 46 Tags: adversarial-robustness, ai-safety, alignment-training, empirical | 2024-03-13 | Adversarial Robustness | adversarial-robustness, ai-safety, alignment-training, empirical | E5 / R3 (96%) | 46 |
| Easy-to-Hard Generalization: Scalable Alignment Beyond Human Supervision Chuang Gan, Longhui Yu, Sean Welleck, Weiyang Liu Published: 2024-03-14Area: Scalable OversightCitations: 103 Tags: ai-safety, alignment-training, empirical, scalable-oversight | 2024-03-14 | Scalable Oversight | ai-safety, alignment-training, empirical, scalable-oversight | E5 / R3 (95%) | 103 |
| Images are Achilles' Heel of Alignment: Exploiting Visual Vulnerabilities for Jailbreaking Multimodal Large Language Models Hangyu Guo, Ji-Rong Wen, Kun Zhou, Wayne Xin Zhao Published: 2024-03-14Area: Multimodal SafetyCitations: 101 Tags: adversarial-robustness, ai-safety, alignment-training, empirical, multimodal-safety | 2024-03-14 | Multimodal Safety | adversarial-robustness, ai-safety, alignment-training, empirical, multimodal-safety | E5 / R3 (94%) | 101 |
| Scaling Data Diversity for Fine-Tuning Language Models in Human Alignment Bowen Yu, Feifan Song, Fei Huang, Haiyang Yu Published: 2024-03-17Area: Alignment TrainingCitations: 25 Tags: ai-safety, alignment-training, empirical | 2024-03-17 | Alignment Training | ai-safety, alignment-training, empirical | E7 / R4 (94%) | 25 |
| Rejection Improves Reliability: Training LLMs to Refuse Unknown Questions Using RL from Knowledge Feedback Da Ma, Hongshen Xu, Kai Yu, Lu Chen Published: 2024-03-27Area: Alignment TrainingCitations: 63 Tags: ai-safety, alignment-training, empirical | 2024-03-27 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (96%) | 63 |
| Understanding the Learning Dynamics of Alignment with Human Feedback Shawn Im, Yixuan Li Published: 2024-03-27Area: Alignment TrainingCitations: 18 Tags: ai-safety, alignment-training, theoretical | 2024-03-27 | Alignment Training | ai-safety, alignment-training, theoretical | E5 / R3 (95%) | 18 |
| Developing Safe and Responsible Large Language Models - A Comprehensive Framework Deepak John Reji, Fatemeh Tavakoli, Oluwanifemi Bamgbose, Shaina Raza Published: 2024-04-01Area: Alignment TrainingCitations: 5 Tags: ai-safety, alignment-training, empirical | 2024-04-01 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (95%) | 5 |
| Regularized Best-of-N Sampling to Mitigate Reward Hacking for Language Model Alignment Kaito Ariu, Kenshi Abe, Tetsuro Morimura, Yuu Jinnai Published: 2024-04-01Area: Deception & FailureCitations: 11 Tags: ai-safety, alignment-training, deception-failure, empirical | 2024-04-01 | Deception & Failure | ai-safety, alignment-training, deception-failure, empirical | E6 / R3 (97%) | 11 |
| What's in Your "Safe" Data? Identifying Benign Data that Breaks Safety Luxi He, Mengzhou Xia, Peter Henderson Published: 2024-04-01Area: Adversarial RobustnessCitations: 95 Tags: adversarial-robustness, ai-safety, alignment-training, empirical | 2024-04-01 | Adversarial Robustness | adversarial-robustness, ai-safety, alignment-training, empirical | E5 / R3 (94%) | 95 |
| Eraser: Jailbreaking Defense in Large Language Models via Unlearning Harmful Knowledge Cen Chen, Huiping Zhuang, Jianwei Wang, Weikai Lu Published: 2024-04-08Area: Adversarial RobustnessCitations: 49 Tags: adversarial-robustness, ai-safety, alignment-training, empirical | 2024-04-08 | Adversarial Robustness | adversarial-robustness, ai-safety, alignment-training, empirical | E5 / R3 (95%) | 49 |
| Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearning Licong Lin, Ruiqi Zhang, Song Mei, Yu Bai Published: 2024-04-08Area: Model EditingCitations: 348 Tags: ai-safety, alignment-training, empirical, model-editing | 2024-04-08 | Model Editing | ai-safety, alignment-training, empirical, model-editing | E5 / R3 (96%) | 348 |
| RLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMs Ameet Deshpande, Ashwin Kalyan, Bruno Castro da Silva, Karthik Narasimhan Published: 2024-04-12Area: Alignment TrainingCitations: 105 Tags: ai-safety, alignment-training, survey | 2024-04-12 | Alignment Training | ai-safety, alignment-training, survey | E5 / R4 (95%) | 105 |
| Foundational Challenges in Assuring Alignment and Safety of Large Language Models Abulhair Saparov, Alan Chan, Aleksandar Petrov, Alexander Pan Published: 2024-04-15Area: Surveys & ReviewsCitations: 211 Tags: ai-safety, alignment-training, survey, surveys-reviews | 2024-04-15 | Surveys & Reviews | ai-safety, alignment-training, survey, surveys-reviews | E6 / R4 (94%) | 211 |
| Social Choice Should Guide AI Alignment in Dealing with Diverse Human Feedback Bob M. Jacobs, Emanuel Tewolde, Eric Pacuit, Hailey Schoelkopf Published: 2024-04-16Area: Alignment TrainingCitations: 69 Tags: ai-safety, alignment-training, position | 2024-04-16 | Alignment Training | ai-safety, alignment-training, position | E5 / R3 (94%) | 69 |
| Stepwise Alignment for Constrained Language Model Policy Optimization Akifumi Wachi, Rei Sato, Takumi Tanabe, Thien Q Tran Published: 2024-04-17Area: Alignment TrainingCitations: 17 Tags: ai-safety, alignment-training, empirical | 2024-04-17 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (97%) | 17 |
| Weak-to-Strong Extrapolation Expedites Alignment Chujie Zheng, Heng Ji, Minlie Huang, Nanyun Peng Published: 2024-04-25Area: Alignment TrainingCitations: 34 Tags: ai-safety, alignment-training, empirical | 2024-04-25 | Alignment Training | ai-safety, alignment-training, empirical | E6 / R3 (96%) | 34 |
| More RLHF, More Trust? On The Impact of Preference Alignment On Trustworthiness Aaron J. Li, Himabindu Lakkaraju, Satyapriya Krishna Published: 2024-04-29Area: Alignment TrainingCitations: 10 Tags: ai-safety, alignment-training, empirical | 2024-04-29 | Alignment Training | ai-safety, alignment-training, empirical | E6 / R4 (96%) | 10 |
| Countering Reward Over-optimization in LLM with Demonstration-Guided Reinforcement Learning Emmanuel Dupoux, Florian Strub, Mathieu Rita, Olivier Pietquin Published: 2024-04-30Area: Alignment TrainingCitations: 15 Tags: ai-safety, alignment-training, empirical | 2024-04-30 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (95%) | 15 |
| A Causal Explainable Guardrails for Large Language Models Kui Ren, Longfei Li, Yan Wang, Zhan Qin Published: 2024-05-07Area: Adversarial RobustnessCitations: 17 Tags: adversarial-robustness, ai-safety, alignment-training, empirical | 2024-05-07 | Adversarial Robustness | adversarial-robustness, ai-safety, alignment-training, empirical | E4 / R3 (94%) | 17 |
| Poser: Unmasking Alignment Faking LLMs by Manipulating Their Internals Caden Juang, Joshua Clymer, Severin Field Published: 2024-05-08Area: Deception & FailureCitations: 7 Tags: ai-safety, alignment-training, benchmark, deception-failure | 2024-05-08 | Deception & Failure | ai-safety, alignment-training, benchmark, deception-failure | E4 / R3 (96%) | 7 |