Instant research discovery
Search and browse arXiv CS/AI/ML papers, enriched with AI-generated insights.
Generate novel research ideas grounded in real arXiv papers with Brainstorm.
PreviousNext
| Paper | Published | Area | Tags | Intel | Citations |
|---|---|---|---|---|---|
Search and browse arXiv CS/AI/ML papers, enriched with AI-generated insights.
Generate novel research ideas grounded in real arXiv papers with Brainstorm.
| Paper | Published | Area | Tags | Intel | Citations |
|---|---|---|---|---|---|
Search and browse arXiv CS/AI/ML papers, enriched with AI-generated insights.
Generate novel research ideas grounded in real arXiv papers with Brainstorm.
| Paper | Published | Area | Tags | Intel | Citations |
|---|---|---|---|---|---|
| Paper | Published | Area | Tags | Intel | Citations |
|---|---|---|---|---|---|
| OpenAssistant Conversations - Democratizing Large Language Model Alignment Abdelpakey Mohamed Hassan, Alexander Mattick, Alexander Rush, Andreas K枚pf Published: 2023-04-14Area: Alignment TrainingCitations: 811 Tags: ai-safety, alignment-training, dataset | 2023-04-14 | Alignment Training | ai-safety, alignment-training, dataset | E5 / R3 (95%) | 811 |
| Fundamental Limitations of Alignment in Large Language Models Amnon Shashua, Noam Wies, Oshri Avnery, Yoav Levine Published: 2023-04-19Area: Formal/TheoreticalCitations: 178 Tags: adversarial-robustness, ai-safety, alignment-training, formaltheoretical, theoretical | 2023-04-19 | Formal/Theoretical | adversarial-robustness, ai-safety, alignment-training, formaltheoretical, theoretical | E4 / R2 (94%) | 178 |
| Principle-Driven Self-Alignment of Language Models from Scratch with Minimal Human Supervision Chuang Gan, David Cox, Hongxin Zhang, Qinhong Zhou Published: 2023-05-04Area: Alignment TrainingCitations: 410 Tags: ai-safety, alignment-training, empirical | 2023-05-04 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (97%) | 410 |
| KGA: A General Machine Unlearning Framework Based on Knowledge Gap Alignment Hongzhi Yin, Kam-Fai Wong, Lingzhi Wang, Tong Chen Published: 2023-05-11Area: Model EditingCitations: 100 Tags: ai-safety, alignment-training, empirical, model-editing | 2023-05-11 | Model Editing | ai-safety, alignment-training, empirical, model-editing | E7 / R4 (96%) | 100 |
| Interpretability at Scale: Identifying Causal Mechanisms in Alpaca Atticus Geiger, Christopher Potts, Noah D. Goodman, Thomas Icard Published: 2023-05-15Area: Mechanistic Interp.Citations: 113 Tags: ai-safety, alignment-training, empirical, interpretability, mechanistic-interp | 2023-05-15 | Mechanistic Interp. | ai-safety, alignment-training, empirical, interpretability, mechanistic-interp | E4 / R3 (95%) | 113 |
| SLiC-HF: Sequence Likelihood Calibration with Human Feedback Misha Khalman, Mohammad Saleh, Peter J. Liu, Rishabh Joshi Published: 2023-05-17Area: Alignment TrainingCitations: 384 Tags: ai-safety, alignment-training, empirical | 2023-05-17 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (94%) | 384 |
| AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedback Carlos Guestrin, Ishaan Gulrajani, Jimmy Ba, Percy Liang Published: 2023-05-22Area: Alignment TrainingCitations: 797 Tags: ai-safety, alignment-training, tool | 2023-05-22 | Alignment Training | ai-safety, alignment-training, tool | E5 / R3 (96%) | 797 |
| Model Evaluation for Extreme Risks Allan Dafoe, Been Kim, Ben Garfinkel, Daniel Kokotajlo Published: 2023-05-24Area: Safety EvaluationCitations: 201 Tags: ai-safety, alignment-training, position, safety-evaluation | 2023-05-24 | Safety Evaluation | ai-safety, alignment-training, position, safety-evaluation | E5 / R4 (93%) | 201 |
| Training Socially Aligned Language Models in Simulated Human Society Andrew M. Dai, Chenyan Jia, Denny Zhou, Diyi Yang Published: 2023-05-26Area: Alignment TrainingCitations: 91 Tags: ai-safety, alignment-training, empirical | 2023-05-26 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (95%) | 91 |
| Direct Preference Optimization: Your Language Model is Secretly a Reward Model Archit Sharma, Chelsea Finn, Christopher D. Manning, Eric Mitchell Published: 2023-05-29Area: Alignment TrainingCitations: 7298 Tags: ai-safety, alignment-training, empirical | 2023-05-29 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (96%) | 7298 |
| Let's Verify Step by Step Bowen Baker, Harri Edwards, Hunter Lightman, Ilya Sutskever Published: 2023-05-31Area: Alignment TrainingCitations: 2550 Tags: ai-safety, alignment-training, empirical | 2023-05-31 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (96%) | 2550 |
| Fine-Grained Human Feedback Gives Better Rewards for Language Model Training Alane Suhr, Hannaneh Hajishirzi, Mari Ostendorf, Noah A. Smith Published: 2023-06-02Area: Alignment TrainingCitations: 430 Tags: ai-safety, alignment-training, empirical | 2023-06-02 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (95%) | 430 |
| Rewarded soups: towards Pareto-optimal alignment by interpolating weights fine-tuned on diverse rewards Alexandre Rame, Corentin Dancette, Guillaume Couairon, Jean-Baptiste Gaya Published: 2023-06-07Area: Alignment TrainingCitations: 216 Tags: ai-safety, alignment-training, empirical | 2023-06-07 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (94%) | 216 |
| Preference Ranking Optimization for Human Alignment Bowen Yu, Feifan Song, Fei Huang, Haiyang Yu Published: 2023-06-30Area: Alignment TrainingCitations: 340 Tags: ai-safety, alignment-training, empirical | 2023-06-30 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (96%) | 340 |
| BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Dataset Boyuan Chen, Ce Bian, Chi Zhang, Jiaming Ji Published: 2023-07-10Area: Alignment TrainingCitations: 767 Tags: ai-safety, alignment-training, dataset | 2023-07-10 | Alignment Training | ai-safety, alignment-training, dataset | E5 / R3 (95%) | 767 |
| Secrets of RLHF in Large Language Models Part I: PPO Binghai Wang, Cheng Chang, Hang Yan, Haoran Huang Published: 2023-07-11Area: Alignment TrainingCitations: 248 Tags: ai-safety, alignment-training, empirical | 2023-07-11 | Alignment Training | ai-safety, alignment-training, empirical | E6 / R4 (95%) | 248 |
| CValues: Measuring the Values of Chinese Large Language Models from Safety to Responsibility Chao Peng, Fei Huang, Guohai Xu, Haotian Xu Published: 2023-07-19Area: Safety EvaluationCitations: 99 Tags: adversarial-robustness, ai-safety, alignment-training, benchmark, safety-evaluation | 2023-07-19 | Safety Evaluation | adversarial-robustness, ai-safety, alignment-training, benchmark, safety-evaluation | E5 / R3 (97%) | 99 |
| Deceptive Alignment Monitoring Andres Carranza, Arnuv Tandon, Dhruv Pai, Rylan Schaeffer Published: 2023-07-20Area: Deception & FailureCitations: 14 Tags: ai-safety, alignment-training, deception-failure, position | 2023-07-20 | Deception & Failure | ai-safety, alignment-training, deception-failure, position | E4 / R3 (95%) | 14 |
| Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback Anand Siththaranjan, Anca Dragan, Andi Peng, Charbel-Raphael Segerie Published: 2023-07-27Area: Alignment TrainingCitations: 761 Tags: ai-safety, alignment-training, survey | 2023-07-27 | Alignment Training | ai-safety, alignment-training, survey | E5 / R3 (97%) | 761 |
| GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Cipher Jen-tse Huang, Pinjia He, Shuming Shi, Wenxiang Jiao Published: 2023-08-12Area: Adversarial RobustnessCitations: 408 Tags: adversarial-robustness, ai-safety, alignment-training, empirical | 2023-08-12 | Adversarial Robustness | adversarial-robustness, ai-safety, alignment-training, empirical | E5 / R3 (94%) | 408 |
| Red-Teaming Large Language Models using Chain of Utterances for Safety-Alignment Rishabh Bhardwaj, Soujanya Poria Published: 2023-08-18Area: Safety EvaluationCitations: 229 Tags: adversarial-robustness, ai-safety, alignment-training, empirical, safety-evaluation | 2023-08-18 | Safety Evaluation | adversarial-robustness, ai-safety, alignment-training, empirical, safety-evaluation | E6 / R3 (97%) | 229 |
| From Instructions to Intrinsic Human Values - A Survey of Alignment Goals for Big Models Jindong Wang, Jing Yao, Xiaoyuan Yi, Xing Xie Published: 2023-08-23Area: Surveys & ReviewsCitations: 62 Tags: ai-safety, alignment-training, survey, surveys-reviews | 2023-08-23 | Surveys & Reviews | ai-safety, alignment-training, survey, surveys-reviews | E5 / R3 (94%) | 62 |
| Identifying and Mitigating the Security Risks of Generative AI Amrita Roy Chowdhury, Ankur Taly, Anupam Datta, Brad Boyd Published: 2023-08-28Area: Surveys & ReviewsCitations: 126 Tags: ai-safety, alignment-training, survey, surveys-reviews | 2023-08-28 | Surveys & Reviews | ai-safety, alignment-training, survey, surveys-reviews | E6 / R4 (95%) | 126 |
| Efficient RLHF: Reducing the Memory Usage of PPO Han Yu, Michael Santacroce, Yadong Lu, Yelong Shen Published: 2023-09-01Area: Alignment TrainingCitations: 42 Tags: ai-safety, alignment-training, empirical | 2023-09-01 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (96%) | 42 |
| RLAIF: Scaling Reinforcement Learning from Human Feedback with AI Feedback Abhinav Rastogi, Colton Bishop, Ethan Hall, Harrison Lee Published: 2023-09-01Area: Alignment TrainingCitations: 538 Tags: ai-safety, alignment-training, empirical | 2023-09-01 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (95%) | 538 |
| RAIN: Your Language Models Can Align Themselves without Finetuning Chao Zhang, Fangyun Wei, Hongyang Zhang, Jinjing Zhao Published: 2023-09-13Area: Alignment TrainingCitations: 161 Tags: ai-safety, alignment-training, empirical, safety-evaluation | 2023-09-13 | Alignment Training | ai-safety, alignment-training, empirical, safety-evaluation | E5 / R3 (98%) | 161 |
| Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructions Dan Jurafsky, Federico Bianchi, Giuseppe Attanasio, James Zou Published: 2023-09-14Area: Alignment TrainingCitations: 343 Tags: ai-safety, alignment-training, empirical | 2023-09-14 | Alignment Training | ai-safety, alignment-training, empirical | E6 / R3 (94%) | 343 |
| Defending Against Alignment-Breaking Attacks via Robustly Aligned LLM Bochuan Cao, Jinghui Chen, Lu Lin, Yuanpu Cao Published: 2023-09-18Area: Adversarial RobustnessCitations: 211 Tags: adversarial-robustness, ai-safety, alignment-training, empirical | 2023-09-18 | Adversarial Robustness | adversarial-robustness, ai-safety, alignment-training, empirical | E4 / R2 (96%) | 211 |
| Large Language Model Alignment: A Survey Chuang Liu, Deyi Xiong, Renren Jin, Tianhao Shen Published: 2023-09-26Area: Surveys & ReviewsCitations: 292 Tags: adversarial-robustness, ai-safety, alignment-training, interpretability, safety-evaluation, survey, surveys-reviews | 2023-09-26 | Surveys & Reviews | adversarial-robustness, ai-safety, alignment-training, interpretability, safety-evaluation, survey, surveys-reviews | E6 / R4 (97%) | 292 |
| The Trickle-down Impact of Reward (In-)consistency on RLHF Baolin Peng, Daniel Khashabi, Dong Yu, Haitao Mi Published: 2023-09-28Area: Alignment TrainingCitations: 28 Tags: ai-safety, alignment-training, empirical | 2023-09-28 | Alignment Training | ai-safety, alignment-training, empirical | E6 / R3 (96%) | 28 |