Instant research discovery
Search and browse arXiv CS/AI/ML papers, enriched with AI-generated insights.
Generate novel research ideas grounded in real arXiv papers with Brainstorm.
PreviousNext
| Paper | Published | Area | Tags | Intel | Citations |
|---|---|---|---|---|---|
Search and browse arXiv CS/AI/ML papers, enriched with AI-generated insights.
Generate novel research ideas grounded in real arXiv papers with Brainstorm.
| Paper | Published | Area | Tags | Intel | Citations |
|---|---|---|---|---|---|
Search and browse arXiv CS/AI/ML papers, enriched with AI-generated insights.
Generate novel research ideas grounded in real arXiv papers with Brainstorm.
| Paper | Published | Area | Tags | Intel | Citations |
|---|---|---|---|---|---|
| Paper | Published | Area | Tags | Intel | Citations |
|---|---|---|---|---|---|
| Learning and Forgetting Unsafe Examples in Large Language Models David Madras, James Zou, Jiachen Zhao, Mengye Ren Published: 2023-12-20Area: Alignment TrainingCitations: 25 Tags: ai-safety, alignment-training, empirical | 2023-12-20 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (93%) | 25 |
| Uncertainty-Penalized Reinforcement Learning from Human Feedback with Diverse Reward LoRA Ensembles Bo Ding, Dawei Feng, Han Zhang, Huaimin Wang Published: 2023-12-30Area: Alignment TrainingCitations: 43 Tags: ai-safety, alignment-training, empirical | 2023-12-30 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (96%) | 43 |
| A Mechanistic Understanding of Alignment Algorithms: A Case Study on DPO and Toxicity Andrew Lee, Itamar Pres, Jonathan K. Kummerfeld, Martin Wattenberg Published: 2024-01-03Area: Mechanistic Interp.Citations: 165 Tags: ai-safety, alignment-training, empirical, mechanistic-interp | 2024-01-03 | Mechanistic Interp. | ai-safety, alignment-training, empirical, mechanistic-interp | E5 / R3 (94%) | 165 |
| Agent Alignment in Evolving Social Norms Qinyuan Cheng, Shimin Li, Tianxiang Sun, Xipeng Qiu Published: 2024-01-09Area: Agent SafetyCitations: 12 Tags: agent-safety, ai-safety, alignment-training, empirical | 2024-01-09 | Agent Safety | agent-safety, ai-safety, alignment-training, empirical | E5 / R3 (96%) | 12 |
| Secrets of RLHF in Large Language Models Part II: Reward Modeling Binghai Wang, Caishuang Huang, Chenyu Shi, Enyu Zhou Published: 2024-01-11Area: Alignment TrainingCitations: 148 Tags: ai-safety, alignment-training, empirical | 2024-01-11 | Alignment Training | ai-safety, alignment-training, empirical | E6 / R4 (94%) | 148 |
| Self-Rewarding Language Models Jason Weston, Jing Xu, Kyunghyun Cho, Richard Yuanzhe Pang Published: 2024-01-18Area: Alignment TrainingCitations: 504 Tags: ai-safety, alignment-training, empirical | 2024-01-18 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (96%) | 504 |
| InferAligner: Inference-Time Alignment for Harmlessness through Cross-Model Guidance Botian Jiang, Chenkun Tan, Dong Zhang, Ke Ren Published: 2024-01-20Area: Model EditingCitations: 78 Tags: ai-safety, alignment-training, empirical, model-editing | 2024-01-20 | Model Editing | ai-safety, alignment-training, empirical, model-editing | E5 / R3 (95%) | 78 |
| Linear Alignment: A Closed-form Solution for Aligning Human Preferences without Tuning and Feedback Dahua Lin, Hang Yan, Junjie Ye, Qiming Ge Published: 2024-01-21Area: Alignment TrainingCitations: 22 Tags: ai-safety, alignment-training, theoretical | 2024-01-21 | Alignment Training | ai-safety, alignment-training, theoretical | E5 / R4 (96%) | 22 |
| WARM: On the Benefits of Weight Averaged Reward Models Alexandre Ram茅, Geoffrey Cideron, Johan Ferret, L茅onard Hussenot Published: 2024-01-22Area: Alignment TrainingCitations: 134 Tags: ai-safety, alignment-training, empirical | 2024-01-22 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (94%) | 134 |
| ARGS: Alignment as Reward-Guided Search Jirayu Burapacheep, Maxim Khanov, Yixuan Li Published: 2024-01-23Area: Alignment TrainingCitations: 95 Tags: ai-safety, alignment-training, empirical | 2024-01-23 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (96%) | 95 |
| Iterative Data Smoothing: Mitigating Reward Overfitting and Overoptimization in RLHF Banghua Zhu, Jiantao Jiao, Michael I. Jordan Published: 2024-01-29Area: Alignment TrainingCitations: 49 Tags: ai-safety, alignment-training, empirical | 2024-01-29 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (95%) | 49 |
| Tradeoffs Between Alignment and Helpfulness in Language Models Amnon Shashua, Binyamin Rothberg, Dorin Shteyman, Noam Wies Published: 2024-01-29Area: Representation AnalysisCitations: 21 Tags: ai-safety, alignment-training, representation-analysis, theoretical | 2024-01-29 | Representation Analysis | ai-safety, alignment-training, representation-analysis, theoretical | E5 / R3 (94%) | 21 |
| Towards Efficient and Exact Optimization of Language Model Alignment Cheng Lu, Haozhe Ji, Hongning Wang, Jie Tang Published: 2024-02-01Area: Alignment TrainingCitations: 32 Tags: ai-safety, alignment-training, empirical | 2024-02-01 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (95%) | 32 |
| KTO: Model Alignment as Prospect Theoretic Optimization Dan Jurafsky, Douwe Kiela, Kawin Ethayarajh, Niklas Muennighoff Published: 2024-02-02Area: Alignment TrainingCitations: 879 Tags: ai-safety, alignment-training, empirical | 2024-02-02 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (97%) | 879 |
| Preference Poisoning Attacks on Reward Model Learning Chaowei Xiao, Chenguang Wang, Jiongxiao Wang, Junlin Wu Published: 2024-02-02Area: Adversarial RobustnessCitations: 12 Tags: adversarial-robustness, ai-safety, alignment-training, empirical | 2024-02-02 | Adversarial Robustness | adversarial-robustness, ai-safety, alignment-training, empirical | E5 / R4 (96%) | 12 |
| Rethinking the Role of Proxy Rewards in Language Model Alignment Minjoon Seo, Sungdong Kim Published: 2024-02-02Area: Alignment TrainingCitations: 5 Tags: ai-safety, alignment-training, empirical | 2024-02-02 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (97%) | 5 |
| Vaccine: Perturbation-aware Alignment for Large Language Models Ling Liu, Sihao Hu, Tiansheng Huang Published: 2024-02-02Area: Alignment TrainingCitations: 88 Tags: ai-safety, alignment-training, empirical | 2024-02-02 | Alignment Training | ai-safety, alignment-training, empirical | E4 / R3 (94%) | 88 |
| DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models Daya Guo, Haowei Zhang, Junxiao Song, Mingchuan Zhang Published: 2024-02-05Area: Alignment TrainingCitations: 4632 Tags: ai-safety, alignment-training, empirical | 2024-02-05 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (98%) | 4632 |
| Prioritizing Safeguarding Over Autonomy: Risks of LLM Agents for Science Arman Cohan, Jian Tang, Kunlun Zhu, Mark Gerstein Published: 2024-02-06Area: Agent SafetyCitations: 55 Tags: agent-safety, ai-safety, alignment-training, survey | 2024-02-06 | Agent Safety | agent-safety, ai-safety, alignment-training, survey | E5 / R3 (95%) | 55 |
| A Roadmap to Pluralistic Alignment Andre Ye, Christopher Michael Rytting, Jared Moore, Jillian Fisher Published: 2024-02-07Area: Alignment TrainingCitations: 161 Tags: ai-safety, alignment-training, survey | 2024-02-07 | Alignment Training | ai-safety, alignment-training, survey | E5 / R3 (94%) | 161 |
| Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modifications Boyi Wei, Kaixuan Huang, Mengdi Wang, Mengzhou Xia Published: 2024-02-07Area: Adversarial RobustnessCitations: 188 Tags: adversarial-robustness, ai-safety, alignment-training, empirical | 2024-02-07 | Adversarial Robustness | adversarial-robustness, ai-safety, alignment-training, empirical | E5 / R3 (96%) | 188 |
| Direct Language Model Alignment from Online AI Feedback Alexandre Rame, Biao Zhang, Bilal Piot, Felipe Llinares Published: 2024-02-07Area: Alignment TrainingCitations: 226 Tags: ai-safety, alignment-training, empirical | 2024-02-07 | Alignment Training | ai-safety, alignment-training, empirical | E6 / R4 (97%) | 226 |
| Self-Alignment of Large Language Models via Monopolylogue-based Social Scene Simulation Bolun Zhang, Rui Ye, Shuo Tang, Siheng Chen Published: 2024-02-08Area: Alignment TrainingCitations: 49 Tags: ai-safety, alignment-training, empirical | 2024-02-08 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R4 (94%) | 49 |
| ODIN: Disentangled Reward Mitigates Hacking in RLHF Bryan Catanzaro, Chen Zhu, Davit Soselia, Heng Huang Published: 2024-02-11Area: Alignment TrainingCitations: 111 Tags: ai-safety, alignment-training, empirical | 2024-02-11 | Alignment Training | ai-safety, alignment-training, empirical | E6 / R4 (95%) | 111 |
| Rethinking Machine Unlearning for Large Language Models Chris Yuhao Liu, Hang Li, Jinghan Jia, Kush R. Varshney Published: 2024-02-13Area: Model EditingCitations: 227 Tags: ai-safety, alignment-training, model-editing, safety-evaluation, survey | 2024-02-13 | Model Editing | ai-safety, alignment-training, model-editing, safety-evaluation, survey | E5 / R3 (95%) | 227 |
| Attacks, Defenses and Evaluations for LLM Conversation Safety: A Survey Chao Yang, Jing Shao, Yu Qiao, Zhanhui Zhou Published: 2024-02-14Area: Surveys & ReviewsCitations: 140 Tags: ai-safety, alignment-training, safety-evaluation, survey, surveys-reviews | 2024-02-14 | Surveys & Reviews | ai-safety, alignment-training, safety-evaluation, survey, surveys-reviews | E5 / R3 (98%) | 140 |
| InfoRM: Mitigating Reward Hacking in RLHF via Information-Theoretic Reward Modeling Dacheng Tao, Lefei Zhang, Liang Ding, Rong Bao Published: 2024-02-14Area: Alignment TrainingCitations: 63 Tags: ai-safety, alignment-training, empirical | 2024-02-14 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (96%) | 63 |
| Soft Prompt Threats: Attacking Safety Alignment and Unlearning in Open-Source LLMs through the Embedding Space David Dobre, Gauthier Gidel, Leo Schwinn, Sophie Xhonneux Published: 2024-02-14Area: Adversarial RobustnessCitations: 83 Tags: adversarial-robustness, ai-safety, alignment-training, empirical | 2024-02-14 | Adversarial Robustness | adversarial-robustness, ai-safety, alignment-training, empirical | E5 / R3 (95%) | 83 |
| Recovering the Pre-Fine-Tuning Weights of Generative Models Eliahu Horwitz, Jonathan Kahana, Yedid Hoshen Published: 2024-02-15Area: Adversarial RobustnessCitations: 13 Tags: adversarial-robustness, ai-safety, alignment-training, empirical | 2024-02-15 | Adversarial Robustness | adversarial-robustness, ai-safety, alignment-training, empirical | E5 / R3 (95%) | 13 |
| Reward Generalization in RLHF: A Topological Perspective Dong Yan, Fanzhi Zeng, Han Yang, Jiaming Ji Published: 2024-02-15Area: Alignment TrainingCitations: 7 Tags: ai-safety, alignment-training, theoretical | 2024-02-15 | Alignment Training | ai-safety, alignment-training, theoretical | E5 / R3 (95%) | 7 |