Instant research discovery
Search and browse arXiv CS/AI/ML papers, enriched with AI-generated insights.
Generate novel research ideas grounded in real arXiv papers with Brainstorm.
PreviousNext
| Paper | Published | Area | Tags | Intel | Citations |
|---|---|---|---|---|---|
Search and browse arXiv CS/AI/ML papers, enriched with AI-generated insights.
Generate novel research ideas grounded in real arXiv papers with Brainstorm.
| Paper | Published | Area | Tags | Intel | Citations |
|---|---|---|---|---|---|
Search and browse arXiv CS/AI/ML papers, enriched with AI-generated insights.
Generate novel research ideas grounded in real arXiv papers with Brainstorm.
| Paper | Published | Area | Tags | Intel | Citations |
|---|---|---|---|---|---|
| Paper | Published | Area | Tags | Intel | Citations |
|---|---|---|---|---|---|
| Safety Arithmetic: A Framework for Test-time Safety Alignment of Language Models by Steering Parameters and Activations Rima Hazra, Sayan Layek, Somnath Banerjee, Soujanya Poria Published: 2024-06-17Area: Model EditingCitations: 26 Tags: ai-safety, alignment-training, empirical, model-editing | 2024-06-17 | Model Editing | ai-safety, alignment-training, empirical, model-editing | E5 / R4 (96%) | 26 |
| SPA-VL: A Comprehensive Safety Preference Alignment Dataset for Vision Language Models Feng Zhao, Guodong Zheng, Jing Shao, Jinlan Fu Published: 2024-06-17Area: Multimodal SafetyCitations: 68 Tags: ai-safety, alignment-training, dataset, multimodal-safety | 2024-06-17 | Multimodal Safety | ai-safety, alignment-training, dataset, multimodal-safety | E6 / R3 (95%) | 68 |
| Super(ficial)-alignment: Strong Models May Deceive Weak Models in Weak-to-Strong Generalization Guangyao Shen, Shiqi Shen, Wenkai Yang, Yankai Lin Published: 2024-06-17Area: Scalable OversightCitations: 20 Tags: ai-safety, alignment-training, empirical, scalable-oversight | 2024-06-17 | Scalable Oversight | ai-safety, alignment-training, empirical, scalable-oversight | E6 / R3 (92%) | 20 |
| Who's Asking? User Personas and the Mechanics of Latent Misalignment Ann Yuan, Asma Ghandeharioun, Emily Reif, Lucas Dixon Published: 2024-06-17Area: Representation AnalysisCitations: 23 Tags: ai-safety, alignment-training, empirical, representation-analysis | 2024-06-17 | Representation Analysis | ai-safety, alignment-training, empirical, representation-analysis | E5 / R4 (95%) | 23 |
| SafeInfer: Context Adaptive Decoding Time Safety Alignment for Large Language Models Animesh Mukherjee, Rima Hazra, Sayan Layek, Shanu Kumar Published: 2024-06-18Area: Adversarial RobustnessCitations: 14 Tags: adversarial-robustness, ai-safety, alignment-training, empirical | 2024-06-18 | Adversarial Robustness | adversarial-robustness, ai-safety, alignment-training, empirical | E5 / R3 (98%) | 14 |
| Jailbreaking Large Language Models Through Alignment Vulnerabilities in Out-of-Distribution Settings Bingda Tang, Dongping Chen, Jingyu Tang, Lichao Sun Published: 2024-06-19Area: Adversarial RobustnessCitations: 7 Tags: adversarial-robustness, ai-safety, alignment-training, empirical | 2024-06-19 | Adversarial Robustness | adversarial-robustness, ai-safety, alignment-training, empirical | E5 / R3 (96%) | 7 |
| Towards Minimal Targeted Updates of Language Models with Targeted Negative Training Arya Tafvizi, Lily H. Zhang, Rajesh Ranganath Published: 2024-06-19Area: Alignment TrainingCitations: 1 Tags: ai-safety, alignment-training, empirical | 2024-06-19 | Alignment Training | ai-safety, alignment-training, empirical | E6 / R3 (93%) | 1 |
| Jailbreaking as a Reward Misspecification Problem Jiahui Gao, Lei Li, Lingpeng Kong, Qi Liu Published: 2024-06-20Area: Adversarial RobustnessCitations: 11 Tags: adversarial-robustness, ai-safety, alignment-training, empirical | 2024-06-20 | Adversarial Robustness | adversarial-robustness, ai-safety, alignment-training, empirical | E5 / R3 (96%) | 11 |
| PKU-SafeRLHF: A Safety Alignment Preference Dataset for Llama Family Models Boren Zheng, Borong Zhang, Boxuan Li, Boyuan Chen Published: 2024-06-20Area: Alignment TrainingCitations: 127 Tags: ai-safety, alignment-training, dataset | 2024-06-20 | Alignment Training | ai-safety, alignment-training, dataset | E5 / R3 (96%) | 127 |
| Raising the Bar: Investigating the Values of Large Language Models via Generative Evolving Testing Han Jiang, Shu Wang, XiaoYuan Yi, Xing Xie Published: 2024-06-20Area: Safety EvaluationCitations: 16 Tags: ai-safety, alignment-training, benchmark, safety-evaluation | 2024-06-20 | Safety Evaluation | ai-safety, alignment-training, benchmark, safety-evaluation | E5 / R3 (95%) | 16 |
| Towards Understanding Safety Alignment: A Mechanistic Perspective from Safety Neurons Jianhui Chen, Juanzi Li, Lei Hou, Xiaozhi Wang Published: 2024-06-20Area: Mechanistic Interp.Citations: 25 Tags: ai-safety, alignment-training, empirical, mechanistic-interp | 2024-06-20 | Mechanistic Interp. | ai-safety, alignment-training, empirical, mechanistic-interp | E5 / R3 (94%) | 25 |
| Robust Reinforcement Learning from Corrupted Human Feedback Alexander Bukharin, Haoming Jiang, Ilgee Hong, Qingru Zhang Published: 2024-06-21Area: Alignment TrainingCitations: 18 Tags: ai-safety, alignment-training, empirical | 2024-06-21 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (93%) | 18 |
| AI Alignment through Reinforcement Learning from Human Feedback? Contradictions and Limitations Adam Dahlgren Lindstr枚m, Dimitri Coelho Mollo, 脥帽igo Mart铆nez de Rituerto de Troya, Lea Krause Published: 2024-06-26Area: Alignment TrainingCitations: 8 Tags: ai-safety, alignment-training, position | 2024-06-26 | Alignment Training | ai-safety, alignment-training, position | E5 / R3 (95%) | 8 |
| The Multilingual Alignment Prism: Aligning Global and Local Preferences to Reduce Harm Aakanksha, Arash Ahmadian, Beyza Ermis, Julia Kreutzer Published: 2024-06-26Area: Alignment TrainingCitations: 55 Tags: ai-safety, alignment-training, empirical | 2024-06-26 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (94%) | 55 |
| ProgressGym: Alignment with a Millennium of Moral Progress Jasmine Xinze Li, Jiaming Ji, Tianyi Qiu, Xuchuan Huang Published: 2024-06-28Area: Alignment TrainingCitations: 9 Tags: ai-safety, alignment-training, benchmark | 2024-06-28 | Alignment Training | ai-safety, alignment-training, benchmark | E7 / R5 (98%) | 9 |
| AI Safety in Generative AI Large Language Models: A Survey Chen Wang, Jaymari Chua, Lina Yao, Shiyi Yang Published: 2024-07-06Area: Surveys & ReviewsCitations: 38 Tags: ai-safety, alignment-training, survey, surveys-reviews | 2024-07-06 | Surveys & Reviews | ai-safety, alignment-training, survey, surveys-reviews | E6 / R3 (95%) | 38 |
| Multilingual Blending: LLM Safety Alignment Evaluation with Language Mixture Jiayang Song, Lei Ma, Yuheng Huang, Zhehua Zhou Published: 2024-07-10Area: Adversarial RobustnessCitations: 19 Tags: adversarial-robustness, ai-safety, alignment-training, empirical, safety-evaluation | 2024-07-10 | Adversarial Robustness | adversarial-robustness, ai-safety, alignment-training, empirical, safety-evaluation | E5 / R3 (95%) | 19 |
| DistillSeq: A Framework for Safety Alignment Testing in Large Language Models using Knowledge Distillation Ling Shi, Mingke Yang, Yi Liu, Yuqi Chen Published: 2024-07-14Area: Safety EvaluationCitations: 9 Tags: adversarial-robustness, ai-safety, alignment-training, empirical, safety-evaluation | 2024-07-14 | Safety Evaluation | adversarial-robustness, ai-safety, alignment-training, empirical, safety-evaluation | E6 / R3 (97%) | 9 |
| Learning Dynamics of LLM Finetuning Danica J. Sutherland, Yi Ren Published: 2024-07-15Area: Training DynamicsCitations: 67 Tags: ai-safety, alignment-training, theoretical, training-dynamics | 2024-07-15 | Training Dynamics | ai-safety, alignment-training, theoretical, training-dynamics | E5 / R3 (94%) | 67 |
| Analyzing the Generalization and Reliability of Steering Vectors Adria Garriga-Alonso, Aengus Lynch, Brooks Paige, Daniel Tan Published: 2024-07-17Area: Representation AnalysisCitations: 64 Tags: ai-safety, alignment-training, empirical, representation-analysis | 2024-07-17 | Representation Analysis | ai-safety, alignment-training, empirical, representation-analysis | E5 / R3 (95%) | 64 |
| Correcting the Mythos of KL-Regularization: Direct Alignment without Overoptimization via Chi-Squared Preference Optimization Akshay Krishnamurthy, Audrey Huang, Dylan J. Foster, Jason D. Lee Published: 2024-07-18Area: Alignment TrainingCitations: 54 Tags: ai-safety, alignment-training, theoretical | 2024-07-18 | Alignment Training | ai-safety, alignment-training, theoretical | E5 / R3 (96%) | 54 |
| Revisiting the Robust Alignment of Circuit Breakers Leo Schwinn, Simon Geisler Published: 2024-07-22Area: Adversarial RobustnessCitations: 8 Tags: adversarial-robustness, ai-safety, alignment-training, empirical | 2024-07-22 | Adversarial Robustness | adversarial-robustness, ai-safety, alignment-training, empirical | E5 / R2 (93%) | 8 |
| Course-Correction: Safety Alignment Using Synthetic Preferences Haiqin Weng, Han Qiu, Renjie Gu, Rongwu Xu Published: 2024-07-23Area: Alignment TrainingCitations: 13 Tags: ai-safety, alignment-training, empirical | 2024-07-23 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (96%) | 13 |
| Supertrust foundational alignment: mutual trust must replace permanent control for safe superintelligence James M. Mazzu Published: 2024-07-29Area: Alignment TrainingCitations: 2 Tags: ai-safety, alignment-training, position | 2024-07-29 | Alignment Training | ai-safety, alignment-training, position | E6 / R3 (93%) | 2 |
| Mission Impossible: A Statistical Perspective on Jailbreaking LLMs Jingtong Su, Julia Kempe, Karen Ullrich Published: 2024-08-02Area: Adversarial RobustnessCitations: 26 Tags: adversarial-robustness, ai-safety, alignment-training, theoretical | 2024-08-02 | Adversarial Robustness | adversarial-robustness, ai-safety, alignment-training, theoretical | E5 / R3 (95%) | 26 |
| Can DPO Learn Diverse Human Values? A Theoretical Scaling Law Shawn Im, Yixuan Li Published: 2024-08-06Area: Alignment TrainingCitations: 7 Tags: ai-safety, alignment-training, theoretical | 2024-08-06 | Alignment Training | ai-safety, alignment-training, theoretical | E4 / R2 (94%) | 7 |
| Multi-Turn Context Jailbreak Attack on Large Language Models From First Principles Deyue Zhang, Dongdong Yang, Hui Li, Quanchen Zou Published: 2024-08-08Area: Adversarial RobustnessCitations: 50 Tags: adversarial-robustness, ai-safety, alignment-training, empirical | 2024-08-08 | Adversarial Robustness | adversarial-robustness, ai-safety, alignment-training, empirical | E5 / R3 (96%) | 50 |
| Antidote: Post-fine-tuning Safety Alignment for Large Language Models against Harmful Fine-tuning Gautam Bhattacharya, Josh Kimball, Ling Liu, Pratik Joshi Published: 2024-08-18Area: Adversarial RobustnessCitations: 55 Tags: adversarial-robustness, ai-safety, alignment-training, empirical | 2024-08-18 | Adversarial Robustness | adversarial-robustness, ai-safety, alignment-training, empirical | E5 / R3 (95%) | 55 |
| Bi-Factorial Preference Optimization: Balancing Safety-Helpfulness in Language Models Adel Bibi, Mohamed Elhoseiny, Philip H.S. Torr, Wenxuan Zhang Published: 2024-08-27Area: Alignment TrainingCitations: 24 Tags: ai-safety, alignment-training, empirical | 2024-08-27 | Alignment Training | ai-safety, alignment-training, empirical | E4 / R2 (94%) | 24 |
| Beyond Preferences in AI Alignment Hal Ashton, Matija Franklin, Micah Carroll, Tan Zhi-Xuan Published: 2024-08-30Area: Alignment TrainingCitations: 44 Tags: ai-safety, alignment-training, position | 2024-08-30 | Alignment Training | ai-safety, alignment-training, position | E5 / R3 (94%) | 44 |