Instant research discovery
Search and browse arXiv CS/AI/ML papers, enriched with AI-generated insights.
Generate novel research ideas grounded in real arXiv papers with Brainstorm.
PreviousNext
| Paper | Published | Area | Tags | Intel | Citations |
|---|---|---|---|---|---|
Search and browse arXiv CS/AI/ML papers, enriched with AI-generated insights.
Generate novel research ideas grounded in real arXiv papers with Brainstorm.
| Paper | Published | Area | Tags | Intel | Citations |
|---|---|---|---|---|---|
Search and browse arXiv CS/AI/ML papers, enriched with AI-generated insights.
Generate novel research ideas grounded in real arXiv papers with Brainstorm.
| Paper | Published | Area | Tags | Intel | Citations |
|---|---|---|---|---|---|
| Paper | Published | Area | Tags | Intel | Citations |
|---|---|---|---|---|---|
| Safety-Aware Fine-Tuning of Large Language Models Hyeong Kyu Choi, Xuefeng Du, Yixuan Li Published: 2024-10-13Area: Alignment TrainingCitations: 38 Tags: ai-safety, alignment-training, empirical | 2024-10-13 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (96%) | 38 |
| Targeted Vaccine: Safety Alignment for Large Language Models against Harmful Fine-Tuning via Layer-wise Perturbation Guozhi Liu, Li Shen, Qi Mu, Ruichao Mo Published: 2024-10-13Area: Alignment TrainingCitations: 31 Tags: ai-safety, alignment-training, empirical | 2024-10-13 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (95%) | 31 |
| On Goodhart's Law, with an Application to Value Alignment El-Mahdi El-Mhamdi, L锚-Nguy锚n Hoang Published: 2024-10-12Area: Formal/TheoreticalCitations: 6 Tags: ai-safety, alignment-training, formaltheoretical, theoretical | 2024-10-12 | Formal/Theoretical | ai-safety, alignment-training, formaltheoretical, theoretical | E5 / R3 (95%) | 6 |
| Unraveling and Mitigating Safety Alignment Degradation of Vision-Language Models Chao Shang, Jie Ma, Ling Liu, Llu铆s M脿rquez Published: 2024-10-11Area: Multimodal SafetyCitations: 16 Tags: ai-safety, alignment-training, empirical, multimodal-safety | 2024-10-11 | Multimodal Safety | ai-safety, alignment-training, empirical, multimodal-safety | E5 / R3 (95%) | 16 |
| ETA: Evaluating Then Aligning Safety of Vision Language Models at Inference Time Bolian Li, Ruqi Zhang, Yi Ding Published: 2024-10-09Area: Multimodal SafetyCitations: 44 Tags: ai-safety, alignment-training, empirical, multimodal-safety | 2024-10-09 | Multimodal Safety | ai-safety, alignment-training, empirical, multimodal-safety | E5 / R3 (95%) | 44 |
| SEAL: Safety-enhanced Aligned LLM Fine-tuning via Bilevel Data Selection Han Shen, Payel Das, Pin-Yu Chen, Tianyi Chen Published: 2024-10-09Area: Alignment TrainingCitations: 55 Tags: ai-safety, alignment-training, empirical | 2024-10-09 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (96%) | 55 |
| RL, but don't do anything I wouldn't do Marcus Hutter, Michael K. Cohen, Stuart Russell, Yoshua Bengio Published: 2024-10-08Area: Alignment TrainingCitations: 2 Tags: ai-safety, alignment-training, theoretical | 2024-10-08 | Alignment Training | ai-safety, alignment-training, theoretical | E5 / R3 (94%) | 2 |
| Rules, Cases, and Reasoning: Positivist Legal Theory as a Framework for Pluralistic AI Alignment Nicholas A. Caputo Published: 2024-10-07Area: Alignment TrainingCitations: 2 Tags: ai-safety, alignment-training, theoretical | 2024-10-07 | Alignment Training | ai-safety, alignment-training, theoretical | E5 / R4 (94%) | 2 |
| SparsePO: Controlling Preference Alignment of LLMs via Sparse Token Masks Fenia Christopoulou, Gerasimos Lampouras, Haitham Bou-Ammar, Jun Wang Published: 2024-10-07Area: Alignment TrainingCitations: 6 Tags: ai-safety, alignment-training, empirical | 2024-10-07 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (95%) | 6 |
| Superficial Safety Alignment Hypothesis Jianwei Li, Jung-Eun Kim Published: 2024-10-07Area: Alignment TrainingCitations: 6 Tags: ai-safety, alignment-training, empirical | 2024-10-07 | Alignment Training | ai-safety, alignment-training, empirical | E6 / R3 (93%) | 6 |
| A Probabilistic Perspective on Unlearning and Alignment for Large Language Models Leo Schwinn, Stephan G眉nnemann, Yan Scholten Published: 2024-10-04Area: Model EditingCitations: 18 Tags: ai-safety, alignment-training, empirical, model-editing, safety-evaluation | 2024-10-04 | Model Editing | ai-safety, alignment-training, empirical, model-editing, safety-evaluation | E5 / R3 (95%) | 18 |
| Challenges and Future Directions of Data-Centric AI Alignment Jeffrey Wang, Leitian Tao, Min-Hsuan Yeh, Seongheon Park Published: 2024-10-02Area: Alignment TrainingCitations: 8 Tags: ai-safety, alignment-training, survey | 2024-10-02 | Alignment Training | ai-safety, alignment-training, survey | E5 / R3 (94%) | 8 |
| Elephant in the Room: Unveiling the Impact of Reward Model Quality in Alignment Daoguang Zan, Jingwei Yi, Jing Yao, Tsung-Yi Ho Published: 2024-09-26Area: Alignment TrainingCitations: 3 Tags: ai-safety, alignment-training, empirical | 2024-09-26 | Alignment Training | ai-safety, alignment-training, empirical | E6 / R3 (97%) | 3 |
| Harmful Fine-tuning Attacks and Defenses for Large Language Models: A Survey Fatih Ilhan, Ling Liu, Selim Furkan Tekin, Sihao Hu Published: 2024-09-26Area: Surveys & ReviewsCitations: 82 Tags: ai-safety, alignment-training, survey, surveys-reviews | 2024-09-26 | Surveys & Reviews | ai-safety, alignment-training, survey, surveys-reviews | E5 / R3 (95%) | 82 |
| RRM: Robust Reward Model Training Mitigates Reward Hacking Abe Ittycheriah, Anastasiia Makarova, Aviral Kumar, Bilal Piot Published: 2024-09-20Area: Alignment TrainingCitations: 53 Tags: ai-safety, alignment-training, empirical | 2024-09-20 | Alignment Training | ai-safety, alignment-training, empirical | E6 / R3 (95%) | 53 |
| Understanding Implosion in Text-to-Image Generative Models Ben Y. Zhao, Cathy Y. Li, Haitao Zheng, Shawn Shan Published: 2024-09-18Area: Multimodal SafetyCitations: 6 Tags: ai-safety, alignment-training, empirical, multimodal-safety | 2024-09-18 | Multimodal Safety | ai-safety, alignment-training, empirical, multimodal-safety | E5 / R4 (96%) | 6 |
| Alignment with Preference Optimization Is All You Need for LLM Safety Ahmed Alzubaidi, Ali Khalifa Almansoori, Hakim Hacid, Mohamed El Amine Seddik Published: 2024-09-12Area: Alignment TrainingCitations: 5 Tags: ai-safety, alignment-training, empirical | 2024-09-12 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (94%) | 5 |
| Booster: Tackling Harmful Fine-tuning for Large Language Models via Attenuating Harmful Perturbation Fatih Ilhan, Ling Liu, Selim Furkan Tekin, Sihao Hu Published: 2024-09-03Area: Adversarial RobustnessCitations: 61 Tags: adversarial-robustness, ai-safety, alignment-training, empirical | 2024-09-03 | Adversarial Robustness | adversarial-robustness, ai-safety, alignment-training, empirical | E5 / R3 (94%) | 61 |
| Beyond Preferences in AI Alignment Hal Ashton, Matija Franklin, Micah Carroll, Tan Zhi-Xuan Published: 2024-08-30Area: Alignment TrainingCitations: 44 Tags: ai-safety, alignment-training, position | 2024-08-30 | Alignment Training | ai-safety, alignment-training, position | E5 / R3 (94%) | 44 |
| Safety Layers in Aligned Large Language Models Lan Zhang, Liuyi Yao, Shen Li, Yaliang Li Published: 2024-08-30Area: Mechanistic Interp.Citations: 88 Tags: ai-safety, alignment-training, empirical, mechanistic-interp | 2024-08-30 | Mechanistic Interp. | ai-safety, alignment-training, empirical, mechanistic-interp | E4 / R3 (95%) | 88 |
| Bi-Factorial Preference Optimization: Balancing Safety-Helpfulness in Language Models Adel Bibi, Mohamed Elhoseiny, Philip H.S. Torr, Wenxuan Zhang Published: 2024-08-27Area: Alignment TrainingCitations: 24 Tags: ai-safety, alignment-training, empirical | 2024-08-27 | Alignment Training | ai-safety, alignment-training, empirical | E4 / R2 (94%) | 24 |
| Antidote: Post-fine-tuning Safety Alignment for Large Language Models against Harmful Fine-tuning Gautam Bhattacharya, Josh Kimball, Ling Liu, Pratik Joshi Published: 2024-08-18Area: Adversarial RobustnessCitations: 55 Tags: adversarial-robustness, ai-safety, alignment-training, empirical | 2024-08-18 | Adversarial Robustness | adversarial-robustness, ai-safety, alignment-training, empirical | E5 / R3 (95%) | 55 |
| Multi-Turn Context Jailbreak Attack on Large Language Models From First Principles Deyue Zhang, Dongdong Yang, Hui Li, Quanchen Zou Published: 2024-08-08Area: Adversarial RobustnessCitations: 50 Tags: adversarial-robustness, ai-safety, alignment-training, empirical | 2024-08-08 | Adversarial Robustness | adversarial-robustness, ai-safety, alignment-training, empirical | E5 / R3 (96%) | 50 |
| Can DPO Learn Diverse Human Values? A Theoretical Scaling Law Shawn Im, Yixuan Li Published: 2024-08-06Area: Alignment TrainingCitations: 7 Tags: ai-safety, alignment-training, theoretical | 2024-08-06 | Alignment Training | ai-safety, alignment-training, theoretical | E4 / R2 (94%) | 7 |
| Mission Impossible: A Statistical Perspective on Jailbreaking LLMs Jingtong Su, Julia Kempe, Karen Ullrich Published: 2024-08-02Area: Adversarial RobustnessCitations: 26 Tags: adversarial-robustness, ai-safety, alignment-training, theoretical | 2024-08-02 | Adversarial Robustness | adversarial-robustness, ai-safety, alignment-training, theoretical | E5 / R3 (95%) | 26 |
| Supertrust foundational alignment: mutual trust must replace permanent control for safe superintelligence James M. Mazzu Published: 2024-07-29Area: Alignment TrainingCitations: 2 Tags: ai-safety, alignment-training, position | 2024-07-29 | Alignment Training | ai-safety, alignment-training, position | E6 / R3 (93%) | 2 |
| Course-Correction: Safety Alignment Using Synthetic Preferences Haiqin Weng, Han Qiu, Renjie Gu, Rongwu Xu Published: 2024-07-23Area: Alignment TrainingCitations: 13 Tags: ai-safety, alignment-training, empirical | 2024-07-23 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (96%) | 13 |
| Revisiting the Robust Alignment of Circuit Breakers Leo Schwinn, Simon Geisler Published: 2024-07-22Area: Adversarial RobustnessCitations: 8 Tags: adversarial-robustness, ai-safety, alignment-training, empirical | 2024-07-22 | Adversarial Robustness | adversarial-robustness, ai-safety, alignment-training, empirical | E5 / R2 (93%) | 8 |
| Correcting the Mythos of KL-Regularization: Direct Alignment without Overoptimization via Chi-Squared Preference Optimization Akshay Krishnamurthy, Audrey Huang, Dylan J. Foster, Jason D. Lee Published: 2024-07-18Area: Alignment TrainingCitations: 54 Tags: ai-safety, alignment-training, theoretical | 2024-07-18 | Alignment Training | ai-safety, alignment-training, theoretical | E5 / R3 (96%) | 54 |
| Analyzing the Generalization and Reliability of Steering Vectors Adria Garriga-Alonso, Aengus Lynch, Brooks Paige, Daniel Tan Published: 2024-07-17Area: Representation AnalysisCitations: 64 Tags: ai-safety, alignment-training, empirical, representation-analysis | 2024-07-17 | Representation Analysis | ai-safety, alignment-training, empirical, representation-analysis | E5 / R3 (95%) | 64 |