Instant research discovery
Search and browse arXiv CS/AI/ML papers, enriched with AI-generated insights.
Generate novel research ideas grounded in real arXiv papers with Brainstorm.
PreviousNext
| Paper | Published | Area | Tags | Intel | Citations |
|---|---|---|---|---|---|
Search and browse arXiv CS/AI/ML papers, enriched with AI-generated insights.
Generate novel research ideas grounded in real arXiv papers with Brainstorm.
| Paper | Published | Area | Tags | Intel | Citations |
|---|---|---|---|---|---|
Search and browse arXiv CS/AI/ML papers, enriched with AI-generated insights.
Generate novel research ideas grounded in real arXiv papers with Brainstorm.
| Paper | Published | Area | Tags | Intel | Citations |
|---|---|---|---|---|---|
| Paper | Published | Area | Tags | Intel | Citations |
|---|---|---|---|---|---|
| Safety Layers in Aligned Large Language Models Lan Zhang, Liuyi Yao, Shen Li, Yaliang Li Published: 2024-08-30Area: Mechanistic Interp.Citations: 88 Tags: ai-safety, alignment-training, empirical, mechanistic-interp | 2024-08-30 | Mechanistic Interp. | ai-safety, alignment-training, empirical, mechanistic-interp | E4 / R3 (95%) | 88 |
| Booster: Tackling Harmful Fine-tuning for Large Language Models via Attenuating Harmful Perturbation Fatih Ilhan, Ling Liu, Selim Furkan Tekin, Sihao Hu Published: 2024-09-03Area: Adversarial RobustnessCitations: 61 Tags: adversarial-robustness, ai-safety, alignment-training, empirical | 2024-09-03 | Adversarial Robustness | adversarial-robustness, ai-safety, alignment-training, empirical | E5 / R3 (94%) | 61 |
| Alignment with Preference Optimization Is All You Need for LLM Safety Ahmed Alzubaidi, Ali Khalifa Almansoori, Hakim Hacid, Mohamed El Amine Seddik Published: 2024-09-12Area: Alignment TrainingCitations: 5 Tags: ai-safety, alignment-training, empirical | 2024-09-12 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (94%) | 5 |
| Understanding Implosion in Text-to-Image Generative Models Ben Y. Zhao, Cathy Y. Li, Haitao Zheng, Shawn Shan Published: 2024-09-18Area: Multimodal SafetyCitations: 6 Tags: ai-safety, alignment-training, empirical, multimodal-safety | 2024-09-18 | Multimodal Safety | ai-safety, alignment-training, empirical, multimodal-safety | E5 / R4 (96%) | 6 |
| RRM: Robust Reward Model Training Mitigates Reward Hacking Abe Ittycheriah, Anastasiia Makarova, Aviral Kumar, Bilal Piot Published: 2024-09-20Area: Alignment TrainingCitations: 53 Tags: ai-safety, alignment-training, empirical | 2024-09-20 | Alignment Training | ai-safety, alignment-training, empirical | E6 / R3 (95%) | 53 |
| Elephant in the Room: Unveiling the Impact of Reward Model Quality in Alignment Daoguang Zan, Jingwei Yi, Jing Yao, Tsung-Yi Ho Published: 2024-09-26Area: Alignment TrainingCitations: 3 Tags: ai-safety, alignment-training, empirical | 2024-09-26 | Alignment Training | ai-safety, alignment-training, empirical | E6 / R3 (97%) | 3 |
| Harmful Fine-tuning Attacks and Defenses for Large Language Models: A Survey Fatih Ilhan, Ling Liu, Selim Furkan Tekin, Sihao Hu Published: 2024-09-26Area: Surveys & ReviewsCitations: 82 Tags: ai-safety, alignment-training, survey, surveys-reviews | 2024-09-26 | Surveys & Reviews | ai-safety, alignment-training, survey, surveys-reviews | E5 / R3 (95%) | 82 |
| Challenges and Future Directions of Data-Centric AI Alignment Jeffrey Wang, Leitian Tao, Min-Hsuan Yeh, Seongheon Park Published: 2024-10-02Area: Alignment TrainingCitations: 8 Tags: ai-safety, alignment-training, survey | 2024-10-02 | Alignment Training | ai-safety, alignment-training, survey | E5 / R3 (94%) | 8 |
| A Probabilistic Perspective on Unlearning and Alignment for Large Language Models Leo Schwinn, Stephan G眉nnemann, Yan Scholten Published: 2024-10-04Area: Model EditingCitations: 18 Tags: ai-safety, alignment-training, empirical, model-editing, safety-evaluation | 2024-10-04 | Model Editing | ai-safety, alignment-training, empirical, model-editing, safety-evaluation | E5 / R3 (95%) | 18 |
| Rules, Cases, and Reasoning: Positivist Legal Theory as a Framework for Pluralistic AI Alignment Nicholas A. Caputo Published: 2024-10-07Area: Alignment TrainingCitations: 2 Tags: ai-safety, alignment-training, theoretical | 2024-10-07 | Alignment Training | ai-safety, alignment-training, theoretical | E5 / R4 (94%) | 2 |
| SparsePO: Controlling Preference Alignment of LLMs via Sparse Token Masks Fenia Christopoulou, Gerasimos Lampouras, Haitham Bou-Ammar, Jun Wang Published: 2024-10-07Area: Alignment TrainingCitations: 6 Tags: ai-safety, alignment-training, empirical | 2024-10-07 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (95%) | 6 |
| Superficial Safety Alignment Hypothesis Jianwei Li, Jung-Eun Kim Published: 2024-10-07Area: Alignment TrainingCitations: 6 Tags: ai-safety, alignment-training, empirical | 2024-10-07 | Alignment Training | ai-safety, alignment-training, empirical | E6 / R3 (93%) | 6 |
| RL, but don't do anything I wouldn't do Marcus Hutter, Michael K. Cohen, Stuart Russell, Yoshua Bengio Published: 2024-10-08Area: Alignment TrainingCitations: 2 Tags: ai-safety, alignment-training, theoretical | 2024-10-08 | Alignment Training | ai-safety, alignment-training, theoretical | E5 / R3 (94%) | 2 |
| ETA: Evaluating Then Aligning Safety of Vision Language Models at Inference Time Bolian Li, Ruqi Zhang, Yi Ding Published: 2024-10-09Area: Multimodal SafetyCitations: 44 Tags: ai-safety, alignment-training, empirical, multimodal-safety | 2024-10-09 | Multimodal Safety | ai-safety, alignment-training, empirical, multimodal-safety | E5 / R3 (95%) | 44 |
| SEAL: Safety-enhanced Aligned LLM Fine-tuning via Bilevel Data Selection Han Shen, Payel Das, Pin-Yu Chen, Tianyi Chen Published: 2024-10-09Area: Alignment TrainingCitations: 55 Tags: ai-safety, alignment-training, empirical | 2024-10-09 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (96%) | 55 |
| Unraveling and Mitigating Safety Alignment Degradation of Vision-Language Models Chao Shang, Jie Ma, Ling Liu, Llu铆s M脿rquez Published: 2024-10-11Area: Multimodal SafetyCitations: 16 Tags: ai-safety, alignment-training, empirical, multimodal-safety | 2024-10-11 | Multimodal Safety | ai-safety, alignment-training, empirical, multimodal-safety | E5 / R3 (95%) | 16 |
| On Goodhart's Law, with an Application to Value Alignment El-Mahdi El-Mhamdi, L锚-Nguy锚n Hoang Published: 2024-10-12Area: Formal/TheoreticalCitations: 6 Tags: ai-safety, alignment-training, formaltheoretical, theoretical | 2024-10-12 | Formal/Theoretical | ai-safety, alignment-training, formaltheoretical, theoretical | E5 / R3 (95%) | 6 |
| Safety-Aware Fine-Tuning of Large Language Models Hyeong Kyu Choi, Xuefeng Du, Yixuan Li Published: 2024-10-13Area: Alignment TrainingCitations: 38 Tags: ai-safety, alignment-training, empirical | 2024-10-13 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (96%) | 38 |
| Targeted Vaccine: Safety Alignment for Large Language Models against Harmful Fine-Tuning via Layer-wise Perturbation Guozhi Liu, Li Shen, Qi Mu, Ruichao Mo Published: 2024-10-13Area: Alignment TrainingCitations: 31 Tags: ai-safety, alignment-training, empirical | 2024-10-13 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (95%) | 31 |
| AdvBDGen: Adversarially Fortified Prompt-Specific Fuzzy Backdoor Generator Against LLM Alignment Furong Huang, Michael-Andrei Panaitescu-Liess, Pankayaraj Pathmanathan, Udari Madhushani Sehwag Published: 2024-10-15Area: Adversarial RobustnessCitations: 2 Tags: adversarial-robustness, ai-safety, alignment-training, empirical | 2024-10-15 | Adversarial Robustness | adversarial-robustness, ai-safety, alignment-training, empirical | E5 / R3 (96%) | 2 |
| Insights from the Inverse: Reconstructing LLM Training Goals Through Inverse Reinforcement Learning Arjun Jagota, Jared Joselowitz, Satyapriya Krishna, Sonali Parbhoo Published: 2024-10-16Area: Alignment TrainingCitations: 4 Tags: ai-safety, alignment-training, empirical | 2024-10-16 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R4 (95%) | 4 |
| POROver: Improving Safety and Reducing Overrefusal in Large Language Models with Overgeneration and Preference Optimization Alon Benhaim, Batuhan K. Karaman, Ishmam Zabir, Mert R. Sabuncu Published: 2024-10-16Area: Alignment TrainingCitations: 3 Tags: ai-safety, alignment-training, empirical | 2024-10-16 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (94%) | 3 |
| Weak-to-Strong Generalization beyond Accuracy: a Pilot Study in Safety, Toxicity, and Legal Reasoning Bo Hui, Ruimeng Ye, Yang Xiao Published: 2024-10-16Area: Scalable OversightCitations: 5 Tags: ai-safety, alignment-training, empirical, scalable-oversight | 2024-10-16 | Scalable Oversight | ai-safety, alignment-training, empirical, scalable-oversight | E8 / R4 (93%) | 5 |
| BiasJailbreak: Analyzing Ethical Biases and Jailbreak Vulnerabilities in Large Language Models Haebin Seong, Isack Lee Published: 2024-10-17Area: Adversarial RobustnessCitations: - Tags: adversarial-robustness, ai-safety, alignment-training, empirical | 2024-10-17 | Adversarial Robustness | adversarial-robustness, ai-safety, alignment-training, empirical | E5 / R3 (94%) | - |
| Persistent Pre-Training Poisoning of LLMs Daphne Ippolito, Eric Michael Smith, Florian Tram猫r, Ivan Evtimov Published: 2024-10-17Area: Adversarial RobustnessCitations: 38 Tags: adversarial-robustness, ai-safety, alignment-training, empirical | 2024-10-17 | Adversarial Robustness | adversarial-robustness, ai-safety, alignment-training, empirical | E8 / R3 (93%) | 38 |
| Towards Understanding the Fragility of Multilingual LLMs against Fine-Tuning Attacks Aobo Yang, Bobbie Chern, Han Zhao, Jianfeng Chi Published: 2024-10-23Area: Adversarial RobustnessCitations: 36 Tags: adversarial-robustness, ai-safety, alignment-training, empirical | 2024-10-23 | Adversarial Robustness | adversarial-robustness, ai-safety, alignment-training, empirical | E5 / R3 (94%) | 36 |
| Inference time LLM alignment in single and multidomain preference spectrum Kishaloy Halder, Manuel Mager, Monica Sunkara, Nikolaos Pappas Published: 2024-10-24Area: Model EditingCitations: 3 Tags: ai-safety, alignment-training, empirical, model-editing | 2024-10-24 | Model Editing | ai-safety, alignment-training, empirical, model-editing | E5 / R3 (95%) | 3 |
| Skywork-Reward: Bag of Tricks for Reward Modeling in LLMs Chaojie Wang, Chris Yuhao Liu, Jiacai Liu, Jujie He Published: 2024-10-24Area: Alignment TrainingCitations: 245 Tags: ai-safety, alignment-training, empirical | 2024-10-24 | Alignment Training | ai-safety, alignment-training, empirical | E6 / R3 (97%) | 245 |
| Enhancing Safety in Reinforcement Learning with Human Feedback via Rectified Policy Optimization Dongqing Zou, Hengquan Guo, Honghao Wei, Jiawei Zhang Published: 2024-10-25Area: Alignment TrainingCitations: 5 Tags: ai-safety, alignment-training, empirical | 2024-10-25 | Alignment Training | ai-safety, alignment-training, empirical | E6 / R3 (96%) | 5 |
| Enhancing Neural Network Interpretability with Feature-Aligned Sparse Autoencoders Alasdair Paren, David Krueger, Fazl Barez, Luke Marks Published: 2024-11-02Area: Mechanistic Interp.Citations: 19 Tags: ai-safety, alignment-training, empirical, interpretability, mechanistic-interp | 2024-11-02 | Mechanistic Interp. | ai-safety, alignment-training, empirical, interpretability, mechanistic-interp | E5 / R3 (95%) | 19 |