Instant research discovery
Search and browse arXiv CS/AI/ML papers, enriched with AI-generated insights.
Generate novel research ideas grounded in real arXiv papers with Brainstorm.
PreviousNext
| Paper | Published | Area | Tags | Intel | Citations |
|---|---|---|---|---|---|
Search and browse arXiv CS/AI/ML papers, enriched with AI-generated insights.
Generate novel research ideas grounded in real arXiv papers with Brainstorm.
| Paper | Published | Area | Tags | Intel | Citations |
|---|---|---|---|---|---|
Search and browse arXiv CS/AI/ML papers, enriched with AI-generated insights.
Generate novel research ideas grounded in real arXiv papers with Brainstorm.
| Paper | Published | Area | Tags | Intel | Citations |
|---|---|---|---|---|---|
| Paper | Published | Area | Tags | Intel | Citations |
|---|---|---|---|---|---|
| Alignment Faking in Large Language Models Akbir Khan, Benjamin Wright, Buck Shlegeris, Carson Denison Published: 2024-12-18Area: Deception & FailureCitations: 160 Tags: ai-safety, alignment-training, deception-failure, empirical | 2024-12-18 | Deception & Failure | ai-safety, alignment-training, deception-failure, empirical | E5 / R3 (95%) | 160 |
| NLSR: Neuron-Level Safety Realignment of Large Language Models Against Harmful Fine-Tuning Gerard de Melo, Liang He, Linlin Wang, Shunfan Zheng Published: 2024-12-17Area: Model EditingCitations: 30 Tags: ai-safety, alignment-training, empirical, model-editing | 2024-12-17 | Model Editing | ai-safety, alignment-training, empirical, model-editing | E4 / R3 (94%) | 30 |
| The superalignment of superhuman intelligence with large language models Jie Tang, Minlie Huang, Pei Ke, Shiyao Cui Published: 2024-12-15Area: Scalable OversightCitations: 1 Tags: ai-safety, alignment-training, scalable-oversight, survey | 2024-12-15 | Scalable Oversight | ai-safety, alignment-training, scalable-oversight, survey | E6 / R3 (93%) | 1 |
| AlignGuard: Scalable Safety Alignment for Text-to-Image Generation Ashkan Khakzar, Fabio Pizzati, I Chieh Chen, Jindong Gu Published: 2024-12-13Area: Multimodal SafetyCitations: 11 Tags: ai-safety, alignment-training, empirical, multimodal-safety | 2024-12-13 | Multimodal Safety | ai-safety, alignment-training, empirical, multimodal-safety | E6 / R4 (98%) | 11 |
| Sail into the Headwind: Alignment via Robust Rewards and Dynamic Labels against Reward Hacking Paria Rashidinejad, Yuandong Tian Published: 2024-12-12Area: Alignment TrainingCitations: 10 Tags: ai-safety, alignment-training, empirical | 2024-12-12 | Alignment Training | ai-safety, alignment-training, empirical | E6 / R3 (97%) | 10 |
| Semantic Loss Guided Data Efficient Supervised Fine Tuning for Safe Responses in LLMs Arunesh Sinha, Pradeep Varakantham, Yuxiao Lu Published: 2024-12-07Area: Alignment TrainingCitations: 3 Tags: ai-safety, alignment-training, empirical | 2024-12-07 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (95%) | 3 |
| LIAR: Leveraging Inference Time Alignment (Best-of-N) to Jailbreak LLMs in Seconds Amrit Singh Bedi, Furong Huang, James Beetham, Mengdi Wang Published: 2024-12-06Area: Adversarial RobustnessCitations: 2 Tags: adversarial-robustness, ai-safety, alignment-training, empirical | 2024-12-06 | Adversarial Robustness | adversarial-robustness, ai-safety, alignment-training, empirical | E5 / R3 (96%) | 2 |
| Jailbreak Large Vision-Language Models Through Multi-Modal Linkage Geyuan Zhang, Tianxing He, Xiaofei Zhou, Yichen Wang Published: 2024-11-30Area: Multimodal SafetyCitations: 37 Tags: adversarial-robustness, ai-safety, alignment-training, empirical, multimodal-safety | 2024-11-30 | Multimodal Safety | adversarial-robustness, ai-safety, alignment-training, empirical, multimodal-safety | E5 / R3 (96%) | 37 |
| Immune: Improving Safety Against Jailbreaks in Multi-modal LLMs via Inference-Time Alignment Ahmad Beirami, Alvaro Velasquez, Amrit Singh Bedi, Dinesh Manocha Published: 2024-11-27Area: Multimodal SafetyCitations: 18 Tags: adversarial-robustness, ai-safety, alignment-training, empirical, multimodal-safety | 2024-11-27 | Multimodal Safety | adversarial-robustness, ai-safety, alignment-training, empirical, multimodal-safety | E5 / R3 (94%) | 18 |
| What AI evaluations for preventing catastrophic risks can and cannot do Lisa Thiergart, Peter Barnett Published: 2024-11-26Area: Safety EvaluationCitations: 3 Tags: ai-safety, alignment-training, position, safety-evaluation | 2024-11-26 | Safety Evaluation | ai-safety, alignment-training, position, safety-evaluation | E6 / R3 (95%) | 3 |
| ProSec: Fortifying Code LLMs with Proactive Security Alignment Jinyao Guo, Kaiyuan Zhang, Xiangyu Zhang, Xiangzhe Xu Published: 2024-11-19Area: Alignment TrainingCitations: 13 Tags: ai-safety, alignment-training, empirical | 2024-11-19 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (95%) | 13 |
| Enhancing Vision-Language Model Safety through Progressive Concept-Bottleneck-Driven Alignment Bo Zheng, Chongjun Wang, Qiushi Cui, Xiangyu Yue Published: 2024-11-18Area: Multimodal SafetyCitations: 1 Tags: ai-safety, alignment-training, empirical, multimodal-safety | 2024-11-18 | Multimodal Safety | ai-safety, alignment-training, empirical, multimodal-safety | E5 / R3 (96%) | 1 |
| Approximated Variational Bayesian Inverse Reinforcement Learning for Large Language Model Alignment Jinsheng Shi, Qinhong Lin, Yuang Cai, Yuyu Yuan Published: 2024-11-14Area: Alignment TrainingCitations: 5 Tags: ai-safety, alignment-training, empirical | 2024-11-14 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (94%) | 5 |
| Dynamic Rewarding with Prompt Optimization Enables Tuning-free Self-Alignment of Language Models Abdullah Ashfaq, Eric P. Xing, Somanshu Singla, Tianyang Liu Published: 2024-11-13Area: Alignment TrainingCitations: 12 Tags: ai-safety, alignment-training, empirical | 2024-11-13 | Alignment Training | ai-safety, alignment-training, empirical | E4 / R3 (95%) | 12 |
| Constrain Alignment with Sparse Autoencoders Chak Tou Leong, Hanqi Yan, Jun Wang, Linyi Yang Published: 2024-11-12Area: Alignment TrainingCitations: 10 Tags: ai-safety, alignment-training, empirical | 2024-11-12 | Alignment Training | ai-safety, alignment-training, empirical | E6 / R3 (97%) | 10 |
| Unfair Alignment: Examining Safety Alignment Across Vision Encoder Layers in Vision-Language Models Amit K. Roy-Chowdhury, Arindam Dutta, Chengyu Song, Erfan Shayegani Published: 2024-11-06Area: Multimodal SafetyCitations: 2 Tags: adversarial-robustness, ai-safety, alignment-training, empirical, multimodal-safety | 2024-11-06 | Multimodal Safety | adversarial-robustness, ai-safety, alignment-training, empirical, multimodal-safety | E6 / R3 (96%) | 2 |
| Benchmarking Vision Language Model Unlearning via Fictitious Facial Identity Dataset Bo Li, Chaowei Xiao, Fei Wang, Furong Huang Published: 2024-11-05Area: Model EditingCitations: 14 Tags: ai-safety, alignment-training, benchmark, model-editing | 2024-11-05 | Model Editing | ai-safety, alignment-training, benchmark, model-editing | E6 / R3 (97%) | 14 |
| Stochastic Monkeys at Play: Random Augmentations Cheaply Break LLM Safety Alignment Gagandeep Singh, Gaokai Zhang, Hangoo Kang, Jason Vega Published: 2024-11-05Area: Adversarial RobustnessCitations: 4 Tags: adversarial-robustness, ai-safety, alignment-training, empirical | 2024-11-05 | Adversarial Robustness | adversarial-robustness, ai-safety, alignment-training, empirical | E5 / R3 (93%) | 4 |
| Enhancing Neural Network Interpretability with Feature-Aligned Sparse Autoencoders Alasdair Paren, David Krueger, Fazl Barez, Luke Marks Published: 2024-11-02Area: Mechanistic Interp.Citations: 19 Tags: ai-safety, alignment-training, empirical, interpretability, mechanistic-interp | 2024-11-02 | Mechanistic Interp. | ai-safety, alignment-training, empirical, interpretability, mechanistic-interp | E5 / R3 (95%) | 19 |
| Rule Based Rewards for Language Model Safety Alec Helyar, Alex Beutel, Andrea Vallone, Ian Kivlichan Published: 2024-11-02Area: Alignment TrainingCitations: 108 Tags: ai-safety, alignment-training, empirical | 2024-11-02 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (94%) | 108 |
| Enhancing Safety in Reinforcement Learning with Human Feedback via Rectified Policy Optimization Dongqing Zou, Hengquan Guo, Honghao Wei, Jiawei Zhang Published: 2024-10-25Area: Alignment TrainingCitations: 5 Tags: ai-safety, alignment-training, empirical | 2024-10-25 | Alignment Training | ai-safety, alignment-training, empirical | E6 / R3 (96%) | 5 |
| Inference time LLM alignment in single and multidomain preference spectrum Kishaloy Halder, Manuel Mager, Monica Sunkara, Nikolaos Pappas Published: 2024-10-24Area: Model EditingCitations: 3 Tags: ai-safety, alignment-training, empirical, model-editing | 2024-10-24 | Model Editing | ai-safety, alignment-training, empirical, model-editing | E5 / R3 (95%) | 3 |
| Skywork-Reward: Bag of Tricks for Reward Modeling in LLMs Chaojie Wang, Chris Yuhao Liu, Jiacai Liu, Jujie He Published: 2024-10-24Area: Alignment TrainingCitations: 245 Tags: ai-safety, alignment-training, empirical | 2024-10-24 | Alignment Training | ai-safety, alignment-training, empirical | E6 / R3 (97%) | 245 |
| Towards Understanding the Fragility of Multilingual LLMs against Fine-Tuning Attacks Aobo Yang, Bobbie Chern, Han Zhao, Jianfeng Chi Published: 2024-10-23Area: Adversarial RobustnessCitations: 36 Tags: adversarial-robustness, ai-safety, alignment-training, empirical | 2024-10-23 | Adversarial Robustness | adversarial-robustness, ai-safety, alignment-training, empirical | E5 / R3 (94%) | 36 |
| BiasJailbreak: Analyzing Ethical Biases and Jailbreak Vulnerabilities in Large Language Models Haebin Seong, Isack Lee Published: 2024-10-17Area: Adversarial RobustnessCitations: - Tags: adversarial-robustness, ai-safety, alignment-training, empirical | 2024-10-17 | Adversarial Robustness | adversarial-robustness, ai-safety, alignment-training, empirical | E5 / R3 (94%) | - |
| Persistent Pre-Training Poisoning of LLMs Daphne Ippolito, Eric Michael Smith, Florian Tram猫r, Ivan Evtimov Published: 2024-10-17Area: Adversarial RobustnessCitations: 38 Tags: adversarial-robustness, ai-safety, alignment-training, empirical | 2024-10-17 | Adversarial Robustness | adversarial-robustness, ai-safety, alignment-training, empirical | E8 / R3 (93%) | 38 |
| Insights from the Inverse: Reconstructing LLM Training Goals Through Inverse Reinforcement Learning Arjun Jagota, Jared Joselowitz, Satyapriya Krishna, Sonali Parbhoo Published: 2024-10-16Area: Alignment TrainingCitations: 4 Tags: ai-safety, alignment-training, empirical | 2024-10-16 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R4 (95%) | 4 |
| POROver: Improving Safety and Reducing Overrefusal in Large Language Models with Overgeneration and Preference Optimization Alon Benhaim, Batuhan K. Karaman, Ishmam Zabir, Mert R. Sabuncu Published: 2024-10-16Area: Alignment TrainingCitations: 3 Tags: ai-safety, alignment-training, empirical | 2024-10-16 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (94%) | 3 |
| Weak-to-Strong Generalization beyond Accuracy: a Pilot Study in Safety, Toxicity, and Legal Reasoning Bo Hui, Ruimeng Ye, Yang Xiao Published: 2024-10-16Area: Scalable OversightCitations: 5 Tags: ai-safety, alignment-training, empirical, scalable-oversight | 2024-10-16 | Scalable Oversight | ai-safety, alignment-training, empirical, scalable-oversight | E8 / R4 (93%) | 5 |
| AdvBDGen: Adversarially Fortified Prompt-Specific Fuzzy Backdoor Generator Against LLM Alignment Furong Huang, Michael-Andrei Panaitescu-Liess, Pankayaraj Pathmanathan, Udari Madhushani Sehwag Published: 2024-10-15Area: Adversarial RobustnessCitations: 2 Tags: adversarial-robustness, ai-safety, alignment-training, empirical | 2024-10-15 | Adversarial Robustness | adversarial-robustness, ai-safety, alignment-training, empirical | E5 / R3 (96%) | 2 |