Instant research discovery
Search and browse arXiv CS/AI/ML papers, enriched with AI-generated insights.
Generate novel research ideas grounded in real arXiv papers with Brainstorm.
PreviousNext
| Paper | Published | Area | Tags | Intel | Citations |
|---|---|---|---|---|---|
Search and browse arXiv CS/AI/ML papers, enriched with AI-generated insights.
Generate novel research ideas grounded in real arXiv papers with Brainstorm.
| Paper | Published | Area | Tags | Intel | Citations |
|---|---|---|---|---|---|
Search and browse arXiv CS/AI/ML papers, enriched with AI-generated insights.
Generate novel research ideas grounded in real arXiv papers with Brainstorm.
| Paper | Published | Area | Tags | Intel | Citations |
|---|---|---|---|---|---|
| Paper | Published | Area | Tags | Intel | Citations |
|---|---|---|---|---|---|
| Mitigating Fine-tuning Risks in LLMs via Safety-Aware Probing Optimization Chengcan Wu, Meng Sun, Yihao Zhang, Zeming Wei Published: 2025-05-22Area: Alignment TrainingCitations: 10 Tags: ai-safety, alignment-training, empirical | 2025-05-22 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (95%) | 10 |
| MPO: Multilingual Safety Alignment via Reward Gap Optimization An Zhang, Bing Qin, Jiahe Guo, Tat-Seng Chua Published: 2025-05-22Area: Alignment TrainingCitations: 8 Tags: ai-safety, alignment-training, empirical | 2025-05-22 | Alignment Training | ai-safety, alignment-training, empirical | E6 / R4 (97%) | 8 |
| MTSA: Multi-turn Safety Alignment for LLMs through Multi-round Red-teaming Daojing He, Jing Li, Jun Yu, Min Zhang Published: 2025-05-22Area: Safety EvaluationCitations: 18 Tags: adversarial-robustness, ai-safety, alignment-training, empirical, safety-evaluation | 2025-05-22 | Safety Evaluation | adversarial-robustness, ai-safety, alignment-training, empirical, safety-evaluation | E4 / R3 (94%) | 18 |
| Refusal Direction is Universal Across Safety-Aligned Languages Barbara Plank, Hinrich Sch眉tze, Mingyang Wang, Xinpeng Wang Published: 2025-05-22Area: Representation AnalysisCitations: 5 Tags: ai-safety, alignment-training, empirical, representation-analysis | 2025-05-22 | Representation Analysis | ai-safety, alignment-training, empirical, representation-analysis | E6 / R3 (94%) | 5 |
| Shape it Up! Restoring LLM Safety during Finetuning Duen Horng Chau, Jianfeng Chi, Pin-Yu Chen, Seongmin Lee Published: 2025-05-22Area: Alignment TrainingCitations: 6 Tags: ai-safety, alignment-training, empirical | 2025-05-22 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (93%) | 6 |
| Towards medical AI misalignment: a preliminary study Allan Tucker, Barbara Puccio, Federico Castagna, Pierangelo Veltri Published: 2025-05-22Area: Adversarial RobustnessCitations: - Tags: adversarial-robustness, ai-safety, alignment-training, empirical | 2025-05-22 | Adversarial Robustness | adversarial-robustness, ai-safety, alignment-training, empirical | E5 / R3 (96%) | - |
| Advancing LLM Safe Alignment with Safety Representation Ranking Chenheng Zhang, Quan Chen, Tianqi Du, Yisen Wang Published: 2025-05-21Area: Adversarial RobustnessCitations: 8 Tags: adversarial-robustness, ai-safety, alignment-training, empirical | 2025-05-21 | Adversarial Robustness | adversarial-robustness, ai-safety, alignment-training, empirical | E4 / R3 (95%) | 8 |
| How Should We Enhance the Safety of Large Reasoning Models: An Empirical Study Fei Mi, Hongning Wang, Junxiao Yang, Lifeng Shang Published: 2025-05-21Area: Alignment TrainingCitations: 16 Tags: ai-safety, alignment-training, empirical | 2025-05-21 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (95%) | 16 |
| Context Reasoner: Incentivizing Reasoning Capability for Contextualized Privacy and Safety Compliance via Reinforcement Learning Haoran Li, Heli Xu, Huihao Jing, Peizhao Hu Published: 2025-05-20Area: Alignment TrainingCitations: 3 Tags: ai-safety, alignment-training, empirical | 2025-05-20 | Alignment Training | ai-safety, alignment-training, empirical | E7 / R3 (96%) | 3 |
| "Haet Bhasha aur Diskrimineshun": Phonetic Perturbations in Code-Mixed Hinglish to Red-Team LLMs Darpan Aswal, Siddharth D Jaiswal Published: 2025-05-20Area: Adversarial RobustnessCitations: - Tags: adversarial-robustness, ai-safety, alignment-training, empirical | 2025-05-20 | Adversarial Robustness | adversarial-robustness, ai-safety, alignment-training, empirical | E6 / R3 (96%) | - |
| SAFEPATH: Preventing Harmful Reasoning in Chain-of-Thought via Early Alignment Albert No, Minsuk Kahng, Sangyeon Yoon, Wonje Jeung Published: 2025-05-20Area: Alignment TrainingCitations: 10 Tags: ai-safety, alignment-training, empirical | 2025-05-20 | Alignment Training | ai-safety, alignment-training, empirical | E6 / R3 (96%) | 10 |
| Safety Subspaces are Not Linearly Distinct: A Fine-Tuning Case Study Kaustubh Ponkshe, Praneeth Vepakomma, Raghav Singhal, Shaan Shah Published: 2025-05-20Area: Alignment TrainingCitations: 1 Tags: ai-safety, alignment-training, empirical | 2025-05-20 | Alignment Training | ai-safety, alignment-training, empirical | E6 / R3 (95%) | 1 |
| The Hallucination Tax of Reinforcement Finetuning Jieyu Zhao, Linxin Song, Taiwei Shi Published: 2025-05-20Area: Alignment TrainingCitations: 21 Tags: ai-safety, alignment-training, empirical | 2025-05-20 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (95%) | 21 |
| Bias Fitting to Mitigate Length Bias of Reward Model in RLHF Dongyun Xue, Houqiang Li, Jianfeng Cai, Jinhua Zhu Published: 2025-05-19Area: Alignment TrainingCitations: 2 Tags: ai-safety, alignment-training, empirical | 2025-05-19 | Alignment Training | ai-safety, alignment-training, empirical | E6 / R4 (95%) | 2 |
| Rethinking Reward Model Evaluation Through the Lens of Reward Overoptimization Dongha Lee, Dongjin Kang, Hyungjoo Chae, Jinyoung Yeo Published: 2025-05-19Area: Alignment TrainingCitations: 3 Tags: ai-safety, alignment-training, empirical, safety-evaluation | 2025-05-19 | Alignment Training | ai-safety, alignment-training, empirical, safety-evaluation | E6 / R3 (96%) | 3 |
| Safety Alignment Can Be Not Superficial With Explicit Safety Signals Jianwei Li, Jung-Eun Kim Published: 2025-05-19Area: Adversarial RobustnessCitations: 3 Tags: adversarial-robustness, ai-safety, alignment-training, empirical | 2025-05-19 | Adversarial Robustness | adversarial-robustness, ai-safety, alignment-training, empirical | E5 / R4 (96%) | 3 |
| ABoN: Adaptive Best-of-N Alignment Hilal Asi, Satyen Kale, Vinod Raman Published: 2025-05-17Area: Alignment TrainingCitations: 4 Tags: ai-safety, alignment-training, empirical | 2025-05-17 | Alignment Training | ai-safety, alignment-training, empirical | E6 / R3 (94%) | 4 |
| Safe Delta: Consistently Preserving Safety when Fine-Tuning LLMs on Diverse Datasets Jiahao Wu, Ke Tang, Ning Lu, Qi Wang Published: 2025-05-17Area: Adversarial RobustnessCitations: 13 Tags: adversarial-robustness, ai-safety, alignment-training, empirical | 2025-05-17 | Adversarial Robustness | adversarial-robustness, ai-safety, alignment-training, empirical | E5 / R3 (95%) | 13 |
| SafeVid: Toward Safety Aligned Video Large Multimodal Models Jiaxin Song, Xingjun Ma, Xin Wang, Yang Yao Published: 2025-05-17Area: Multimodal SafetyCitations: 4 Tags: ai-safety, alignment-training, empirical, multimodal-safety | 2025-05-17 | Multimodal Safety | ai-safety, alignment-training, empirical, multimodal-safety | E5 / R4 (95%) | 4 |
| Two Minds Better Than One: Collaborative Reward Modeling for LLM Alignment Jiahuan Li, Jiazheng Zhang, Jingang Wang, MingXu Chai Published: 2025-05-15Area: Alignment TrainingCitations: 2 Tags: ai-safety, alignment-training, empirical | 2025-05-15 | Alignment Training | ai-safety, alignment-training, empirical | E6 / R4 (95%) | 2 |
| On the Robustness of Reward Models for Language Model Alignment Aman Gupta, Eunki Kim, Guijin Son, James Thorne Published: 2025-05-12Area: Alignment TrainingCitations: 7 Tags: ai-safety, alignment-training, empirical | 2025-05-12 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (95%) | 7 |
| Think in Safety: Unveiling and Mitigating Safety Alignment Collapse in Multimodal Large Reasoning Model Chi Chen, Jinan Xu, Kaiyu Huang, Xiangyu Shi Published: 2025-05-10Area: Multimodal SafetyCitations: 11 Tags: adversarial-robustness, ai-safety, alignment-training, empirical, multimodal-safety, safety-evaluation | 2025-05-10 | Multimodal Safety | adversarial-robustness, ai-safety, alignment-training, empirical, multimodal-safety, safety-evaluation | E5 / R3 (95%) | 11 |
| An Alignment Safety Case Sketch Based on Debate Benjamin Hilton, Geoffrey Irving, Jacob Pfau, Marie Davidsen Buhl Published: 2025-05-06Area: Scalable OversightCitations: 8 Tags: ai-safety, alignment-training, scalable-oversight, theoretical | 2025-05-06 | Scalable Oversight | ai-safety, alignment-training, scalable-oversight, theoretical | E5 / R3 (94%) | 8 |
| LlamaFirewall: An Open Source Guardrail System for Building Secure AI Agents Abraham Montilla, Alekhya Gampa, Beto de Paola, Cyrus Nikolaidis Published: 2025-05-06Area: Adversarial RobustnessCitations: 41 Tags: adversarial-robustness, ai-safety, alignment-training, tool | 2025-05-06 | Adversarial Robustness | adversarial-robustness, ai-safety, alignment-training, tool | E5 / R4 (98%) | 41 |
| A Survey on Progress in LLM Alignment from the Perspective of Reward Design Jian Yang, Mark Dras, Miaomiao Ji, Shoujin Wang Published: 2025-05-05Area: Surveys & ReviewsCitations: 10 Tags: ai-safety, alignment-training, survey, surveys-reviews | 2025-05-05 | Surveys & Reviews | ai-safety, alignment-training, survey, surveys-reviews | E6 / R4 (95%) | 10 |
| On the Limitations of Steering in Language Model Alignment Chebrolu Niranjan, Gerard Christopher Yeo, Kokil Jaidka Published: 2025-05-02Area: Representation AnalysisCitations: 3 Tags: ai-safety, alignment-training, empirical, representation-analysis | 2025-05-02 | Representation Analysis | ai-safety, alignment-training, empirical, representation-analysis | E4 / R3 (94%) | 3 |
| Characterizing AI Agents for Alignment and Governance Atoosa Kasirzadeh, Iason Gabriel Published: 2025-04-30Area: Agent SafetyCitations: 28 Tags: agent-safety, ai-safety, alignment-training, theoretical | 2025-04-30 | Agent Safety | agent-safety, ai-safety, alignment-training, theoretical | E6 / R5 (95%) | 28 |
| XBreaking: Explainable Artificial Intelligence for Jailbreaking LLMs Antonino Nocera, Marco Arazzi, Vignesh Kumar Kembu, Vinod P Published: 2025-04-30Area: Adversarial RobustnessCitations: - Tags: adversarial-robustness, ai-safety, alignment-training, empirical | 2025-04-30 | Adversarial Robustness | adversarial-robustness, ai-safety, alignment-training, empirical | E5 / R3 (95%) | - |
| Toward Evaluative Thinking: Meta Policy Optimization with Evolving Reward Models Chanwoo Park, Dongyeop Kang, Suin Kim, Vipul Raheja Published: 2025-04-28Area: Alignment TrainingCitations: 3 Tags: ai-safety, alignment-training, empirical | 2025-04-28 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (96%) | 3 |
| Adaptive Helpfulness-Harmlessness Alignment with Preference Vectors Chan-Hung Yu, Chin-Ting Hsu, Kuan-Hao Huang, Ren-Wei Liang Published: 2025-04-27Area: Alignment TrainingCitations: 2 Tags: ai-safety, alignment-training, empirical | 2025-04-27 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (94%) | 2 |