Instant research discovery
Search and browse arXiv CS/AI/ML papers, enriched with AI-generated insights.
Generate novel research ideas grounded in real arXiv papers with Brainstorm.
PreviousNext
| Paper | Published | Area | Tags | Intel | Citations |
|---|---|---|---|---|---|
Search and browse arXiv CS/AI/ML papers, enriched with AI-generated insights.
Generate novel research ideas grounded in real arXiv papers with Brainstorm.
| Paper | Published | Area | Tags | Intel | Citations |
|---|---|---|---|---|---|
Search and browse arXiv CS/AI/ML papers, enriched with AI-generated insights.
Generate novel research ideas grounded in real arXiv papers with Brainstorm.
| Paper | Published | Area | Tags | Intel | Citations |
|---|---|---|---|---|---|
| Paper | Published | Area | Tags | Intel | Citations |
|---|---|---|---|---|---|
| Rule Based Rewards for Language Model Safety Alec Helyar, Alex Beutel, Andrea Vallone, Ian Kivlichan Published: 2024-11-02Area: Alignment TrainingCitations: 108 Tags: ai-safety, alignment-training, empirical | 2024-11-02 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (94%) | 108 |
| Benchmarking Vision Language Model Unlearning via Fictitious Facial Identity Dataset Bo Li, Chaowei Xiao, Fei Wang, Furong Huang Published: 2024-11-05Area: Model EditingCitations: 14 Tags: ai-safety, alignment-training, benchmark, model-editing | 2024-11-05 | Model Editing | ai-safety, alignment-training, benchmark, model-editing | E6 / R3 (97%) | 14 |
| Stochastic Monkeys at Play: Random Augmentations Cheaply Break LLM Safety Alignment Gagandeep Singh, Gaokai Zhang, Hangoo Kang, Jason Vega Published: 2024-11-05Area: Adversarial RobustnessCitations: 4 Tags: adversarial-robustness, ai-safety, alignment-training, empirical | 2024-11-05 | Adversarial Robustness | adversarial-robustness, ai-safety, alignment-training, empirical | E5 / R3 (93%) | 4 |
| Unfair Alignment: Examining Safety Alignment Across Vision Encoder Layers in Vision-Language Models Amit K. Roy-Chowdhury, Arindam Dutta, Chengyu Song, Erfan Shayegani Published: 2024-11-06Area: Multimodal SafetyCitations: 2 Tags: adversarial-robustness, ai-safety, alignment-training, empirical, multimodal-safety | 2024-11-06 | Multimodal Safety | adversarial-robustness, ai-safety, alignment-training, empirical, multimodal-safety | E6 / R3 (96%) | 2 |
| Constrain Alignment with Sparse Autoencoders Chak Tou Leong, Hanqi Yan, Jun Wang, Linyi Yang Published: 2024-11-12Area: Alignment TrainingCitations: 10 Tags: ai-safety, alignment-training, empirical | 2024-11-12 | Alignment Training | ai-safety, alignment-training, empirical | E6 / R3 (97%) | 10 |
| Dynamic Rewarding with Prompt Optimization Enables Tuning-free Self-Alignment of Language Models Abdullah Ashfaq, Eric P. Xing, Somanshu Singla, Tianyang Liu Published: 2024-11-13Area: Alignment TrainingCitations: 12 Tags: ai-safety, alignment-training, empirical | 2024-11-13 | Alignment Training | ai-safety, alignment-training, empirical | E4 / R3 (95%) | 12 |
| Approximated Variational Bayesian Inverse Reinforcement Learning for Large Language Model Alignment Jinsheng Shi, Qinhong Lin, Yuang Cai, Yuyu Yuan Published: 2024-11-14Area: Alignment TrainingCitations: 5 Tags: ai-safety, alignment-training, empirical | 2024-11-14 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (94%) | 5 |
| Enhancing Vision-Language Model Safety through Progressive Concept-Bottleneck-Driven Alignment Bo Zheng, Chongjun Wang, Qiushi Cui, Xiangyu Yue Published: 2024-11-18Area: Multimodal SafetyCitations: 1 Tags: ai-safety, alignment-training, empirical, multimodal-safety | 2024-11-18 | Multimodal Safety | ai-safety, alignment-training, empirical, multimodal-safety | E5 / R3 (96%) | 1 |
| ProSec: Fortifying Code LLMs with Proactive Security Alignment Jinyao Guo, Kaiyuan Zhang, Xiangyu Zhang, Xiangzhe Xu Published: 2024-11-19Area: Alignment TrainingCitations: 13 Tags: ai-safety, alignment-training, empirical | 2024-11-19 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (95%) | 13 |
| What AI evaluations for preventing catastrophic risks can and cannot do Lisa Thiergart, Peter Barnett Published: 2024-11-26Area: Safety EvaluationCitations: 3 Tags: ai-safety, alignment-training, position, safety-evaluation | 2024-11-26 | Safety Evaluation | ai-safety, alignment-training, position, safety-evaluation | E6 / R3 (95%) | 3 |
| Immune: Improving Safety Against Jailbreaks in Multi-modal LLMs via Inference-Time Alignment Ahmad Beirami, Alvaro Velasquez, Amrit Singh Bedi, Dinesh Manocha Published: 2024-11-27Area: Multimodal SafetyCitations: 18 Tags: adversarial-robustness, ai-safety, alignment-training, empirical, multimodal-safety | 2024-11-27 | Multimodal Safety | adversarial-robustness, ai-safety, alignment-training, empirical, multimodal-safety | E5 / R3 (94%) | 18 |
| Jailbreak Large Vision-Language Models Through Multi-Modal Linkage Geyuan Zhang, Tianxing He, Xiaofei Zhou, Yichen Wang Published: 2024-11-30Area: Multimodal SafetyCitations: 37 Tags: adversarial-robustness, ai-safety, alignment-training, empirical, multimodal-safety | 2024-11-30 | Multimodal Safety | adversarial-robustness, ai-safety, alignment-training, empirical, multimodal-safety | E5 / R3 (96%) | 37 |
| LIAR: Leveraging Inference Time Alignment (Best-of-N) to Jailbreak LLMs in Seconds Amrit Singh Bedi, Furong Huang, James Beetham, Mengdi Wang Published: 2024-12-06Area: Adversarial RobustnessCitations: 2 Tags: adversarial-robustness, ai-safety, alignment-training, empirical | 2024-12-06 | Adversarial Robustness | adversarial-robustness, ai-safety, alignment-training, empirical | E5 / R3 (96%) | 2 |
| Semantic Loss Guided Data Efficient Supervised Fine Tuning for Safe Responses in LLMs Arunesh Sinha, Pradeep Varakantham, Yuxiao Lu Published: 2024-12-07Area: Alignment TrainingCitations: 3 Tags: ai-safety, alignment-training, empirical | 2024-12-07 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (95%) | 3 |
| Sail into the Headwind: Alignment via Robust Rewards and Dynamic Labels against Reward Hacking Paria Rashidinejad, Yuandong Tian Published: 2024-12-12Area: Alignment TrainingCitations: 10 Tags: ai-safety, alignment-training, empirical | 2024-12-12 | Alignment Training | ai-safety, alignment-training, empirical | E6 / R3 (97%) | 10 |
| AlignGuard: Scalable Safety Alignment for Text-to-Image Generation Ashkan Khakzar, Fabio Pizzati, I Chieh Chen, Jindong Gu Published: 2024-12-13Area: Multimodal SafetyCitations: 11 Tags: ai-safety, alignment-training, empirical, multimodal-safety | 2024-12-13 | Multimodal Safety | ai-safety, alignment-training, empirical, multimodal-safety | E6 / R4 (98%) | 11 |
| The superalignment of superhuman intelligence with large language models Jie Tang, Minlie Huang, Pei Ke, Shiyao Cui Published: 2024-12-15Area: Scalable OversightCitations: 1 Tags: ai-safety, alignment-training, scalable-oversight, survey | 2024-12-15 | Scalable Oversight | ai-safety, alignment-training, scalable-oversight, survey | E6 / R3 (93%) | 1 |
| NLSR: Neuron-Level Safety Realignment of Large Language Models Against Harmful Fine-Tuning Gerard de Melo, Liang He, Linlin Wang, Shunfan Zheng Published: 2024-12-17Area: Model EditingCitations: 30 Tags: ai-safety, alignment-training, empirical, model-editing | 2024-12-17 | Model Editing | ai-safety, alignment-training, empirical, model-editing | E4 / R3 (94%) | 30 |
| Alignment Faking in Large Language Models Akbir Khan, Benjamin Wright, Buck Shlegeris, Carson Denison Published: 2024-12-18Area: Deception & FailureCitations: 160 Tags: ai-safety, alignment-training, deception-failure, empirical | 2024-12-18 | Deception & Failure | ai-safety, alignment-training, deception-failure, empirical | E5 / R3 (95%) | 160 |
| Deliberative Alignment: Reasoning Enables Safer Language Models Alec Helyar, Alex Beutel, Amelia Glaese, Andrea Vallone Published: 2024-12-20Area: Alignment TrainingCitations: 210 Tags: adversarial-robustness, ai-safety, alignment-training, empirical | 2024-12-20 | Alignment Training | adversarial-robustness, ai-safety, alignment-training, empirical | E5 / R3 (95%) | 210 |
| Chained Tuning Leads to Biased Forgetting Adina Williams, Alicia Sun, Bhaktipriya Radharapu, Levent Sagun Published: 2024-12-21Area: Alignment TrainingCitations: 1 Tags: ai-safety, alignment-training, empirical | 2024-12-21 | Alignment Training | ai-safety, alignment-training, empirical | E6 / R3 (95%) | 1 |
| OpenAI o1 System Card Aaron Jaech, Adam Kalai, Adam Lerer, Ahmed El-Kishky Published: 2024-12-21Area: Safety EvaluationCitations: - Tags: adversarial-robustness, ai-safety, alignment-training, empirical, safety-evaluation | 2024-12-21 | Safety Evaluation | adversarial-robustness, ai-safety, alignment-training, empirical, safety-evaluation | E6 / R3 (97%) | - |
| The Road to Artificial SuperIntelligence: A Comprehensive Survey of Superalignment HyunJin Kim, Jianxun Lian, Jing Yao, JinYeong Bak Published: 2024-12-21Area: Surveys & ReviewsCitations: 9 Tags: ai-safety, alignment-training, survey, surveys-reviews | 2024-12-21 | Surveys & Reviews | ai-safety, alignment-training, survey, surveys-reviews | E6 / R3 (94%) | 9 |
| The Task Shield: Enforcing Task Alignment to Defend Against Indirect Prompt Injection in LLM Agents Anna Squicciarini, Feiran Jia, Tong Wu, Xin Qin Published: 2024-12-21Area: Agent SafetyCitations: 26 Tags: agent-safety, ai-safety, alignment-training, empirical | 2024-12-21 | Agent Safety | agent-safety, ai-safety, alignment-training, empirical | E5 / R3 (96%) | 26 |
| When Can Proxies Improve the Sample Complexity of Preference Learning? Alexander D'Amour, Daniel Augusto de Souza, Matt J. Kusner, Mengyue Yang Published: 2024-12-21Area: Alignment TrainingCitations: 1 Tags: ai-safety, alignment-training, theoretical | 2024-12-21 | Alignment Training | ai-safety, alignment-training, theoretical | E4 / R3 (95%) | 1 |
| Large Language Model Safety: A Holistic Survey Bojian Jiang, Chuang Liu, Dan Shi, Deyi Xiong Published: 2024-12-23Area: Surveys & ReviewsCitations: 47 Tags: adversarial-robustness, ai-safety, alignment-training, interpretability, survey, surveys-reviews | 2024-12-23 | Surveys & Reviews | adversarial-robustness, ai-safety, alignment-training, interpretability, survey, surveys-reviews | E7 / R5 (99%) | 47 |
| Are the Values of LLMs Structurally Aligned with Humans? A Causal Perspective Fangwei Zhong, Hengli Li, Junqi Wang, Mengmeng Wang Published: 2024-12-31Area: Alignment TrainingCitations: 2 Tags: ai-safety, alignment-training, empirical | 2024-12-31 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (96%) | 2 |
| Analyzing Fine-tuning Representation Shift for Multimodal LLMs Steering alignment Jayneel Parekh, Matthieu Cord, Mustafa Shukor, Pegah Khayatan Published: 2025-01-06Area: Representation AnalysisCitations: 8 Tags: ai-safety, alignment-training, empirical, representation-analysis | 2025-01-06 | Representation Analysis | ai-safety, alignment-training, empirical, representation-analysis | E5 / R3 (96%) | 8 |
| Understanding Impact of Human Feedback via Influence Functions Haeone Lee, Hanho Ryu, Kimin Lee, Taywon Min Published: 2025-01-10Area: Alignment TrainingCitations: 7 Tags: ai-safety, alignment-training, empirical | 2025-01-10 | Alignment Training | ai-safety, alignment-training, empirical | E6 / R4 (94%) | 7 |
| Iterative Label Refinement Matters More than Preference Optimization under Weak Supervision Cassidy Laidlaw, Jacob Steinhardt, Yaowen Ye Published: 2025-01-14Area: Alignment TrainingCitations: 3 Tags: ai-safety, alignment-training, empirical | 2025-01-14 | Alignment Training | ai-safety, alignment-training, empirical | E7 / R3 (95%) | 3 |