Instant research discovery
Search and browse arXiv CS/AI/ML papers, enriched with AI-generated insights.
Generate novel research ideas grounded in real arXiv papers with Brainstorm.
PreviousNext
| Paper | Published | Area | Tags | Intel | Citations |
|---|---|---|---|---|---|
Search and browse arXiv CS/AI/ML papers, enriched with AI-generated insights.
Generate novel research ideas grounded in real arXiv papers with Brainstorm.
| Paper | Published | Area | Tags | Intel | Citations |
|---|---|---|---|---|---|
Search and browse arXiv CS/AI/ML papers, enriched with AI-generated insights.
Generate novel research ideas grounded in real arXiv papers with Brainstorm.
| Paper | Published | Area | Tags | Intel | Citations |
|---|---|---|---|---|---|
| Paper | Published | Area | Tags | Intel | Citations |
|---|---|---|---|---|---|
| Helping or Herding? Reward Model Ensembles Mitigate but do not Eliminate Reward Hacking Adam Fisch, Ahmad Beirami, Alekh Agarwal, Alex D'Amour Published: 2023-12-14Area: Deception & FailureCitations: 147 Tags: ai-safety, deception-failure, empirical | 2023-12-14 | Deception & Failure | ai-safety, deception-failure, empirical | E5 / R3 (92%) | 147 |
| Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations Damai Dai, Deli Chen, Lei Li, Peiyi Wang Published: 2023-12-14Area: Alignment TrainingCitations: 725 Tags: ai-safety, alignment-training, empirical | 2023-12-14 | Alignment Training | ai-safety, alignment-training, empirical | E6 / R4 (96%) | 725 |
| Successor Heads: Recurring, Interpretable Attention Heads In The Wild Arthur Conmy, Euan Ong, George Ogden, Rhys Gould Published: 2023-12-14Area: Mechanistic Interp.Citations: 69 Tags: ai-safety, empirical, mechanistic-interp | 2023-12-14 | Mechanistic Interp. | ai-safety, empirical, mechanistic-interp | E6 / R3 (97%) | 69 |
| The Earth is Flat because...: Investigating LLMs' Belief towards Misinformation via Persuasive Conversation Brian S. Lin, Han Qiu, Rongwu Xu, Shujian Yang Published: 2023-12-14Area: Deception & FailureCitations: 101 Tags: ai-safety, deception-failure, empirical | 2023-12-14 | Deception & Failure | ai-safety, deception-failure, empirical | E5 / R3 (96%) | 101 |
| Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervision Adrien Ecoffet, Bowen Baker, Collin Burns, Ilya Sutskever Published: 2023-12-14Area: Scalable OversightCitations: 409 Tags: ai-safety, empirical, scalable-oversight | 2023-12-14 | Scalable Oversight | ai-safety, empirical, scalable-oversight | E5 / R3 (95%) | 409 |
| Challenges with Unsupervised LLM Knowledge Discovery Johannes Gasteiger, Rohin Shah, Sebastian Farquhar, Vikrant Varma Published: 2023-12-15Area: Representation AnalysisCitations: 35 Tags: ai-safety, empirical, representation-analysis | 2023-12-15 | Representation Analysis | ai-safety, empirical, representation-analysis | E5 / R3 (98%) | 35 |
| A Mutation-Based Method for Multi-Modal Jailbreaking Attack Detection Cen Zhang, Chao Shen, Tianlin Li, Xiaofei Xie Published: 2023-12-17Area: Adversarial RobustnessCitations: 44 Tags: adversarial-robustness, ai-safety, empirical | 2023-12-17 | Adversarial Robustness | adversarial-robustness, ai-safety, empirical | E6 / R4 (95%) | 44 |
| Bypassing the Safety Training of Open-Source LLMs with Priming Attacks Changming Xu, Gagandeep Singh, Isha Chaudhary, Jason Vega Published: 2023-12-19Area: Adversarial RobustnessCitations: 43 Tags: adversarial-robustness, ai-safety, empirical | 2023-12-19 | Adversarial Robustness | adversarial-robustness, ai-safety, empirical | E5 / R3 (96%) | 43 |
| Learning and Forgetting Unsafe Examples in Large Language Models David Madras, James Zou, Jiachen Zhao, Mengye Ren Published: 2023-12-20Area: Alignment TrainingCitations: 25 Tags: ai-safety, alignment-training, empirical | 2023-12-20 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (93%) | 25 |
| Exploiting Novel GPT-4 APIs Adam Gleave, Euan McLean, Kellin Pelrine, Micha艂 Zaj膮c Published: 2023-12-21Area: Adversarial RobustnessCitations: 28 Tags: adversarial-robustness, ai-safety, empirical | 2023-12-21 | Adversarial Robustness | adversarial-robustness, ai-safety, empirical | E5 / R3 (96%) | 28 |
| Observable Propagation: Uncovering Feature Vectors in Transformers Arman Cohan, Jacob Dunefsky Published: 2023-12-26Area: Mechanistic Interp.Citations: 2 Tags: ai-safety, empirical, mechanistic-interp | 2023-12-26 | Mechanistic Interp. | ai-safety, empirical, mechanistic-interp | E5 / R3 (93%) | 2 |
| Jatmo: Prompt Injection Defense by Task-Specific Finetuning Basel Alomair, Chawin Sitawarin, David Wagner, Elizabeth Sun Published: 2023-12-29Area: Adversarial RobustnessCitations: 102 Tags: adversarial-robustness, ai-safety, empirical | 2023-12-29 | Adversarial Robustness | adversarial-robustness, ai-safety, empirical | E5 / R3 (95%) | 102 |
| Uncertainty-Penalized Reinforcement Learning from Human Feedback with Diverse Reward LoRA Ensembles Bo Ding, Dawei Feng, Han Zhang, Huaimin Wang Published: 2023-12-30Area: Alignment TrainingCitations: 43 Tags: ai-safety, alignment-training, empirical | 2023-12-30 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (96%) | 43 |
| A Mechanistic Understanding of Alignment Algorithms: A Case Study on DPO and Toxicity Andrew Lee, Itamar Pres, Jonathan K. Kummerfeld, Martin Wattenberg Published: 2024-01-03Area: Mechanistic Interp.Citations: 165 Tags: ai-safety, alignment-training, empirical, mechanistic-interp | 2024-01-03 | Mechanistic Interp. | ai-safety, alignment-training, empirical, mechanistic-interp | E5 / R3 (94%) | 165 |
| Evaluating Brain-Inspired Modular Training in Automated Circuit Discovery for Mechanistic Interpretability Jatin Nainani Published: 2024-01-08Area: Mechanistic Interp.Citations: 3 Tags: ai-safety, empirical, interpretability, mechanistic-interp | 2024-01-08 | Mechanistic Interp. | ai-safety, empirical, interpretability, mechanistic-interp | E5 / R3 (95%) | 3 |
| Agent Alignment in Evolving Social Norms Qinyuan Cheng, Shimin Li, Tianxiang Sun, Xipeng Qiu Published: 2024-01-09Area: Agent SafetyCitations: 12 Tags: agent-safety, ai-safety, alignment-training, empirical | 2024-01-09 | Agent Safety | agent-safety, ai-safety, alignment-training, empirical | E5 / R3 (96%) | 12 |
| Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training Adam Jermyn, Amanda Askell, Ansh Radhakrishnan, Buck Shlegeris Published: 2024-01-10Area: Deception & FailureCitations: 303 Tags: adversarial-robustness, ai-safety, deception-failure, empirical | 2024-01-10 | Deception & Failure | adversarial-robustness, ai-safety, deception-failure, empirical | E8 / R3 (97%) | 303 |
| EraseDiff: Erasing Data Influence in Diffusion Models Jing Wu, Mehrtash Harandi, Munawar Hayat, Trung Le Published: 2024-01-11Area: Model EditingCitations: 28 Tags: ai-safety, empirical, model-editing | 2024-01-11 | Model Editing | ai-safety, empirical, model-editing | E5 / R3 (94%) | 28 |
| Manipulating Feature Visualizations with Gradient Slingshots Alexander Warnecke, Dilyara Bareeva, Kirill Bykov, Klaus-Robert M眉ller Published: 2024-01-11Area: Adversarial RobustnessCitations: 6 Tags: adversarial-robustness, ai-safety, empirical | 2024-01-11 | Adversarial Robustness | adversarial-robustness, ai-safety, empirical | E5 / R3 (95%) | 6 |
| Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Models Adam Pearce, Asma Ghandeharioun, Avi Caciularu, Lucas Dixon Published: 2024-01-11Area: Representation AnalysisCitations: 174 Tags: ai-safety, empirical, interpretability, representation-analysis | 2024-01-11 | Representation Analysis | ai-safety, empirical, interpretability, representation-analysis | E5 / R3 (95%) | 174 |
| Secrets of RLHF in Large Language Models Part II: Reward Modeling Binghai Wang, Caishuang Huang, Chenyu Shi, Enyu Zhou Published: 2024-01-11Area: Alignment TrainingCitations: 148 Tags: ai-safety, alignment-training, empirical | 2024-01-11 | Alignment Training | ai-safety, alignment-training, empirical | E6 / R4 (94%) | 148 |
| Universal Vulnerabilities in Large Language Models: Backdoor Attacks for In-context Learning Fengjun Pan, Jinming Wen, Luu Anh Tuan, Meihuizi Jia Published: 2024-01-11Area: Adversarial RobustnessCitations: 73 Tags: adversarial-robustness, ai-safety, empirical | 2024-01-11 | Adversarial Robustness | adversarial-robustness, ai-safety, empirical | E4 / R3 (97%) | 73 |
| How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMs Diyi Yang, Hongpeng Lin, Jingwen Zhang, Ruoxi Jia Published: 2024-01-12Area: Adversarial RobustnessCitations: 529 Tags: adversarial-robustness, ai-safety, empirical | 2024-01-12 | Adversarial Robustness | adversarial-robustness, ai-safety, empirical | E6 / R4 (97%) | 529 |
| Intention Analysis Makes LLMs A Good Jailbreak Defender Dacheng Tao, Lefei Zhang, Liang Ding, Yuqi Zhang Published: 2024-01-12Area: Adversarial RobustnessCitations: 63 Tags: adversarial-robustness, ai-safety, empirical | 2024-01-12 | Adversarial Robustness | adversarial-robustness, ai-safety, empirical | E5 / R3 (96%) | 63 |
| Revisiting Jailbreaking for Large Language Models: A Representation Engineering Perspective Changze Lv, Muling Wu, Shihan Dou, Tianlong Li Published: 2024-01-12Area: Adversarial RobustnessCitations: 34 Tags: adversarial-robustness, ai-safety, empirical | 2024-01-12 | Adversarial Robustness | adversarial-robustness, ai-safety, empirical | E5 / R3 (94%) | 34 |
| The Unreasonable Effectiveness of Easy Training Data for Hard Tasks Mohit Bansal, Peter Clark, Peter Hase, Sarah Wiegreffe Published: 2024-01-12Area: Scalable OversightCitations: 47 Tags: ai-safety, empirical, scalable-oversight | 2024-01-12 | Scalable Oversight | ai-safety, empirical, scalable-oversight | E5 / R3 (95%) | 47 |
| Self-Rewarding Language Models Jason Weston, Jing Xu, Kyunghyun Cho, Richard Yuanzhe Pang Published: 2024-01-18Area: Alignment TrainingCitations: 504 Tags: ai-safety, alignment-training, empirical | 2024-01-18 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (96%) | 504 |
| BadChain: Backdoor Chain-of-Thought Prompting for Large Language Models Bhaskar Ramasubramanian, Bo Li, Fengqing Jiang, Radha Poovendran Published: 2024-01-20Area: Adversarial RobustnessCitations: 85 Tags: adversarial-robustness, ai-safety, empirical | 2024-01-20 | Adversarial Robustness | adversarial-robustness, ai-safety, empirical | E5 / R3 (98%) | 85 |
| InferAligner: Inference-Time Alignment for Harmlessness through Cross-Model Guidance Botian Jiang, Chenkun Tan, Dong Zhang, Ke Ren Published: 2024-01-20Area: Model EditingCitations: 78 Tags: ai-safety, alignment-training, empirical, model-editing | 2024-01-20 | Model Editing | ai-safety, alignment-training, empirical, model-editing | E5 / R3 (95%) | 78 |
| PsySafe: A Comprehensive Framework for Psychological-based Attack, Defense, and Evaluation of Multi-agent System Safety Feng Zhao, Hongzhi Gao, Huchuan Lu, Jing Shao Published: 2024-01-22Area: Agent SafetyCitations: 75 Tags: agent-safety, ai-safety, empirical, safety-evaluation | 2024-01-22 | Agent Safety | agent-safety, ai-safety, empirical, safety-evaluation | E6 / R4 (95%) | 75 |