Instant research discovery
Search and browse arXiv CS/AI/ML papers, enriched with AI-generated insights.
Generate novel research ideas grounded in real arXiv papers with Brainstorm.
PreviousNext
| Paper | Published | Area | Tags | Intel | Citations |
|---|---|---|---|---|---|
Search and browse arXiv CS/AI/ML papers, enriched with AI-generated insights.
Generate novel research ideas grounded in real arXiv papers with Brainstorm.
| Paper | Published | Area | Tags | Intel | Citations |
|---|---|---|---|---|---|
Search and browse arXiv CS/AI/ML papers, enriched with AI-generated insights.
Generate novel research ideas grounded in real arXiv papers with Brainstorm.
| Paper | Published | Area | Tags | Intel | Citations |
|---|---|---|---|---|---|
| Paper | Published | Area | Tags | Intel | Citations |
|---|---|---|---|---|---|
| Universal Neurons in GPT2 Language Models Dimitris Bertsimas, Neel Nanda, Qinyi Sun, Tara Rezaei Kheirkhah Published: 2024-01-22Area: Mechanistic Interp.Citations: 83 Tags: ai-safety, empirical, mechanistic-interp | 2024-01-22 | Mechanistic Interp. | ai-safety, empirical, mechanistic-interp | E5 / R3 (93%) | 83 |
| WARM: On the Benefits of Weight Averaged Reward Models Alexandre Ram茅, Geoffrey Cideron, Johan Ferret, L茅onard Hussenot Published: 2024-01-22Area: Alignment TrainingCitations: 134 Tags: ai-safety, alignment-training, empirical | 2024-01-22 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (94%) | 134 |
| A Reply to Makelov et al. (2023)'s "Interpretability Illusion" Arguments Aryaman Arora, Atticus Geiger, Christopher Potts, Jing Huang Published: 2024-01-23Area: Mechanistic Interp.Citations: 9 Tags: ai-safety, empirical, interpretability, mechanistic-interp | 2024-01-23 | Mechanistic Interp. | ai-safety, empirical, interpretability, mechanistic-interp | E5 / R3 (94%) | 9 |
| ARGS: Alignment as Reward-Guided Search Jirayu Burapacheep, Maxim Khanov, Yixuan Li Published: 2024-01-23Area: Alignment TrainingCitations: 95 Tags: ai-safety, alignment-training, empirical | 2024-01-23 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (96%) | 95 |
| Fluent dreaming for language models Michael Sklar, T. Ben Thompson, Zygimantas Straznickas Published: 2024-01-24Area: Mechanistic Interp.Citations: 4 Tags: adversarial-robustness, ai-safety, empirical, mechanistic-interp | 2024-01-24 | Mechanistic Interp. | adversarial-robustness, ai-safety, empirical, mechanistic-interp | E4 / R2 (97%) | 4 |
| Fortifying Ethical Boundaries in AI: Advanced Strategies for Enhancing Security in Large Language Models Jianling Qiu, Wei Zhang, Yunhong He, Zhengqing Yuan Published: 2024-01-27Area: Adversarial RobustnessCitations: 3 Tags: adversarial-robustness, ai-safety, empirical | 2024-01-27 | Adversarial Robustness | adversarial-robustness, ai-safety, empirical | E6 / R3 (94%) | 3 |
| Iterative Data Smoothing: Mitigating Reward Overfitting and Overoptimization in RLHF Banghua Zhu, Jiantao Jiao, Michael I. Jordan Published: 2024-01-29Area: Alignment TrainingCitations: 49 Tags: ai-safety, alignment-training, empirical | 2024-01-29 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (95%) | 49 |
| Robust Prompt Optimization for Defending Language Models Against Jailbreaking Attacks Andy Zhou, Bo Li, Haohan Wang Published: 2024-01-30Area: Adversarial RobustnessCitations: 141 Tags: adversarial-robustness, ai-safety, empirical | 2024-01-30 | Adversarial Robustness | adversarial-robustness, ai-safety, empirical | E5 / R4 (97%) | 141 |
| Weak-to-Strong Jailbreaking on Large Language Models Chao Du, Lei Li, Tianyu Pang, William Yang Wang Published: 2024-01-30Area: Adversarial RobustnessCitations: 96 Tags: adversarial-robustness, ai-safety, empirical | 2024-01-30 | Adversarial Robustness | adversarial-robustness, ai-safety, empirical | E5 / R3 (98%) | 96 |
| Investigating Bias Representations in Llama 2 Chat via Activation Steering Dawn Lu, Nina Rimsky Published: 2024-02-01Area: Representation AnalysisCitations: 13 Tags: ai-safety, empirical, representation-analysis | 2024-02-01 | Representation Analysis | ai-safety, empirical, representation-analysis | E5 / R4 (93%) | 13 |
| Towards Efficient and Exact Optimization of Language Model Alignment Cheng Lu, Haozhe Ji, Hongning Wang, Jie Tang Published: 2024-02-01Area: Alignment TrainingCitations: 32 Tags: ai-safety, alignment-training, empirical | 2024-02-01 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (95%) | 32 |
| KTO: Model Alignment as Prospect Theoretic Optimization Dan Jurafsky, Douwe Kiela, Kawin Ethayarajh, Niklas Muennighoff Published: 2024-02-02Area: Alignment TrainingCitations: 879 Tags: ai-safety, alignment-training, empirical | 2024-02-02 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (97%) | 879 |
| Preference Poisoning Attacks on Reward Model Learning Chaowei Xiao, Chenguang Wang, Jiongxiao Wang, Junlin Wu Published: 2024-02-02Area: Adversarial RobustnessCitations: 12 Tags: adversarial-robustness, ai-safety, alignment-training, empirical | 2024-02-02 | Adversarial Robustness | adversarial-robustness, ai-safety, alignment-training, empirical | E5 / R4 (96%) | 12 |
| Rethinking the Role of Proxy Rewards in Language Model Alignment Minjoon Seo, Sungdong Kim Published: 2024-02-02Area: Alignment TrainingCitations: 5 Tags: ai-safety, alignment-training, empirical | 2024-02-02 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (97%) | 5 |
| Vaccine: Perturbation-aware Alignment for Large Language Models Ling Liu, Sihao Hu, Tiansheng Huang Published: 2024-02-02Area: Alignment TrainingCitations: 88 Tags: ai-safety, alignment-training, empirical | 2024-02-02 | Alignment Training | ai-safety, alignment-training, empirical | E4 / R3 (94%) | 88 |
| Data Poisoning for In-context Learning Han Xu, Hui Liu, Jiliang Tang, Makoto Yamada Published: 2024-02-03Area: Adversarial RobustnessCitations: 27 Tags: adversarial-robustness, ai-safety, empirical | 2024-02-03 | Adversarial Robustness | adversarial-robustness, ai-safety, empirical | E4 / R3 (97%) | 27 |
| Safety Fine-Tuning at (Almost) No Cost: A Baseline for Vision Large Language Models Ondrej Bohdal, Timothy Hospedales, Tingyang Yu, Yongshuo Zong Published: 2024-02-03Area: Multimodal SafetyCitations: 125 Tags: adversarial-robustness, ai-safety, empirical, multimodal-safety | 2024-02-03 | Multimodal Safety | adversarial-robustness, ai-safety, empirical, multimodal-safety | E6 / R3 (94%) | 125 |
| Jailbreaking Attack against Multimodal Large Language Model Gang Hua, Haodong Ren, Rong Jin, Xinbo Gao Published: 2024-02-04Area: Multimodal SafetyCitations: 128 Tags: adversarial-robustness, ai-safety, empirical, multimodal-safety | 2024-02-04 | Multimodal Safety | adversarial-robustness, ai-safety, empirical, multimodal-safety | E8 / R3 (95%) | 128 |
| The Developmental Landscape of In-Context Learning Daniel Murfet, George Wang, Jesse Hoogland, Liam Carroll Published: 2024-02-04Area: Training DynamicsCitations: 19 Tags: ai-safety, empirical, training-dynamics | 2024-02-04 | Training Dynamics | ai-safety, empirical, training-dynamics | E5 / R3 (96%) | 19 |
| DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models Daya Guo, Haowei Zhang, Junxiao Song, Mingchuan Zhang Published: 2024-02-05Area: Alignment TrainingCitations: 4632 Tags: ai-safety, alignment-training, empirical | 2024-02-05 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R3 (98%) | 4632 |
| GUARD: Role-playing to Generate Natural-language Jailbreakings to Test Guideline Adherence of Large Language Models Andy Zhou, Haibo Jin, Haohan Wang, Peiyan Zhang Published: 2024-02-05Area: Adversarial RobustnessCitations: 49 Tags: adversarial-robustness, ai-safety, empirical | 2024-02-05 | Adversarial Robustness | adversarial-robustness, ai-safety, empirical | E8 / R3 (97%) | 49 |
| Shadowcast: Stealthy Data Poisoning Attacks Against Vision-Language Models Furong Huang, Jiarui Yao, Manli Shu, Ning Yu Published: 2024-02-05Area: Multimodal SafetyCitations: 42 Tags: ai-safety, empirical, multimodal-safety | 2024-02-05 | Multimodal Safety | ai-safety, empirical, multimodal-safety | E5 / R3 (95%) | 42 |
| Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modifications Boyi Wei, Kaixuan Huang, Mengdi Wang, Mengzhou Xia Published: 2024-02-07Area: Adversarial RobustnessCitations: 188 Tags: adversarial-robustness, ai-safety, alignment-training, empirical | 2024-02-07 | Adversarial Robustness | adversarial-robustness, ai-safety, alignment-training, empirical | E5 / R3 (96%) | 188 |
| Direct Language Model Alignment from Online AI Feedback Alexandre Rame, Biao Zhang, Bilal Piot, Felipe Llinares Published: 2024-02-07Area: Alignment TrainingCitations: 226 Tags: ai-safety, alignment-training, empirical | 2024-02-07 | Alignment Training | ai-safety, alignment-training, empirical | E6 / R4 (97%) | 226 |
| Opening the AI black box: program synthesis via mechanistic interpretability Anish Mudide, Chloe Loughridge, Eric J. Michaud, Isaac Liao Published: 2024-02-07Area: Mechanistic Interp.Citations: 19 Tags: ai-safety, empirical, interpretability, mechanistic-interp | 2024-02-07 | Mechanistic Interp. | ai-safety, empirical, interpretability, mechanistic-interp | E5 / R4 (96%) | 19 |
| Rapid Optimization for Jailbreaking LLMs via Subconscious Exploitation and Echopraxia Guangyu Shen, Guanhong Tao, Kaiyuan Zhang, Lu Yan Published: 2024-02-08Area: Adversarial RobustnessCitations: 18 Tags: adversarial-robustness, ai-safety, empirical | 2024-02-08 | Adversarial Robustness | adversarial-robustness, ai-safety, empirical | E5 / R3 (94%) | 18 |
| Self-Alignment of Large Language Models via Monopolylogue-based Social Scene Simulation Bolun Zhang, Rui Ye, Shuo Tang, Siheng Chen Published: 2024-02-08Area: Alignment TrainingCitations: 49 Tags: ai-safety, alignment-training, empirical | 2024-02-08 | Alignment Training | ai-safety, alignment-training, empirical | E5 / R4 (94%) | 49 |
| Debating with More Persuasive LLMs Leads to More Truthful Answers Akbir Khan, Ansh Radhakrishnan, Dan Valentine, Edward Grefenstette Published: 2024-02-09Area: Scalable OversightCitations: 215 Tags: ai-safety, empirical, scalable-oversight | 2024-02-09 | Scalable Oversight | ai-safety, empirical, scalable-oversight | E5 / R3 (96%) | 215 |
| Feedback Loops With Language Models Drive In-Context Reward Hacking Alexander Pan, Erik Jones, Jacob Steinhardt, Meena Jagadeesan Published: 2024-02-09Area: Deception & FailureCitations: 61 Tags: ai-safety, deception-failure, empirical | 2024-02-09 | Deception & Failure | ai-safety, deception-failure, empirical | E5 / R3 (95%) | 61 |
| StruQ: Defending Against Prompt Injection with Structured Queries Chawin Sitawarin, David Wagner, Julien Piet, Sizhe Chen Published: 2024-02-09Area: Adversarial RobustnessCitations: 185 Tags: adversarial-robustness, ai-safety, empirical | 2024-02-09 | Adversarial Robustness | adversarial-robustness, ai-safety, empirical | E5 / R3 (96%) | 185 |