publications

Publications by categories in reversed chronological order. Generated by jekyll-scholar.

2026

  1. DenseOn with the LateOn: Fully Open Dense and Late-Interaction Models for Multilingual, Long-Context, and Code Search
    Raphaël Sourty, Antoine Chaffin,  Orion Weller, Paulo Roberto Moura Junior, and Amélie Chatelain
    Preprint 2026
    PDF Code
  2. On the Theoretical Limitations of Embedding-Based Retrieval
    Orion Weller, Michael Boratko, Iftekhar Naim, and Jinhyuk Lee
    In International Conference on Learning Representations (ICLR) 2026
    PDF Code
  3. Seq vs Seq: An Open Suite of Paired Encoders and Decoders
    Orion Weller, Kathryn Ricci, Marc Marone, Antoine Chaffin, Dawn Lawrie, and Benjamin Van Durme
    In International Conference on Learning Representations (ICLR) 2026
    PDF Code
  4. CacheNotes: Task-aware key-value cache compression for reasoning-intensive knowledge tasks
    Giulio Corallo,  Orion Weller, Fabio Petroni, and Paolo Papotti
    In European Chapter of the Association for Computational Linguistics (EACL) 2026
    PDF
  5. Humanity’s Last Exam
    Long Phan, Alice Gatti, Ziwen Han, Nathaniel Li, Josephina Hu, Hugh Zhang, Chen Bo Calvin Zhang, Mohamed Shaaban, John Ling,  ...,  Orion Weller, and others
    Nature 2026
    PDF

2025

  1. mmBERT: A Modern Multilingual Encoder with Annealed Language Learning
    Marc Marone,  Orion Weller, William Fleshman, Eugene Yang, Dawn Lawrie, and Benjamin Van Durme
    Preprint 2025
    PDF Code
  2. Auto-ARGUE: LLM-Based Report Generation Evaluation
    William Walden, Marc Mason,  Orion Weller, Laura Dietz, John Conroy, Neil Molino, Hannah Recknor, Bryan Li, Gabrielle Kaili-May Liu, Yu Hou, Dawn Lawrie, James Mayfield, and Eugene Yang
    Preprint 2025
    PDF
  3. Rank1: Test-Time Compute for Reranking in Information Retrieval
    Orion Weller, Kathryn Ricci, Eugene Yang, Andrew Yates, Dawn Lawrie, and Benjamin Van Durme
    In Conference on Language Models (CoLM) 2025
    PDF Code
  4. Rank-K: Test-Time Reasoning for Listwise Reranking
    Eugene Yang, Andrew Yates, Kathryn Ricci,  Orion Weller, Vivek Chari, Benjamin Van Durme, and Dawn Lawrie
    Preprint 2025
    PDF Code
  5. mFollowIR: a Multilingual Benchmark for Instruction Following in IR
    Orion Weller, Benjamin Chang, Mahsa Yarmohammadi, Sam Barham, Sean MacAvaney, Arman Cohan, Luca Soldaini, Benjamin Van Durme, and Dawn Lawrie
    In European Conference on Information Retrieval (ECIR) 2025
    PDF Code
  6. Harder, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference
    Benjamin Warner, Antoine Chaffin, Benjamin Clavié,  Orion Weller, Oskar Hallström, Said Taghadouini, Alexis Gallagher, Raja Biswas, Faisal Ladhak, Tom Aarsen, Nathan Cooper, Griffin Adams, Jeremy Howard, and Iacopo Poli
    In Association for Computational Linguistics (ACL) 2025
    PDF Code
  7. MMTEB: Massive Multilingual Text Embedding Benchmark
    Kenneth Enevoldsen, Isaac Chung, Imene Kerboua, Márton Kardos,  ..., Sara Hooker, Chenghao Xiao, Vaibhav Adlakha,  Orion Weller, Siva Reddy, and Niklas Muennighoff
    In International Conference on Learning Representations (ICLR) 2025
    PDF Code
  8. From Models to Microtheories: Distilling a Model’s Topical Knowledge for Grounded Question Answering
    Nathaniel Weir, Bhavana Dalvi,  Orion Weller, Oyvind Tafjord, Sam Hornstein, Alexander Sabol, Peter Jansen, Benjamin Van Durme, and Peter Clark
    In International Conference on Learning Representations (ICLR) 2025
    PDF Code
  9. CLERC: A Dataset for Legal Case Retrieval and Retrieval-Augmented Analysis Generation
    Abe Bohan Hou,  Orion Weller, Guanghui Qin, Eugene Yang, Dawn Lawrie, Nils Holzenberger, Andrew Blair-Stanek, and Benjamin Van Durme
    In North American Chapter of the Association for Computational Linguistics (NAACL) 2025
    PDF Code
  10. Self-[In]correct: LLMs Struggle with Refining Self-Generated Responses
    Dongwei Jiang, Jingyu Zhang,  Orion Weller, Nathaniel Weir, Benjamin Van Durme, and Daniel Khashabi
    In Association for the Advancement of Artificial Intelligence (AAAI) 2025
    PDF
  11. Promptriever: Instruction-Trained Retrievers Can Be Prompted Like Language Models
    Orion Weller, Benjamin Van Durme, Dawn Lawrie, Ashwin Paranjape, Yuhao Zhang, and Jack Hessel
    In International Conference on Learning Representations (ICLR) 2025
    PDF Code
  12. FollowIR: Evaluating and Teaching Information Retrieval Models to Follow Instructions
    Orion Weller, Benjamin Chang, Sean MacAvaney, Kyle Lo, Arman Cohan, Benjamin Van Durme, Dawn Lawrie, and Luca Soldaini
    In North American Chapter of the Association for Computational Linguistics (NAACL) 2025
    PDF Code

2024

  1. Dated Data: Tracing Knowledge Cutoffs in Large Language Models
    Jeffrey Cheng, Marc Marone,  Orion Weller, Dawn Lawrie, Daniel Khashabi, and Benjamin Van Durme
    In Conference on Language Models (CoLM) 2024
    PDF Code
  2. Learning to Reason via Program Generation, Emulation, and Search
    Nathaniel Weir, Muhammad Khalifa, Linlu Qiu,  Orion Weller, and Peter Clark
    In Neural Information Processing Systems (NeurIPS) 2024
    PDF Code
  3. A Waterlog for Detecting and Tracing Synthetic Text from Large Language Models
    Brennon Brimhall,  Orion Weller, Matthew Green, and Ian Miers
    In Cryptology ePrint Archive 2024
    PDF
  4. L-FRESCO: Factual Recall Evaluation Score for Legal Analysis Generation
    Abe Bohan Hou, Zhengping Jiang, Guanghui Qin,  Orion Weller, Andrew Blair-Stanek, and Benjamin Van Durme
    In Generative AI + Law Workshop at ICML 2024
  5. On the Evaluation of Machine-Generated Reports
    James Mayfield, Eugene Yang, Dawn Lawrie, Sean MacAvaney, Paul McNamee, Douglas W Oard, Luca Soldaini, Ian Soboroff,  Orion Weller, Efsun Kayi, and others
    In ACM Special Interest Group on Information Retrieval (SIGIR) 2024
    PDF
  6. Enhancing Systematic Decompositional Natural Language Inference Using Informal Logic
    Nathaniel Weir, Kate Sanders,  Orion Weller, Shreya Sharma, Dongwei Jiang, Zhengping Zhang, Bhavana Dalvi Mishra, Oyvind Tafjord, Peter Jansen, Peter Clark, and others
    In Empirical Methods in Natural Language Processing (EMNLP) 2024
    PDF
  7. NevIR: Negation in Neural Information Retrieval
    Orion Weller, Dawn Lawrie, and Benjamin Van Durme
    In European Chapter of the Association for Computational Linguistics (EACL) 2024
    PDF Code
  8. When do Generative Query and Document Expansions Fail? A Comprehensive Study Across Methods, Retrievers, and Datasets
    Orion Weller, Kyle Lo, David Wadden, Dawn Lawrie, Benjamin Van Durme, Arman Cohan, and Luca Soldaini
    In Findings of the European Chapter of the Association for Computational Linguistics (EACL) 2024
    PDF Code
  9. “According to..." Prompting Language Models Improves Quoting from Pre-Training Data
    ∗Orion Weller, ∗Marc Marone, Nathaniel Weir, Dawn Lawrie, Daniel Khashabi, and Benjamin Van Durme
    In European Chapter of the Association for Computational Linguistics (EACL) 2024
    PDF Code
  10. Defending Against Misinformation Attacks in Open-Domain Question Answering
    ∗Orion Weller, ∗Aleem Khan, Nathaniel Weir, Dawn J Lawrie, and Benjamin Van Durme
    In European Chapter of the Association for Computational Linguistics (EACL) 2024
    PDF Code

2023

  1. When Do Decompositions Help for Machine Reading?
    Kangda Wei, Dawn J Lawrie, Benjamin Van Durme, Yunmo Chen, and Orion Weller
    In Empirical Methods in Natural Language Processing (EMNLP) 2023
    PDF Code
  2. Synthetic Cross-language Information Retrieval Training Data
    James Mayfield, Eugene Yang, Dawn Lawrie, Samuel Barham,  Orion Weller, Marc Mason, Suraj Nair, and Scott Miller
    Preprint 2023
    PDF
  3. MegaWika: Millions of reports and their sources across 50 diverse languages
    Samuel Barham,  Orion Weller, Michelle Yuan, Kenton Murray, Mahsa Yarmohammadi, Zhengping Jiang, Siddharth Vashishtha, Alexander Martin, Anqi Liu, Aaron Steven White, and others
    Preprint 2023
    PDF Code

2022

  1. Pretrained Models for Multilingual Federated Learning
    ∗Orion Weller, ∗Marc Marone, Vladimir Braverman, Dawn Lawrie, and Benjamin Van Durme
    In North American Chapter of the Association for Computational Linguistics (NAACL) 2022
    PDF Code
  2. When to Use Multi-Task Learning vs Intermediate Fine-Tuning for Pre-Trained Encoder Transfer Learning
    Orion Weller, Kevin Seppi, and Matt Gardner
    In Association for Computational Linguistics (ACL) 2022
    PDF Code
  3. End-to-End Speech Translation for Code Switched Speech
    Orion Weller, Matthias Sperber, Telmo Pires, Hendra Setiawan, Christian Gollan, Dominic Telaar, and Matthias Paulik
    In Findings of the Association for Computational Linguistics (ACL) 2022
    PDF Code

2021

  1. Exploring the Relationship Between Algorithm Performance, Vocabulary, and Run-Time in Text Classification
    Wilson Fearn,  Orion Weller, and Kevin Seppi
    In North American Chapter of the Association for Computational Linguistics (NAACL) 2021
    PDF Code
  2. Streaming Joint Speech Translation and Transcription
    Orion Weller, Matthias Sperber, Christian Gollan, and Joris Kluivers
    In European Chapter of the Association for Computational Linguistics (EACL) 2021
    PDF
  3. Predicting Mental Health and Suicidal Ideation Among Adolescents Using the Risk and Protective Factor Framework: A Large Scale Machine Learning Approach
    Orion Weller, Luke Sagers, Carl Hanson, Quinn Snell, Michael Barnes, and Shannon Tass
    PLoS One 2021

2020

  1. Learning from Task Descriptions
    Orion Weller, Nicholas Lourie, Matt Gardner, and Matthew Peters
    In Empirical Methods in Natural Language Processing (EMNLP) 2020
    PDF Code
  2. You Don’t Have Time to Read This: an Exploration of Document Level Reading Time Prediction
    Orion Weller, Jordan Hildebrandt, Ilya Reznik, Christopher Challis, E. Shannon Tass, Quinn Snell, and Kevin Seppi
    In Association for Computational Linguistics (ACL) 2020
    PDF Code
  3. The rJokes Dataset: a Large Scale Humor Collection
    Orion Weller, and Kevin Seppi
    In International Conference on Language Resources and Evaluation (LREC) 2020
    PDF Code
  4. Can Humor Prediction Datasets be used for Humor Generation? Humorous Headline Generation via Style Transfer
    Orion Weller, Nancy Fulda, and Kevin Seppi
    In Second Workshop on Figurative Language Processing @ ACL 2020 2020
    PDF Code

2019

  1. Humor Detection: A Transformer gets the Last Laugh
    Orion Weller, and Kevin Seppi
    In Empirical Methods in Natural Language Processing (EMNLP) 2019
    PDF Code