Orion Weller

I’m a final-year PhD student at the Center for Language and Speech Processing at Johns Hopkins University, advised by Benjamin Van Durme and Dawn Lawrie. My research is graciously supported by a NSF Graduate Research Fellowship.

My current research interests are situated between the LM and IR fields, where I work to improve how models find, understand, and generate information. These days my research interests fall in three main categories, although I can get distracted by other LM-based topics.

• Retrieval models: figuring out how to evaluate them comprehensively and giving them new capabilites such as being instructable/prompted retrievers or using test-time compute.
• Retrieval-Augmented Generation (RAG): working towards better RAG evaluations and improving RAG performance (often through better retrieval).
• Language model pre-training (analysis and modeling). On the analysis side, studying why LMs act they way they do and using pre-training information to help LM generation. On the modeling side improving encoder pre-training with ModernBERT, comparing architecture differences with Ettin, and improving multilingual models with mmBERT.

Previously I graduated with my Bachelor’s degree from Brigham Young University in computer science and statistics, where I was advised by Kevin Seppi and Quinn Snell.

For Fall 2025 I am currently interning at the FAIR language team at Meta’s Superintelligence Lab with Xilun Chen and Scott Yih. In the past, I’ve been lucky to intern from amazing mentors at Google Deepmind, Samaya AI, AI2, and Apple.

If you’re interested in getting in contact with me, please email me at {last_name}{first_name}@gmail.com.