alt text 

Daniel Khashabi (دانیال خشابی)
Young Investigator
Allen Institute for Artificial Intelligence
2157 N Northlake Way Suite 110, Seattle, WA 98103 | danielk¯\_(ツ)_/¯

I am a post-doctoral researcher at Allen Institute for AI, Seattle. I am affiliated with the Mosaic team and I frequently collaborate with other teams like AllenNLP and Aristo. Prior to this, I was a doctoral student at Computer and Information Sciences at the University of Pennsylvania (2017-2019) and at the University of Illinois, Urbana-Champaign (2012-2016), under Prof. Dan Roth. I got my B.Sc. from Tehran Polytechnic (2008-2012) where I was greatly fortunate to work with Prof. Hamid Sheikhzadeh.

Research Themes

My research focuses on the computational foundations of intelligent behavior, through the lens of natural language. My goal is to explore the different ways in which we can develop and evaluate systems that understand and reason with (and about) natural language in different contexts. I mostly identify with natural language processing (ACL, NAACL, EMNLP) and artificial intelligence communities (AAAI, IJCAI) and, to some extent, with the machine learning community (NeurIPS, ICML).

Here are several themes I am interested in:

  • Generalism*: AlphaGo may be the world champion at Go, although it can't solve any other problem! How can we incentivize building models that can address a broader scope of tasks and abilities? (papers: ACL’22, EMNLP-Findings’20)

  • Self-supervised learning: How can we build task-independent representations that utilize cheap signals available in-the-wild (web data or physical environment) and support better model generalization? (papers: ACL’20, EMNLP’18)

  • Reasoning: I view “reasoning” as the process of explaining or justifying decisions via “reasons”. How can we enable machines to communicate via reasons, for a broad-ranging spectrum of tasks? (papers: TACL’21, NAACL’21)

  • AI + Society: What are the algorithmic biases in our models and how can we alleviate them? How do the models contribute to echo chambers (and our growing socio-political divisions) and how can address these issues? (papers: EMNLP’20, NAACL’19)


Disclaimer: This material is presented to ensure the timely dissemination of scholarly works. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms invoked by each author's copyright.

  • Benchmarking Generalization via In-Context Instructions on 1,600+ Language Tasks
    Yizhong Wang et al.
    arXiv, 2022
    Paper, Website, Data

  • NeuroLogic A*esque Decoding: Constrained Text Generation with Lookahead Heuristics
    Ximing Lu, Sean Welleck, Peter West, Liwei Jiang, Jungo Kasai, Daniel Khashabi, Ronan Le Bras, Lianhui Qin, Youngjae Yu, Rowan Zellers, Noah A. Smith and Yejin Choi
    NAACL, 2022

  • Promp Waywardness: The Curious Case of Discretized Interpretation of Continuous Prompts
    Daniel Khashabi, Shane Lyu, Sewon Min, Lianhui Qin, Kyle Richardson, Sameer Singh, Sean Welleck, Hannaneh Hajishirzi, Tushar Khot, Ashish Sabharwal and Yejin Choi
    NAACL, 2022
    Paper, Slides

  • Time Waits for No One! Analysis and Challenges of Temporal Misalignment
    Kelvin Luu, Daniel Khashabi, Suchin Gururangan, Karishma Mandyam and Noah A. Smith
    NAACL, 2022
    Paper, Data

  • Toward Automatic Discovery of Diverse Perspectives
    Sihao Chen, Daniel Khashabi and Dan Roth
    Book chapter in “Creating a More Transparent Internet: The Perspective Web”
    Cambridge University Press, 2022

  • COLD Decoding: Energy-based Constrained Text Generation with Langevin Dynamics
    Lianhui Qin, Sean Welleck, Daniel Khashabi and Yejin Choi
    aXiv, 2022
    Paper, Code, Slides

  • UnifiedQA-v2: Stronger Generalization via Broader Cross-Format Training
    Daniel Khashabi, Yeganeh Kordi and Hannaneh Hajishirzi
    aXiv, 2022

  • Hey AI, Can You Solve Complex Tasks by Talking to Agents?
    Tushar Khot, Kyle Richardson, Daniel Khashabi and Ashish Sabharwal
    ACL - Findings, 2022
    Paper, Slides, Poster

  • Reframing Instructional Prompts to GPTk's Language
    Swaroop Mishra, Daniel Khashabi, Chitta Baral, Yejin Choi and Hannaneh Hajishirzi
    ACL - Findings, 2022
    Paper, Code

  • Cross-Task Generalization via Natural Language Crowdsourcing Instructions
    Swaroop Mishra, Daniel Khashabi, Chitta Baral and Hannaneh Hajishirzi
    ACL, 2022
    Paper, Data, Data2, Slides, Blog

  • Findings of the 2021 Conference on Machine Translation (WMT21)
    Farhad Akhbardeh et al.
    WMT, 2021

  • GooAQ: Open Question Answering with Diverse Answer Types
    Daniel Khashabi, Amos Ng, Tushar Khot, Ashish Sabharwal, Hannaneh Hajishirzi and Chris Callison-Burch
    EMNLP - Findings, 2021
    Paper, Data, Slides

  • Ethical-Advice Taker: Do Language Models Understand Natural Language Interventions?
    Jieyu Zhao, Daniel Khashabi, Tushar Khot, Ashish Sabharwal and Kai-Wei Chang
    ACL-IJCNLP - Findings, 2021
    Paper, Data & Code

  • Text Modular Networks: Learning to Decompose Tasks in the Language of Existing Models
    Tushar Khot, Daniel Khashabi, Kyle Richardson, Ashish Sabharwal and Peter Clark
    NAACL-HLT, 2021
    Paper, Resources, Demo, Slides

  • Think you have Solved Direct-Answer Question Answering? Try ARC-DA, the Direct-Answer AI2 Reasoning Challenge
    Sumithra Bhakthavatsalam, Daniel Khashabi, Tushar Khot, Bhavana Dalvi Mishra, Kyle Richardson, Ashish Sabharwal, Carissa Schoenick, Oyvind Tafjord and Peter Clark
    arXiv, 2021
    Paper, Data

  • GENIE: Leaderboard for Human-in-the-Loop Evaluation of Text Generation
    Daniel Khashabi, Gabriel Stanovsky, Jonathan Bragg, Nicholas Lourie, Jungo Kasai, Yejin Choi, Noah A. Smith and Daniel S. Weld
    arXiv, 2021
    Paper, Website, Media coverage: VentureBeat

  • ParsiNLU: A Suite of Language Understanding Challenges for Persian
    Daniel Khashabi, Arman Cohan, Siamak Shakeri et al.
    TACL, 2021
    Paper, Code & Data, Slides

  • Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies
    Mor Geva, Daniel Khashabi, Elad Segal, Tushar Khot, Dan Roth and Jonathan Berant
    TACL, 2021
    Paper, Dataset, Leaderboard, Slides

  • UnQovering Stereotypical Biases via Underspecified Questions
    Tao Li, Tushar Khot, Daniel Khashabi and Ashish Sabharwal
    EMNLP - Findings, 2020
    Paper, Code, Demo

  • UnifiedQA: Crossing Format Boundaries With a Single QA System
    Daniel Khashabi, Sewon Min, Tushar Khot, Ashish Sabharwal, Oyvind Tafjord, Peter Clark and Hannaneh Hajishirzi
    EMNLP - Findings, 2020
    Paper, Resources, Demo, Slides

  • Evaluating Models’ Local Decision Boundaries via Contrast Sets
    Matt Gardner et al.
    EMNLP - Findings, 2020
    Paper, Data

  • More Bang for Your Buck: Natural Perturbation for Robust Question Answering
    Daniel Khashabi, Tushar Khot and Ashish Sabharwal
    EMNLP, 2020
    Paper, Data, Slides, Talk

  • Commonsense Knowledge Discovery from Linguistic Graphs
    Hongming Zhang, Daniel Khashabi, Yangqiu Song and Dan Roth
    IJCAI, 2020
    Paper, Data

  • From ‘F’ to ‘A’ on the N.Y. Regents Science Exams: An Overview of the Aristo Project
    Peter Clark et al.
    AI Magazine, 2020
    Paper, Talk, Media coverage: NYTimes, GeekWire, Vox, Forbes

  • Temporal Common Sense Acquisition with Minimal Supervision
    Ben Zhou, Qiang Ning Daniel Khashabi and Dan Roth
    ACL, 2020
    Paper, Code, ACL slides

  • Not All Claims are Created Equal: Choosing the Right Approach to Assess Your Hypotheses
    Erfan Sadeqi Azer, Daniel Khashabi, Ashish Sabharwal and Dan Roth
    ACL, 2020
    Paper, Software, Slides, Slides2, ACL slides

  • “Going on a vacation” takes longer than “Going for a walk”: A Study of Temporal Commonsense Understanding
    Ben Zhou, Daniel Khashabi, Qiang Ning and Dan Roth
    EMNLP, 2019
    Paper, Dataset, Leaderboard, Slides, Talk

  • PerspectroScope: A Window to the World of Diverse Perspectives
    Sihao Chen, Daniel Khashabi, Chris Callison-Burch and Dan Roth
    ACL - Demos, 2019
    Paper, Video, Code, Poster

  • On the Possibilities and Limitations of Multi-hop Reasoning Under Linguistic Imperfections
    Daniel Khashabi, Erfan Sadeqi Azer, Tushar Khot, Ashish Sabharwal and Dan Roth
    arXiv, 2019

  • Zero-Shot Open Entity Typing as Type-Compatible Grounding
    Ben Zhou, Daniel Khashabi, Chen-Tse Tsai and Dan Roth
    EMNLP, 2018
    Paper, Code, Poster

  • Looking Beyond the Surface: A Challenge Set for Reading Comprehension over Multiple Sentences
    Daniel Khashabi, Snigdha Chaturvedi, Michael Roth, Shyam Upadhyay and Dan Roth
    NAACL, 2018
    Paper, Dataset, Poster

  • CogCompNLP: Your Swiss Army Knife for NLP
    Daniel Khashabi, Mark Sammons, Ben Zhou, Tom Redman, Christos Christodoulopoulos et al.
    LREC, 2018
    Paper, Code, Poster

  • Question Answering as Global Reasoning over Semantic Abstractions
    Daniel Khashabi, Tushar Khot, Ashish Sabharwal and Dan Roth
    AAAI, 2018
    Paper, Code, Slides-1, Slides-2

  • Relational Learning and Feature Extraction by Querying over Heterogeneous Information Networks
    Parisa Kordjamshidi, Sameer Singh, Daniel Khashabi, Christos Christodoulopoulos, Mark Summons, Saurabh Sinha and Dan Roth
    Seventh International Workshop on Statistical Relational AI (StarAI), 2017
    Paper, Code, Poster

  • Learning What is Essential in Questions
    Daniel Khashabi, Tushar Khot, Ashish Sabharwal and Dan Roth
    CoNLL, 2017
    Paper, Code, Poster, Spotlight

  • Better call Saul: Flexible Programming for Learning and Inference in NLP
    Parisa Kordjamshidi, Daniel Khashabi, Christos Christodoulopoulos, Bhargav Mangipudi, Sameer Singh and Dan Roth
    COLING, 2016
    Paper, Slides, Code

  • Question Answering via Integer Programming over Semi-Structured Knowledge
    Daniel Khashabi, Tushar Khot, Ashish Sabharwal, Peter Clark, Oren Etzioni and Dan Roth
    IJCAI, 2016
    Paper, Code, Demo, UW talk, IJCAI talk, Poster

  • EDISON: Feature Extraction for NLP, Simplified
    Mark Sammons, Christos Christodoulopoulos, Parisa Kordjamshidi, Daniel Khashabi, Vivek Srikumar and Dan Roth
    LREC, 2016
    Paper, Poster

  • Combining Retrieval, Statistics, and Inference to Answer Elementary Science Questions
    Peter Clark et al.
    AAAI, 2016

  • Online Learning with Adversarial Delays
    Kent Quanrud and Daniel Khashabi
    NeurIPS, 2015
    Paper, Poster

  • Illinois-Profiler: Knowledge Schemas at Scale
    Zhiye Fei, Daniel Khashabi, Haoruo Peng, Hao Wu and Dan Roth
    IJCAI Workshop on Cognitive Knowledge Acquisition and Applications (Cognitum 2015)
    Paper, Slides, Poster

  • Solving Hard Co-reference Problems
    Haoruo Peng, Daniel Khashabi and Dan Roth
    NAACL, 2015
    Paper, Slides, Poster

  • Adaptive Tiled Neural Networks
    Mohammad Nokhbeh-Zaeem et al.
    in IEEE International Conference on Systems, Man, and Cybernetics (SMC), 2011
    Paper, Code


  • Image demosaicing
    Sebastian Nowozin, Daniel Khashabi, Jeremy Jancsary, Bruce Lindbloom, Andrew Fitzgibbon, Microsoft Technology Licensing LLC
    United States patent US 9,344,690. 2016 May 17.


  • Reasoning-Driven Question-Answering for Natural Language Understanding
    PhD Thesis, University of Pennsylvania, 2019
    PDF, Slides

  • Analysis and Implementation of Bayesian Methods to Model Correlated Information (Multitask Learning)
    BSc Thesis, Tehran Polytechnic, 2012 (in Persian.)
    PDF, Slides

Select Talks


  • Guest Lecturer: CS446 (Machine Learning) @ UPenn, Spring 2018.

  • Guest Lecturer: CS446 (Machine Learning) @ UPenn, Fall 2018: Lecture 1 Lecture 2

  • Guest Lecturer: CS446 (Machine Learning) @ UIUC, Spring 2016: Lecture 1

  • Guest Lecturer: CS446 (Machine Learning) @ UIUC, Fall 2015: files Lecture 1 Lecture 2

  • Teaching Assistant: CS446: Machine Learning, Dan Roth, UIUC (Fall, 2015).

  • Teaching Assistant: CS473: Fundamental Algorithms, Jeff Erikson, UIUC (Fall, 2013).

  • Teaching Assistant: CS473: Fundamental Algorithms, Sariel Har-Peled and Alexandra Kolla, UIUC (Spring, 2013).

  • Teaching Assistant: “Probability and Statistics”, Instructor: Dr. Gholamreza Moradi, AUT (Spring, 2012).

  • Teaching Assistant: “Digital Signal Processing”, Instructor: Dr. Hamid Sheikhzadeh, AUT (Spring 2012).

  • Teaching Assistant: “C++ II : Numerical C++ Programming”, Instructor: Dr.Bahram Taheri, Joint program with University of Birmingham and AUT (Fall, 2011).

  • Teaching Assistant: “Introduction to Computers and Programming(C++)”, Instructor: Dr.Hassan Taheri (Spring, 2011).