Zachary Lipton, PI
Zachary Lipton is the Raj Reddy Associate Professor of Machine Learning at Carnegie Mellon University, where he directs the Approximately Correct Machine Intelligence (ACMI) lab. He is also the Chief Technology Officer and Chief Scientist of Abridge, the leading platform for AI-based ambient listening technology in healthcare. Abridge’s technology performs clerical tasks, liberating doctors to focus on patient care. Research focuses include the theoretical and engineering foundations of robust and adaptive machine learning algorithms, applications to both prediction and decision-making problems in clinical medicine, natural language processing, and the impact of machine learning systems on society. A key theme in his current work is to take advantage of causal structure underlying the observed data while producing algorithms that are compatible with the modern deep learning power tools that dominate practical applications. He is the founder of the Approximately Correct blog (approximatelycorrect.com) and a co-author of Dive Into Deep Learning, an interactive open-source book drafted entirely through Jupyter notebooks that has reached millions of readers. He can be found on X (@zacharylipton), GitHub (@zackchase), or his lab's website (acmilab.org).
Research Interests: Pickled Herring, Philosophy of Artificial Intelligence
Publications
Red-Teaming for Generative AI: Silver Bullet or Security Theater?
Michael Feffer, Anusha Sinha, Wesley H. Deng, Zachary Lipton, Hoda Heidari
AAAI/ACM Conference on Artificial Intelligence, Ethics and Society (AIES), 2024GenAudit: Fixing Factual Errors in Language Model Outputs with Evidence
Kundan Krishna, Sanjana Ramprasad, Prakhar Gupta, Byron Wallace, Zachary Lipton, Jeffrey Bigham
arXiv (preprint), 2024Auditing Fairness under Unobserved Confounding
Emily Byun, Dylan Sam, Michael Oberst, Zachary Lipton, Bryan Wilder
AISTATS, 2024DeepDrake ft. BTS-GAN and TayloRVC: A Survey of Musical Deepfake Models
Michael Feffer, Zachary Lipton, Chris Donahue
Workshop on Human-Centric Music Information Research (HCMIR@ISMIR), 2023From Preference Elicitation to Participatory ML: A Critical Survey & Guidelines for Future Research
Michael Feffer, Michael Skirpan, Zachary Lipton*, Hoda Heidari*
AAAI/ACM Conference on Artificial Intelligence, Ethics and Society (AIES), 2023USB: A Unified Summarization Benchmark Across Tasks and Domains
Kundan Krishna, Sanjana Ramprasad, Prakhar Gupta, Byron Wallace, Jeffrey Bigham, Zachary Lipton
EMNLP (Findings), 2023Downstream Datasets Make Surprisingly Good Pretraining Corpora
Kundan Krishna, Saurabh Garg, Jeffrey Bigham, Zachary Lipton
ACL 2023, 2023Moral Machine or Tyranny of the Majority?
Michael Feffer, Hoda Heidari*, Zachary Lipton*
Association for the Advancement of Artificial Intelligence (AAAI), 2023Discovering Optimal Scoring Mechanisms in Causal Strategic Prediction
Tom Yan, Shantanu Gupta, Zachary Lipton
arXiv (preprint), 2023Model Evaluation in Medical Datasets Over Time
Helen Zhou*, Yuwen Chen*, Zachary Lipton
Conference on Health, Inference, and Learning (CHIL), 2023Domain Adaptation under Missingness Shift
Helen Zhou, Sivaraman Balakrishnan, Zachary Lipton
Artificial Intelligence and Statistics (AISTATS), 2023Disentangling the Mechanisms Behind Implicit Regularization in SGD
Zachary Novack, Simran Kaur, Tanya Marwah, Saurabh Garg, Zachary Lipton
International Conference on Learning Representations (ICLR), 2023Chils: Zero-shot image classification with hierarchical label sets
Zachary Novack, Julian McAuley, Zachary Lipton, Saurabh Garg
arXiv preprint arXiv:2302.02551, 2023RLSbench: Domain Adaptation Under Relaxed Label Shift
Saurabh Garg, Nick Erickson, James Sharpnack, Alex Smola, Sivaraman Balakrishnan, Zachary Lipton
arXiv preprint arXiv:2302.03020, 2023Local Causal Discovery for Estimating Causal Effects
Shantanu Gupta, David Childers, Zachary Lipton
Causal Learning and Reasoning (CLeaR), 2023Assistive alignment of in-the-wild sheet music and performances
Michael Feffer, Zachary Lipton, Chris Donahue
23rd Int. Society for Music Information Retrieval Conf. (ISMIR LBD), 2022Learning Clinical Concepts for Predicting Risk of Progression to Severe COVID-19
Helen Zhou, Cheng Cheng, Kelly Shields, Gursimran Kocchar, Tariq Cheema, Zachary Lipton, Jeremy Weiss
American Medical Informatics Association (AMIA) Annual Symposium, 2022Unsupervised Learning under Latent Label Shift
Manley Roberts*, Pranav Mani*, Saurabh Garg, Zachary Lipton
Advances in Neural Information Processing Systems (NeurIPS), 2022Domain Adaptation under Open Set Label Shift
Saurabh Garg, Sivaraman Balakrishnan, Zachary Lipton
Advances in Neural Information Processing Systems (NeurIPS), 2022Counterfactual Metrics for Auditing Black-Box Recommender Systems for Ethical Concerns
Nil-Jana Akpinar*, Liu Leqi*, Dylan Hadfield-Menell, Zachary Lipton
Workshop on Responsible Decision Making in Dynamic Environments. International Conference on Machine Learning (ICML 2022), 2022Resolving the Human Subjects Status of Machine Learning's Crowdworkers
Divyansh Kaushik, Zachary Lipton, Alex John London
arXiv Preprint, 2022Practical Benefits of Feature Feedback Under Distribution Shift
Anurag Katakkar*, Clay Yoo*, Weiqin Wang*, Zachary Lipton, Divyansh Kaushik
BlackBox NLP workshop, 2022Modeling Attrition in Recommender Systems with Departing Bandits
Omer Ben-Porat*, Lee Cohen*, Liu Leqi*, Zachary Lipton, Yishay Mansour
Association for the Advancement of Artificial Intelligence (AAAI), 2022Off-Policy Risk Assessment for Markov Decision Processes
Audrey Huang, Liu Leqi, Kamyar Azizzadenesheli, Zachary Lipton
International Conference on Artificial Intelligence and Statistics (AISTATS), 2022Explain, Edit, and Understand: Rethinking User Study Design for Evaluating Model Explanations
Siddhant Arora*, Danish Pruthi*, Norman Sadeh, William Cohen, Zachary Lipton, Graham Neubig
Association for the Advancement of Artificial Intelligence (AAAI), 2022Leveraging Unlabeled Data to Predict Out-of-Distribution Performance
Saurabh Garg, Sivaraman Balakrishnan, Zachary Lipton, Hanie Sedghi, Behnam Neyshabur
International Conference on Learning Representations (ICLR), 2022Mixture Proportion Estimation and PU Learning: A Modern Approach
Saurabh Garg, Yifan Wu, Alex Smola, Sivaraman Balakrishnan, Zachary Lipton
Advances in Neural Information Processing Systems (NeurIPS), 2021Rebounding Bandits for Modeling Satiation Effects
Liu Leqi, Fatma Kilinc-Karzan, Zachary Lipton, Alan Montgomery
Advances in Neural Information Processing Systems (NeurIPS), 2021Evaluating Explanations: How much do explanations from the teacher aid students?
Danish Pruthi, Rachit Bansal, Bhuwan Dhingra, Livio Baldini Soares, Michael Collins, Zachary Lipton, Graham Neubig, William Cohen
Transactions of the Association for Computational Linguistics (TACL), 2021Parametric Complexity Bounds for Approximating PDEs with Neural Networks
Tanya Marwah, Zachary Lipton, Andrej Risteski
Advances in Neural Information Processing Systems (NeurIPS), 2021Does Pretraining for Summarization Require Knowledge Transfer?
Kundan Krishna, Jeffrey Bigham, Zachary Lipton
Findings of EMNLP, 2021Efficient Online Estimation of Causal Effects by Deciding What to Observe
Shantanu Gupta, Zachary Lipton, David Childers
Advances in Neural Information Processing Systems (NeurIPS), 2021The Impact of Algorithmic Risk Assessments on Human Predictions and its Analysis via Crowdsourcing Studies
Riccardo Fogliato, Alexandra Chouldechova, Zachary Lipton
Computer-supported cooperative work (CSCW), 2021When Curation Becomes Creation: Algorithms, microcontent, and the vanishing distinction between platforms and creators
Liu Leqi, Zachary Lipton, Dylan Hadfield-Menell
Communications of the ACM (CACM), 2021Off-Policy Risk Assessment in Contextual Bandits
Audrey Huang, Liu Leqi, Zachary Lipton, Kamyar Azizzadenesheli
Advances in Neural Information Processing Systems (NeurIPS), 2021On the Efficacy of Adversarial Data Collection for Question Answering: Results from a Large-Scale Randomized Study
Divyansh Kaushik, Douwe Kiela, Zachary Lipton, Wen-tau Yih
Association for Computational Linguistics and International Joint Conference on Natural Language Processing (ACL-IJCNLP), 2021Unpacking the Drop in COVID-19 Case Fatality Rates: A Study of National and Florida Line-Level Data
Cheng Cheng*, Helen Zhou*, Jeremy Weiss, Zachary Lipton
American Medical Informatics Association (AMIA) Annual Symposium, 2021Correcting Exposure Bias for Link Recommendation
Shantanu Gupta, Hao Wang, Zachary Lipton, Yuyang Wang
International Conference on Machine Learning (ICML), 2021Estimating Treatment Effects with Observed Confounders and Mediators
Shantanu Gupta, Zachary Lipton, David Childers
Uncertainty in Artificial Intelligence (UAI), 2021On the Validity of Arrest as a Proxy for Offense: Race and the Likelihood of Arrest for Violent Crimes
Riccardo Fogliato, Alice Xiang, Zachary Lipton, Daniel Nagin, Alexandra Chouldechova
Conference on AI, Ethics, and Society (AIES), 2021RATT: Leveraging Unlabeled Data to Guarantee Generalization
Saurabh Garg, Sivaraman Balakrishnan, J. Zico Kolter, Zachary Lipton
International Conference on Machine Learning (ICML), 2021On Proximal Policy Optimization’s Heavy-Tailed Gradients
Saurabh Garg, Emilio Parisotto, Adarsh Prasad, J. Zico Kolter, Zachary Lipton, Sivaraman Balakrishnan, Ruslan Salakhutdinov, Pradeep Ravikumar
International Conference on Machine Learning (ICML), 2021Fair Machine Learning Under Partial Compliance
Jessica Dai, Sina Fazelpour, Zachary Lipton
Conference on Artificial Intelligence, Ethics, and Society (AIES), 2021Explaining The Efficacy of Counterfactually-Augmented Data
Divyansh Kaushik, Zachary Lipton
International Conference on Learning Representations (ICLR), 2021Decoding and Diversity in Machine Translation
Nick Roberts, Davis Liang, Graham Neubig, Zachary Lipton
NeurIPS Resistance AI Workshop, 2020Weakly and Semisupervised Evidence Extraction
Danish Pruthi, Bhuwan Dhingra, Graham Neubig, Zachary Lipton
Findings of EMNLP, 2020On Negative Interference in Multilingual Models:Findings and A Meta-Learning Treatment
Zirui Wang, Yulia Tsvetkov, Zachary Lipton
Empirical Methods in Natural Language Processing (EMNLP), 2020A Unified View of Label Shift Estimation
Saurabh Garg, Yifan Wu, Sivaraman Balakrishnan, Zachary Lipton
Advances in Neural Information Processing Systems (NeurIPS), 2020Algorithmic Fairness from a Non-Ideal Perspective
Sina Fazelpour, Zachary Lipton
AAAI/ACM Conference on Artificial Intelligence, Ethics and Society (AIES), 2020Predicting Mortality Risk in Viral and Unspecified Pneumonia to Assist Clinicians with COVID-19 ECMO Planning
Helen Zhou*, Cheng Cheng*, Zachary Lipton, George Chen, Jeremy Weiss
International Conference on Artificial Intelligence in Medicine (AIME), 2020Uncertainty-Aware Lookahead Factor Models for Quantitative Investing
Lakshay Chauhan, John Alberg, Zachary Lipton
International Conference on Machine Learning (ICML), 2020Learning to Deceive with Attention-Based Explanations
Danish Pruthi, Mansi Gupta, Bhuwan Dhingra, Graham Neubig, Zachary Lipton
Association for Computational Linguistics (ACL), 2020Efficient candidate screening under multiple tests and implications for fairness
Lee Cohen, Zachary Lipton, Yishay Mansour
Foundations of Responsible Computing Foundations of Responsible Computing (FORC), 2020Learning the Difference that Makes a Difference with Counterfactually-Augmented Data
Divyansh Kaushik, Zachary Lipton
International Conference on Learning Representations (ICLR), 2020Extracting Structured Data from Doctor-Patient Conversations by Predicting Noteworthy Utterances
Kundan Krishna, Amy Pavel, Benjamin Schloss, Jeffrey Bigham, Zachary Lipton
AAAI Workshop Health Intelligence, 2020Predicting Mortality in Liver Transplant Candidates
Jonathon Byrd, Sivaraman Balakrishnan, Xiaoqian Jiang, Zachary Lipton
AAAI Workshop Health Intelligence, 2020Causal Inference With Selectively-Deconfounded Data
Kyra Gan, Andrew Li, Zachary Lipton, Sridhar Tayur
NeurIPS Causality Workshop, 2019Are Perceptually-Aligned Gradients a General Property of Robust Classifiers?
Simran Kaur, Jeremy Cohen, Zachary Lipton
Neurips 2019 Science Meets Engineering of Deep Learning Workshop, 2019Game Design for Eliciting Distinguishable Behavior
Fan Yang, Liu Leqi, Yifan Wu, Zachary Lipton, Pradeep Ravikumar, William Cohen, Tom Mitchell
Advances in Neural Information Processing Systems (NeurIPS), 2019Learning Robust Global Representations by Penalizing Local Predictive Power
Haohan Wang, Songwei Ge, Eric Xing, Zachary Lipton
Advances in Neural Information Processing Systems (NeurIPS), 2019Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shift
Stefan Rabanser, Stephan Günnemann, Zachary Lipton
Advances in Neural Information Processing Systems (NeurIPS), 2019Entity Projection via Machine Translation for Cross-Lingual NER
Alankar Jain, Bhargavi Paranjape, Zachary Lipton
Empirical Methods in Natural Language Processing (EMNLP), 2019Practical Obstacles to Deploying Active Learning
David Lowell, Zachary Lipton, Byron Wallace
Empirical Methods in Natural Language Processing (EMNLP), 2019AmazonQA: A Review-Based Question Answering Task
Mansi Gupta, Nitish Kulkarni, Raghuveer Chanda, Anirudha Rayasam, Zachary Lipton
International Joint Conference on Machine Learning (IJCAI), 2019Embryo staging with weakly-supervised region selection and dynamically- decoded predictions
Tingfung Lau, Nathan Ng, Julian Gingold, Nina Desai, Julian McAuley, Zachary Lipton
Machine Learning for Healthcare (MLHC), 2019Domain Adaptation with Asymmetrically-Relaxed Distribution Alignment
Yifan Wu, Divyansh Kaushik, Ezra Winston, Zachary Lipton
International Conference on Machine Learning (ICML), 2019What is the Effect of Importance Weighting in Deep Learning?
Jonathon Byrd, Zachary Lipton
International Conference on Machine Learning (ICML), 2019Estimating brain age based on a uniform healthy population with deep learning and structural MRI
Xinyang Feng, Zachary Lipton, Jie Yang, Scott Small, Frank Provenzano
Neurobiology of Aging, 2019Combating Adversarial Misspellings with Robust Word Recognition
Danish Pruthi, Bhuwan Dhingra, Zachary Lipton
Association for Computational Linguistics (ACL), 2019Troubling Trends in Machine Learning Scholarship
Zachary Lipton*, Jacob Steinhardt*
Communications of the ACM (CACM), 2019Reinforcement Learning with Unknown Reward Functions
Benjamin Eysenbach*, Jacob Tyo*, Shane Gu, Ruslan Salakhutdinov, Zachary Lipton, Sergey Levine
Task-Agnostic Reinforcement Learning Workshop, ICLR, 2019Learning Robust Representations by Projecting Superficial Statistics Out
Haohan Wang, Zexue Hu, Zachary Lipton, Eric Xing
International Conference on Learning Representations (ICLR), 2019Active Learning with Partial Feedback
Peiyun Hu, Zachary Lipton, Anima Anandkumar, Deva Ramanan
International Conference on Learning Representations (ICLR), 2019Does Mitigating ML’s Impact Disparity Require Treatment Disparity?
Zachary Lipton, Alexandra Chouldechova, Julian McAuley
Advances in Neural Information Processing (NeurIPS), 2018How Much Reading Does Reading Comprehension Require? A Critical Investigation of Popular Benchmarks
Divyansh Kaushik, Zachary Lipton
Empirical Methods in Natural Language Processing (EMNLP), 2018Deep Bayesian Active Learning for Natural Language Processing: Results of a Large-Scale Empirical Study
Aditya Siddhant, Zachary Lipton
Empirical Methods in Natural Language Processing (EMNLP), 2018Learning Noise-Invariant Representations for Robust Speech Recognition
Davis Liang, Zhiheng Huang, Zachary Lipton
IEEE Speech and Language Technology (SLT), 2018Born Again Networks
Tommaso Furlanello, Zachary Lipton, Michael Tschannen, Laurent Itti, Anima Anandkumar
International Conference on Machine Learning (ICML), 2018Detecting and Correcting for Label Shift with Black Box Predictors
Zachary Lipton*, Yu-Xiang Wang*, Alex Smola
International Conference on Machine Learning (ICML), 2018Semantically Decomposing the Latent Spaces of Generative Adversarial Networks
Chris Donahue, Zachary Lipton, Akshay Balsubramani, Julian McAuley
International Conference on Learning Representations (ICLR), 2018Stochastic Activation Pruning for Robust Adversarial Defense
Guneet Dhillon Singh, Kamyar Azizzadenesheli, Jeremy Bernstein, Aran Khanna, Zachary Lipton, Anima Anandkumar
International Conference on Learning Representations (ICLR), 2018Learning from Noisy Singly-Labeled Data
Ashish Khetan, Zachary Lipton, Anima Anandkumar
International Conference on Learning Representations (ICLR), 2018Deep Active Learning for Named Entity Recognition
Yanyao Shen, Hyokun Yun, Zachary Lipton, Anima Anandkumar
International Conference on Learning Representations (ICLR), 2018Efficient exploration for dialogue policy learning with BBQ-networks
Zachary Lipton, Xiujun Li, Jianfeng Gao, Lihong Li, Li Deng
Association for the Advancement of Artificial Intelligence (AAAI), 2018Dance Dance Convolution
Chris Donahue, Zachary Lipton, Julian McAuley
International Conference on Machine Learning (ICML), 2017Predicting Surgery Duration with Neural Heteroscedastic Regression
Nathan Ng, Julian McAuley, Charles Elkan, Zachary Lipton
Machine Learning for Healthcare (MLHC), 2017Context Matters: Refining Object Detection in Video with Recurrent Neural Networks
Zachary Lipton, Subarna Tripathi, Serge Belongie, Truong Nguyen
British Machine Vision Conference (BMVC), 2016Modeling Missing Data in Clinical Time Series with RNNs
Zachary Lipton, David Kale, Randall Wetzel
Machine Learning for Healthcare (MLHC), 2016The Mythos of Model Interpretability
Zachary Lipton
Communications of the ACM (CACM), 2016Learning to Diagnose with LSTM Recurrent Neural Networks
Zachary Lipton, David Kale*, Charles Elkan, Randall Wetzel
International Conference on Learning Representations (ICLR), 2016Playing the Imitation Game with deep learning
Zachary Lipton, Charles Elkan
IEEE Spectrum, 2016Optimal thresholding of classifiers to maximize F1 measure
Zachary Lipton, Balakrishnan Narayanaswamy, Charles Elkan
European Conference on Machine Learning (ECML), 2014