Fairness/Accountability

Inspired by recent breakthroughs in predictive modeling, practitioners in both industry and government have turned to machine learning with hopes of operationalizing predictions to drive automated decisions. Unfortunately, many social desiderata concerning consequential decisions, such as justice or fairness, have no natural formulation within a purely predictive framework. In efforts to mitigate these problems, researchers have proposed a variety of metrics for quantifying deviations from various statistical parities that we might expect to observe in a fair world and offered a variety of algorithms in attempts to satisfy subsets of these parities or to trade off the degree to which they are satisfied against utility. We are interested in studying the wider systems. What must we know about how disparities arose in the first place, what decision is influenced by the prediction, the normative principles for assigning responsibility, and the impacts of interventions to give practical guidance to ML practitioners?

• Red-Teaming for Generative AI: Silver Bullet or Security Theater?

  Michael Feffer, Anusha Sinha, Wesley H. Deng, Zachary Lipton, Hoda Heidari
  AAAI/ACM Conference on Artificial Intelligence, Ethics and Society (AIES)

• Auditing Fairness under Unobserved Confounding

  Emily Byun, Dylan Sam, Michael Oberst, Zachary Lipton, Bryan Wilder
  AISTATS

• DeepDrake ft. BTS-GAN and TayloRVC: A Survey of Musical Deepfake Models

  Michael Feffer, Zachary Lipton, Chris Donahue
  Workshop on Human-Centric Music Information Research (HCMIR@ISMIR)

• The AI Incident Database as an Educational Tool to Raise Awareness of Harms: A Classroom Exploration of Efficacy, Limitations, & Future Design Improvements

  Michael Feffer, Nikolas Martelaro, Hoda Heidari
  ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (EAAMO)

• From Preference Elicitation to Participatory ML: A Critical Survey & Guidelines for Future Research

  Michael Feffer, Michael Skirpan, Zachary Lipton*, Hoda Heidari*
  AAAI/ACM Conference on Artificial Intelligence, Ethics and Society (AIES)

• A Suite of Fairness Datasets for Tabular Classification

  Martin Hirzel, Michael Feffer
  arXiv

• Moral Machine or Tyranny of the Majority?

  Michael Feffer, Hoda Heidari*, Zachary Lipton*
  Association for the Advancement of Artificial Intelligence (AAAI)

• Searching for Fairer Machine Learning Ensembles

  Michael Feffer, Martin Hirzel, Samuel C. Hoffman, Kiran Kate, Parikshit Ram, Avraham Shinnar
  International Conference on Automated Machine Learning (AutoML)

• A Sandbox Tool to Bias(Stress)-Test Fairness Algorithms

  Nil-Jana Akpinar, Manish Nagireddy, Logan Stapleton, Hao-Fei Cheng, Haiyi Zhu, Steven Wu, Hoda Heidari
  Poster at the Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (EAAMO 2022)

• Counterfactual Metrics for Auditing Black-Box Recommender Systems for Ethical Concerns

  Nil-Jana Akpinar*, Liu Leqi*, Dylan Hadfield-Menell, Zachary Lipton
  Workshop on Responsible Decision Making in Dynamic Environments. International Conference on Machine Learning (ICML 2022)

• Resolving the Human Subjects Status of Machine Learning's Crowdworkers

  Divyansh Kaushik, Zachary Lipton, Alex John London
  arXiv Preprint

• Long-Term Dynamics of Fairness Intervention in Connection Recommender Systems

  Nil-Jana Akpinar, Cyrus DiCiccio, Preetam Nandy, Kinjal Basu
  Conference on Artificial Intelligence, Ethics, and Society (AIES)

• Modeling Attrition in Recommender Systems with Departing Bandits

  Omer Ben-Porat*, Lee Cohen*, Liu Leqi*, Zachary Lipton, Yishay Mansour
  Association for the Advancement of Artificial Intelligence (AAAI)

• Off-Policy Risk Assessment for Markov Decision Processes

  Audrey Huang, Liu Leqi, Kamyar Azizzadenesheli, Zachary Lipton
  International Conference on Artificial Intelligence and Statistics (AISTATS)

• Rebounding Bandits for Modeling Satiation Effects

  Liu Leqi, Fatma Kilinc-Karzan, Zachary Lipton, Alan Montgomery
  Advances in Neural Information Processing Systems (NeurIPS)

• Evaluating Explanations: How much do explanations from the teacher aid students?

  Danish Pruthi, Rachit Bansal, Bhuwan Dhingra, Livio Baldini Soares, Michael Collins, Zachary Lipton, Graham Neubig, William Cohen
  Transactions of the Association for Computational Linguistics (TACL)

• The Impact of Algorithmic Risk Assessments on Human Predictions and its Analysis via Crowdsourcing Studies

  Riccardo Fogliato, Alexandra Chouldechova, Zachary Lipton
  Computer-supported cooperative work (CSCW)

• When Curation Becomes Creation: Algorithms, microcontent, and the vanishing distinction between platforms and creators

  Liu Leqi, Zachary Lipton, Dylan Hadfield-Menell
  Communications of the ACM (CACM)

• On the Validity of Arrest as a Proxy for Offense: Race and the Likelihood of Arrest for Violent Crimes

  Riccardo Fogliato, Alice Xiang, Zachary Lipton, Daniel Nagin, Alexandra Chouldechova
  Conference on AI, Ethics, and Society (AIES)

• Fair Machine Learning Under Partial Compliance

  Jessica Dai, Sina Fazelpour, Zachary Lipton
  Conference on Artificial Intelligence, Ethics, and Society (AIES)

• The effect of differential victim crime reporting on predictive policing systems

  Nil-Jana Akpinar, Maria De-Arteaga, Alexandra Chouldechova
  Conference on Fairness, Accountability, and Transparency (FAccT 2021)

• Explaining The Efficacy of Counterfactually-Augmented Data

  Divyansh Kaushik, Zachary Lipton
  International Conference on Learning Representations (ICLR)

• Decoding and Diversity in Machine Translation

  Nick Roberts, Davis Liang, Graham Neubig, Zachary Lipton
  NeurIPS Resistance AI Workshop

• Weakly and Semisupervised Evidence Extraction

  Danish Pruthi, Bhuwan Dhingra, Graham Neubig, Zachary Lipton
  Findings of EMNLP

• Algorithmic Fairness from a Non-Ideal Perspective

  Sina Fazelpour, Zachary Lipton
  AAAI/ACM Conference on Artificial Intelligence, Ethics and Society (AIES)

• Uniform Convergence of Rank-weighted Learning

  Justin Khim, Liu Leqi, Adarsh Prasad, Pradeep Ravikumar
  International Conference on Machine Learning (ICML)

• Automated Dependence Plots

  David I. Inouye, Liu Leqi, Joon Sik Kim, Bryon Aragam, Pradeep Ravikumar
  Uncertainty in Artificial Intelligence (UAI)

• Learning to Deceive with Attention-Based Explanations

  Danish Pruthi, Mansi Gupta, Bhuwan Dhingra, Graham Neubig, Zachary Lipton
  Association for Computational Linguistics (ACL)

• Fairness Evaluation in the Presence of Biased Noisy Labels

  Riccardo Fogliato, Max G'Sell, Alexandra Chouldechova
  International Conference on Artificial Intelligence and Statistics (AISTATS)

• A Case for Humans-in-the-Loop: Decisions in the Presence of Erroneous Algorithmic Scores

  Maria De Arteaga*, Riccardo Fogliato*, Alexandra Chouldechova
  Conference on Human Factors in Computing Systems (CHI)

• Efficient candidate screening under multiple tests and implications for fairness

  Lee Cohen, Zachary Lipton, Yishay Mansour
  Foundations of Responsible Computing Foundations of Responsible Computing (FORC)

• Learning the Difference that Makes a Difference with Counterfactually-Augmented Data

  Divyansh Kaushik, Zachary Lipton
  International Conference on Learning Representations (ICLR)

• On Human-Aligned Risk Minimization

  Liu Leqi, Adarsh Prasad, Pradeep Ravikumar
  Advances in Neural Information Processing Systems (NeurIPS)

• Game Design for Eliciting Distinguishable Behavior

  Fan Yang, Liu Leqi, Yifan Wu, Zachary Lipton, Pradeep Ravikumar, William Cohen, Tom Mitchell
  Advances in Neural Information Processing Systems (NeurIPS)

• Troubling Trends in Machine Learning Scholarship

  Zachary Lipton*, Jacob Steinhardt*
  Communications of the ACM (CACM)

• Does Mitigating ML’s Impact Disparity Require Treatment Disparity?

  Zachary Lipton, Alexandra Chouldechova, Julian McAuley
  Advances in Neural Information Processing (NeurIPS)

• The Mythos of Model Interpretability

  Zachary Lipton
  Communications of the ACM (CACM)