Derived from tens of thousands of qualitative evaluations, collected over three decades, by a representative sample of thousands of legal experts.
The Jurist-Derived Judicial Ideology Scores (JuDJIS) project is a new measure of judicial ideology — and other traits — that will locate on a single scale nearly every Article III U.S. federal judge serving since 1990 (approximately 4,900 judges). Project development began in 2015, and the first phase, Circuit: Ideology, was released in summer 2024; this release estimates the ideology of essentially every U.S. Court of Appeals judge who's served from 1990–2022: at least 450 judges.
The quantitative measure is derived from tens of thousands of qualitative evaluations — an ongoing, third-party initiative conducted over three decades — by a representative sample of thousands of legal experts, i.e., jurists, familiar with the judges' approaches to judging.
By drawing on expert evaluations, this method hopes to overcome measurement issues such as endogeneity, sample bias, lack of dynamic capability, and difficulty measuring the lower courts. Notably, the JuDJIS scores are perhaps the first to take account of the subtle differences in judges' opinions and other lawmaking behavior, allowing them to differentiate between the vast majority of judges in the dataset.
Analysis of a set of appellate-decision data indicates that the time-aggregated point estimates predict appellate outcomes more accurately than existing appellate-judge ideology measures. This project aims to help foster breakthroughs in fields such as judicial behavior, jurisprudence, and empirical legal studies, opening new avenues of research.
I thank participants at the 2024 meeting of the American Law & Economics Association, 2024 NYU School of Law External Law & Economics Workshop, 2019 University College Dublin Quantitative Text Analysis Dublin (QTA-DUB) Workshop, 2019 Hebrew University Empirical Study of Public Law & Human Rights Workshop, 2019 ETH Zurich Conference on Data Science and Law, 2019 Princeton University Political Economy and Public Law Conference, 2018 Meeting of the American Political Science Association (APSA), University of Michigan Inter-disciplinary Workshop in American Politics, and 2018 Conference on Empirical Legal Studies (CELS). I thank Megan Rosen, editor of the Almanac of the Federal Judiciary, for providing access to the archived Almanac files.
I also thank Deborah Beim, Adam Chilton, Michael Gilbert, Joshua Fischman, Richard Hynes, Michael Nelson, Kevin Quinn, Kelly Rader, Kyle Rozema, Megan Stevenson, and Mariah Zeisberg for helpful comments. I especially thank Charles Crabtree and Adam Feldman for many constructive formative conversations.
I thank Kevin Breiner, Ruixing Cao, Husnain Choudhry, Eddie Colombo, Danielle Gibbons, Jake Greenberg, Conor Hargen, Jeffrey Horn, Samuel Lin, Joseph Park, Vaghif Salem, Leighton Schnedler, Jacob Smith, and Latrell Williams for outstanding research assistance or website support.
Finally, I thank Li Zhang of the UVA Legal Data Lab for his tireless and invaluable assistance with text analysis over the last two years.