Past Performance is Not Indicative of Future Results (2020)

Let's talk about what machine learning is: it's a statistical inference tool. Anyone who's ever stared at clouds knows there are plenty of face-like elements of our real world, and no statistical picture of "Face-ness" is a perfect substitute for understanding what a face actually is. The problems of theory-free statistical inference go far beyond hallucinat­ing faces in the snow. Anyone who's ever taken a basic stats course knows that "Correlation isn't causation." For example, maybe the reason cops find more crime in Black neighborhoods because they harass Black people more with pretextual stops and searches that give them the basis to unfairly charge them, a process that leads to many unjust guilty pleas because the system is rigged to railroad people into pleading guilty rather than fighting charges. The only way to find out was to go and talk to both people and uncover the qualitative, internal, uncomputable parts of the experience. The role this deficit plays in magnifying bias has been well-theorized and well-publicized by this point: feed a hiring algorithm the resumes of previously successful candidates and you will end up hiring people who look exactly like the people you've hired all along; do the same thing with a credit-assessment system and you'll freeze out the same people who have historically faced financial discrimination; try it with risk-assessment for bail and you'll lock up the same people you've always slammed in jail before trial. Brilliant people have done remarkable things with it.