Date: December 11th, 2017 9:23 PM
Author: pungent avocado house puppy
And in general, I think it's going to be an increasingly important question about the way that we handle protected classes generally, and maybe race specifically, in data science models of this type. Because otherwise it’s like: okay, you can’t directly model if a person is black. Can you use their zip code? Can you use the racial demographics for the zip code? Can you use things that correlate with the racial demographics of their zip code? And at what level do you draw the line?
And we know what we're doing for mortgage lending—and the answer there is, frankly, as a data scientist, a little bit offensive—which is that we don't give a shit where your house is. We just lend. That's what Rocket Mortgages does. It’s a fucking app, and you're like, “How can I get a million dollar loan with an app?” And the answer is that they legally can't tell where your house is. And the algorithm that you use to do mortgages has to be vetted by a federal agency.
That's an extreme, but that might be the extreme we go down, where every single time anybody gets assessed for anything, the actual algorithm and the inputs are assessed by a federal regulator.
---
I think it's the same thing where it's like, okay, you can't look at race, but can you look at correlates of race? Can you look at correlates of correlates of race? How far do you go down before you say, "Okay, that's okay to look at?”
---
You can load in tons and tons of demographic data, and it's disturbing when you see percent black in a zip code and percent Hispanic in a zip code be more important than borrower debt-to-income ratio when you run a credit model. When you see something like that, you're like, “Ooh, that's not good.” Because the frightening thing is that even if you remove those specific variables, if the signal is there, you're going to find correlates with it all the time, and you either need to have a regulator that says, “You can use these variables, you can't use these variables,” or, I don't know, we need to change the law.
(http://www.autoadmit.com/thread.php?thread_id=3825418&forum_id=2#34894210)