Technology

Could a human have judged it better? Maybe not. I think a better question to ask is, "Should anyone be sent back into a violent domestic situation with no additional protection, no matter the calculated risk?" And as someone who has been on the receiving end of that conversation and later narrowly escaped a total-family-annihilation situation, I would say no...no one should be told that, even though they were in a terrifying, life-threatening situation, they will not be provided protection, and no further steps will be taken to keep them from being injured again, or from being killed next time. But even without algorithms, that happens constantly...the only thing the algorithm accomplishes is that the investigator / social worker / etc doesn't have to have any kind of personal connection with the victim, so they don't have to feel some kind of way for giving an innocent person a death sentence because they were just doing what the computer told them to.

Final thought: When you pair this practice with the ongoing conversation around the legality of women seeking divorce without their husband's consent, you have a terrifying and consistently deadly situation.

Thank you, this is why I came to the Fediverse from Reddit.

An algorithm is never to blame, some pencil necked desk jockey decided the criteria to get help that was used to create the algorithm, the blame is entirely on them.

That said, I doubt it would make any difference if a human was in the loop. An algorithm is still al algorithm, even if it's applied by a human. We usually just call that a "policy" though. People have been murdered by the paper sea for decades before we started calling it "algorithms".

Critical thinking spotted, proper authorities have been notified.

We will fix you!

It reminds me of the debate around self driving cars. Tesla has a flawed implementation of self driving tech, that's trying to gather all the information it needs through camera inputs vs using multiple sensor types. This doesn't always work, and has led to some questionable crashes where it definitely looks like a human driver could have avoided the crash.

However, even with Tesla's flawed self driving, They're supposed to have far fewer wrecks than humans driving. According to Tesla's safety report, Tesla's in self driving mode average 5-6 million miles per accident vs 1-1.5 million miles for Tesla drivers not using self driving (US average is 500-750k miles per accident).

So a system like this doesn't have to be perfect to do a far better job than people can, but that doesn't mean it won't feel terrible for the unlucky people who things go poorly for.

Here's another quote further down:

Since 2007, about 0.03 percent of Spain’s 814,000 reported victims of gender violence have been killed after being assessed by VioGén, the ministry said. During that time, repeat attacks have fallen to roughly 15 percent of all gender violence cases from 40 percent, according to government figures.

“If it weren’t for this, we would have more homicides and gender-based violence,” said Juan José López Ossorio, a psychologist who helped create VioGén and works for the Interior Ministry.

So no, not a scandal, it seems it is helping, but perhaps could be better. At least that's my read.

Is there evidence that a human would have judged those cases differently?

It implies that a human would have been worse. Or at least that an average human would be worse, the ones making the decision.

The article is not about how the AI is responsible for the death. It's likely that the woman would have died in the counterfactual.

The question is not "how effective is AI"? The question is should life or death decisions be made by an electrified Oracle at Delphi. You must answer this question before "is AI effective" becomes relevant.

If somebody was adjudicating traffic court with Tarot cards, would you ask: well how accurate are the cards compared to a judge?

My impression from the article is more that they're not doing any kind of garbage-in assessment: nobody is making sure they're getting answers about the right person (eg: some women date more than one guy) and some women don't feel safe giving accurate answers to the police, and there aren't good failsafes available for when it's wrong; you're forced to hire legal counsel and pursue a change via the courts.

The crucial point is: 8% of the decisions turn out to be wrong or misjudged.

The article says:

Yet roughly 8 percent of women who the algorithm found to be at negligible risk and 14 percent at low risk have reported being harmed again, according to Spain’s Interior Ministry, which oversees the system.

Granted, neither "negligible" or "low risk" means "no risk", but I think 8% and 14% are far too high numbers for those categories.

Furthermore, there's this crucial bit:

At least 247 women have also been killed by their current or former partner since 2007 after being assessed by VioGén, according to government figures. While that is a tiny fraction of gender violence cases, it points to the algorithm’s flaws. The New York Times found that in a judicial review of 98 of those homicides, 55 of the slain women were scored by VioGén as negligible or low risk for repeat abuse.

So in the 98 murders they reviewed, the algorithm put more than 50% of them at negligible or low risk for repeat abuse. That's a fucking coin flip!