this post was submitted on 07 Oct 2024
82 points (100.0% liked)
TechTakes
1427 readers
118 users here now
Big brain tech dude got yet another clueless take over at HackerNews etc? Here's the place to vent. Orange site, VC foolishness, all welcome.
This is not debate club. Unless it’s amusing debate.
For actually-good tech, you want our NotAwfulTech community
founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Despite what the tech companies say, there are absolutely techniques for identifying the sources of their data, and there are absolutely techniques for good faith data removal upon request. I know this, because I've worked on such projects before on some of the less major tech companies that make some effort to abide by European laws.
The trick is, it costs money, and the economics shift such that one must eventually begin to do things like audit and curate. The shape and size of your business, plus how you address your markets, gains nuance that doesn't work when your entire business model is smooth, mindless amotirizing of other people's data.
But I don't envy these tech companies, or the increasing absurd stories they must tell to hide the truth. A handsome sword hangs above their heads.
The other reason they don't do it is because many models are trained on a large corpus of pirated texts, and documenting this would be a confession.
Not just in an 'I scraped the new york times without permission' kind of way, but in a 'I illegally downloaded a torrent containing bestsellers from the last 30 years' kind of way.
Bestsellers? There used to be torrents of basically all releases. My provider blocks torrent sites and I dont use a vpn so im not sure if people still do this, but downloading basically all books (in english) at once released in a certain period was possible
occasionally i see this for music (weekly new tracks)