Training data producers and consumers
I'm trying as hard as I can not to write about LLMs, or to write positively about them. It feels bad to be negative, but it also feels bad not to express ideas - writing, for me and many people, is a way to get ideas out of the head and to process them. Anyway, this is the rant I'm allowing myself this month.
This came from a shadcn tweet. He's a guy who wrote a very popular UI library, which tons of startups use and which helped land him a job at Vercel. So, a very prolific open source creator as well as someone who has done very well at the startup game.
You can just read the first bit of the tweet and get it. My quick summary is that he worked really hard and cared a lot about making a new feature and someone quickly and effortlessly ported it to another framework, probably under a different label.
The replies to the tweet are mostly braindead, like most junk on Twitter and the internet. The usual reckons - license it GPL if you don't want people to copy it, or "business execution" or whatever will matter, or this is the deal you signed up for, etc. Some people are sympathetic.
Anyway, it reminded me of some recent conversations and it made me think of a framing: there are people who create training data, and people who consume it.
I was at a party a while ago and my friends were talking about how good Wikipedia is but wondering who edits it. I do. I answered questions on Stack Overflow too. I've mapped stuff on OpenStreetMap. Written a lot of English and a lot of code and made them public.
I've created a lot of training data. Some people have created even more training data.
A lot of people have created very little training data, but they love to use training data in the form of LLM outputs.
Maybe this is an explanatory factor for people's feelings. Maybe some people have created about as much training data as they've consumed and they're fine with AI.
This is partly why I'm a little bitter about the whole situation. (But trying not to be! Feeling bad feels bad!) A lot of the people chipping in their input about shadcn's situation don't have projects of their own. They don't think much about how it feels to really focus on a project, to invest in it, and to publish it. They don't understand the complicated sense of ownership involved in open source. Or the warm fuzzy feelings of writing things and seeing them in the wild were-are-still incredible. How does it feel when you see more people cloning your work and erasing your name than using it directly?
They don't feel the loss of meaning because they never had it.
This is kind of an elitist thing to say, but it's not that different from the general problem that people take the work of other people for granted. Someone wrote all those books that taught the LLMs how to speak. They did it partly because it made them feel good. People will stop writing books and publishing their output if they feel like it will just get stolen. We'll end up in a world with less openness, less trust, and less good fuzzy feelings.
I think the worst part is that the norms of publishing open source software and publishing writing on the web were mostly community and interpersonal norms, not legal or technical ones. Before, you could rip off someone's code, but you'd be an asshole for doing it and the community would judge you. You could ignore robots.txt and scrape websites that asked you not to, but people would block you and make a fuss.
Now there just isn't a way to opt out. Whatever software you publish will be cloned and whatever you write on the internet will be scraped. The people involved don't really care and their investors don't either. Web scrapers disregard robots.txt and do nearly DDoS-levels of traffic nowadays to feed the machine.
So, trying hard is kind of a waste - your project will just get cloned, your ideas whitelabeled. Might as well ship a bunch of vibecoded projects with no new ideas instead, who cares.
That's enough ranting for now, I'll get back to partially using LLMs and trying to find ways to look at it that feel better.
Examples: