The LLM provider/creator would have to disclose this. Easier said than done but you would need open-source models and not just open-weight models.
Programming
Welcome to the main community in programming.dev! Feel free to post anything relating to programming here!
Cross posting is strongly encouraged in the instance. If you feel your post or another person's post makes sense in another community cross post into it.
Hope you enjoy the instance!
Rules
Rules
- Follow the programming.dev instance rules
- Keep content related to programming in some way
- If you're posting long videos try to add in some form of tldr for those who don't want to watch videos
Wormhole
Follow the wormhole through a path of communities !webdev@programming.dev
What's wrong with copyleft code?
Nothing in itself. I just felt it was antithetical to the intention behind copyleft to train on it then share the result.
It's a good question, I'll think on it more.
I don't see why it would be. Isn't 'copyleft' basically: "You may modify and redistribute this code for free, but whatever you produce with it has to be copyleft as well"?
Extending that to coding agents trained on copyleft code, I would say that any code that agent writes is also copyleft. How is that anti-ethical?
You're probably looking for explicitly public domain code, not just avoiding copyleft. That will be rare to find. Permissive licenses such as MIT share with GPL and other copyleft licenses the aspect that they require attribution, while LLMs act as an attribution stripper.
Olmo might be the closest you can get
https://allenai.org/olmo (website straight up crashes my phone)
https://huggingface.co/allenai/OLMo-7B
Olmo is trained on the Dolma dataset, “a diverse mixture of web content, scientific papers, code, public-domain books, social media, and encyclopedic materials”, so this might be a more ethical source, but I’m not sure if it’s copyleft free.
https://huggingface.co/papers/2402.00159
You could rent a cluster of GPUs and remove copyleft material and retrain
Dolma uses starcoder https://huggingface.co/blog/starcoder, which
are Large Language Models for Code (Code LLMs) trained on permissively licensed data from GitHub
so this might do. Thanks.
Just because code isn't copyleft doesn't mean it does not require attribution or consent to being scraped by LLMs!
Also, I don't think there's any model out there that has asked for permission for scraping, and if any claim they do, they're almost certainly liars.
In my opinion, you should ideally avoid LLMs if you can do the task yourself. It definitely has some genuine applications, especially translation, but functionally, ethically, and legally, it is not a suitable option for most people.
Just because code isn’t copyleft doesn’t mean it does not require attribution or consent to being scraped by LLMs!
I am painfully aware. For my own work, I don't use LLMs, but I'm not convinced I'm going to be able to do that at work. I'm trying to work out the details of what that means and if it's irretrievably immoral. It's not looking good.