What if people start sending malicious code to github? Poisoning the AI model
this post was submitted on 26 Mar 2026
177 points (100.0% liked)
Technology
7078 readers
161 users here now
News community around technology, social media platforms, information technology and governmental policy surrounding it.
What doesn't fit here?
The core of the story has to be technology focused.
- If article mentions "AI" in a sentence and then talks about business economics that doesn't make it tech news.
- Gaming is too many layers removed from technology. There are many dedicated communities that are a better fit for it.
- Transporation is too many layers removed from technology. EVs while use many cool technologies have many dedicated communities that are a better fit for it.
- Entertainment is too many layers removed from technology. While sometimes it can fit here, business or cultural aspects of it are a better fit for dedicated communities.
- Cybersecurity. While it heavily focuses on technology, most of the time it's too technical for most people who are not already invested in it. Should be posted in a dedicated communities unless it has broader connection to other tech areas.
Post guidelines
Title format
Post title should mirror the news source title. If you don't like the title of article, look for an alternative source instead of editorializing it.
URL format
Post URL should be the original link to the article (even if paywalled) and archived copies left in the body. It allows avoiding duplicate posts when cross-posting.
[Opinion] prefix
Opinion (op-ed) articles must use [Opinion] prefix before the title. Opinion articles refer to articles that their publisher doesn't explictly endorse.
Country prefix
Country prefix can be added to the title with a separator (|, :, etc.) if the news is from a local publisher who doesn't clearly mention the country.
Rules
1. English only
Title and associated content has to be in English.
2. Use original link
Post URL should be the original link to the article (even if paywalled) and archived copies left in the body. It allows avoiding duplicate posts when cross-posting.
3. Respectful communication
All communication has to be respectful of differing opinions, viewpoints, and experiences.
4. Inclusivity
Everyone is welcome here regardless of age, body size, visible or invisible disability, ethnicity, sex characteristics, gender identity and expression, education, socio-economic status, nationality, personal appearance, race, caste, color, religion, or sexual identity and orientation.
5. Ad hominem attacks
Any kind of personal attacks are expressly forbidden. If you can't argue your position without attacking a person's character, you already lost the argument.
6. Off-topic tangents
Stay on topic. Keep it relevant.
7. Instance rules may apply
If something is not covered by community rules, but are against lemmy.zip instance rules, they will be enforced.
Companion communities
!globalnews@lemmy.zip
!interestingshare@lemmy.zip
Icon attribution | Banner attribution
If someone is interested in moderating this community, message @brikox@lemmy.zip.
founded 2 years ago
MODERATORS
That's what I tell myself I'm doing when I push more poorly written code to one of my repos
The data GitHub wants includes:
- Model outputs that have been accepted or modified;
- Model inputs including code snippets shown;
- Code context surrounding your cursor position;
- Comments and documentation you've written;
- File names and repo structure;
- Interactions with Copilot features (e.g. chats); and
- Feedback (e.g. thumbs up/down ratings)...As the FAQs explain: "If a Copilot user has their settings set to enable model training on their interaction data, code snippets from private repositories can be collected and used for model training while the user is actively engaged with Copilot while working in that repository."
Source: https://www.theregister.com/2026/03/26/github_ai_training_policy_changes/
Yay, it's not enough that most companies store highly confidential code on GitHub, now we will let a PUBLIC agent be trained on them.
Wonder how long it will take for people to find ways around guardrails and have the model essentially copy the entire codebase of a specific company with a simple prompt.
Do note that GitHub explicitly excludes those with an Enterprise (or other corporate) plan for precisely those reasons.
They do so now, but here's an update. What's the chance for a few years to reach another one?:
- "To keep up with global changes, we enabled Copilot for everyone, but there's still an option to opt-out! Plan Pro and up offers such an option just you decide on it! Meanwhile, we hope Copilot will help everyone for free, and with whole 10 tokens we gift you right now!"
To opt out, GitHub users should visit /settings/copilot/features and disable "Allow GitHub to use my data for AI model training" under the Privacy heading.
Do you guys really believe these opt-out buttons do anything?
Not /s btw, genuine question.
Maybe. But I wonder how would it apply: are my contributions to another user's repository still used for training if that user didn't opt out?
To mirror your question: do you really believe that a significant fraction of users will uncheck this checkbox?
Personally, I think only a few percent will do this and Microsoft does not care about losing their data.
But why would they honor it in the first place? You have absolutely no way to check what they do with your data.
So we're just supposed to take their word for it?
Would you also say that we have absolutely no way to confirm that Facebook managers know how addictive their apps are for children? I’m pretty sure that just a few years ago cynical people would say that. But only a couple of days ago this was proved in the court and Facebook lost the case.
So yeah, I believe that taking the company’s word for it is generally a winning strategy, and that blatant violations are relatively rare. Here, MS has a lot to lose and not much to win.
Placbo buttons to boil the frog.
Color me shocked! Jk everybody saw that coming... It was probably hinted at to get reactions, then went ahead when people didn't bitch too much.
I'd say I should pull my code from it, but I'm bad enough at coding that I think I'll leave it to poison the well
People still won't leave 🤷
I'm home sick today. Gonna start migrating my shit away from there
I stand corrected. Luckily 👍