Access to and the use of copyrighted material has been legal since copyright was invented. You can find a copy of a (physical) textbook in a library. You can read it. You can use the theory and formulas you find in it to design a product. All without any permission from the copyright holder being required. And you will have full copyright over that product.
This is essential to a free society. When you can’t freely use your own time and your own physical property to earn a living without paying tribute to whoever owns the relevant (copy)rights, rights that were often obtained before you were born[1], you’re living in a feudal society. The land has been replaced by intellectual property. I’m worried we’re moving scarily far in that direction.[2]
I think we should all push for copyright to be weakened whenever we’re in a position to make a difference. Yes that means being an outlier. Neutrality without a moral anchor is a moral hazard, so I don’t think we should always strive for neutrality
due to copyrights being extended to the date of the death of all authors plus 70 years ↩︎
With Disney losing the mickey mouse copyright I thought we were finally moving in the right direction. But the AI panic has handed the new nobility a golden opportunity, and creators are helping them take it ↩︎
All people I’ve seen argue about the abandonment of copyright don’t seem to have a good replacement. I think if FOSS licenses end up effectively meaningless, many will leave the ecosystem. Same for artists no longer able to sell their art if AI companies just turn it into competing copies for free.
I’ve also not seen a study showing a significant(!) productivity increase wih LLMs.
I have no interest in debating this video or anything, but the video seems to show literal text duplication examples too. If you watch the clip on Github I linked, it seems to have two of those.
The field study explains this “conflict”: LLMs simply seem to do both types a lot.
I didn’t watch the video. Half an hour to watch a talking head is too big an ask
Note that I didn’t claim literal duplication never happens. I said that wasn’t my general experience across decades, since long before LLMs were invented. And still isn’t. “AI slop” routinely shows up in Python spaces now, and is almost never a case of duplicated text. Instead it’s fundamentally senseless word salad. That’s the real time drain we’re facing.
One large instance of which I presented to Copilot, and posted that AI’s analysis here. It nailed it, instantly picking up on numerous strong signs of AI provenance, but nothing about text duplication. Everything about style, semantic incoherence, and falling into kinds of hallucinations common to LLMs. Also signs of that the human who posted it didn’t know enough about the subject matter to detect the incoherence themselves, and compounded the incoherence by adding their own confusions in their attempts to edit the assistant’s outputs to disguise where it came from.
Which aren’t AI problems, but “people problems”.
“Cranks” have always been with us too. Sometimes they can employ AI to make their rambling more plausible on the surface, but not good enough yet even to fool another AI
Libraries pay for their physical books. That is different from using copyrighted works without any payment. If there is a cache of pirated, copyrighted works online do you think that you can use them in your product because you can then gain access to the pirated works? If that product is shown to benefit from from copyrighted works, should it have to pay? Should it only have to pay off those with expensive lawyers? I have decided to leave the question marks as people have differing opinions.
OK! I took the half hour to watch the full video. It contained 3 instances of “literal text duplication” in all. The 3rd was the best (code to compute reciprocal square roots quickly, exceedingly novel for both the algorithm and the snarky comments), but not included in the brief snippet in the comment you linked directly to.
Most of the talk swung to the Chardet case, though, which the audience too was much more engaged with. No literal text duplication. Claude AI rewrote an entire, non-trivial, LGPL’ed code base, and Claude’s human slapped an MIT license on it.
Best I can tell, that hasn’t yet made it to a court. In my eyes it was wrong, because I’m a Golden Rule guy, and care about decency much more than legalistic hair-splitting. Regardless of what courts make up about what laws “really mean”, it obviously violated the intent of the original author, who used the LGPL for their own reasons. “Honor their wishes” is the decent thing to do, on the face of it.
It’s unusual too because the copier has been very open about what they did, and insists there was nothing wrong with it.
People have always been jerks at times, and copiers have always existed. AI can make widespread superficial changes much faster then people, but there’s nothing fundamentally new about the human dynamics in play.
“The law” will stumble into new approaches. The big money players will fund think tanks and lobbyists to write “model legislation” for clueless legislators to pass, and get what they want in the end.
In the meantime, I can only suggest we continue reviewing PRs for quality, and best we can for legality, regardless of source. And asking for the help of an AI assistant to judge possible plagiarism seems thoroughly reasonable to me. In my experience so far, Copilot is better at that than most humans.
I understand the sentiment and welcome you clear statement of facts and opinions, even though I don’t agree with your conclusions.
To achieve your goals, a ban does look more straightforward. But I see many pragmatic issues, even accepting your premises.
E.g. how would it work if an LLM-based code review suggests a trivial, one-line fix, as code? Should it be left unfixed due to the provenance of the suggestion? Or should the issue be closed, then we wait for some human to make the same suggestion (hopefully without having read the LLM suggestion)? What if it’s a bigger contribution, would it require clean room re-implementation? Even if it’s an urgent security issue?
How should cases where the contributor uses LLMs to be able to discuss in English, including their code contributions (what if the LLM flags an error in the code, are they supposed to can the whole contribution?), be handled?
And similar issues that have been raised, not about the intent, but the day-to-day feasibility. I’m not worried about contributors lying, the system cannot guard against that IMO and it’s the contributor’s responsibility. I’m worried about how core developers would have to thread the fine line of handling code that has been contaminated with LLM input.
In any case, thank you for raising this discussion and for doing so in such a careful, evidence-based way.
Leaving aside the questions about copyright, ethics etc my experience of dealing with people using LLMs in pull requests, issues, mailing lists etc is that the worst part is LLM text in communication rather than using LLMs to write code. It leads to a complete breakdown in human to human communication and is massively demotivating.
I would rather talk to someone in broken English or read through typos than see the output of an LLM any time. Someone thinks that the LLM has improved their grammar but once I see the em dash then that is far worse than bad grammar because now any part of the text can be a hallucination and I have no idea whether the human understands anything. It is actually worse than talking to an LLM directly because you don’t know which parts are from the LLM and which are from the human and humans and LLMs are prone to different forms of (mis)understanding and confusion: it is very hard to interpret what the human understands and is trying to communicate in this situation.
Tim quite rightly pointed out above that he would never post LLM output without clearly marking it as such and distinguishing it very explicitly from the human text. I think that for open source communication between strangers on the internet it is absolutely essential that people do this. Many people do not do this though and the most common reason given is that English is a second language. I have some sympathy with this but I think it still needs to be considered unacceptable because the people doing it inevitably use the LLM for more than just translation. If there are LLM-based tools that can stick to pure translation with very low hallucination rate then maybe that would be acceptable although it would still need to be explicitly marked but I am not aware of tools like that (google translate?).
Reviewing some PRs now feels like prompting a broken AI agent. You comment on the PR and then the human on the other side garbles your prompt, feeds it into an LLM and then spits more LLM weirdness back at you. The humans who are doing this need to be shown a very clear statement about why this is unacceptable. It is not just a case that it should be forbidden for ethical or other reasons but that practically they will destroy the motivation for anyone to review PRs or reply to them. They even make it worse for the people who are not using LLMs because now I am suspicious of any new person online which undermines the general trust on which open source is based.
I would say that even if LLMs are allowed to write code there needs to be very clear rules about how LLMs can and cannot be used in communication.
If you want to stick to just that, while I haven’t belabored the point, I’m with those who already explained why they don’t believe “a ban” could be effective. At best it would drive those using bots to work a bit harder at disguising the source. It certainly would not stop bot-generated code from showing up in PRs, It’s already the case that no core dev would knowingly accept a PR with plagiarized code (whether via literal text duplication or via “uncanny similarity” to other projects’ work).
Suggesting alternatives is a way of acknowledging that the original issues raised are real and worthy of addressing, but proactively suggesting other approaches that may be more effective in real life. Yes, that does expand the scope of the original, by looking at the problem to be solved rather than just picking on a suggested approach. If the latter is all you want, then I’m “-1” on the suggested approach, and that’s the end of it.
We had a mildly interesting case of LLM-enabled confusion this week on the PSRT list (private list for Python security issues). A user thought they saw a possible use-after-free vulnerability in the array.array code. According to me, there isn’t one where they thought, but the code is a bit subtle and easy to misunderstand.
Anyway, in the end it turned out that they engaged an LLM to help write up the report, in the process of which the LLM rewrote parts of their proof-of-concept code in ways that made the confusion worse. The only Big Clue of that we had is that their report had a section labelled “Expected output” (which would have confirmed a security hole) but output their code didn’t actually produce.
Which is a little-recognized danger of LLMs too: even when used as intended, they tend to be thought amplifiers, expanding on what you already believe. In this case, the user and their bot took turns pushing each other deeper into a rabbit hole.
Always push back! I find bots very useful, but much more for the creative (“pseudo” or not I don’t care) dialog they can sustain than for what either of us actually says. It’s ego-free brainstorming with a partner that never gets tired, touchy, or tunes out.
In this case, I certainly would have challenged the bot to explain why the “expected output” wasn’t seen.
I’m sorely tempted to point Copilot at the chain of messages that went into this report, to see whether it could have caught signs of AI-amplified confusion earlier than any of us did (we never did - at the end, the user volunteered details of what actually happened).
But communications with the PSRT are confidential, so that’s a non-starter. In hindsight, I imagine I see some clues that Copilot may well have picked up on. But that may be my own human hallucination.
i do have people suggesting on how to better break free from AI/LLM tendencies to please their users (but also their brand, if they are provider online like OpenAI & Anthropic), mostly by asking the AI for steelmanning themselves and other philosopical disciplines, which maybe too tiring for humans to do (i have tried a debate club in university a bit)
so it is actually kind of interesting reading above posts, i really appreciate it
Sorry, I revised this a little bit too much so let me post this cleaned up as a new item:
I see, but e.g. this article puts it at 8-10%, so how would you even know when you encounter a plagiarized item? https://www.theatlantic.com/technology/2026/01/ai-memorization-research/685552/
I don’t understand why you would put a gut feeling over data. Your impression may be one thing, but the data apparently says the opposite. Edit: but I’d be happy to be corrected!
And I don’t get the confidence that the average well-meaning LLM user or reviewer would notice once (apparently inevitably) plagiarism is injected into the CPython code.
No policy is perfect, all policies have enforcement issues. It could still be worth doing anyway.
If in doubt, yes. Point out the problem to another coder that hasn’t seen the LLM suggestion, and have them fix it.
However, clearly I’m not the person to make any of the decisions. I hope my clarifications help somebody anyway.
But I’m not going to hassle with finding a way around the paywall
I wouldn’t be surprised if some bots save all their training data verbatim. But it doesn’t matter to me: plagiarism has been a fact of life forever, both literal duplication and attempts to disguise. That’s a human problem. The bots themselves (best I can tell) don’t submit PRs, only people do.
The answer to your question is that I generally wouldn’t know. Already related that the only case of literal duplication in CPython I can recall had nothing to do with AI, but with a contributor who copied some glibc code because they didn’t know any better. No intent to deceive: a comment before the copied code plainly said that what followed was copied from glibc.
It it weren’t for that comment, I don’t think anyone would have noticed (and know for sure that I would not have). I expect that AI is actually much better at detecting such stuff than people are, and especially so when human intent to deceive is in play. Already posted a real-life example of Copilot nailing numerous strong clues of “AI slop” in a message on this forum that people missed. That thread “seemed off” to many, but only Copilot produced a cogent and evidence-based analysis of exactly what was “off” about it.
Does the data establish that “put up words banning AI?” is an effective mitigation?
I want approaches that have a chance of being effective even in a future world where all AI assistants have total access to all the world’s information, and the companies that market them are up-front about that … and ever-more people are keen to deceive.
I have, to the contrary, every expectation that people in general are poor at recognizing plagiarism. I also expect (but don’t know) that there’s already plagiarized code in CPython, and dating from before bot days. How could we possibly know? Brute force “longest substring” matching against a massive corpus of all the world’s code could catch literal duplication 'in theory", but I don’t know of a practical way to apply it. I know there are various tools there claiming to have abilities like that, but have no experience of them.
I don’t think so. Based on age and lifelong repeated experience of “so ban it!” approaches being actively counterproductive in many areas of human behavior. I’d be all in favor, though, of beefing up what we already say to point out the dangers of unintended plagiarism, which AI assistants have surely intensified (whether via literal duplication or substantial rewriting makes scant difference to me - or to the people whose work was copied).
When I review a PR now, and get suspicious. I’ll ask Copilot for its opinion. It’s already proved to me that it’s better at this than I am. Although I fancy I have gotten better at it lately, thanks to learning from Copilot some patterns to look for that I wasn’t attuned to before.
But I don’t want to force that approach on others.
As a brief tangent, this has indeed begun to happen. I’ve personally received PRs submitted by AI agents on projects I maintain, and there was at least one somewhat newsworthy case of it in recent memory: An AI Agent Published a Hit Piece on Me – The Shamblog
Perhaps CPython has avoided it so far, but I would probably treat it as an inevitability.
Very interesting! Thanks. Sounds like a lone rapscallion set up their own bot as a “social experiment”, and it went far beyond what they expected. But no way to know for sure.
I’m open to that. Indeed, it’s a large part of why I don’t think “ban it!” can be effective in real life. Submissions from bots don’t come with “NOTE: a bot produced this” at the top. and there’s no limit on how perverse people can be.
All we’ll usually have to go on is the submission itself. Mining that for telltale signs may be helpful, and asking a bot to try its hand at looking for signs even more helpful. Or not. Only way to know is to try it.
To my eyes, the rant reproduced in the piece you linked to did shows signs of AI provenance.
But they should. Bots should be labelled as bots and it should be possible to trace back to who controls them. Anyone who is thinking of making a bot and violating this rule should be told very clearly that that is not acceptable.
GitHub and other things like this forum should enforce this rule: no bots or if there are bots then they are clearly marked as such, and it is clear which humans control which bots. Right now GitHub does not even have a button I can use to report “this account is a bot”.
And if they ignore that rule? Many will. Make “a rule” and many humans will scheme to violate it “just for fun” at best.
I believe some government agencies can reliably trace Internet traffic back to its true source, and usually also to the people who control it, but by exploiting massive resources and the legal power to compel providers to honor secret requests for supposedly private info.
For the rest of us, the web wasn’t designed for accountability - and it shows
That said, I do think I’ve seen signs of homegrown bots spamming Python’s issue tracker, but still pretty rare and very crude. They get squashed quickly. And all but certain to get worse, alas. But it’s impossible for me to imagine how saying “that’s forbidden!” could help. As you say, GitHub adding a clear way to say “such-and-such an account appears to be malicious” might help.
You can see an example here. AI companies have trained bots and put me in their training data so that they tag me on GitHub. They make their bots look like human accounts and don’t leave any indication of how you can contact the human controller. It should be possible for me to report that to GitHub have them banned for breaking the bot rules (that don’t currently exist). These problems are going to get much worse if not addressed.
I can’t say. The human (if it was one!) who engaged in the discussion there didn’t come off like AI at all, and explained that tagging you was an unintended consequence of a mistake in how their bot was configured - and, indeed, that submitting the issue to the sympy repo to begin with was due to the same mistake.
If if stopped then, I’d take them at their word. Did it stop then? I assume that if it had continued, you would have also linked to more recent instances.
They make their bots look like human accounts and don’t leave any indication of how you can contact the human controller.
According to their story, the issue report never should have been pushed to begin with. So in the absence of more evidence, the Principle of Charity applies for me. “Stuff happens - move on.”
It should be possible for me to report that to GitHub have them banned for breaking the bot rules (that don’t currently exist). These problems are going to get much worse if not addressed.
I agree GitHub should have a way to report malicious actions - of all kinds, including, but not just limited to, AI-related bad actions. They don’t need special rules to block spammers, and garbage reports are very arguably spam regardless of source.
But happy to change my mind if an army of bots starts targeting me, of course
↩︎