They can sometimes be correct and helpful, but they can also be misleading and inaccurate, and unlike human posts which typically have certainly patterns and characteristics that make it usually possible to tell the difference between the two, it can be very hard to tell with ChatGPT, as it does a good job emulating the form of a good answer without necessarily having the substance, and it can require subject matter knowledge to determine if its misleading or not. It can also be used somewhat abusively by users and bots to gather rep/trust level in order to spam, etc. later.
Stack Overflow banned all such ChatGPT answers; their post and the responses there provide some useful context about this situation.
How should we handle it? Ban them like SO? Just consider them case by case like other human answers that aren’t helpful, off-topic, etc? Or something in between?
ChatGPT is entirely inappropriate as a means of constructing code or answers relating to code, since it has no concept of correctness. A policy stating that it is banned would go a long way to making it clear that people answer questions, not language models.
I would also suggest that questions about AI generated code be required to clearly state that the code is AI generated. At a glance, AI generated code gives the impression of a certain level of proficiency on the part of the user asking the question, which in turn informs the kind of answer they are likely to get. Then, as happened in the recent ELOCKUNMAPPED thread, it turns out that the code is actually nonsense and the thread devolves into a meta discussion about this very topic.
However, how would this policy actually be enforced? The thing with AI generated content is that it does a reasonably good job at masquerading as human-made, and it’s only going to become more difficult to tell the two apart.
Such content is harmful. It pollutes the informational space with a noise and reduces the value of other posts. It increases entropy and brings the heat death of the universe closer. It should be treated like environmental pollution and vandalism.
I think banning is reasonable. Posting a ChatGPT-generated reply is not more useful than pasting the results of a Google search (nobody does that, fortunately), and it’s much more misleading.
Copy-pasting a search results page is bad. Copy-pasting AI-generated output whole is bad.
Carefully picking one or two links out of search results from a search you made yourself is perfectly acceptable to me. So I guess I would not mind if someone were to ask some AI first, judge the output, and then finally rewrite it for me in their own words – but I guess that is not what is happening.
[And yes, it is becoming quite a pain on StackOverflow. I hope this wave is going to pass, quickly would be preferable. Can’t wait for the new AIs to train on content generated by previous, less capable AIs.]
For now, we should ban ChatGPT on Python Discourse, in order hold back the flood as we maintain our bearings. However, it is almost inevitable that the dam will break as it becomes increasingly difficult to distinguish content generated by machines from that generated by humans. It may already be true that a good strategy, among others, for solving a problem is to take content generated by AI as raw material, and to shape it into a final product via varying degrees of editing. As it may become routine for much of written and other creativity to consist of a blend of content generated by humans and machines, where should the line be drawn between what is permitted and what is not?
With its seeming likely that a ban on ChatGPT and the like will fail over time, we need to consider, among other things, what value content generated by humans will continue to have in an age of emerging AI. Perhaps one launching point, among many, for a discussion could be this:
I would suggest that if the person in the hypothetical room were enabled to inject a generous amount of human judgment into the translation process, we would have a much more creative, and hence valuable, translation than one generated solely by rigid rules. That is because a human brain works much differently from a digital machine, introducing a type of serendipity into the final product that a digital machine by itself cannot offer. Likewise, with our maintaining a great deal of caution, there may eventually be a regulated role for something like ChatGPT to play in generating content for a forum, so long as it is not allowed to smother the human element.
So, let’s institute a ban on ChatGPT here, as soon as we can institute it with a well-stated policy. While the ban is in place, let’s figure out how we can harness the best of what ChatGPT and kin can offer, while not allowing such products to damage the culture.
This forum is useful because it’s a source of users personally interacting and verifying. ChatGPT and other LLMs don’t do either of those things accurately, only convincingly.
I think banning AI generated content is entirely reasonable.
However, I do have one note of caution. Please be careful about accusing people of using AI, particularly so for questions posted to this forum (as opposed to answers / replies).
In the past couple months, many people have shared examples of weird looking code to me. In my informed opinion, most of these have not been ChatGPT generated (for example, the ELOCKUNMAPPED referenced above is certainly not ChatGPT generated and probably just an unfortunate interaction with an IDE). Confused beginners have and always will do bizarrely confusing things, and we should do our best to help them instead of dehumanising them or casting aspersions.
Yeah, despite the strong consensus (which I share) for banning it at least for now, this I think is the real difficulty about doing so—absent some smoking gun or the poster being honest and actually attributing it (which in >90% of cases its likely to be new users who wouldn’t bother to) it’s never possible to be 100% confident, especially as AIs continue to improve. You’d basically need another AI training on the output of the first AI to detect whether a given post matches the AI’s output…which basically leads to a star-bellied/plain-bellied Sneetches situation. Not sure if you happen to have any secret sauce for that.
I think there might be some confusion with the other thread—the post I referenced here is the almost certainly AI one, rather than the ELOCKUNMAPPED example I reference on the other thread where I was mostly joking about it being AI-generated, which I shouldn’t have in hindsight, and if I’d put a bit more thought into it, it was pretty obviously not.
Too late… I just did exactly this (not here, on StackOverflow, wrongly accusing a contribution to have been AI-generated). In hindsight it was obviously not AI. I feel bad and this is frustrating for all (?) parties. I am worried this will keep happening and I will need to learn how to behave to prevent false accusations. Online interactions were already quite tense in some parts of the Internet, this will not make it any easier. Right now it is manageable, but it feels like it is getting worse by the day, let’s see how it evolves.
Do we need a “Suspicion of AI-generated content” choice in the “Flag post” dialog? Or how should we flag, what is the recommendation?
[Should we avoid using the term “ChatGPT”? As far as I know there are more than just this one]
I think we should generally avoid accusing other users directly of using ChatGPT or the like for composing a post. Flagging suspicious material is the best action. If necessary, describe what you feel is the tell. Subsequently, the Moderators and Administrators can do their best to make the appropriate determination.
It’s Off-Topic
This post is not relevant to the current discussion as defined by the title and first post, and should probably be moved elsewhere.
It’s Inappropriate
This post contains content that a reasonable person would consider offensive, abusive, to be hateful conduct or a violation of our community guidelines.
It’s Spam
This post is an advertisement, or vandalism. It is not useful or relevant to the current topic.
Something Else
This post requires staff attention for another reason not listed above.
I would recommend instead of accusing them for using AI, just ask.
So instead of: “surely you used AI to make this up, there’s no way a human could come up with this”
Do: “This code doesn’t quite make sense. Did you come up with this or copy pasted from somewhere?” (because it’s possible that they copy paste from untrustworthy blog post / tutorial, but as a newbie, they didn’t know that such place isn’t trustworthy)
For policy change within our own discourse forum, I would be ok if we start with “people must disclose if they used AI”.
I think that choice allows the flagger to cite specific details, so it might suffice. But if ChatGPT posts become particularly abundant, an additional dedicated choice could be added that invites the flagger to cite the perceived evidence.
So, let’s institute a ban on ChatGPT here, as soon as we can institute
it with a well-stated policy. While the ban is in place, let’s figure
out how we can harness the best of what ChatGPT and kin can offer,
while not allowing such products to damage the culture.
Large Language Models (LLMs) in general I presume; let’s not go down the
path of specific “apps”. No objection to citing ChatGPT as the common
example of course.
By my count, there are fewer than a dozen unique posters who have contributed to this thread, and only a handful of those are regular contributors to the Help topic. I think it is premature to talk about a “consensus” to ban ChatGTP based on so few people.
I agree with a complete ban on using ChatGTP output for answers to questions. I doubt that’s enforceable though.
But otherwise posting honest questions based on ChatGTP output, such as “How does this work?” or “How do I fix this so it does work?” should be allowed, so long as the poster is open and upfront about the code coming from ChatGTP.
Similarly, I don’t mind people sharing especially spectacular examples of ChatGTP code (whether good or bad) for discussion.
Either way, only if they’re up-front about it being ChatGPT-generated code. In the latter case that won’t be hard (and let’s be honest, taking cheap shots at bad AI-generated text may be cheap, but it really is funny sometimes), but in the former case, it really needs to be said clearly.
Unfortunately AI-generated code often makes a really REALLY bad basis for learning. It’s one thing if it leads you to ask a question, but much worse if you don’t know to ask. AI generation is a tool for experts, not a tutor for new programmers.
Just in case: please flag for such situations on Stack Overflow - if you’re wrong, it will generally be no harm, no foul unless it becomes a pattern. It also directly gets the attention of those in a position to investigate the claim and act upon it. “Accusing” makes it sound like you left a comment; please don’t do that. It won’t likely be noticed by moderators without a flag, and your comment could get flagged as rude (CoC-violating).
