I’m not sure if its possible to add a custom flag reason, or have a flag reason that also allows user-written report text. That would be something the @admins would need to check. For now, I would suggest just using Something else and clearly articulating the basis for your suspicion and any obvious flaws in the post.
Well, ISTM that we’ve got most of the current regular contributors in that category here (you, me, Cameron, Chris, Barry, Quercus, Sinoroc, and Karl; @tjreedy and @rob42 are the two most frequent posters whose names are missing, though there are certainly others who post answers less frequently), and everyone I saw who expressed an opinion was in favor of either mandatory notification or a ban on ChatGPT answers. I tagged them to get their input, and we’re certainly not going to do anything quite yet before the discussion fully plays out.
Sorry, I should have been more specific—there seems to be a developing consensus against ChatGPT answers, at least those not identified as such. For questions, the use, abuse and harm seem less compelling.
I don’t think that I can say anything that has not already been said.
ChatGPT (and the likes of) are here and here to stay. I know that some institutions have embraced that while others are railing against it. If we do introduce a policy for such generated content, the issue is (and it’s already been pointed out) enforcement. There are tools that can detect C-GPT generated text, but can that be implemented here? I know that unformatted code detection was tried and it’s clearly failed to stop such content, which leads me to believe that it’s not so easy to do.
If we (as contributors) try to enforce a policy by flagging content that we believe is C-GPT generated, then what if we’re wrong? And won’t that increase the workload for those that have to review and take action on the flagged content? Do you have the time to do that? And do we (again as contributors) have the time to read and judge each post in each thread? I for one do not have that time and I’m selective about what I read and to what I respond.
Moving forward, these LLMs are only going to get better and as such, harder to detect (again, that’s been said), unless the developers of LLMs introduce some mechanism that allows for detection, which I think, ultimately, is what needs to happen.
Thanks all, so I take note that I should flag with “Something else” and describe in the free text field, if I suspect some contribution to be AI-generated (assuming the contribution is also a net negative for the discussion thread). I agree, the volume is way too low here so far, that it warrants having a specific choice for this in the “Flag Post” dialog.
Agree on all this. We can ban as a policy, but it does not seem to be technically enforceable upfront, so the ban will be on a case by case basis. Someone flags, a moderator decides if it is worth acting upon it or not.
If a contribution is clearly labelled as “AI-generated” or “AI-assisted”, I would probably not complain (as long as it is within reason). If we end up with a topic “Let’s study (and laugh at) this AI-generated Python code” then I am up for it as well.
If I recall correctly, there is some kind of “wizard” presented by the Discourse software powering this forum for new members. Well, I do not remember how the onboarding in general looks like, but maybe there is a way to place a notice in there to inform new members that AI-generated contributions are “banned” (or whatever exact policy is chosen).
Thus far, all of the replies within this thread are from users who are recognizable as individuals of high integrity. It appears that we all agree that within this forum, replies consisting wholly or primarily of content generated by large language models (LLMs) should be prohibited. There also seems to be a consensus here that such a ban would be difficult to enforce. For sure, all of us, as would nearly all other current users, would abide by such a prohibition. In terms of volume of generated posts, will this have the unintended consequence of favoring those who do not abide by the ban? That would be unfortunate, as it would constitute the opposite of the effect that we are trying to achieve. Such is one of the dangers of unenforceable regulations. Yet I do remain convinced that some form of the proposed ban needs to be instituted.
Just to note, I meant to mention it before but we do have one tool that can help enforce this, at least in the cases that are most concerning (and most match the profile of the new users who are likely to intentionally or not violate any such rule). Discourse automatically flags posts by new users if they type too fast, i.e. if they copy/paste a bunch of text in from somewhere else or otherwise aren’t actually typing it into the dialog like a human would. In some cases this happens, e.g. users pasting a long error message or code block in, but if a prose answer (as opposed to a code block) is copy/pasted, which is what folks are most concerned with here, this is a good sign of it being AI-generated or otherwise pasted from somewhere else.
Indeed, Discourse actually flagged the post I linked above as such and held it in the review queue, but another mod approved it before I saw it presumably not realizing it was AI-generated. Still, it permanently logs the fact it did so, and thus any mod reviewing a later report will see that if they check for it, which can help verify that the post was indeed taken from somewhere else at least. If this is missing cases, we can strengthen this by increasing the setting min_first_post_typing_time from the default 3000 ms to 5000 ms, 10000 ms or more.
It certainly wouldn’t catch an established user doing this, or one who extensively edited or added content to the original AI text, but these are inherently a lot less concerning from an enforcement perspective for the reasons discussed.
Within this forum, the most affected areas in the near future are likely to be the Python Help and Welcome to Discourse! categories, so this might not yet be a pressing issue where the Admins and other developers congregate. Since they are the ones who would need to institute any new policy, a nudge may be needed. Would it be a good step to call the attention of the Admins to this discussion, so that they may either consider possible actions, or so they and others may first offer their opinions here? If a calling-of-attention is appropriate at this time, I believe you, @CAM-Gerlach, would be a great ambassador for this.
I did bring it up previously on the core dev Discord, and all active admins/mods should presumably be watching this category, but for any others that aren’t… @staff , thoughts?
Thanks for that calling-of-attention to this discussion.
Where would the proposed policy change be announced? Perhaps it would be on one of the following pages, but there’s so much text on each one already, making it likely that the announcement would go largely unnoticed:
I find it interesting that “typing too fast” is a warning sign of
potential abuse, while interacting with the forum via SMTP (for
which it has no idea how fast I’m typing messages into my mail
client) isn’t. I suppose the assumption there is that people who
compose posts with a LLM are unlikely to be savvy enough to consider
that the forum might allow them to bypass such filters with it? Or
is the concern only over very new users to the forum composing posts
with an LLM, in which case their E-mail might also get flagged for
moderator attention similarly because they have very few posts?
Speaking not on behalf of all staff but as one member of staff, I would love it if we could ban posts made with ChatGPT and about code produced with ChatGPT. In answers and questions both, honestly.
However, the practical problem with that is that it’s not possible for us to reliably categorize content as AI-generated, unless the poster themselves disclose the origin of their post or code example.
There’s a risk of awkward interactions if non-native speakers get flagged or code examples by newbies get flagged. Therefore, unless somebody here suggests a decent way to differentiate between those cases and AI-generated content, I would have to say I’m not in favor of letting people flag content as AI-generated. In the end, those flags would have to be evaluated by moderators, and currently, it seems too arbitrary to judge one way or the other.
I find it interesting that “typing too fast” is a warning sign of
potential abuse, while interacting with the forum via SMTP (for
which it has no idea how fast I’m typing messages into my mail
client) isn’t.
I’d consider it a case of measurement where that measurement is
possible.
I suppose the assumption there is that people who
compose posts with a LLM are unlikely to be savvy enough to consider
that the forum might allow them to bypass such filters with it? Or
is the concern only over very new users to the forum composing posts
with an LLM, in which case their E-mail might also get flagged for
moderator attention similarly because they have very few posts?
I’ve not encountered this, so my opinion is uninformed, but I can
imagine a user new to this forum and any forum might eg copy/paste a
slab fro stackoverflow or whatever (eg their same question there)
without editing. I also don’t know if it differentiates between typing
(single characters arriving, as the compose/view pair is “live”) versus
a paste-of-text as one does into code fences.
I can well imagine very rapid single characters might be a hallmark of
some kind of robot.
The “typed too fast” filter only checks new users’ posts, and there are a number of other spam detection methods and measures used beyond just that which get applied to all post types.
Yeah, accurate detection is the main difficulty here, so especially as AIs improve we’re going to miss some false negatives. However, given the main concern is false positives (removing a post that is not AI on the grounds that it is), and given the tools available, we can be quite confident that if an answer was from a new user and was flagged by the “new user typed too quickly” filter, we know it must have been copy/pasted from somewhere else, and if it isn’t attributed and there isn’t any other reason for it (code blocks, etc) and it has the hallmarks of ChatGPT (or in some cases, clear giveaways like the post linked above), we can be confident that place was ChatGPT/AI.
I.e., the value of the policy is partially deterrence (we won’t catch everyone, but we can try to catch enough to deter others), and in allowing us to take action when we can be highly confident that the post was AI-generated.
When it comes to “user typed too fast” detection in particular, it’s been useful so far but it’s not bullet-proof. Even before GPTs there’s been examples of fully articulated posts flagged like this, which otherwise are perfectly innocent.
There are plausible reasons for this, too. Some users prefer to type outside of the browser so they have better “undo” control, or some particular workflow tool they depend on. One tool like that can be Google Translate, which, while not ideal, lets many of our community members interact with everybody else successfully. Some users copy&paste from other forums they posted on before. Etc. etc.
So, even here we have to be a little careful.
I think what would be uncontroversial would be to ask users in “pinned posts” and ToS to disclose usage of ML-based generators in their posts. Even if we cannot fully police it, it would set the tone.
Yes, I used chatgpt here and did some research since I thought it would be beneficial to him or her. If I had known that I shouldn’t have used ai here, I would not have done so.
Assume that people are looking for human generated content rather than LLM generated junk. If they were looking for ChatGPT, they would have gone to it, you don’t need to be a relay for that. Remember, LLMs don’t know anything, they just sound like they might to people who don’t know better. This advice is good for anywhere, not only this forum.
Thanks for letting us know about that, and just to be clear as far as we’re concerned, you’ve acted in good faith and didn’t do anything against the rules, since we didn’t have a policy prior to you posting it. If anything, it was a good thing that it was you that posted it, since it was bound to start happening at some point soon and most new users probably wouldn’t have gone out of their way to reply in this thread owning up to it. And as a new member of the community, your feedback about this proposed policy and how we might announce and enforce it would be quite useful
Thanks for your acknowledgment regarding the use of ChatGPT. We really appreciate that. How to operate amidst this emerging phenomenon of large language models, such as ChatGPT, is a learning experience for all of us.
If you don’t want to become the AI police, which is only going to get harder, you could also add language to the effect of
For AI-assisted answers, please make sure that they work and solve the problem. Otherwise, your post might be removed, and your account suspended.
The point being that nonsensical answers are the problem, not the manner in which they are contrived. AI just makes it very easy to produce wrong code that is hard to check even by experts.
As far as I can tell, AI-generated content is barely having an impact on this forum and the existing rules are working fine. The one example cited is not clearly deleterious IMO; in fact it seems rather helpful. What am I missing?
First from my point of view it is because when I come here I expect to discuss with humans. And second, for me the analogy with the search engine works well.
I can go consult a search engine myself, I do not need anyone else to do it for me, or at best I would take suggestions from other humans to help me input the right keywords into the search engine. And this, for me, applies equally to an AI content generator. See above: Community policy on AI-generated answers (e.g. ChatGPT) - #5 by pitrou
If in the example cited above there was a clear label “generated by this AI with the following prompt: …”, then we might be having a rather different discussion here in this thread. If someone posts something along the lines of “I asked [AI content generator brand name] and this was the output, which I found rather relevant and accurate: …”, then I guess I could be fine with that. At least we are all on the same page, and know on what basis we are discussing and debating. If there is an error in the presented solution, we know it is not human error, we know not to try to find what lead to the human error, and we know not to try and help the counterparty improve their (coding) skills.
Generally I do not feel like I want to discuss, reason, argue, or debate a topic without knowing that the counterparty only copy-pastes AI generated content. This feels degrading to me.
So yes, I anticipated a bit, the situation is really not bad here. As far as I know, this is the first occurrence, it had virtually no consequences (apart from this discussion, which is good to have regardless). I guess my take is that we do not necessarily need to ban AI-generated content, but I think there should be at the very least a mandatory disclaimer that it is AI-generated content (even for AI-generated translations, I think it would be healthier to expect a disclaimer).
