You have no idea how much your message means to me, so thank you!
If you’re a personal friend of @Rosuav, please tell him or her that I’m sorry if I’ve offended or tired him or her by asking so many questions. I’m learning to be polite and ask first if it’s okay if I have more questions.
I have more respect for all of you, so thank you again.
It’s okay, I’m not offended. I just stop responding in threads that aren’t going to be productive.
Its not just ChatGPT that is in the running.
Open source AI is already matching it according to a leaked memo from Google: Google "We Have No Moat, And Neither Does OpenAI"
FWIW, of interest to this thread—it seems that the previous ban on LLM (ChatGPT)-generated content on the Stack Exchange network has been mostly reversed by order of SE management (perhaps in a vain attempt to partially reverse the major drop in site traffic due to ChatGPT, or due to the company’s recent investments in generative AI for its enterprise product), and moderators are now generally prohibited from removing LLM-generated content on such grounds.
The previous measures banning LLM-generated content seemed to be strongly supported by most of the platform’s active members, and the reversal has prompted huge backlash especially from the site’s volunteer moderators against new the ban on removing ChatGPT-generated posts, to the point where there is now a mass network-wide strike of SE mods for the first time I am aware of in the platform’s history, and many of the individual communities have outright refused to re-allow ChatGPT content, to the approval of most people there.
I don’t have the time to follow those links, and what you wrote is so full of double negatives that I don’t understand what’s happening. Is there still a ban or not?
@CAM-Gerlach, would it be accurate to state that the the gist of your most recent post is that on some platforms, moderators are being ordered, against their will, to allow posts to remain intact that include content generated by LLMs?
It has been particularly concerning to see a thread initiated by a new user on this forum entitled Jython isn’t executing some valid Python 2 code, and I’ve interpreted that to mean that the Python trademark is invalid, which cites legal advice from an AI in an effort to challenge Python’s trademark. If we wish to discourage the use of LLM content here, perhaps we should prepare now to notify new users upon their arrival that they should not use such content.
What official stated policy is foreseen on Python Discourse regarding the use of content generated by LLMs?
No. Stack Exchange management lifted the ban, but we don’t know details, since the instructions they gave to moderators are not public.
Volunteer moderators, feeling mistreated, are now on strike.
I honestly don’t see the problem with quoting “legal advice” from ChatGPT, given that the poster used blockquote markup and prefixed it with “this is what ChatGPT said”. (And IIRC they even drew the conclusion “talk to your lawyer”.) That’s no different to me than saying “here’s what Google told me”.
I continue having a problem with anyone posting LLM-generated text as if they wrote it themselves. But I could see an exception for cases where they did the research and found they agreed with the generated text (similarly to how you’d trust a a spelling or grammar checker). I personally wouldn’t use ChatGPT for legal advice, but I’ve used it to remind me how to cook a certain dish (being no great chef myself).
I wonder if eventually we’ll reach a stage where we just shake our heads and move on, just like we do when we see a “security issue” reported by some “script kiddie”.
Sorry, I finished writing that in a bit too much of a rush before a series of meetings. I edited it to hopefully make it substantially clearer. There are still some double-negatives, as they are somewhat implicit in the situation that is being described—SE specifically banned of all SE communities moderators from banning ChatGPT-generated content.
In this case, I would tend to agree with the latter position—while it’s certainly a bit concerning to rely on ChatGPT for this, at least without carefully cross-checking its citations and confirming it with an actual lawyer, its not that much different from the perspective of the present discussion then if the user had cited a summary of that particular generally issue from any other source.
In that context ChatGPT was used as a source of background information and clearly cited as such, not to write the actual direct content of the user’s post here as is the issue here. And furthermore, the actual response was a general summary of the issue as one could have easily obtained from any other website, which the person used to then support their own (inaccurate) conclusions, rather than actually (say) asking ChatGPT directly whether the Python trademark is valid, and the LLM itself stating the conclusion that it wasn’t—the latter would be an actual problem specific to a generative LLM.
Let’s just hope their lawyer isn’t the guy in this video…
(TL;DR: Lawyer uses ChatGPT to write a chunk of his legal arguments for a case, ChatGPT makes up a bunch of non-existent cases, and now lawyer is facing sanctions for it by the judge. His response: that he “has never utilized Chat GPT [sic] as a source for conducting legal research prior to this occurrence and therefore was unaware of the possibility that its content could be false.” Yes, that’s an actual quote from his legal filing.)
My understanding was that the median consensus had roughly converged on, for now, requiring users to disclose their sources for any substantial amount of content in their posts that they did not write themselves, and to do due diligence to check that content for basic accuracy, which would cover both LLMs and plagiarism from other sources, which interestingly enough was exactly what the user who’s post initially prompted this thread ended up doing as a spam tactic and had to be banned for it.
However, I’d written a draft of such a policy above and it attracted some feedback, but I was urged to wait for further discussion before going ahead and implementing anything, particularly given some forum members were opposed to allowing use of LLMs at all.
To (try to) clarify about Stack Overflow.
First, some background: the Stack Exchange network is operated (as in, the servers are kept running) by a company, Stack Exchange Inc. In 2021 the Stack Overflow site itself was sold to Prosus - which in turn has Naspers as its parent. However, the Stack Exchange Inc. company appears to remain, and own all the other sites (although Stack Overflow dwarfs them all put together, to my understanding).
It was never a network-wide policy to ban ChatGPT content; individual sites - Stack Overflow as well as each Stack Exchange site - discussed it in their own meta spaces, and many of them decided to ban the content (following Stack Overflow’s lead). Each site has its own volunteer moderator team, who are expected to liaise with “community managers” and staff employed by Stack Exchange, Inc. While they operate mostly independently, they are apparently bound by some form of moderator agreement. (Moderators who won’t abide by it can get replaced; but replacement is a community election process, so one can imagine the potential for larger-scale future conflicts…)
In theory, the buyout of Stack Overflow was supposed to allow for independent operation. However, Jeff Atwood left the company over a decade ago, while Joel Spolsky stepped down from the CEO role. Supposedly he is still “chairman”, but he is not listed on the current “company leadership” page.
Meanwhile, the new CEO, Prashanth Chandrasekar, has been active on meta.stackexchange.com lately, as well as the company blog, generally promoting AI concepts and suggesting ways they might “enhance” the site going forward. The company has also been working with Microsoft, which likely also informs policy.
In theory, the ban on ChatGPT content is still in effect on Stack Overflow and across the Stack Exchange sites that were already banning it. There was never a formal “ban on the ban”.
However, the ban on ChatGPT content is not enforceable for two reasons:
Stack Exchange, Inc. has updated the moderator agreement such that moderators are - per their description - effectively disallowed to suspend anyone for using ChatGPT, unless they admit to doing so. (If I understood it properly, they may also not remove this content effectively unless the author admits to it.) Essentially, the company argues this is an abuse of moderator power and that potential false positives (for which they have shown no real evidence) are not worth it.
In particular, the Stack Overflow moderators were chastised (falsely) for relying on detection tools to identify ChatGPT content, which the company claims (unclear whether Spolsky agrees, or can do anything about it) are not accurate enough for the purpose.
The exact moderator agreement is kept secret; we cannot confirm the details unless it is leaked (or the company is convinced to publish it). Moderators have claimed (per discussion at @CAM-Gerlach 's first link) that the company has also misrepresented that policy/guidance in the original post on Meta.
Because of the first point (along with worsening conditions resulting from stuff mentioned in the background section), moderators are on strike anyway.
The whole network was sold. The legal name is Stack Exchange Inc. but the brand is Stack Overflow. A comment from a staff member in the first link:
“Stack Overflow” is the publicly-used name of the company - the entire company was purchased, not only stackoverflow.com.
See also Did Prosus acquire just Stack Overflow or the whole Stack Exchange Network? - Meta Stack Exchange
Reviving this because I noticed a related problem: the spammers will use ChatGPT to try to gain legitimacy before posting spam. There’s a classic pattern in this thread.
A new user shows up and posts a vague, poorly written question - totally normal stuff, maybe not what we’d like to see but certainly not suspicious. The username looks like a throwaway, but plenty of people do that sort of thing for various reasons, fine.
A week later after no other attention, another new user shows up, “likes” the OP, and pastes in an obvious, lazy ChatGPT answer. Okay, so we can talk to that user about ChatGPT, sure. The new user also has a throwaway name.
Then after some other engagement from the regulars, the ChatGPT-using user just straight up posts a spam link. Oh. Well, then. In retrospect it feels like all the signs were there and the whole effort was coordinated.
In cases like this it seems worth checking if the two new user accounts are from the same, or a similar IP. I would also propose that ChatGPT content as the first reply on a post that was initially ignored by the regulars, should be treated with heightened suspicion.
Btw, what’s the value of this strategy? Is it to bypass some kind of discourse filter that doesn’t let users with no previous interaction post links?
Having read that thread I don’t see any signs this was coordinated? If your strategy needs low quality questions on technical help forums with no interactions they are already an abundant resource.
P.S. seems like the thread has been cleaned up between my starting to write this and posting it.
Possibly. They don’t seem to be trying to build community trust or sneak the spam links in with actual useful content (that takes way too much effort). I guess that as long as a significant number of Discourse sites use such a filter, the spammers will take a strategy of automatically trying to work around it, rather than putting in the effort to check whether it’s needed.
True. I did find it suspicious that a throwaway name responded to another throwaway name, but I guess there are enough of those, too. And of course a spammer can “like” the OP without the OP having anything to do with it.
Thanks for bringing this up. Funny enough, despite their apparent cooperation at first, the original user who’s ChatGPT answer prompted this thread actually ended up being exactly that, as they then switched to a pattern of plagiarizing Stack Overflow questions and hiding links to their spam site within them, which quickly got them blocked as a spammer once the pattern was spotted (which I believe either you or Chris here originally brought to our attention).
I went ahead and blocked the offending spammer and their username/IP, along with removing their spam posts.
Hmm, that’s definitely a spam pattern I’ve commonly seen on a number of other platforms, e.g. YouTube, Reddit, Q&A sites, etc. However, digging deeper, there doesn’t appear to be clear evidence that is the case here, as the two user accounts were created at different dates, have different IPs from very different countries, different emails, and nothing else that would appear to connect them.
The only potential point of interest is both had their first posts flagged for “typing too fast” (i.e. copy/pasting text from another source or using a bot), though that filter is rather crude and can have a number of false positives, particularly on very short text like in this case (since its a simple flat time check) or if it was copy/pasted for other reasons.
It seems a rather plausible alternative that given the number of new questions asked here and the fact that there is no need for a particular question to be asked (that would lead into the spam, as in many other cases), spammers could simply pick any suitable existing one and reply to that, without needing to create another fake account and have it post its own questions.
There are a number of filters and antispam checks that only apply to a user’s first post, a bunch of limitations that posting a couple times, getting a few likes and otherwise interacting a bit with the site will remove (including with posting links) via going from TL0 (New User) to TL1 (Basic user), and users with a higher TL need more flags by higher-ranked users to be auto-hidden. Thus, this is a not-uncommon phenomenon.
The pattern that Karl discovered is really beyond the pale - I wasn’t even aware spammers were targetting forums like this. Doesn’t this also pose extra security risks for regular users?
Spammers have targeted forums since the days when PHPBB was dominant, if not longer.
I just flagged two spam posts in quick succession that were blatant ChatGPT responses to previous content of the thread, with a spam link to some Python course at the bottom. I guess a useful heuristic is “the user’s first post is long, contains a URL and is not an OP”.
Of course, there are much more sophisticated filters and blacklists out there. It’s worth looking in to how Charcoal works for Stack Exchange.
One thing worth considering is perhaps the spammers are simply testing multiple ways to get their links clicked or to rise higher in search engine results. We may only be viewing one thing they’re trying, when over on some other Discourse-hosted site they’re trying a different approach. This would be akin to Facebook running experiments to see which of multiple approaches get users to engage more(and thus view more ads).
Just a thought…
Yeah, unfortunately spam is pretty pervasive everywhere, including here especially lately. As far as I can see, we typically get multiple spam posts per day, some really obvious (male enhancement, drug-filled gummies and various technology, marketing and similar services appear to be some of the main recent, or not so recent trends), and others a good deal more subtle and requiring knowlege of relatively subtle patterns. Most are caught immediately by the filters, and only a small proportion of the much more subtle and “long con” ones make it through to be seen and reported by the general membership. And a hopefully small but likely still non-trivial proportion may never get reported at all, at least until the user involved is detected and blocked. Its not quite the volume we get on, e.g., a large popular subreddit, but its actually not that far off.
But I digress, sorry…
GitHub has today updated their code of conduct regarding generative AI in community discussions:
The code of conduct: GitHub Community Code of Conduct - GitHub Docs
Announcement: Code of Conduct Update: Generative AI Content · community · Discussion #69006 · GitHub
TL;DR: While we encourage non-AI answers to questions in the community, we also love experimenting and using new technologies. If you wish to use AI generated content when answering questions in the community, you must do so by the following rules:
- Take personal responsibility for everything you post.
- Read and revise the content before you post it; use your own authentic voice.
- Use your expertise as a developer to verify that the answer works and makes sense.
- Do not just post AI-generated content verbatim to inflate your reputation or give a false impression of product expertise.
- AI tools will often answer in an authoritative tone that sounds like a tech support professional. Be careful not to mislead other users into thinking that this authoritative tone means they are receiving an official response from GitHub.
Additionally, all of the guidelines listed in Contribute in a Positive Way also apply here.
Failure to adhere to these guidelines could result in the following:
- Content Removal
- Content Blocking
- Community access temporarily or permanently blocked
- GitHub Account Suspension
- GitHub Account Termination