I think there’s a strong argument to be made that that informed discussion should have happened before these tools were dumped on the world, and should still happen before they are promulgated any further.
3 Likes
Unfortunately, the horse is out of the barn already. We have to play the hand the broligarchs dealt us. Besides, how is Sam Altman (or Zuck, or whoever) going to afford that next yacht/island/rocket?
Our deliberative/slow-and-contentioussclerotic governments were never going to run at the pace of technological advancement. As another — much smaller — example, consider the modern e-bike. There are a huge range of e-bikes available which all got plopped on the market in the US with essentially zero regulation. Individual cities are trying to come up with regulations which seem to be driven largely by the loudest voices at city council meetings. The states are far behind that. Forget the federal government.
6 Likes
Yeah, I went through a few options before converging on this as being probably best. Thanks for confirming.
I had a conversation with a colleague recently where they were explaining how ChatGPT helps them to write emails. It went like this:
Colleague: It’s great because I can put all the important information in a few bullet points and then ChatGPT writes a long fully formatted email…
Me: Why can’t you just send me the bullet points? Do you think I want to read a long email?
12 Likes
And @bwoodsend. When I (tell myself I) control the scope of env vars going in to the container in the compose file, I know env files all live in a system file somewhere, but do they end up “all over hidden corners of /var/lib/docker/” too?
I’m a hobbyist programmer too, but I’ve always been comfortable with using other people’s ideas and code (think using StackOverflow solutions, examples from literature, tips or even code from community members, PRs from people I don’t know, etc.). I’ve been extensively using LLMs (first Gemini, now Claude and Claude Code) in my latest projects and feeling bad about using code from LLMs for this reason, even though they’re not “people”, never went through my mind.
Feeling bad due to the externalities does happen, though.
One thing I believe we should do is some sort of cost/benefit analysis for each significant use of LLMs. I think that your branch cutting example and your decision is exactly that, and I expect everyone to have their own inputs and formulas, based on their values and knowledge, to make that analysis and naturally reach very different conclusions.
I wish people made those decisions while knowing about the impacts and dangers, and also knowing that the cost will always have a large societal component. The benefit can be purely individual, but hopefully some LLM usage will result in societal benefits too.
I try to follow these principles myself and IMO the projects I’ve been using LLMs on do have positive results for others. Are they enough to offset the costs? I think (and hope) so.
3 Likes
Mine either. Whether you think of them as “just tools” or (as I’ve come to view them) as “collaborators”, they’re sources of ideas I may have otherwise missed. So are countless authors, web pages, and millions of lines of open source code written by others.
But I’m not shy about it either. I have no “shame” in using any of those, and happily give credit to bots (and authors, and web pages, and other projects) from which I’ve taken ideas. It’s not the source that matters, it’s the results.
1 Like
Once you have a stable coding pattern for something, Agent are good at doing the grunt work of making similar objects. But, when it comes to the details of a particularly different algo within one of those objects, the agents can go off-the-rails and will require iterations of clear instruction and code-review. Sometime they say, “All done, all tests pass! 40 out of 40, 100% pass!”. Then you run the tests yourself.. 2 are failing.
Still, I find coding with Agents highly productive.
But you have to review the code and review the tests.
Slop is easily created by inexperienced undisciplined devs. Still, much more productive without AI, but potentially digging a hole they can climb out of.
2 Likes
We surely do not. We can just not use the tools, and most especially not pay for them.
2 Likes
We – you and me – do not, but there are many more companies and individuals who are clearly using this stuff. As a society, the horse has already left the barn.
4 Likes
I should have mentioned that I am in favour of AI use; that I call what they do thinking; that I consider their ideas, literature, images, videos, a creative act as much as those of the next human. Even more, I think they are, and not space colonization, the way humanity can survive in the very very long term.
The problems that I listed above are of a type that I describe to myself as Thin, Long, and Convoluted. Not properly defined terms.
- Thin, because small deviations from the solution are not a solution anymore.
- Long, in that they have infinitely many (or an intractably large) number of instances, such that it is not possible to fit all its solutions.
- Convoluted … I haven’t thought about it enough to have a definition. Constant answer, periodic answers are not convoluted. It should not be entirely trivial, a la Kolmogorov complexity, to answer all instances of the problem.
The problems (at least the last two types) that I gave as examples, I was expecting them to fail solving them. The first, while proving theorems in general is TLC, I was expecting the specific instance that solutions were available online.
My beefs:
-
The answers are nicely packaged; better language skills than mine, well-typeset formulas, able to bring up concepts and terminology with a breadth that only an expert would have, but then one needs to be alert for mistakes in places that don’t seem at par with all that knowledge, like adding integers wrong. I am being unfair, probably, but I am used to computers’ precision/reliability when solving problems for which we already have algorithms. They probably just need a mechanism to “lock in” into a problem for which we already have better solutions than a neural network, and send it through the corresponding logic-based library.
a. Is it fair to say that they have ulterior motives? Maybe not, but it is not the entire picture. They do have the motivation of whatever cost function or metric of deviation from the training data and the implied knowledge that from it it gets extrapolated. None of those are certificates of correctness for the problems that do have certificates of correctness.
There is some incentive in placing the gaps in reasoning in places that are as disguised as possible.
-
The problems for which we have simple enough algorithmic solutions, when being part of what the LLMs need to solve, are solved at multiple times the energy cost. Both theirs and ours. Talking to a customer service chat bot, he asked me for my date of birth or something that I could have typed in the phone number pad. Instead, they had to process the digitized sound wave of my voice the many times they didn’t understand what I said.
-
They bring consequences of this type
I am not saying that this specific instance was a consequence, only using it as example of the type. We have the tool, and with it the temptation of using it. In turn, when that happens often, this brings consequences for our own brain development. This time it is qualitative different from loosing mental arithmetic skills from using calculators. This time we can go to them for every reason that we can put in words.
I compare that with the style of some of the strongest problem solvers that I personally know. They like to try the problem for a while before researching the state of the art. Two reasons: Their own development and enjoyment, and that the state of partial ignorance has the potential to yield new ways to look at a problem.
-
It is not the only innocent purchase we make daily with these characteristics, and some are worse, but they are part of the accumulation of wealth by very unsavoury characters.
Euclid defined “Def. 1.1. A point is that which has no part.”, but now we see the benefits of structuralism and define points only by the properties that they exhibits in the structure that we want to study. For example, “by two points it passes only one line”, etc. One benefit is that whatever properties we find that the structure has, are also implied for anything we want to call points, as long as they behave the same.
If a bot acted the same way as a person who knows, that counts as knowing to me. When they act the same way as a person thinking, creating, feeling, that counts as thinking, drawing, feeling to me. Incidentally, if I believe that a person that is very clever and nice can have ulterior motives of which niceness is part of their toolbox, I should believe that too of the bot.
1 Like
Indeed. I realize Dario’s entire Adolesence Of Technology essay is long and may be read as dark because of the societal state of the world topics it covers. But the final concluding section “Humanity’s Test” (which i linked to via the anchor tag above so you can choose to skip the larger thing if you’re not up for a deep read right now) nicely sums up the situation we’re in as a world.
3 Likes
For you, don’t miss out on the opportunities here. Experienced
programmers will also tell you to /read/ lots and lots of code,
written by more experienced programmers.
Agree, but are those coding agents/LLMs really experienced
programmers? I guess the answer must be – an at least partial –
“yes”, if such a high-profile programmer like you is suggesting that.
Granted, I only have a superficial understanding about how LLMs
work, but I really have problems believing, that a “stochastic
parrot”, a probability-driven text-concatenator could ever be called
an experienced programmer.
A bot is a limitless, tireless, source of ideas to /explore/. It’s
never too busy for you, never annoyed by “too simple” questions,
never impatient, always willing to follow wherever you lead. Pick a
task. Say, binary search over a sorted list. That’s notoriously hard
for newbies to get right in all cases. Try it! Ask a bot to critique
Well, even if I’m just a completely auto-didactic hobby-programmer
(who never got really good at it), I guess I’m no newbie anymore,
not with 25+ years of experiences, but I get what you’re saying. 
You have 24/7 access to an infinitely patient tutor now. Use it. not
to write your code for you, but to learn from it. Or you could ask a
human expert instead - but they’d just brush you off
That’s the other thing. I’m highly social anxious and slightly
autistic. I don’t think, that it is wise (for me and for the society
in general) to replace social interactions with machine interactions
any further. And when asking in the appropriate place, being brushed
off isn’t what’s in my experience.
@devdanzin wrote:
I’m a hobbyist programmer too, but I’ve always been comfortable with using other people’s ideas and code (think using StackOverflow solutions, examples from literature, tips or even code from community members, PRs from people I don’t know, etc.).
Other people’s ideas and code, yes (although I always try to do it
myself before looking how others do it.) I realize that my
opinion/ethics in this regard is somewhat nebulous and inconsistent.
When I take code (or ideas) from people, I feel the obligation to
pay tribute by naming them as the source and hence highlight that
this was not my work or idea. I have no such feelings when taking
from an LLM.
Anyway, maybe I try LLMs again some day, but for now, I will program
“by hand” even if LLMs grow to prefect programmers. I just like the
thinking, the pondering, the head-scratching, the creativity, the
wonderful feeling you get when after hours and hours and days and
days of head-scratching, you finally got it working.
PS: I guess there’s no way to add a “Like” using E-Mail?
2 Likes
I’m not a compose user but I’m guessing you mean like this?
# docker-compose.yml
services:
app:
image: alpine
environment:
- TOKEN=verysecret
In which case, yes. It’ll be somewhere in /var/lib/docker/containers/*/config.v2.json.
/t/28146> docker compose up
/t/28146> docker container ls -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
2241bcef258e alpine "/bin/sh" 13 seconds ago Exited (0) 12 seconds ago 28146-app-1
# cat /var/lib/docker/containers/2241bcef258ef0f991e9c700ab915aaeee76e986a72dd3cc4ac6700ae068a565/config.v2.json
...
"Config": {
...
"Env": [
"TOKEN=verysecret",
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
],
...
(The container ID is the prefix of the directory that the config.v2.json lives in)
Likewise if you use .env file or vanilla docker run -e SECRET=value.
1 Like
This quote in the Dario Amodei essay @gpshead referenced surprised me.
Because AI is now writing much of the code at Anthropic, it is already substantially accelerating the rate of our progress in building the next generation of AI systems.
It suggests that they are, in fact, experienced programmers. They didn’t develop their skills over a decade of software development experience as a human programmer would, but they are experienced just the same.
2 Likes
I wondered when it would reach this forum.
If you want better writing than mine, ignore me and read this instead, since it captures my own feelings better than I could:
I’ve used Claude, and Copilot. I tried them last year and I’ve been coerced by my employer into using them this year. I’ve built, because I need health insurance, RAG-based conversational assistants backed by OpenAI models. I say this only because I fully expect someone to reply “but have you tried it though?”
I’m going to copy a comment from an Ars Technica article about experiences with “agentic” coding tools because I don’t have much to say differently here.
As a software engineer, what [this situation] and many of the comments reflect the most, to me, is the long-standing profound myopia of software developers and the tech industry more broadly. We continue to ignore our own societal context and the input of, seemingly, every other academic discipline outside computer science.
So often the analysis [in these articles and blogs] is along the lines of:
- Does this make my work easier?
- Is the output quality good enough for me?
- Will we be able to ship products faster?
- Do I enjoy the process like I used to?
Over time the answers to these have become “yes” for more of us. But this entire framing ignores that technology isn’t neutral and can’t be separated from the context in which it is developed, deployed, and sold.
No one in the tech sector can claim to be ignorant at this point. We all know about the theft, the exploited contract labor, the economic distortion, the ecological costs. We know how those things disproportionately impact minority communities. We all know these companies are run by liars, nazis, or zealots who think they’re building the Machine God. We all know what they’re doing to artists, to authors, to actors, to education. We know what it’s doing to news and information. We’ve all heard from the sociologists, cognitive scientists, historians, and psychologists.
But for some reason none of that is relevant once The Tools Are Good Enough For My Code. At least as long as we add a paragraph to our blog posts acknowledging the “potential ethical issues”.
So I’m glad you got to make some games, [author]. Maybe you get a pass here as a journalist - since apparently someone does need to engage with these products - for handing hundreds of dollars to these monsters and your voice to normalizing them. But the rest of us need to grow the fuck up and acknowledge that our actions have consequences beyond our own industry.
Our productivity, our enjoyment, does not matter and should not matter.
If you really, truly think that your personal productivity outweighs the societal costs of these tools, I don’t know what else to say. I don’t even know where to start a dialog from that.
If nothing else, all of you are giving money to people who think my partner should be killed for being queer and disabled. Please stop.
6 Likes
The ecological devestations may be real in the short term, but in the long term as the technology matures it is looking far more likely that it will more than make up for its great environmental costs with even greater amplification of human ingenuity and acceleration of human capacities to innovate so we can get to the next level on the Kardashev scale orders of magnitude quicker than without the technology. It’s highly likely that all the environmental sacrifices that help us get to that point will eventually bring an overall net positive benefit to the environment because of solutions it enables us to find.
And in the short term, I wouldn’t call the technology fraudulent either because although it still can’t yet reason through a large code base with production-quality outputs, the fact that it can help both coders and non-coders bring visions to life quickly by iterating through possibilities with working prototypes in less than 1/100 of conventional timeframes already proves is worth. At my work, a large backlog of small internal apps with too small a user base to be deemed economically viable are now getting completed with very little time and labor costs.
Half a year ago I too admittedly called AI-generated codes AI slops. Not today anymore. If only one is willing to learn to harness it correctly.
3 Likes
Are you joking? Are those innovations the ones wrecking higher education, or the ones filling academic journals with a tsunami of fake citations? Maybe you mean the medical advice that’s incorrect only half the time, or the automated transcripts with hallucinated text in them? I’m sure we can fix silly things like those by simply investing more money and resources than literally exist on earth into scaling the models up an asymptotic cliff, right?
Next will you say Tesla will deploy full self driving within the next six months?
We needed to cut emissions yesterday. People are being poisoned and displaced now. How convenient that you can excuse any and all harms in the present with an imaginary future for which there is no evidence. I have lost all my patience for this long-termist bullshit.
For anyone who cares, I can highly recommend the book “More Everything Forever” by Adam Becker.
6 Likes
Indeed. I’ve been expecting this topic to come up as well.
To put my own position out there: I don’t think these tools will ever work out to be net beneficial for our society. Look at all of the billions spent on this tech in the US alone, mostly to make number go up, and imagine if it were spent on healthcare and trains.
That said, I feel very strongly that we should keep these discussions not only “civil” but genuinely welcoming. Which is really difficult when some of us feel that not only creating but using this tech is wrong, and others of us rely on the current array of LLM businesses for salary and health insurance. I don’t know where we go from here as a software community, but I’d like it to be possible for us to stay a community at all – these are trying years for many of us but I’ve always felt that Python has provided one if the most inclusive communities in my life.
There are hard lines. No Nazis, for example. But I’d like the community to be broad enough to include people with different views on ethical issues, not just technical ones.
7 Likes
I wouldn’t be so quick to pass judgment to a work clearly still in progress. The accuracy of gen-AI output is improving at an astounding rate and in many areas it is already outperforming professional human experts. Remember that even human experts make mistakes too, so as long as AIs make mistakes at a rate on par with or lower than human experts they can provide meaningful value to practical works. And particularly in areas where there’s a severe shortage of human experts (such as radiologists), having help from replicable AI instances can be hugely beneficial.
There’s no point in approaching a new paradigm with old mental models. Higher education and the society as a whole can and will evolve and adapt to a new AI-native world.
1 Like