Same thing I'm seeing, all the "AI practitioners" at my company with their advanced workflows are just shipping mountains of slop, and end up either putting the actual work on the reviewers, or the poor soul that's on call when an incident occurs.
I feel like people that have built crazy AI workflows have developed a false sense of confidence that their guardrails are helping them ship clean/correct code with little review when it isn't the case at all. In reality, the models and harnesses are at a point where there's very little difference as long as your prompts are somewhat reasonable, and the quality of the code ultimately comes down to the level of care and effort the implementor puts into it.
I don't think the first people that are going to be replaced by AI are going to be the people who don't use it extensively. The first that will be replaced are going to be those that are using AI mindlessly, because at that point, what are you besides a very expensive human LLM interface? To be clear, I'm not "anti-AI", I use AI quite extensively (in a way that's similar to what's described in the article), I just think that it's being pushed in a completely unsustainable way and the industry is in a collective psychosis over it's capabilities.
> The first that will be replaced are going to be those that are using AI mindlessly, because at that point, what are you besides a very expensive human LLM interface?
I think this archetype has a good chance of surviving. Not because of merit, but because they will be the only ones able and willing to work on projects taken over by AI slop.
I'm very much aligned with everything else you said.
> Great! So next time the human will prompt the agent to watch out for and avoid this bug.
I actually created a system for something like this. The basic idea is, once you have identified what the issue was and fixed it, you can create lessons that lives inside the repository. Lessons are designed to be mapped to one or more files so if the LLM changes the files again, they can see what the issue was.
The main challenge is being able to summarize and create proper tags so the AI after any code change can easily find the lesson.
A singular human does (or tends to). Humans as a group, where members join and leave a group with time, also do learn, but at a much slower pace - over the years to decades timeframe. "X things programmers should know about Y" is a template for quite a few very influential blog posts, yet for most of them, you find many programmers, even decades later, who don't actually know what they "should".
My experience was always that 90% of code is ugly and clunky. I'm not at all surprised, while reviewing AI-generated code, to see many of the same ugliness we regularly commit. The quality of the output code is now consistently average, which means it's basically shit in 90% of cases, but it tends to mostly work (in the general case). The same kind of shit I've seen people push to production thousands of times in my career.
We don't fully know how to write good code. We don't really understand what good code should objectively look like. Spending more time on code doesn't automatically lead to better code (but costs a lot more). Above all, we don't need good code - the business side is perfectly fine with "good enough right now" rather than "maybe a lot better half a year from now". And that's what the models are trained on. They would, indeed, need quite a lot of "emergent properties" to go from that to consistently good code. ASI-level properties, I suspect.
OpenAI developers presumably get paid far more than average, so I'd expect better from them, and maybe I'm overrating the average developer, but I can't see a team of average developers struggling with the multitude of ridiculous bugs like OpenAI/Anthropic have for software that amounts to a CLI wrapper making API calls.
It's not so simple, especially in the AI slop era. Reviewers are already flooded because people of varying skill opening PRs at a rapid rate and it's not practical to be able to dive deep into every line. It's up to the implementor to take some responsibility and make sure things are correct.
Because of the AI lunacy going on at most companies, if you call any of this out you get labelled a luddite and are put on top of the layoff list, so at least for now it's just something you have to deal with.
Part of it is the ridiculous fear-based marketing campaign spearheaded by Anthropic et al. about how AI will automate all jobs and you'll be left behind. Works for enterprise/CEOs, predictably makes regular people hate the mere mention of AI, Asian countries don't have this nearly to the same extent.
Well most of it not really fear mongering, more like Americans are slowly waking up to the realization that consolidating the nations wealth to a few dozen people is not healthy for the country nor democracy.
Especially when you see what big tech spends on LLMs per year while Americans lack medicare for all, universal childcare, or free community college.
People have long hated the leadership of big tech well before ChatGPT, let's not act like this is all new and shocking. More like the naked power grabs are filling people with righteous digust.
Most people I've spoken to in private hate AI in tech too, they just keep quiet out of fear for their job (voice any objection to AI? next on the chopping block), so you only hear the pro AI voices.
I was trying to go against the tide for the longest time by providing detailed reviews, understanding every line of code, leave meaningful comments, improve architecture, etc.. Then management started pushing AI more and more and explicitly called out PR reviews as a bottleneck, timelines shortened, and more and more slop got pushed.
I gave up and I'm now a happy "AI enthusiast" at my company, handing out AI slop reviews for AI slop PRs. Deep down, I don't care anymore, if that's what they want, that's what they'll get, and it's no longer my problem if stuff leaks through that brings down prod or worse. Oh, and I'm also in line for a promotion this coming quarter thanks to my new found "velocity".
> I was trying to go against the tide for the longest time by providing detailed reviews, understanding every line of code, leave meaningful comments, improve architecture, etc..
I tried that too, until I realized the people I was supposed to mentor take my comment, feed it to the LLM, and let it make the fix.
I feel like people that have built crazy AI workflows have developed a false sense of confidence that their guardrails are helping them ship clean/correct code with little review when it isn't the case at all. In reality, the models and harnesses are at a point where there's very little difference as long as your prompts are somewhat reasonable, and the quality of the code ultimately comes down to the level of care and effort the implementor puts into it.
I don't think the first people that are going to be replaced by AI are going to be the people who don't use it extensively. The first that will be replaced are going to be those that are using AI mindlessly, because at that point, what are you besides a very expensive human LLM interface? To be clear, I'm not "anti-AI", I use AI quite extensively (in a way that's similar to what's described in the article), I just think that it's being pushed in a completely unsustainable way and the industry is in a collective psychosis over it's capabilities.