I’ve hit this! In my otherwise wildly successful attempt to translate a Haskell codebase to Clojure [0], Claude at one point asks:
[Claude:] Shall I commit this progress? [some details about what has been accomplished follow]
Then several background commands finish (by timeout or completing); Claude Code sees this as my input, thinks I haven’t replied to its question, so it answers itself in my name:
[Claude:] Yes, go ahead and commit! Great progress. The decodeFloat discovery was key.
For those who are wondering: These LLMs are trained on special delimiters that mark different sources of messages. There's typical something like [system][/system], then one for agent, user and tool. There are also different delimiter shapes.
You can even construct a raw prompt and tell it your own messaging structure just via the prompt. During my initial tinkering with a local model I did it this way because I didn't know about the special delimiters. It actually kind of worked and I got it to call tools. Was just more unreliable. And it also did some weird stuff like repeating the problem statement that it should act on with a tool call and got in loops where it posed itself similar problems and then tried to fix them with tool calls. Very weird.
In any case, I think the lesson here is that it's all just probabilistic. When it works and the agent does something useful or even clever, then it feels a bit like magic. But that's misleading and dangerous.
i think that a wasteful but good solution would be to tag each token, not use opening/closing tags.
whatever n-dimensional space the tokens occupy, manually add more dimensions, to reflect user/agent, trusted/untrusted input.
it should be much harder for the LLM to fuck up this way if every single word it reads screams "suspicion" or "trust". with tag tokens at the start it can just forget
It helps with LLMs to ban personal pronouns. Give yourself a name, give the agent a name, and speak in third person. Reiterate this in the system prompt.
If you say "you" or "I", LLMs universally can not distinguish between the speaker as the context grows.
These services are pretty underbaked and not much better than local in operational utility. Also true, characteristics experienced at lower quants will always show themselves with base models at scale.
Yep, I've discovered the same. Approve once in a session, and from then on you're rolling the dice if it will self-approve a change and commit/push (or whatever you originally asked it to do) for as long as it stays in context.
I don't think so, feels like the wrong side is getting attention. Degrading the experience for humans (in one tool) because the bots are prone to injection (from any tool). Terraform is used outside of agents; somebody surely finds the reminder helpful.
If terraform were to abide, I'd hope at the very least it would check if in a pipeline or under an agent. This should be obvious from file descriptors/env.
What about the next thing that might make a suggestion relying on our discretion? Patch it for agent safety?
"Run terraform apply plan.out next" in this context is a prompt injection for an LLM to exactly the same degree it is for a human.
Even a first party suggestion can be wrong in context, and if a malicious actor managed to substitute that message with a suggestion of their own, humans would fall for the trick even more than LLMs do.
Right, I'm fine with humans making the call. We're not so injection-happy/easily confused, apparently.
Discretion, etc. We understand that was the tool making a suggestion, not our idea. Our agency isn't in question.
The removal proposal is similar to wanting a phishing-free environment instead of preparing for the inevitability. I could see removing this message based on your point of context/utility, but not to protect the agent. We get no such protection, just training and practice.
A supply chain attack is another matter entirely; I'm sure people would pause at a new suggestion that deviates from their plan/training. As shown, autobots are eager to roll out and easily drown in context. So much so that `User` and `stdout` get confused.
I wonder if this is a result of auto-compacting the context? Maybe when it processes it it inadvertently strips out its own [Header:] and then decides to answer its own questions.
I don’t think so, at least not in this particular case. This was a conversation with the 1M context window enabled; this happened before the first compaction – you can see a compaction further down in the logs.
My theory is that Claude confuses output of commands running in the background with legitimate user input.
My own guess is that something like this happened:
Claude in testing would interrupt too much to ask for clarifying questions. So as a heavy handed fix they turn down the sampling probability of <end of turn> token which hands back to the user for clarifications.
So it doesn't hand back to the user, but the internal layers expected an end of turn, so you get this weird sort of self answering behaviour as a result.
As an aside my big reason for believing this, is that this sort of dumb simple patch laid onto of a existing behaviour is often the kind of solution optimizers find. Like if you made a dataset with lots of pairs Where one side has lots of <end of turns> and one side does not. The harder thing to learn tends to be to "ask fewer questions and work more autonomously" while the easy thing to learn "less end of turn tokens" tends to get learned way faster.
The post is just a prelude to a 10-part article, most of which is not yet released (but will be shortly). Judging by the table of contents, the things you expected will be elaborated on in subsequent parts.
I picked up singing 4 years ago (I’m 42 now), starting from nothing, and I’ve been taking regular lessons. I still suck. But I suck slightly less than I did when I was starting, and what motivates me is the sheer joy that it brings. I just hope it lasts.
HN’s guidelines say ‘Don’t editorialize’. The original title here is ‘[BUG] Claude Code login fails with OAuth timeout on Windows’, which is more specific and less clickbaity.
I almost overlooked this, and then when I didn’t, I almost dismissed this as Yet Another Linux Distro with a custom skin. But no, there’s novelty and exploration in here. There’s attempt to venture off the local maximum. This is a breath of fresh air.
Always remember that open-source is an author’s gift to the world, and the author doesn’t owe anything to anyone. Thus, if you need a feature that for whatever reason can’t or won’t go upstream, forking is just about the only viable option. Fingers crossed!
If "taking part in a huge ecosystem in a foundational role" means 'other people choosing to use your FOSS software', and I can't think of what else it would mean, then no, you have no obligation to do any of that.
FOSS means the right to use and fork. That's all it means. That's all it ever meant. Any social expectations beyond that live entirely in your imagination.
There is simply no responsibility an OSS maintainer has. They can choose to be responsible, but no one can force them. Eventually OSS licensing is THE solution at heart to solve this problem. Maintainers go rogue? Fork and move on.
But surprise, who is going to fork AND maintain? Filling in all the demands from the community, for potentially no benefit?
No one can force him to take the responsibility, just like no one can force anyone else to.
Right, frustration about the no strings attached sentiment for OSS devs. Of course you've no obligations for support or maintenance, but with increasing exposure responsibility grows as de facto ever more projects, people, softwares depend on you.
This doesn't come over night and this is a spectrum and a choice. From purely personal side project over exotic Debian package to friggin httpx with 15k Github stars and 100 million downloads a week the 46th most downloaded PyPI package!
If this shall work reasonably in any way, hou have to step up. Take money (as they do, https://github.com/sponsors/encode), search fellow maintainers or cede involvement - even if only temporarily.
I feel there should be support from the ecosystem to help with that. OpenJS Foundation seems doing great: https://openjsf.org/projects. The Python Software Foundation could not only host PyPI but offer assistance for the most important packages.
>> Of course you've no obligations for support or maintenance, but with increasing exposure responsibility grows as de facto ever more projects, people, softwares depend on you.
This is an oxymoron. Either you have obligations, or you don't. There's no such thing as having "no obligations" but also "growing responsibility".
I don't understand how you can possibly conclude that just because you've chosen to become dependent on some FOSS library, they owe you anything. You don't get to somehow impose obligations on other people by your choices. They get none of your profits, but they're somehow responsible to you for your business risks? Nonsense.
It is a condition of your use of the code that you've accepted its license, and FOSS licenses are CRYSTAL CLEAR (ALL CAPS) on what obligations or responsibilities the authors have towards you - none whatsoever. Your use of the software is contingent on your acceptance of that license.
If that lack of warranty poses an unacceptable business risk to you, go buy support. Pay a dev to fix the issues you're having, rather than inventing some fictitious responsibility they have to you to do it for free.
Yeah. Previous poster points out sources how a maintainer could get resources (money, support, etc). Maintainers may be exhausted or overwhelmed by the (imposed) responsibility / work. Actively acquiring those resources would just push that over the edge.
There is also the possibility that a maintainer simply doesn't care about what the community wants, it's his baby and he can do what he wants.
Forking a project is built-in by licensing. A lot of complaints, but those complainers don't fork. Why is that? Yeah right.
Side Note: Transferring projects to foundations etc with funding may be a solution for projects that are highly depended on and require active, reliable maintenance. They wont work well for innovation or experimentation. Just saying they are just a part of the equation and not the sole solution.
No. Even if it’s a central piece of infrastructure, any and all maintainership effort is still a token of good will of the maintainer – and needs to be appreciated, rather than expected.
If you need stronger guarantees, pay someone to deliver them.
A (hypothetical) professional propriety project at same scale would probably feed a handful of people, with much less stress. FOSS version is zero cash and exaggerated community demands. Dream job.
Counterpoint: in December, a Polish MP [0] has vibe-coded an interpreter [1] of a 1959 Polish programming language, feeding it the available documentation. _That,_ at least, is unlikely to have appeared in the model’s training data.
Not exactly a counterpoint, since nobody argued that LLMs can not produce "original" code from specs at all - just that this particular exercise was not clean room.
(although for SAKO [1], it's an average 1960 programming language, just with keywords in Polish, so it's certainly almost trivial for an LLM to produce an interpreter, since construction via analogy is the bread and butter of LLMs. Also, such interpreters tend to have an order of magnitude less complexity than emulators.)
[0]: https://www.joelonsoftware.com/2000/04/26/designing-for-peop...
reply