A very simple cli tool, consuming basic txt format. You can use it in a second window while waiting for your compilation to finish.
Recently I’ve been also experimenting with defining QA pairs in my note files (in a special section). I then use a custom function in emacs to extract these pairs and push to a file as well as Anki.
It’d be probably useful to include this very comment in your system prompt or a separate file which you ask the coding agent to read at the beginning of each session.
It’s great to read in the comments about experiences of others with vibe coding. But I also feel like lots of opinions are not coming from actual experience, or “serious” attempts at vibe coding, and more from theoretical deliberations. I might be wrong.
Here are some of my own high-level experiences / thoughts:
- Perhaps contrary to popular belief I think vibe coding will bring the best software / system architects. This is due to massively shortened feedback loop between architectural idea and seeing it in action, easiness with which it can be changed, and the ability to discuss it at any moment.
- We’re not really coding anymore. This is a new role, not a role of a senior dev reviewing PRs of junior devs. Devs are just best suited (currently) to take on this new role. I came to realization that if you’re reviewing all generated code in detail you’re doing it wrong. You just shifted bottleneck by one step. You’re still coding. You should skim if the code is in line with your high-level expectation and then make LLM maintain an architecture doc and other docs that describe what and how you’re building (this is the info you should know in detail). You can do audits with another LLM whether the implementation is 100% reflecting the docs, you can chat with LLM about implementation at any moment if you ever need. But you should not know the implementation the way you know it today. The implementation became the implementation detail. The whole challenge is to let go of the old and embrace and search for efficiency in the new setup.
- Connected to the above: reading through LLM outputs is a massive fatigue. You are exhausted after the day, because you read hundreds of pages. This is a challenge to fight. You cannot unlock full potential here if you aim at reading and reviewing everything.
- Vibe coding makes you work on the problem level much more. I never liked the phrase “ideas are cheap”. And now finally I think the tides will turn, ideas are and will be king.
- Devil is in the detail, 100%. People with ability to see connections, distill key insights, communicate and articulate clearly, think clearly, are the ones to benefit.
I implemented HRM for educational purposes and got good results for path finding. But then I started to do ablation experiments and came to the same conclusions as the ARC-AGI team (the HRM architecture itself didn’t play a big role): https://github.com/krychu/hrm
This was a bit unfortunate. I think there is something in the idea of latent space reasoning.
> The problem with c is that you must have a comprehensive dictionary in your brain with tons of corner cases to know what is or is not undefined in any given compiler setting.
The cases of undefined behavior in the C standard are independent of compiler settings or options.
> If C could have a consistent set of rules …
The C language has a well-defined standard, but the presence of undefined behavior is a deliberate aspect of that standard.
Using an LLM framework at this moment doesn’t make sense and can be damaging, in my humble opinion. Ways to extract value from LLMs are in early exploration stage. Look at research in prompting: chain of thought, react, reflection, tree of thoughts, zero vs few hot etc. Then completion vs conversational interfacing. Then memory management via vector databases and prompt expansion vs compression vs progressive summarization etc. All these are fairly recent developments. They are not abstractions worth cementing, this is search and creative phase. LLMs threw everything in the air, but the dust is far from settling. I think it’s important to recognize the phase we’re in and pick your weapon accordingly. You have to stay nimble and light, ready to experiment with a new idea that will come out next week. You should be hacking these things together by yourself. If you pick a framework at this stage know that the framework will have to pay the price of trying to cement things in the times of storm. And you’ll be a few steps behind. Of course this is my personal take.
> Look at research in prompting: chain of thought, react, reflection, tree of thoughts, zero vs few hot etc. Then completion vs conversational interfacing. Then memory management via vector databases and prompt expansion vs compression vs progressive summarization etc.
Wow, that's a list of things that's completely new to me - thanks! Do you have any particular resources that you'd recommend for learning about these different topics?
I didn’t think gp was recommending writing every thing from scratch, but rather advocating for more piecemeal tools that can be refactored or replaced, rather than buying into a whole framework. Those libraries you mentioned are much lighter weight. I like using guidance partially since it provides a unified interface to language models, but it’s not trying to do too other than managing prompts, output and iteration within prompts
I've looked at the documentation for both Microsoft Guidance and LMQL and they look like LangChain to me: frameworks where I'd have to spend a whole lot more time learning how to use them than if I just imported the OpenAI Python library directly and started running prompts through my own thin set of custom functions.
It's kind of strange to lump both of those (far simpler) "frameworks" (more like template standards) with LangChain. Indeed, they do seem to provide far more value than LangChain does.
A very simple cli tool, consuming basic txt format. You can use it in a second window while waiting for your compilation to finish.
Recently I’ve been also experimenting with defining QA pairs in my note files (in a special section). I then use a custom function in emacs to extract these pairs and push to a file as well as Anki.