> the model has to re-run the generation, and the client has to start receiving tokens from scratch again.
I don't understand. The payload can be designed to have sequence number. In case of reconnect, send the last known sequence number. Sounds like a application level protocol problem and not transport. Am I missing something?
The pub/sub mentioned in the article essentially does the same thing.
The blog author is confusing SSE the protocol itself, with how the application is typically implemented. SSE is great and can trivially be implemented in a way that allows history, catch-up, and resuming. The "Pub/Sub" mentioned at the end of the exact application of SSE that the author wants.
Exactly, I use ADK (https://google.github.io/adk-docs/runtime/) and it handles all that bookkeeping for me. If my client disconnects, the engine continues to run until the turn is up, recording all events.
We need more frameworks like ADK that handle the bookkeeping and give us great abstractions for building agentic systems.
If you don't have a proper grasp of what is the transport, what the is the protocol and what is your application protocol, I think chucking in libraries to try to help often makes things too complicated. You still would need to understand the differences and nuances.
yup, I have recently started saying "building blocks over batteries included"
particularly as it comes to people trying to sell me on their Agent "framework", which amounts to little more than some well built tools and prompts, but pigeon holes me into how they think about solving certain issues in the agentic space, based on how things work today. If I go out 2 years, do I have to wait for the "framework" to realize their ideas are now out-of-touch and wait for them to course correct, or have I selected a framework that allows me to easily experiment, evaluate, and adjust any technique, with an ecosystem of building blocks for both the provider and user side of what I am building
> The blog author is confusing SSE the protocol itself, with how the application is typically implemented
Yeah, pretty common misunderstand among us self-taught developers who at one point never came across things like the OSI model (https://en.wikipedia.org/wiki/OSI_model) or similar before, that we confuse what layer things actually happens at.
What you're doing is fine for a homelab, or learning. But barring any very specific reason other than just not liking Kafka, its bad. The second that pattern needs to be fanned out to support even 50+ producers/consumers, the overhead and complexity needed to manage already-solved problems becomes a very bad design choice.
Kafka already solves this problem and gives me message durability, near infinite scale out, sharding, delivery guarantees, etc out of the box. I do not care to develop, reshard databases or production-alize this myself.
Some people don't and won't need 50+ producers/consumers for a long while, if ever. Rewriting the code at that point may be less costly than operating Kafka in the interim. Kafka is also has a higher potential for failure than sqlite.
Ofc, and not everybody needs or cares for all the features Kafka has. Then use another known and tested messaging system. Use NATS or ZMQ. Or any cloud native pubsub system
My main point is, I have zero interest in creating novel solutions to a solved problem. It just artificially increases the complexity of my work and the learning curve for contributors.
Have a table level seqno as monotonically increasing number stamped for every mutation. When a subscriber connects it asks for rows > Subscriber's seqno-last-handled.
True but the webdev idiom is injecting things such as mathjax from a cdn. I guess one can pre-render the page and save that, but that's kind of like a PDF already
To add, (in UK atleast), the parking space in some spots could be really small [1]. I have seen bulky cars parked leaving no room for neighboring slots.
The ability to ask "stupid" questions without feeling embarrassed is a superpower IMO.
I had the sweetest manager once. Someone stared talking about iphone and she [1] casually asked "what is iphone?" (this is after 6-7 months after iPhone was launched). Everyone's jaw dropped ... what? In which world u live in? ... to which she said with a wide smile and not an ounce of embarrassment .. "what? I don't know what iphone is?"
But she was otherwise so good in every other aspect ...
[1] She is/was mother of 4 kids and that left her very little time for anything else.
Once I got over the embarrassment hurdle of asking “dumb” questions, I grew a lot in my early career. Then people saw me as highly engaged, and my questions and understanding got better over time.
In particular, I encourage all the new joiners on my team to play the “newbie” card to allow them to ask as many questions as they want.
“Hey, new guy here… what is X?”
I even tell them to set a goal of one question per day if it helps.
I think it’s so important not to be passive because you absorb / understand less that way.
After 2 months working at a new department, i awkwardly asked my boss what the department acronym stood for. He stared at me confused, and couldn't figure it out; hes worked there for 30 years.
Technology isn't intuitive and there ares loads of things to remember. Sometimes the dumb question is needed by more than the person that asked.
[1] https://developer.mozilla.org/en-US/docs/Web/API/Server-sent...
reply