Kafka is really not intended to improve on this. Instead, it's intended for very high-volume ETL processing, where a classical message queue delivering records would spend too much time on locking. Kafka is hot-rodding the message queue design and removing guard rails to get more messages thru faster.
Generally I say, "Message queues are for tasks, Kafka is for data." But in the latter case, if your data volume is not huge, a message queue for async ETL will do just fine and give better guarantees as FIFO goes.
In essence, Kafka is a very specialized version of much more general-purpose message queues, which should be your default starting point. It's similar to replacing a SQL RDBMS with some kind of special NoSQL system - if you need it, okay, but otherwise the general-purpose default is usually the better option.
Of course this is not the same as Kafka, but the comment I'm replying to:
> Ah yes, and every consumer should just do this in a while (true) loop as producers write to it. Very efficient and simple with no possibility of lock contention or hot spots. Genius, really.
Seemed to imply that it's not possible to build a high performance pub/sub system using a simple SQL select. I do not think that is true and it is in fact fairly easy to build a high performance pub/sub system with a simple SQL select. Clearly, this design as proposed is not the same as Kafka.
No, I implied that implementing pub/sub with just a select statement is silly because it is. Your implementation accounts for the downfalls of this approach with smart design using a message queue and intelligent locking semantics. Parent of my comment was glib and included none of this.
I have NEVER had any such problem. Subway puts what you want on the sandwich. You tell them directly. You watch them make it. If you ask for soggy things on your sandwich, it will be soggy.
I do not ask for soggy things on my sandwich. That is why I have never eaten a soggy subway sandwich, EVER.
I did have a fairly loud argument with one - only one - of their sandwich artists ONCE, who refused to microwave the bacon (add bacon to any sandwich for $0.50) before putting it on sandwich, per STANDARD SUBWAY PRACTICE. I finally walked out on the dirty bastard, but I did not file a formal complaint with Subway mgmt. I am a busy person. Maybe next time. Perhaps the sandwich would've also been soggy, had I agreed to uncooked bacon, which I will NEVER do, but I don't know why that would make it soggier.
But I digress: Soggy sandwiches have specific reasons. Wanting a cheaper sandwich is not one of them. Bacon will make your sandwich slightly crispier, if that helps. It also tastes EXCELLENT.
EDIT: To be clear, it is important to be SPECIFIC about how much of something you want. The average sandwich artist has a tendency to dump the contents of an entire bottle of a given condiment on your sandwich, which I admit, could cause sogginess. This is because they think they are doing you a favor. I often use bold & dramatic hand gestures while bent over slightly, face pressed against the glass enclosure of the sandwich-making zone. "STOP! RIGHT THERE!" I will yell, several times, if necessary. Then again, I assume everyone knows this. Or else now you do.
> The average sandwich artist has a tendency to dump the contents of an entire bottle of a given condiment on your sandwich, which I admit, could cause sogginess. This is because they think they are doing you a favor. I often use bold & dramatic hand gestures while bent over slightly, face pressed against the glass enclosure of the sandwich-making zone. "STOP! RIGHT THERE!" I will yell, several times, if necessary. Then again, I assume everyone knows this. Or else now you do.
This is mediated by culture and/or region and/or personal taste. I generally find that they default to fairly reasonable amounts and that their notion of "a little" or "a lot" of a sauce etc. matches mine well enough that I can make it work. The much bigger communication problem is getting them to stop assuming you want lettuce and tomato without being asked.
IMX, soggy sandwiches happen mainly because either the sauce wasn't applied neatly, or because (as you say) there are soggy things on it (the lettuce often has water on it, and condiment sauces add up, not to mention the marinara on pizza/meatball type subs).
> I often use bold & dramatic hand gestures while bent over slightly, face pressed against the glass enclosure of the sandwich-making zone. "STOP! RIGHT THERE!" I will yell, several times, if necessary.
1: Methane leaves the atmosphere a lot faster than Co2
2: The methane is a result of breaking down food where the carbon was captured from the air by the plants that were the source of the food.
3: (And I'll let you figure out the numbers) You need to calculate the methane to Co2 ratio of the expected release of methane vs Co2. I suspect there is significantly less methane released than (equivalent) Co2 from cars.
That being said, who wants to go back to horses? I don't.
I do think this is one of Redford's very best films. A lot of detractors make the mistake of assuming the character is intended to represent an expert & experienced sailor, leading to a complete misread of the story. It is possible to be both clever & unwise.
Generally I say, "Message queues are for tasks, Kafka is for data." But in the latter case, if your data volume is not huge, a message queue for async ETL will do just fine and give better guarantees as FIFO goes.
In essence, Kafka is a very specialized version of much more general-purpose message queues, which should be your default starting point. It's similar to replacing a SQL RDBMS with some kind of special NoSQL system - if you need it, okay, but otherwise the general-purpose default is usually the better option.
reply