His whole thing is to make obvious incontestable claims about AI (LLM’s make mistakes) and connect it to unfalsifiable grand prognostications (It’s all gonna crash… any day now). Its the same tactic any preacher who harps on about the impending apocalypse uses.
His increasing accuracy derives from him wisening up and criticizing LLM’s only in domains where there is a consensus, and keeping his more grand criticisms unfalsifiable.
I’m not the OP, and haven’t read much Marcus, but an analysis linking “chains of claims” together could be interesting, devaluing specific true claims when they are used to support a false claim. The claims dataset appears to evaluate each claim independently.
It costs money to run AI models. The company serving you tokens has to make it up somehow.
This demo however undersells the tactically insidious way ads could be run in an AI chat. All it would need to do is merely recommend a product at a slightly higher percentage. In fact the chat could be biased in imperceptible ways which drive the user's thinking, aims and behavior patterns towards an outcome which leads them to seeking out a specific brand, website, app, etc. In aggregate, the ads are served, just not without making it ever obvious.
Even if there is "auditing" on the behavior of models, it is possible to train preferences into models without any of those preferences being specifically stated in the training data:
> In 8 experiments on 5 prominent and diverse adversarial imagesets, human subjects correctly anticipated the machine’s preferred label over relevant foils—even for images described as “totally unrecognizable to human eyes”.
You're doing the common thing where multiple criticisms you have of LLM's are individually valid but mutually incompatible.
If the point was that an AI is a fancy Markov chain, then it would produce a more uniformly random distribution of random numbers (perhaps with 7 more often due to humans preferring that as it feels "more random")
However the idea that it's an issue that it always produces 7 implies that LLM's are not random enough, and rather have collapsed to a definite mode of thought which restricts stochastic variation.
Both cases taken together, you are claiming that LLM's are too random to be truly "intelligent", but also not random enough.
All of this is a distraction from the fact that LLM's can write thousands of lines of genuinely useful, novel code. I feel the only way to reasonably reconcile this with their varying failure cases is to entertain the notion that LLM's not "unintelligent", but merely a different kind of intelligence than humans, and will have distinct deficits and distinct aptitudes. (This isn't a comprehensive assessment, but it, as a general description, has more predictive power than claiming that LLM's are just a "slot machine")
This is part of the reason why American tech companies are so successful though. Being unable to lay off workers causes stagnation at companies where fast-development is paramount.
That helps, but there's a reason why the collective economy of the EU has produced only like 1/10 the number of unicorns over the past 30 years despite their GDP being around 2/3 the US's
I was concerned originally when I heard that Anthropic, who often professed to being the "good guy" AI company who would always prioritize human welfare, opted to sell priority access to their models to the Pentagon in the first place.
The devil's advocate position in their favor I imagine would be that they believe some AI lab would inevitably be the one to serve the military industrial complex, and overall it's better that the one with the most inflexible moral code be the one to do it.
I was concerned originally when I heard that Anthropic, who often professed to being the "good guy" AI company who would always prioritize human welfare, opted to sell priority access to their models to the Pentagon in the first place.
The devil's advocate position in their favor I imagine would be that they believe some AI lab would inevitably be the one to serve the military industrial complex, and overall it's better that the one with the most inflexible moral code be the one to do it.
AI was always particularly well suited to military use and mass surveillance. It can take huge amounts of raw data and parse it for your, provide useful information from that. And let's face it, companies exist for profit.
True, and that has been going on for awhile now. But what does that have to do with Anthropic's genai chatbots with comparatively tiny context windows?
> opted to sell priority access to their models to the Pentagon
The bottom of all of this is that companies need to profit to sustain themselves. If "y'all" (the users) don't buy enough of their products, they will seek new sources of revenue.
This applies to any company who has external investors and shareholders, regardless of their day 0 messaging. When push comes to shove and their survival is threatened, any customer is better than no customer.
It's very possible that $20 Claude subscriptions isn't delivering on multiple billions in investment.
The only companies that can truly hold to their missions are those that (a) don't need to profit to survive, e.g. lifestyle businesses of rich people (b) wholly owned by owners and employees and have no fiduciary duty.
Anthropic cares first and foremost about extinction risk. This is not what everyone who professes to care about human welfare thinks should be at the top of the priority list. See e.g. the Voluntary Human Extinction Movement for an example of a humanistic approach to letting humanity die off with no replacement.
One of the most challenging problems in AI safety re/ x-risk is that even if you can get one country to do the right thing, getting multiple countries on board is an entirely different ballgame. Some amount of intentional coercion is inevitable.
On the low end, you could pay bounties to international bounty hunters who extract foreign AI researchers in a manner similar to an FBI's most wanted lost, and let AI researchers quickly do the math and realize there are a million other well paid jobs that don't come with this flight risk. On the high end you can go to war and kill everyone. Whatever gets the job done.
Either way, if you want to win at enforcing a new kind of international coercion, you need to be at the top of the pack militarily and economically speaking. That is the true goal here, and I don't think one can make coherent sense out of what Anthropic is doing without keeping that in the back of their mind at all times.
If you can't distinguish the actual utility and progress of AI from it's annoying hype-men then it's hard to take your dismissal of AI seriously.
Failure to appreciate changes in AI will have left you calling every shot wrong over the past 5 years. While AI models continue to improve at an exponential rate, you'll cling to your facile maxims like "dude it's just predicting the next token it isn't real intelligence".
> In England, where 28 percent of all bus passengers are on concessionary fares for age or disability, Prime Minister Margaret Thatcher is supposed to have said, ‘If a man finds himself a passenger on a bus having attained the age of 26, he can count himself a failure in life’.
I think this general attitude is the source of the general poor quality of public transit in the US (even though the quote refers to the UK). Public transit is considered a mode of the designated underclass, and nobody making policy decisions considers themselves a member of that class. In the US you aren't really considered an "active participant" in white-collar society unless you have a car (though NYC is a notable exception to this maxim, it holds true in essentially all other cities)
reply