Genuine question, why have you chosen to phrase this scraping and distillation as an attack? I'm imagining you're doing it because that's how Anthropic prefers to frame it, but isn't scraping and distillation, with some minor shuffling of semantics, exactly what Anthropic and co did to obtain their own position? And would it be valid to interpret that as an attack as well?
I don't think that learning from textbooks to take an exam and learning from the answers of another student taking the exam are the same.
Joking aside, I also don't believe that maximum access to raw Internet data and its quantity is why some models are doing better than Google. It seems that these SoTA models gain more power from synthetic data and how they discard garbage.
Firehosing Anthropic to exfiltrate their model seems materially different than Anthropic downloading all of the Internet to create the model in the first place to me. But maybe that's just me?
I don't see the material difference in firehosing anthropic vs anthropic firehosing random sites on the internet. As someone who runs a few of those random sites, I've had to take actions that increase my costs (and burn my time) to mitigate a new host of scrapers constantly firing at every available endpoint, even ones specifically marked as off limits.
Yes, what the LLM providers did was worse and impacted people financially a whole lot more in lost compensation for works as well as operational costs that would never reach the heights they did solely because of scrapers on behalf of model providers.
It's really not. It was a fun toy but had very little utility. It could generate plausible looking text that collapsed immediately upon any amount of inspection or even just attention. Code generation wasn't even a twinkle in Altman's eye scanning orbs at that point.
And the "too dangerous to release" capability was writing somewhat plausible news articles based on a headline or handwritten beginning of an article. In the same style as what you had written
Today we call that "advanced autocomplete", but at the time OpenAI managed to generate a lot of hype about how this would lead to an unstoppable flood of disinformation if they allowed the wrong people access to this dangerous tool. Even the original gpt3 was still behind waitlists with manual approval
I think you misunderstand the comment you replied to. They are saying the above comment was a rhetorical exaggeration of GPT-2's capabilities as a commentary on how low quality Samsung TV software is. They don't actually think GPT-2 was very capable. It is a figure of speech, not a literal statement.
As far as I can tell, every major version of GTK should be thought of as an entirely separate project, and nothing in GTK 4 made GTK 3 or GTK 2 harder to use.
Please link me to the python3 gtk2 library so that I can migrate all my python2 gtk2 software to python3 without rewriting the entire UI. Thanks in advance!
what does make GTK2 harder to use is that it is not supported anymore. you can't build or run GKT2 based apps on new systems without building the GTK2 libraries yourself.
Swartz died in 2002, decades before LLMs. It is distasteful to put words in the mouths of the dead by invoking him here.
Even local AI concentrates power in the hands of a few, the few who can afford the hardware to run it, and the few who have the luxury of enough time and energy to devote to engaging with the intricate, technical rabbit hole of local models.
It is lauding his accomplishments, yes. Why bring him up in specific if there is no relation intended? There are many broad shouldered giants in this space.
Because it can't. I'm aware of only two projects that have attempted this.
1. Anthropic's C compiler. This is a buggy, unoptimized, unwieldy mess. One that can compile the Linux kernel, which is no mean feat. But it is not "fully featured." And it had an incredible array of tests to start with.
2. Cursor's web browser. This is, flatly, a trainwreck. It's built on top of massive dependencies that do incredible amounts of heavy lifting despite claims to the contrary, and never seems to have ever produced a successful build.
The tests are the real clincher here. That’s the main reason that this project has made so much progress: the FATE suite highlights specific bugs to fix and tracks regressions.
As for optimization, that seems to be more of a question of effort than whether it’s possible. I was able to take down the performance gap on Rust vs C (without Assembly) from 10x to 1.5x through detailed profiling and iterative improvements with Claude.
It also looks like the Anthropic C compiler was built from scratch. By contrast, `wedeo` was directly based on FFmpeg’s existing code. Going by spec and test suite only would have taken a lot longer, and the quality would have been significantly lower.
Anthropic's C compiler also had an incredible test suite available, and was trained on compiler codebases. It might have been built "from scratch" but compilers are incredibly well-trodden ground.
And to contrast, Cursor's browser explicitly reviewed Servo's architecture before setting out, and still wound up like that.
The difference between using general knowledge and summaries and doing code-for-code rewrites cannot be understated. When creating `wedeo`, despite DIRECTLY reviewing FFmpeg's code, Claude would frequently make mistakes that would have to be fixed. I would constantly have it go and refer back to the ground truth FFmpeg source implementation. It did eventually work, but only after several rounds of reviews, and often dead-end investigations. Being able to directly inspect (and in some cases modify for extra debug output) FFmpeg at any given time was invaluable, and I doubt any rewrite would succeed without having that.
Yes, but this does come with differences and tradeoffs. If the terminal isn't managing the scrollback, you don't get scrollbars and you lose any smooth/high resolution scrolling. You also lose fancy terminal features like searching the scrollback, all that needs to be implemented in your application. Depending on the environment it can also wind up being quite unpleasant to use with a trackpad, sometimes skipping around wildly for small movements.
The other part (which IMO is more consequential) is that once the LLM application quits or otherwise drops out of the alternate screen, that conversation is lost forever.
With the usual terminal mode, that history can outlive the Claude application, and considering many people keep their terminals running for days or sometimes even weeks at a time, that means having the convo in your scrollback buffer for a while.
It's not usefully deterministic in the way computers usually are. Sensitively identical input can still lead to wildly different outputs even if all randomness is crushed out.
reply