Another great example of the unintuitiveness of heritability is the fact that earrings are highly heritable. Earrings are highly correlated to a specific genetics (being female), so they're very "heritable", even though that correlation is an arbitrary cultural fashion.
Highly recommend using the minimum release age setting, though I think a week is probably overkill. Did any of the recent supply-chain attacks have a malicious version up for more than a day?
Maybe not, but how much of that was luck? I think it's only a matter of time until a similar compromise happens but nobody notices it for a few days, better safe than sorry.
People are already scanning npm constantly. You can limit yourself to pre-scanned packages by setting npm's minimum release age setting to 1 or 2 days (a timeframe that all the recent high-profile malicious package versions were unpublished within).
Note to self: the test suite for vetting a package should include setting the system date some time in the future, to check if an exploit is trying to sleep long enough to defeat the age limit.
Npm's package-lock.json already handles pinning everything to exact versions, including subdependencies. Pinning exact versions in package.json doesn't affect your subdependencies.
You aren't wrong. However, this article does offer some additional advice on this matter, and some potential reasons why it might still be desirable to pin your deps in package.json.
> If a lock file gets out of sync with its package.json, it can no longer be guaranteed to lock anything, and the package.json will be the source of truth for installs.
> provides much less visibility than package.json, because it's not designed to be human readable and is quite dense.
> If the package.json has a range, and a new in-range version is released that would break the build, then essentially your package.json is in a state of "broken", even if the lock file is still holding things together.
It's silly to act like they've got mud on their face when Mythos and Opus are apparently some of the very best models. Anyone that has found value out of previous LLMs is likely to find more value out of the newest ones. The only thing Mythos looks bad against is the very tall bar some people have imagined. People are putting too much weight on marketing and then reaction to marketing.
> People are putting too much weight on marketing and then reaction to marketing.
No, what others are doing, which I've done myself in the past too, is to evaluate how much their marketing matches up with reality, then share our experience about that. Very different than just "putting too much weight on marketing".
Hasn't almost every new frontier model had an early period of limited access? I don't get why everyone is acting like Mythos is particularly egregious for this.
> We formed Project Glasswing because of capabilities we’ve observed in a new frontier model trained by Anthropic that we believe could reshape cybersecurity.
> Claude Mythos Preview is a general-purpose, unreleased frontier model that reveals a stark fact: AI models have reached a level of coding capability where they can surpass all but the most skilled humans at finding and exploiting software vulnerabilities.
Yeah, the answer almost certainly has to be this, or that they were using an old version of the package which didn't use the system RNG correctly (the current version appears to do it correctly, but I didn't dive into older versions), or their project has loaded an old broken polyfill re-implementing the JS crypto API, or they were running this on a hosting setup that does something jank like resuming the same VM snapshot with its RNG state on multiple servers. This category of explanation is many orders of magnitude more likely than a true random collision.
There's already an okay solution to supply-chain attacks against dependency managers like npm, PyPI, and Cargo: set them to only install package versions that are more than a few days old. The recent high-profile attacks were all caught and rolled back within a day, so doing this would have let you safely avoid the attacks. It really should be the default behavior. Let self-selected beta testers and security scanner companies try out the newest versions of packages for a day before you try them. Instructions: https://cooldowns.dev/
An artifact manager. Only get what you approve. So you can get fast updates when needed and consistently known stable when you need it. Does need a little config override - easy work.
I had my own janky tooling for something like it. This is a good project.
Does that really scale well? Thanks to cascading dependencies, even a medium sized project can import hundreds of dependencies. Can a developer really review them all to figure out if they are safe and that there's not security fix that was fixed in a newer version of the package?
Yes, that is what is required. Every dependency needs an internal owner and reviewer. Every change needs to be reviewed and brought into the internal repository.
If no one is willing to stand up and say "yes this is safe and of acceptable quality", why use it?
It's a software engineering version of the professional engineering stamp.
So you get security updates late too? Many vulnerabilities are in the wild for years before being noticed, and patched.
Once noticed, that's where the exploit explosion erupts, excited exploiters everywhere, emboldened... enticed... excessively encouraged, by your delayed updates.
Presumably npm exempts security updates from its minimum release age, but even if it doesn't, I think the times where you need an important security update are relatively rare enough that handling the real cases on a case-by-case basis with whitelisting is fine. Outside of Next.js's React2Shell vulnerability last year, I'm not sure I've ever had a security update of a dependency written in a memory-safe language (ie. not C/C++) which I've installed through npm/PyPI/Cargo that patched a security vulnerability that had been making my application exploitable to others in practice. Almost all security vulnerabilities I've personally seen flagged through npm are about things I only use at build-time and are only relevant if a user can create and pass an arbitrary object to the function, which is rarely the case. Most security vulnerabilities I've encountered and fixed in working on web apps were things like XSS, SQL injections, and improperly enforced permissions, and they nearly always happened in the application's own code rather than inside a dependency.
> exempts security updates from its minimum release age
If it does, doesn't that defeat the purpose? If a package is compromised, of course the compromiser will just label their new version as a "security update".
> Presumably npm exempts security updates from its minimum release age
Why would it? Then an attacker would just push compromised code as a "security update". Since the majority of these npm attacks are account-based, the attacker can do everything the actual owner could.
IMO, the most sustainable version is either the linux distros/bsd ports/homebrew models. You don't push new libraries to the public registry, instead you write a packaging script that gets reviewed for every new changes.
Another model is Perl's CPAN where you publish source files only.
Trust me, as someone who has contributed to such a package set, almost nobody is inspecting diffs between upstream versions when updating a package. Only the package definitions themselves are reviewed, but they are typically only version + hash bumps.
Reviewing upstream diffs for every package requires a lot of man hours and most packagers are volunteers. I guess LLMs might help catching some obvious cases.
Not really talking about upstream. Most supply attacks I’ve heard about are stolen secrets and artifacts uploading. They’re not about repositories or websites being taken over. As the packaging scripts are often in repos, you detect easily if people are trying to update where upstream points to.
That usually ends up as proxies to the upstream repos, because the people managing the company repos don't have time to review every new version of a package.
At that point you're just as vulnerable to a supply chain attack.
Shellshock was in 2014 and Log4Shell was 2021. It's far more likely that you're going to get pwned by using a too-recent unreviewed malicious package than to be unknowingly missing a security update that keeps you vulnerable to easy RCEs. And if such a big RCE vuln happens again, you're likely to hear about it and you can whitelist the update.
reply