We detached this comment from https://news.ycombinator.com/item?id=48929037 and marked it off topic.

You may be new here. This is Simon’s de facto benchmark for models. I happen to find it a really good one.

Small aside: It’s crazy to me that while it’s improved over time it does seem like most of the models haven’t been trained specifically to defeat this one.

The pelican thing is definitely getting deeper into model weights over time just by getting fed threads like this.