Here is the relevant quote by Trenton Bricken from the transcript:
One example I didn't talk about before with how the model retrieves facts: So you say, "What sport did Michael Jordan play?" And not only can you see it hop from like Michael Jordan to basketball and answer basketball. But the model also has an awareness of when it doesn't know the answer to a fact. And so, by default, it will actually say, "I don't know the answer to this question." But if it sees something that it does know the answer to, it will inhibit the "I don't know" circuit and then reply with the circuit that it actually has the answer to. So, for example, if you ask it, "Who is Michael Batkin?" —which is just a made-up fictional person— it will by default just say, "I don't know." It's only with Michael Jordan or someone else that it will then inhibit the "I don't know" circuit.
But what's really interesting here and where you can start making downstream predictions or reasoning about the model, is that the "I don't know" circuit is only on the name of the person. And so, in the paper we also ask it, "What paper did Andrej Karpathy write?" And so it recognizes the name Andrej Karpathy, because he's sufficiently famous, so that turns off the "I don't know" reply. But then when it comes time for the model to say what paper it worked on, it doesn't actually know any of his papers, and so then it needs to make something up. And so you can see different components and different circuits all interacting at the same time to lead to this final answer.
> these agents are trained on bad code - which is open source.
This is doubtful and not what I've seen in over 30 years in the industry. People who are ashamed of their source code don't make it Open Source. In general, Open Source will be higher quality than closed source.
Sure, these days you will need to avoid github repositories made by students for their homework assignments. I don't think that's a problem.
> Even the charging costs for 1GWh is absurdly high compared to Heavy Fuel Oil. An orders of magnitude more expensive.
Burning 250 tons of oil to get 1GWh of energy releases around 800 tons of CO2.
Let's assume a $100 CO2 tax. We want to prevent the worst of global warming, right? That would add ~25% to the price of oil.
There is likely to be an oversupply of renewable (solar) energy less than 5 years from now.
So I wouldn't be so sure about that 100 year prediction.
Even under the EU Emissions Trading System (ETS), where shipping companies already must buy allowances for CO2 emissions from large vessels calling at EU ports, costing roughly €80-€90 per tonne of CO2 emitted, batteries aren't remotely competitive with HFO/LSMFO.
Even if the electricity was free, the cost (both CAPEX and in mass/volume) is not close. We need an improvement in mass energy density and volumetric energy density of 200-1,000% and a complete redesign of all shipping and ports to migrate to battery transoceanic shipping.
SMRs, renewably cracked hydrocarbons, and fusion will all be mainstream beforehand.
Once again, this is one of those areas where HN commenters believe they can understand a complex industry based on Wikipedia-level stats.
> Once again, this is one of those areas where HN commenters believe they can understand a complex industry based on Wikipedia-level stats.
Please elaborate your math here. I just outlined the cost of batteries, the cost of fuel, the cost of electricity. Is my math wrong? Because if it isn't, it's at least feasible. You seem to assume very different numbers here. Which I would argue are probably a combination of dated and wrong.
> SMRs, renewably cracked hydrocarbons, and fusion will all be mainstream beforehand.
We'll know in a few years how wrong you or I will be. I don't find your argumentation very persuasive though. You might be eating your proverbial hat by the 2040s. I'm betting somebody will manage to stuff a few gwh of battery in a ship by then. A shipment of 7-9K EVs at 50kwh each, pretty much gets you there. That's the capacity of some of the new ships that BYD uses for transporting their EVs around the world. 2000EVs is basically about 1 gwh of power.
Yes, your maths is off by roughly an order of magnitude because the starting assumptions are wrong.
> The mwh price is 80–90$, so 3 gwh (3000mwh) would be about 240K $
You’re effectively designing around ~3GWh for an ocean leg, but a large container vessel at 40–60MW continuous draw burns roughly 1–1.5GWh per day at sea.
A 20–30 day crossing needs on the order of 20–40GWh of shaft power, not 3GWh. 5,000t of HFO actually corresponds to ~50–60GWh of chemical energy and ~25–30GWh delivered to the propeller at realistic engine efficiencies.
> The mwh price is 80–90$
$80–90/MWh is a generation/LCOE number, which you're comparing to $500/t HFO delivered, stored, with global bunkering logistics already in place.
You're not accounting for the cost of delivering tens of GWh at hundreds of MW into a hull, in a tight port stay, via infrastructure that simply doesn’t exist. Even if you grant free electricity at the fence, the capex for multi-hundred-MW substations, converters, cabling, connectors, etc completely dominates.
> A shipment of 7-9K EVs at 50kwh each, pretty much gets you there. That's the capacity of some of the new ships that BYD uses for transporting their EVs around the world. 2000EVs is basically about 1 gwh of power.
2,000 EVs * 50kWh ≈ 100MWh. 9,000 EVs * 50kWh ≈ 450MWh. That’s still one to two orders of magnitude below what a long-range deep-sea vessel actually needs on a single leg.
> We'll know in a few years how wrong you or I will be.
Anyone in or close to the maritime industry knows now. Not even the most bullish consider economic transoceanic shipping by battery-powered vessels by the 2040s remotely possible. Realistically pure-battery transoceanic cargo ships will never happen, because other superior zero-carbon options will become viable long before batteries close the energy density and infrastructure gap.
We'll obviously see batteries in tugs, ferries, short-sea and hybrid ECA work become the standard much sooner.
The upper weight limit for both 20ft and 40ft shipping containers is around 28,300 kg. Since SodiumLion batteries in containers will be weight limited, it makes sense to use 20ft shipping containers. One container can probably store around 3MWh of energy. For a 2700nm trip at 15 knots with 15MW of power usage, you need 2700Mwh divided by 3MWh per container gives us 900 20 ft shipping containers (TEU) for the battery.
OTOH if we look at popular shorter range routes like within Asia or Europe the calculation looks a lot more favorable.
I don't think "wrong memory" is accurate, it's missing information and doesn't know it or is trained not to admit it.
Checkout the Dwarkesh Podcast episode https://www.dwarkesh.com/p/sholto-trenton-2 starting at 1:45:38
Here is the relevant quote by Trenton Bricken from the transcript:
One example I didn't talk about before with how the model retrieves facts: So you say, "What sport did Michael Jordan play?" And not only can you see it hop from like Michael Jordan to basketball and answer basketball. But the model also has an awareness of when it doesn't know the answer to a fact. And so, by default, it will actually say, "I don't know the answer to this question." But if it sees something that it does know the answer to, it will inhibit the "I don't know" circuit and then reply with the circuit that it actually has the answer to. So, for example, if you ask it, "Who is Michael Batkin?" —which is just a made-up fictional person— it will by default just say, "I don't know." It's only with Michael Jordan or someone else that it will then inhibit the "I don't know" circuit.
But what's really interesting here and where you can start making downstream predictions or reasoning about the model, is that the "I don't know" circuit is only on the name of the person. And so, in the paper we also ask it, "What paper did Andrej Karpathy write?" And so it recognizes the name Andrej Karpathy, because he's sufficiently famous, so that turns off the "I don't know" reply. But then when it comes time for the model to say what paper it worked on, it doesn't actually know any of his papers, and so then it needs to make something up. And so you can see different components and different circuits all interacting at the same time to lead to this final answer.
reply