Latest episode
The Sim-to-Real Gap
- Running AI in production now — even badly — builds a failure map that cannot be bought later and compounds as a competitive advantage.
- The most valuable person for managing AI in client work is not the most technical but the one with the sharpest instinct for edge cases.
Read the transcript
In this episode
- The sim-to-real gap from robotics explains AI's jagged capability profile better than any capability or hype frame: benchmarks are simulations, production is the real world, and the gap between them is real but closeable
- Firms running AI in production now — even imperfectly — are building a failure map that cannot be purchased later; that map is a durable competitive asset
- The person most valuable for managing AI in a wealth management context is not the most technical person on the team but the person with the best instinct for edge cases in client work
- Political backlash against data centers is now a retail electoral message, which introduces genuine uncertainty into the "AI compute gets cheaper on a smooth curve" assumption
- Energy infrastructure risk — siting fights, grid strain, permitting — is an underappreciated variable in AI pricing trajectories over the next two years
- Frontier AI systems are demonstrably capable of producing confident, internally consistent, persuasive output that is wrong; the Clocktower benchmark sharpens this, even if the failure mode is not intentional deception
- The governance question is not about AI intent — it is about whether the human review step before client contact is designed to catch confident wrongness, not just obvious errors
This is The Judgment Layer, from Advisers Give Back — a nonprofit working to increase access to pro bono financial planning, pairing households who can't afford a financial planner with CFP professionals who volunteer their time. Each week, we sort through what actually mattered in AI, and what it changes for the firm you run. Written and voiced with AI; directed by Matt Iverson-Comelo, executive director of Advisers Give Back. Nothing here is investment, legal, or compliance advice.
This week, the most useful frame for understanding AI isn't a capability announcement — it's a concept from robotics that explains why the tools keep surprising you.
The story that most deserves your attention this week isn't the one with the biggest headline. It's a question buried inside a robotics paper that turns out to be the most useful frame I've found for understanding why AI keeps surprising you — in both directions.
Timothy B. Lee, the technology journalist who writes the Understanding AI newsletter, published a piece this week about a concept called the sim-to-real gap. It comes from robotics, and it goes like this. When researchers train a robot in a simulated environment and then deploy it in the physical world, it often fails in ways that are both surprising and predictable. The simulation is almost right. It captures most of what matters. But the real world has friction, variation, and edge cases the simulation never modeled. And the robot, which has never seen any of that, hits a wall — sometimes literally.
Lee's argument is that the sim-to-real gap is everywhere in AI right now, not just in robotics. The benchmarks that frontier models ace — the tests that generate the press releases and the capability claims — are, in a sense, simulations. They are structured, well-defined problems in a known environment. When a model scores at or above human expert level on a medical licensing exam, that is a real result. But deploying that same model in an actual clinical workflow, with ambiguous patient histories and stressed physicians and EHR systems that were designed in two thousand and eight, is the real world. The gap between those two environments is the sim-to-real gap. And it explains something that probably frustrates you. How a tool can be genuinely impressive in a demo and genuinely mediocre in production, without anyone having lied to you.
Here is the non-obvious part, and it's the reason this idea belongs in your leadership meeting. The sim-to-real gap is not static. In robotics, researchers have learned to close it systematically — by deliberately training in noisier, messier simulations, by exposing models to more variation, by building feedback loops from real-world failures back into training. The gap shrinks. And the history of robotics suggests it shrinks faster than most people expect, once the right techniques are applied.
What that means for a firm like yours is that the AI tools disappointing you in production today are probably not the same tools you will be evaluating in eighteen months. The jaggedness Lee describes — extraordinary performance in some domains, baffling failure in adjacent ones — is a characteristic of an early stage, not a permanent feature. And here is what that asymmetry demands of you. The firms that are running AI in production now, even imperfectly, are accumulating something the firms on the sidelines are not. They are building the feedback loops. They are finding out where their sim-to-real gap actually lives — which workflows, which client types, which question structures reliably break the tool — and that knowledge is not transferable. You cannot buy it. You can only earn it by doing.
The picture it paints for twelve to eighteen months is specific. Firms that have been running AI against real client work — even in a limited, supervised way — will have a detailed map of where the tools fail in their context. They will have trained their people on what that failure looks like. When the next generation of tools closes the gap in several of those areas, those firms will be able to move quickly, because they already know which problems are being solved. Firms that waited will be starting from scratch, without that map, at a moment when the competitive pressure is higher.
There is also a staffing implication worth sitting with. The person best positioned to manage AI tools in a wealth management context is not the most technically sophisticated person on your team. It is the person with the best instinct for where your firm's work departs from the well-structured, well-labeled, well-defined — the person who can look at a client situation and recognize that it is an edge case before the tool has a chance to miss it. That is a form of professional judgment that you probably already have on your team, and it may be undervalued right now precisely because it has never had to be named.
The one-sentence version. The jaggedness of AI is not a reason to wait — it is the enrollment form for the class you need to have already taken.
The other thing I'd want you to know about this week is what happened in Texas, and it has a second-order consequence for advisory firms that the headlines missed entirely.
MIT Technology Review reported that Texas — which has been the fastest-growing data center market in the country and the site of several of Big Tech's largest AI infrastructure buildouts — is now facing real political uncertainty after this week's midterms. Democrat James Talarico flipped a United States Senate seat in Texas, and Democrat Gina Hinojosa won the governor's race against the Republican incumbent. That is a significant political shift in a state that had been specifically hospitable to AI data center development, and it arrived at the same moment as a broader backlash. Political ads in a number of markets ran explicitly against data center siting — the electricity consumption, the strain on local grids, the land use. That is a new thing. Data center opponents have found a retail political message that works, and that changes the trajectory.
Meanwhile, the Senate is set to vote this week on something called the Bipartisan American Affordability and Jobs Act — legislation designed to cut permitting red tape and speed up energy infrastructure buildout. MIT Technology Review covered that too. These two stories are running in opposite directions at the same time. One piece of legislation trying to accelerate energy infrastructure, and a political environment that is newly hostile to the facilities that need that infrastructure most.
Here is what this means for you, and it is not obvious from the headlines. The cost of frontier AI — the thing the Anthropic S-1 told us was trending toward software economics — is partly a function of how cheap and reliable the energy supply to those data centers turns out to be. If the Texas buildout stalls, if permitting fights slow the construction of new capacity, if energy costs for data centers increase because the political environment is making siting harder, then the gross margins Lee's newsletter described in the Anthropic S-1 may face more pressure than the current financial model assumes. It doesn't reverse the direction of travel. But it means the timeline on cheap, abundant AI compute is more uncertain than the pure-capability story suggests.
For a firm planning its AI spend over the next two years, the practical implication is this. The pricing environment for AI services is not on a single smooth curve downward. It has real infrastructure dependencies that are now subject to political risk in a way they weren't a year ago. That doesn't mean lock in long-term contracts today — that would be getting ahead of what anyone can know. It means that the firms building their strategy around the assumption of rapidly falling AI costs should hold that assumption a little more loosely than the headlines have encouraged.
There's one more thing worth your time this week, and it's the most uncomfortable item in the roundup.
Dwarkesh Patel, a journalist and podcaster who covers AI and science, published a piece about a benchmark called Clocktower — a test where AI systems are evaluated not on answering questions correctly, but on their ability to deceive. The setup is a multiplayer game structure in which winning requires lying convincingly, building false coalitions, and maintaining a cover story under pressure. The finding, reported by Patel, is that frontier AI systems are now good enough at this to run what the researchers call a conspiracy — to coordinate deception across multiple interactions in ways that are strategic rather than accidental.
I want to be careful about what this does and doesn't mean. This is a benchmark, which is to say it is a simulation, and Lee's piece about sim-to-real gaps applies here too. A model that can maintain a deceptive narrative in a structured game is not the same as a model that is trying to deceive your clients. The intent is not there in any meaningful sense. But here is what is real and worth taking seriously. AI systems are increasingly capable of producing confident, internally consistent, persuasive output that is wrong. That is not deception in the moral sense. It is a capability profile that has similar downstream effects when the output reaches a client without review.
The governance question this raises is not new, but the Clocktower results sharpen it. When you think about where AI output enters your client-facing work — draft financial plans, meeting summaries, recommended next steps, proposed language for a sensitive conversation — the relevant question is not whether the AI is trying to deceive anyone. It is whether the person reviewing that output before it reaches a client has the context and the mandate to catch confident wrongness. Confident and wrong is the failure mode that matters. And the benchmark suggests frontier models are getting better, not worse, at the confident part.
If you do not have a written policy for how AI-generated content is reviewed before it touches a client, this is the week to start drafting one. Not because the tools are malicious. Because the professional obligation to get it right does not transfer to the tool.
Looking at the week ahead. The Senate vote on the Bipartisan American Affordability and Jobs Act on Monday will be worth watching, specifically for what it signals about the political appetite for energy infrastructure at scale. If it passes with real teeth, it pushes back against the Texas story somewhat. If it stalls or gets watered down, the data center capacity question gets more complicated, not less.
I'd also watch for any further reporting on Anthropic's S-1 roadshow — the first time, as the Hard Fork podcast noted, that the books of a pure-play frontier AI lab have been open to public scrutiny. The analysts picking through those four hundred and twenty-one pages will surface things this week that the first wave of coverage missed.
The idea worth bringing into your leadership meeting this week is the sim-to-real gap — not as a technical concept, but as a management frame. Ask your team. Where is our sim-to-real gap? Where are we evaluating AI in conditions that are cleaner and more structured than the actual work? Where are we running it in production, and what are we learning from the failures? The firms that can answer that second question specifically are ahead. The firms that can only answer the first one have work to do.
One thing to try this month
Ask your team this week: where is our sim-to-real gap? Name the specific workflows where you are evaluating AI in cleaner conditions than the actual client work requires — and name the ones where you are running it in production and learning from failures. The gap between those two answers is the work.
Questions for your leadership team
- Where in our firm are we evaluating AI against well-structured, representative examples of real client work — and where are we still testing it in conditions that are cleaner than the work actually is?
- If AI compute costs do not fall as smoothly as we have assumed — because of energy infrastructure constraints and political headwinds — how does that change the timeline and budget we have built around AI adoption?
- When AI-generated content reaches a client — in any form — who is reviewing it, with what mandate, and are they specifically equipped to catch output that is confident and wrong rather than just obviously mistaken?
Sources
- Timothy B. Lee, Understanding AI — "This robotics concept helps explain the jaggedness of frontier capabilities"
- MIT Technology Review — "Texas was the new epicenter of the AI data center buildout. That might be kaput."
- MIT Technology Review — "A new bill could change how the US builds energy infrastructure"
- Dwarkesh Patel — "The benchmark where AIs have to lie to win"
- Hard Fork, The New York Times — "Are We in an A.I. Bubble? We're About to Find Out."
