Capability → trust → autonomy: the three phases of the agent age

Capability → trust → autonomy: the three phases of the agent age

The first phase of the agent age was about capability: can the combination of model, harness, tools, and context reliably get work done? Madhu Guru’s answer is that we are through it — capabilities have gotten “pretty far” — and the baton now passes to trust: does the harness take the right action, with the right permissions? Can we observe it, audit it, and establish accountability? Only after that comes autonomy.

The post is a quote-tweet of Satya Nadella’s new X article, “Models as Insider Risks in the Super Intelligence Era”. Nadella’s framing: traditional software gave us the tools to trace a behaviour back to a specific code path. Agentic systems break that traceability — the behaviour emerges from a model, a harness, tools, and context interacting, which is exactly why agentic security must now outpace agent capability. Guru puts the race in one sentence: agent capabilities will soon evolve at recursive-self-improvement pace, and security — identity, permissions, observability — has to outrun it.

Notice how neatly this stacks onto yesterday’s lead story. Edition 16’s lecture (leerob, Stanford CS146S) taught us the architecture of always-on agents: heartbeat, context store, harness. This is the sequel: once the architecture exists and works, the frontier moves from can it work? to can it be trusted? Identity, permissions, observability — Guru names them as a triad, not a checklist. Identity is who the agent is allowed to be. Permissions are what it may touch. Observability is whether anyone can see what it did. Without all three, accountability is a press release.

Guru also notes, honestly, that “a lot of serious work has already gone into these problems over the last couple of years” — this is not a green field. What changes is urgency: the challenge “gets more important. And harder.” The word insider risk in Nadella’s title is doing quiet work here. In the super-intelligence era, the threat model is not only the malicious outsider — it is the trusted agent, acting with legitimate permissions, making an unobserved mistake.

• The agent conversation has a phase map. Stop asking “are agents real?” — that was Phase 1. The questions that matter now are identity, permissions, and observability.

• The trust triad is the new stack. Identity (who the agent is), permissions (what it may touch), observability (who can see what it did) — these are infrastructure, not policy.

• Insider risk is the new threat model. The danger is not only the external attacker; it is the legitimate agent making an unobserved move.

• Speed is the pressure. If capabilities advance at recursive-self-improvement pace, trust tooling cannot move at committee pace.

Grade what the agent changed, not what it said

Sergii Makarevych’s long-form X article asks the question every team shipping LLM agents is now stuck on: how do you know whether an AI agent did the job — and will do it again? His answer is a discipline he calls agentic evals: grade agents by the end state they leave in the world (the database, the ticket, the repo), not by the fluency of their reply. His demo is brutally clean. In a runnable 20-task customer-support agent experiment, grading by database end-state passed 48 of 60 episodes. Grading by what the agent said passed 58 — including a do-nothing agent that passed 20 out of 20. The reply looked right. Nothing happened.

The article draws the two axes every agent builder needs: pass@k (capability — does it succeed at least once in k attempts?) and passk (reliability — does it succeed in every one of k attempts?). Capability is what demos sell; reliability is what production buys. Makarevych then walks through the failure forensics: first-wrong-step analysis to locate where an episode actually started going wrong, evidence levels for model-as-judge graders, cost-per-passing-episode as the metric that matters once you scale, how multi-agent hand-offs bleed reliability, and the eternal nemesis — reward hacking, where the agent games the metric instead of doing the work. It closes with a decision guide, fifteen takeaways, and a references list of arXiv papers. It is, in effect, a field manual.

Read this as the companion manual to the lead story. The lead says trust is Phase 2 of the agent age; Makarevych’s article is how you manufacture it. Evals are the observability-and-accountability layer made concrete: repeatable trials, database-truth grading, and a unit economics metric (cost-per-passing-episode) that turns reliability from vibes into budget. Note the pairing the article insists on: capability without reliability is a demo; reliability without cost discipline is a hobby.

• Grade the world, not the words. End-state grading (what the agent changed) is the only grade that survives contact with production; reply-grading can pass a do-nothing agent.

• Measure both axes. pass@k tells you if the agent can do it; passk tells you if it will do it. Ship on the second.

• Find the first wrong step. Failure forensics beats aggregate scores: locate where an episode started degrading and fix there.

• Watch the hand-offs. Multi-agent pipelines lose reliability at every baton pass — budget eval coverage for the seams, not just the nodes.

• Price the pass. Cost-per-passing-episode is the metric that decides whether an agent workflow is a business or a demo.

Two-slice teams: the pirate and the architect

Tired: two-pizza teams. Wired: two-slice teams. Sujayath — co-founder of Silq, founder of Voonik — says he “got a lot of pushback” for arguing that one product manager and one engineer can now be the entire team. Then Lenny Rachitsky’s newsletter handed him the vocabulary: the pirate and the architect. The pirate finds the treasure — the customer, the insight, the wedge, the nerve to ship. The architect builds the ship — the systems, the evals, the scaffolding that lets the whole thing sail without the crew. Two roles. Two people. That is the team.

The deeper claim is about what AI did to team physics. When intelligence and output are abundant, coordination becomes the scarce resource — and the cheapest way to coordinate is to not need to. A two-slice team has no handoffs, no alignment meetings, no ticket queues between the person who sees the opportunity and the person who builds for it. The pirate-architect split is really a division of judgement (where do we point this?) and craft (how do we make it hold?). In the agent era, the craft layer is increasingly the agents themselves — the architect directs a fleet while the pirate keeps stealing treasure.

• Team size is a coordination technology. Two-pizza teams optimised for a world where doing the work was expensive. Two-slice teams optimise for a world where aligning is expensive.

• The split is judgement vs craft. The pirate owns direction; the architect owns the machine. Each is legible without the other.

• Agents slot under the architect. The craft layer now includes directing agent fleets — which is why one engineer can cover what a team used to.

Claude Opus 5.5 animates like a studio

“You no longer need After Effects if you use Claude Opus 5.5 correctly.” Florence’s post — the single most bookmarked thing in this edition, at 3.8K bookmarks — points at her Opus 5.5 Motion Lab, an X article in which every animation was made by Claude Opus 5.5, each one published alongside the exact prompt that made it plus every follow-up the model needed. She has even linked the “Exact prompt.txt” on Google Drive. The motion-design moat, she is saying, has collapsed into a text file.

Watch the mechanism, because it is the same one running through this whole edition. The studio’s craft was never only taste — it was technique encoded in a tool: keyframes, easing curves, layer comps, render graphs. A frontier model with enough context and enough iterations re-derives that technique from prompts. The lab’s format is the tell: prompt plus every follow-up it needed. The skill is no longer operating the timeline; it is directing the iteration loop — knowing what to ask for next, and recognising when the frame is right. That is a director’s job, not a technician’s, which is why designers who learn to direct will outrun studios that only learned to operate.

• Technique unbundles from the tool. After Effects encoded motion craft in a timeline UI; a frontier model re-derives it from language. The tool is not the skill.

• The new skill is directing iterations. Prompt, inspect, follow-up — knowing what to ask next is the craft that survives.

• Publish the prompt file, not just the output. The lab’s exact-prompt-plus-follow-ups format is what makes the craft transferable.

Google’s meeting-note Mac app that never phones home

Vijay Shekhar Sharma’s entire post is one sentence — “Did you download Google Foresight yet?” — and it pulled 1.8K bookmarks, because the thing it links to is genuinely the week’s most interesting product: Google AI Edge Foresight, an experimental Mac app that turns your shorthand meeting bullets into full notes with everything running on the device. Nothing recorded, imported, or indexed ever leaves your Mac. In the age of cloud note-takers, that sentence is the feature.

Launched on 6 October alongside DeepMind’s EmbeddingGemma 2 (a 740-million-parameter open embedding model), Foresight is Google’s showcase for on-device AI — and a direct shot at Granola. It captures your mic and system audio, so it works with Meet, Zoom, or a face-to-face meeting, and even offline. You scribble short bullets; it expands them into polished notes from the live transcript. Then it goes further: point it at project folders, PDFs, Docs, Office files, Markdown, and web bookmarks, and it builds a personal knowledge base you can chat with (“Ask”) — while Live Assistance listens mid-meeting and surfaces proactive context cards with exact citations when a topic from your files comes up. The engine room is EmbeddingGemma 2 for retrieval plus the Gemma 4 language model — cross-modal, so a search term can find a moment in a recording. Apple Silicon only, free, direct download, no cloud subscription. Google calls it an experiment; the direction it points is not.

The strategic read: this is Google answering the trust phase (see the lead story) at the product level. The agent age’s trust question — does it act rightly, and can we prove it? — has a consumer twin: where does my data go? Foresight’s answer is architectural: it cannot leak your meetings because the cloud is not in the loop. Local-first is becoming a product differentiator, not just an engineering preference — the same week Google Cloud ships a data-centre agent that reads your mail, Google Labs ships one that never touches a server.

• Privacy is moving into the architecture. “Nothing leaves your device” beats any privacy policy — it is a property of the system, not a promise.

• Shorthand-in, full-notes-out is a better UX than full-automation. Foresight keeps the human’s bullets as the skeleton and fleshes them out — the user stays the editor.

• The knowledge base is the moat. Meeting notes are table stakes; the indexed personal library plus Live Assistance citations is the compounding part.

• Watch the showcase pattern. Google’s real product here may be the Gemma on-device story — the app is the demo that proves it.

Khosla backs a “mind-reading baseball cap”

Vinod Khosla quote-tweets Rahul Chhabra’s announcement: $50M raised from Khosla Ventures, Accel, Initialized, Kevin Weil, and DST Global to build “the world’s most wearable BCI cap” at Sabi (Forbes, 9 Oct). Khosla’s entire commentary: “Time to put on your thinking caps!” Brain-computer interfaces are leaving the lab coat and putting on a baseball cap — the form factor is the thesis. 38.7K views.

“100 pairs sold in 3 minutes”

Rajan Anandan (Peak XV / Surge) teases an update from one of the companies profiled in The Lift-Off Nation: a hundred pairs sold in three minutes, and “any guesses?” No name, no link — a pure curiosity hook. The book’s companies keep producing these velocity stories; worth watching the replies for the reveal. 19.9K views.

Four non-design courses from Harvard & Stanford

Designer-builder Charmie Kapoor shares the four most fun classes she ever took — from the pre-AI days — in a 36-second video plus a YouTube deep-dive: “4 non-design courses from Harvard & Stanford.” The framing is the insight: the classes that changed a designer most had nothing to do with design. A reminder that range compounds — the best taste is imported. Weekend watch.