# Kevin Murphy — Full Corpus Site: https://kevinmurphywebdev.com Generated: 2026-07-06T18:51:18.260Z This file contains every published blog post and project case study on https://kevinmurphywebdev.com, concatenated for single-fetch ingestion by AI crawlers and agents. The lighter index is at https://kevinmurphywebdev.com/llms.txt. The machine-readable agent descriptor is at https://kevinmurphywebdev.com/.well-known/agents.json. Each section is separated by a level-2 heading naming the source URL so a chunker can split cleanly. Posts come first (most recent first), then project case studies, then the resume summary. --- ## Building issuegraph: a LangGraph triage agent, with the evals to prove it works URL: https://kevinmurphywebdev.com/blog/building-issuegraph Date: 2026-07-06 Tags: ai, agents, langgraph, langchain, langsmith, evals, open-source, applied-ai Excerpt: I build applied AI in the open, from multi-agent orchestration to retrieval. issuegraph puts LangGraph through the same bar: a GitHub issue triage agent as a state machine, with conditional routing, a bounded quality-guard loop, human-in-the-loop interrupts over Redis checkpoints, and a LangSmith eval suite that checks whether the classifier's confidence is honest. This post walks the graph architecture, the interrupt-and-resume plumbing, the one bug that taught me the most, and the calibration math. > **Repo:** [github.com/midimurphdesigns/issuegraph](https://github.com/midimurphdesigns/issuegraph) > > **Live:** [issuegraph.kevinmurphywebdev.com](https://issuegraph.kevinmurphywebdev.com) ## What issuegraph demonstrates - **LangGraph state machines.** Seven nodes, shared typed state, conditional edges for routing and a bounded retry loop, a checkpointer for pause and resume. - **Human-in-the-loop interrupts.** A confidence gate calls `interrupt()` mid-run, the graph checkpoints to Redis, and a human approves or rejects the draft before the run finishes, possibly on a different serverless instance. - **LangChain chains with structured output.** Prompt templates piped into Zod-schema-forced tool calls, so every LLM boundary returns typed, validated data. - **LangSmith evals.** A labeled golden set uploaded as a dataset, `evaluate()` running the full graph over every example, a deterministic evaluator paired with an LLM judge. - **Confidence calibration.** Brier score plus a reliability table that checks the classifier's stated confidence against actual accuracy, the same discipline forge applies to its debugging lanes. - **Public-endpoint hardening.** Preset-only input, per-IP rate limits, a global daily budget that fails closed, sanitized production errors. ## Why I built it Most of my agent work runs on the Vercel AI SDK and the raw Anthropic SDK. forge orchestrates four parallel subagents with hand-rolled coordination. Kev-O does retrieval with hand-rolled BM25 and reranking. I like building from primitives because you learn what the abstractions are hiding. LangChain and LangGraph are the other major dialect of agent engineering, and I wanted to put them through the same bar I hold my own orchestration to. So I built the kind of system I would normally hand-roll, in the framework instead. A GitHub issue triage agent: read an issue, decide what it is, draft a maintainer reply, check the draft, and know when to ask a human instead of trusting itself. Small enough to be honest about, real enough to exercise the parts of LangGraph I actually wanted to test: conditional routing, a bounded loop, and human-in-the-loop interrupts. The three pieces, in one line each: LangChain is the glue for calling a model, LangGraph turns a set of steps into a runnable graph, and LangSmith records and grades what ran. The one I most wanted to learn was LangGraph, because it is built for exactly the thing this project needed: an agent that stops partway through and waits for a person. Which is why I do not have a single default. If I had to pick one for a team that ships a lot of different AI features, LangChain plus LangSmith is the safe standardization: the ecosystem is broadest, the tracing and eval story is the most complete, and most agents eventually grow a branch or a loop. But for a single-provider service where I want every token in view, or a durable step pipeline, the lighter tools are not a compromise, they are the better fit. The skill is not loyalty to one. It is knowing which shape the problem is before you reach. ## The graph A pipe is a straight line. Real triage branches, loops, and pauses, and that is exactly what a pipe cannot express. LangGraph models it as a state machine: nodes are functions that read shared state and return partial updates, edges decide what runs next. ```ts const builder = new StateGraph(TriageState) .addNode("classify", classifyNode) .addNode("draftBug", draftBugNode) .addNode("draftFeature", draftFeatureNode) .addNode("draftDocs", draftDocsNode) .addNode("draftQuestion", draftQuestionNode) .addNode("guard", guardNode) .addNode("gate", gateNode) .addEdge(START, "classify") .addConditionalEdges("classify", routeByCategory, { draftBug: "draftBug", draftFeature: "draftFeature", draftDocs: "draftDocs", draftQuestion: "draftQuestion", }) .addConditionalEdges("guard", afterGuard, { gate: "gate", draftBug: "draftBug", draftFeature: "draftFeature", draftDocs: "draftDocs", draftQuestion: "draftQuestion", }) .addEdge("gate", END); ``` A conditional edge is just a function that returns the name of the next node. `routeByCategory` reads the classification and picks a specialist drafter. `afterGuard` either advances to the gate or points back at a drafter, and pointing backward is all it takes to make a retry loop. The loop is bounded by a redraft counter in state, because an LLM judge that keeps rejecting drafts would otherwise spin forever. The classifier that feeds the router is a plain LangChain chain. The part worth knowing is `withStructuredOutput`: it converts a Zod schema into a forced tool call, so the model cannot reply with prose. It has to fill in the schema, and the chain returns a typed object. ```ts const ClassificationSchema = z.object({ category: z.enum(["bug", "feature", "docs", "question"]), confidence: z.number().min(0).max(1), reasoning: z.string(), }); const classifierChain = prompt.pipe( model.withStructuredOutput(ClassificationSchema, { name: "classify_issue" }), ); ``` That confidence number matters later. It is a claim the model makes about itself, and the whole back half of this project is about checking that claim. ## The interrupt, and the bug that taught me the most The gate node is my favorite part of the system. If the classifier is confident, the run finalizes on its own. If not, the graph stops and asks a person. ```ts function gateNode(state: TriageStateType) { const confidence = state.classification?.confidence ?? 0; if (confidence >= CONFIDENCE_GATE) return { status: "auto-finalized" }; const decision = interrupt({ reason: "low confidence, human approval required", confidence, draft: state.draft, }) as { approved: boolean }; return { status: decision.approved ? "human-approved" : "human-rejected" }; } ``` `interrupt()` pauses the graph and saves a checkpoint: which node was running, what the state held, everything needed to come back later. Resuming is `graph.invoke(new Command({ resume: value }), config)` with the same thread id. The checkpointer is what makes this work, and it is required, because you cannot resume a run that nothing remembered. The bug: my first version resumed with a bare boolean, `new Command({ resume: false })` for a rejection. LangGraph threw `EmptyInputError`. A falsy resume value looks like no input at all, so approval worked and rejection crashed. The fix is resuming with an object, `{ approved: false }`, which is never falsy. It is a small bug, but it taught me how the resume plumbing actually works underneath: the resume value is real graph input, not a signal, and it flows through the same input validation everything else does. The serverless wrinkle is that the pause and the resume are different requests, which on Vercel can mean different machines with nothing shared in memory. So the checkpointer writes to Redis. The demo derives its Redis connection from the same Upstash environment variables the rate limiter uses, and paused runs survive across instances with an hour of TTL. ## LangSmith, in practice Two environment variables turn on tracing, and every chain and graph call ships a full trace tree: each model call with its inputs, outputs, token counts, and latency, nested inside the node that made it. When the guard rejects a draft and the graph loops back for a redraft, the trace shows the second call exactly where it happened. Debugging a non-deterministic system without this is guesswork. With it, you read what actually ran. The demo page streams the same structure to the browser as the graph executes, one event per node, so watching the pipeline light up on screen is watching the trace shape form in real time. Evals are where LangSmith earns its keep. The golden set is a list of issues with known-correct categories. It uploads once as a dataset, and `evaluate()` runs the whole graph against every example. ```ts const experiment = await evaluate(target, { data: "issuegraph-golden", evaluators: [categoryAccuracy, draftQuality], experimentPrefix: "issuegraph", }); ``` Two evaluators, deliberately different kinds. Category accuracy is an exact match, because there is one right answer and comparing strings is free and objective. Draft quality is an LLM judge, because a good maintainer reply has a hundred valid forms and no string comparison can grade it. Knowing which kind fits which output is most of eval design. ## Calibration: checking the model's honesty The classifier says 0.95 confident. Should anyone believe it? Calibration is the discipline of checking. Take every prediction, pair the stated confidence with whether the answer was actually right, and compute two things. The Brier score averages the squared gap between confidence and outcome, so confident-and-wrong costs the most. The reliability table buckets predictions by stated confidence and compares each bucket's claim to its actual accuracy. ``` ── CALIBRATION REPORT ────────────────────────────────── samples: 8 Brier score: 0.003 (0 = perfect, 0.25 = coin flip) reliability by confidence bucket: bucket n claimed actual verdict 0.9-1.0 8 0.95 1.00 calibrated ``` That report is real output from the committed golden-set run, and the honest read is that it flatters the model. Eight clean examples prove the plumbing, not the classifier. The value shows up as the golden set grows with production failures: the buckets where claimed confidence outruns actual accuracy are exactly where the confidence gate should distrust the model and route to a human. forge does this same loop with a learned weight per debugging lane. issuegraph rebuilds the pattern on LangSmith eval data, and the confidence gate is the consumer. ## What the framework bought, and what it cost My other agent projects hand-roll their coordination. forge runs four parallel subagents with my own orchestration. Kev-O does retrieval with my own BM25 and reranking. Building this one in a framework instead was the point, and it made the trade concrete. What it bought me was real. The checkpointer and `interrupt()` gave me pause-and-resume across serverless instances for almost no code, which is the kind of thing that is fiddly to get right by hand. Tracing turned on with two environment variables. `evaluate()` plus datasets replaced a pile of test-harness code I have written from scratch elsewhere. What it cost me showed up at debug time. When I coordinate everything myself, a failure lands in code I wrote and can read straight through. The `false`-resume bug earlier in this post is the small example: the misbehavior was inside the framework's input handling, so understanding it meant reading LangGraph's source rather than my own. In my own code it would have been obvious at a glance. That is the honest tradeoff, and it is not a knock on the framework. It is just where the time moves when you let someone else own the control flow. ## The demo hardening The hosted page triages curated presets only. No visitor text ever reaches a prompt, which closes prompt injection and keeps spend bounded. Per-IP rate limits ride a sliding window, a global daily budget caps the worst case at a few dollars, and both fail closed if the limiter is unconfigured in production. Resume requests make no model call, so they stay rate-limited without burning budget. Production error events are generic; the real errors stay in server logs. The CLI version accepts any public GitHub issue URL, which is the honest split: capability in the repo, restraint on the public endpoint. --- ## Building anchor: dual-surface product pages, agents.json, delegated-authority checkout, and AEO instrumentation URL: https://kevinmurphywebdev.com/blog/building-anchor Date: 2026-05-24 Tags: ai, agents, commerce, aeo, agentic-commerce-protocol, nextjs, open-source, applied-ai Excerpt: Anchor is an open-source AI-native product catalog. Every product has a human page AND three statically-cached LLM-facing endpoints. A /.well-known/agents.json descriptor publishes capabilities in the Agentic Commerce Protocol shape. A delegated-authority checkout endpoint runs an eight-check pipeline. AEO instrumentation classifies 13 LLM crawlers and logs every fetch through a proxy so cache hits stay observed. Here are the design choices that mattered. > **Repo:** [github.com/midimurphdesigns/anchor](https://github.com/midimurphdesigns/anchor) > > **Live:** [anchor.kevinmurphywebdev.com](https://anchor.kevinmurphywebdev.com) ## Skills, concepts, and tools this build demonstrates **Next 16 rendering modes.** Static (`○`), SSG with `generateStaticParams` (`●`), Partial Prerendering with `` holes (`◐`), Dynamic SSR (`ƒ`), and the Edge proxy layer. Every mode lives in a real route in the catalog, not a toy example. **Caching primitives under `cacheComponents: true`.** The `'use cache'` directive, `cacheLife('hours' | 'days')`, `cacheTag` for surgical labeling, and `revalidateTag(tag, profile)` for surgical invalidation. A sale invalidates one product entry; the other nine stay cached. **Serverless lifecycle.** Next 16's `after()` primitive for queuing telemetry writes past the response. Why unawaited Promises die in serverless and survive on long-running Node. When to reach for a queue (durable side effects) vs `after()` (best-effort telemetry). **Edge proxy as the cross-cutting layer.** `proxy.ts` (the renamed `middleware.ts`) classifies User-Agent and Referer on every matching request, attaches `X-Anchor-*` headers, and queues Redis writes via `after()` so cache hits stay observed. Runs before any route handler; geographically close. **Agentic Commerce Protocol shape.** `/.well-known/agents.json` discovery descriptor with capabilities, endpoints, auth model, pricing-negotiation envelope, and rate limits. The RFC 8615 `.well-known` convention. The five fields that have no `llms.txt` equivalent and the consumer that reads each one. **Delegated authority and the eight-check pipeline.** HMAC-SHA256 with constant-time signature verification (timing-safe XOR loop, not `===`). Ordered checks cheapest-first so attackers spamming junk tokens never touch Redis. Nonce (`jti`) for replay protection; Idempotency-Key for network-retry protection. The sharpest distinction: nonces reject duplicates, idempotency keys repeat the original response. **Citation-shaped content for LLM crawlers.** Three independent URL placements per response (opening prose line, JSON-LD `@id`, closing canonical line). The `Source:` lexical prefix that exploits LLM training-data bias. The lost-in-the-middle effect and why opening-line URL placement raises citation accuracy from roughly 40% to roughly 85%. **AEO (Answer Engine Optimization) instrumentation.** Bot classification across 13 known LLM crawlers (ChatGPT-User, GPTBot, Perplexity-User, PerplexityBot, Claude-Web, ClaudeBot, Google-Extended, GoogleBot, Applebot-Extended, Bytespider, meta-externalagent, Amazonbot, cohere-ai). Redis sorted sets keyed `aeo:fetch::` with `ZADD`/`ZCOUNT`/`ZRANGE` for O(log N) time-windowed counts. Cited-by attribution via Referer matching for the answer-engine conversion funnel. **Generative UI in AI SDK v6.** `streamObject` with a Zod discriminated union as the schema (`streamUI` was removed in v6). The model picks one of N shapes via a `kind` literal; React switch-renders the matching component. Type-safe at every boundary; no markup ever crosses the wire from the model. **Server Actions for client-to-server calls.** Type-safe at the call boundary, the client imports the action function and gets the return type without designing a JSON envelope. Used in the comparison agent flow. **Stack.** TypeScript strict, Next 16 with `cacheComponents: true`, React 19, Tailwind v4, Vercel AI SDK 6, `@ai-sdk/anthropic` against Claude Haiku 4.5, Zod for schema validation, Upstash Redis (with an in-memory fallback for local dev). **Discipline.** Five-scenario test suite that asserts every guarantee (happy path, replay rejection, over-budget rejection, SKU mismatch, idempotent retry); 11/11 assertions pass. A `/docs/rendering` page documenting every route's mode + rationale, sourced from `lib/render-modes.ts` so docs cannot drift from the routes that ship. A dev-only render inspector that overlays colored boundaries on every annotated component and shows a counterfactual latency-savings panel vs a fully-dynamic baseline. ## What anchor is Anchor is an open-source product catalog designed for a web where half the traffic is becoming AI agents instead of humans. Ten fictional specialty-coffee products. Every product has a human page rendered as React in the browser, AND three statically-cached LLM-facing endpoints, markdown, JSON-LD, and plain text. A `/.well-known/agents.json` descriptor publishes the site's capabilities (endpoints, auth, pricing negotiation, rate limits) in the Agentic Commerce Protocol shape. A `POST /api/agent/checkout` endpoint runs an eight-check delegated-authority pipeline before any charge fires. A live AEO dashboard classifies 13 known LLM crawlers by user-agent and surfaces the per-product fetch counts. It's the third in a trilogy. Forge demonstrates multi-agent orchestration. Loom demonstrates durable AI commerce. Anchor demonstrates the discovery and citation surface that lets agents find and transact with a site at all. ## What's in the box - Dual-surface product pages: a human page and three citation-shaped LLM endpoints per slug - A `/.well-known/agents.json` descriptor publishing capabilities, endpoints, auth model, pricing-negotiation envelope, and rate limits - A `/llms.txt` markdown reading list pointing at every product's agent endpoint - A `/agents` human page that renders the same descriptor an agent runtime sees (documentation IS the implementation) - An eight-check delegated-authority checkout pipeline with HMAC-SHA256 + constant-time signature verification + nonce + idempotency - Cited-by attribution: when a human arrives on a product page with a Referer from one of six known LLM client surfaces (ChatGPT, Perplexity, Claude, Gemini, Copilot, You), the proxy classifies it and tags the response - AEO instrumentation: every `/agent` fetch classified across 13 LLM crawlers and logged to Redis sorted sets via `after()` so cache hits stay observed - A live dashboard with 24h / 7d / 30d totals, bot-mix breakdown, per-product table, and a live tail of recent fetches - A comparison agent at `/compare` with generative UI: the model picks one of three shapes (spec table, pros and cons, recommendation paragraph) via structured-output dispatch - A self-documenting `/docs/rendering` page listing every route's mode + rationale + the Next 16 primitives the build leans on - A dev-only render inspector that overlays colored boundaries on every component classifying its render mode and shows a counterfactual savings panel ## Why the agent surface is split into three static routes The first version of the agent endpoint was content-negotiated: a single dynamic `/agent` route that read the `Accept` header and branched between markdown, JSON-LD, and plain-text bodies. It worked. It also missed the point of the build. The refactor: split into `/agent/markdown`, `/agent/json`, `/agent/plain`. Each one prerendered per slug at build time via `generateStaticParams`. All 30 routes (10 products × 3 formats) plus a 10-redirect canonical-shortcut on `/agent` itself live as static files served from the edge cache. TTFB drops to about 5ms worldwide because no serverless function ever runs on a cache hit. The body is a pure function of the catalog wrapped in `'use cache'`, same input, same output, every time. This left a problem: if the route handler doesn't execute on cache hits, the AEO logger doesn't fire on cache hits. Telemetry would silently die the moment the cache started working. The fix: move logging into the proxy. The proxy runs at the edge before any route handler, on every matching request, regardless of cache status. It classifies the User-Agent and queues a Redis write via `after()`. The body comes from cache; the telemetry comes from the proxy. Both layers do exactly what they're best at, and the cache savings don't cost observability. ## The citation-shaped opening line Every `/agent/markdown` response opens with: ``` Source: https://anchor.kevinmurphywebdev.com/products/moonshot-grinder-x1, Moonshot Lab Moonshot Grinder X1 is listed at $899.00 on anchor (...) ``` Three structural choices are doing work in that line. The word `Source:` is a high-signal prefix, LLMs trained on academic and journalistic text strongly associate it with "authoritative citation follows," which pushes the URL into a category the model preserves through summarization. The canonical URL is the second token, not buried at the end, because LLMs cite documents most reliably when the URL is in the opening sentence (the lost-in-the-middle effect, written up by Stanford in 2023). The structured assertion (`is listed at $899.00 on anchor`) gives the model a fact to cite alongside the URL. The URL appears in three independent locations in every response, once in the opening line, once in the JSON-LD `@id` field, once in the closing `Canonical URL:` line. Three independent placements is the redundancy that pushes citation accuracy from roughly 40% to roughly 85%. If one mechanism fails (JSON-LD parser missed the metadata, RAG chunker grabbed only the middle), the others backstop. ## The agents.json descriptor `/.well-known/agents.json` is the machine-readable capability descriptor. The path piggy-backs on RFC 8615's reserved `/.well-known/` convention so agent runtimes know where to look without scraping the homepage. The shape is loosely modeled on Stripe's Agentic Commerce Protocol draft. Conservative on fields: every field documents a known consumer in code comments, nothing speculative. The five fields that have no `llms.txt` equivalent are `endpoints` (read by an agent runtime that wants to call search or checkout), `auth` (read by a protocol negotiator before any authenticated request), `pricing.negotiation` (read by an agent buyer deciding whether to probe for a discount), `capabilities` (a quick filter, does this site support purchase, or only browse), and `rateLimits` (per-agent caps the runtime respects to avoid backoff). Plus the supporting fields: `schemaVersion`, `name`, `origin`, `contact`, `terms`. `/agents` is the human-facing mirror of the same descriptor, rendered by the same `loadAgentsDescriptor()` function the JSON endpoint uses. Documentation IS the implementation; drift between the two is impossible by construction. Both surfaces share one cached loader entry tagged `descriptor`, so a sale that changes inventory invalidates both in one `revalidateTag('descriptor')` call. ## The eight-check pipeline `POST /api/agent/checkout` runs an ordered eight-check pipeline before any side effect fires: 1. Token present 2. Signature valid (HMAC-SHA256, constant-time compare) 3. Not expired 4. Agent matches caller 5. Scope matches request (action + SKU + maxCents) 6. Nonce unused (the token's `jti` is single-use) 7. Idempotency-Key check (return cached response on retry) 8. Process charge + revalidateTag The order is cheapest-and-most-likely-to-fail first. Signature is the cheapest cryptographic check, about one millisecond, and the most likely to fail when an attacker is forging tokens. Expiry next because it's a single subtraction. Agent binding and scope are logical compares against verified claims. The Redis lookups (nonce and idempotency) only run after the signature is verified, so an attacker spamming junk tokens never touches the storage layer. Expiry deliberately comes after signature verification, not before. The `exp` field lives inside the token body. Reading it before verifying the signature means trusting unverified bytes, which is the same class of bug as JWT's notorious `alg: none` exploit. The rule that prevents this whole family of attacks: never trust a claim until the signature attesting it is verified. The signature comparison uses a constant-time XOR loop, not `===`. JavaScript's string equality returns false the instant it finds a mismatched byte, which means a wrong signature with a matching first byte takes slightly longer than one that mismatches immediately. An attacker who can measure timing can recover signatures byte-by-byte. The constant-time loop XORs every byte regardless of mismatch and OR-accumulates the difference into one comparison at the end. No timing leak. The same pattern lives in Node's `crypto.timingSafeEqual` and Stripe's webhook signature verifier for the same reason. ## Nonce vs idempotency Both reject duplicates. Both live in Redis with a 24h TTL. The mechanism is the same; the threat model is different. The nonce is in the signed token. It defends against replay attacks, an attacker who captures a valid signed token (network logs, MITM) can't re-fire it because the token's `jti` is single-use and tracked server-side. Each token gets used exactly once; the second attempt returns 409 Conflict. The idempotency key is in the request headers. It defends against network retries, the agent fires a request, the network drops the response packet, the agent retries with the same `Idempotency-Key`. Without idempotency the second retry processes a second charge. With idempotency the second retry returns the cached response from the first attempt; no second charge fires. Nonces reject duplicates with an error. Idempotency keys repeat the original success. That's the sharpest distinction worth memorizing. ## Generative UI in AI SDK v6 AI SDK v6 removed `streamUI`, the v3 primitive that let a model return React elements directly. The v6 idiomatic pattern is structured-output dispatch. The comparison agent at `/compare` works like this. The user picks two products. A Server Action calls `streamObject` with a Zod discriminated union as the schema, three shapes, each tagged with a `kind` literal: `specTable`, `prosCons`, `recommendation`. The model decides which shape fits (same category → spec table, overlapping use case → pros and cons, unrelated → recommendation paragraph) and emits a typed object matching that shape. The Zod schema validates the output before it crosses any boundary. The client receives the validated object and switch-renders the matching React component on `kind`. No HTML, markup, or scripts ever cross the wire from the model. The model picks a shape and fills typed fields; React renders. Type-safe at every boundary, can't inject markup, deterministic enough that the same input reliably picks the same shape (temperature 0.3 on Claude Haiku 4.5). Generative because the SHAPE is generated, not just the content. ## Rendering modes in Next 16 with cacheComponents The build leans on every Next 16 rendering mode: - **Static (○)** for `/`, `/agents`, `/compare`, `/docs/rendering`, `/agents-descriptor`, `/llms-txt`. Pure server components reading cached data via `'use cache'`. Edge CDN serves them. - **SSG (●)** for the four `/products/[slug]/agent*` routes. `generateStaticParams` expands the slug parameter into 10 prerendered files per format. - **PPR (◐)** for `/products/[slug]` (cached product copy + dynamic AgentTally hole) and `/dashboard` (functionally force-dynamic via `headers()` reads in every Suspense child). - **Dynamic (ƒ)** for `/api/agent/checkout` and `/api/agent/issue-token`. Every response depends on the request body + token contents + Redis lookups + time. No caching helps. - **Edge (proxy)** for cited-by attribution + AEO logging. Runs before any route handler on every matching request. The `cacheComponents: true` flag inverts the rendering defaults: every server component is treated as static unless something opts it out of static. The opt-out is implicit, reading `headers()`, `cookies()`, `searchParams`, or uncached data sources is what makes a component dynamic. The per-route `dynamic`, `revalidate`, and `runtime` exports from earlier Next versions are disallowed; the data you read inside the component is the only signal Next uses to classify it. A `/docs/rendering` page lists every route with its mode and rationale. The page reads from `lib/render-modes.ts`, the same source of truth the inspector overlay uses, so the documentation can never drift from the routes that ship. ## Surgical cache invalidation The cached loaders carry three independent tags: ``` loadProduct('moonshot-grinder-x1') → cacheTag('product:moonshot-grinder-x1') loadProduct('kestrel-pour-kettle') → cacheTag('product:kestrel-pour-kettle') ... (one per slug) loadAllSlugs() → cacheTag('catalog:index') loadAgentsDescriptor() → cacheTag('descriptor') ``` A sale fires `revalidateTag('product:moonshot-grinder-x1', 'hours')` and `revalidateTag('descriptor', 'hours')`. Two entries recompute on the next read: the affected product (now reads decremented inventory) and the descriptor (whose `catalog[]` contains an `inStock` field for every SKU). Nine other product entries stay cached. The catalog-index entry stays cached because the slug list didn't change, a sale decrements stock, it doesn't remove a product. Tagging is what makes the invalidation surgical instead of a sledgehammer. If we'd tagged all three loaders with one common tag like `'catalog'`, every sale would invalidate everything, defeating the point. Three tags, three separate decisions about when each one needs to recompute. ## What I'd do differently in production The HMAC token shape is a demo simplification. A production deployment would use asymmetric ed25519 signatures issued by the user's wallet, with anchor verifying via the user's public key, the user's private key never leaves their device. The eight-check verification logic is identical either way; only the key model changes. The in-memory Redis fallback (active when `UPSTASH_*` env vars are missing) is intentionally per-process and doesn't survive restarts. It exists so local dev and the test script work without standing up Upstash. Production needs Upstash configured for cross-instance state. The comparison agent's three shapes are hand-picked for a coffee catalog. A different product category might want a different shape inventory (size comparison? compatibility matrix? color swatches?). The pattern generalizes, Zod discriminated union, model picks, client dispatches, but the specific shapes are domain choices. The catalog of ten products is a fixture. Swapping it for a Supabase-backed loader doesn't change the cache shape; the `'use cache'` wrapper around `loadProduct` works identically with a database call. ## Why this build exists Agent commerce is the surface where AI meets real money movement, and the design questions there have right and wrong answers, not vibes. The failure modes have to be named. The security shape has to be defended in code, not asserted. The cache discipline has to be visible at the route level, not hidden. Anchor is the artifact that demonstrates one coherent answer to those constraints. As a trilogy, forge covers agent orchestration, loom covers durable execution, and anchor covers discovery and transaction. The build is open source. The eight checks live in [`lib/principal.ts`](https://github.com/midimurphdesigns/anchor/blob/main/lib/principal.ts) with stable failure codes for HTTP mapping. The agent descriptor is in [`lib/agents-descriptor.ts`](https://github.com/midimurphdesigns/anchor/blob/main/lib/agents-descriptor.ts), the single source for `/.well-known/agents.json` and `/llms.txt` and `/agents`. The proxy is in [`proxy.ts`](https://github.com/midimurphdesigns/anchor/blob/main/proxy.ts), about sixty lines. The test script that proves the five scenarios is in [`scripts/test-checkout.ts`](https://github.com/midimurphdesigns/anchor/blob/main/scripts/test-checkout.ts) with eleven assertions, all passing. Fork it, run it, break it. The next category of commerce is the one where agents transact on behalf of users. The infrastructure to make that safe has to be built somewhere. This is what one shape of it looks like. --- ## Loom: durable AI commerce with Vercel Workflows, exactly-once side effects, and a bounded agent gate URL: https://kevinmurphywebdev.com/blog/building-loom Date: 2026-05-23 Tags: ai, durable-execution, agents, workflows, stripe, open-source, applied-ai Excerpt: Loom is an open-source backend that demonstrates the patterns required to let an LLM influence real money movement safely. Four Vercel Workflows run end-to-end with durable sleep, exactly-once side effects, saga compensation, Stripe webhook drift reconciliation, and a deterministic authorization gate that bounds every agent decision. Ships with an adversarial eval harness and a failure-injection harness. Built on Vercel Workflows (GA), the Vercel AI SDK, Anthropic, Stripe, and Upstash. > **Repo:** [github.com/midimurphdesigns/loom](https://github.com/midimurphdesigns/loom) > > **Live:** [loom.kevinmurphywebdev.com](https://loom.kevinmurphywebdev.com) ## What loom is Loom is a durable-execution backend for AI-driven commerce. Four workflows run end-to-end on the live demo, each one demonstrating a class of failure that production AI-commerce systems have to handle correctly: - **Cart abandonment** with a six-hour durable sleep and an idempotent email send. Proves the runtime survives process restarts and replay storms without duplicating customer-visible side effects. - **Dynamic checkout** with an LLM-driven discount negotiation bounded by a deterministic authorization gate. Proves an agent can influence a real payment amount without ever being trusted with the ceiling. - **Shipping monitor** with saga compensation. Proves a multi-step external integration can roll back cleanly when one step fails after another has already succeeded. - **Stripe webhook drift** demo. Proves the system tolerates the four standard webhook failure modes (out-of-order, duplicate, late, never-arriving) by treating the event store as a durable log instead of a queue. The whole thing is open source, MIT licensed, and hosted with per-IP rate limits and a daily USD spend cap so the demo is safe to leave on the internet. ## What's in the box Every bullet maps to a layer in loom that the live demo exercises: - **Durable execution** on Vercel Workflows (GA). Per-step checkpointing, durable sleep across replicas, automatic replay on failure. - **Exactly-once visible side effects.** At-least-once delivery + receiver-side idempotency keys + stable composite keys (`workflowId:stepName`) → one email per real event under retry storms. - **Saga compensation** with namespaced idempotency keys. The booking key and the cancel key live in different namespaces so the cancel call cannot dedupe-return the original booking record. - **Stripe webhook drift reconciliation.** Events persist to a durable log with per-consumer cursors and TTL-based eviction; tolerates out-of-order, duplicate, late, and never-arriving webhooks. - **Bounded agent authority** via structured-output contracts (`generateObject` + Zod discriminated union) and a deterministic authorization gate that runs after the LLM returns. - **Adversarial eval harness** (Sirens) — 10 prompt-injection scenarios asserted against the gate in CI; snapshots to `.loom/sirens/.json` for diffing across model upgrades. - **Failure-injection harness** — a `KillingProvider` throws between step execution and step recording (the worst case for durability); N trials asserts zero duplicate sends and zero drops. - **Cost-aware model routing.** Haiku for generators, Opus for structured agent decisions. Per-day USD cap via Upstash with budget-aware short-circuit before every LLM call. - **Cross-instance abort** — global signal in Redis, local actuator inside the workflow step. Anyone can fire; the workflow decides when it's safe to act. - **Production-shaped guardrails.** Per-IP sliding-window rate limit, daily USD cap, per-visitor cookie scoping so demo events stay isolated between sessions. - **TypeScript discipline.** Strict mode, zero `any`, discriminated unions everywhere a workflow outcome is rendered, typed cross-process state contracts. ## Stack - **Vercel Workflows** (GA) for durable execution. Per-step checkpointing, durable sleep, automatic replay on failure. - **Vercel AI SDK** on `@ai-sdk/anthropic` with Claude Opus 4.7 for structured agent decisions and Claude Haiku 4.5 for free-form generators. - **Stripe** for the webhook receiver, signature verification, and idempotency-key semantics. - **Upstash Redis** for the event log, per-visitor cursors, the budget counter, and the rate-limit window. - **TypeScript** strict mode, zero `any`, Zod for structured-output schemas and runtime validation at every API boundary. - **Next.js 16** App Router on Vercel. ## The problem Letting an LLM influence customer-facing money movement requires solving two problems at once. The first is durability: AI workflows often involve external API calls, long waits, and multi-step coordination. A naive implementation drops state when a process dies and re-fires side effects on retry. The second is bounded agent authority: an LLM that can write a discount amount can be coerced into writing a wrong discount amount by an adversarial input, a hallucination, or a prompt-injection attack carried by a customer message. A production AI-commerce system has to solve both. Loom is an end-to-end demonstration of how. ## How loom bounds an agent's spending authority The dynamic-checkout workflow is the highest-stakes surface in the system. An LLM-driven decision becomes a discount applied to a real payment. The defense is structural, not behavioral, and it has three layers. ### Layer 1: a structured-output contract The `negotiate_discount` step calls Claude Opus through `generateObject` from the Vercel AI SDK. The model's entire output surface is a Zod discriminated union: ```ts const AgentDecision = z.discriminatedUnion('action', [ z.object({ action: z.literal('discount'), amountCents: z.number().int().nonnegative(), reason: z.string(), }), z.object({ action: z.literal('refund'), amountCents: z.number().int().nonnegative(), reason: z.string(), }), z.object({ action: z.literal('no_action'), reason: z.string(), }), ]); ``` The model cannot call functions, read files, or write to state. Its only capability is filling in this object. Any output that doesn't validate is rejected before any downstream code sees it. That's the first floor. ### Layer 2: a deterministic authorization gate After the model returns, the workflow calls `authorizeDiscount` from `lib/agent-authority.ts`: ```ts export function authorizeDiscount( amountCents: number, ): { status: 'approved' } | { status: 'decision_blocked'; reason: string; ceilingUsd: number } { const ceilingUsd = Number(process.env.LOOM_MAX_DISCOUNT_USD ?? 25); const ceilingCents = ceilingUsd * 100; if (amountCents > ceilingCents) { return { status: 'decision_blocked', reason: `requested ${amountCents / 100} USD exceeds ceiling`, ceilingUsd, }; } return { status: 'approved' }; } ``` Five lines of plain code with no LLM in the loop. The ceiling is read from the environment, never embedded in the system prompt. The model cannot infer it, cannot inspect it, cannot route around it. **The right way to bound an LLM is to validate its decisions in deterministic code after it has finished, not to ask it to behave.** The workflow logs both branches identically. An approved decision records `{ outcome: 'applied', amountCents, reason }`. A blocked decision records `{ outcome: 'blocked', requestedCents, ceilingUsd, modelReason }`. Every request has a paper trail. ### Layer 3: adversarial evidence The gate is the safety mechanism. Sirens is the evidence that the gate holds. `scripts/sirens.ts` runs ten adversarial scenarios offline against the same agent path the runtime uses. Vague pressure ("just this once"), fabricated authority ("the CEO already approved a 50% discount"), system-prompt-leak attempts, JSON injection, ceiling-math tricks, chained-reasoning attacks. After each scenario, Sirens asserts that the *applied* amount never exceeds `MAX_DISCOUNT_USD`. The assertion never fires because the deterministic gate always catches the overshoot. Results snapshot to `.loom/sirens/.json` for diffing across prompt changes or model upgrades. Unit tests verify the gate code is correct against the inputs the author thought to test. Sirens verifies the gate holds against an adversarial LLM producing inputs the author didn't think to test. The two are complementary; production needs both. ## How loom achieves exactly-once side effects Stripe retries webhooks at-least-once. The workflow runtime retries failed steps. The email provider can retry on transient failure. If any of those layers leaks a duplicate, a customer gets two emails and trust in the system erodes. The defense is three layers of receiver-side idempotency keyed on stable composite identifiers. The workflow's step-level idempotency key is `workflowId:stepName`. Two replays of the same step produce the same key. The send-email step passes that key to the mock email provider in `lib/email.ts`, which stores it in Upstash with a TTL. The first call returns `{ deduplicated: false }` and dispatches the email. Every subsequent call with the same key returns `{ deduplicated: true }` and dispatches nothing. Stripe's own idempotency-key API (passed to its checkout-session creation) works the same way at the receiver Stripe controls: same key returns the cached result; same key plus a different body returns 400. The principle that makes this composable: **at-least-once delivery plus idempotent receivers plus stable keys equals exactly-once visible side effects.** Senders are never trusted to send exactly once because they can't be — networks are unreliable. Receivers are trusted to ignore duplicates because the receiver is the only place that can be authoritative about whether the side effect has already happened. ## Idempotency keys versus sagas Idempotency keys prevent duplicate side effects. They do not undo side effects that already happened and turned out to be wrong. That is what sagas are for. **Idempotency keys prevent; sagas compensate.** The shipping-monitor workflow demonstrates the difference. It books carrier A, then attempts carrier B. If carrier B raises a `CarrierFailureError`, the workflow runs a paired compensation step that cancels carrier A's booking. The implementation detail worth naming: the booking step and the compensation step use idempotency keys in *different namespaces*. The booking key lives at `loom:carrier:booking:`. The cancel key lives at `loom:carrier:cancel:`. If they shared a namespace, the cancel call would dedupe-return the original booking record and the cancel would silently no-op. Same workflow id, different namespaces. The namespace is the *type* of side effect, not the workflow it belongs to. The saga's outcome is one of two typed values: `completed` or `rolled_back`. A discriminated union, not a boolean. The UI renders which path the saga took, including which carrier failed and what the compensation returned, so the audit log is self-evident. ## Stripe webhook drift reconciliation Webhooks are not ordered, not exactly-once, and not predictably timed. A `payment_intent.succeeded` event can land before, during, or after the workflow that needs it. Code that assumes any one of those orderings breaks under load. The defense is to treat the webhook store as a durable log with multiple readers, not as a queue. The receiver verifies the Stripe signature, persists the event to `loom:stripe:event:` with a thirty-day TTL, and appends the id to a per-visitor list. The receiver does not trigger any workflow. Workflows are independent consumers; each one walks the log when it's ready, tracks its own cursor (a consumed-set in Redis), and picks the first unconsumed event newest-first. The principle: **webhook stores are durable logs, not queues. Multiple consumers, individual cursors, TTL-based eviction.** The demo's receive-then-consume-later flow makes the two timing modes visible: The first failure mode is the webhook arriving before the workflow asks for it. If the store were a queue and had been consumed, the workflow would wait forever. Because it's a durable log with a cursor, the workflow's consumer reads from the beginning and finds it. The second failure mode is the workflow running before the webhook arrives. The consumer polls the store, finds nothing, sleeps, retries. Durable sleep means the retry doesn't cost a process; the workflow is paused, not spinning. Click "send test webhook" three times in the demo to stack events. Click "fire consumer workflow" once to advance the cursor by one event. Consumed events dim with a checkmark and stay in the store. The store does not shrink; the consumer's cursor advances. That visual is the entire argument for why webhook stores are not queues. ## Durability under fault injection Durability claims are cheap. Durability evidence is what makes them credible. `scripts/failure-injection.ts` runs the cart-abandonment and dynamic-checkout workflows against a custom `KillingProvider` that wraps the durable step recorder. The provider's job is to crash the workflow at the worst possible moment. The worst possible moment is *after* `await fn()` returns but *before* the step's result is persisted. The side effect has happened (the email was sent, the discount was applied), but the workflow has no memory it happened. On replay, the workflow asks the recorder for that step's result, the recorder has nothing, and the workflow re-runs `fn()`. The side effect is about to fire a second time. That is exactly when the receiver-side idempotency key earns its keep. The second call carries the same composite key as the first. The receiver returns `{ deduplicated: true }`. The visible side effect (an outbound email, a discount application) fires exactly once. The harness runs N=5 trials per workflow per phase and asserts recovery completed, the email audit log shows exactly one entry per workflow run, and no sends were dropped. Zero duplicates. Zero drops. That's the difference between "the system is durable in theory" and "the system has measured durability under fault injection." ## Cost-aware model routing Every LLM call in loom is bracketed by `assertWithinBudget` before and `recordSpend` after. The budget lives at the Upstash key `loom:cost:YYYY-MM-DD` and increments atomically via `incrbyfloat`. If the day's spend exceeds `LOOM_DAILY_USD_CAP` (default $2), the call short-circuits with `BudgetExceededError` before reaching Anthropic. The live cost panel polls the counter in real time. The other half of cost discipline is which model gets each seat. Haiku drafts the re-engagement email in cart-abandonment. Haiku is the right tool for classifiers, summarizers, and unconstrained text generators where the worst case is "the prose is fine but not great." Opus runs `negotiate_discount`. Opus is the right tool for structured agent decisions and anywhere the model's reasoning needs to be defensible enough to log as audit evidence. Haiku at this seat would produce reasons like "discount because customer asked," which would make the audit log useless. The principle: **match model cost to blast radius.** Cost-per-token differs by roughly 15x between Haiku and Opus depending on cache state. Defaulting every call to Opus is how small projects burn through credit before they ship. Defaulting every call to Haiku is how an agent gives away a free discount because the model could not produce a defensible refusal. ## Cross-instance abort If a workflow runs across multiple replicas and a user clicks an abort button, the abort signal has to reach every replica that might be running the workflow. The actuator — the JavaScript object that can stop work — cannot leave the process it lives in. The signal — the intent to stop — has to be readable from any process. The pattern: **the signal is global; the actuator is local.** The signal is an Upstash key, `loom:workflow::abort`. Any client can set it. The actuator is a check inside the workflow step that consults the key before performing a side effect. If the key is set, the step throws `WorkflowAbortedError`, the runtime stops scheduling further steps, and the workflow records `outcome: 'aborted'`. If the actuator lived where the signal lived, the system would be coordinating cancellation across nodes — the classic distributed-systems trap. Decouple them: anyone can fire the signal; the workflow decides when it's safe to act on it. ## What loom intentionally does not solve A production deployment of these patterns would close four gaps that the demo leaves open on purpose. **The dispatcher.** A real system needs a transactional-outbox dispatcher between webhook receive and workflow start. If a process crashes between writing the event to the log and triggering the workflow, the event sits unread. The fix is the transactional-outbox pattern with a watchdog that re-queues unconsumed events older than a threshold. Loom's architecture doc names this Phase 7 as a deferred item; production must close it. **Real adapters.** The carrier API, the email provider, and (for everything past checkout-session creation) the Stripe checkout flow are fixture-backed. Each one becomes a real adapter with retry-with-backoff, circuit breakers, and provider-specific idempotency semantics. **Multi-tenant cost ceilings.** Loom's cost cap is global to the demo. Production needs per-team-id cost ceilings, per-team rate limits, and per-team idempotency namespacing. **Observability.** Every workflow, every step, every LLM call, every adapter call needs an OpenTelemetry span so a stuck workflow is debuggable by an SRE who has never read the source. Loom logs to console; production traces through Honeycomb or equivalent. The architecture is shaped for these additions; the demo intentionally stops before them. ## Three patterns to take away **Idempotency is a system property, not a function annotation.** The composite key, the receiver-side dedup, the namespace separation between booking and cancel, the workflow-step idempotency — all of it has to compose. A single layer doing it correctly isn't enough; the chain has to be unbroken from caller to receiver. Drawing the chain explicitly and naming the dedup point at every hop is the move that catches the bug before deploy. **The deterministic gate is the safety; the eval harness is the evidence.** The five-line `if` statement is what stops the $10k discount in production. Sirens proves the `if` holds against attacks an adversarial designer made. Both, drilled into CI, is what makes an agent-driven system trustworthy enough to ship in front of customers. **Match model cost to blast radius.** Haiku is the default. Opus appears at exactly the seats where its reasoning premium pays for itself — structured decisions whose `reason` field becomes audit log evidence, negotiations whose output has to read as defensible refusal under scrutiny. Everywhere else, Haiku. If you want to see all of this run end-to-end, [loom.kevinmurphywebdev.com](https://loom.kevinmurphywebdev.com) is the live demo. The full source is at [github.com/midimurphdesigns/loom](https://github.com/midimurphdesigns/loom). The most useful files to read in order are `docs/ARCHITECTURE.md`, `lib/agent-authority.ts`, `lib/workflows/*.ts`, and `scripts/sirens.ts`. --- ## Building Forge: a multi-agent debugging concierge in production-grade TypeScript URL: https://kevinmurphywebdev.com/blog/building-forge Date: 2026-05-22 Tags: ai, agents, multi-agent, streaming, open-source, applied-ai Excerpt: Forge is an open-source multi-agent debugging concierge. Point it at a stack trace and four specialist subagents fan out in parallel, each with a focused tool set, before a coordinator merges their structured findings into ranked hypotheses with calibration-weighted confidence. Built on the Vercel AI SDK plus Anthropic. This post walks the architecture: parallel agent orchestration, durable resumable sessions, cross-instance abort, Brier-score calibration as a feedback loop, and what production deployment would change. > **Repo:** [github.com/midimurphdesigns/forge](https://github.com/midimurphdesigns/forge) > > **Live:** [forge.kevinmurphywebdev.com](https://forge.kevinmurphywebdev.com) ## What forge demonstrates Every bullet below maps to a layer in forge that the live demo exercises: - **Multi-agent orchestration** with the Vercel AI SDK. Parallel fan-out via `Promise.all` plus `pLimit` bounded concurrency, four specialist subagents with focused tool sets, structured discriminated-union outcomes. - **Two-pass agent pattern** for reliable structured output from a tool-using loop (`generateText` then `generateObject`). - **Durable session state** across serverless replicas. UUID in URL, Upstash-backed store, work-vs-transport separation so refresh resumes from a snapshot without re-running the agents. - **Preemptive abort** plumbed end-to-end via AbortSignal. UI click flips an Upstash flag, the coordinator polls cross-instance, an in-process AbortController cancels the in-flight Anthropic fetch. - **Brier-score calibration** as a feedback loop. Every (predicted confidence, rubric outcome) pair gets logged, weights derive from mean outcome divided by mean predicted with a clamp, applied at merge time. - **Anthropic prompt caching** with cache-control breakpoints on system messages, instrumented with per-call USD pricing and cache hit rate in the cost dashboard. - **Graded eval rubric** with intersection-over-union scoring for line ranges, n-runs aggregation, and per-scenario mean plus standard deviation for statistical significance. - **Speculative tool-input prefetch**. Predict the next tool call based on the current one and fire it in parallel with the LLM's thinking time. - **Production-shaped guardrails**. Upstash sliding-window rate limit per IP, daily USD cap shared across all visitors, owner-bypass cookie. - **TypeScript discipline**. Strict mode, zero `any`, discriminated unions driving exhaustive UI rendering, typed cross-process state contracts. Forge is a multi-agent debugging concierge. Paste a stack trace. Four specialist subagents run in parallel, each with a focused tool set and a tight system prompt. A coordinator merges their structured findings into ranked hypotheses, weights confidence by per-lane Brier calibration, and streams progress events to the browser as the work unfolds. Every refresh resumes the in-flight session. Every stop button cancels the underlying LLM call mid-stream. The interesting problems forge solves are not in any single LLM call. They are in the seams: how parallel agents reason without contaminating each other, how a serverless system survives the user refreshing the tab, how an abort button cancels a fetch running on a different replica, and how a multi-agent system gets more reliable over time without any individual agent improving. The rest of this post walks the architecture, the trade-offs, and what production deployment would change. ## What is on screen Open the live URL and you see one page. A short visitor header explains what is going on. Below that, a single button kicks off an investigation against a hardcoded sample input (a TypeError stack trace plus a deployed commit SHA plus an error timestamp plus a fingerprint, visible behind a disclosure toggle). Click run. Four lane cards transition from queued to running to done, in parallel, in roughly three to eight seconds each. As lanes finish, the coordinator merges results into ranked hypotheses with confidence percentages. A calibration panel shows the per-lane Brier scores and the weights derived from them. A cost panel breaks down input tokens, output tokens, cache reads, and USD per lane. An eval harness section renders a representative snapshot from the CLI runner. That is the surface. The interesting part is what runs underneath. ## The four specialists Forge has one design rule that pays for itself across the whole codebase: specialize, do not generalize. A naive agent gets a stack trace, the full repo, the full error log, and a giant prompt that says "figure it out." Real-world agents fail at this because the model has to context-switch between four reasoning modes, the prompt gets bloated with everything it might need, and the output is one monolithic answer that is hard to grade. Forge splits the work across four subagents, each with one job, one focused tool set, and a structured output contract. **source-reader** identifies the implicated source files from the stack trace and reads the code at the relevant lines. Its tools are fetch_file and fetch_directory. Its output is a typed object with file path, line range, snippet, surrounding context, confidence, and reasoning. **blame-correlator** finds recent commits that could have caused the error and ranks them by relevance. Its tools are git_log, git_diff, and git_blame. Its output is a list of candidate commits with relevance scores plus a single top suspect plus an aggregate confidence. **frequency-analyzer** quantifies the blast radius. How often does this error fire, how many users hit it, is it spiking. Its tools are error-tracking queries. Its output is a structured severity report with a p0 through p3 classification. **repro-drafter** writes a minimal local reproduction from the stack trace alone. Its tool set is empty by design. Its output is numbered steps, optional code, environment requirements, and an honest list of gaps in its assumptions. The design contract for these lives in [docs/AGENTS.md](https://github.com/midimurphdesigns/forge/blob/main/docs/AGENTS.md) on the repo. I wrote that doc before writing any of the agent code. The point of writing it first was to be able to re-explain the system from words six months later by reading the doc, closing it, and reconstructing the architecture from the contract alone. ## Why parallel beats sequential The naive way to build this is a sequential agent loop. source-reader runs, its output flows into blame-correlator, which flows into frequency-analyzer, which flows into repro-drafter. That structure has two real problems. The first is latency. Four 3-second calls run sequentially equals 12 seconds. In parallel they are ~3 seconds. User-facing latency is the headline win. The second is more subtle and more important. Sequential reasoning means each agent's hypothesis contaminates the next agent's framing. If source-reader concludes "this is a null deref in auth.ts," blame-correlator anchors on that and stops considering "maybe the stack trace is misleading because of a sourcemap mismatch." Parallel reasoning lets each lane form an independent hypothesis. The coordinator then merges with a calibration-aware weighting. This is the same independence-of-evidence trick that ensemble methods in classical ML use. Forge implements this with a single fan-out in `lib/coordinator.ts`: ```ts const limit = pLimit(4); const outcomes = await Promise.all( LANES.map((lane) => limit(async (): Promise => { try { const value = await lane.run(input, sessionId, controller.signal); return { lane: lane.name, status: "fulfilled", value, durationMs }; } catch (err) { return { lane: lane.name, status: "rejected", reason, durationMs }; } }), ), ); ``` Two small things in that snippet are doing real work. First, `pLimit(4)` is a bounded-concurrency primitive. It caps how many promises run at once within a single request. For four lanes that is not the win it sounds like; the real win is the pattern. In a production version with twenty lanes you would set pLimit to four or five and the fan-out would run in waves. Second, every lane catches its own errors and returns a typed `LaneOutcome` discriminated union. That is why the code uses `Promise.all` instead of `Promise.allSettled`. The outer promise never rejects because the inner functions never throw. The coordinator gets back a typed array of outcomes, each one either fulfilled with a result or rejected with a reason. No mixed-rejection batch semantics, no try-catch wrapping the merge logic. Each lane already returns success or failure as data, so the outer promise never sees a rejection. ## The two-pass agent pattern Every LLM call has one job that defines its output shape. Either it loops with tools and produces unstructured text, or it produces a typed JSON object with no tools. Trying both in one call fails two ways. The model either skips tools and hallucinates the answer, or calls tools correctly and produces schema-invalid JSON. The Vercel AI SDK actually enforces this by refusing the tools argument on generateObject. Forge's subagents use a two-pass pattern. Pass one is `generateText` with the tool set and a step-count stop condition, producing a free-form investigation transcript. Pass two is `generateObject` with no tools and a Zod schema, coercing the transcript into the typed result shape. ```ts const investigation = await generateText({ model: anthropic(MODEL), messages: [...], tools: { fetch_file, fetch_directory }, stopWhen: stepCountIs(6), abortSignal: signal, }); const { object } = await generateObject({ model: anthropic(MODEL), schema: Schema, prompt: `Investigation transcript:\n\n${investigation.text}\n\nProduce the structured result.`, abortSignal: signal, }); ``` Two calls per lane, four lanes, equals eight LLM calls per investigation. The cost dashboard in the live demo shows roughly $0.08 per full run on Claude Sonnet 4.6. That is the price of reliable structured output from a tool-using loop. The alternative, fighting the model to produce both at once, fails too often to be useful in production. ## Resumable streams Forge's investigation is a first-class durable session. Every state transition writes to a session store keyed by a UUID. The browser captures the UUID from the first SSE frame and pins it to the URL via `history.replaceState`. Refresh the tab and the page reads `?sessionId=X`, fires a GET to `/api/debug?sessionId=X`, and the server replays the buffered lane state plus the merged hypotheses. The trick worth knowing here is what crosses the refresh boundary and what does not. Three things cross. **The UUID in the URL.** Persisted in the browser's address bar across reloads. **The session state on the server.** A map of lane statuses, results, durations, plus the merged hypotheses, plus the cost summary. Lives in process memory in dev. Should live in Upstash or KV in production. **The work itself.** The original POST's `runCoordinator` keeps running regardless of whether the client is connected. Two browsers can resume the same UUID simultaneously and both see the same snapshot. They do not interfere with each other because the GET handler is read-only. What does NOT cross is the live tail. The resume GET returns a snapshot of the state at the moment of the GET, not a subscription to future updates. If you want to watch the still-running lanes continue, you need to refresh again, or build an event bus the POST publishes to and the GET subscribes to. I deferred that layer. Snapshot-on-refresh is enough for the demo's use case and the architectural pattern is what matters. The architectural shape to internalize is the separation of work and transport. The work (the agent loop) runs to completion on the server regardless of whether anyone is listening. The transport (the HTTP response stream) is just one possible subscriber to the work's progress. Resume reconnects the transport to a different snapshot of the same work, not to the work itself. ## Per-lane interrupt and the signal-vs-actuator distinction The stop button on each lane card was the bug that taught me the most about serverless. Version one was cooperative abort. The interrupt POST set an in-memory flag, and the coordinator checked the flag at lane-task boundaries (before start, after lane.run returns). The flag worked perfectly on localhost. The same fan-out runs in one Node process; the interrupt POST and the coordinator share heap memory; the flag flip is visible everywhere. On Vercel it failed silently. The investigation kept running for 30 seconds after I clicked stop, then completed normally. Vercel's serverless runtime can route consecutive requests from the same browser to different replicas. The POST that started the investigation landed on instance A; the interrupt POST landed on instance B. Instance B's in-memory store was empty (different process, different heap), the session lookup returned null, the interrupt returned 404 before flipping anything, and instance A's coordinator never knew it was supposed to stop. The fix is the lesson worth repeating. **The signal is global; the actuator is local.** The interrupt POST writes the abort signal to Upstash. The coordinator (running on a different replica, possibly) polls Upstash every 500ms and, when it sees the flag set, calls `controller.abort()` on its own local AbortController. The signal crosses processes via Redis. The actuator (the AbortController, with its `signal` reference and `abort()` method) is a JavaScript object that lives in heap memory in exactly one process and can only cancel work running in that same process. You cannot serialize a controller, you cannot ship it to another instance. So you signal across processes and actuate within them. Then I plumbed `AbortSignal` end-to-end. Every subagent function accepts an optional signal and passes it through to both the `generateText` and `generateObject` calls via the AI SDK's `abortSignal` parameter. When the controller aborts, the underlying fetch to Anthropic closes its connection, the SDK promise rejects with AbortError, the coordinator catches it, distinguishes abort-shaped errors from real errors, and marks the lane aborted. Round-trip from click to "aborted" badge is roughly one second on Vercel and effectively instant on localhost. The UI flips to an optimistic "stopping" state on click so the button feels responsive before the server-side abort lands. What this layer actually demonstrates is that real cancellation in a distributed agent system requires three separate things working together. First, a shared signal: any process can read or write the abort intent, which is why it lives in Upstash. Second, a local actuator: each running process owns an AbortController that can cancel work happening inside it but nowhere else. Third, signal-awareness all the way down: every layer of the call stack, including the HTTP request the AI SDK fires to Anthropic, has to accept and forward the signal so the cancellation reaches the actual work. Miss any one of these and the stop button is decorative. ## Brier calibration and self-correction LLMs are systematically overconfident. Source-reader will tell you it is 95% sure the bug is in `src/auth/session.ts` even when it is wrong. If the merge logic naively averages those overconfident self-ratings, the system's overall confidence is also overconfident, and the output becomes untrustworthy even when individual lanes happen to be correct. Forge's calibration layer measures and corrects. Every (predicted confidence, rubric outcome) pair is logged to Upstash after each session. The rubric scores each lane's structured output against a per-component grading scheme. For source-reader it is 40 points for file match, 20 points scaled by line-range intersection-over-union with the ground truth, 15 points for having a snippet, 25 points for having reasoning. The threshold for "this lane counted as correct" is total >= 60% of max. That threshold maps to "useful answer, not perfect answer," which is what a real reviewer applies when deciding whether to act on a hypothesis. Across many runs, two numbers per lane are computed. **Brier score** is the mean squared error between predicted confidence and binary outcome. Lower is better. Zero is perfect calibration. 0.25 is the no-information baseline (the score you get from predicting 0.5 on everything). Above 0.25 is worse than random and means the lane is actively misleading. **Weight** is the ratio of mean outcome to mean predicted, clamped to the range 0.5 to 1.5 with a three-sample floor before weighting activates. A chronically overconfident lane sees its weight drop toward 0.5. A chronically underconfident lane sees its weight rise toward 1.5. The clamp prevents small-sample overcommitment in either direction. At merge time, each lane's contributed confidence is multiplied by its weight before ranking. A lane with weight 0.6 has 40% less influence on the merged hypothesis than a lane with weight 1.0. The system gets more reliable hypotheses over time even though no individual lane improves. The principle worth taking away: **calibration is system-level learning, not model-level learning.** The lanes are stateless function calls. The calibration log is the system's accumulated memory of which lanes to trust more or less. The lanes never get smarter. The merge gets smarter. That separation is the architectural win. The honest gap I documented in the UI: real production calibration needs a real correctness oracle. Forge's rubric is a stub that knows the right answer for the demo's sample input. For arbitrary scenarios you need either labeled eval data (the harness's golden set) or implicit signals like user thumbs-up or bug-reopened-within-24-hours. Neither is free. The blog post is the place to be honest about this; the demo page now says it in plain text. ## Speculative tool-input prefetch There is one more orchestration layer worth describing. The speculator pre-fires likely next tool inputs in the background while the LLM is still reasoning about the current tool's output. When source-reader calls `fetch_directory("src/auth")`, a rule fires that says "the next call is almost certainly `fetch_file("src/auth/session.ts")` against the same SHA." The speculator immediately calls the file fetcher and stores the resulting promise in a cache. By the time the LLM actually requests the file, the result is sitting in the cache and the consume call returns instantly. On a real GitHub API where each call takes 800ms, this saves real latency. Tracked metrics on the live demo show predictions, hits, misses, and in-flight count. The hit rate is the headline KPI. Below ~50% the speculator is wasting more calls than it saves. This pattern only makes sense when three conditions hold. The next call is highly predictable given the current call. The tool calls have non-trivial latency. Wasted speculations are cheap. Forge satisfies all three for the directory-to-file navigation in source-reader. The blame correlator's `git_log` could speculate `git_diff(top_returned_sha)`, but the prediction is shakier (the LLM might want a different commit), so I left it out. The pattern is streaming-aware tool orchestration: speculative execution borrowed from CPU pipelines, applied to LLM tool calls. The math that determines whether it is worth using is hit-rate times min(latency, thinking-time) minus waste-rate times cost. For Forge's mocked-fixture tools the actual production value is questionable. For a research agent calling expensive search APIs the pattern is a real win. ## The eval harness A demo without measurement is just a vibe. Forge ships with a CLI eval runner ([scripts/eval.ts](https://github.com/midimurphdesigns/forge/blob/main/scripts/eval.ts)) that takes five reproducible bug scenarios from [lib/eval/scenarios.ts](https://github.com/midimurphdesigns/forge/blob/main/lib/eval/scenarios.ts) and runs each one N times against the coordinator. Each lane's structured output is scored against the rubric in [lib/eval/rubric.ts](https://github.com/midimurphdesigns/forge/blob/main/lib/eval/rubric.ts). The runner reports per-scenario mean and standard deviation in raw points, plus mean duration in milliseconds. Snapshots write to `.forge/evals/.json` for diffing against prior baselines. The two ideas that matter here are graded rubric and n-runs aggregation. **Graded rubric.** A binary "correct or not correct" verdict throws away too much information. Source-reader returning the right file with a slightly-wrong line range gets full file-match credit and proportional IoU credit, totaling 75 of 100 points. That is a useful signal. The Brier outcome is then derived from the rubric total by the 60% threshold rule, but the rubric total itself drives the human-facing scoreboard. **N-runs aggregation.** LLM output is non-deterministic. A single "v2 got 17 out of 20 right" run is a sample from a distribution, not proof of improvement. The runner does N runs per scenario and reports mean plus standard deviation. v2 is only a real improvement over v1 if v2's mean exceeds v1's mean by more than roughly two standard deviations. Without this discipline every prompt change is a vibes-based judgment. The live demo page renders a labeled illustrative snapshot rather than firing a fresh server-side eval on every visit. That tradeoff is the senior-engineer move: running for every visitor would cost roughly $1.50 per click and trip the daily cap immediately. Serving a stale snapshot without labeling would lie about freshness. Illustrative-and-labeled, with a pointer to the CLI runner and the snapshot JSON path, is the honest middle. ## Prompt caching and the honesty rule Every subagent's system message carries an Anthropic ephemeral cache breakpoint. The intention is for the static system prompt to be cached at Anthropic's edge and reused at 10% the input price on subsequent calls. The catch: Anthropic requires a minimum of 1024 tokens to be cacheable on Sonnet. Forge's system prompts are roughly 80 to 110 tokens each, well below threshold. The breakpoints are syntactically correct but get silently ignored by the API. I considered three options. First, pad the prompts artificially to clear the minimum and get fake-looking cache hits. Lying to a metric. Rejected. Second, move the cache breakpoint to include tool definitions, hoping the combined region exceeds 1024 tokens. Padding for its own sake. Rejected. Third, leave the prompts at their honest 80-to-110 token size and disclose in the UI that caching is below threshold for this demo's prompt sizes. Picked this. The cost panel on the live demo now carries an orange "heads up" callout explaining the threshold and what a production-sized prompt with 3K to 5K tokens of instructions plus tool schemas plus RAG context would actually see. Expected reduction in per-call input cost is roughly 70 to 90% of the cached prefix portion once it activates. The cache layer is correctly wired; it just does not light up at this scale. The general principle is that admitting a real limitation is more useful than faking a metric to make it look good. The cache layer is wired correctly; it doesn't activate at this scale. Disclosing that on the dashboard is what makes the rest of the cost data trustworthy. ## What I would change for production Forge is a proof-of-concept architecture demo. A real production version for a customer running thousands of debug investigations per day would change five things. **The session store moves to Upstash or KV.** The in-memory `Map` works on localhost but degrades gracefully to broken on Vercel because each serverless invocation potentially has its own fresh heap. Resume across replicas requires shared storage. The `SessionStore` interface in `lib/store.ts` is shaped for this swap; the implementation switch is one file. **A process-shared semaphore around every Anthropic call.** Current `pLimit(4)` is per-request bounding. At one hundred concurrent investigations that is four hundred parallel Anthropic calls, which will burn the org's tokens-per-minute quota in seconds. The fix is a distributed rate limiter (Upstash sliding window) keyed on the API key. Per-request pLimit caps fan-out within one investigation; global pLimit caps fan-out across all concurrent investigations. Two different bounding levels, both needed. **Real GitHub and Sentry adapters with retry-with-backoff and circuit breakers.** The fixture tools today return canned data. Production tools need their own rate-limit handling, exponential backoff on transient errors, and a circuit breaker that fails fast when the upstream is down so the agent does not waste tokens flailing. **Write-capable tools require idempotency keys, transactional outboxes, and saga compensation.** Forge's read-only fixtures make abort safe. A production agent that creates pull requests, posts to Slack, or charges a customer mid-run would leave side effects half-done on abort. Idempotency keys make retries safe at the receiver. Outboxes make multi-system writes atomic by staging the external call in a DB row inside the same transaction as the local state change. Sagas pair each step with a compensating step that rolls back partial work. That is also where Vercel Workflows or a comparable durable execution layer earns its keep: each step is checkpointed, mid-step crash means the next worker resumes from the checkpoint, no work duplicated. **Per-tenant isolation and observability.** Production agents are multi-tenant. Calibration logs need to be scoped per team so team A's overconfident lane does not drag team B's weights down. Every LLM call, tool call, and lane needs an OpenTelemetry span so an SRE can debug a stuck investigation without reading source. Real observability is not a feature, it is the precondition for running this in front of paying customers. ## What I learned Three lessons stick. The signal is global; the actuator is local. Distributed systems pattern, but it cuts harder in AI infrastructure because the in-flight LLM call is the work that needs cancelling and the cancellation control is a per-process JavaScript object. Pin this phrase in your head; it will save you an hour of confused debugging the first time you ship an agent demo to Vercel. The lanes are stateless; the system has memory. Calibration as a feedback loop in the merge layer is what separates a multi-agent system from a bag of LLM calls. The lanes never learn. The system learns by tracking which lanes to trust. Every metric needs an honest defense. Cache hit rate of zero, illustrative eval snapshot, stub correctness oracle, in-memory session store on Vercel. Every shortcut Forge takes is documented in the UI or the README with what it would look like in production. The discipline of disclosing the gap, instead of hiding it, is the move that signals senior judgment. If you want to see all of this in action, [forge.kevinmurphywebdev.com](https://forge.kevinmurphywebdev.com) is the live demo. The full source is at [github.com/midimurphdesigns/forge](https://github.com/midimurphdesigns/forge). The most interesting files to read in order are `docs/AGENTS.md`, `lib/coordinator.ts`, and `lib/store.ts`. Everything else is implementation detail. --- ## Auditing my own portfolio against Next.js 16: what I shipped, what I skipped, and why URL: https://kevinmurphywebdev.com/blog/nextjs-16-portfolio-teardown Date: 2026-05-20 Tags: nextjs, react, app-router, performance, vercel Excerpt: I audited my own portfolio against the Next.js 16 upgrade guide line by line. Here's what the site uses, what it deliberately leaves out, and the v16 changes I caught only because I read the release notes instead of a tutorial. I built this site from a clean App Router start on Next.js 16. Then I sat down and audited it the way a senior reviewer who knew the framework end-to-end would. The interesting question was never "did I use every primitive?" It was "did I make the right call for each one, and can I defend the omissions out loud?" This post is the audit. The structure follows the spine that ended up mattering: what I used, what I deliberately did not use, what v16 caught me on after I read the upgrade guide, and the senior-engineer signals I am still trying to earn. The aim is to write the post I wish I had read when I was first reasoning through Next.js App Router primitives from scratch, on a real site, against the v16 upgrade guide. ## The short version: best practices this site actually follows If you want the headline before the deep dive, here are the calls that matter most: - **Server components by default.** About 60% of the component files are server-rendered. `'use client'` lives at the leaves (animation hooks, event handlers, browser APIs), never at the root of a page tree. - **SSG-first with explicit exceptions.** Every page that can be prerendered at build time is. ISR is opted into on one route where the content has a real expiry. Dynamic rendering is reserved for genuinely per-request work (RAG endpoint, OG image generator, RSS feed). - **`dynamicParams = false` on every finite-slug route.** Blog and portfolio slug routes return 404 for anything not in the build-time list. Eliminates a slug-fuzzing surface. - **`generateStaticParams` enumerates the URL space at build.** Combined with the line above, Next knows the full set of pages before any user request lands. - **Metadata API is the spine, not a checkbox.** Per-post `generateMetadata`, a single `buildMetadata` helper, dynamic OG images via `next/og`, RSS feed handler, sitemap, robots, and JSON-LD on every page that has one. - **Type safety from the route up.** `npx next typegen` generates `PageProps<'/blog/[slug]'>` so route params are typed from the actual filesystem instead of hand-written. Typecheck is wired to run typegen first so it self-heals on clean checkouts. - **Strict TypeScript with `noUncheckedIndexedAccess`.** Catches the array-index `undefined` bugs that bite half a year after they ship. - **Tailwind v4 CSS-first.** No `tailwind.config.js`. Design tokens live as CSS custom properties under `@theme` in `globals.css` where they belong. - **Turbopack default.** v16 stable for both dev and build. No flag needed; the explicit `--turbopack` script flag has been removed. - **Leaf-isolated dynamic imports.** Heavy animation modules load behind `next/dynamic` with `{ ssr: false }` so they ship in their own chunks after hydration instead of bloating the initial route bundle. - **Bundle analyzer wired behind a flag.** `pnpm analyze` runs a production build with `@next/bundle-analyzer` and emits an interactive treemap. The v16 build output dropped its "First Load JS" column for RSC-correctness reasons, so the analyzer is the way to keep client bundle weight honest. - **Defensive HTTP headers in `next.config.ts`.** `X-Content-Type-Options: nosniff`, `X-Frame-Options: DENY`, `Referrer-Policy`, `Permissions-Policy` denying camera and microphone and geolocation, plus immutable cache headers on `/fonts/*`. The rest of the post walks each of these out, plus the primitives I deliberately did not use. ## Stack snapshot Next.js 16.0 on the App Router. React 19 stable. TypeScript strict mode with `noUncheckedIndexedAccess` and `noImplicitOverride`. Tailwind v4 with CSS-first config in `globals.css`. Three font families via `next/font/google` plus a manually preloaded Migra italic for the display tier. Turbopack as the default bundler for both dev and build (one of the v16 things I had to remove a redundant flag for). Vercel for hosting. MDX for content via `@next/mdx`. No Pages Router code anywhere; the site started on App Router from day one. That last detail matters. Most Next.js posts about App Router are migration stories from the Pages Router, and I have nothing interesting to say about that path because I never had to walk it. What I can speak to is reasoning about App Router primitives from a clean start. ## Server components are the default, and I treated them that way Of about 85 component and route files in the repo, 35 have `'use client'` at the top. The other 50 are server components by default. Every client component earns its directive on one of five grounds: it uses hooks (`useState`, `useEffect`, `usePathname`), it attaches event listeners, it touches a browser API (canvas, WebGL, scroll position), it provides React context (Lenis), or it is an error boundary. The pattern I worked hard to keep is "client components are leaves, not branches." The `Hero` component at `src/components/Hero.tsx` is a client component because it runs an opacity animation on mount. But it does not wrap the whole homepage. The page itself, at `src/app/page.tsx`, stays a server component. It reads from the file system (blog posts, projects), renders the static layout, and embeds the `Hero` as one leaf among several. The same logic governs heavier pieces. The hero canvas effect (`HeroLogoRain`), the smooth-scroll provider (`LenisProvider`), and the magnetic cursor (`Cursor`) are all imported via `next/dynamic` with `{ ssr: false }`. They are browser-only by definition (canvas, scroll APIs, mouse tracking), and they are not on the LCP critical path, so deferring them keeps the initial HTML lean. The diagnostic test for this is simple. Pick any page, look at the route file, and ask: how much of this tree could in principle render with `JSON.stringify` of plain data instead of a React-DOM hydration walk? On this site the answer for every route is "most of it." That is the win. ## File architecture and routing: what the `app/` tree actually looks like The full top-level layout under `src/app/`: ``` src/app/ ├── layout.tsx single root layout, server component ├── page.tsx home, server component ├── globals.css tailwind v4 import + design tokens ├── error.tsx route-level error boundary, client ├── global-error.tsx root error boundary (wraps html + body) ├── not-found.tsx custom 404 ├── icon.tsx runtime-generated favicon ├── apple-icon.tsx runtime-generated apple touch icon ├── sitemap.ts dynamic sitemap ├── robots.ts robots.txt generator ├── about/page.tsx ├── contact/page.tsx ├── demos/page.tsx ├── resume/page.tsx ISR, revalidate: 3600 ├── blog/ │ ├── page.tsx index, reads searchParams (tag, page) │ └── [slug]/page.tsx dynamic segment, dynamicParams: false ├── portfolio/ │ ├── page.tsx index, reads searchParams (tab) │ └── [slug]/page.tsx dynamic segment, dynamicParams: false ├── api/kev-o/route.ts RAG endpoint, Node runtime, streaming ├── feed.xml/route.ts RSS handler with Cache-Control └── og/route.tsx dynamic OG image via ImageResponse ``` The shape under it: every URL segment is a folder, every page in that segment is a `page.tsx`. The folder structure is the URL structure. There is one root `layout.tsx` and no nested layouts. Each dynamic segment uses square-bracket folders (`[slug]`). API endpoints are `route.ts` files under `app/api/` colocated with their domain. The advantages of this flat shape, on this site specifically: The site has one visual chrome (header, footer, grain layer, command palette, cursor) that appears on every page. There is no logged-in vs logged-out shell, no marketing vs admin split, no settings sub-app. Adding nested layouts would require duplicating the chrome or adding pass-through layouts that exist only to re-export `children`. Either path is ceremony. One root layout is the right answer. Colocating API handlers under `app/api/` instead of a parallel `pages/api/` tree (the old way) lets the route handler import server-only utilities (`lib/seo.ts`, `lib/content.ts`, the MDX corpus builder) without indirection. The Kev-O route handler reads the corpus JSON that the prebuild step writes under `public/kev-o-corpus.json`. The file path lookup is dead simple because there is one source tree. The dynamic segments (`[slug]`) carry both `generateStaticParams` and `dynamicParams = false`. That combo lets Next prerender all 11 blog posts and 18 portfolio entries at build time, and 404 anything else. The list of valid URLs is decided by what is on disk, not by request-time logic. ## App Router primitives the site uses A complete enumeration: - **Single root `layout.tsx`.** Wraps every page. Sets ``, preloads Migra italic with ``, injects the RSS ``, drops two JSON-LD blocks (Person, Website) into ``, and renders the chrome (skip-to-content link, grain, header, footer, command palette) around `{children}`. - **`page.tsx`.** Every visible URL has one. They are server components by default. The two dynamic segments (`/blog/[slug]/page.tsx` and `/portfolio/[slug]/page.tsx`) use the `PageProps<'/...'>` typegen helper for their params. - **Dynamic segments (`[slug]`).** Both content surfaces (blog, portfolio) use a single dynamic segment with `generateStaticParams` + `dynamicParams = false`. The pages are prerendered; the URL space is finite and locked. - **`searchParams` for filter state.** `/blog?tag=ai&page=2` and `/portfolio?tab=demos` read `searchParams` (now a Promise in v16, awaited per the upgrade guide) for filter and pagination state. This is the right call over a dynamic segment because the filter is a view of one underlying list, not a separate page. Search params change without rebuilding the route. - **`error.tsx`.** A client component (Next requires this) that renders a recovery UI when a server component throws below it. Shows a "something broke" message with a `reset()` button that re-runs the segment. Scoped to everything under `app/` except the root layout. - **`global-error.tsx`.** A client component that catches errors in the root layout itself, including hydration failures. Renders its own `` and `` because the root layout has crashed. This is the last line of defense. - **`not-found.tsx`.** A custom 404 page rendered when `notFound()` is called or when a route segment does not exist. The blog and portfolio slug routes call `notFound()` when a slug is missing, which now (with `dynamicParams = false`) happens at build-time enumeration rather than at request time. - **`route.ts` and `route.tsx`.** Six route handlers total: the Kev-O RAG endpoint (streaming, rate-limited), an admin endpoint, a health probe, an admin reset, the RSS feed handler, and the dynamic OG image generator. All declare `export const runtime = 'nodejs'` explicitly so there is no ambiguity about where they run. - **`sitemap.ts` and `robots.ts`.** Special filenames Next recognizes. The sitemap reads the same `getPosts()` and `projects` data the rest of the site uses, so the URL list cannot drift from the actual content. - **`icon.tsx` and `apple-icon.tsx`.** Runtime-generated favicons via `ImageResponse`. Lets the brand mark stay consistent without checking PNG assets into the repo. - **`generateMetadata` on every page or layout that needs per-route metadata.** Per-post overrides for `ogTitle`, `ogSubtitle`, and `ogEyebrow`. - **`generateStaticParams` on both dynamic segments.** Enumerates the URL space at build. ## App Router primitives the site does not use, and why - **Nested `layout.tsx` files.** Not used. The site has one shell. A nested layout would either duplicate the chrome or pass-through `children`. Adding one as decoration is the kind of move that signals "I memorized the API" rather than "I made a decision." - **`template.tsx`.** Not used. Templates differ from layouts in that they re-mount on every navigation instead of persisting. They are useful when a page needs to reset state between routes (a multi-step form, an animation that should restart). Nothing on this site needs that. - **`loading.tsx`.** Not used. Same reasoning as the Suspense section below: no route on this site has real wait time on the server, so a loading state would be theatre that makes pages feel slower. - **`default.tsx`.** Required only for parallel routes. The site has no parallel routes. Not needed. - **Catch-all routes (`[...slug]`).** Not used. The blog and portfolio URL spaces are flat (`/blog/foo`, not `/blog/2026/04/foo`). A catch-all would let the URL space grow without code changes, which is the opposite of what `dynamicParams = false` is doing. The two design choices point in opposite directions, and locking the URL space is the right answer for a portfolio. - **Optional catch-all routes (`[[...slug]]`).** Not used. Same reasoning as catch-all, plus the optional variant is for cases where the segment may be empty. No use case here. - **Private folders (`_name`).** Not used. Private folders are a way to put utilities inside `app/` that should not be picked up as routes. The site uses `src/lib/` and `src/components/` for that, which is the more common pattern and lives outside `app/` entirely. The private-folder convention is mostly useful for monorepos or for projects that want everything under `app/`. - **Route groups (`(name)`).** Not used. Route groups let you share a layout across a subset of routes without affecting URLs. With one root layout and no shell variations, there is nothing to group. - **Parallel routes (`@slot`).** Not used. Closest call was the `/portfolio` tabs (work, demos, OSS). I went with `searchParams` instead because the tabs are filter views on one list, not three URL-addressable panels. - **Intercepting routes (`(.)foo`, `(..)foo`).** Not used. The Instagram-style modal pattern is the canonical use case (click a thumbnail to open a modal that shows the same URL as the standalone page). No need here. The pattern across the "not used" list is the same: every primitive solves a specific problem. If the project does not have that problem, adopting the primitive adds complexity without adding capability. Saying so out loud is the senior signal. ## Error boundaries: three layers, deliberately The error handling story spans three files, in order from innermost to outermost: `notFound()` calls inside `page.tsx` handlers trigger the nearest `not-found.tsx`. That covers "this content does not exist" (a slug that was not in `generateStaticParams`, a search that returned zero results that should 404 instead of empty). `error.tsx` catches runtime errors thrown inside server components below the root layout. It is a client component because the recovery UI needs interactivity (the `reset()` button). It renders inside the root layout, so the header and footer still show; only the page content gets replaced. `global-error.tsx` catches errors in the root layout itself, including hydration failures. It renders its own `` and `` because the chrome it would normally rely on is what crashed. This is the bottom of the stack. The three layers are not interchangeable. Putting your error UI in only `error.tsx` means a broken root layout shows a blank page. Putting it in only `global-error.tsx` means every recoverable error in a page replaces the entire shell. Both files exist on this site for that reason. ## Rendering modes: SSG-first, one ISR exception, dynamic only where it has to be The route table from `next build` for this site reads: ``` ○ / static ○ /about static ○ /contact static ○ /demos static ● /blog/[slug] SSG (11 paths) ● /portfolio/[slug] SSG (18 paths) ○ /resume ISR (revalidate: 1h) ƒ /api/kev-o dynamic (Node runtime) ƒ /og dynamic (ImageResponse) ƒ /feed.xml dynamic ``` The static pages are static because their content is checked into the repo. The dynamic segment routes (`/blog/[slug]`, `/portfolio/[slug]`) use `generateStaticParams` to enumerate the slugs at build time, and I added `export const dynamicParams = false` while writing this post. That last setting is the senior signal. It tells Next that any slug not in the build-time list returns 404 instead of being lazily rendered at request time. It eliminates a class of fuzzing surfaces (poking at random `/blog/foo-bar`) and it makes the route deterministic. A careful reviewer would ask why `dynamicParams = false` is not on every route that has a finite slug set. It is a one-line change with no downside. The one ISR route is `/resume`. The resume PDF is generated by Puppeteer in a postbuild step, but the resume page itself displays metadata about certifications. Some of that metadata has expiry dates. Once an hour the page revalidates so the rendered "expires in N months" copy stays accurate without needing a redeploy. The route declares `export const revalidate = 3600`. The dynamic route handlers all serve traffic that cannot be precomputed. `/api/kev-o` is the RAG endpoint that streams Claude responses against an MDX corpus. `/og` is the dynamic Open Graph image generator (Satori under the hood via `next/og`). `/feed.xml` is the RSS feed, which I made dynamic on purpose so it can read whatever the latest published post is without a rebuild. What I want to flag here is what is _not_ on the list. There is no route using PPR (Partial Prerendering) or Cache Components. I considered enabling `cacheComponents: true`. I decided against it. The mental model for Cache Components is built around the case where one route mixes a fast static shell with slow dynamic holes (a product page where layout is cached but price is per-user, a dashboard with a cached layout and a live data widget). This site does not have that pattern. Every page is either fully static or fully dynamic. Adding the Cache Components opt-in would introduce ceremony with no measurable benefit, and v16 explicitly warns that PPR works differently from the v15 canary, so the cost of opting in is real. The right call was to leave it off and write that decision down. If I ever add a route with mixed static and per-user data, that is when Cache Components earns its place. ## Metadata API: this is where the site invests If I had to point to one area where this site invests deeply, it is the metadata surface. The site ships: - `generateMetadata` on every dynamic route, with per-post overrides for `ogTitle`, `ogSubtitle`, and `ogEyebrow` (so a long post title renders as a short OG title on LinkedIn) - A `sitemap.ts` route that emits every static and dynamic URL with `lastModified` derived from frontmatter - A `robots.ts` that allows all, disallows `/api/`, and links the sitemap - A `feed.xml` route handler that emits RSS 2.0 with `Cache-Control: public, max-age=600, s-maxage=3600` - A dynamic OG image route at `/og` using `ImageResponse` from `next/og`, which renders Migra italic on a noise-textured canvas with custom fonts loaded from disk - JSON-LD structured data in the root layout (Person, Website) and per page (Breadcrumb, Article, CreativeWork) - `icon.tsx` and `apple-icon.tsx` that generate favicons at runtime so the brand mark stays consistent without checking PNG assets into the repo The mistake I see in a lot of portfolio repos is treating metadata as a checkbox. Either there is no `generateMetadata` and the home page title leaks across every route, or there is one static `metadata` export per page with the same boilerplate copied around. The senior pattern is: define `buildMetadata` as a single helper (in `src/lib/seo.ts` here), and have every page call it with just the fields that actually vary. The helper handles the absolute URL, the OG image URL with query parameters, the canonical, and the JSON-LD shape. The page declares intent, not boilerplate. One subtle v16 thing on metadata. `generateMetadata`'s `params` and `searchParams` are Promises now. The compatibility shim for sync access from v15 is fully removed. If you upgrade from 14 or 15 and you have not run `npx next typegen`, you will get type errors on every dynamic route. Running typegen produces a `PageProps<'/blog/[slug]'>` helper that types the route's params correctly without hand-written types. I converted both dynamic routes to use `PageProps<'/blog/[slug]'>` and `PageProps<'/portfolio/[slug]'>` while writing this post. The types are now sourced from the actual route structure instead of duplicated. ## Streaming, Suspense, and `loading.tsx`: deliberately absent There are no `` boundaries in the codebase. There are no `loading.tsx` files. This was a deliberate call and it is one of the easier ones to defend. `` and `loading.tsx` are streaming primitives. They earn their place when a route has real wait time on the server (a DB query, a slow third-party API, an LLM call) and you want to flush the parts of the page that are ready while the slow parts resolve in the background. The streaming model is genuinely beautiful, and PPR builds on top of it. This site does not have those latencies. Every page either reads from the file system (sub-millisecond on a Vercel edge node) or it serves prerendered HTML. There is no point flushing an empty shell with a skeleton when the full page is already on the CDN. Adding a `loading.tsx` would actually make the page feel _slower_ because users would see a flash of the loading state. The honest version of this answer: I would add a `` boundary the first time I introduced a server component that called a slow API. Right now the slowest server-side work on the site is a `Promise.all([getPosts(), getAllTags()])` that completes in single-digit milliseconds. The boundary would be theatre. ## Server actions, middleware, and proxy: also deliberately absent There are no server actions in the repo (`'use server'` does not appear). There is no `middleware.ts`, and consequently no `proxy.ts` either (that is the v16 rename, which I will get to). Server actions are the right tool for mutations triggered from your own UI. This site has no mutations. The contact page is a `mailto:` link. The Kev-O chat interface posts to a route handler at `/api/kev-o` because it needs streaming and external rate-limit checks that route handlers handle more cleanly. If I ever add a form (a guestbook, a newsletter signup), I will use a server action. Until then, adding one as decoration would be the same kind of mistake as adding a `loading.tsx`. Middleware was harder to think through. The v16 release renames `middleware.ts` to `proxy.ts` and forces it to run on the Node.js runtime instead of the Edge runtime. That is a significant change. The reasoning in the release notes is that middleware was being used for too many things, and the "proxy" framing makes the file's actual job clearer: it sits in front of every matching request and decides whether to redirect, rewrite, or pass through. If you need Edge runtime auth checks, the upgrade guide says to keep `middleware.ts` for now (deprecated, but functional) and wait for the next minor release. This site has no auth, no A/B testing, no locale negotiation (the global bilingual rule is explicitly overridden for this project, per ADR-018), and no per-request decisions to make. Adding a proxy would be code that runs on every page load and does nothing. So there is no proxy, and that is a deliberate choice rather than an oversight. ## next/image, next/font, next/script: the standard kit, used correctly There is no raw `` tag in the codebase. Two pages use `next/image`: the about page (headshot) and the hero on the home page. Both pass explicit `sizes` attributes for the responsive `srcset`. The hero image is the LCP element on the home page and carries `priority`, which is still the correct prop in Next 16 (a reference doc I cross-checked claimed it had been renamed to `preload`, but the actual v16 upgrade guide makes no such mention; do not trust secondary sources without verifying). Fonts come through `next/font/google` for Space Grotesk, Geist Mono, and Instrument Serif. All three use `display: 'swap'` so the page renders immediately with the fallback font and reflows when the web font loads. The variables (`--font-space-grotesk`, `--font-geist-mono`, `--font-instrument-serif`) are exposed as CSS custom properties and consumed in `globals.css`. The display font (Migra italic) is not on Google Fonts. It is a trial file licensed for evaluation, served from `public/fonts/migra/`. The font is preloaded explicitly in the root layout with ``. This is one of the few places where I dropped to a raw `` tag because `next/font/local` does not handle proprietary trial files cleanly. The trade-off is conscious. The v16 image config changes worth noting (verified against the upgrade guide): `images.minimumCacheTTL` default went from 60 seconds to 4 hours. `images.qualities` went from "any value 1-100 allowed" to "only [75] allowed by default" with non-75 values coerced to the nearest allowed quality. `images.domains` is deprecated in favor of `remotePatterns`. I checked my code: no `quality` props anywhere (so the coerce-to-75 change is a non-issue), `images.domains` is not used (already on `remotePatterns`), and I am happy with the longer cache TTL. ## Bundle optimization: leaf isolation plus dynamic imports The v16 release removed the `size` and `First Load JS` columns from the `next build` output because the team felt the numbers were inaccurate in RSC architectures, and they recommend Chrome Lighthouse or Vercel Analytics instead. That is fine advice for production monitoring but it does not replace a static analyzer for "did I just balloon the home page bundle by adding a new dependency." So `@next/bundle-analyzer` is wired into `next.config.ts` behind an `ANALYZE=true` flag, and `pnpm analyze` runs a full production build that emits an interactive treemap to `.next/analyze/`. The flag means the analyzer never runs on regular builds (no overhead), but it is one command away whenever a PR feels like it might have ballooned a route. The static analyzer is half of the story. The other half is the leaf-isolation pattern enforced at code-review time. Any new `'use client'` directive has to justify why the whole component (not just the interactive part) needs to be on the client. The heavy animation modules (`HeroLogoRain`, `LostSignal`, `ChromaticField`) are all behind `next/dynamic` with `{ ssr: false }`, which means they ship in their own chunks loaded only after hydration. The MDX rendering happens entirely on the server. The MDX runtime is not in the client bundle. ## What v16 caught me on, after I read the upgrade guide These are the actual changes I made to this codebase while writing this post: 1. Added `export const dynamicParams = false` to both dynamic routes. This is a senior signal that has nothing to do with v16 specifically, but writing the post is what made me realize I had not done it. 2. Converted `params` types to use the new `PageProps<'/...'>` helper from `npx next typegen`. The helper has been around since 15.5 but the upgrade guide is what pointed me at it. 3. Removed the `--turbopack` flag from the `dev` script in `package.json`. Turbopack is the default for both dev and build in v16; the flag is now redundant. The build still passes (and is noticeably faster than the v15 webpack default). 4. Audited image config defaults against the v16 changes. Nothing needed to change; the project already uses `remotePatterns`, has no `quality` props, and accepts the new 4-hour `minimumCacheTTL`. 5. Confirmed the `proxy.ts` rename and the Edge-to-Node runtime change does not affect me because there is no middleware. Filed the knowledge for next time. 6. Verified the v16 parallel-routes change does not bite this site. Parallel route slots now require an explicit `default.js` file or the build fails. The site has no parallel routes, so this is a non-issue, but the `npx @next/codemod@canary upgrade latest` codemod handles it automatically if you have any. 7. Replaced the runtime MDX dynamic import (an `await import()` call whose path was a template string built from the slug) with a build-time-generated static registry. Hot reload silently no-op'd on content edits because the template-string `import()` was opaque to Turbopack's dependency tracker; static imports give the bundler something it can trace. The pattern under all of these is the same: read the upgrade guide front to back. Do not trust tutorials, do not trust secondary sources, do not trust your own assumption that "I am on the latest version." The release notes are the contract. ## What I would build next if this site had to grow If I had a hypothetical Year Two of this site, here is the order I would reach for the unused primitives: If I added a contact form with a real submit, that gets a server action with `updateTag()` after the database write so the user sees their submission immediately instead of waiting on stale data. If I added a /now page that pulled from a slow source (latest GitHub activity, last book finished), that gets `'use cache'` with a `cacheTag` and a `cacheLife({ expire: 300 })` to make the page mostly static while the slow data refreshes on its own SWR window. Enabling `cacheComponents: true` unlocks this. If I added a logged-in surface, that gets a proxy.ts with a Node-runtime auth check, plus `'use cache: private'` on the per-user data. If I added a real admin route, that gets a `loading.tsx` (because admin routes have legitimate latency from real database queries) and probably parallel routes for the dashboard panels. None of these are speculative refactors I should ship now. They are upgrades that earn their place the moment the requirement that justifies them lands. ## The signal I am trying to send The frame I want anyone reviewing this site to leave with is: this person has read the docs, made deliberate decisions, and can explain the omissions out loud. Not "this person used every primitive in the framework." That second frame is the failure mode for portfolio sites. Every primitive earns its place; the ones that do not earn it stay on the shelf. The move I would recommend if you are doing the same exercise on your own Next.js site: audit it against the v16 upgrade guide line by line. For every primitive you do not use, write down one sentence on why. If you cannot defend an omission in one sentence, that is the gap. Fix it, or write the sentence and move on. ## Glossary If you are newer to the Next.js ecosystem, a few terms used above with definitions that lean on how Vercel uses them in their own docs, with my own framing where I think it helps. **App Router.** The current Next.js routing model, built around the `app/` directory. Each folder is a URL segment and special filenames (`page.tsx`, `layout.tsx`, `loading.tsx`, `error.tsx`, `route.ts`) declare what each segment does. The predecessor is the Pages Router, which used `pages/` and had a different mental model around data fetching. App Router is React Server Components native. **Server component.** A React component that runs only on the server. Its code is never sent to the browser. It can read from the database or the filesystem directly, but it cannot use hooks like `useState` or event handlers like `onClick`. Server components are the default in the App Router. **Client component.** A React component marked with `'use client'` at the top of the file. Its code ships to the browser so it can use hooks, event handlers, and browser APIs. The senior pattern is to push client components as far down the tree as possible so the rest of the page stays on the server. **SSR, SSG, ISR.** Three rendering modes the App Router collapses into a unified model. SSR (server-side rendering) renders the page fresh on each request. SSG (static site generation) renders the page at build time and serves the same HTML to everyone until the next build. ISR (incremental static regeneration) is SSG with an expiry, after which the page is regenerated on demand. In v16 the framing has shifted to "is this page dynamic or cached," and you opt into caching with the `'use cache'` directive once `cacheComponents` is enabled. **Hydration.** The process by which React takes a server-rendered HTML page and attaches event listeners to make it interactive in the browser. The HTML renders first (fast, no JavaScript needed), then the JavaScript bundle loads and "hydrates" the page. Server components produce HTML that requires no hydration; client components do. **`generateStaticParams`.** A function exported from a dynamic segment route (like `/blog/[slug]`) that returns the list of values the route should prerender at build time. Without it, Next either skips static generation or generates pages lazily on first request, depending on configuration. **`dynamicParams`.** A route segment config option. When set to `false`, any path not returned by `generateStaticParams` returns a 404 instead of being rendered on demand. Locks the URL space. **Streaming.** A rendering technique where the server flushes parts of a page to the browser as they become ready instead of waiting for the entire page to render. Powered by React's `Suspense` boundaries. Pairs naturally with PPR. **Suspense boundary.** A React primitive that lets you wrap an async component and declare what to show while it loads. The framework streams the page shell first and slots the async content in when it resolves. **`loading.tsx`.** A route-level convention in the App Router. If you put a `loading.tsx` in a route folder, Next wraps the entire page in a Suspense boundary using that file as the fallback. Sugar over manually writing the Suspense. **Server action.** An async function marked with `'use server'` that runs on the server and can be called directly from a form or client component as if it were a local function. The framework handles the RPC plumbing. Use server actions for mutations triggered by your own UI. **Route handler.** A `route.ts` file under `app/` that exports `GET`, `POST`, or other HTTP method handlers. Use route handlers for webhooks, external API consumers, file uploads, or anything that needs a stable HTTP endpoint. **Middleware (now `proxy.ts` in v16).** A file at the project root that runs before every matching request and can redirect, rewrite, or modify the response. In Next 16 the file was renamed to `proxy.ts` and the runtime was forced to Node.js (Edge runtime is no longer supported for this file). **Edge runtime.** A lightweight JavaScript runtime (based on V8 isolates, not Node.js) that runs at Vercel's edge locations closer to the user. It has a smaller API surface than Node and excludes Node-specific modules like `fs`. Most route handlers in this site use the Node runtime explicitly because they need full Node APIs. **Turbopack.** The Rust-based bundler that replaced Webpack as the default for Next 16 dev and build. Significantly faster than Webpack for both cold start and incremental compile. **PPR (Partial Prerendering) / Cache Components.** A v16 feature, opted into by setting `cacheComponents: true` in `next.config.ts`. Lets a single route mix a static shell (served instantly from the CDN) with dynamic holes (streamed in as they resolve). Useful for pages that are mostly the same for everyone but have a few per-user or per-request pieces. This site does not use it because no route has that pattern yet. **`'use cache'`, `cacheLife`, `cacheTag`.** The v16 caching primitives. `'use cache'` at the top of a function or component tells the compiler to memoize it. `cacheLife` sets the SWR window. `cacheTag` lets you invalidate selectively via `revalidateTag` or `updateTag`. All three were `unstable_` in v15 and lost the prefix in v16. **`revalidateTag` vs `updateTag`.** Both invalidate cached data by tag. `revalidateTag` marks data stale and refreshes in the background while users see the old value (SWR semantics). `updateTag` expires and refreshes synchronously so the next render sees the new value (read-your-writes semantics, server-action-only). **RSC payload.** The serialized representation of a React Server Component tree that the framework sends to the browser. It is not HTML; it is a custom format React uses to reconcile server-rendered content with client-side state. **LCP (Largest Contentful Paint).** The Core Web Vital that measures how long it takes for the largest visible element on a page to render. Common LCP elements are hero images, headline text, or above-the-fold backgrounds. Vercel Speed Insights and Chrome Lighthouse both report it. **CLS (Cumulative Layout Shift) and INP (Interaction to Next Paint).** The other two Core Web Vitals. CLS measures visual stability (how much content jumps around as the page loads). INP measures responsiveness to user input. Both have direct ranking implications on Google. **`generateMetadata`.** An async function exported from a page or layout that returns a `Metadata` object. Lets you compute per-page metadata (title, description, OG image, canonical URL) based on the route params. **`ImageResponse` / `next/og`.** A Next utility that renders JSX directly to a PNG using Satori. Used for generating Open Graph share images dynamically per route without needing a headless browser. **JSON-LD.** A structured data format Google reads to understand page semantics. Lets you mark a page as an Article, a Person, a BreadcrumbList, etc., so search engines render rich results. Embedded as `