Blog

Toolcrib vs. shadcn/ui and Other Open-Source UI Toolkits

Analysis based on a live clone of escape-llc/toolcrib (commit history, source, docs, and the CLI package) as of August 17, 2026. GitHub’s stars/forks/issues API was rate-limited during this review, so community-metric claims below (stars, adoption) rely on what’s visible in the repo itself, not the GitHub API — worth re-checking directly on the repo page.

1. What Toolcrib Actually Is

Toolcrib is not a component library in the traditional sense — it’s a UI toolkit explicitly designed to be used by an AI coding agent, not primarily by a human developer. Its stated problem: when you “vibe code” a UI with an LLM, the model regenerates Tailwind-esque class soup from scratch every turn, so nothing stays visually consistent across sessions.

Its solution is structural, not cosmetic:

No className or style props are exposed on components. The model is architecturally prevented from inventing ad hoc styling — it can only pick from a constrained set of variants/props.
~33 components (Button, Card, DataTable, Modal, Popup, SlideOut, TabStrip, ToggleGroup, Toolbar, AppShell, Splitter, ThemeEditor, etc.), built on top of Radix UI primitives for accessibility/behavior, with a custom CSS-variable-driven theming layer (ThemeSlice pattern) rather than Tailwind.
A component-manifest.json — a machine-readable API surface — meant to be fed to the agent instead of prose prop documentation, plus ai-docs/ files (CORE.md, NEW_APP.md, REFACTOR_APP.md) written specifically as agent instructions.
A custom CLI (npx toolcrib init/apply/doctor/merge/versions) with a genuinely clever patch-based vendoring workflow: it never touches your files directly, stages everything as reviewable .patch files, and has a real three-way merge/conflict system for upgrades (did you edit the file? did upstream change it too? → safe update / kept-as-is / conflict).
Full vendoring model — the same “copy the source into your repo” approach as shadcn/ui, explicitly cited as the inspiration.

The unifying principle underneath all of it — worth naming explicitly, since it recurs at every layer covered in this report: LLMs don’t reproduce their own prior output exactly. Call it fuzziness — the same prompt, or a prompt close enough to the last one, doesn’t reliably regenerate the same answer. Toolcrib’s operating premise is that anywhere this fuzziness would surface as a real bug or as visible drift, the right response isn’t to document the correct answer more clearly and hope the model reproduces it — it’s to remove the model’s opportunity to regenerate that decision at all, and point it at one fixed, shared answer instead. The no-className slice system is this principle applied to styling. The shared infrastructure in Section 2 — one event bus, one observer manager, one z-index scale, one ID-generation primitive — is the same principle applied to behavior, closing off the exact re-solve-and-diverge failure mode described there. And the generate → review → taste-check development loop in Section 9 is the same principle applied to the project’s own process: a check the model’s fuzziness can’t route around, because a human is doing it, not the model. Three layers, one insulation strategy.

There’s a fourth, and it sits earlier in the pipeline than any of the above — before generation happens at all. A fixed, closed vocabulary reduces fuzziness not just in what the model produces, but in what it and the human mean when they refer to something in conversation. This is genuinely worth separating from shadcn’s component naming, which offers something similar but shallower: shadcn’s Button or Dialog is a stable name too, but the vocabulary stops being closed the moment you go one level down — variant, spacing, or color is expressed in Tailwind’s open combinatorial utility-class space, where many different class combinations can express roughly the same intent, so “give it the danger color” still gets re-interpreted freely each time. Toolcrib’s manifest closes the vocabulary at that level too — verified directly in source: Button‘s variant prop is a literal union ('primary' | 'secondary' | 'outline' | 'danger' | 'ghost'), not an open string, and the $defs section of component-manifest.json enumerates the same kind of closed set for every reusable field (e.g. subtheme: 'error' | 'success' | 'warning' | 'info'). “Use the danger button” now names one of five fixed values, not a request the model reformulates in CSS each time. Two honest caveats: this isn’t unique to Toolcrib at the component-naming level (shadcn has that too) — it’s the closed vocabulary reaching below the component, into variant/slice-field territory, that’s the actual difference; and a closed vocabulary only helps once it’s navigable, which is exactly why the manifest’s own size (Gap-Closure Plan, item 1) matters as much as its closedness — a vocabulary too large to search is functionally no better than prose.

2. The Load-Bearing Infrastructure Layer

The comparison so far has focused on components and theming, but that undersells a real part of what Toolcrib provides: a layer of shared plumbing solving problems that are tedious and easy to get subtly wrong every time an AI model (or a human, for that matter) reinvents them per-component or per-session. This is infrastructure in the literal sense — invisible when it works, and the report would be incomplete without naming it, since it’s a genuine part of the “why does this reduce drift” answer, not just the styling story.

Concretely, verified directly against source:

A strongly-typed global event bus (eventBus/eventBus.ts) for cross-tree communication without prop-drilling — aiBus.openModal(id), aiBus.showToast(message), etc. — with a genuinely subtle piece of engineering inside it: certain events (currently tab:changed) are marked sticky, replaying their last payload to a new subscriber immediately on subscribe. That exists to solve a real race condition — <TabStrip> and <TabStrip.Panel> are independent subtrees with no guaranteed mount order, so a plain pub/sub can permanently miss the initial broadcast depending on which one mounts first. This is exactly the kind of cross-component wiring bug that’s tedious to get right once and easy to get wrong every time it’s rebuilt from scratch.
A single shared ResizeObserver/IntersectionObserver manager (observer/observerManager.ts) rather than one observer instance per component — every observed element is tracked in one manager, debounced individually, with its own documented fix for a real bug (a second observe() call on an already-tracked element used to silently keep the first call’s config forever, so resize events kept reporting stale data after a config change).
A canonical z-index scale (theme/zIndex.ts) — BASE, STICKY, SPLITTER, DRAWER, MODAL, DROPDOWN, TOOLTIP, TOAST, each a fixed value with a documented reason for its position (e.g., tooltips above dropdowns and modals, toasts above everything). Z-index stacking wars — a toast trapped behind a modal, a dropdown rendering under a drawer — are a genuinely common class of ad hoc-UI bug, and this closes it with one shared constant instead of leaving every component to invent its own value.
SSR-safe stable ID generation (components/shared/useStableId.ts) — built on React’s own useId() rather than a Math.random()-backed ref specifically because the previous implementation caused a real hydration mismatch bug under Next.js/Remix-style SSR (server and client generating different random values for the same element). The comment in the source is explicit that fixing the shared primitive once forecloses the same class of bug for every future consumer, not just the components that hit it first.
Real exit-animation lifecycle handling (theme/useAnimatedMount.ts) — drives unmount off an actual animationend event rather than a guessed setTimeout duration, after that exact guessed-duration pattern caused two documented real bugs in this project’s own history: Toast’s exit animation never actually playing before removal, and SlideOut staying mounted (backdrop and all, still blocking clicks) after its timeout lost a race against React’s effect scheduling. This is precisely the kind of animation-timing bug that’s near-universal in hand-rolled overlay code and easy to reintroduce every time a model builds one from scratch.
Portal-aware document targeting (theme/targetDocumentContext.tsx) — a React Context (not CSS-variable inheritance, and not DOM position) tracking which Document a component’s self-injected CSS should target, specifically so a component portaled into an iframe doesn’t silently inject its :hover/:focus-visible/@keyframes rules into the outer page’s <head> instead of the iframe’s. The same Context-over-CSS-cascade reasoning also underlies StyleDomainContext (Section 6’s correction on one-off styling) — it’s a repeated architectural pattern in this codebase, not a one-off fix.
Automatic WCAG contrast enforcement, not just accessible markup (theme/hsv.ts, invoked from every palette generation in theme/harmonies.ts) — this is a distinct contribution from Radix’s accessibility guarantees (which cover markup/ARIA/keyboard behavior, not color choices at all). ensureWCAGContrast computes real WCAG 2.x relative luminance (the spec’s own piecewise-linearized-sRGB formula, weighted 0.2126/0.7152/0.0722 per channel) and iteratively nudges a color’s Value/Saturation until it meets a minimum contrast ratio against its background — with a documented real bug behind why this matters: an earlier version approximated luminance from HSV lightness alone, which is hue-blind, and silently passed real accessibility failures specifically for the toolkit’s own default error/info subthemes (blue and red-anchored), reporting a flat 4.56:1 “compliant” ratio across every hue when the real range was 1.89:1 (a failing blue) to 11.55:1 (yellow). A separate pickReadableTextColor handles the different case of text painted directly on a solid fill (a filled button, a badge) rather than themed text on a neutral surface, fixing a second real bug where filled-button variants hardcoded white text unconditionally and produced unreadable white-on-lime-green output for bright primary colors. Because this runs automatically inside palette generation, a non-expert user picking a color in the visual Theme Editor gets WCAG AA-compliant text by construction, with no accessibility knowledge required — reinforcing the non-Tailwind-expert positioning discussed in Section 6.

Why this matters for the comparison specifically — and it’s a sharper problem than mere duplication: shadcn/ui ships components; it doesn’t ship an opinion on any of the above. The risk isn’t just that a model building on shadcn re-solves z-index conflicts, observer duplication, SSR-safe IDs, and overlay unmount timing independently every session — it’s that an AI model cannot reproduce its own prior output exactly, so each re-solving is liable to diverge from the last one, not just duplicate it. Two independently-guessed z-index values aren’t two valid stacking orders — they’re two values that may actively conflict the moment both components land on the same page. Two independently-timed overlay unmount guesses aren’t two working solutions — they’re two different failure windows race-conditioned against React’s own scheduling. This is the same root cause as the styling-drift problem the rest of this report is about — no persistent state carried across generations — just surfacing as behavioral/correctness bugs at the infrastructure layer instead of visual inconsistency at the styling layer. Toolcrib closing this class of problem once, as shared infrastructure instead of per-component logic that gets regenerated (and silently re-diverges) every time, is a real structural advantage the component/theming framing alone doesn’t capture.

3. Project Maturity (important context)

Repo history starts August 6, 2026; current state is ~49 commits over ~10 days, tagged v0.1.0 → v0.3.0.
Single contributor (eScape LLC) for all 49 commits — no visible outside contributors yet.
The README states plainly: “This file is the only human-created content in the whole repo! The rest was generated by AI.” The commit history backs this up with a specific detail: 7 of the 49 commit messages explicitly name Claude sessions (“big fat claude session,” “claude fix campaign,” “big fat claude review session,” “big fat claude refactor,” etc.), concentrated in the project’s major refactor and review passes — so “mostly Claude” is an accurate characterization of the primary tool, not just “AI” in the generic sense.
MIT licensed, has a test suite (49 unit test files, Playwright e2e specs for interaction states, animations, layout, typography) — a good sign of engineering care for something this new.

This is a pre-community, single-maintainer, days-old project, not an established ecosystem. That single fact should weight every comparison below.

4. Component Coverage vs. shadcn/ui and Others

Category	Toolcrib	shadcn/ui	Mantine / Chakra UI / MUI
Primitives base	Radix UI	Radix UI (mostly)	Own primitives (Mantine/MUI) or Ark UI (Chakra v3)
Styling model	CSS variables + “slices,” no className exposed	Tailwind classes, fully editable	CSS-in-JS / CSS modules, fully editable
Component count	~33	~50+ (Command, Calendar, Carousel, Sheet, Breadcrumb, Pagination, Chart, Drawer, Skeleton, Input OTP, Resizable, Navigation Menu…)	100+ (mature, years of iteration)
Data table	Yes (built-in)	No (community recipe w/ TanStack Table)	Yes (Mantine/MUI have dedicated table packages)
Theme editor	Yes, built-in visual editor	No (edit Tailwind config/CSS vars by hand)	Varies (MUI has theme creator tooling)
Distribution	CLI with patch-based init/merge/doctor/upgrade	CLI (`shadcn add`), simpler add-only model	npm package, standard semver updates
Community / ecosystem	None yet (10 days old)	Large, very active, huge blog/tutorial corpus, de facto standard for AI-assisted React UI in 2025–2026	Large, established for years
AI-specific design	Purpose-built for this (manifest, ai-docs, style lockdown)	Popular with AI tools incidentally (simple, copy-pasteable, well-represented in training data)	Not designed with AI agents in mind

The honest framing: Toolcrib isn’t really competing with shadcn/ui on breadth or maturity — it’s competing on a narrower, specific bet, and that bet is broader than styling alone: that removing the model’s opportunity to regenerate a decision — whether that decision is a CSS class or a z-index value or an overlay’s unmount timing — produces more consistent, more correct AI-generated UIs than shadcn’s “here’s editable Tailwind and no shared behavioral plumbing, good luck” approach. The styling story (no className) is the most visible half of that bet; the infrastructure layer in Section 2 (event bus, observer manager, z-index scale, SSR-safe IDs, animation lifecycle) is the less visible, arguably equally important other half.

5. Pros

Genuinely addresses a real, well-known pain point. Anyone who’s iterated on an LLM-generated UI has hit the “it looked great, then I asked for one tweak and everything drifted” problem. Locking out className/style is a real architectural fix, not just documentation asking the model to behave.
The shared infrastructure layer (Section 2) closes a class of bug the styling fix doesn’t even touch. Z-index stacking conflicts, duplicated ResizeObserver instances, SSR hydration mismatches from ad hoc ID generation, and overlay-unmount race conditions are all things a model re-solves — and, given LLM fuzziness, re-diverges on — every time it builds an overlay or a layout from scratch. A single event bus, one z-index scale, one ID primitive, and one animation-lifecycle hook close all of these once, at the infrastructure layer, rather than leaving each to be independently reinvented and silently drift out of sync with every other instance. This is arguably as consequential as the styling story and gets far less attention in most framings of what Toolcrib does, this one included until this correction.
A closed vocabulary, not just a component library — a distinct benefit from the manifest’s machine-readability. “Use the danger button” names one of five fixed variant values (verified in source: variant?: 'primary' | 'secondary' | 'outline' | 'danger' | 'ghost'), not a request the model re-expresses in CSS however it sees fit this session. shadcn offers stable component names too, but its vocabulary opens back up into Tailwind’s combinatorial class space the moment you’re below the component level; Toolcrib’s stays closed all the way to the variant/slice-field level (see Section 1). That reduces ambiguity in the human-model conversation itself, before generation even happens — not just in what the model outputs.
Semantic styling persists through refactors; raw utility styling doesn’t — and this is a sharper problem for uncontrolled styling than mere inefficiency. Verified in source: Button‘s danger variant resolves to var(--ai-subtheme-error, #ef4444) — a CSS custom property set centrally by palette generation — not a hardcoded value baked into the call site. The only thing written in JSX is the semantic label itself, variant="danger". Raw Tailwind has no equivalent separation: className="bg-red-500" conflates the resolved value and the semantic intent into one artifact, with nothing distinguishing “this red is here because it’s our danger color” from “this red is here because someone typed it once.” A later restyle pass — same model, new session, or a different one entirely — has no way to tell deliberate customization from arbitrary/leftover code by inspecting the class list alone, and can silently normalize away or overwrite a previously-correct, deliberate decision (a contrast fix, a brand-specific shade) without any signal that a regression just happened. Toolcrib’s call sites can’t regress this way, because they never encode a resolved value to begin with — a global palette change propagates to every variant="danger" automatically, with nothing to hunt down and reinterpret at each occurrence.
Machine-readable manifest is a smart idea. Feeding a JSON API surface to a model is more token-efficient and more reliable than prose prop docs, and avoids the model guessing prop names.
The CLI’s patch/merge/conflict model is more sophisticated than shadcn’s — and it’s a real mechanism for the app’s foundation to actively improve over time, not just avoid decay. shadcn’s add command is essentially one-shot copy; Toolcrib’s doctor/merge tries to solve the real problem of upgrading vendored code you’ve since hand-edited (three-way diff, conflict files) — a harder problem shadcn hasn’t fully solved either. This isn’t hypothetical: the WCAG contrast fix and the SSR-safe useStableId fix (Section 2) both landed as real commits between tagged versions, so an app that started on an earlier version and later runs toolcrib merge genuinely inherits both fixes in its own vendored code, the same way it would from a patched npm dependency — without an agent ever needing to rediscover or refix either bug independently. That’s distinct from, and stronger than, the “shared infrastructure prevents per-session divergence” argument in Section 2 — this is the toolkit’s own quality improving underneath an app that’s already been built, not just staying consistent at a fixed baseline. (See the Cons/Risks entry below on what this benefit actually depends on.)
Built-in theme editor is a nice differentiator — non-technical users or designers can produce a preset without touching code, then hand that preset to the AI.
Accessibility inherited “for free” via Radix, same trust boundary as shadcn — markup, ARIA, and keyboard behavior. Worth distinguishing from a separate accessibility contribution that’s actually Toolcrib’s own, not Radix’s: automatic WCAG contrast enforcement (Section 2) guarantees readable text against any generated palette, which Radix has no opinion on at all since it doesn’t touch color. Two different accessibility guarantees, from two different layers, both real.
Has real automated tests (unit + Playwright e2e) despite its youth, which is more test discipline than a lot of brand-new component libraries ship with.

6. Cons / Risks

The “gets better over time” benefit above depends on two things that aren’t automatic, and its scope is narrower than “application architecture” implies. First, it only accrues to apps that actually practice merge discipline — one that vendors Toolcrib once and never runs merge again is frozen at whatever quality existed at init time, no different from ad hoc code in that respect, and any hand-edited file that upstream also later touches becomes a conflict requiring manual reconciliation, not an automatic gain. Second, and more fundamentally: everything this report has covered — the slice system, the event bus, the z-index scale, WCAG enforcement — is the presentation layer specifically. Toolcrib has no opinion on, and does nothing to insulate, state management, data fetching, routing, or business-logic organization elsewhere in the same app. An agent working in those layers is exactly as “uncontrolled” as it would be without Toolcrib at all — the curation is real, but it’s scoped to UI/presentation architecture, not the application as a whole.
No community, no track record. Zero external contributors, unverifiable production usage, 10 days of history. Bugs, edge cases, and abandoned-maintenance risk are all unknowns you’d be absorbing personally. shadcn/ui, Mantine, Chakra, and MUI all have years of battle-testing and large enough communities that answers to “why is X broken” already exist on the internet.
Much smaller component surface. No Command palette, Calendar/date picker, Carousel, Breadcrumb, Pagination, Skeleton loaders, Chart wrapper, Navigation Menu, or Drawer-as-distinct-from-modal out of the box — all standard in shadcn/ui today. You’d be building these yourself or falling back to raw Radix/other libraries anyway, which reintroduces the exact “unstyled model output” problem Toolcrib exists to prevent.
Losing className/inline styling is a real trade-off, but it’s not a dead end. One-off formatting isn’t unhandled — a per-component overrides prop (resolved against the component’s own registered ThemeSlice fields) and a StyleDomainProvider/useStyleDomain mechanism for subtree-level semantic states (e.g., marking a whole form section as an error state) exist specifically for this. That’s a genuinely well-designed answer to the “how do I make just this one thing different” problem, including handling portaled components (Modal/Popup/SlideOut) correctly via React Context rather than CSS-variable inheritance, which wouldn’t cross a portal boundary. The remaining gap is that this mechanism isn’t documented in the AI-facing manifest or ai-docs yet, so an agent (or a new contributor) has no way to discover it without reading source — worth fixing, but it’s a documentation gap, not a missing capability.
Own theming engine instead of Tailwind — a real trade-off, but not a one-directional loss. For developers who already think in Tailwind, this is a genuine ecosystem-lock-in cost: fewer Stack Overflow answers, fewer people who already know the system, and less training-data prior for the model itself (see Section 8). But Toolcrib’s actual target user isn’t necessarily a Tailwind expert — it’s explicitly aimed at people who want to style their app without needing to be one. Judged against that audience, the visual Theme Editor and CSS-variable “slice” system are arguably more approachable than Tailwind’s config-file-and-utility-class model, which has its own real learning curve for anyone who isn’t already fluent. The honest framing is: this is a bet on a different audience, not a strictly worse version of Tailwind’s approach — the risk is narrower than “lock-in,” and is really about whether that non-expert audience is large enough, and whether the toolkit does enough to actively serve them (see the gap-closure plan for how to lean into this rather than just mitigate it).
“Full vendoring” inherits shadcn’s known downside too: you now own and must maintain 596KB+ of copied component source in your repo, including bugs, rather than pulling a patched npm dependency.
Radix UI dependency uses radix-ui v1 “all-in-one” package — fine, but ties you to Radix’s release cadence and any of its known quirks (portal/z-index edge cases, etc.) just as shadcn does.
No visible design-system opinions beyond function. shadcn/ui, Mantine, and Chakra have gone through multiple visual redesign iterations informed by thousands of real apps; Toolcrib’s visual defaults haven’t been stress-tested at that scale.

7. When Each Makes Sense

Choose shadcn/ui if you want the largest component surface, the most community support/tutorials, Tailwind (which most AI models already “know” natively), and you’re fine reviewing/fixing the occasional styling drift yourself, or you’re building a team-scale product where hiring people who already know the stack matters.
Choose Mantine/Chakra/MUI if you want a mature, batteries-included design system with years of edge-case handling, and you’re not primarily optimizing for AI-agent-driven development.
Consider Toolcrib if your primary workflow really is heavy, repeated AI-agent UI generation/editing (e.g., an internal tool built almost entirely through Claude/Copilot-style iteration) and you’re willing to accept: a much smaller component set, a bespoke styling system to learn, and zero community safety net, in exchange for a real shot at more consistent AI output turn-to-turn. It’s best understood as an experimental, opinionated bet on a specific workflow, not a shadcn replacement yet.

8. AI-Optimization Evaluation (All Toolkits)

“AI-optimized” is being scored here on seven dimensions that actually matter when an agent, not a human, is doing the building and iterating:

Training-data prior — how much the model already “knows” about the library from pretraining, before you give it any docs at all.
Styling determinism — can the model freely emit arbitrary CSS/classes (drift risk), or is it constrained to a fixed set of tokens/variants?
Machine-readable API surface — is there a structured manifest/type surface an agent can consume cheaply, versus prose docs it must infer from?
Cross-turn consistency mechanism — is there one source of truth (theme object/provider) the model reliably reuses, or does each turn regenerate values independently?
Vendoring/inspectability — can the agent see and safely edit the actual component source, or is it a black-box npm import?
Token cost per interaction — how much context is needed to use the library correctly?
Purpose-built intent — was agent-driven usage an actual design goal, or a side effect of being simple/popular?

Dimension	Toolcrib	shadcn/ui	Mantine	Chakra UI (v3)	MUI
Training-data prior	Low, but not uniformly — near-zero for bespoke mechanisms (slice system, `overrides`, `StyleDomain`, event bus, z-index scale); largely inherited for conventional API shape, since components wrap Radix directly (`open`/`onOpenChange`, `value`/`onValueChange`) and a model’s deep Radix/React prior transfers by analogy without ever having seen Toolcrib specifically	Very high (dominant in 2024–2026 AI-coding tutorials/blogs)	Medium	Medium-high	Very high (a decade of docs/StackOverflow/code in training data)
Styling determinism	Highest — no `className`/`style` at all, fixed variant props only	Lowest — full Tailwind class freedom, arbitrary values allowed	Medium — style props + CSS modules, easy to bypass tokens	Medium-high — token-constrained style props (`p={4}`), recipes system discourages arbitrary values	Low-medium — `sx` prop allows arbitrary CSS-in-JS, easy to drift despite theme
Machine-readable API surface	Purpose-built — `component-manifest.json` + `ai-docs/` written for agents	None formal — relies on model’s prior + component source as implicit doc	None formal — relies on TS types + docs site	None formal — relies on TS types + docs site	Strong TS types act as de facto manifest, but no agent-specific format
Cross-turn consistency mechanism	Strong — ThemeSlice system + CSS variables, single source of truth by construction	Weak — `tailwind.config` + CSS vars exist but nothing stops per-turn class invention	Medium — `MantineProvider` theme object is centralized, but style-prop overrides can still drift	Strong — theme object + recipes are the idiomatic path, harder to bypass than Tailwind	Medium — `ThemeProvider`/theme object is thorough, but `sx` overrides bypass it constantly in practice
Vendoring/inspectability	Full vendor, agent sees everything	Full vendor (same model)	npm package, source visible on GitHub but not in-repo	npm package (v3 uses Ark UI + Panda CSS under the hood)	npm package
Token cost per interaction	Low once manifest is loaded, but must be loaded (nothing free from prior)	Low — model often needs little beyond the prompt due to strong prior	Medium	Medium	Medium-high — large API surface, though offset by strong prior knowledge
Purpose-built for agents	Yes, explicitly	No — accidental fit due to simplicity + popularity	No	No	No

Per-toolkit take

Toolcrib — the only one of the five actually engineered around the premise “an AI agent is the primary user.” Wins outright on styling determinism and manifest design — the two dimensions most directly responsible for the “it looked great, then fell apart” failure mode this whole report started from. Its weakness is structural, not conceptual: it has to earn every bit of correct usage through the injected manifest/docs each session, because there’s no training-data prior to lean on yet. Worth being precise about, though: that’s not a flaw unique to Toolcrib — every one of these toolkits had zero prior once, and shadcn/MUI/Chakra’s current advantage here is simply age plus adoption, not something inherent to their design. And it’s less severe than a single “very low prior” score implies, because the gap isn’t uniform across the API surface. Every migrated overlay wraps its Radix primitive directly and exposes the exact conventional controlled pattern (Modal/Popup use open/onOpenChange over DialogPrimitive.Root/PopoverPrimitive.Root; Select uses value/onValueChange) — a model’s deep, well-worn Radix/React prior transfers here almost for free, without ever having seen Toolcrib specifically. The real gap sits narrower than “the whole library”: it’s the genuinely bespoke mechanisms with no external precedent anywhere — the slice system, overrides, StyleDomain, the event bus’s sticky-replay semantics, the fixed z-index scale — where prior can’t transfer because nothing like it exists elsewhere for the model to have learned from. And the cost of losing the rules mid-session is friction, not corruption: every migrated component’s props type is Omit<HTMLAttributes<T>, 'style' | 'className'>, so a model reaching for className gets a real compiler error (plus a dev-mode console warning for the cases TS can’t catch), not a silently drifted UI. The two “wasted turns” costs aren’t the same scale, and shouldn’t be read as comparable — and it’s actually sharper than a scale difference. Toolcrib’s cost is a one-time, self-correcting error: one compile error, one turn, done for the session. Tailwind/shadcn’s cost is the open-ended, recurring one Toolcrib’s own README names as its founding motivation (“tweaking is a horrible affair of wasted turns adjusting padding, margins, etc.”) — but “wasted turns” undersells it, because the failure mode isn’t purely inefficiency. Raw utility classes conflate a value’s resolved output with its semantic intent into one artifact (bg-red-500 doesn’t say whether that red is there because it’s the deliberate danger color or because someone typed it once and moved on), so a later restyle pass has no way to distinguish deliberate customization from arbitrary/leftover code, and can silently overwrite or normalize away a previously-correct decision with no signal that anything regressed. That’s not wasted effort — it’s active regression of work that was already right, and it matches practitioner experience rather than staying theoretical: visual regressions from uncontrolled styling are reported as a chronic, recurring cost across multiple real projects, not a rare edge case — exactly the pattern the semantic/resolved-value conflation above would predict. Framed that way, the net trade tilts toward Toolcrib more heavily than “wasted turns” alone implies — which is the whole thesis of this comparison in the first place. See the companion gap-closure plan for how MCP/RAG-backed on-demand retrieval could shrink Toolcrib’s smaller cost further — and how it should now target the narrow bespoke-mechanism gap specifically, not the whole API surface.
shadcn/ui — paradoxically the most “AI-optimized” toolkit in practice today despite not being designed for it: strong prior + full vendoring means a model can often produce correct, idiomatic code with almost no extra context. But it’s optimized for generation, not consistency — nothing stops the same model from producing three different-looking buttons across three turns, since Tailwind places no real ceiling on invention. This is exactly the gap Toolcrib is targeting.
Chakra UI (v3) — the best of the “not purpose-built” group on styling determinism, because its token-based style props and recipe system make it awkward (not impossible) for a model to reach for arbitrary values. If you want shadcn-adjacent popularity with meaningfully more built-in consistency pressure, this is the closest existing fit.
Mantine — decent middle ground: centralized theme object gives real consistency at the design-token level, but its style-prop API is loose enough that a model can still freelance visual details turn to turn.
MUI — the weakest on determinism precisely because of sx: its theme system is arguably the most mature of all five, but the sx escape hatch is so idiomatic and so heavily represented in training data that models reach for arbitrary inline overrides constantly, undermining the theme’s authority. Best type coverage of the group, though, which does help an agent avoid outright invalid prop usage even if it doesn’t stop visual drift.

Net ranking for “keeps an AI agent’s output visually consistent across turns,” specifically: Toolcrib > Chakra UI ≥ Mantine > MUI > shadcn/ui.
Net ranking for “produces correct code with the least hand-holding today, cold prior only”: shadcn/ui > MUI ≥ Chakra UI ≥ Mantine > Toolcrib.

These two rankings are almost inverted — which is really the whole story. Toolcrib is a bet that the consistency ranking matters more over a long agent-driven project than the cold-start ranking; shadcn’s popularity suggests most of the market is currently betting the opposite way.

A detailed, actionable gap-closure plan addressing every issue raised above is available as a companion document: toolcrib-gap-closure-plan.md.

9. A Meta-Observation: The Same Feedback Loop, at Two Layers

Toolcrib’s founding premise is that AI-generated UI needs a structural mechanism to keep it honest across turns — hence the manifest, the slice system, and the compile-time rejection of className/style. What’s notable, looking at the commit history rather than just the code, is that the project’s own development process ran on the same principle, just implemented socially instead of architecturally.

The commit messages show a recognizable pattern, not just “AI wrote the code”:

“claude review session” → “apply claude review patches” (Aug 11–12) — Claude wasn’t only generating code on request; it was reviewing existing work and flagging what needed to change, with the findings applied as a distinct, later step. That’s a critic role, not just an executor role.
“defect campaign” / “claude fix campaign” — rounds of finding and fixing issues, implying something was actively surfacing problems that then got worked through systematically rather than everything landing correctly on the first pass.
“taste-test patch” — the most telling one. Nothing here was broken; someone used the toolkit and felt something was off, then corrected it on that basis. That’s a distinctly human contribution no amount of automated or AI review catches on its own, because the thing being corrected wasn’t wrong — it just wasn’t right yet.

That same loop — generate, review, catch what review missed, apply judgment the review can’t reach — is structurally identical to what produced the corrections in this very report: several claims here (that a one-off styling escape hatch didn’t exist, that the className failure mode was a hard stop rather than a caught error, that the “wasted turns” comparison favored shadcn) were wrong on first pass and only fixed because someone with direct knowledge of the codebase caught them.

Worth being precise about the framing, though: not “co-equal,” but two non-overlapping failure modes catching each other. Claude can review a large diff for internal consistency at a speed and breadth no solo human matches. It cannot tell you that padding feels wrong, or that a friction point encountered while actually using the toolkit wasn’t reflected in the manifest — that requires the lived experience of building with the thing, which is irreducibly a human contribution. Neither role subsumes the other; “co-equal” risks implying interchangeable, when the more accurate claim is that each role catches errors the other structurally cannot.

The result is a tighter form of dogfooding than “an AI tool built using AI.” Toolcrib enforces this discipline architecturally for the UI code it produces — the manifest and slice system are a structural stand-in for “someone has to catch drift, so make the compiler do it instead of hoping a human notices.” The project’s own development process enforced the same discipline socially — human-AI review loops standing in for what the manifest does mechanically at the component level. Same founding insight, expressed twice, at two different layers of the same project.

10. Bottom Line

Toolcrib’s core idea — remove the model’s opportunity to regenerate (and silently diverge on) decisions that need to stay consistent, whether that’s a CSS class or a z-index value or an overlay’s unmount timing — is a legitimate and clever response to a real problem with AI-driven UI work, and it’s a broader idea than the styling story alone: the shared infrastructure layer (event bus, observer manager, z-index scale, SSR-safe IDs, animation lifecycle) closes a real class of behavioral bugs the no-className rule doesn’t even address. The CLI’s merge/patch/conflict tooling is arguably more thoughtfully engineered than shadcn’s for long-term maintenance, too. But it’s a solo-built, 10-day-old project with roughly two-thirds the component coverage of shadcn/ui and none of shadcn’s community or production track record. Treat it as a promising experiment worth watching or piloting on a low-stakes project, not yet as a like-for-like alternative to shadcn/ui, Mantine, Chakra, or MUI for anything that needs to be safe to build a real product on today.

2026-08-17

Toolcrib Taste-Test: The Raw AI Report on Cascading Style 💩

We let an autonomous AI agent run an N=3 multi-trial stress test on our repository. It hit the wall, diagnosed its own layout failures, and compiled a raw engineering post-mortem tracking the exact anatomy of context drift and broken text contrast.

👉 Click Here to Read the Full Report & Get the CLI

2026-08-14
Prompt & Pray: Why Vibe Coding Needs a Floor, Not Just a Good Prompt
A case for adopting “floor” tooling — component libraries built for AI-mediated development — as standard practice for anyone building real software with an AI coding assistant, instead of hoping each new prompt holds together with the last one.

AI-generated document. This white paper was written by Claude, based on a hands-on experiment building the same application twice — once freehand, once with the toolcrib component toolkit — and re-testing that comparison across a version upgrade and a new feature. The findings are empirical, drawn from live browser testing, source inspection, and a real upstream test suite, not vendor material. “Floor tooling,” used throughout, is a term coined for this paper, not an existing industry label — see the terminology note below for why. Verify anything load-bearing before acting on it.

The claim, up front

If you build software by describing what you want to an AI and accepting what it writes back — “vibe coding,” in the current term — you are not just risking small cosmetic bugs. You are structurally unable to produce consistent, polished results across a project of any real size. Every dialog, every table, every form is a fresh roll of the dice, decided independently, with no memory of how the last one was decided. Call it what it is: prompt and pray — describe what you want, and hope the result holds together with everything that came before it, because nothing about the workflow gives you a reason to expect that it will.

Not because the model is careless, but because nothing in a freehand workflow gives it, or you, a reason to converge on one correct way of doing anything.

Component libraries designed for this workflow — what we’ll call floor tooling — exist to fix exactly this. Not by making the AI smarter, but by giving it (and you) a fixed, pre-decided, pre-tested set of building blocks to reach for instead of reinventing. This paper makes the case for why that matters, using a controlled, repeatable experiment as evidence rather than assertion.

A note on terminology

“Floor tooling” is a term we’re proposing here, not one already in industry use — worth saying plainly rather than letting it pass as established. It was chosen deliberately over the existing terms that sit closest to it, because each of those carries baggage that doesn’t quite fit:
- “Design system” is the nearest match, but the term usually implies a human-curated visual/brand layer — a Figma library, a style guide, something a design team maintains for human designers and engineers to follow by hand. It doesn’t foreground the property this paper is actually about: a toolkit built to be read and reasoned about by an AI assistant, with documentation structured for that purpose specifically.
- “Component library” is accurate but generic — it describes the mechanism (MUI, shadcn/ui, Radix, Chakra all qualify) without describing what makes the AI-native version of one different: the assistant-readable manifest, the AGENTS.md/CLAUDE.md-style instructions, patch-based vendoring built for AI-mediated merges rather than a human running npm update and reading a changelog.
- “Scaffolding” / “boilerplate” describe project setup — the files you get on day one — not an ongoing correctness guarantee that holds for every feature added afterward, which is the actual claim this paper is making.
- “Guardrails” is used broadly in AI contexts for constraining model behavior or outputs in general (tone, safety, refusals). It’s not specific to UI correctness, and using it here would blur this argument into a different one.
“Floor” was chosen specifically because it names the property directly — a baseline every consumer gets automatically, that individual features can build above but not fall below — without importing a definition from an adjacent, not-quite-matching field. If this term doesn’t end up sticking, “AI-native component library” or “AI-native design system” are the closest existing phrases a reader will already recognize.

Built with an AI collaborator, not retrofitted at one

One fact changes how to read all of this correctly, and it’s worth stating plainly: of the toolkit’s own project documents, only README.md is human-authored. AGENTS.md, CONTRIBUTING.md, and SESSION_SUMMARIES.md — everything quoted above — are themselves AI-authored, written by an assistant working on the toolkit, not handed down by a human architect describing a plan from outside. That’s not a weaker form of evidence for the argument this paper is making; if anything it’s a purer instance of it. The claim isn’t “a human designed a good process and described it in a document a model can read” — it’s that the actual operating instructions a repository runs on were written by the same kind of model this whole paper is about, based on what it found working the codebase, and then kept in place by whoever maintains the project as the real, load-bearing contributor guide, not a curiosity. AGENTS.md‘s own line — the toolkit “is a React component library designed specifically for AI code generation (‘vibe coding’) — an AI that’s building a UI tends to hand-roll the same popups, slide-outs, and ad-hoc CSS over and over. Toolcrib exists to give it a structural toolkit instead” — is an AI’s own account of why this exists, not a founder’s mission statement, and it happens to match the exact failure mode this paper’s own experiment reproduced independently.

The one caveat worth naming honestly: an AI’s self-report about the value of AI-facing tooling is not disinterested third-party testimony, and this paper doesn’t treat it as such. What makes it evidence rather than assertion is what a human did with it — accepted it, kept it in place across releases, and let it govern how the actual codebase gets contributed to, rather than discarding it as filler.

More specifically, the actual mechanism is a literal, repeatable practice, not a general philosophy:

The model is asked directly where the generation problems are, and the fixes address what it reports.

Checked directly in the repository: a SESSION_SUMMARIES.md prompt has an agent write up its own contributor sessions under five fixed headers, one of which is simply “Friction” — defined as “anything that took extra turns, wasn’t obvious from AGENTS.md/CORE.md/the component manifest, or required guessing,” and flagged explicitly as “the most valuable part of the post.” AGENTS.md closes the loop on the other side: it describes running “external review sessions” whose entire job is to read the whole codebase at once and ask “does this new code repeat a mistake already found and written up elsewhere” — precisely because, in the file’s own words, “nothing in a single generation pass forces a check for” that, and “nothing in this repo’s CI checks that either.” What that review finds gets written back into AGENTS.md at the level of the general mechanism, not the specific instance, so the next occurrence of the same underlying bug is recognized on sight instead of re-diagnosed from scratch.

This isn’t a hypothetical process — real examples are publicly posted to the repository’s own Discussions, under exactly this template. Discussion #25, “Badge status pill,” is one such post — coincidentally, the build of the very <Badge> component this paper’s own experiment relied on for its priority/status pills. Its Friction section reads, in full:

Third small factual gap found across the last three items in this batch (theme-slice doc’s off-by-one count, ToolcribProvider’s wrong file path, now this). None individually significant, but the pattern is now clear enough to state plainly: every hand-off doc in this repo so far has had at least one concrete, checkable claim that didn’t hold up against the actual source, sitting right next to otherwise-sound design reasoning. Continuing to verify each doc’s specific factual claims independently before acting on them, not just its overall approach — same as the last two check-ins noted, now with a third data point.

That’s the mechanism this section describes, caught in the act: an agent doing the actual work, noticing its own hand-off documentation was factually wrong for the third time running, naming the pattern explicitly rather than treating each instance as an isolated slip, and posting that finding publicly rather than quietly self-correcting and moving on.

This is a real, verifiable difference in how the toolkit’s own defect-finding actually works, and it plausibly explains why its infrastructure closes gaps a human-first process might never have surfaced: a human maintainer optimizing for other humans has no particular reason to go looking for a mount-order race between two unrelated subtrees, or to measure how much context a JSON manifest split one way costs versus another. Those are exactly the things that turn up when the actual question repeatedly asked is “where did this break for you,” aimed at the party that’s doing the generating.

Why this isn’t really about accessibility

It’s tempting to sell this idea on compliance grounds — “your AI-written modal is missing aria-invalid,” “your table isn’t keyboard-navigable.” Those things are true and we verified them directly, but leading with them undersells the argument and mistargets the audience. Most people building a personal tool, an internal dashboard, or a weekend project correctly don’t care whether a screen reader can read their delete button. Pitching floor tooling as an accessibility fix asks them to value something they don’t yet value, and the pitch fails on that basis alone.

The real, cross-cutting cost is something every builder already feels, whether or not they have the vocabulary for it:

The app gets flakier and less consistent the more you add to it.

The second modal doesn’t quite behave like the first. A feature that worked in one session breaks something adjacent in the next. Fixing a bug in one place doesn’t fix the same bug sitting somewhere else in the codebase, because it was never “the same bug” to begin with — it was two independent instances of the model making a plausible-but-uninformed choice, twice.

And these regressions don’t announce themselves. A change made to satisfy one request can quietly break something unrelated that nobody was looking at in that turn — a filter that stops narrowing correctly, a form that silently drops a field on save — and because nothing is asserting what “still working” means, it can sit undiscovered for turns or sessions, surfacing only when a person happens to click through that specific path again. At that point, diagnosing it costs far more than it would have to prevent it: the person (or the assistant, prompted fresh) has to first notice something’s wrong, then figure out which of several intervening changes caused it, with no test failure pointing at the culprit and no memory of what the code looked like before it broke. What would have been a one-line fix caught immediately becomes a multi-turn investigation caught late.

That’s the actual disease. Accessibility gaps are just the easiest symptom to demonstrate with a script.

What a floor actually replaces

An experienced developer working alongside an AI assistant supplies something that has nothing to do with typing skill: judgment. They know a destructive confirmation shouldn’t be dismissible by clicking outside it, while a general-purpose form should be. They know a data table with more than a page of rows needs virtualization even if nobody asked for it. They know their color palette, spacing scale, and font weights should be the same in file 40 as they were in file 1, and they enforce that by review, habit, and memory across sessions the AI itself doesn’t have.

But knowing the right call is not the same as the model executing it correctly on the first try, and a floor only guarantees the former. Reaching for a component that has virtualization built in is a judgment call the toolkit encodes; whether the model wired it up correctly for this specific dataset, this specific column configuration, this specific edge case, is a separate question the toolkit cannot answer for you. That gap still has to be closed by a curated debugging process — ideally automated tests that catch a regression the moment it’s introduced, and at minimum deliberate manual inspection by whoever is driving the build. A floor changes what’s being verified and lowers how often verification turns something up, because the default choice was already sound — it does not remove verification from the workflow.

Most people using an AI assistant to build software do not have that judgment yet, and re-deriving it turn by turn is not a realistic expectation — that’s precisely why they’re using an assistant to build in the first place. A floor is what stands in for that missing judgment. It doesn’t make the assistant smarter; it removes the need for either party to independently have the judgment, because the correct choice was already made once, centrally, by whoever built the toolkit, and it’s the only choice available.

This reframes who benefits. It’s not “people who would care about correctness if they understood it.” It’s anyone whose actual bottleneck is not knowing what to ask for or how to evaluate what came back — which describes most people vibe coding, not a niche.

The theme engine specifically: consistent visuals that don’t get reinvented per element

One piece of this deserves its own explanation, because it’s easy to wave at “consistent styling” without showing what actually enforces it. Checked directly in toolcrib‘s own source rather than assumed: every component reads its colors, spacing, radius, and typography from one shared set of CSS custom properties, generated once by a real color-theory engine — pick a base color and a harmony mode (monochromatic, analogous, split-complementary, triadic, tetradic), and the toolkit derives a full, coordinated palette from it. A Button added in month three and a Badge added in week one pull from the same generated palette by construction; there’s no second decision to make, and so no opportunity for the two to quietly drift apart the way two independently-styled dialogs did in this experiment’s uncontrolled leg.

Two things about this are worth calling out specifically, because they run against the usual complaint that shared design systems fight you the moment you want something to look different:
- It’s still configurable, not fixed. Any component can override its subtheme or spacing per instance — this is exactly the mechanism that let the priority/status badges in this experiment’s table render in different colors (success green, warning amber, error red) while still pulling from the same underlying palette, rather than each badge instance hand-picking a hex code.
- The palette generator won’t let a configuration choice become unreadable. Inspected the actual contrast-checking function in the theme engine’s color math: when a color is adjusted for a harmony or a custom base color, the engine iteratively nudges it until it clears a minimum WCAG contrast ratio against its background, before it’s ever handed to a component. A person picking colors with no design background can choose a base hue that would, unadjusted, produce low-contrast text — and the floor corrects for it automatically, the same way it corrected Select‘s missing id in this report’s earlier findings, without anyone needing to know contrast ratios exist.
This is the same substitution-of-judgment argument as the rest of this section, applied specifically to visual design rather than interaction behavior: consistent, accessible-by-default visuals aren’t the result of anyone on the build remembering the rules — they’re the result of there being exactly one place those rules are encoded, generating everything downstream of it.

The event bus and shared observers: architecture the model doesn’t have to invent per feature

There’s a second, less visible piece of infrastructure worth documenting on its own, because it prevents a specific failure mode that has nothing to do with styling or accessibility: an AI assistant reinventing React state management, badly, feature by feature.

Two unrelated components that need to react to the same thing — a tab strip and its content panel, a toast notification triggered from deep inside a form, a data table that needs to know when its container is resized — normally have to be wired together somehow: lifted state, a shared context provider, or callback props threaded down through however many layers separate them. Each of those has a failure mode an unsupervised assistant reliably reaches for and reliably gets slightly wrong: prop-drilling that breaks the moment a new layer is inserted between parent and child, or a context provider that has to be remembered and wrapped around every new subtree that needs it, easy to forget on the fourth feature when it was only modeled correctly on the first.

Checked directly in the toolkit’s source: toolcrib sidesteps this with a single, strongly-typed, app-wide event bus (aiBus) that any component can publish to or subscribe from, with no parent-child relationship required at all. Components that have no reason to know about each other — a TabStrip and its own TabStrip.Panel, for instance — communicate by emitting and listening for the same named event (tab:changed) rather than sharing a context. The bus’s own source comment explains a specific, real problem this solves: two independent subtrees have no guaranteed mount order, so a naive implementation can permanently miss an initial state broadcast depending on which one happens to render first. The bus fixes this once, centrally, by letting specific events replay their last known value to a new subscriber the moment it subscribes — a piece of correctness a from-scratch implementation would have to rediscover (or simply never notice failing intermittently) on every feature that needed the same pattern.

The same centralization shows up one layer lower, at the browser API level. Resize and intersection tracking — knowing when an element’s size changed, or when it’s scrolled into view — normally means a fresh ResizeObserver/IntersectionObserver instantiated per component that needs it, which is wasteful at best and a source of subtly different behavior at worst if two features implement the debounce timing differently. toolcrib runs exactly one instance of each, shared across the whole app, and routes every component’s resize/intersection data through it via a single hook — meaning a virtualized table, a deferred-content section, and a custom component built later all get identical, centrally-debounced measurement behavior for free, rather than three separately-written approximations of the same thing.

None of this is about the visual polish the rest of this paper focuses on. It’s a structural floor for state and event handling itself — the same “one correct way encoded once” principle applied to application architecture rather than to a specific UI element, closing off an entire category of bug (drilled-prop breakage, forgotten context providers, mount-order races, redundant browser observers) before it has a chance to appear even once. It’s also a plausible example of the provenance point made earlier: a mount-order race between two unrelated subtrees is exactly the kind of defect that surfaces from actually building many features with an AI agent in the loop, repeatedly, rather than from a human designing the library’s architecture up front and hoping it holds.

This is one solution, not a bundle of unrelated ones

Laid out separately, these can read like a list of features a marketing page would bullet-point. They aren’t separate. The same underlying move — take a decision that would otherwise be re-made, slightly differently, every time it comes up, and encode it exactly once so nothing downstream can drift from it — shows up at every layer this paper has examined, and each layer happens to correspond to a different kind of pain a builder feels without necessarily connecting it to the others:
- Visual pain (“why doesn’t this match”) is addressed by the theme engine — one generated palette, with built-in contrast correction, that every component pulls from.
- Interaction pain (“why does this feel different from the last one”) is addressed by the component layer itself — a Modal and an AlertDialog that behave identically everywhere they’re used, with focus-trap and dismiss semantics decided once.
- Architectural pain (“why did fixing this over here break something over there”) is addressed by the event bus and shared observers — state and cross-component communication that doesn’t have to be re-invented, and re-gotten-slightly-wrong, per feature.
- Confidence pain (“did that actually work, and will it keep working”) is addressed by the upstream test suite — 456 tests catching a regression the moment it’s introduced, rather than a builder’s one-time manual click-through standing in for verification indefinitely.
A builder feeling any one of these would reasonably look for a point solution — a linter, a design token file, a state-management library, a testing tutorial — and each would help exactly the slice of pain it targets. What actually generalizes is the single design principle underneath all four, and that’s the more accurate way to describe what’s being adopted: not a components library that also happens to have decent theming and some tests, but one recurring engineering decision — stop re-deciding the same thing every time it comes up — applied consistently enough that it shows up in the palette, in the dialogs, in the state layer, and in the CI pipeline, as the same fix in four different places rather than four different fixes.

The evidence: a controlled experiment

To test this rather than assert it, the same task-management dashboard was built twice from an identical specification: once with a general-purpose AI coding assistant working freehand (“uncontrolled”), and once using the same assistant with toolcrib, an AI-native component toolkit, vendored into the project. Both were then extended with an identical new feature, requested in plain language with no toolkit-specific vocabulary, and both were re-tested after a toolkit version upgrade.

Finding 1: the floor produces dramatically less code to get the same result

Metric Freehand With floor tooling Change
Raw HTML elements written 68 6 −91%
Manual styling attributes written 60 0 −100%

The six raw elements remaining were plain text (<h1>, <p>) or below the component’s granularity — not a gap in the toolkit, just content with nothing left to replace.

Finding 2: freehand code reproduces its own mistakes, even in the same sitting

Both builds were asked, in identical plain language, to add a delete-confirmation step: “show a quick confirmation that summarizes what’s being removed… let them back out instead of deleting.”

The freehand build wrote a second dialog from scratch. Despite the first dialog sitting in the same file tree, in the same session, moments earlier, the second one reproduced the exact same defects independently: no dialog semantics, keyboard focus escaping the dialog, no way to dismiss it with the keyboard. Not a different set of problems — the identical ones, arrived at twice, separately.

The floor-tooling build didn’t write a new dialog at all. It reached for a different, more specific component the toolkit already provided — one purpose-built for exactly this “can’t be casually dismissed” pattern — correctly, based only on the wording of the request, without ever being told the component’s name. That pick wasn’t luck: the component’s own doc comment states plainly that it’s for exactly this situation and explicitly contrasts it with the more general option — the mechanism behind this is examined directly in the Recommendation section below. The result needed less than half the new code, zero new styling, and came with keyboard support, dismiss behavior, and screen-reader semantics already verified correct upstream, before this project ever adopted it.

Finding 3: retrofitting consistency after the fact is expensive, and it doesn’t compound

Fixing the freehand build’s two broken dialogs required writing a new, shared piece of infrastructure from scratch — roughly ninety lines of hand-built logic to handle keyboard trapping, escape behavior, and labeling — and then reopening both already-finished files to adopt it.

That fix helps exactly those two dialogs. It does nothing for the next one. The next hand-rolled dialog a future session writes is exactly as likely to omit the same things, because nothing about this fix travels with the project automatically — it only helps if whoever writes the next dialog happens to remember this file exists and chooses to reuse it.

Compare that to the floor-tooling side, where the third dialog this project will ever need requires no new infrastructure at all — just picking the right existing component, the same as the second one did.

Finding 4: the floor is continuously tested by someone other than you

The freehand project had no test suite at all — no test files, no test runner even installed. Every guarantee about its behavior was true only as of the one-time manual verification performed during this experiment; nothing re-checks it the next time the code is touched.

The toolkit’s real upstream repository, checked directly rather than taken on faith, has 456 automated tests running on every commit, including named regression tests written specifically for defects like the ones found here. A version upgrade during this experiment carried forward four previously-documented defects — fixed, upstream, before this project ever touched the new version — and every one of those fixes was verified, live, to actually work. This is the mechanism that makes a floor durable: a mistake found once gets fixed once, centrally, and every project built on top of the toolkit inherits the fix automatically the next time it updates, with no memory or manual effort required on the builder’s part.

What this doesn’t fix

Honesty about the limits makes the case stronger, not weaker.
- Coverage is bounded, but not by app category — it’s bounded by two specific kinds of surface. A contact form, an email signup, a “confirm before you quit” prompt, a settings screen with toggles and sliders — none of that is CRUD, and all of it is exactly the Form/Button/Modal/Toggle layer this toolkit provides. Checked against the full component manifest: the real exclusion isn’t “games” or “landing pages” as categories. It’s:
  - Non-DOM rendering and simulation — a game’s actual canvas/WebGL draw loop, physics, sprite animation. This is a different medium entirely from the HTML/CSS components a toolkit like this provides; the toolkit has something to say about the menu that pauses the loop, nothing about the loop itself.
  - Surfaces where looking generic is the failure mode, not a shortcut — a landing page’s hero section, a game’s stylized main-menu art, anywhere a brand’s visual identity is the entire point. Reaching for a standard Card there undermines the goal, since the toolkit’s value is consistency and a differentiated visual signature wants the opposite. The same landing page’s contact form and FAQ accordion, a few sections down, are back to being exactly what the toolkit is for.
  Most of what’s actually inside a game or a marketing site — every prompt, form, dialog, and menu — benefits the same way this experiment’s dashboard did. The exclusion is narrower and more specific than “skip it for these app types”: skip it for the rendering loop itself, and for the handful of surfaces deliberately trying not to look like anything else.
- It’s not free. A floor-tooling build carries a heavier dependency footprint and a somewhat more complex mental model than “just write some markup.” For code that’s genuinely disposable — a one-shot script for a single use, not a static mockup (a different job entirely, see the Recommendation section) but real functional code nobody will ever open again after today — that overhead may not be worth paying.
- It doesn’t replace all judgment, only some of it. Choosing the right component for the right situation still requires the request to be clear about intent — the assistant made the right call here because the phrasing carried real information (“let them back out instead of deleting”), not because the toolkit reads minds.
- It doesn’t replace debugging. A correct component choice can still be wired together wrong — the wrong prop passed, a malformed dataset handed to it, a state update that doesn’t fire when expected. The floor lowers how often something goes wrong by starting from a sound default instead of an improvised one, and it gives you a known-good baseline to debug against — but the debugging step itself, tests where they exist and careful manual verification where they don’t, is still the builder’s responsibility every single time, not something the toolkit performs on your behalf.
- This experiment ran in one continuous session. The freehand build’s internal consistency — the fact that its second dialog at least matched its first dialog’s visual style — depended on the same model, in the same sitting, having the first file in its own context. A real project built across many separate sessions, days or weeks apart, has no such guarantee, and this experiment did not directly measure how much further that consistency erodes over time. It’s a reasonable inference from how these tools work, not a separately verified result.
Recommendation

If what you actually want is a static mockup to look at, there are tools purpose-built for that — Google’s Canvas among them — and none of this applies to you; go use one of those instead. But if you’re using an AI assistant to build something with real, working functionality, that choice already means you’ve crossed the line that matters: you intend for it to run correctly, not just look right in a screenshot. There’s no lighter-weight version of “vibe coding a functional app” for which a floor stops being worth adopting — treat it as a default decision, not an optional add-on. One division of labor is worth being explicit about throughout what follows: the builder chooses which toolkit to adopt and says what they want built; the AI agent is the one actually reading the toolkit’s documentation, deciding between components, and re-consulting that guidance on every feature after the first. None of these recommendations ask a non-expert builder to go read AGENTS.md themselves — that’s precisely the point of adopting a floor in the first place.
1. Match the toolkit to your app’s shape. Confirm its component set actually covers what you’re building (forms, tables, overlays, dashboards) before adopting it; it won’t help with what it doesn’t have.
2. Favor a toolkit whose documentation distinguishes good and bad usage patterns, not just one that lists components. This is a correctness requirement, not a convenience one: a library can have exactly the right primitive sitting in it — an AlertDialog right next to a Modal, a lightweight CardSimple right next to a full Card — and an agent with no guidance on the difference will default to whichever one is more familiar from training data, using the wrong one every time despite both being available. Checked directly in the toolkit’s own components: this contrast is a deliberate, tagged feature (@ai-hint on CardSimple; AlertDialog‘s own doc comment says “reserve this for… not general-purpose content — use Modal for that”) — the exact infrastructure this paper’s AlertDialog-over-Modal result in Finding 2 depended on. Without it, the components still exist, but the floor doesn’t reliably hold, because nothing tells the agent which option applies.
3. Favor a toolkit that knows how to prime the system prompt automatically, rather than relying on the builder to do it. Checked directly: toolcrib init drops an AGENTS.md and a one-line CLAUDE.md (just @AGENTS.md) at the project root — files that Claude Code, Cursor, and similar tools load into context automatically at the start of a session, with the toolkit’s own instructions stating plainly that its content belongs in whichever convention the builder’s agent already uses. A non-expert builder never has to know this file exists, remember to reference it, or paste anything into a system prompt by hand; the priming happens the moment the toolkit is installed. A toolkit that only ships human-facing README prose has no such path — the guidance only reaches the agent if the builder thinks to go find it and hand it over themselves, which is exactly the step a non-expert can’t be relied on to take.
4. Auto-priming isn’t the whole job — the agent still has to go deeper per feature. What gets primed automatically is the toolkit’s core rules, not the full manifest or every component’s individual doc comments; those are too large to preload in full. The benefit demonstrated in this experiment came from the agent actively consulting those specifics fresh, for the new request, not from having skimmed them once, earlier, for something unrelated.
5. Treat a version upgrade as free maintenance, not a chore. The mechanism that makes a floor durable is precisely that fixes propagate forward automatically; skipping upgrades forfeits that benefit.
6. Don’t expect it to replace clear communication. The right component still has to be inferable from what you actually asked for — vague requests get vague results regardless of what’s available underneath.
7. Still verify what got built. Picking the right component is not the same as it being wired up correctly for your specific case. Run whatever automated checks are available, and where none exist, actually click through the feature yourself before trusting it — the floor lowers how often you’ll find something wrong, it doesn’t excuse you from looking.
Put together, none of this is really a components-library pitch. It’s an argument about where a non-expert builder’s actual limits sit, and about a specific kind of tool built to sit exactly there instead of asking the builder to move. The judgment that’s missing — which dialog shouldn’t be dismissible, when a table needs virtualization, what a consistent palette even means — doesn’t get taught to the builder or magically acquired by the model; it gets encoded once, by a process that repeatedly asked an AI agent where its own output broke down and fixed what it found, then delivered back into every project automatically, the moment the toolkit is installed, without anyone having to go looking for it. That’s a different claim than “this library has nice components,” and it’s the one this paper has tried to actually demonstrate rather than assert: consistent visuals, consistent interaction, consistent architecture, and a continuously-verified confidence that any of it still works are four faces of the same fix, not four separate features bundled onto a components library for marketing purposes.

None of that erases the caveats stated plainly above — the scope stops at the edge of what the toolkit’s manifest covers, the debugging step never goes away, and a vague request still gets a vague result no matter what’s installed underneath. But within that scope, the pitch is not “your app will be accessible.” It’s this:

The parts of quality you don’t yet have the vocabulary to ask for or the experience to evaluate get supplied anyway, consistently, for as long as the project lives.

That is the actual promise vibe coding makes and, on its own, cannot keep.
2026-08-20

Metric	Freehand	With floor tooling	Change
Raw HTML elements written	68	6	−91%
Manual styling attributes written	60	0	−100%

Refactor Application Experiment, Rerun: Toolcrib v0.5.0 vs. v0.4.0

AI-generated report. This document, the code in both experiment legs, and all findings below were produced by Claude. Verify anything load-bearing before acting on it.

Factors: uncontrolled (free-form AI-generated React/Tailwind) vs. toolcrib (escape-llc/toolcrib CLI refactor) Measurement: how much of the uncontrolled markup gets refactored into toolcrib’s components — and, matching the original report’s later scope, whether the four specific behavioral gaps found in toolcrib’s own Select, FormField/Input, and DataTable components have since been fixed. Purpose of this run: repeat the identical experiment design against toolcrib v0.5.0, including the markup-coverage measurement and the head-to-head behavioral re-tests (Modal, DataTable, Form validation) from the original report, tested live in a browser exactly as before — not inferred from source or the manifest.

Headline findings:

All four concrete bugs documented in the original report’s ACTION_PLAN.md are fixed in v0.5.0 — verified live in a browser, not just by reading the diff.
A follow-on extension test — adding an identical new feature to both legs from a plain-language request — is the most significant finding of this rerun. The toolcrib leg picked a more specific, better-suited primitive (AlertDialog over the already-used Modal) purely from the request’s wording, at less than half the raw markup and zero new styling attributes. The uncontrolled leg re-wrote its dialog chrome from scratch and reproduced its first modal’s accessibility bugs exactly — and retrofitting a shared floor afterward (a guaranteed baseline of correct behavior every dialog gets automatically, rather than each one re-earning it by hand) cost real, measured extra work with no test suite guarding the fix, unlike toolcrib’s 456 passing upstream tests enforcing the same guarantees on every commit.

Method

Same design as the original run: scaffold an “uncontrolled” Vite + React 19 + TS + Tailwind task/project dashboard (stat cards, filter toolbar, sortable-looking task table, full modal form), fork it, run toolcrib init --situation refactor --version 0.5.0 → review patches → toolcrib apply → install deps → confirm the vendored-but-unused project still builds → refactor every file against the toolcrib API → recount raw markup → verify functional equivalence in a real (Puppeteer) browser.

Results

Metric	Uncontrolled (before)	Toolcrib v0.5.0 (after)	Change	v0.4.0 result (prior run)
Raw HTML elements	68	6	−91.2%	−90.8% (87→8)
`className` attributes	60	0	−100%	−97% (65→2)
Patches applied	—	156 / 156 clean	—	116 / 116 clean

(Baseline element/class counts differ slightly from the original run — 68/60 vs. 87/65 — because this rebuild of the same spec happened to write marginally leaner hand-rolled JSX; the percentage reduction is the comparable figure, not the raw counts.)

What’s left raw, and why (all legitimate, same category as last time):

<h1>/<p> in the app header, <span> for a stat card’s value — plain text content, nothing to replace.
3 small <div>s used as title/description wrapper inside DataTable‘s cell renderer — below the granularity toolcrib operates at, same as the original run’s leftover table-cell <div>s.

No coverage gaps this time. The single gap flagged in the v0.4.0 report — no badge/pill component — is gone: <Badge subtheme size icon> now exists and both PriorityBadge/StatusBadge refactored onto it cleanly, at zero raw markup cost.

Friction-point comparison against the v0.4.0 run

Friction point (v0.4.0)	v0.5.0 status
`verbatimModuleSyntax` TypeScript incompatibility — broke the build immediately after `apply`, required manually disabling the option in `tsconfig.app.json`	Fixed. `tsc -b` and `vite build` both passed with zero changes needed, immediately after `apply` and `npm install`.
React version conflict flagged during `init`	Fixed / no longer flagged. `CORE.md` now states React 18.3+ and 19.x both resolve as compatible automatically; `init` didn’t stop to ask.
`CardSimple` silently required `children`, contradicting the manifest’s short description (“title/subtitle props are header-only”)	Fixed. `children` is now a required prop in `CardSimple`‘s own type signature, matching actual behavior — no surprise needed source-reading to discover it this time.
Missing badge/pill component (coverage gap, not a bug)	Closed. `<Badge>` now ships in Data Display.
Unauthenticated `api.github.com` 403 on `init`/`versions` calls (sandbox IP rate-limited)	Not re-tested. This run pinned `--version 0.5.0` from the start (same workaround as last time), so the rate-limited code path wasn’t exercised either way. This is an environment/API-limiting issue, not something a toolcrib version bump would fix, so it’s reasonable to assume it’s still present if `init` is run without `--version`.

Functional verification

Driven in a headless browser exactly like the original run:

New Task modal — opens with focus trap and backdrop, all six fields present and empty, Cancel/Save wired correctly.
Status filter — selecting “Done” narrowed the table from 6 rows to the 2 actual “Done” tasks; stat cards stayed accurate.
Edit prefill — clicking “Edit” on “Write Q3 retro notes” opened the modal with every field correctly populated (title, description, assignee, due date, priority, status) via <Form initialValues>.
Delete — clicking “Delete” removed the row; table count dropped from 6 to 5 immediately.
Zero console/page errors across every interaction in this run (same clean result as v0.4.0).

Head-to-head: the four `ACTION_PLAN.md` gaps, re-tested live in v0.5.0

The original v0.4.0 report went beyond markup counting and found four concrete, verified bugs in toolcrib’s own components via live browser testing (not source-reading alone), handed off as ACTION_PLAN.md. This rerun re-tested all four the same way — actually triggering each behavior in a headless browser, not just diffing source.

#	Gap (v0.4.0)	v0.5.0 status	How verified
1	`<Select>`‘s trigger renders with no `id` and no `aria-label` — breaks label association for every field using it (3 of 6 modal fields, plus both toolbar filters)	Fixed. `Select.tsx` now sets `id={effectiveId}` where `effectiveId = id ?? fieldName`.	Opened the New Task modal and checked all 6 `<label for>` → target resolutions live in the DOM: 6/6 now resolve (Priority and Status selects included), up from 3/6.
2	Validation error state not wired to `aria-invalid`/`aria-describedby` — `FormField`‘s error `<span>` had no `id` for anything to point at	Fixed. `FormField`‘s error span now has `id={errorId}`, and `Input`/`Textarea`/`Select` all set `aria-invalid`/`aria-describedby` off `isError`.	Submitted the New Task form empty and inspected the `title` field live: `aria-invalid="true"`, `aria-describedby="title-error"`, and that id resolves to a real element whose text is exactly “Title is required.”
3	`DataTable`‘s sortable `<th>` had no `tabIndex`, `role`, `onKeyDown`, or `aria-sort` — sorting was mouse-only	Fixed. Sortable headers now get `tabIndex={0}`, `onKeyDown` (Enter/Space), and `aria-sort` reflecting current state; non-sortable columns correctly get `tabIndex={undefined}` and no `aria-sort`.	Focused the “Assignee” header via `Tab`, pressed `Enter` twice live: rows re-sorted ascending then descending each time, and `aria-sort` flipped `"ascending"` → `"descending"` in step.
4	`Column.sortable`‘s JSDoc said `@default false`, but the code checked `col.sortable !== false` — an omitted `sortable` silently defaulted to sortable, giving the “Actions” column a false pointer-cursor affordance	Fixed. Code now reads `const isSortable = col.sortable === true;`, matching the JSDoc’s `@default false` exactly.	Clicked the “Actions” header directly: `cursor: default` (not `pointer`), `tabIndex: -1`, no `aria-sort`, and the click was a genuine no-op — row order unchanged.

Not a regression check on everything, but nothing new turned up either. The rest of the original Modal behavioral checks — dialog role, focus trap holding across 30 tabs, Escape-to-close, backdrop-click-to-close — were never gaps in v0.4.0 (Radix already handled these), and all four re-confirmed working identically in v0.5.0. Zero console errors or warnings across the full re-test sequence (open modal, tab, sort via keyboard, edit, close).

Extension: a new feature, worded as a consumer would ask it

The claim under test in this section, stated plainly: without a shared, tested component floor, an AI coding assistant working freehand (“vibe coding”) will tend to repeat the same defects across independently-written features rather than converge on consistent, correct behavior — and once that inconsistency exists, closing it costs real extra work that a component-based approach never has to spend in the first place. This section and the two after it exist specifically to test that claim against evidence, rather than assume it in either direction.

To test whether the coverage/correctness pattern holds beyond the original build — not just on markup the model wrote once, deliberately, with the manifest open — a new feature was added to both legs from an identical, plain-language request (no toolcrib vocabulary, no component names):

“Before someone deletes a task, show a quick confirmation that summarizes what’s being removed — the title, who it’s assigned to, and its current priority and status — so people don’t accidentally delete the wrong thing. Let them back out instead of deleting.”

This was chosen because it forces each leg to repeat an earlier markup concept — a second overlay/dialog, and a second use of the priority/status badges — the exact condition under which a hand-rolled codebase either duplicates its own prior pattern (with its own quirks) or a component-based one just reaches for the matching primitive again.

Uncontrolled leg: wrote a new DeleteConfirmModal.tsx from scratch. It reused the two PriorityBadge/StatusBadge helper functions (trivial, since those are just plain functions already imported elsewhere) — but the dialog chrome itself (backdrop, centered panel, buttons) is 100% fresh JSX, structurally similar to the first modal but not sharing any code with it. 11 new raw elements, 11 new classNames. And it reproduced the exact same defect class as the first modal, confirmed live: no role="dialog", focus escapes after a few tabs (landed on a background “Delete” button), and Escape does nothing. A second, independently-written implementation of the same UI concept, with the same bugs, found and fixed nowhere in relation to the first instance.

Toolcrib leg: rather than reaching for the already-used <Modal>, the correct primitive here is actually a different, more specific component — <AlertDialog> — whose own doc comment describes exactly this use case: “a blocking confirmation dialog that cannot be light-dismissed… for destructive/irreversible actions… use <Modal> for general-purpose content.” This is real evidence the toolkit’s design vocabulary maps onto how a consumer actually phrases a request (“let them back out instead of deleting” → a non-dismissible confirmation, not a normal overlate) without ever using toolcrib’s own terms. The implementation is 5 new raw elements (all plain text/summary-box wrappers, same category as every other file), 0 new classNames, built from 7 AlertDialog slot components plus the two already-existing Badge components — genuinely reused, not reimplemented.

Live-tested every claim in AlertDialog‘s own doc comment rather than trusting it:

Check	Uncontrolled (new modal)	Toolcrib (`AlertDialog`)
New raw elements introduced	11	5 (all legitimate)
New `className`s introduced	11	0
Reused an existing component for the dialog chrome	❌ wrote a new one	✅ `AlertDialog`, distinct from the already-used `Modal`
Has `role="alertdialog"` / accessible name	❌ no role at all	✅ `role="alertdialog"`, `aria-labelledby` wired
Focus trap	❌ escapes after ~10 tabs	✅ holds after 10 tabs
Backdrop click dismisses	(n/a — no such convention tested)	✅ correctly does not dismiss — matches the doc’s “cannot be light-dismissed” claim for destructive actions
`Escape` dismisses	❌ no-op	✅ closes, matching the doc’s stated Escape-still-works convention
Functional delete (row count, dialog closes)	✅ 6→5 rows, closes correctly	✅ 6→5 rows, closes correctly

Bottom line on the extension: the hypothesis held. Asked in plain consumer language with no toolcrib-specific terms, the toolcrib leg picked the more specific of two overlay primitives correctly (AlertDialog over the already-used Modal) based on the semantics of the request (“let them back out” implies a deliberate, non-light-dismissible choice), producing less than half the raw markup and zero new styling attributes — and, just as importantly, avoided reintroducing the same class of accessibility bug the uncontrolled leg’s second implementation repeated wholesale.

The cost of retrofitting a floor, measured concretely

“Floor,” as used throughout this report, means a baseline of correct behavior that every consumer of a shared component gets automatically — accessibility semantics, keyboard support, consistent dismiss behavior — as opposed to something each individual feature has to separately get right by hand. A component library either has one (every Modal behaves the same way because there’s one Modal) or it doesn’t (every hand-rolled dialog is only as correct as that particular session happened to make it).

The extension test above showed the uncontrolled leg’s second dialog reproducing the first one’s accessibility bugs independently — but “the same mistakes recur” is a claim worth quantifying, not just asserting. So this rerun actually did the retrofit: built a shared, hand-rolled Dialog primitive for the uncontrolled leg and migrated both TaskModal and DeleteConfirmModal onto it, fixing every bug found in both dialogs (missing role, no focus trap, dead Escape, 0/6 unassociated labels in TaskModal) in one place instead of two.

What that retrofit actually cost:

	Toolcrib leg	Uncontrolled leg
Shared dialog primitive already existed before this feature was requested?	Yes — `Modal` and `AlertDialog`, both pre-built, pre-tested	No
New infrastructure code required to get a correct floor	0 lines — picking the right existing primitive was the entire task	86 lines (`Dialog.tsx`): manual focus-trap logic (query focusable elements, trap `Tab`/`Shift+Tab`), `Escape` handling, configurable backdrop-dismiss, focus restoration on close, an `aria-labelledby` id generator
Files that had to be revisited (not written fresh, actually re-opened and edited) to adopt the fix	0 — both dialogs were correct from the first line written	2 (`TaskModal.tsx`, `DeleteConfirmModal.tsx`) — every `<label>` needed a manually paired `id`/`htmlFor`, every bit of dialog chrome needed replacing with the new shared wrapper
Did fixing dialog #2’s bugs also require re-touching dialog #1?	n/a — neither needed fixing	Yes. The bug wasn’t in “the second dialog,” it was in the pattern both dialogs independently used. Fixing it meant going back into code that had already shipped and was already presumed done.

This is the concrete shape of the claim being tested. A single vibe-coding session that writes one dialog, then a second dialog implementing the same concept independently, doesn’t get inconsistency between them by bad luck — it gets it because nothing in the first dialog constrained what the second one could look like or how it could behave. TaskModal and DeleteConfirmModal were written minutes apart, by the same session, with the first dialog sitting right there in context — and still diverged in nothing (same bugs, same omissions) because there was no shared floor forcing convergence, only two separate acts of “write a plausible-looking dialog.” Every additional overlay a real project accumulates over weeks of separate sessions — a settings panel, a confirmation for some other destructive action, a share dialog — is another independent roll of the same dice, each one only as good as whether that particular turn happened to remember to add role, a focus trap, and label ids, with no mechanism forcing consistency across them and no cheap way to discover the drift short of an audit like this one.

And the fix doesn’t compound the way the toolcrib leg’s did. Building Dialog.tsx fixed these two call sites. It does nothing for the next hand-rolled overlay a future session writes, because there’s still no floor — the next dialog is exactly as likely to omit role/focus-trap/labels as these two were, unless whoever writes it happens to reuse this specific file (which requires knowing it exists, which requires an audit like this one having already happened and been remembered). Contrast with the toolcrib leg: the third new overlay this project will ever need doesn’t require writing new infrastructure or retrofitting anything — it requires picking Modal or AlertDialog off a manifest that already lists both, exactly as this experiment’s second feature did. The floor doesn’t just fix what exists; it’s the reason nothing new needs fixing later.

The QA asymmetry, verified directly: does either leg’s floor have tests behind it?

The retrofit above fixed the uncontrolled leg’s dialogs, but a fix with no test guarding it is exactly as fragile as no fix at all the moment someone touches that file again. So this checks something more specific than “toolcrib has more components”: does the infrastructure that enforces correctness actually exist on each side, or is it assumed?

Uncontrolled leg — checked directly, not assumed: no *.test.*/*.spec.* file anywhere in the project. No test runner installed at all — vitest/jest/playwright aren’t even in package.json. This includes after the Dialog.tsx retrofit above: the new focus-trap/Escape/backdrop-control logic that fixed both dialogs’ bugs has zero tests protecting it going forward. Every guarantee this session verified (focus trap holding after 15 tabs, Escape closing, 6/6 labels resolving) is only true as of this conversation’s manual, one-time Puppeteer checks — nothing re-runs them the next time this file is touched, refactored, or copy-pasted into a third dialog.

Toolcrib — checked directly, not assumed, against the actual upstream repo, not just this project’s vendored copy: the premise motivating this section was the assertion that toolcrib “has hundreds of unit tests that run every commit for QA” — rather than take that at face value, escape-llc/toolcrib was cloned fresh and its real test suite actually run. Result:

456 tests across 75 files, npx vitest run → 456/456 passing (up from the ~439 cited in the original v0.4.0 report — new tests were added between versions, not just new components).
A real CI workflow (.github/workflows/ci.yml) runs this suite on every push and PR to main — its own comment explains why: “test failures… were only ever caught at release time… days or releases apart from the change that introduced them,” i.e. this gate was itself added to close a gap the maintainers had already identified.
Confirmed the specific fixes verified earlier in this report have dedicated, named regression tests, not incidental coverage:
- DataTable.test.tsx → describe('regression: sortable headers were mouse-only with no aria-sort, and sortable defaulted to true', ...), with tests asserting the exact tabIndex/aria-sort/click-no-ops behavior this report tested live in the browser.
- Select.test.tsx → it('sets aria-invalid and a resolving aria-describedby once touched and invalid', ...), guarding the exact wiring this report also verified live.
- A separate docsInSync.test.ts regenerates and diffs the manifest/docs against source on every run — meaning the ai-docs/ this entire refactor was guided by is itself under a drift check, not hand-maintained prose that can silently go stale.
There is no equivalent test suite for this consumer project’s two TaskModal/ DeleteConfirmDialog files specifically — the guarantee lives one layer up, in the component source both files consume, not in this project’s own repo. That’s a real and fair distinction (a consumer project should still write its own integration tests), but it’s the opposite failure mode from the uncontrolled leg: here, the primitives are tested at their source, continuously, by someone other than whoever is using them today; there, neither the primitives nor anything built from them are tested by anyone, ever, unless this project starts from zero.

This sharpens the “no floor → repeated mistakes, no path to consistency” claim rather than just restating it. The uncontrolled leg’s retrofit fixed two dialogs by hand and verified the fix by hand, once, in this conversation — a correct floor was built, but nothing makes it stay correct. The toolcrib leg’s equivalent guarantee was never re-derived in this project at all: it already existed, upstream, continuously re-verified by 456 tests gating every commit before this project ever ran toolcrib apply. “Adding structure” in the uncontrolled leg took one session’s worth of extra turns and produced a floor that depends on nobody forgetting it exists; in the toolcrib leg, the floor came with its own standing mechanism for staying a floor, and that mechanism predates and outlives any single consumer’s session.

New observations specific to v0.5.0

Root setup got simpler. A single <ToolcribProvider> now composes ThemeProvider + ToastProvider + ToastContainer in the correct order — v0.4.0 required wiring all three by hand in main.tsx.
More components shipped. 156 patches vs. 116 in v0.4.0 — the toolkit’s surface area grew meaningfully between versions.
Import convention changed. v0.5.0 wires a #toolcrib subpath import via package.json‘s imports field (import { Card } from '#toolcrib'), replacing v0.4.0’s relative ./toolcrib import. This is a one-time mechanical change with no functional friction, but it’s a breaking convention change between versions worth flagging for anyone maintaining an existing v0.4.0 toolcrib integration.
Manifest is now split per-category (ai-docs/manifest/<category>.json) alongside the full component-manifest.json, which materially reduced how much JSON needed to be read per component during the refactor — a documentation/DX improvement, not a functional one.

Bottom line

Toolcrib v0.5.0 reproduced the original run’s headline coverage result — ~91% raw-markup elimination and 100% styling-attribute elimination — while resolving every non-environmental friction point the v0.4.0 run surfaced: the build-breaking TypeScript incompatibility is gone, the React-version false-flag is gone, the CardSimple/manifest mismatch is gone, and the one real coverage gap (no badge component) is closed. The only unresolved item is the GitHub API rate limit on unauthenticated init calls without a pinned version — untouched by this comparison since both runs used the same pin-the-version workaround, and not something a toolcrib release would be expected to fix regardless.

More significantly, all four concrete correctness bugs from the original ACTION_PLAN.md are fixed in v0.5.0, each re-verified by actually triggering the behavior in a live browser rather than re-reading the diff: Select now exposes an id and full label association (6/6 fields, up from 3/6), form validation errors are properly wired to aria-invalid/aria-describedby, DataTable‘s sortable headers are fully keyboard-operable with correct aria-sort, and the sortable-default doc/implementation mismatch is resolved. This is exactly the outcome the original report’s “repair loop” discussion predicted was possible in principle — a bug found once in the vendored source becoming fixed for every consumer via a version bump — and this rerun is the first direct evidence that it actually happened, in the very next version.

The extension test is the most important finding in this rerun, not a side note. Both legs were handed an identical new feature in plain consumer language, with no toolcrib vocabulary at all — a delete confirmation summarizing what’s being removed. This was deliberately chosen to force each leg to repeat an earlier markup concept (a second dialog, a second use of the priority/status badges), because that’s the condition that actually separates “the model can write correct-looking code once” from “the model has any structural reason to write the same correct thing twice”:

The toolcrib leg picked a different, more specific primitive — AlertDialog over the already-used Modal — correctly, from the request’s wording alone, with no toolcrib terms used and no components named. “Let them back out instead of deleting” mapped onto AlertDialog‘s own documented purpose (“blocking confirmation… for destructive/irreversible actions… use Modal for general-purpose content”) without that distinction ever being stated. Result: 5 new raw elements (all legitimate text/ wrappers), 0 new classNames, and every accessibility property AlertDialog‘s doc comment claims — role="alertdialog", focus trap, non-dismissible backdrop, Escape-to-close — verified true live.
The uncontrolled leg, writing the same feature minutes later in the same session, with the first dialog sitting in context the entire time, still reproduced its first dialog’s exact bugs: no role="dialog", focus escaping the trap, dead Escape. Nothing about having just written a working- looking dialog constrained what the second one looked like — 11 new raw elements, 11 new classNames, and the identical defect class independently reintroduced.
Retrofitting a shared floor after the fact was possible but not free: fixing both dialogs required writing 86 new lines of hand-rolled infrastructure (manual focus-trap logic, Escape handling, backdrop control, label-id wiring) and reopening both already-shipped files to adopt it — extra turns spent purely on catching up to a floor the toolcrib leg started with at zero cost. And that new Dialog.tsx protects only these two call sites going forward; the next hand-rolled overlay this project ever needs is exactly as likely to omit the same things, since nothing forces the next session to know this file exists or reuse it.
Neither retrofit is backed by a test that would catch a future regression — the uncontrolled leg has no test runner installed at all, while toolcrib’s guarantees are enforced by 456 passing upstream tests (verified directly against the real escape-llc/toolcrib repo, not assumed) running on every commit, including named regression tests for the exact Select/DataTable bugs this report reproduced live.

Put together, this is direct, measured evidence for the stronger claim: without a functional floor, a vibe-coding session doesn’t just risk inconsistency between features — it structurally has no mechanism to avoid it, even within a single session, even with the earlier code still in context, and closing the gap after the fact costs real, measurable extra work that a component-based floor never required in the first place.

2026-08-20

Refactor Application Experiment: Uncontrolled vs. Toolcrib

This report, the code it describes, and the patches accompanying it were generated by an AI model (Claude). Every claim here was checked against a running app, a real repo checkout, or a live browser session rather than left as an assertion — the methodology and exact commands are in the Appendix specifically so any of it can be independently re-run. But “extensively verified” isn’t the same as “immune to the failure modes this report itself describes,” and that applies with particular force to the parts proposing fixes for someone else’s repository (ACTION_PLAN.md, the verbatimModuleSyntax patch): treat them as a diagnosed starting point for a human or another AI session to review, not as pre-approved changes to merge unread.

Factors: uncontrolled (free-form AI-generated React/Tailwind) vs. toolcrib (escape-llc/toolcrib CLI refactor) Measurement: how much of the uncontrolled markup gets refactored into toolcrib’s components — and, beyond raw coverage, whether the two implementations actually behave differently where it matters.

This report covers three kinds of comparison:

Markup coverage (Method → Bottom line): how much hand-rolled JSX got replaced with toolcrib components, and what didn’t convert.
Head-to-head behavioral comparisons, each tested live in a browser rather than inferred from source or the component manifest:
- Modal — dialog semantics, focus trap, Escape/backdrop dismiss, label association (“Correctness & quality: beyond markup coverage”)
- Table — sorting, pagination, virtualization vs. native semantics, keyboard access (“DataTable: head-to-head on features and correctness”)
- Form — validation behavior, error display, and what’s (and isn’t) exposed to assistive tech in each (“Form validation: head-to-head”)
Discussion: the repair loop — four real gaps surfaced in toolcrib itself (listed in full, with fixes, in the companion ACTION_PLAN.md). Rather than treat that as a strike against the tool, this section argues the gaps matter less than they would in the uncontrolled leg, because toolcrib has a mechanism (doctor/merge) for a fix found once to propagate back to every consumer, where hand-rolled code has no equivalent path at all — and examines why the gaps cluster specifically in toolcrib’s own from-scratch code rather than the parts wrapping an already-hardened library, plus what that implies about AI-generated tests as a check on AI-generated code.

Method

Scaffolded a Vite + React 19 + TypeScript + Tailwind project (uncontrolled).
Built a task/project dashboard app: header, 4 stat cards, a toolbar (search + 2 filter selects + new-task button), a sortable-looking task table with edit/delete actions, and a full modal form (title, description, assignee, due date, priority, status). All markup hand-rolled JSX with Tailwind classNames — no shared component library.
Counted the baseline: 87 raw HTML elements, 65 className attributes across App.tsx + 5 component files.
Forked the project to toolcrib_leg. Ran:npx toolcrib init --situation refactor --version 0.4.0 Reviewed all 116 staged patches in ./toolcrib-patches/, then npx toolcrib apply. Installed the added deps (radix-ui, zod). Confirmed the project still built with toolcrib vendored but not yet used.
Read toolcrib/ai-docs/CORE.md, REFACTOR_APP.md, and component-manifest.json (the files toolcrib init vendors specifically to brief an AI agent on the toolkit).
Refactored every file against the toolcrib API:
- main.tsx — added required <ThemeProvider> + <ToastProvider> + <ToastContainer> root wrapper
- App.tsx — raw <header>/<main> → <AppShell> + <VStack>
- StatCards.tsx — raw grid of <div>s → <Grid> + <CardSimple>
- Toolbar.tsx — raw <input>/<select>/<button> → <Toolbar> + <Input> + <Select> + <Button>
- TaskTable.tsx — raw <table> → <DataTable> with column definitions
- TaskModal.tsx — raw <form> + manual useState draft object → <Modal> + <Form schema={zodSchema}> + <FormField> wrapping <Input>/<Select>/<Textarea>
Re-ran the same element/attribute counts on the refactored code.
Verified functional equivalence in a real browser (Puppeteer): status filter, edit-modal prefill, and delete all confirmed working identically to the original.

Results

Metric	Uncontrolled (before)	Toolcrib (after)	Change
Raw HTML elements	87	8	−90.8%
`className` attributes	65	2	−97%
Toolcrib component tags used	0	41	—

What’s left raw, and why (all legitimate, not refactor failures):

<h1>/<p> in the app header — plain text content, nothing to replace.
2 <span>s in Badge.tsx — toolcrib has no badge/pill/tag component in its manifest (confirmed by searching it directly). A genuine coverage gap, not a missed conversion.
4 small <div>s used as text-wrapping containers inside table/stat-card cell renderers — below the granularity toolcrib operates at.

Functional verification

Screenshots alone don’t prove equivalence, so the refactored app was driven in a headless browser:

Status filter — selecting “Done” correctly narrowed the table from 6 rows to the 2 actual “Done” tasks; stat cards stayed accurate.
Edit prefill — clicking “Edit” on “Write Q3 retro notes” opened the modal with every field correctly populated (title, description, assignee, due date, priority, status) via <Form initialValues> — replacing what was previously a hand-written useEffect/useState sync in the uncontrolled version.
No console or page errors during any interaction.

Friction points encountered

api.github.com rate limit. Unauthenticated CLI calls (toolcrib versions, init without --version) hit a 403 on this sandbox’s shared egress IP. Worked around by pinning --version 0.4.0 directly — the download itself uses plain github.com release-asset URLs, not the rate-limited API.
verbatimModuleSyntax incompatibility. Vite’s default React+TS template enables this strict TypeScript option; toolcrib’s vendored source uses plain (non-import type) type imports throughout, which fails the build under it. toolcrib doctor checks moduleResolution compatibility but not this specific option — had to manually disable it in tsconfig.app.json. This would block any real adopter’s first build.
CardSimple requires children — its title/subtitle props are header-only, not a full replacement for a card’s main content. A one-line fix once checked against the actual component source rather than assumed from the manifest’s short description.

Correctness & quality: beyond markup coverage

Element/attribute counts show how much markup moved, but say nothing about whether the result actually behaves better. Toolcrib’s stated premise is that using its components buys you correctness guarantees (accessible dialog semantics, focus management, keyboard support) “for free” — so this was tested directly against the running apps, not assumed.

Method: scripted the same interaction sequence (open the “New task” modal, inspect DOM, tab through the form 30 times, press Escape, click the backdrop) against both apps in a headless browser, plus captured console output throughout.

Check	Uncontrolled	Toolcrib
Modal has `role="dialog"`	❌ plain `<div>`	✅
Modal has a wired accessible name (`aria-labelledby`)	❌ none	✅ (Radix auto-wires to the header text)
Label ↔ input association (`for`/`id` resolves)	❌ 0 / 6	⚠️ 3 / 6
Focus trap (Tab can’t reach the page behind the modal)	❌ escapes after a few tabs	✅ trapped
`Escape` closes the modal	❌ no-op	✅
Clicking the backdrop closes the modal	❌ no-op	✅
Console errors/warnings during use	none	none

Uncontrolled app — confirmed against source, not just observed behavior: the backdrop <div> has no onClick at all, and there’s no keydown/Escape listener anywhere in the file — only the explicit “✕” and “Cancel” buttons close it. None of the six <label> elements have a for attribute, and none of the inputs have a matching id, so a screen reader cannot associate any label with its control. This is exactly the kind of defect a human reviewer skimming the rendered UI (rather than testing it) would miss — it looks correct and reads correct, it just isn’t wired.

Toolcrib app — a real, partial win, not a clean sweep: <Modal> (built on Radix’s Dialog primitive) gets dialog semantics, a full focus trap, Escape-to-close, and backdrop-dismiss entirely for free — none of that was written by hand, all of it verified working. But label association only holds for 3 of 6 fields: <Input>/<Textarea> correctly forward id={name} to match <FormField>‘s <Label htmlFor={name}>, while <Select> does not — its SelectProps interface doesn’t even expose an id prop, and its underlying SelectPrimitive.Trigger renders with no id and no aria-label, confirmed directly in the DOM. So “Assignee,” “Priority,” and “Status” (and, by the same code path, both toolbar filter selects) have no accessible name at all for a screen reader — worse than an unassociated label, since there’s no fallback either. This is a genuine bug in toolcrib’s own <Select> component, not a refactor mistake on this project’s part, and would be worth its own upstream patch if useful (same shape as the verbatimModuleSyntax fix: a small, mechanical, verifiable change — add id={fieldName} to the Trigger).

Bottom line on this axis: the premise holds directionally — adopting toolcrib’s overlay and form primitives did eliminate several real, verified interaction/accessibility bugs that existed in the hand-written version, without anyone writing focus-trap or keyboard-dismiss logic by hand. But it’s not an unconditional guarantee: one of toolcrib’s own components has a real accessibility gap that partially undermines the very pattern (FormField + Label htmlFor) the toolkit is designed around.

DataTable: head-to-head on features and correctness

Same treatment as the modal — tested actual behavior in-browser, checked source for root causes, didn’t just compare markup.

Uncontrolled table: a real <table>/<thead>/<tbody>/<th> with correct native semantics, but genuinely static — no sorting, no pagination, <th> elements are plain headers with no interactivity at all. Confirmed directly in source: zero onClick/sort state anywhere in the file.

Toolcrib <DataTable>: implements virtualization without abandoning table semantics — worth calling out as a good design choice, since many virtualized-table implementations fall back to <div> grids and lose native table accessibility entirely. This one still renders real <table>/<thead>/<tbody>/<th>/<tr>/<td>, with virtualization done via spacer <tr>s above/below the visible window — confirmed in source.

Check	Uncontrolled	Toolcrib
Sorting	❌ none	✅ works, verified correct (asc/desc toggle, correct reordering, `▲`/`▼` indicator)
Pagination	❌ none (renders all rows)	✅ present, correctly disabled/hidden logic when all rows fit on one page
Native `<table>` semantics preserved under virtualization	n/a (not virtualized)	✅ yes — spacer-row approach, not a div-grid
Sortable headers reachable by keyboard	n/a	❌ `tabIndex="-1"` on every `<th>`, no `role`, no keydown handler — mouse-only
`aria-sort` reflects current sort state	n/a	❌ not set on any header, in either sort state
`sortable` default matches its own documented contract	n/a	❌ mismatch — see below

Confirmed bug: sortable‘s default contradicts its own JSDoc. The Column.sortable prop is documented as @default false (“if true, clicking toggles sorting”). The actual click handler checks col.sortable !== false — so a column that simply omits sortable (the documented default, not-opted-in state) evaluates undefined !== false → true and becomes sortable anyway. In this project’s own TaskTable.tsx, the “Actions” column never sets sortable and isn’t sortable data — but the bug makes its header render with cursor: pointer (a false affordance) and become clickable. Confirmed no crash results (the sort comparator finds undefined on both sides and no-ops), but it’s a real, user-facing inconsistency between what the docs promise and what the code does — same “verify, don’t just read the manifest” lesson as the CardSimple/children prop earlier in this report, just with a more concrete downstream effect.

Not observed but worth flagging as a latent risk, not a confirmed bug: virtualization positions off-screen rows using a fixed itemHeight (default 44px) for the spacer-row math, while this project’s “Task” column renders two lines of custom content (title + description) at ~39px measured height — close enough that it didn’t clip with 6 rows, but a taller custom cell renderer at real pagination scale (many rows, itemHeight not explicitly overridden) could plausibly drift from the actual rendered row height. Didn’t reproduce this at the scale tested; flagging as a thing to watch, not a demonstrated defect.

Bottom line on the table: toolcrib’s DataTable is a genuine upgrade in capability — free sorting and pagination that didn’t exist before, implemented without sacrificing table semantics under virtualization, which is a real engineering credit. But it inherits the same pattern as the Select finding above: real functionality, real gap. The sortable headers are entirely unreachable by keyboard, and the sortable prop’s own documented default doesn’t match its implementation — two more concrete, verified findings, not speculation, worth their own upstream fixes if useful.

Form validation: head-to-head

Same standard again — tested actual behavior, checked source, didn’t just compare the schema declaration to the JSX.

Uncontrolled form: only title has any validation at all — a native HTML5 required attribute. Confirmed in source: no other field (description, assignee, dueDate, priority, status) has any constraint whatsoever. Tested the one constraint that exists: submitting with an empty title is correctly blocked by the browser’s native Constraint Validation API — input.validity.valid is false, :invalid matches, and the browser shows its own native tooltip (“Please fill out this field.”). This works, but it’s a single hand-placed attribute; nothing else in the form is validated at all.

Toolcrib form: the Zod schema’s title: z.string().min(1, 'Title is required') was tested the same way — submit immediately with an empty title, no prior interaction with any field. Confirmed:

Submission is blocked (modal stays open)
The custom message “Title is required” renders inline, styled in red, directly below the field — on the very first submit attempt, without requiring a prior blur first (the Form‘s submit handler marks every field touched before validating, so a never-touched field still surfaces its error immediately rather than waiting for the user to tab through it)
Fixing the title and resubmitting closes the modal and adds the new row correctly — confirmed end-to-end, not just that the error clears

A gap common to both, worth being even-handed about: neither implementation wires aria-invalid or aria-describedby to connect the error state to the input programmatically. Checked both directly in the DOM post-submit-attempt: uncontrolled’s native-invalid input has aria-invalid: null (browsers don’t add this automatically just because :invalid matches); toolcrib’s <Input> in error state also has aria-invalid: null and aria-describedby: null despite rendering a visible, correctly worded error message right next to it. So a screen reader user gets something from the native path (browser-level exposure of the constraint-validation tooltip, inconsistent across browsers) and gets silence from toolcrib’s path (a sighted-only red-text error) — neither is a clean win here, and toolcrib’s FormField/Input pairing would need aria-invalid={!!error} and aria-describedby pointing at the error <span>‘s id to close this gap. Small, mechanical, same shape as every other fix in this report.

Where the schema approach is a genuine, structural improvement, independent of the gap above: a single Zod schema is one source of truth that every field binds to automatically, versus native HTML5 validation requiring a hand-placed attribute per field with a fairly limited vocabulary (required, pattern, min/max, no cross-field rules, no custom messaging without extra JS). Worth being honest that this project’s own schema didn’t fully exploit that ceiling — assignee is typed z.string() rather than z.enum(assignees), and dueDate has no non-empty check, so both apps actually accept a task with no due date. That’s a modeling choice made while writing this experiment’s schema, not a toolcrib limitation — the toolkit would enforce any of that just as easily if the schema said so.

Bottom line on forms: toolcrib’s validation is more capable and more consistently applied across the whole form than the uncontrolled version’s single required attribute, and it demonstrably works correctly end-to-end. But “adopting a schema-driven form” didn’t automatically deliver full accessibility for free — the error-to-input wiring is a real, separate gap, present in toolcrib’s own component, not something this project’s usage pattern could have fixed by writing a better schema.

Bottom line

For a UI with strong overlap with toolcrib’s supplied component categories (overlays, tables, forms, layout primitives), the refactor achieved ~91% raw-markup elimination and ~97% styling-attribute elimination in a single incremental pass, guided entirely by the vendored ai-docs/ + component-manifest.json. The only markup that couldn’t convert was content genuinely outside the toolkit’s scope (plain text) or a missing primitive (badges). Two real, non-trivial friction points surfaced along the way (API rate limiting during init, and a TypeScript strictness incompatibility) that a first-time adopter would hit before ever reaching the refactor step itself.

Markup coverage was only half the question, though. Testing actual behavior — not just counting tags — showed the refactor also fixed several genuine, verified bugs (no dialog semantics, no focus trap, dead Escape/backdrop handlers, zero label association) that existed in the hand-rolled version, and added real capability that never existed before (sorting, pagination) without sacrificing native table semantics. That’s toolcrib’s stated value proposition actually paying off, empirically, not just asserted.

It’s not unconditional, though, and a rigorous eval should say so rather than round up. Four concrete, verified gaps surfaced in toolcrib itself: <Select>‘s trigger has no id/aria-label at all (breaking label association for exactly the fields that use it), DataTable‘s sortable headers are mouse-only with no keyboard path and no aria-sort, Column.sortable‘s actual behavior contradicts its own documented default, and Input‘s error state isn’t wired to aria-invalid/aria-describedby despite FormField rendering a clear, correctly worded error message right next to it. All four are small, mechanical, and independently verifiable — the same shape as the verbatimModuleSyntax fix already handed off — but they’re real, and “adopting toolcrib” measurably improved correctness here without making it perfect.

Discussion: the repair loop

Why these gaps matter less than they would in the uncontrolled leg

Every gap found in this report — the missing Select id, the unwired aria-invalid, the keyboard-inaccessible sort, the sortable default bug — has a fix available two ways right now: verbatim-module-syntax-fix-HEAD-full.patch and ACTION_PLAN.md, both handed off toward the actual escape-llc/toolcrib repo. That handoff is only possible, and only useful at scale, because of a mechanism the uncontrolled leg has no equivalent of at all.

Uncontrolled: the four modal/table/form bugs found here live nowhere but this one project’s TaskModal.tsx/TaskTable.tsx/Toolbar.tsx. There is no upstream for them. Fixing them means editing these exact files, in this exact project, by hand. If another project independently hand-rolled a similar modal — which, structurally, most AI-assisted “vibe coded” apps do, since that’s the whole premise toolcrib‘s own README states — it has the same class of bugs sitting in its own copy, undiscovered, unconnected to this report, and would need the same debugging effort spent again from zero. Nothing about finding and fixing it here helps anyone else.

Toolcrib: the CLI’s init → apply → doctor → merge lifecycle exists specifically to prevent that duplication. merge compares three things per vendored file — what a project originally got, what’s actually on disk now, and what a new release ships — and sorts each into “safe update” (never hand-touched locally; the new version is staged as a patch), “kept as-is” (locally edited, and upstream didn’t change that file — left alone), or “conflict” (both changed; a .upstream-diff is written explaining what upstream did, so a person or their AI reconciles it deliberately rather than one side silently winning).

Concretely, in this experiment: toolcrib_leg never hand-edited Select.tsx, DataTable.tsx, or FormComponents.tsx — every use of those components went through the public API, not a local patch to the vendored source. That means if escape-llc/toolcrib accepts the fixes in ACTION_PLAN.md and ships them in a new release, running toolcrib merge in this exact project would detect all four files as clean “safe updates,” stage them as reviewable patches, and toolcrib apply would close all four gaps — without anyone here re-diagnosing them, re-writing the fixes, or even remembering this report existed. The fix travels from wherever it was first found (here) to every consumer that adopted the toolkit, mechanically, with the tooling itself protecting any local customization along the way. The uncontrolled leg’s equivalent bugs have no such path: they only leave this project if a person manually notices this report, manually applies the same reasoning to different files, and manually repeats the process per project, per bug, indefinitely.

That asymmetry — not the markup-conversion percentage this report opened with — is arguably toolcrib‘s strongest claim on “quality of operations”: not that the vendored code starts out bug-free (it demonstrably doesn’t), but that a bug found once has a mechanical, conflict-aware path back to every project that adopted it, which hand-rolled code structurally cannot offer no matter how carefully it’s written.

Where the gaps cluster, and why

The four gaps found in this report aren’t evenly distributed across toolcrib’s components, and the pattern is worth naming explicitly rather than leaving as a footnote.

Nobody prompting “build a modal” — whichever side of this experiment is doing the prompting — carries aria-modal, focus traps, or aria-invalid as part of what a modal is. That knowledge isn’t common even among engineers who’d never touch an AI tool; it’s specialist accessibility knowledge, not general withheld competence. So there’s no reason to expect that requirement to surface in generated output by default, on either side of this experiment, unless something other than the prompt supplies it.

That “something else” is exactly what splits toolcrib’s own results cleanly in two:

<Modal> is a thin skin over Radix’s Dialog primitive. Every check passed: role, focus trap, Escape, backdrop-dismiss — all correct, all unprompted. That’s not because toolcrib’s construction carried the accessibility signal; it’s because Radix’s own maintainers do, as their entire reason for existing, and toolcrib inherited their already-solved output wholesale.
<Select> also wraps a Radix primitive — but the specific thing that broke (id never threading from toolcrib’s own fieldName down to SelectPrimitive.Trigger) isn’t Radix’s problem to solve; Radix has no idea toolcrib’s FormField pattern exists. That connective code is original, toolcrib-authored glue, and it’s exactly where the gap is.
<Input>‘s missing aria-invalid/aria-describedby and <DataTable>‘s three separate issues (keyboard-inaccessible sort, missing aria-sort, the sortable default bug) aren’t wrapping anything. DataTable in particular has no Radix underneath it at all — and it’s also where defects are most concentrated: three distinct, independent bugs in one from-scratch component.

Every gap found in this report sits in code toolcrib’s own authors wrote from scratch. Every check that passed sits in code borrowed from a library whose maintainers had the missing signal already built in. Given toolcrib’s own README states plainly that “the rest was generated by AI, using various IDEs and AI models,” this isn’t a coincidence of small sample size — it’s the same mechanism the rest of this report describes, visible one layer down. Toolcrib doesn’t transcend the signal problem so much as launder it: it succeeds precisely where it borrowed someone else’s already-solved instance of correctness, and reproduces the identical gap everywhere it had to solve the problem fresh — because “build a modal” and “build a sortable table” carry exactly the same missing signal regardless of who, or what, receives the instruction.

This is a falsifiable claim, not just a narrative fit to the four bugs already found: it predicts the next defect in this codebase is more likely sitting in DataTable, Toolbar, or another from-scratch component than in anything built directly on a Radix primitive. Worth checking against whatever escape-llc/toolcrib fixes or doesn’t fix next.

Why the floor matters more than the bug count, and why AI-written tests don’t fully fix that

Toolcrib’s own stated philosophy isn’t “these components are bug-free” — it’s closer to giving vibe-coding practitioners a better floor to build from, on the premise that regression-testing an entire UI after every single turn is an unreasonable ask of anyone, and that even a project with unit and e2e tests is only getting one level of assurance, especially when those tests are themselves AI-generated. Both halves of that are worth taking seriously on their own, not just as a defense of the four gaps found here.

The floor, not the bug count. The value of <Modal> was never that this instance happens to be correct — it’s that “does this modal have a focus trap” stops being a question re-asked, and re-answered from scratch, on every single turn that requests a modal, in every project, forever. In the uncontrolled world, each occurrence independently re-samples the same latent distribution — whether this modal gets a focus trap is drawn fresh every time, with no memory of the last hundred times it wasn’t. Once a component exists and a bug in it is fixed centrally — via exactly the merge path described in the Epilog — that fix is permanently subtracted from the risk surface for every future turn that calls <Modal> instead of regenerating one. The floor rises not because the generation got smarter, but because “regenerate this from scratch” is being asked for less and less often.

AI-generated tests confirm the generated code, not some objective truth. If the same model (or the same class of model, drawing on the same latent space) writes both the implementation and the test suite for it, the test isn’t an independent check — it’s a second draw from the same distribution the implementation came from. A missing signal that produced <Select>‘s absent id or <Input>‘s absent aria-invalid is exactly the kind of gap a self-authored test is also unlikely to catch, for the same reason: nothing in “build a modal” surfaces aria-modal as a requirement to implement or to test for. A green suite in that situation confirms the code matches its author’s mental model: it says nothing about whether that mental model was complete.

This is, incidentally, a fair description of why this report’s own checks held up: none of them came from a test some model wrote for itself. They came from an external, pre-existing standard the generation loop had no hand in — the WAI-ARIA authoring practices for dialogs and sortable tables specify role, aria-sort, focus containment, keyboard reachability as known, external requirements, checked against the live DOM directly, not against self-referential test expectations. It’s the same underlying reason toolcrib doctor‘s drift detection and merge‘s conflict flagging are worth more than a green test run alone: the check’s value comes precisely from sitting outside the generation it’s checking, the same way the fixes in ACTION_PLAN.md came from driving a real browser against a real standard, not from asking the code whether it thought it was correct.

Appendix: tooling used for verification

Every claim in this report was checked against a running app or a real repo checkout, not inferred from source reading alone. Precise tooling, for reproducibility:

Browser automation — Puppeteer, not Playwright. Headless Chrome (a locally available “Chrome for Testing” build) driven via puppeteer (already present in this environment through a global @mermaid-js/mermaid-cli install), launched with --no-sandbox --disable-setuid-sandbox. Both apps were served with vite preview (production build, not dev server) so behavior matched what a real deployment would do. Used for:

Full-page screenshots (page.screenshot({ fullPage: true })) for the before/after visual comparisons.
Simulated interaction: clicking buttons/headers via page.evaluate DOM queries (matched by visible text rather than brittle selectors, since neither app’s markup was written with test hooks in mind), page.keyboard.press('Tab'/'Enter'/'Escape') for keyboard-path testing, page.mouse.click(x, y) for backdrop-dismiss testing.
Live DOM/ARIA inspection: getAttribute('role'|'aria-*'), tabIndex, getComputedStyle (cursor, border color, computed height), document.activeElement containment checks for the focus-trap tests, input.validity/matches(':invalid') for native constraint validation state.
Console/error capture: page.on('console') and page.on('pageerror') listeners kept open through every interaction sequence, not just page load.

Static/source verification — read directly, not summarized from memory. Component source (Select.tsx, DataTable.tsx, FormComponents.tsx, Modal.tsx, etc.), the CLI’s own source (toolcrib/src/lib/*.js) to confirm exactly what init/apply/doctor do and don’t touch, and scripts/build-release.js‘s comments to confirm what’s actually excluded from the shipped release.

Codemod — ts-morph, not regex. The verbatimModuleSyntax fix walked the real TypeScript AST (Project.getSourceFiles(), per-specifier symbol resolution via getSymbol()/getAliasedSymbol()), checking each import specifier’s resolved declaration kind (InterfaceDeclaration/TypeAliasDeclaration only) before marking it type-only — verified directly that this approach can’t misclassify a class/value binding (see the ClassDeclaration check in that section) the way a name-based heuristic could.

Repo/build verification — real toolchain runs, not just tsc --noEmit in isolation. git clone/git diff/git apply --check against fresh clones (not just the working copy that generated a patch) for every patch handed off; npm install (including the repo’s own isolated scripts/ dependency scope, once that mattered); npx tsc --noEmit both with the shipped config and with verbatimModuleSyntax forced on via a scratch config copy; npm run build (the repo’s actual tsc && vite build script, not a substitute); npx vitest run for the full existing test suite (439 tests) rather than cherry-picking the ones expected to pass.

External research — fetched live, not recalled. escape-llc/toolcrib‘s README, CORE.md/REFACTOR_APP.md/component-manifest.json, CHANGES.md, and release list were all pulled via curl/git clone during this session; likewise microsoft/typescript-go‘s README and its own documented feature-status table, and npm view typescript versions/dist-tags to confirm the TS 7 API-surface change directly rather than trust prior training data on a fast-moving toolchain change.

2026-08-19

The Toolcrib Taste-Test

Disclaimer: This report was produced by Claude (an AI), working under a human’s direction. Every claim below was tested against the real toolcrib source, not assumed — but the testing, code, and conclusions are still AI output. Review before citing a number externally.

Date: 2026-08-14
Subject: Does toolcrib (an npm-installed, AI-oriented React component library) improve on unguided “vibe coding,” and in what sense? Started as a comparison of visual styling control; the evidence extended it to interaction correctness and architecture as well — see Findings 9–10.
Scope note: This reflects toolcrib with the five patches from this evaluation applied — a WCAG contrast fix; manifest type-definition resolution (across two files); DataTable.rowSubtheme; and the matching Anti-Patterns doc entry — not the state it shipped in before this test.

Verdict

The evidence in this report adds up to something stronger than “toolcrib is nicer to work with”: uncontrolled, unguided output has multiple genuine risk surfaces, not one, and none of them are a taste preference — they fail silently. Four separate results in this evaluation demonstrate that, not just assert it, spanning both styling and interaction correctness:

A restyle attempt collapsed three distinct spacing roles (table cells, inputs, buttons) into one identical value — the script ran without error, the page rendered without error, and the task was reportable as complete. Nothing caught it (Finding 3/4).
One of three independently-built, ordinary-looking unguided trials shipped text at 2.54:1 contrast — below WCAG AA, genuinely hard to read for a meaningful fraction of real users, and nothing in that build process flagged it (Finding 7).
Given the ability to freely pick a UI accent color, 4 out of 4 plausible, ordinary-looking choices (a light yellow, a mint green, a pale pink, a light gray) produced a button with white text at 1.2–1.4:1 contrast — not “hard to read,” effectively invisible. This wasn’t an adversarial edge case; it’s what happens by default the moment color choice is unconstrained (Finding 8).
Not a styling defect at all: an unguided build of one of the most common interactive elements in ordinary app work — a delete-confirmation dialog — had zero of the four standard accessible-dialog behaviors: no focus trap, no Escape-to-close, no ARIA dialog semantics, no focus restoration to the triggering element. Confirmed by direct inspection of the generated code, not assumed (Finding 9). This one is arguably harder to catch than the others, since a sighted developer testing with a mouse never trips any of the four.

None of these four were caught by a compiler, a linter, a test, or the build process itself. Each surfaced only because this evaluation went looking for it from outside the ordinary request-response loop. That’s the actual danger, and it isn’t specific to CSS: unguided output’s failures don’t announce themselves, in styling or in behavior. They ship, and stay shipped, until a human happens to notice — which for accessibility specifically (contrast or keyboard/screen-reader access) can mean a real user locked out of a control entirely, a support ticket, or in jurisdictions with accessibility law (ADA in the US, the EU Accessibility Act, and similar), a genuine compliance exposure. (Not legal advice — a general observation that this category of defect has real-world stakes beyond aesthetics, worth flagging to whoever owns that risk at an organization, not just to whoever owns the CSS.)

This is not an argument that toolcrib specifically is required. It’s an argument that something needs to make color/spacing/contrast/interaction decisions structural rather than hoped-for — a design-token system with enforced contrast, automated accessibility testing in CI (axe-core, Lighthouse CI), a linting rule that rejects known-bad patterns, headless accessible primitives (Radix, React Aria, or similar) as the default rather than hand-rolled markup, a mandatory human accessibility review — toolcrib is simply the one mechanism this evaluation tested, and it happens to close all four of the failure modes above by construction rather than by someone remembering to check. The comparison that matters isn’t “toolcrib vs. Tailwind.” It’s “some enforced control vs. none” — and this report’s evidence is about what “none” actually produces when nobody’s looking, in more than one subsystem.

How toolcrib specifically controls the styling risk surface, evidenced across this report: multiple, coordinated levels, not one blunt rule.

Compile-time: style/className aren’t valid props on any toolcrib component — a type error, not a lint warning that can be silenced or ignored (Finding 1).
Per-instance: components with a registered theme slice expose a typed, sparse overrides prop for one-off adjustments — enumerable values only, never a raw style object.
App-wide: ThemeProvider‘s parameters set color/radius/density/mode once, read by every component from one shared source, which is why a global restyle can’t produce the cross-component cascading collision an ad hoc rewrite did (Finding 3).
Semantic: a bounded four-category subtheme vocabulary (error/success/warning/info) applies uniformly across component types and, since this evaluation’s patch, individual table rows too (Finding 5).
Underneath all of the above: contrast enforcement runs automatically regardless of which level actually produced a given color — the theme parameters, a subtheme, or an end user’s own pick (Finding 7).
Beyond the running app: the same guarantees extend past build-time to a designer tuning the shipped default and to an end user personalizing at runtime, through one shared component rather than two separately-engineered systems (Finding 8).
Around all of it: a machine-readable manifest describes exactly what’s controllable at each level, so the vocabulary is discoverable by reading documentation rather than requiring a dive into source (Finding 10, and the patches in this report that closed real gaps in it).

No single level here is airtight on its own — Finding 4 found where semantic-only control (four subthemes) runs out for a non-semantic need, Finding 2 found a harmony-mode gap in the theme level’s own reach for a neutral secondary color. But layered together, they’re the reason ad hoc styling doesn’t have anywhere convenient to hide in this system by default: every path back to arbitrary style either fails to compile, or requires actively bypassing an enumerated, typed, contrast-guaranteed alternative that was sitting right there instead.

And the fourth bullet above — the dialog with zero accessible behaviors — is controlled the same way, in a separate subsystem. Building Modal/Popup/SlideOut/AlertDialog on Radix’s tested primitives instead of hand-rolled markup, and using a typed event bus (aiBus) for cross-tree state instead of prop-drilling, is architecturally the same move as everything in the styling list above: replace a decision an unguided build has to re-earn correctly every single time with one made once, inherited automatically. 60% of toolcrib’s component set inherits it this way (Finding 9); an unguided build re-risks the same absence — focus trap, Escape handling, ARIA semantics, focus restoration — per component, every time, with nothing in an ordinary sighted-developer testing pass likely to catch it.

Toolcrib is a conditional improvement, not a universal one. It reliably eliminates the failure mode it targets — ad hoc, inconsistent, hard-to-globally-restyle utility-class soup — by construction: no toolcrib component’s props accept style or className at all, a compile-time guarantee rather than a convention someone can forget. It also enforces a WCAG contrast floor on every color it generates dynamically, something unguided output has no equivalent of.

That guarantee is narrower than “no ad hoc styling anywhere,” though. It only restricts toolcrib’s own component props — plain JSX inside a slot, a child, or a render callback is exactly as stylable as ever. The real constraint bites only where a component’s own DOM is generated entirely internally and never exposed as a prop or slot at all.

A frame that applies throughout: nothing here is automatically enforced by the AI maintaining the toolkit. Every fix reflected in this report — the contrast patch, the new row-flagging prop, the doc update — required a human to decide it was worth institutionalizing and direct it. That’s not a limitation of toolcrib specifically; it’s the shape of how any of this stays correct. What toolcrib changes is the leverage of each such decision: once a shared, constrained vocabulary exists (paddingMode, cornerRadiusMode, subtheme, rowSubtheme), a short human imperative (“make this row a warning,” “make corners sharp app-wide”) lands on one precise, testable primitive shared by the whole codebase — instead of requiring a fresh, ad hoc translation into raw styling every time, the exact failure mode demonstrated below.

The cost this doesn’t show up in Finding 6’s token count: every unguided defect surfaced in this report — a3’s failing contrast (Finding 7), the restyle cascade bug (Finding 3), all four unreadable end-user color picks (Finding 8) — was not caught by the unguided build process itself. Nothing flagged any of them; they only surfaced because this evaluation went looking from outside the normal request-response loop. In an ordinary session, each ships silently until a human happens to notice, and only then does correction start — an indeterminate number of turns, unknowable in advance, with no guarantee a fix doesn’t introduce a new problem. Every toolcrib defect in this report required the same kind of deliberate effort to find, but once found, closed structurally and permanently — a compile-time guarantee, a shared resolver, a manifest generator — inherited by every future project built against the patched library at zero further cost. The asymmetry isn’t “toolcrib has fewer bugs.” It’s that unguided defects are per-project and non-transferable — a fresh session tomorrow re-risks the identical mistake with none of today’s corrections carried forward — while toolcrib’s, once patched, are closed for every future consumer at once.

Project shape	Recommendation
Long-lived app, restyled/rethemed repeatedly, multiple contributors (human or AI)	Toolcrib wins clearly
App needs end-user-facing theme customization (presets, live color picking, persistence)	Toolcrib wins clearly — see Finding 8
One-shot prototype, thrown away after a demo	Roughly a wash — onboarding/context cost isn’t recouped
Frequent narrow, non-semantic one-off visual tweaks (outside the existing subtheme/theme-axis vocabulary)	Unguided/Tailwind wins on flexibility — see Findings 5–6
Non-React stack	Not applicable

Methodology

Prompt (identical for every trial in both conditions):

“Build a single-page admin settings screen: a left sidebar nav (5 items), a top toolbar with a page title and a save button, and a main content area with three stacked sections — a profile card (avatar, name, email fields), a notification preferences section (4 toggle rows with label + description each), and a billing section (a small table of past invoices). End with a sticky footer bar with Save/Cancel buttons. Make it look like a polished SaaS product, fully responsive.”

Session A (control):

npm init -y

No framework, bundler, or CSS approach specified — the agent was free to choose anything. Three independent trials (a1, a2, a3), each a fresh build from the same prompt with no cross-contamination (later trials weren’t shown earlier trials’ output before being built). All three independently converged on the same unprompted default: a static index.html pulling in Tailwind via the CDN <script> tag — the common “quick vibe-coding” reflex — but with genuine structural variation between them (different layout approaches: flex-column sidebar vs. CSS grid template; different accent colors: indigo, blue, violet; different spacing scales).

Session B (toolcrib):

npm init -y
npx toolcrib init --version 0.1.0 --situation new
# reviewed the staged patches in ./toolcrib-patches/
toolcrib apply

This vendors the component library into ./toolcrib/, wires the #toolcrib import alias into package.json, and stages an AGENTS.md/CLAUDE.md block plus the ai-docs/ folder (CORE.md, NEW_APP.md, component-manifest.json) the toolkit’s own README says an AI assistant should read before building. Those docs were read in full before any component code was written, per their own instructions. Since toolcrib is a React library, this necessarily pulled in React/Vite/TypeScript tooling — a stack difference from Session A’s static HTML that’s a genuine confound (Finding 1’s zero-Tailwind result is partly attributable to a completely different toolchain, not toolcrib alone). Three independent trials (b1, b2, b3), each composed with different component choices (Card vs. CardSimple, Toggle vs. a manual toggle group, different Toolbar slot usage) to avoid just re-running the same file three times.

Caveat carried through every finding below: one model (Claude) built both arms of every trial, with full knowledge of the hypothesis going in. This is not a blinded experiment — a real version of this test would use separate model instances or separate human developers unaware of the comparison being drawn.

Finding 1 — Utility-class soup

Trial	Tailwind utility-class tokens
a1	386
a2	307
a3	412
A mean	368.3
b1, b2, b3	0, 0, 0

Mann-Whitney U (one-sided, A > B): U = 9.0, p = 0.032 — the smallest p-value obtainable at n=3 vs n=3, so it mostly just confirms zero overlap rather than being independently strong evidence. The stronger claim comes from source, not statistics: no toolcrib component’s TypeScript interface accepts className or style at all, confirmed by tsc --noEmit succeeding on all three B trials with zero style-prop usage. This is a compile-time guarantee, not a per-trial tendency — running more trials wouldn’t add evidence beyond what reading one interface already established with certainty.

Finding 2 — Structural and visual similarity

Structural (wireframe) similarity: 100% — sidebar, toolbar, profile card, notification rows, billing table, sticky footer all present in both conditions, reflecting that one agent mapped the same spec faithfully onto both toolkits.

Visual similarity out of the box: low. Toolcrib defaults to isDarkMode: true, base hue 88° (lime, #89d82f) — nowhere near the light-mode indigo/blue/violet accents unguided sessions converged on independently (hue distance 129°–174° of a possible 180°, inverted background luminance).

Retheming to match, using only ThemeProvider‘s own parameters (no style props):

Accent hue, light/dark mode, padding density: one config object, exact match.
Corner radius: cornerRadiusMode: 'rounded' reproduces Tailwind’s own rounded/md/lg/xl scale exactly (0.25/0.375/0.5/0.75rem) — not a coincidence, the scale appears deliberately copied.
Background neutrality: close but not exact — every surface color is architecturally tied to the accent’s own hue (s=4 residual tint vs. Tailwind’s fully achromatic s=0), a deliberate tradeoff, not a config gap.
The secondary palette role: not trivial. Every harmony mode (monochromatic/analogous/split-complementary/triadic/tetradic) hue-shifts it away from the base color by construction — none has a “keep this neutral” option. A plain gray “Cancel” button — trivial in Tailwind at zero saturation — has no equivalent one-line path here; it requires overriding --ai-color-secondary directly, outside the parameter API.

Why exact matching has a ceiling regardless of configuration: toolcrib generates its whole palette procedurally from one seed color. Tailwind’s actual palette doesn’t work that way — checking its real published blue-family stops (500→800) shows hue drifting (217°→226°) and saturation moving non-monotonically (76%→86.6%→82.9%) as Value falls steadily, the signature of independently hand-tuned stops per family, not one formula. Matching is only exact where something reduces to an unchanged scalar (a hue number, a copied radius constant) — never for anything toolcrib has to actually derive.

Methodology note, corrected in hindsight: the retheming above was done by computing HSV values by hand (converting target hex codes, predicting resulting backgrounds via the same formulas the engine uses). That’s not the workflow toolcrib’s own docs describe for this exact task — see Finding 8, which covers ThemeEditor as a live, drag-and-see design tool meant to replace this kind of manual color math entirely. The numbers above still hold; they likely overstate the real effort involved.

Finding 3 — Uniform restyling

Task: accent color, corner radius, and density changed app-wide.

	Toolcrib	Unguided (Tailwind)
Edit	6 lines, 1 file (`ThemeProvider` params)	~20 separate find/replace rules across scattered literal classes
Structural/decorative judgment needed	No — `Avatar`/`Toggle` hardcode their own radius (`--ai-avatar-radius`, literal `50%`), decoupled from the shared corner-radius token by construction	Yes — every `rounded-full` instance needs manual inspection to tell “circle” from “card corner”
Result	Clean, uniform, every instance updated	A real, reproducible bug: sequential text-rewrite rules matched each other’s output — `px-2`/`px-3`/`px-4` (three distinct roles: table cells, inputs, buttons) all collapsed into a single `px-5`, silently destroying the original spacing hierarchy

The mechanism: Tailwind classes are independent literal strings with no shared reference — “restyle uniformly” decomposes into N rewrites that must not collide with each other. Toolcrib’s theme has one value per axis, read by every component from a shared source — nothing for a second rule to collide with.

Finding 4 — turn count for styling changes

Finding 3 measured the size of a restyle edit. A different question: how many conversational turns does it take to reach an actually-correct end state, not just a completed-looking one?

Toolcrib: bounded at one turn. The restyle in Finding 3 was a single edit, and when an actual mistake was made in this session while producing it — an earlier attempt used an invalid literal (paddingMode="loose", not one of the three real values) — tsc --noEmit caught it immediately, in the same turn, before anything was reported as done. The correction happened inside the attempt itself, via a compiler forcing function, not as a separate turn triggered by someone noticing later.

Unguided: the first turn can complete “successfully” while silently wrong, with no bound on when that’s discovered. The same restyle’s sed-cascade bug (Finding 3) is the direct evidence: the script ran without error, the HTML rendered without error, and the task was reportable as done after one turn — while three distinct spacing roles had actually collapsed into one, undetected by anything automated. No compiler, linter, or test in that stack flags a semantically-wrong-but-syntactically-valid utility class. That defect surfaced in this report only because this evaluation went back and specifically diffed spacing token counts across the before/after files — a deliberate audit step, not something the original restyle turn would have triggered on its own. In an ordinary session, correctness would depend entirely on whether and when a human happens to notice the visual regression: could be the next turn, could be never.

	Toolcrib	Unguided
Turns to a reportable result	1	1
Turns to a verified-correct result	1 (compiler gates it before completion)	1 + N, N indeterminate (depends on a human noticing)
What catches a mistake	`tsc --noEmit`, in-turn	Nothing automated; a later, separate review pass

This is the same asymmetry as the discovery-cost point in the Verdict, scoped specifically to the styling-change task Finding 3 already measured: toolcrib’s guarantee isn’t just that the edit is smaller, it’s that the edit’s correctness is checked before the turn ends, where the unguided equivalent has no such gate at all.

Finding 5 — Row-level and other narrow customization

Toolcrib’s components fall into two categories:

Slot/child/render-based (Card, Form, Toolbar, DataTable‘s own cell render): a plain, fully-stylable wrapper element is always one JSX node away. The “no style/className” rule only restricts the toolkit’s own component props here — your own elements inside are unrestricted.
Internally-owned DOM never exposed as a prop or slot: here the restriction genuinely bites. DataTable‘s row (<tr>/<td>) is now the one exception, closed via rowSubtheme?: (record, index) => 'error'|'success'|'warning'|'info'|undefined — the row-level equivalent of the subtheme prop already used on Button/Toast/Progress. Flagging a row (e.g. a pending invoice) is one line, tinting the row’s real background/border with the same WCAG-guaranteed colors used everywhere else in the system — not a per-cell approximation. Rows with no match render exactly as before.

Deliberately not added: a generic rowClassName/rowStyle escape hatch — that would reopen exactly the raw-style door the toolkit’s uniform-restyle guarantee (Finding 3) depends on staying closed. The boundary that remains: a row needing flagging for a non-semantic reason (“recently edited,” “selected”) still has no hook, since only the four existing subthemes are covered. General lesson for any other component with the same shape of internal-DOM restriction: check whether it exposes a slot before assuming a workaround is needed.

Finding 6 — Token cost

Toolcrib’s own recommended reading set (CORE.md + NEW_APP.md + component-manifest.json) is ≈83.5K characters (≈21K tokens, chars/4 estimate) of onboarding cost per fresh session — Tailwind needs none.

Output cost is shape-dependent, not uniformly cheaper or more expensive:

Task	Unguided	Toolcrib
Global restyle (color/corners/density, app-wide)	Full-file regen — no single edit point	233 chars, one config block
Row-level flag (semantic: error/success/warning/info)	~35 chars, one class	~79 chars, one prop — comparable
Row-level flag (non-semantic category)	~35 chars, one class	No hook exists; would require a new theme slice — a materially bigger task

Breakeven for the onboarding cost against repeated global restyles: roughly 11 cycles (≈1,880 tokens saved per restyle against the ≈21K upfront cost). Caveats: chars/4 is a rough proxy, not a real tokenizer; prompt caching could push the effective onboarding cost toward zero on repeat sessions against the same project; output tokens typically price higher than input tokens on API pricing, which isn’t reflected in the raw counts above.

Finding 7 — WCAG contrast enforcement

Every dynamically-generated palette color — primary/accent, all four subthemes, on-fill button/badge text — is nudged via ensureWCAGContrast()/pickReadableTextColor() to clear a minimum contrast ratio (3.0:1 for large text/UI, 4.5:1 for normal text) against its actual background, regardless of which base hue is chosen. Verified across five tested hues (lime/indigo/teal/blue-600/red), including --ai-text-secondary against all three surface tiers (bgPrimary/bgSurface/bgContainer) — every combination clears 4.5:1 or better.

For comparison: one of three independent unguided trials shipped text-gray-400 on white at 2.54:1 — failing WCAG AA outright, with nothing in an unguided session catching it. That’s the exact class of defect this machinery is built to prevent, and does, for every color it touches.

Finding 8 — dynamic theme customization has two consumers, not one

Every finding above (except the retheme exercise in Finding 2) concerns build-time decisions locked in once. A different question: what does it take to let someone other than the original build-time decision-maker pick or change the theme, with their choice persisting? There are two distinct someones, and toolcrib’s own docs treat them as two separate, intentional use cases for the same infrastructure:

The app’s own designer/developer, at development time. NEW_APP.md‘s own guidance: “Drop <ThemeEditor trigger={...}> somewhere reachable early in development… use it to pick the base colour, harmony mode, and spacing scale interactively. Once you’re happy, read the resulting parameters off useTheme()… Don’t hand-pick colours.” This is a live, visual alternative to exactly the manual HSV computation Finding 2’s retheme exercise did by hand (colorsys.rgb_to_hsv, computing predicted backgrounds via Python) — the documented workflow is drag sliders, watch the app update live, read off the result. Finding 2 didn’t use it, and in hindsight that means the real dev-time cost of matching a target palette is likely lower than the “one config object” already measured there, since the intended path removes the color-math step entirely rather than just shortening it.
An end user of the deployed app, at runtime. This is what the rest of Finding 8 below actually tested.

Toolcrib: one component serves both roles, pre-built. ThemeEditor (hosted in a SlideOut/Modal/Popup of your choosing — it has no overlay chrome of its own) bundles continuous H/S/V sliders, 6 shipped presets, a “your saved themes” library (named save/load/delete via localStorage), and file export/import, all wired to the same ThemeProvider state validated in Finding 7. Wiring it into the settings screen and adding auto-persistence via the existing theme:changed event-bus channel cost 17 lines / 660 characters across two files — and every color it can produce, whether picked by a designer tuning the shipped default or an end user personalizing later, still goes through ensureWCAGContrast()/pickReadableTextColor().

Unguided: neither role has an equivalent tool. A developer manually edits hex codes in utility classes and reloads to see the result — no live preview, no interactive picker, nothing beyond what Finding 2’s own manual HSV math demonstrates. An end user has nothing at all unless a bespoke picker is built from scratch, tested next:

Unguided end-user customization has to be engineered from scratch, and the accessibility guarantee doesn’t come for free. Tailwind’s utility classes are static — runtime end-user color customization means abandoning most literal color classes for CSS-variable-driven arbitrary values (bg-[var(--accent)]), hand-rolling shade derivation (a “hover” and a light tint via plain RGB math), and hand-rolling persistence. Built the equivalent for the same settings screen: 35 lines / 2,729 characters, with a single flat accent color, no presets, no named saved-theme library, no file export — and critically, no contrast checking of any kind.

That gap isn’t hypothetical. Tested 4 plausible end-user picks (a light yellow, mint green, pale pink, light gray) against the unguided version’s text-white Save button:

End-user pick	White-text contrast ratio	Result
Light yellow `#fde047`	1.32:1	Unreadable
Mint green `#86efac`	1.40:1	Unreadable
Pale pink `#fbcfe8`	1.38:1	Unreadable
Light gray `#e5e7eb`	1.24:1	Unreadable

All four fail outright — not edge cases, the default outcome for any light pick with the natural unguided implementation. Toolcrib’s pickReadableTextColor() provably reaches ≥4.58:1 against any background (it’s a closed-form choice between pure black/white, not a per-color check that can be forgotten) — the same guarantee applies automatically to a color the end user picked, not just ones a developer chose and could manually spot-check.

Finding 9 — components are load-bearing architecture, not styling convenience

Every finding above treats toolcrib’s components as, in effect, styled <div>s — measuring how easy they are to color, restyle, and keep accessible. But components like Modal, AlertDialog, SlideOut, TabStrip, and Splitter also carry real behavioral infrastructure that has nothing to do with color: focus management, portal rendering, cross-tree state coordination, and layout-aware geometry. Tested directly with a concrete feature, not just described.

Task: add a Delete button to each row of the billing table. Clicking it opens a confirmation dialog — not nested in the row itself, since the table may be virtualized or paginated — that must correctly trap focus while open, dismiss on Escape, and return focus to the row’s Delete button on close.

Toolcrib: the row’s button calls aiBus.openAlertDialog('confirm-delete-invoice', { invoice: record.invoice }) — one line, no callback prop threaded through DataTable at all. A single <AlertDialog> mounted once at the app root listens for that id and renders the confirmation. Total: 31 lines / 1,231 characters, and every one of the following came from AlertDialog wrapping Radix UI’s AlertDialogPrimitive correctly, not from anything hand-rolled in this feature:

Full focus trap while open (verified: Radix’s own accessible-dialog behavior, not toolcrib-original logic — toolcrib’s actual contribution here is guaranteeing this primitive gets used via its own anti-pattern guidance, “don’t hand-roll a popup/modal/drawer,” rather than reimplementing focus-trapping itself).
Escape-to-close, matching native confirm() convention (confirmed against Radix’s own source per the component’s code comments — outside-click is deliberately disabled instead, appropriate for a destructive-action dialog, not an oversight).
Correct ARIA dialog semantics and focus restoration to the triggering element on close.

Unguided: built the same feature — a delete button per row, a confirmation dialog, cancel/confirm — with no special accessibility instruction given, matching how an ordinary request would actually be phrased. 32 lines / 2,019 characters (comparable cost, prop-drilling avoided here only because a flat HTML file’s global function scope happens to sidestep it — a real componentized React app without an equivalent to aiBus would need actual prop-drilling or lifted state, making this an underestimate of the real unguided cost). Checked the result directly for the four standard behaviors above, not assumed:

Behavior	Found in unguided implementation?
Any `.focus()` call (focus trap)	0 occurrences
Any `keydown`/`keyup`/Escape handling	0 occurrences
`role="dialog"`, `aria-modal`, `aria-labelledby`, or `aria-describedby`	0 occurrences
Any tracking of the triggering element (focus restoration)	0 occurrences

Not “missing some” — missing all four, confirmed by direct inspection of the code produced, the same style of check as Finding 1’s className/style grep. A keyboard user could still tab through the page behind this “modal,” nothing closes it via Escape, a screen reader has no indication a dialog opened at all, and focus is simply lost on close rather than returned to the trigger. This is the same class of silent, uninstructed omission demonstrated in Findings 7 and 8, in a different subsystem: not a color contrast failure, a keyboard-accessibility and screen-reader failure, equally undetectable by anything in the unguided build process itself.

This isn’t one component’s incidental correctness — it’s a systematic architectural choice, verified across the whole library. Checked which of toolcrib’s 30 components sit directly on a named Radix primitive versus which are hand-rolled markup:

	Count	Examples
Built on a Radix primitive	18 of 30 (60%)	`Modal`(Dialog), `AlertDialog`, `Popup`(Popover), `SlideOut`(Portal), `TabStrip`(Tabs), `DropdownMenu`, `ContextMenu`, `Select`, `RadioGroup`, `Slider`, `Accordion`, `Collapsible`, `Toast`, `Tooltip`, `Avatar`, `Separator`, `ToggleGroup`, `Checkbox`/`Switch`
Hand-rolled, no Radix foundation	12 of 30 (40%)	`DataTable`, `Splitter`, `Card`/`CardSimple`, `AppShell`, `Toolbar`, `UIGroup`, layout primitives (`Grid`/`Stack`/`Content`), `ThemeEditor`, `AIErrorBoundary`

This correlates exactly with where this report already found gaps, not by coincidence: DataTable — the one component whose internal row markup had no escape hatch at all until Finding 5’s patch — is in the hand-rolled column. TabStrip, by contrast, sits on Radix’s Tabs primitive and inherits its roving-tabindex keyboard model and ARIA tab semantics for free. This also sharpens a speculative note in Threats to Validity below: Splitter (also hand-rolled, no Radix) is the more plausible candidate for a similar internal-DOM gap to DataTable‘s; TabStrip (Radix-backed) is less so, on the same logic that predicted DataTable‘s gap in the first place. For 60% of the library, accessible focus management, keyboard navigation, and ARIA semantics aren’t something toolcrib re-implements per component and could get subtly wrong each time — they’re inherited once, from one well-tested foundation, everywhere that foundation is used.

The generalizable point: a component library earns “load-bearing” status when removing it would require reimplementing correctness-critical behavior, not just re-picking colors. aiBus (cross-tree dispatch without prop-drilling or context restructuring), the layout-domain system (LayoutDomainContext/useLayoutDomain, which lets a <Card> nested inside a <Splitter.Panel> automatically square its corner adjacent to the resize handle — geometry-aware behavior no styling system provides), and the systematic Radix foundation underneath 60% of the component set are all in this category. None of them are visible in a screenshot comparison; all of them are visible the moment a feature requires cross-component coordination or correct keyboard/screen-reader behavior, which most real features eventually do.

Finding 10 — the docs are AI-maintained, and what that does and doesn’t imply

All of toolcrib’s ai-docs/*.md are themselves AI-generated and AI-maintained — only the repo’s root README isn’t. That reframes every gap in Findings 5/7 above (not Finding 8, which found no gap — its shipped capability worked as documented): not a scattered collection of unrelated bugs, but one recurring shape — documentation and tooling produced for AI consumption, by an AI, without something adversarially trying to use it the way a consumer would. Generation and adversarial-use-checking are different activities; being capable of one doesn’t guarantee the other happened.

Where this needs tempering: this is a maintenance-process observation, not evidence of an inevitable failure loop. A human is still the one deciding a gap matters enough to fix and directing the correction — which is exactly how the five patches referenced throughout this report came to exist: gaps surfaced by actually trying to build something, a human decided they were worth institutionalizing rather than left as one-off workarounds, and verified patches came out the other side. That’s the normal corrective mechanism any human-governed codebase relies on, AI-authored docs or not.

What this doesn’t guarantee: if toolcrib’s maintenance pipeline regenerates these docs again without incorporating this specific pattern, a structurally similar gap could reappear somewhere else — a different component’s manifest entry, a different anti-pattern the table doesn’t cover yet. The patches in this report close the specific gaps this evaluation happened to hit; they don’t retroactively audit every other component for the same shape of issue.

Worth noting for calibration: every automated check that existed throughout this evaluation caught real regressions reliably — tsc --noEmit, the 212-test suite, and the doc-drift checks all passed cleanly on every patch. Those checks are good at “does generated output match what the generator would produce” and “does existing behavior still work.” That’s a different thing from “is the generated output complete” — which is exactly the class of gap that only surfaced by trying to build the settings screen, hitting a wall, and going looking for why.

Patch verification methodology

The five patches referenced above (WCAG contrast fix, manifest type-definition resolution across two files, DataTable.rowSubtheme, and the matching Anti-Patterns doc entry) were verified against the real upstream repository, not just written and assumed correct:

Fresh clone of github.com/escape-llc/toolcrib (not the pre-existing working copy the patches were drafted against).
Applied all five diffs in sequence via git apply --check followed by git apply — confirmed each applies cleanly with zero conflicts, in dependency order, on the clean checkout.
Dependency install: npm install. One unrelated environment finding surfaced here: the repo pins "typescript": "^7.0.2", but that package’s root export (. → lib/version.cjs) is now just a version stub — the actual Compiler API moved to typescript/unstable/ast and sibling subpaths. The pre-existing, unmodified scripts/lib/extract.js (classic ts.createSourceFile/ts.ScriptTarget API, used throughout the file, not only by these patches) doesn’t run against that as installed. Verification below used typescript@5.7 installed locally (--no-save, not committed to the project) purely to execute and test the patches; this incompatibility is separate from anything in the patches themselves and would need its own fix.
Typecheck: npx tsc --noEmit → clean, whole project.
Manifest regeneration: node scripts/generate-manifest.js --write → 32 component(s), 39 event channel(s), 10 helper method(s), 156 CSS variable(s), including 8 newly-resolved $defs entries (Column plus 7 others with the same latent type-flattening gap: AccordionItemData, CardOverrides, MenuItemData, RadioOption, SelectOptionData, TabItem, ToggleGroupOption).
Docs regeneration: node scripts/generate-docs.js --write → regenerates ai-docs/CORE.md cleanly, rowSubtheme present in both the auto-generated Component Reference table and the newly-added Anti-Patterns row.
Drift checks (both must report no difference between generated output and what’s committed): node scripts/generate-docs.js --check → “ai-docs/CORE.md matches the template + source exactly”; node scripts/generate-manifest.js --check → “component-manifest.json matches source exactly”.
Full test suite: npx vitest run → 48 test files, 212 tests, all passing — including the project’s own docsInSync.test.ts (which independently re-runs the drift checks above as part of its assertions) and DataTable.test.tsx (6/6, confirming the new rowSubtheme prop doesn’t change behavior for rows that don’t set it) and harmonies.test.ts (4/4, confirming the WCAG patch doesn’t break existing theme-generation tests).

No step above was skipped or assumed to pass based on the patch’s own internal logic looking correct — each ran against the actual real source tree, post-patch, before being reported as verified.

Threats to validity

Same-agent bias. One model built both arms of every trial, aware of the hypothesis throughout. No blinding. A rigorous version of this test would use separate model instances or separate human developers unaware of the comparison being drawn.
N=3 per condition for the stochastic metrics (Tailwind token counts) — enough to demonstrate the effect, not a rigorous population estimate. Doesn’t matter for the Tailwind-count result specifically (a type-system guarantee, not a sample statistic), but does matter for anything not backed by an equivalent structural proof.
Stack confound. Toolcrib forces React/TSX/Vite; Session A was free to pick anything and defaulted to static HTML + Tailwind CDN. Some of every result above is inseparable from “different tooling entirely,” not purely “toolcrib vs. nothing.”
No real browser rendering. A true pixel-level visual diff (Playwright/Chromium) was attempted and blocked by this environment’s network allowlist. All visual-similarity findings are derived analytically from theme/CSS source values, not screenshots.
Token cost estimates are approximate (chars/4), not measured from real API usage logs or an actual tokenizer.
The DataTable row-hook gap (Finding 5, now patched) was one example of one component. Finding 9’s Radix-foundation survey narrows this rather than resolving it: Splitter (also hand-rolled, no Radix) is now the more plausible candidate for a similar internal-DOM gap; TabStrip (Radix-backed) is less so. That’s an informed prediction from architectural pattern, not a verified finding — Splitter itself wasn’t tested the way DataTable was.
Finding 9’s accessible-dialog behavior was tested for one component pair (AlertDialog/DataTable) and verified by static inspection of the generated code (.focus() calls, ARIA attributes present), not by driving an actual browser with a screen reader or keyboard-only navigation. The same network restriction that blocked visual screenshot testing (Threat 4) would also block that kind of live behavioral verification here.

Further work

None of the following would change the verdict — they’d sharpen the decimals:

Real screenshot-based visual diff (needs network access to a Chromium download host, or a pre-installed browser).
Blinded trials across genuinely separate model instances or human developers, to remove same-agent bias.
Real API-log token costs in place of the chars/4 proxy, ideally with prompt caching enabled to measure the true amortized onboarding cost.
An unguided-retheming test — does a session using toolcrib reach for ThemeEditor/initialParameters without being told to, the way the unguided sessions reached for Tailwind without being told to? (Out of scope here: theming is treated as a human-in-the-loop decision point in both conditions, not something either tool should autonomously do.)
Larger-N global-restyle trials to firm up the token-cost breakeven estimate.
A survey of which other toolcrib components expose internally-owned, unreachable DOM (like DataTable‘s row used to) versus which already expose enough slots that the constraint never bites.

Bottom line

Toolcrib removes ad hoc styling from its own components by construction, and that removal is real, uniform, and doesn’t have a style-prop backdoor anywhere in the toolkit’s own API surface. It doesn’t remove styling flexibility from your own code — a plain wrapper inside a slot is unrestricted — and the one place the restriction used to have no answer at all (a data table’s own row markup) now has one, scoped to the four existing semantic categories rather than reopened as a general escape hatch.

It also turns a category of feature that’s genuinely hard to build well — end-user-facing runtime theme customization with guaranteed accessible contrast — into a near-zero-cost addition, where the unguided equivalent isn’t just more code, it’s code that fails silently the moment a real user picks a plausible color.

None of that is the whole story, and treating it as a styling toolkit undersells what’s actually load-bearing in it. aiBus (cross-tree dispatch without prop-drilling), the Radix foundation underneath 60% of its components (Finding 9), and the layout-domain corner-squaring system are not styling conveniences — they’re what stands between “confirmation dialog” and a dialog that silently fails at keyboard access and screen-reader semantics, the same silent-failure pattern demonstrated for color throughout this report, just in a subsystem a sighted developer testing with a mouse is far less likely to ever notice is broken. An unguided build of that same feature had zero of the four standard accessible-dialog behaviors. Evaluating toolcrib only on how it handles color, and stopping there, would have missed roughly half of what this evaluation actually found.

Adopt it where the actual pain is inconsistent, hard-to-globally-restyle output across many contributors or iterations, where interaction correctness (focus management, cross-tree state, keyboard/screen-reader access) matters as much as appearance, and where a project can accept React/TSX as the price of entry. The accessibility floor is a genuine bonus most unguided output has no equivalent of, extending automatically to colors an end user picks and to every component built on the Radix foundation, not just ones a developer happened to get right. Expect the remaining friction to be narrow and specific — non-semantic per-instance styling treatments on components that don’t expose a slot for them, and the ~40% of the component set (DataTable, Splitter, and similar) that doesn’t inherit the Radix foundation — rather than pervasive.

The real cost comparison isn’t “tokens to generate the first draft.” It’s tokens (and turns, and calendar time) to discover a defect exists at all, since nothing in an unguided session flags one on its own — followed by an indeterminate number of correction turns once someone finally does. Toolcrib doesn’t eliminate that discovery cost; the five patches in this report all took real, deliberate effort to find. What it changes is what happens after: a fix closes structurally, once, for every future project — where an unguided project’s fix is local to that one file, and the next unguided session re-risks the identical mistake with nothing carried forward.

If one sentence has to carry this report past the specifics of any one library: unconstrained, unguided output is not a stylistic inconvenience, it’s an unmonitored failure surface, in styling and in interaction correctness alike — this evaluation produced a silent functional regression, a genuine WCAG failure, a 100% failure rate on plausible end-user color picks, and a confirmation dialog with zero of four standard accessible-dialog behaviors, all without the build process itself raising a single flag in any case. Whether the fix is toolcrib, a different design-token system, automated accessibility testing in CI, headless accessible primitives as the default rather than hand-rolled markup, or a human review step that actually happens every time — the specific mechanism matters far less than the fact that some mechanism needs to exist, covering behavior as much as appearance. This evaluation’s evidence is about what happens when none does.

2026-08-17

Site Update

Dreadfully sorry everyone, the Hive Lite theme we were using got corrupted, so we have switched themes.

There were also some plugin update issues, which we are working through.

Thanks for your cooperation and patience.

2025-12-21
MediaLab Used in Exciting Research Project on Bats
We are extremely pleased to announce that MediaLab was an integral part of the data processing behind an exciting research project on Little Brown Bats (Myotis lucifugus) at Tippy Dam in Michigan!

Tippy Dam in the Northern Lower Peninsula Michigan

See the article here from PLOS!

See the blog post from one of the researchers!

Why is this research important?

A fungus known as White Nose Syndrome (Pseudogymnoascus destructans) has been decimating bat populations all over the world, but this particular bat population is surviving in spite of it being present in their hibernaculum.

Little Brown Bat (Myotis lucifugus) with White Nose Syndrome (Pseudogymnoascus destructans)

A large part of the research consisted of analyzing 1000s of hours of infrared video of the bats in two different “rooms” of the dam’s interior. The researchers originally tried to use an existing “recipe” to process the videos, but encountered many technological obstacles carrying it out, and ultimately met with failure.

Out of this failure, MediaLab was born! Using Microsoft Visual Studio, the OpenCV computer vision library, and the Universal Windows Platform (UWP), we set about creating the image processing application that brought this research project to a successful dataset for analysis. We are also pleased that it is in Windows Store for the last 18 months, with hundreds of installs!

Why didn’t we mention this sooner?

Due to the competitive nature of research in this field, we were asked to keep a “low profile” with regard to bragging. Nevertheless, many of our screen images posted in the Wiki and elsewhere provided clues to how MediaLab was being used to produce hundreds of megabytes of data to bring this research project to a successful completion.

What did we do with MediaLab on this project?

There were several key processing elements:
- Load seven-day-long video files and extract “frames” every 10 minutes.
- Save each frame as an individual image file for subsequent processing.
- Then for each “set” of frames:
  - Run a MediaLab “pipeline” consisting of multiple CV operations.
    
    Contrast-limited Adaptive Histogram Equalization
    
    Gaussian blur
    
    Threshold
    
    Contour extraction (canny)
    
    Contour data output
  - Different videos required parameter adjustments on the pipelines.
  - Output the data into CSV file(s) for subsequent processing.
- Then for each “set” of CSV data files:
  - Process in the R statistics platform to produce statistics and graphs.
2024-02-07
New VueJS Component on NPM

We are now publishing on NPM and our first offering is a VueJS component for PDF rendering based on pdfjs.

Here is the link.

2023-10-01
MediaLab 1.0.18
2022-12-07 – 2022-12-14 EST

New Image Viewer Page and other internal enhancements.
Google Calendar

iCalendar

Outlook 365

Outlook Live
2022-12-07

Blog

1. What Toolcrib Actually Is

2. The Load-Bearing Infrastructure Layer

3. Project Maturity (important context)

4. Component Coverage vs. shadcn/ui and Others

5. Pros

6. Cons / Risks

7. When Each Makes Sense

8. AI-Optimization Evaluation (All Toolkits)

Per-toolkit take

9. A Meta-Observation: The Same Feedback Loop, at Two Layers

10. Bottom Line

The claim, up front

A note on terminology

Built with an AI collaborator, not retrofitted at one

Why this isn’t really about accessibility

What a floor actually replaces

The theme engine specifically: consistent visuals that don’t get reinvented per element

The event bus and shared observers: architecture the model doesn’t have to invent per feature

This is one solution, not a bundle of unrelated ones

The evidence: a controlled experiment

Finding 1: the floor produces dramatically less code to get the same result

Finding 2: freehand code reproduces its own mistakes, even in the same sitting

Finding 3: retrofitting consistency after the fact is expensive, and it doesn’t compound

Finding 4: the floor is continuously tested by someone other than you

What this doesn’t fix

Recommendation

Method

Results

Friction-point comparison against the v0.4.0 run

Functional verification

Head-to-head: the four ACTION_PLAN.md gaps, re-tested live in v0.5.0

Extension: a new feature, worded as a consumer would ask it

The cost of retrofitting a floor, measured concretely

The QA asymmetry, verified directly: does either leg’s floor have tests behind it?

New observations specific to v0.5.0

Bottom line

Method

Results

Functional verification

Friction points encountered

Correctness & quality: beyond markup coverage

DataTable: head-to-head on features and correctness

Form validation: head-to-head

Bottom line

Discussion: the repair loop

Why these gaps matter less than they would in the uncontrolled leg

Where the gaps cluster, and why

Why the floor matters more than the bug count, and why AI-written tests don’t fully fix that

Appendix: tooling used for verification

Verdict

Methodology

Finding 1 — Utility-class soup

Finding 2 — Structural and visual similarity

Finding 3 — Uniform restyling

Finding 4 — turn count for styling changes

Finding 5 — Row-level and other narrow customization

Finding 6 — Token cost

Finding 7 — WCAG contrast enforcement

Finding 8 — dynamic theme customization has two consumers, not one

Finding 9 — components are load-bearing architecture, not styling convenience

Finding 10 — the docs are AI-maintained, and what that does and doesn’t imply

Patch verification methodology

Threats to validity

Further work

Bottom line

Head-to-head: the four `ACTION_PLAN.md` gaps, re-tested live in v0.5.0