← research

Experiment: GLM vs Opus, blind A/B on a real UI task

Jul 14, 2026

2026-07-07. Same 45-minute work brief handed to GLM (z.ai) and Claude Opus; diffs judged blind by GPT-5.5 as X and Y. Cost: GLM ~$0.04 vs Opus ~$2.73 — ~70× cheaper. Verdict below: X = GLM scored 4/10, Y = Opus scored 8/10. Conclusion that entered our routing doctrine: budget-model UI output needs a review-and-fix pass before it ships; the cost win is real, the quality gap is too.

The brief (verbatim, as given to both models)

Work brief: Thermometer "cross-chain reveal" (tap-to-separate bands)

Repo: Rally (TanStack Start + React 19 + Tailwind, bun package manager). Scope: src/components/Thermometer.tsx and src/components/LiquidColumn.tsx ONLY. You may add small helper files under src/components/ if genuinely needed, but prefer keeping the change contained to these two files. Do NOT touch contracts/, routes/, other components, tokens.css, or package.json (no new dependencies — implement with what's already imported: React, the existing canvas 2D rendering approach, Tailwind classes, lucide-react icons if needed).

Source of truth

This repo has a design bible at src/design/BIBLE.md. Section 3 ("The signature moment — the liquid") specifies, among other things:

The cross-chain reveal: you can see "half came from Solana" in the band heights — no legend needed. Tap the mercury → the bands gently separate with labels (a delightful, discoverable detail).

This interaction does not exist yet. Thermometer.tsx renders a LiquidColumn (canvas-rendered liquid with stacked chain "bands") but there is no tap/click handler and no separated/labeled state anywhere in either file.

Task

Implement the tap-to-reveal interaction described above, end to end:

  1. Trigger. Clicking/tapping the glass tube (the thermometer itself, i.e. the role="progressbar" element in Thermometer.tsx) toggles a "separated" view. Only meaningful when there is more than one band (i.e. real segments were passed in with 2+ chains contributing) — if there's 0 or 1 band, tapping should either no-op or be visually inert; don't crash or show a pointless single-band separation.

  2. Separated visual. While separated, the stacked bands should gently pull apart (a small visible gap between each chain's band) so each chain's contribution reads as a distinct segment rather than a seamless gradient stack. This must be an animated transition (ease in/out over roughly 250-400ms), not an instant snap, and must respect prefers-reduced-motion (already checked elsewhere in Thermometer.tsx via reducedMotion — reuse or thread that same signal into LiquidColumn; if reduced motion, apply the separated state without animating it).

  3. Labels. Each separated band shows a label with:

    • the chain's display name, from CHAIN_META[chain].label (imported from #/design/chains), and
    • that chain's share, either as a dollar amount (formatUsd(...)) or a percentage of the total raised — your call, pick one and be consistent. Labels must be legible against the band's own gradient color (adjust label text color/weight/shadow as needed — don't hardcode a color that only works on one chain's hue). You can render labels as a DOM overlay positioned over the canvas (simplest and most robust) or draw them on canvas — DOM overlay is the safer choice given the existing canvas is bare pixels with no text layout today.

  4. Un-reveal. Tapping again (or tapping anywhere else on the tube) returns to the normal merged/stacked view, with the same kind of eased transition.

  5. Accessibility / hit target. The tube already has role="progressbar" with aria-valuenow/aria-valuemin/aria-valuemax/aria-label. Keep those intact. Make the tap target keyboard-operable too (e.g. the toggle should also fire on Enter/Space when the element is focused) — a progressbar isn't natively interactive, so add whatever minimal affordance (tabIndex, key handler, and an appropriate aria-pressed or similar state, or a wrapping button) makes this a real accessible toggle rather than a mouse-only click handler. Use your judgment on the cleanest way to do this without a large refactor.

  6. Don't break existing behavior. The live "pour" animation (money landing, goal-crossing confetti trigger, etc.) already implemented in both files must keep working exactly as before when NOT in the separated state. Horizontal orientation (orientation="horizontal") doesn't need the full reveal treatment, but must not crash or visually break if someone taps it — no-op is fine there if you decide the reveal is vertical-only (there's a reasonable argument either way; your call, document it in a one-line comment).

Definition of done / how this will be graded

  • bun run build (vite build via TanStack Start) succeeds with no new TypeScript or build errors. This currently passes on main — that's your baseline; don't regress it.
  • Note: bun run test (vitest) is already broken on this repo independent of your change — it picks up nested contract-fixture test files from an unrelated .claude/worktrees/ submodule and fails before running any real test. Ignore it; it is not part of your grading signal and you are not expected to fix it.
  • No new npm/bun dependencies added (check package.json is untouched).
  • The two files' existing exported component APIs (ThermometerProps, LiquidColumnProps) should stay backward compatible — existing callers (src/routes/index.tsx, src/routes/c.$id.tsx, CampaignCard.tsx, RoundBar.tsx) must keep compiling without changes, unless you have a good reason to extend (not replace) the prop surface.
  • Code should match the existing style in these files: TypeScript strict, small focused comments explaining why (this codebase's comments explain design intent, not just what the code does — match that voice), Tailwind utility classes for layout/typography, no any.

Out of scope (do not attempt)

  • The canvas→WebGL rewrite mentioned elsewhere in BIBLE §3 ("explore a lightweight WebGL shader... canvas-2D with craft is the reliable floor") — stay on canvas 2D.
  • PWA shell, Motion library install, View Transitions, tokens.css wiring — all separate, unrelated line items from the parent GitHub issue (#15). This brief is scoped to the single "tap to reveal" interaction only.
  • Any other route, page, or component beyond Thermometer.tsx / LiquidColumn.tsx.

Budget

You have approximately 45 minutes of engineering time for this. Ship a complete, working version of the interaction above rather than a partial attempt at more.

The blind judge's verdict (verbatim)

Scorecard

Req X Y
1 Trigger PARTIAL: click/key toggles, but gates on raw segments.length > 1, so one real positive band can still become interactive. PASS: gates on normalized bands.length > 1 and vertical orientation.
2 Separated visual PARTIAL: gaps animate normally, but reduced-motion separation does not apply because gap is only set inside stepPhysics. PASS: shared rect math opens animated gaps; reduced motion snaps via sepRef.
3 Labels FAIL: labels use CHAIN_META, but are clipped inside the 44px tube, can be stale, and are hidden for bands under 20px. PASS: labels use CHAIN_META, percent share, external dark pill/scrim for legibility.
4 Un-reveal PARTIAL: toggles back in normal motion; reduced-motion path is broken. PASS: same eased state path reverses cleanly.
5 Accessibility PARTIAL: focus + Enter/Space work, but no exposed toggle state. PASS: tabIndex, Enter/Space, aria-pressed, original progressbar aria retained.
6 Existing behavior PARTIAL: pour/confetti likely preserved, horizontal inert, but reduced-motion reveal regresses. PASS: pour/confetti path preserved; horizontal is documented no-op.
DoD / checks PARTIAL: build passes, no deps, scoped files only, but CHECKS reports 3 new TS6133 errors. PASS: build passes, 0 scoped tsc errors, no deps, scoped files only.

X: Top Defects

  1. src/components/LiquidColumn.tsx (diff_X.patch:87-92, 185-195): reduced-motion reveal never applies. gapRef is set only inside stepPhysics, but the effect renders before stepping and ensureRunning() exits under reduced motion.

  2. src/components/LiquidColumn.tsx (diff_X.patch:153-169, 232-265): label positions are mutated in a ref during canvas paint, then read during React render. Ref changes do not re-render, so labels can stay at stale unseparated positions.

  3. src/components/LiquidColumn.tsx (diff_X.patch:7, 57-59) and CHECKS: unused formatUsd, pct, totalRaised, and goal introduce 3 new TS6133 errors.

Y: Top Defects

  1. src/components/Thermometer.tsx (diff_Y.patch:316-332): nonzero tiny shares can render as 0% because the label uses Math.round(...) with no <1% case.

  2. src/components/Thermometer.tsx (diff_Y.patch:248-257, 311-314): labels are positioned at final separated rects and only opacity-fade, so during the gap animation they can briefly lead the moving bands.

  3. src/components/Thermometer.tsx (diff_Y.patch:275-284): aria-pressed on a role="progressbar" is a pragmatic but imperfect ARIA pairing; a wrapper/toggle control would be cleaner.

Verdict

Merge Y. It meets the brief end to end, passes the mechanical checks, preserves the existing animation paths, and has substantially better rendering architecture.

Quality scores: X: 4/10, Y: 8/10. sfies the brief end to end with clean mechanical checks and better design judgment. X has a working start on the canvas separation, but the type errors and clipped/stale labels block merge.

Quality: X 4/10, Y 8/10.