Evals + Claude-Code Usage/Cost Observability (code-level)

Treasure-trove extraction for Claude Mission Control (a Claude-Code observability app that parses ~/.claude/projects/**/*.jsonl and wants an evals layer). Doctrine: patterns mined from the actual code, with real file-path citations, not the authors' blog posts.

Repos cloned to ~/Developer/reference-repos/:

• evalite/ — mattpocock/evalite (local-first TS eval runner on Vitest)
• ccusage/ — ryoppippi/ccusage (v20 — now a Rust binary; the classic TS parser was rewritten)
• CodexBar/ — steipete/CodexBar (Swift menu-bar usage-stats app)

The single highest-value finding for Mission Control's render-storm/perf problem is CodexBar's incremental append-only JSONL cache (§3.1) combined with ccusage's prefilter-before-parse pipeline (§2.1). Both are transcribed in full below.

⚠️ Scope note: ccusage v20 shipped its parsing logic as a compiled Rust crate (rust/crates/ccusage/). The npm ccusage package is now just a thin native-binary launcher (apps/ccusage/src/cli.js). All ccusage parsing citations below are Rust. The techniques (byte-line splitting, SIMD prefilter, typed lenient deserialize, size-balanced parallel chunks) port cleanly to a Node/Bun worker-thread pipeline.

1. evalite — evals layer patterns

1.1 The prod no-op boundary (env-var gate + identity wrapper)

The single most important design decision: the AI-SDK trace wrapper is always safe to leave in production code because it compiles to an identity function unless the runner turns it on.

packages/evalite/src/traces.ts:8

export const shouldReportTrace = (): boolean => {
  return !!process.env.EVALITE_REPORT_TRACES;
};

export const reportTrace = (trace: Evalite.Trace): void => {
  if (!shouldReportTrace()) return;           // <-- prod: early return, zero cost
  const _reportTrace = reportTraceLocalStorage.getStore();
  if (!_reportTrace) throw new Error("reportTrace must be called inside an evalite eval");
  _reportTrace(trace);
};

packages/evalite/src/ai-sdk.ts:82

export const traceAISDKModel = (model: LanguageModelV2): LanguageModelV2 => {
  if (!shouldReportTrace()) return model;     // <-- prod: returns the raw model unwrapped
  return wrapLanguageModel({ model, middleware: { wrapGenerate: ..., wrapStream: ... } });
};

The env var is set only by the runner, never by the app: packages/evalite/src/run-evalite.ts:233 → process.env.EVALITE_REPORT_TRACES = "true";

How to apply to Mission Control: if MC ships an SDK/middleware users embed in their own agents to emit richer telemetry, gate it the same way — one env var, identity-return when off. Users traceAISDKModel(model) unconditionally in their code; it's inert until MC's runner sets the flag. No if (dev) branches sprinkled through user code.

1.2 Per-concurrent-test trace collection via AsyncLocalStorage

Evalite runs every dataset row as a concurrent Vitest test, yet each row's nested LLM calls must attribute their traces to the right row with no global mutable state. Solved with AsyncLocalStorage:

packages/evalite/src/traces.ts:4

export const reportTraceLocalStorage =
  new AsyncLocalStorage<(trace: Evalite.Trace) => void>();

packages/evalite/src/evalite.ts:320 (inside each it.concurrent):

const traces: Evalite.Trace[] = [];
reportTraceLocalStorage.enterWith((trace) => traces.push(trace));

The middleware calls reportTrace(...) (traces.ts:12), which does reportTraceLocalStorage.getStore() and pushes into the collector bound to this async context. Same trick for a file-write queue: writeFileQueueLocalStorage.enterWith(...) at evalite.ts:309.

How to apply: MC's app server handles concurrent requests. If you add per-request tracing/cost attribution, AsyncLocalStorage gives you request-scoped collection without threading a context object through every function — the canonical Node pattern for exactly this.

1.3 Don't build a runner — hijack Vitest

evalite() is not a bespoke test runner. It's a thin wrapper that registers a Vitest describe with one it.concurrent per row (packages/evalite/src/evalite.ts:185 registerEvalite, :278 it.concurrent). This inherits watch mode, concurrency, reporters, filtering, .only/ .skip, TS/ESM loading — for free.

Notable sub-patterns in evalite.ts:

• Tests created in a loop, not it.each (:277) — deliberately, so closures can capture non-serializable data (Zod schemas, class instances). Comment at :275: "Create individual tests manually to avoid serialization."
• Results shipped to the reporter as Vitest annotations (serializeAnnotation({type: "RESULT_STARTED"|"RESULT_SUBMITTED", ...}), :290/:357). The annotation channel is the transport from test process → reporter → SQLite. A RESULT_STARTED fires immediately so the UI shows "running" before the (slow) LLM call finishes.
• Streaming tasks normalized via executeTask (:50) — if the task returns an async iterable, it drains chunks and concatenates; scoring sees a finished value either way.
• makeSerializable (:27) — structuredClone probe, fall back to a JSON.stringify replacer that stringifies functions/symbols/bigints. Belt-and-suspenders before persisting arbitrary user objects.

How to apply: if MC adds "run these evals against my agent," build on Vitest rather than a custom loop. You get the ecosystem; you write ~400 lines (all of evalite.ts) instead of a runner.

1.4 Scorers are just async functions returning a number (or {score, metadata})

packages/evalite/src/create-scorer.ts:3 — the entire scorer contract:

export const createScorer = (opts) => async (input) => {
  const score = await opts.scorer(input);
  if (typeof score === "object") {
    if (typeof score.score !== "number") throw new Error(`must return a number`);
    return { score: score.score, metadata: score.metadata, description, name };
  }
  if (typeof score !== "number") throw new Error(`must return a number`);
  return { description, name, score };
};

Scorers receive { input, output, expected } and return 0..1. That's it — no framework. Autoeval (LLM-as-judge) scorers are the same shape, they just call a model inside. Scoring is parallelized: Promise.all(opts.scorers.map(...)) at evalite.ts:89.

How to apply: MC's eval layer should adopt this exact minimal contract. A scorer = (ctx) => number. Ship a few built-ins (exact-match, JSON-diff, LLM-judge) and let users write inline functions. Don't invent a scorer DSL.

1.5 AI-SDK middleware for BOTH tracing and caching (same seam)

Evalite and its example both use wrapLanguageModel middleware — one composable seam does tracing, and a separate wrapper does response caching:

packages/example/src/cache-model.ts:28 (deterministic eval reruns without paying for tokens):

export const cacheModel = (model, storage) => wrapLanguageModel({ model, middleware: {
  wrapGenerate: async (opts) => {
    const key = createHash("sha256").update(JSON.stringify(opts.params)).digest("hex");
    const cached = await storage.get(key);
    if (cached && typeof cached === "object") {
      const result = createResultFromCachedObject(cached);
      result.usage.inputTokens = 0;   // zero out so UI SHOWS it was a cache hit
      result.usage.outputTokens = 0;
      result.usage.totalTokens = 0;
      return result;
    }
    const generated = await opts.doGenerate();
    await storage.set(key, JSON.stringify(generated));
    return generated;
  }}});

Key = sha256(JSON.stringify(params)). On hit, tokens are forced to 0 so the cost UI visibly distinguishes cached calls. Timestamps are rehydrated to Date on read (:15).

How to apply: MC can offer a "replay without re-billing" mode for eval reruns using this exact cache. The zero-token-on-hit trick is a nice UX detail worth stealing for MC's cost view.

1.6 SQLite with WAL + idempotent additive migrations

packages/evalite/src/storage/sqlite.ts:8

const db = new Database(url);
db.pragma("journal_mode = WAL");
db.exec(`CREATE TABLE IF NOT EXISTS runs (...); ... traces (...);`);
// then every schema evolution is an additive ALTER wrapped in try/catch:
try { db.exec(`ALTER TABLE evals ADD COLUMN status TEXT NOT NULL DEFAULT 'success';`); } catch {}
try { db.exec(`ALTER TABLE results ADD COLUMN rendered_columns TEXT`); } catch {}
try { db.exec(`ALTER TABLE traces RENAME COLUMN prompt_tokens TO input_tokens; ...`); } catch {}

Schema: runs → evals → results → scores/traces (star schema; scores and traces are child rows, JSON blobs for input/output/metadata). Migrations are try/catch'd ALTERs — no migration framework, no version table; re-running is a no-op because ADD COLUMN throws on the second run and is swallowed. WAL mode = concurrent reader (the UI server) + writer (the reporter) without locking.

How to apply: MC almost certainly wants a local SQLite of parsed usage. WAL is mandatory if the UI reads while a scan writes. The try/catch-ALTER migration style is crude but zero-dependency and perfectly fine for a single-file local DB.

2. ccusage (Rust) — how it parses thousands of session files fast

Entry: rust/crates/ccusage/src/adapter/claude/mod.rs and .../claude/daily.rs. ccusage does a full re-parse every run (no persistent cross-run cache — see §4), but each run is fast because of the pipeline below. For MC, combine this pipeline with CodexBar's incremental cache (§3) to get both.

2.1 The four-stage read pipeline (the core steal)

Documented in rust/crates/ccusage/src/adapter/jsonl.rs:1:

1. Read the whole file once, split into byte slices with byte_lines — no String alloc per line.
2. Skip lines that cannot match using a precompiled memmem substring prefilter, before any JSON parsing.
3. Deserialize survivors directly into a typed struct with serde_json::from_slice — unused fields skipped, no intermediate Value tree.

Stage 1+2 — read whole file, byte-split, SIMD prefilter (claude/mod.rs:346):

let Ok(content) = fs::read(path) else { return loaded_file; };
let usage_marker = memmem::Finder::new(br#""usage":{"#);   // built ONCE per file
for line in byte_lines(&content) {
    if usage_marker.find(line).is_none() { continue; }     // SIMD memmem, skip non-usage lines
    if has_unsupported_null_field(line) { continue; }      // byte-level null-field reject
    let Ok(data) = serde_json::from_slice::<UsageEntry>(line) else { continue; };
    ...
}

byte_lines is a zero-alloc iterator using memchr for newline splitting (rust/crates/ccusage/src/fast.rs:83). The LinePrefilter (fast.rs:24) wraps reusable memmem::Finder needles so the "is this line worth parsing?" check runs on the SIMD path instead of str::contains. Prefilter can require ALL markers or ANY (fast.rs:39/:44).

Stage 3 — typed lenient deserialize. Instead of parsing to serde_json::Value and hand-walking with .get(), it deserializes straight into structs. The trick that makes this safe against messy real-world logs is a set of lenient deserializers (jsonl.rs:75) used as #[serde(default, deserialize_with = "...")]:

• lenient_u64 — floats/strings/negatives/null → 0 (matches JS Value::as_u64)
• lenient_i64 / lenient_f64 — bad type → None, doesn't fail the record
• lenient_object — non-object → None instead of erroring the whole line
• lenient_array / lenient_vec — skip bad elements, keep good ones
• non_empty_string — trim, empty→None

So a single malformed token field degrades to 0 instead of dropping the entire usage record.

How to apply to MC (this is the perf fix): MC's Node/Bun parser almost certainly does JSON.parse(line) on every line of every file. Replace with:

1. fs.readFileSync (or a streaming read) → get a Buffer, not a string.
2. Before JSON.parse, do buffer.includes(Buffer.from('"usage":{')) on the raw bytes to skip lines that can't contain usage (Claude logs are ~50%+ non-usage lines: user turns, tool results, summaries). Buffer.indexOf is native/fast. This alone can halve parse work.
3. Only JSON.parse the survivors. In Node there's no cheap typed-deserialize equivalent, but skipping the non-usage lines is the big win.

2.2 Size-balanced parallel file reading

rust/crates/ccusage/src/adapter/claude/mod.rs:136 chunk_file_indexes_by_size — greedy longest-processing-time bin-packing: sort files by byte size desc, assign each to the currently smallest-total worker. Then thread::scope spawns one worker per core (available_parallelism().min(files.len()), :172), each parses its chunk, results are reassembled in original order via an index-keyed Vec<Option<_>> (:200).

let worker_count = thread::available_parallelism().map(usize::from).unwrap_or(1).min(files.len());
let chunks = chunk_file_indexes_by_size(files, worker_count);   // balanced by total bytes
thread::scope(|scope| { for chunk in chunks { scope.spawn(move || { ...parse... }); } ... })

Balancing by bytes, not file count matters: one 200MB session file among a thousand 5KB files would otherwise pin one core while the rest idle.

How to apply: MC should fan file parsing across Worker threads (Node worker_threads / Bun workers), chunked by summed file size, not count. os.availableParallelism() exists in Node 20+.

2.3 File discovery + project/session extraction

rust/crates/ccusage/src/adapter/claude/paths.rs:

• claude_paths() (:13) — resolves data dirs in priority order: CLAUDE_CONFIG_DIR (comma-sep, ~ expansion, accepts either the config dir or the projects/ dir itself), else $XDG_CONFIG_HOME/claude then ~/.claude, deduped, must contain projects/.
• collect_files_with_extension (:97) — plain recursive read_dir walk collecting *.jsonl.
• usage_files (:69) sorts results by path for stable ordering; supports a single-segment project_filter that scopes the walk to projects/<filter> (is_project_path_segment at :85 rejects ./..///\ to prevent traversal).
• extract_session_parts (:150) — handles modern flat projects/<proj>/<session>.jsonl, nested .../<session>/chat.jsonl, and .../<session>/subagents/<worker>.jsonl layouts.

How to apply: MC must handle CLAUDE_CONFIG_DIR (users relocate .claude) and the subagents/nested session layouts, or it silently under-counts. These are load-bearing edge cases.

2.4 Deduplication by (messageId, requestId)

Claude logs replay the same assistant message across files (resumes, sidechains//btw). ccusage dedupes on a hash of (message.id, requestId): rust/crates/ccusage/src/adapter/claude/mod.rs:292

fn usage_dedupe_hash(message_id: &str, request_id: Option<&str>) -> u64 {
    let mut hasher = FxHasher::default();
    message_id.hash(&mut hasher);
    request_id.hash(&mut hasher);
    hasher.finish()
}

push_deduped_entry (:239) keeps an FxHashMap<u64, SmallVec<[usize;1]>> of seen hashes. Collisions are resolved by re-checking the actual ids. There's a second, looser key on message_id alone to catch sidechain replays that reuse a message with a new request id (:254), preferring the parent (non-sidechain) / higher-token / higher-cost entry (should_replace_deduped_entry, :223). Extensively tested (mod.rs:657, daily.rs:519).

How to apply: If MC sums tokens/cost naively it double-counts on session resume and sidechains. Dedup on (message.id, requestId) is mandatory for correct totals — this is a correctness bug hiding as a data-modeling choice.

2.5 Cost model — embedded pricing + tiered/cache-aware calc

rust/crates/ccusage/src/cost.rs:

• Three modes (:23): Display (trust logged costUSD), Auto (logged cost else compute), Calculate (always compute from tokens).
• calculate_cost_from_tokens (:84) sums input/output/cache-create/cache-read, each via tiered_cost (:137) which applies an above-200k-token tier when present. Cache-creation is split by TTL: 5-minute at base rate, 1-hour at 2× input rate (CACHE_CREATE_1H_INPUT_MULTIPLIER = 2.0, :7), read from the cache_creation.ephemeral_5m/1h breakdown when the log provides it, else flat.

Pricing is embedded in the binary at build time and refreshed from network opportunistically: rust/crates/ccusage/src/pricing.rs:15

const BUILD_TIME_LITELLM_JSON: &str = include_str!(concat!(env!("OUT_DIR"), "/litellm-pricing.json"));
const BUILD_TIME_MODELS_DEV_JSON: &str = include_str!("models-dev-pricing.json");

load_with_overrides(offline, ...) (:213) starts from the embedded snapshot, then (unless --offline) fetches LiteLLM's live pricing over ureq and falls back to embedded on any parse/ fetch failure (:229/:235). So it works fully offline and never blocks on the network.

How to apply: MC should embed a pricing snapshot (LiteLLM's model_prices_and_context_window.json is the de-facto source both ccusage and CodexBar use) so cost works offline and on day one, with an optional background refresh. Don't forget the 200k tier and the 1h-cache 2× multiplier or cost will be wrong for heavy-cache Claude sessions.

3. CodexBar (Swift) — incremental scanning + cheap live polling

This is the repo most aligned with MC's problem: a long-lived app that must re-read a growing local corpus repeatedly without melting the CPU. Its cost-usage engine lives in Sources/CodexBarCore/Vendored/CostUsage/.

3.1 ★ The incremental append-only JSONL cache (top steal for MC)

Claude/Codex session files are append-only. CodexBar exploits this to never re-parse bytes it has already seen.

Cache shape — CostUsageCache.swift:106. Per file it stores mtime, size, the byte offset it parsed up to, and the already-aggregated result:

struct CostUsageFileUsage: Codable {
    var mtimeUnixMs: Int64
    var size: Int64
    var days: [String: [String: [Int]]]   // dayKey -> model -> packed usage
    var parsedBytes: Int64?                // <-- how far into the file we've parsed
    var claudeRows: [ClaudeUsageRow]?      // cached parsed rows for incremental merge
    ...
}
struct CostUsageCache: Codable {
    var version: Int = 1
    var producerKey: String?               // parser-hash guard (see below)
    var files: [String: CostUsageFileUsage] = [:]   // filePath -> usage
    var days: [String: [String: [Int]]] = [:]       // pre-aggregated grand totals
    var roots: [String: Int64]?
}

The three-way decision per file — CostUsageScanner+Claude.swift:535 processClaudeFile:

// 1. Unchanged (same mtime AND size) -> skip entirely, contribute cached aggregate.
if let cached = state.cache.files[path],
   cached.mtimeUnixMs == mtimeMs, cached.size == size, !state.forceFullScan {
    return
}
// 2. Grew (append-only) -> parse ONLY the new tail bytes, merge with cached rows.
if let cached = state.cache.files[path], !state.forceFullScan {
    let startOffset = cached.parsedBytes ?? cached.size
    let canIncremental = size > cached.size && startOffset > 0 && startOffset <= size
        && cached.claudeRows != nil
    if canIncremental {
        let delta = try Self.parseClaudeFileCancellable(fileURL: url, startOffset: startOffset, ...)
        let mergedRows = Self.mergeClaudeRows(existing: cached.claudeRows ?? [], delta: delta.rows)
        state.cache.files[path] = Self.makeClaudeFileUsage(..., parsedBytes: delta.parsedBytes)
        return
    }
}
// 3. Shrank/rewritten/new -> full parse.
let parsed = try Self.parseClaudeFileCancellable(fileURL: url, ...)

The seek-and-read-tail primitive — CostUsageJsonl.swift:27 scan(offset:...):

let handle = try FileHandle(forReadingFrom: fileURL)
if startOffset > 0 { try handle.seek(toOffset: UInt64(startOffset)) }   // <-- skip already-parsed bytes
while true {
    let chunk = try handle.read(upToCount: 256 * 1024) ?? Data()        // 256KB chunks
    ... scan for 0x0A newline manually, emit each Line ...
    // returns startOffset + bytesRead so the caller persists the new parsedBytes
}

Two more memory-bounding tricks in the same scanner:

• prefixBytes / maxLineBytes (:51 appendSegment): it only keeps the first prefixBytes of each line (enough to hold the usage JSON) and flags the rest wasTruncated — a 10MB pasted- file line never materializes in full.
• autoreleasepool per 256KB chunk (:76) bounds peak memory during a big scan.

Cache invalidation by parser version — CostUsageCache.swift:9/:97: the cache file path embeds an artifactVersion per provider (codex: 8, claude/vertexai: 4) and the payload carries a producerKey ("codex:cu:p<parserHash>"). On load (:53 loadCache) a mismatched producerKey or version is discarded. So when you change parsing logic, you bump the version and every stale cache is ignored automatically — no manual cache-clear, no silently-wrong reused aggregates.

Atomic cache write — CostUsageCache.swift:70 save: encode → write to .tmp-<uuid>.json with .atomic → replaceItemAt. A crash mid-write never corrupts the cache.

The directory-walk gotcha (documented, learned the hard way) — CostUsageScanner+Claude.swift:613:

Always enumerate the directory tree. The per-file mtime/size cache already skips unchanged files, so the only cost is the walk itself. The previous root-mtime optimization skipped enumeration when the root dir mtime was unchanged, but on POSIX a directory mtime only updates for direct child changes — not for files modified inside subdirectories. This caused new session logs to go undetected until the cache was manually cleared.

How to apply to MC — this is the render-storm cure:

1. Persist a JSON/SQLite cache keyed by absolute file path → { mtimeMs, size, parsedBytes, aggregate }.
2. On each scan: stat every file (cheap). If mtime and size match cache → skip, reuse aggregate. If size grew → open, seek(parsedBytes), parse only the tail, merge. Else full parse.
3. Always walk the dir tree (don't trust dir mtime); the per-file skip is what saves you.
4. Embed a parserVersion in the cache; bump it to invalidate on logic changes.
5. Atomic-write the cache (tmp + rename).

For a user with thousands of sessions where only the active one is being appended, this turns an O(total corpus bytes) rescan into O(bytes appended since last scan) — typically kilobytes.

3.2 Scans pinned to one dedicated off-pool thread (don't starve the UI)

Sources/CodexBarCore/CostUsageScanExecutor.swift:9 — header comment states the exact failure mode MC is at risk of:

Cost-usage scans read and parse the full local session corpus synchronously and can run for minutes on large archives. Executing that inline on Swift's cooperative thread pool starves every other async task — menus freeze while the main thread sits idle — and overlapping provider scans multiply pool pressure and disk load. This executor pins all corpus scans to a single serial utility queue off the cooperative pool.

private static let queue = DispatchQueue(label: queueLabel, qos: .utility)  // serial, low priority

It also bridges task cancellation into a cooperative checkCancellation callback threaded through the whole scanner (CostUsageJsonl.scan(checkCancellation:), parseClaudeFileCancellable, etc.), so a superseded scan stops promptly instead of running to completion. Work still queued when cancelled resumes immediately with CancellationError rather than waiting behind an in-flight scan (RunState.install, :27).

How to apply: MC should run corpus scans in a worker thread, never on the request/render path, and make them cancellable so a newer scan (or a closed window) aborts the old one. The "serial queue, one scan at a time, cancel-and-supersede" model prevents overlapping scans from compounding disk I/O.

3.3 Adaptive refresh cadence (cheap when idle, live when watched)

Sources/CodexBar/AdaptiveRefreshPolicy.swift:6 — a pure function: (now, lastMenuOpenAt, lowPowerMode, thermalState) → delay. No clock reads inside, so it's trivially testable.

recentInteraction (<5min since menu opened) -> 2 min
warm             (<1h)                       -> 5 min
idle             (<4h)                       -> 15 min
longIdle         (>=4h / never opened)       -> 30 min
constrained      (low-power OR thermal>=serious) -> 30 min   // takes priority

Refresh frequency scales with recency of user attention; low-power/thermal pressure forces the slow cadence regardless.

How to apply: MC's live polling should back off when the window is unfocused/hidden and speed up when the user is actively looking. Modeling the cadence decision as a pure (signals) → interval function keeps it testable and tunable. (Browser analog: document.visibilityState + last-focus timestamp instead of menu-open.)

4. Efficient JSONL-corpus parsing — the combined playbook for Mission Control

Neither tool alone is the full answer; the ideal MC parser is CodexBar's incremental cache wrapping ccusage's per-file pipeline:

Concrete MC parser sketch (Node/Bun):

scan():
  files = walkTree(claudePaths())              # always walk (dir mtime lies — §3.1)
  chunks = binPackBySize(files, numCPUs)       # §2.2
  results = await Promise.all(chunks.map(c => worker.parse(c, cache)))  # §3.2 off-thread
  persistCacheAtomic(mergedCache)              # §3.1 tmp+rename

worker.parse(file, cache):
  st = stat(file)
  hit = cache[file.path]
  if hit && hit.mtime==st.mtime && hit.size==st.size: return hit.aggregate   # skip
  start = (hit && st.size>hit.size) ? hit.parsedBytes : 0                     # tail vs full
  fd.seek(start); for line in readLines(fd):
     if !line.includes('"usage":{'): continue                                # prefilter §2.1
     e = JSON.parse(line); if dupe(e.message.id, e.requestId): continue      # dedup §2.4
     aggregate += cost(e, pricing)                                           # §2.5
  cache[file.path] = { mtime, size, parsedBytes:end, aggregate: merge(hit,delta) }

5. Top ~6 highest-ticket steals (ranked)

1. Incremental append-only cache — CodexBar/.../CostUsageCache.swift + CostUsageScanner+Claude.swift:535 + CostUsageJsonl.swift:41. Store per-file {mtime,size,parsedBytes,aggregate}; skip unchanged files, seek() and parse only appended tail bytes, version-guard the cache. This directly fixes MC's render-storm/rescan perf. O(corpus) → O(bytes appended).

2. Prefilter-before-parse — ccusage/.../fast.rs + adapter/claude/mod.rs:350. Byte-substring test ("usage":{) on the raw line to skip the ~half of Claude-log lines that can't contain usage, before paying for JSON.parse. In JS: Buffer.indexOf. Cheapest big win, pairs with #1.

3. Scans on a dedicated cancellable off-render thread — CodexBar/.../CostUsageScanExecutor.swift. Header comment is a warning label for exactly MC's UI-freeze risk. Serial utility queue, cancel-and-supersede, cooperative checkCancellation threaded through the parser.

4. Prod no-op eval/trace SDK via env-gate + identity wrapper — evalite/.../traces.ts:8 + ai-sdk.ts:82. If MC ships embeddable telemetry middleware, make it inert unless the runner sets one env var. Plus AsyncLocalStorage for per-request/-test trace attribution (traces.ts:4, evalite.ts:320).

5. (messageId, requestId) dedup — ccusage/.../claude/mod.rs:292. Without it MC double-counts tokens/cost on session resume and sidechains. Correctness, not just perf.

6. Embedded pricing snapshot with offline fallback + correct cost model — ccusage/.../pricing.rs:15 + cost.rs. Bake in the LiteLLM pricing JSON so cost works offline/ day-one; background-refresh optional. Don't miss the 200k tier (cost.rs:137) and the 1-hour cache 2× multiplier (cost.rs:7).

Runner-ups: build evals on Vitest, not a custom runner (evalite/.../evalite.ts); **SQLite WAL

• try/catch-ALTER migrations** (storage/sqlite.ts:8); size-balanced parallel file chunking (ccusage/.../mod.rs:136); adaptive refresh cadence as a pure function (CodexBar/.../AdaptiveRefreshPolicy.swift).

Appendix — key file map

• evalite: packages/evalite/src/{traces.ts, ai-sdk.ts, evalite.ts, create-scorer.ts, run-evalite.ts, storage/sqlite.ts}, packages/example/src/cache-model.ts
• ccusage (Rust): rust/crates/ccusage/src/{fast.rs, cost.rs, pricing.rs, adapter/jsonl.rs, adapter/claude/{mod.rs, daily.rs, paths.rs}}; npm launcher apps/ccusage/src/cli.js
• CodexBar: Sources/CodexBarCore/Vendored/CostUsage/{CostUsageCache.swift, CostUsageJsonl.swift, CostUsageScanner+Claude.swift, CostUsageScanner+CacheHelpers.swift, ModelsDevPricing.swift}, Sources/CodexBarCore/CostUsageScanExecutor.swift, Sources/CodexBar/AdaptiveRefreshPolicy.swift