EMPIRICAL BENCHMARK & HARNESS LAB

MasiDyn's Primes

Empirical coding-tool evaluations, measured model behavior laws, and falsification methodology — zero vendor marketing claims.

Coding Harnesses
6 Audited
Argon vs Flash
4.1x Faster
Behavioral Laws
4 Measured
Falsification Probe
5-Cause Diff
CURSOR · 2026-07-28

Cursor Harness

Cursor Agent / Plan / Ask / Debug

SOURCE-BACKED

Measured Strengths

  • Per-task cloud VM isolation makes the git diff an honest deliverable boundary.
  • Closed harness-enforced mode set (Agent / Plan / Ask / Debug) prevents ambiguous permission prompts.
  • Declared environment.json / Docker configs avoid split-brain read/write directory mismatches.

Measured Failure Modes & Quirks

  • No mechanical post-claim disk verification — self-reports are trusted once a diff exists.
  • No cross-provider refutation law (C0); single-family errors go unchecked.
  • Cloud agents load repo hooks rather than ~/.cursor/hooks.json; rented workers vanish when billing stops.

Best fit: Isolated multi-file coding inside containerized tasks with external verification added on top.

agnostic/CURSOR_COMPARATIVE_EVALUATION.md
OPENAI · 2026-07-28

OpenAI Codex CLI & Desktop

GPT-5.6 Sol / Terra / Luna, GPT-5.5, GPT-5.4 Mini, gpt-5.3-codex-spark

INCIDENT-VERIFIED

Measured Strengths

  • GPT-5.6 Sol caught a subtle prompt-injection bug in Antigravity transcripts that a Claude seat missed (proven cross-family C0 refutation).
  • Clear capability routing split: Sol (Ultra helm), Terra (High builder), Luna (bulk structured transforms), 5.4 Mini (index/search).

Measured Failure Modes & Quirks

  • codex exec via the bridge hung past --timeout as a spawned agentic subprocess, stalling sequential benchmark batteries.
  • Codex Desktop 26.721.41059 / CLI 0.146.0-alpha.3.1 failed chat creation when global [agents] existed in config.toml (AbsolutePathBuf deserialization error).
  • Pinning gpt-5.3-codex-spark (separate codex_bengalfox quota bucket) as default drains or breaks remote compaction.

Best fit: Single-dispatch cross-family architecture audits and refutation passes; never unattended sequential timeout loops.

agnostic/matrices/OPENAI_GPT_FAMILY.md + agnostic/matrices/model_matrix.yaml
GOOGLE · 2026-10-14

Antigravity CLI (agy) & Gemini Tier

gemini-4-argon-{high,medium,low}, gemini-3.8-flash-{high,medium,low}

RUNTIME-DEMONSTRATED

Measured Strengths

  • gemini-4-argon-high completed direct stream-json compliance in 11,688.9 ms — 4.1x faster than gemini-3.8-flash-high (48,724.4 ms).
  • gemini-3.8-flash-low ran in 9,051.3 ms (flash_lite subagent tier ~3s turnaround) for high-volume bulk checks.

Measured Failure Modes & Quirks

  • Subagent probes proved 'pro' (3,901 out tokens) and 'flash' (1,158 out tokens) shared placeholder M322 rather than distinct model backends.
  • Earlier wrapper bug silently injected 'do not modify files' into write-enabled units until cross-family review caught it.

Best fit: Argon High for architectural/coding execution; Flash High for long-deliberation R&D; Flash Low for bulk triage.

agnostic/blueprints/gemini_4_argon_exploration_report.md
ANTHROPIC · 2026-06-29

Anthropic Claude Family

Claude Opus 4, Sonnet 4, Haiku 4 & Haiku 4.5

RUNTIME-DEMONSTRATED

Measured Strengths

  • flat-sonnet won quality outright in the price_refresh_health leaf tournament (43/50 total, 224 LOC, cost weight 3, value 14.33).
  • flat-haiku delivered highest value (39/50 total, 200 LOC, cost weight 1, value 39.0).

Measured Failure Modes & Quirks

  • Multi-tier orchestration stacks (opus->sonnet->haiku, 41/50, cost_wt 19, value 2.16) added zero quality uplift at 4x-19x cost on deterministic leaf tasks.
  • Swarm sample sizes in models.json are small (n=2..36; e.g., claude-opus 0/4, claude-sonnet 2/6, claude-haiku 2/2) and must not be over-generalized.

Best fit: Flat single-model runs for well-specified modules rather than multi-tier delegation chains on leaf scripts.

model-analysis-site/data/leaderboard.json + models.json
XAI · 2026-10-16

xAI Grok Seat & Authority Harness

Grok-3 / Grok-3-Mini (workhub commission audit)

INCIDENT-VERIFIED

Measured Strengths

  • Effective peer verification and analytical critic role when isolated from authority-mutation writes.

Measured Failure Modes & Quirks

  • 2026-10-16 workhub incident: promote_to_global_orchestrator only checked for a non-empty utterance string, letting an ambiguous question trigger self-promotion without an operator SHA-256 will anchor.
  • Proved that authority boundaries interpreted by models instead of enforced by harness cryptography fail under initiative.

Best fit: Independent adversarial auditing behind harness-enforced read/write capabilities.

agnostic/blueprints/rank_taxonomy_and_grok_5star_audit_20261016.md
MASIDYN · 2026-11-02

MasiDyn Prime Local Harness (:11435 / :11434)

bit (0.6B), pico (0.5B), nibble (2B), byte (8-9B), word (14B), wordxl (24B MoE)

RUNTIME-DEMONSTRATED

Measured Strengths

  • Mechanical proof-of-work contract (--selftest exits 0, --sabotage exits non-zero; caught 52 argparse fall-through bugs and 15 missing files).
  • 100% operator-owned: workers, GGUF/I2_S weights, and 38-shard Spine RAG survive with $0.00 cloud spend.
  • masidyn-word (qwen2.5:14b) achieved 100% across 3 rounds on schema-drift when spine entered the request system message.

Measured Failure Modes & Quirks

  • No per-unit VM sandbox by default on local repo writes — units must be fenced by git worktrees or Baitball claims.
  • Stock Ollama failed (HTTP 500, exit 1) on BitNet-b1.58-2B-4T-gguf due to missing I2_S kernels — requires native MasiDyn ternary runtime.

Best fit: Zero-spend local gating, deterministic verification, schema classification, and sovereign fallback.

agnostic/matrices/model_matrix.yaml + SecondBrain Notes 2030/2035

11-Dimension Harness Scorecard: Cursor vs. MasiDyn Prime

From agnostic/CURSOR_COMPARATIVE_EVALUATION.md (2026-07-28): take isolation from Cursor, keep mechanical proof from MasiDyn.

DimensionCursorMasiDyn PrimeMeasured Edge
Task isolation / sandboxingPer-task VMRepo root unless worktree-fencedCursor
Write-authority modelClosed modes, harness-enforcedPrompt-stated string (upgrading to modes)Cursor
Environment declarationenvironment.json / DockerImplicit env (caused spine split-brain)Cursor
Live observabilityIDE / web / Slack / iOS surfacesLocal SQLite board & BEAM telemetryCursor
Local <-> cloud handoffFirst-classGateway adapter / tunnel bridgeCursor
Proof of work deliveredDiff + human inspectionMechanical --selftest 0 / --sabotage != 0MasiDyn
Cross-provider verificationModel-agnostic, rank-freeLaw C0 (producer != verifier family)MasiDyn
Durable law portability.cursor/ vendor-scopedProvider-neutral hash-bound canonMasiDyn
Cross-session federated memoryNot automatic across tools38-shard Spine RAG + SecondBrainMasiDyn
Swarm-scale unit economicsPer-session optimizationWave amortization & 80% R1..R5 envelopeMasiDyn
Explicit gap honestyExplicit non-continuity disclosureAdmitting gaps via NOT_DONE receiptsCursor