Harness engineering: beyond vibe coding.

Imagine a coding agent tasked with fixing an expensive query analyzer. It makes a patch, runs the test script, and confidently reports success. The wrapper script exits with status zero even when a test command fails. The agent has optimized for a signal that never goes red.

This is a constructed example, not a reported incident. But the failure class is real: testing infrastructure, evaluators, and agent tool wrappers are software too. Before trusting them, deliberately introduce a failing assertion and confirm that the entire CI job fails.

A good prompt cannot compensate for a false-positive verifier. A different model reviewing the same false result is not an independent check.

An effective workflow may use one agent to propose acceptance tests, another to implement the behavior, and a third to review the diff. The separation helps only if the reviewer sees independent evidence and the implementer cannot rewrite the definition of success.

First ask the test author to demonstrate that new tests detect missing or incorrect behavior. Commit approved criteria separately. Let the implementation agent work in an isolated branch. Run tests plus previously withheld checks. Then review not only whether the code passes, but whether it changes any permissions, dependencies, error handling, or surrounding interfaces.

Strict red-green-refactor loops remain valuable for small deterministic functions. For broader infrastructure tasks, high-level acceptance tests and contract tests may precede the shorter unit-test cycles. Tests are evidence conditioned on the adequacy of their specification—not a proof of general correctness.

Consider a Python tool that recommends PostgreSQL index candidates from captured plans. The input files are read-only. The expected output is a report—not SQL executed against a live database.

Run the same scenarios after each model or prompt change. Compare outcomes, false positives, tool-call costs, and failure explanations. The goal is a reproducible evaluation record—not a persuasive agent demo.

A low-risk documentation update may need linting, link checks, and human review. A database migration needs dependency mapping, rollback or roll-forward procedures, isolated rehearsals, and production change authority. Applying the same expensive multi-agent process to every typo would make the workflow less useful.

Independent checks are imperfect: two agents can share training-derived blind spots, test fixtures can omit important production states, and automated policy gates can themselves be misconfigured. Higher-risk operations require stronger controls and real-world observation beyond a successful CI run.

1. An unambiguous statement of intended and forbidden behavior.
2. A reviewable, minimally scoped implementation diff.
3. Reproducible verification with evidence the tests can fail.
4. Clear permission boundaries and a record of tool actions.
5. A bounded deployment decision, with a human accountable for approval.

The human engineer still needs to understand the system well enough to reject a flawed specification, recognize architectural damage, and challenge contradictory results.

OpenAI — Harness Engineering: Leveraging Codex in an Agent-First World ↗

Anthropic — Claude Code Best Practices ↗

Anthropic — Demystifying Evals for AI Agents ↗

OWASP — Excessive Agency ↗

These sources describe evolving techniques; the six-boundary framework and example experiments above are DBRE.tech editorial synthesis, not a vendor-certified standard.