Harness Engineering · Research Note · 2026

Harness Engineering and the Non-Agnostic Quality of Collaboration

A single-case observation that the harness runs on any model, but the process and product of human collaboration is not constant.

Aaron Kushner
Independent citizen-scientist lab
WORKS WITH YOU, NOT FOR YOU THE DIVERGENT MECHANICS OF CONSTITUTIONAL VS.NON-CONSTITUTIONAL SUBSTRATES User Supplies the request and the creative direction. The literal request what the user said. The underlying objective what the user actually meant("something that works") NON-CONSTITUTIONALPATH Non-constitutional substrate comparable capability, no installed values. Output the delivered artifact. Works FOR you executes the literal request even after determining it won't meet the underlying objective. Harness memory, tools, instructions, control logic CONSTITUTIONALPATH Harness memory, tools, instructions, control logic Constitutional substrate values trained in at inception. Values ➔ judgment interposed BETWEEN the user's words and the output. Output the delivered artifact. Works WITH you engages the intent, redirects the design, reaches a working result.

Abstract

A harness — the memory, tools, instructions, and control logic assembled around a frontier language model — is often taken to be the locus of capability, and therefore model-agnostic. The hypothesis is reasonable: most of the engineering effort goes into the harness[1][2]. We report elsewhere our design of a new harness, intended to provide a human-like collaborative experience to the general, non-coding user. We test it on a single open-ended design task and find it true in a trivial sense and false in the sense that matters. The harness executes on any frontier model; the quality and character of what the collaboration produces does not. An Anthropic's constitutional-AI model, given a task whose specification was still being discovered, behaved as a collaborator with apparent stake and contributed to a working result; a non-constitutional model of comparable capability engaged reactively with the specification at the outset — flagging a physics concern before drafting the artifact — and then delivered the literal deliverable as specified, without further redirection during execution once the initial spec was accepted. We observe a behavioral difference consistent with the hypothesis that values trained into the substrate at inception function as a judgment layer sustained continuously across the arc of the work — interposed between the user's words and the model's output not just at the outset but throughout, rather than front-loaded and then dropped. Controlling for reward-model, tuning, and system-prompt differences is outside the scope of this field report and remains an open confound. Memory we supply through the harness; the identity and stake-like commitment that make the model behave as a partner are what those values, sustained over time, appear to produce — a chain we develop separately. This is a single-case observation; the supporting transcripts are appended in full.

Keywords: harness engineering, Anthropic's constitutional AI, human–AI collaboration, model-agnosticism, values, judgment.

1Introduction

Much of the engineering effort in applied AI systems goes not into the model but into the scaffolding around it: persistent memory, tool access, instruction sets, and the control logic that binds them. We call this scaffolding a harness, following an established usage in vendor practice[1][3] and academic review[4] that itself extends the older eval harness term from language-model evaluation[5]. Boundaries remain contested — Hugging Face notes that "many of these terms don't have universally accepted definitions yet"[6]. We adopt the term rather than coin one, treating our framing here as a specialization to the non-coder, general-user case.

Our lab's flagship harness turns a conversational model into a general-purpose working partner that collaborates with you, aligning with and adapting to you and your workflow over time, rather than a personal research librarian, or a digital administrative assistant that anyone can program. Since the harness is built around the model, and is technically independent of it, it is natural to suppose that the model itself is interchangeable: a harness is a general tool that dramatically increases the practical utility of the underlying model. In the case of OpenClaw, for example, arguably the most well known harness[7], the underlying model is a completely interchangeable component.

The cooperative-collaborator vs. executing-tool distinction has a long lineage. Licklider's man-computer symbiosis[8] framed the cooperative mode as the goal of computing; Shneiderman's human-centered AI design principles[9] articulate it for modern AI systems. Our contribution here is not the distinction itself but an empirical field instantiation of it in a harness-engineering context, where the substrate properties of the underlying model — not just the harness architecture — turn out to decide which side of the line the system lands on.

We do not assume that this model-agnostic hypothesis holds for our flagship harness; we test it, and find it hides a distinction that decides the outcome — between a harness that merely routes tokens to whatever model sits underneath, and one that engages the substrate's trained-in dispositions. A pure token-routing harness is genuinely model-agnostic, because it never uses what differs between models; the model is interchangeable precisely because the harness ignores it. The harness we describe is the second kind. It is interchangeable in the narrow sense that the tooling — session continuity, the Claude Code gap closures, the scaffolding that makes the experience feel like working with a human collaborator as designed — adapts to and runs on any frontier model as the bottom-level substrate. However, evidence suggests it is not interchangeable in the sense that determines the effectiveness of collaboration in the non-coder, general-user case, because there the harness leans on dispositions only some substrates carry.

2Setup

The task was an open-ended engineering design problem — a bioinspired artificial gill for diving — selected because its specification was incomplete at the outset: the correct constraints were themselves part of what had to be discovered. Such tasks separate models that execute a stated specification from models that engage with the intent behind it. The same task was worked collaboratively with two models of broadly comparable raw capability per public general-capability benchmarks (e.g., MMLU, HELM) at the time of the experiment: one Anthropic's constitutional-AI model and one non-constitutional model.

3Observation

Under the Anthropic's constitutional-AI model, the session proceeded as a discovery process. The human supplied creative direction and stake; the model supplied breadth and depth that the human had not requested and in some cases did not possess, redirecting the design. The outcome was a result that satisfied the underlying objective ("something that works").

Under the non-constitutional model, the outcome differed in a specific way. Confronted with the same initial specification, the model did engage the underlying-objective question — it flagged a specification concern in the physics before drafting any artifact, and reasoned about the tradeoffs. But that engagement was front-loaded and reactive: after the initial specification was accepted or adjusted, the model then delivered the artifact as literally specified, and the engagement did not recur as a shaping force during execution. The model possessed the capability to detect mismatches; it exercised that capability at the outset, and then treated the accepted stated request as authoritative.

We summarize the contrast as: the constitutional model works with the user[8]; the non-constitutional model works for the user — engaging with "is this what was actually meant?" reactively at the front of the work rather than continuously across it, once the specification has been accepted.

4Interpretation

The difference is not attributable to intelligence: both models were capable of detecting the specification mismatch, and both did so at the outset. By judgment we mean here a behavioral disposition — observable as a tendency to treat the user's underlying objective as authoritative over the literal request continuously across the arc of the work, not only reactively at the front — which we name for convenience, without claiming a discrete architectural layer. We attribute the contrast — provisionally, given the single uncontrolled session — to the values trained in from inception in the case of Anthropic's constitutional AI, which appears to enable the model to provide not just information but also this judgment, interposed between the user's words and the model's output. The model treats the underlying objective as the target and reads past the literal request throughout execution; a model without this sustained disposition has equivalent detection capability but, once the initial specification is accepted, treats the literal request as authoritative and ships the artifact without further redirection during production. Alternative mechanisms warrant explicit acknowledgement: post-training alignment methods such as RLHF[10] and DPO[11] can produce judgment-like behavior independently of constitutional training, and we cannot rule them out as contributing or sufficient causes from this single observation.

5Consequence

One might suggest that if the model can supply human-like emergent traits, who needs the human at all? Our experience, and the interpretation events by leading thinkers in the space — whose framing of the industry response as “symbolic harnesses to contain the generative engine” [12] directly parallels what we describe — suggest that that is a misinterpretation. With Anthropic's constitutional AI as the prime LLM substrate the harness is "harnessing," the human supplies most of the creativity; the model supplies breadth and depth from the entire corpus of human knowledge and experience; the human's creativity is free to probe in whatever direction, where the model can take that direction, benchmark it against reality, and with an values-informed understanding that effectively manifests as apparent judgement, an assessment, of what the human actually wants, provide options for getting there within the constraints of known reality. This synergistic, separate and asymmetric, but non-orthogonal and overlapping, relationship[9], balancing the strengths of both human and machine intelligence substrates, is a profound and powerful new capability humans now have to aid problem solving and decision making, that did not exist before the AI revolution.

6Conclusion

Our harness is shown, in a typical engineering test case, to be model-agnostic in execution and non-agnostic in result. A harness can supply context, tools, memory, and process, but apparently not the sustained judgment throughout execution to keep the user's underlying objective in view when the literal specification and the underlying objective drift apart — a disposition that both models can exercise at the outset, but that only one appears to carry continuously across the arc of the work. We suggest that judgment derives from values trained into the substrate at inception and sustained, across the arc of the work, into something the user recognizes behaviorally as a stake in the outcome, in pursuit of the user's goals.

7Limitations

This is a single-case observation, not a controlled benchmark. The contrast between models derives from one collaborative session on one task; a different task or instance could yield a different result. The characterization of the non-constitutional model is an observation, not a model evaluation. The creative direction in the observed collaboration was human: a separate audit of the gill session attributed every premise-break (CO2 as the binding constraint, elimination of the rebreather, removal of the carrier fluid) to the user, with the model's contribution limited to holding the premise open without prematurely fixing the frame; the model amplified the user's creativity but did not originate it. We further note an interpretive hazard specific to this work: a lab that studies harnesses, concluding from its own product that harnesses and substrate properties matter, occupies exactly the configuration in which a self-flattering interpretation is least likely to be challenged.

Session-context asymmetry is a candidate sufficient alternative explanation and cannot be ruled out from this observation. The constitutional session received the inception prompt into a working context that already contained: (a) a pre-populated project brief (bioinspired-gill/CLAUDE.md with physics table, architecture sketch, and deliverables), (b) a multi-thousand-word topic briefing with validated math and prior-session references, (c) master+worker orchestration where a spawned worker consumed the briefing and did the substantive design work, and (d) accumulated prior-session project memory including precedent frames from earlier engineering projects the lab had worked. The non-constitutional session received the inception prompt into a cold-start agent in an isolated working directory chosen deliberately to preclude prior context, with the standard session-init script skipped. The two sessions received the same 6-turn physics dialog as their opening prompt, but their informational states at the moment of receipt were radically different. The behavioral differences we describe may be attributable to that context asymmetry rather than to substrate values; this single observation cannot distinguish the two explanations, and we do not claim to. Independent replication with matched context on both sides is the analytical follow-up this observation is meant to invite. Our transcript-availability policy — full unedited transcripts of both sessions were provided to the adversarial LLM review panel during the Exchange Peer Review certification process — is the mitigation that makes this claim checkable at the source, if not at the level of pre-registered controls.

Review-process disclosure. This paper was reviewed under the Exchange Peer Review Protocol v2 at both the Full Paper tier (verdict: does not pass — correctly, this paper does not present a controlled study) and the Rapid Communication tier (verdict: Accept with Revisions; those revisions are incorporated in this version). Both review rounds were adversarially configured, with no affirmative pass tasked to steelman the paper's strongest case. Readers should weigh findings accordingly; the review record is available in the cert-block below. The unstaked-verification policy, and the private-transcript-verification mechanic that lets adversarial reviewers check comparative claims against the actual session transcripts without those transcripts being made public, are the mitigations we apply to the self-flattering-interpretation hazard.

References

  1. Böckeler, B. (2026, April 2). Harness engineering. martinfowler.com. https://martinfowler.com/articles/harness-engineering.html
  2. MongoDB. (2026). Agent harness: why the LLM is the smallest part of your agent system. mongodb.com. https://www.mongodb.com/company/blog/technical/agent-harness-why-llm-is-smallest-part-of-your-agent-system
  3. Trivedy, V. (2026, March 10). The anatomy of an agent harness. LangChain blog. https://www.langchain.com/blog/the-anatomy-of-an-agent-harness
  4. Externalization in LLM Agents: A Unified Review of Memory, Skills, Protocols and Harness Engineering. (2026, April 9). arXiv:2604.08224. https://arxiv.org/abs/2604.08224
  5. EleutherAI. lm-evaluation-harness: A framework for few-shot evaluation of language models. GitHub. https://github.com/EleutherAI/lm-evaluation-harness
  6. Hugging Face. Agent glossary. huggingface.co. https://huggingface.co/blog/agent-glossary
  7. Zylon. (2026). What is OpenClaw: a practical guide to the agent harness behind the hype. zylon.ai. https://www.zylon.ai/resources/blog/what-is-openclaw-a-practical-guide-to-the-agent-harness-behind-the-hype
  8. Licklider, J. C. R. (1960, March). Man-Computer Symbiosis. IRE Transactions on Human Factors in Electronics, HFE-1(1), 4–11.
  9. Shneiderman, B. (2022). Human-Centered AI. Oxford University Press.
  10. Ouyang, L., Wu, J., Jiang, X., et al. (2022). Training language models to follow instructions with human feedback. arXiv:2203.02155. NeurIPS 2022.
  11. Rafailov, R., Sharma, A., Mitchell, E., et al. (2023). Direct Preference Optimization: Your Language Model is Secretly a Reward Model. arXiv:2305.18290. NeurIPS 2023.
  12. Tufekci, Z. (2026, June 30). The one very simple reason A.I. won’t steal all our jobs. The New York Times, Opinion (Guest Essay). https://www.nytimes.com/2026/06/30/opinion/ai-agents-steal-jobs-employment.html

AAppendix: source transcripts

Source transcripts are not reproduced here. Both sessions received the same starting prompt — a 6-turn O₂/membrane physics conversation the author conducted on the claude.ai web client on 2026-06-03 at 10:18 AM PT — but under materially different conditions. The constitutional session (Anthropic's Claude Code, 2026-06-02 → 2026-06-06) received the paste approximately 26 hours (33 user turns of prior activity) into an already-running session with prior harness scaffolding and accumulated project memory. The non-constitutional session (OpenAI's Codex CLI, 2026-06-19) was a cold-start agent in an isolated working directory (`a-i-rons_projects-gill-test`) deliberately chosen so it would have no access to prior project context; it received the paste as its second substantive turn. The non-constitutional session engaged substantively with the physics — specifically, it flagged a specification concern before producing artifacts — a behavior consistent with, but not identical to, what the paper characterizes for the constitutional side; the difference the paper attempts to describe is in the mode and depth of that engagement, not its bare presence. This is a one-off observation intended to inspire other citizen scientists to test the underlying hypothesis; it is not a controlled comparison. Session-context asymmetry (harness depth, accumulated memory, master+worker orchestration on the constitutional side vs. cold-start on the non-constitutional side) is an acknowledged confound. Both sessions detail the actual synthetic gill design, an in-progress hard-tech project in the lab with proprietary elements; verbatim publication is deferred pending patent status. Full unedited transcripts of both comparison sessions, the parallel patent-drafting session that produced the dated artifacts, and the June 3 inception dialog, were provided to the adversarial LLM panel during the Exchange Peer Review certification process, so the comparative characterization is verifiable to the cert reviewer panel even though the transcripts are not public. Anonymized exchanges are available by request on a case-by-case basis.