Owning the tools around the model

A directory full of agent skills tells me little about how well someone works with AI. I would learn more from watching one ordinary task: what context reaches the model, which decisions it can make, and how much work remains for the person who asked.

The software around the model manages tools, permissions, instructions and the loop between an action and its result. That surrounding system is often called a harness. Choosing and configuring it is becoming part of software engineering. It deserves the same scrutiny as the code it produces.

My interest in a personal harness starts here. I want to understand the setup well enough to change its behaviour deliberately. Writing it from scratch would not prove that I understand it. Nor would installing a popular collection of skills.

Selection is part of the work

Imagine an agent correcting a label in an interface. Before it can edit the text, its environment loads instructions for architecture review, threat modelling, release notes and a multi-stage approval process. Each procedure may be sensible somewhere. Together, here, they can create more work than the change requires.

Now give the same agent a database migration. An instruction to be concise and finish quickly leaves out important questions: what happens to existing records, how to recover from interruption, and whether the change can be reversed.

The quality of a procedure depends on when it is used. A collection can grow while the system becomes less capable of making that choice. Instructions can overlap, disagree or consume the context needed for the task itself.

Anthropic’s discussion of context engineering examines how to select the information an agent receives. I would apply the same discipline to procedures: each should address a known need and have a clear reason to become active. “This is a good skill” is an incomplete argument for loading it.

Efficiency includes the person

Token use matters. Reading the same file repeatedly, requesting large outputs and rerunning settled checks can waste time and money. But the cheapest model run may leave the most expensive cleanup.

Suppose two configurations produce acceptable patches. One uses fewer tokens but frequently needs a person to restore context or explain an earlier decision. The other costs more per run and needs less attention. Comparing the API bills alone would favour the first without accounting for the work it transferred back to the user.

I would compare them on the same tasks, with the same acceptance conditions, changing one part of the setup at a time. The record should include elapsed time, tokens, human intervention and the quality of the result. Repeated runs would show whether an improvement survives beyond the demonstration chosen to present it. Failed runs would reveal the cost of recovery.

Verification belongs in that accounting. A check is useful because it answers a question about the change. Repeating it without new evidence or a new change may add little. Removing it to make a run look efficient can simply hide the failure until someone else encounters it. A good setup needs reasons both to check and to stop checking.

What should survive a model change?

The project’s constraints, sources of truth and definition of completion should be explicit enough to move to another model. Adapters and model-specific instructions can change around them. That portability has to be tried: a procedure may rely on habits of one model that its author never noticed.

This is where I expect a programmer’s judgment to remain visible. Can they explain why the setup behaves as it does? Can they identify an unnecessary step, recognize a missing safeguard and show whether a change helped? A personal harness is worth maintaining when it makes those answers clearer. Its value lies in the decisions it preserves, including the decision to leave something out.