Requirement or Constraint: Why Our Agents Stopped Thinking in Design

We recently built a website relaunch using an agentic approach. About 1,000 URLs, the existing content system was to become a pure data store, and an agent was to take over editorial operations. To make clear what the agents should produce, we pre-specified the project in great detail.

The result was very good execution—and effectively no design.

The agents delivered. But at every design decision they deferred to what already existed: the briefings, the functional requirements, the module list, the data fields. Nothing emerged that we hadn’t already written down. The creative process never happened. It had been specified away.

My first reflex was to blame too much context. That diagnosis was too blunt.

A constraint shapes a solution space; a requirement closes a problem space

The mistake wasn’t the amount of context. The mistake was the status we gave that context. We folded the design phase into a roadmap made up of requirements. That turned design itself into a requirement.

There’s a clean piece of evidence for this that predates the current agent debate.

Rahul Mohanani, Paul Ralph, and Ben Shreeve presented a randomized study at ICSE in 2014 with 42 participants. Both groups received content-identical instructions for the same design task. For one group the instructions were labeled “requirements,” for the other “ideas.” Nothing else differed.

The requirements group produced significantly less original designs (Mann-Whitney U = 116.5, p = 0.004).

Only the label. The authors call the effect “requirement fixation.”

A follow-up from the same team, published in 2022 in IEEE Transactions on Software Engineering, observed 42 designers working in 21 pairs. Those who received template-style requirements accepted their own first solution uncritically and never returned to it. The authors’ recommendation is explicit: when creativity is the priority, avoid requirement templates and the over-structuring, over-simplifying, and over-rationalizing of problem statements.

We had done all three in the same file.

• 42 participants – Randomized study with identical instructions
• Mann-Whitney U = 116.5, p = 0.004 – Significantly less originality under “requirements”
• 42 designers in 21 pairs – Requirement templates promote early fixation

Why agents fail here just like people

Design fixation is not new. Jansson and Smith described it in Design Studies in 1991: when people see an example before a design task, they adopt its features—including deliberately introduced flaws. The result is the same number of drafts, but less flexibility and less originality. This held true for professional engineers as well as students.

We now see the same effect in agents. Sanjith Vigraham’s work this year examines our exact case—agentic design exploration rather than implementation—across more than 2,700 runs and seven context conditions. Two findings matter in practice.

First, the same artifact type can substantially improve exploration in some tasks and worsen it by up to 46 percent in others. In several tasks, an irrelevant document outperformed each relevant one.

Second, and this explains our project: narrowness that comes from the training data can be disrupted by perturbations. Narrowness that comes from an explicit instruction cannot. If you nail an agent to a solution in the briefing, no additional context will pull it back out.

A note on robustness: this is a single-author study, one model, not peer-reviewed. It supports a mechanism, not a natural law.

Complementing this, Laban and colleagues at Microsoft and Salesforce studied over 200,000 simulated conversations. The average performance drop across multiple conversation rounds is 39 percent compared to a single round. The mechanism is the same: models form early assumptions, produce a premature final answer, rely on it, and then fail to backtrack.

• 2,700 runs, 7 context conditions – Agentic design exploration analyzed
• up to 46 percent – Exploration worsened depending on artifact type
• 39 percent – Performance drop across multiple conversation rounds

Context is a budget, not a reservoir

Anthropic frames the mechanics in their Context Engineering guide. Models have an “attention budget,” and every additional token consumes part of it. Architecturally, n tokens create n² relationships, so a tension exists between context size and attention sharpness. Their goal: the smallest number of tokens with high signal value. Explicitly against our mistake: they advise not to stuff a checklist of edge cases into the prompt.

Two empirical measurements support this view.

Chroma tested 18 models across roughly 195,000 calls while keeping task difficulty constant and varying only input length. That isolates the effect of quantity. The striking result: 300 tokens of focused input beat 113,000 tokens of full text across all tested models.

Distyl AI stressed 20 models with up to 500 concurrent instructions. At 500, the best frontier model reached 68 percent instruction-following accuracy. A direct implication for long specifications: there’s a systematic preference for early instructions. What’s later in your document is followed less faithfully than what’s earlier.

In our project this was measurable before it became noticeable. Before the first content question, 1,500–3,000 lines of preloaded context had accumulated, plus two to three minutes of wait time per query.

• 18 models, 195,000 calls – Input-length effect isolated
• 300 tokens vs. 113,000 tokens – Focused input beats full text across models
• 20 models, 500 instructions – Best model at 68% instruction adherence
• 1,500–3,000 lines + 2–3 minutes – Preloaded context and per-query wait time in the project

The obvious fix is the same mistake in different clothes

My first repair idea was to free the design phase and feed it with desired references, best practices, and visual examples. That would have repeated the mistake.

Wadinambiarachchi and colleagues tested this at CHI 2024 with 60 participants. Support from an AI image generator during ideation increased fixation on the first example. Participants produced fewer ideas, with lower variance and less originality than the control group.

Reference pages at the start of a design phase are not extra degrees of freedom. They are the anchor where everything hangs.

What we do differently now

1. Mission briefing instead of a requirements list. Purpose, audience, and success criteria. Consciously no module list.
2. One run with zero context. It shows how broadly the system explores on its own. It’s the cheapest diagnostic in the process.
3. Divergence with minimal exemplars. If we need examples, we use anti-patterns (“not like this, and here’s why”), not best-practice templates.
4. Requirements only after form is decided. Scope, data fields, and governance limits come after, not in parallel.
5. Guardrails last. They belong in execution, where long autonomous runs are the point.

Does this mean we plan less?

No. The relationship between specification and creativity is not linear but inverse-U shaped. Acar, Tarakci, and van Knippenberg synthesized many studies in the Journal of Management in 2019: too few constraints reduce creativity just as surely as too many. The middle amount stimulates it.

In agentic development the current reflex goes the other way. Spec-driven development—the full up-front specification as the leading artifact—is fashionable. Colin Eberhardt at Scott Logic implemented the same feature twice, once with such a tool and once without. With the spec tool: 689 lines of code, 2,577 lines of specification text, three and a half hours of review, and one bug the spec didn’t prevent. Without the tool: a 15-minute review and no bug. Thoughtworks’ Technology Radar placed the method at “Assess” in November, noting that we may relearn a bitter lesson: hand-crafted, detailed rules for AI do not scale.

• 2,577 lines of specification – Volume in the spec-driven approach
• 3.5 hours review – Effort with full up-front specification
• 15 minutes review – Effort without the spec tool
• 0 errors vs. 1 error – Outcome quality comparison

Our roadmap wasn’t wrong. It only had one mode, and that mode was execution.

Frequently asked questions about Requirement vs. Constraint in agentic design (FAQ)

How do I tell a requirement from a constraint in practice?

Requirements specify what a system must deliver and often exclude options prematurely. Constraints define the frame within which solutions may be found. For creative phases, tight, meaningful constraints help more than exhaustive requirement catalogs. The order matters: first form and direction, then details.

How much context should I give agents?

Context is a scarce attention budget, not a storage pool you can fill. Start with the smallest, most relevant set of tokens and expand only deliberately. Long, fuzzy prompts reduce focus and adherence, especially for instructions placed later in the document. Measure impact, not volume: short, precise tokens beat full-text dumps.

Can I use examples without causing fixation?

Yes, but sparingly and preferably as negative examples or contrasts. Early, strong exemplars narrow the search space and anchor first solutions. Use examples after an initial divergence phase or explicitly mark what to avoid. That way they guide rather than prescribe.

What does this imply for specifications and roadmaps?

Formalize requirements only after the broad form is decided. Guardrails and detail rules belong in execution; they hamper exploration. Explicitly separate creative divergence from subsequent convergence. Keep roadmaps multimodal: discover, then specify, then deliver.

How do I spot early if my agents are becoming “stuck”?

Plan an initial run with no context and compare the system’s breadth, variance, and originality. If diversity drops sharply or early assumptions repeat, fixation is likely. Watch for growing wait times with heavy pre-context and for the absence of backtracking to earlier alternatives. Use perturbations only when narrowness stems from implicit knowledge, not hard instructions.

In the end, it’s the same question that applies even without agents: who in your process is still allowed to propose something that’s not already in the briefing?