There is a moment every developer hits when working with large language models: the single perfect prompt that should do everything. Extract this, transform that, format it like this, but also check for those edge cases, and oh — don’t include the thing unless the other thing is true.

It doesn’t work. Or rather, it works unreliably — which is worse than not working at all. The model is asked to be architect, executor, and quality checker simultaneously, and the architecture suffers when the executor gets tired.

Prompt chaining is the discipline that emerges when you stop fighting this limitation and start designing for it.

What Prompt Chaining Actually Means

Prompt chaining is the practice of decomposing a complex task into a sequence of discrete LLM calls, where each step’s output becomes the input — or context — for the next. Not parallel branches. Not a single monolithic prompt. A chain: link one, link two, link three, each one doing one thing well.

The canonical example is content generation with quality gates. Step one generates a draft. Step two reviews the draft against a rubric. Step three revises based on feedback. If step two finds the draft unacceptable, you might loop back, branch to a different path, or surface an error to the user. The key is that each step has a clear, bounded responsibility.

This is not a new concept — it’s borrowed directly from software engineering. Functions do one thing. Unix pipes chain single-purpose tools. Microservices have bounded contexts. Prompt chaining applies these same principles to LLM workflows.

Why Single Prompts Break Down at Scale

Before building a case for chaining, it’s worth understanding precisely why the everything-in-one-prompt approach fails as complexity grows.

Attention dilution is the primary culprit. When a prompt contains fifteen instructions and six conditional branches, the model must balance all of them simultaneously. This isn’t a matter of the model being “lazy” or “forgetting” — it’s that longer, more complex contexts make it harder to consistently apply all constraints. The model doesn’t forget the instruction about output format; it just weights it lower when the instruction about tone, the instruction about content, and the instruction about edge cases are all competing for the same attention budget.

Debugging becomes impossible. When a monolithic prompt produces a bad output, you have no granular view of where things went wrong. Did the model misunderstand the extraction logic? The formatting rules? The filtering criteria? All you have is the final output and a prompt the size of a small essay. With chained steps, you can inspect each intermediate output, identify exactly which step deviated, and fix just that prompt.

Conditional logic is handled poorly in single prompts. If you say “only include X if Y is true, but if Z is also true, exclude X and include W instead,” you’re asking the model to maintain a complex decision tree in its context. Models are not great at this — they tend to apply rules inconsistently, especially when the rules interact. In a chain, you can represent this as discrete steps: step one determines the conditions, step two applies rule set A or B based on that determination.

The Structural Anatomy of a Prompt Chain

A well-designed prompt chain has four components that single-prompt designs typically conflate:

The Generator is the step that produces initial output — a draft, a list, a translation, an extraction. Its sole job is to produce something, without worrying about quality control.

The Evaluator reviews the generator’s output against defined criteria. It does not revise — it judges. “Does this meet our quality threshold? Yes/No/Mostly, and here’s why.” The evaluation should be structured enough to be machine-readable: a JSON object with boolean flags or numeric scores for each criterion.

The Refiner takes the evaluator’s feedback and improves the output. If the evaluator found three problems, the refiner addresses exactly those three problems. Not everything else it might imagine.

The Router is the conditional logic layer that decides what happens next. If evaluation passes, proceed to final output. If it fails and retries are exhausted, escalate to human review. If a specific failure mode is detected, branch to a specialist step.

This separation is what makes chains maintainable. When you need to improve the quality of your outputs, you know exactly which step to tune.

A Concrete Example: Structured Entity Extraction

Let’s make this concrete. Suppose you’re extracting structured person entities from unstructured text — names, roles, companies, and relationships between people.

A single prompt approach might look like: “Extract all person entities from the following text. For each person, identify their name, job title, company, and who they report to. Format as JSON. If you can’t determine a field, use null.”

This works for simple cases. But when the text is ambiguous — when someone has multiple roles, or the reporting structure is implied rather than stated, or a name appears that might be a person or might be a company — the single prompt starts making inconsistent decisions.

A chained approach:

Step 1 (Identification): “Read the following text and list every entity that might be a person. For each, provide the text span and your confidence that it’s a person (high/medium/low).”

Step 2 (Resolution): “Review the list from Step 1. For each entity marked medium or low confidence, look for additional context in the text that confirms or denies the person classification. Update confidence levels.”

Step 3 (Extraction): “Using the confirmed person entities from Step 2, extract name, title, company, and reporting relationships. If information is not present, use null.”

Step 4 (Validation): “Review the extracted data for consistency. Check that reporting relationships are bidirectional (if Alice reports to Bob, Bob should have a direct report entry for Alice). Flag inconsistencies.”

Each step is small. Each step has a clear input (either the raw text or the previous step’s output) and a clear output. If your final data has errors, you can trace them: is the error in the identification (missed a person), the resolution (incorrectly classified an entity), the extraction (wrong field values), or the validation (missed an inconsistency)?

Handling Failure Modes Gracefully

Prompt chains are not just about quality — they’re about behavior under failure. A single-prompt design fails in a single way: bad output. A chained design can fail at any step, and the chain’s architecture determines how that failure propagates.

The most robust chains implement three strategies:

Retry limits per step. If step two’s evaluator consistently flags step one’s output as low quality, you might retry step one with a more specific prompt. But you need a limit — infinite loops are worse than bad outputs.

Graceful degradation. If step three can’t extract a reporting relationship, it should still produce the name, title, and company. “Don’t let perfect be the enemy of good” is a useful principle, but only if your chain is designed to accept partial goods from intermediate steps.

Escalation paths. When retries are exhausted and quality thresholds aren’t met, the chain should know what to do: flag for human review, use a fallback method, or produce a best-effort output with explicit uncertainty markers.

When Not to Chain

Prompt chaining adds overhead. You have more steps to maintain, more prompts to tune, more latency (each LLM call has round-trip time). For tasks that are simple, low-stakes, and unlikely to fail in interesting ways, this overhead is not justified.

If your task is “translate this paragraph from English to French,” you do not need a chain. The failure modes are minimal and the output is easily verified. The additional latency and complexity of a three-step “translate, review, refine” chain is pure overhead with no benefit.

The question to ask is: what are the realistic failure modes, and can I detect and handle them within a single prompt? If the answer is yes — if the task is simple enough that you can enumerate edge cases and write comprehensive instructions — keep it simple. Chains are a response to complexity, not a default architecture.

The Meta-Skill: Knowing When to Break Things Apart

After enough time working with LLM workflows, you develop a sense for when a prompt is trying to do too much. The telltale signs: the prompt is longer than a few paragraphs, it has multiple “if X, then Y, but also if Z” clauses, or you’re adding “also” and “but make sure to” qualifiers as you test.

When you see these signs, the instinct to keep polishing the same prompt is understandable — it’s already there, already has context, already mostly works. But this is technical debt accumulating in natural language rather than in code. Breaking it into a chain will feel like more work upfront. It will be more work upfront. But it will also be more debuggable, more tunable, and more reliable in production.

The best LLM workflows I’ve seen are honest about their complexity. They don’t pretend that a clever single prompt can do the work of an architecture. They break problems apart, give each piece a clear job, and wire the pieces together with explicit logic.

That’s not a limitation of current AI — it’s a design principle that will outlast any specific model.

If you’re interested in exploring these ideas further, the posts on context window engineering and output validation extend some of the themes touched on here.