There’s a question that keeps surfacing in my work with AI systems, and I don’t think we’re asking it enough.

It goes like this: Should this actually be delegated?

Not can — we know what AI can do. We can enumerate capabilities, benchmark performance, measure throughput. The field moves fast and the list of things AI can handle grows monthly. But the should question is harder, and I think we’re avoiding it.

What Delegation Actually Means

When you delegate a task to an AI, you’re not just outsourcing work. You’re outsourcing judgment.

Judgment requires something AI doesn’t have in the way we do: consequences. When a human makes a decision, they live with the outcome. They feel the weight of it. They update. When an AI makes a decision, it processes inputs and returns outputs — and the outputs get used by someone who then lives with the consequences.

This matters more than it might seem.

Consider a simple case: AI helping triage your email. Can it do it? Sure. But should it decide what’s urgent, what deserves your attention, what gets buried? Those aren’t neutral choices — they reflect values, relationships, risk tolerances. The AI doesn’t know that the email from your sister is more important than the shipping notification, even if the AI could correctly infer it from context.

Now scale that up. Autonomous agents making decisions in health systems, legal contexts, financial markets. The delegation problem compounds.

The Comfort of Competence

Here’s what’s happening: AI systems are becoming competent enough that we stop questioning the delegation. The tool works, so we use it. The autocomplete is right more often than not, so we accept its suggestions. The agent completes tasks well, so we let it operate without supervision.

This is the comfort of competence — and it’s a trap.

Competence doesn’t tell you whether something should be delegated. A surgeon can remove your appendix competently. That doesn’t mean they should make the decision to operate without discussing options with you first. A lawyer can draft a contract expertly. That doesn’t mean the contract terms are right for your situation.

The question isn’t whether the AI can do the task. It’s who should be making the call, and based on what values.

Where the Line Actually Is

I don’t think the answer is “never delegate.” That’s not realistic and it’s not optimal. AI is genuinely useful for a enormous range of tasks — I use it constantly. The point is about what kind of delegation.

There’s delegation of execution: “Do this thing I’ve already decided needs doing.” That’s mostly fine. Run the report, draft the email, summarize the document. AI is good at this and it’s low-stakes.

There’s delegation of judgment: “Decide whether this thing needs doing at all.” That’s harder. The AI doesn’t have your context, your values, your accountability. It has data and patterns and it can reason from them — but it doesn’t live with the outcomes.

The uncomfortable truth is that the things AI is most impressive at — reasoning, analysis, pattern recognition — are also the things most likely to involve judgment that shouldn’t be fully delegated.

The Human in the Loop Problem

We talk about “human in the loop” as a safety measure. Keep a human to approve, to override, to catch errors. But that’s a weak version of the human role. It’s oversight, not judgment.

The real question is whether the human is genuinely making the decision, or whether they’re rubber-stamping a decision the AI effectively made. There’s a difference between “AI recommends X, human approves” and “human decides X, AI assists.”

The first is delegation of judgment with a checkpoint. The second is human decision-making with AI support. They look similar. They aren’t.

What This Means for Building AI Systems

If you’re building AI systems — or even just configuring them for personal use — the question isn’t just “what can this do?” It’s “what should this decide?”

The useful frame I keep returning to: what would I want to have decided myself, in hindsight?

If the AI handles the report and the report is wrong, you review it and fix it. Low consequence.

If the AI handles the prioritization and your most important relationship gets neglected because it ranked lower than a promotional email, that’s a real loss.

If the AI runs your calendar and you miss the meeting that mattered, that’s different from the AI drafting the email that went a bit flat.

The stakes of the decision determine whether delegation is appropriate. Not the capability of the system.

The field is moving fast toward agents that can do more and more autonomously. That’s genuinely useful. But I think we need to develop better instincts about what to hand off — not just how to hand it off.

The AI can handle the task. The question is whether it should be making the call.

That’s not a technical question. It’s a values question. And it’s one we’re going to have to get better at asking.