When to build with AI, and when not to
At TheyDo we are enabling every product team to deliver agentic value to customers. Today that work is concentrated in one team. Over time more and more business logic will be captured by agents. And we work in qualitative data, where AI shines. The move is happening for well-known reasons.
Raw intelligence keeps compounding. The models are ahead of us. They are more capable than anyone has productised yet. That is a product opportunity for fast-moving teams.
It unlocks use cases that were impossible before. Reading qualitative data at scale was impossible for software and unaffordable for people. Now we mine millions of records into insights and answer questions across the whole context graph.
We can deliver customer value faster than ever. Business logic in deterministic software is expensive. In the AI domain it becomes a prompt, and changing one word can entirely change the logic. This makes an AI feature an MVP for hundreds of deterministic features. You can discover with the agent, and harden deterministically what proves itself. Each different way of prompting the agent is a use case, albeit crude and UI-less.
Yet AI can be a hammer that makes everything look like a nail. For the foreseeable future there are still strong cases for deterministic software. The answer on when is "it depends". At a high level:
- Does it need intelligence? AI.
- Is the workflow narrow, well-defined, and frequently used? Non-AI.
I wrote the twelve questions below as guidance for teams joining the agentic layer of our product. Answering them should build the intuition for whether to build with AI or without it.
Definitions
AI: any shape the model takes. An agent, an AI workflow, a single AI call. If a model produces the output, it is AI.
Non-AI: deterministic, traditional, designed, coded software. All mean the same thing. The same input produces the same output.
The capability ladder:
- Rules. Non-AI. Hand-written logic. If X then Y. Exact, fast, cheap to run, expensive to extend.
- A single AI call. One prompt, one response, inside an otherwise deterministic feature. Classify this quote, summarise this doc. The model produces content; code decides everything else.
- An AI workflow. Several model calls orchestrated by code we wrote. We define the steps, the model fills them in. Control flow stays deterministic, content is probabilistic. You own the plumbing.
- An agent. The model decides the steps, using tools in a loop. The agent owns the plumbing.
Find direction
Workflow
1. How well defined is the workflow? Is the user problem fuzzy or deterministic? If you can map every step today, it is a good candidate to build deterministically. If the workflow is still finding its shape, AI serves it while it crystallises. An agent can even serve workflows we never designed for.
One caveat. Hardening a workflow as non-AI is a bet that it is stable, and that it won't be absorbed by agents in the near future.
2. What is the step in the workflow? The decision unit is the step within a workflow, not the feature. Hybrid workflows, where AI proposes and deterministic software commits, are becoming the norm.
3. How broad is the workflow? Count the valid ways to do the thing. Three ways: encode them with non-AI. Dozens: every one is UI to design and code, while with AI each variation is just a prompt.
4. Can we define what good looks like? Before building, write down what a good output is and how we will check it. If we cannot, the problem is not the testing. It is that we do not understand the job well enough yet, and neither will users.
The stakes
5. What does a mistake cost? Reversibility and consequence set the ceiling on autonomy. When mistakes are high-stakes or hard to detect, autonomy becomes a liability.
6. Does it need to be exact? AI output varies between runs. If the same input must always produce the same output, that is non-AI territory, or AI kept behind a deterministic gate. Accepting variance means designing for it: acceptance thresholds instead of pass and fail.
UX
7. Who are we serving? Which segment of users will use this? Agent adoption is rarely universal. An agent-only capability leaves everyone who has not adopted the agent without it.
8. How fast does it need to feel, and for whom? How often users run a workflow determines the latency they will accept. Thirty times a day cannot wait for a model; once a day can. Agentic systems trade latency and cost for better task performance. Make that trade only where it pays off.
9. Does it fit the users' mental model? The anti-pattern is replacing good UX with a chatbot. If a table, filter, or form solves it, that is the better product.
10. Can users verify the output faster than doing the work themselves? With AI, verification tends to become the bottleneck. If checking the output takes as long as producing it by hand, the AI path defeats itself.
Economics and ownership
11. What does it cost to run at scale? "Cheap to innovate" conflates build cost with total cost. A deterministic feature costs mostly at build time. An AI feature costs on every invocation, and tokens on a high-volume workflow can dwarf the build cost of the deterministic alternative. Don't build a $100 solution for a $5 problem, in either direction: heavy UI for a rare workflow, or heavy tokens for a frequent one.
12. What does it take to keep it running? An AI feature carries recurring liabilities a deterministic one does not. Eval sets, acceptance thresholds, drift monitoring.
The bottom line
Every workflow lands in one of three places. Some belong in AI: the problem stays fuzzy or broad, and intelligence is the feature. Some belong in non-AI: frequent, exact, speed-critical work. A lot falls in between, in hybrid workflows where AI proposes and deterministic software commits.
There is no scorecard. Answer the twelve questions and the decision tends to make itself. Some workflows will never leave the AI domain, because intelligence is the product. Others should never enter it. The intuition you build by asking these questions is the point.