Context Engineering: The Next Evolution of AI

For CTOs, Engineering Leads & Senior Product Managers  ·  Umaku field notes · Pillar essay

Two years ago, prompt engineering was a role. Today, most of the people who held that role are doing something different, and it does not yet have a settled name. This essay argues the name is context engineering — the discipline of deciding what information an AI system receives, how it is retrieved and organized, and how it stays accurate over time. As frontier models commoditize, this is where competitive advantage now lives.

The reason is structural. In 2023 the gap between the best model and the one most teams could actually get was wide enough that prompt tricks — few-shot examples, chain-of-thought triggers, careful phrasing — produced dramatic differences in output. In 2026 the gap is narrow. Frontier models sit within a few points of each other on the benchmarks that describe real work. Prompt tricks still work — they just work equally well on any of them.

Figure 1. The lever moved. In 2023 it was the words in a prompt. In 2026 it is everything the model is given to work with.

What still moves the needle is what the model is given to work with. And that turns out to be a much larger, harder, and more compounding discipline than simply  “writing a good prompt.”

1. The Commoditization Argument

Every technology adoption curve looks similar: an early period where the raw capability is the differentiator, followed by a period where the capability is universal and the differentiator moves to what surrounds it. Databases went through this in the 1990s — the advantage moved from “having a database” to schema design, ETL, and warehousing. Cloud went through it in the 2010s — from “having cloud” to orchestration and cost engineering.

AI is having its analogous moment now. Model APIs are cheap. Inference latency is falling. The top labs’ offerings differ in personality more than raw capability at most useful tasks. Open-weight models within reach of a small team are strong enough to run production workloads.

Figure 2. Frontier closed models, mid-tier commercial models, and the best open-weight releases are all converging toward the same benchmark ceiling. The gap that used to decide who won is closing.

When the same underlying capability is available to everyone at similar cost, the outcome of the technology is decided by everything you build around it. That everything now has a name.

2. Prompt Engineering, In Context

Prompt engineering was the first response to the AI wave. The natural lever seemed to be the words in the prompt, and the words did matter — teams that were good at it produced obviously better AI products than teams that were not.

But prompt engineering is a bounded discipline. The lever is limited by the length of a prompt and the ingenuity of the person writing it. Once you have written the good prompt, you have written it; the next hour of prompt engineering does not produce another 10× gain.

Context engineering is what happens when the industry realizes that the good prompt was one component of a larger, more interesting system — the system that decides what the model sees at all.

Figure 3. Prompt engineering and context engineering are related but different disciplines. Only one of them compounds.

3. What Context Engineering Is

Context engineering is the discipline of deciding what information an AI system receives, how that information is retrieved and organized, and how it stays accurate over time.

Four decisions structure the work.

What to include. The model has a finite context window. Anything it needs to reason about must be selected from the larger corpus. Selection has costs (tokens, latency), benefits (correctness), and failure modes (missing material).

How to retrieve. For any large corpus, the system needs a way to find the right pieces at the right time. Embeddings, keyword search, hybrid retrieval, graph traversal, structured queries — each has its uses. Choosing which to use, and when, is the retrieval design.

How to organize. Information given to the model as unstructured text is treated as unstructured text. Information given with structure — sections, tags, provenance, timestamps — is treated with more discipline by modern models. Organization is a lever.

How to keep fresh. Every source of context has a half-life. Documents change. Systems change. Decisions get overturned. A retrieval system indexed a month ago is a source of confidently wrong answers about a system that has moved on.

Figure 4. The four decisions of context engineering. Get any of them wrong, and the model performs badly on a problem, and that is not the model’s fault.

4. The Six Sources of Context

Every production AI system draws from some subset of six sources of context. Each has a different cost, half-life, and failure mode.

System instructions are the fixed frame — role, constraints, guardrails. Cheap to write, hard to update at scale.

Retrieval brings in documents, code, tickets, prior conversations, and decisions of record. This is the largest and most malleable source.

Tool outputs are the results of API calls, database reads, and calculations. Fresh by construction — the model does not know your inventory number, but a tool call does.

Conversation history is what has already been said in this session. Short-lived, essential for coherence, prone to overflow.

Structured memory is persistent facts about the user, the project, or the world that the system has learned to keep — preferences, project charters, prior decisions.

Live state is the current time, current on-call rotation, current status of the production system. Ephemeral and vital.

Figure 5. The six sources of context. A production system that skips any of these is smaller than it needs to be. A system that treats them as equivalent will confuse a stale document with a live tool result.

5. The Freshness Problem

Static context is a well-known trap. A team indexes their docs on Monday. A week later, half of it is out of date. A month later, most of it is. The retrieval system does not know this — it returns confident chunks, the model answers with equal confidence, and the trust budget is spent.

The failure is not in the model. It is in the assumption that context can be indexed once and forgotten. Production systems are, definitionally, moving targets.

Figure 6. Every source has a half-life. Freshness is not a nice-to-have — it is the difference between an assistant that helps and one that misleads.

A context engineering approach treats freshness as a first-class concern — with re-indexing pipelines, source-of-truth policies, and provenance metadata that lets the system say “as of last Tuesday” instead of “as of forever.” The teams building this well are subtracting the source of a large class of production incidents.

6. Where Advantage Compounds

The reason context engineering matters more than prompt engineering, structurally, is that it compounds.

Every user interaction produces a signal — an outcome, a correction, a preference — that can feed back into retrieval, memory, or structure. Systems that treat these signals as data improve their own context loop over time. The next user gets a slightly better answer than the previous one, on the same model, at the same cost.

Figure 7. The context flywheel. Every interaction produces a signal. Systems that treat signals as data compounds. Signals + refinement is where the moat lives.

This is the context flywheel. It is not a metaphor: it is the shape of the moat that the leading AI products in 2026 are quietly building. The moat is not in the model. The moat is in the accumulated context that has been curated, filtered, refreshed, and structured through millions of interactions the competition has not seen.

7. What This Means for Engineering Organizations

Context engineering has structural consequences.

There is a new discipline. It sits between application engineering, ML engineering, and data engineering, and it borrows from all three. Some teams are already hiring for it — the job description says “AI engineer” or “applied AI engineer” and, in the body, describes retrieval systems, memory design, and freshness pipelines.

There is a new infrastructure requirement. Embeddings pipelines, vector stores, retrieval routers, MCP servers, and evaluation harnesses that measure retrieval quality separately from model quality. A production AI system now has as much operational surface as a production web service.

Figure 8. A production context stack. Every horizontal layer is a place a competitor cannot reach without their own engineering.

There is a new failure mode to monitor. The failure is not “the model got it wrong.” The failure is “the model was given the wrong context.” A team without observability on their retrieval layer will spend the next year misdiagnosing production incidents.

Figure 9. The stack from silicon to product. Advantage moves upward as each layer commoditizes. In 2026, it lives in the context layer.

8. Umaku Is Context Engineering for Software Delivery

The discipline this essay describes at the AI-system level is the same one Umaku applies at the team-software level.

A team’s tickets, commits, ADRs, PRs, and post-mortems are a corpus. Deciding what to expose to a code-review agent, how to retrieve it, how to organize it, and how to keep it fresh is context engineering. The four Umaku post-sprint agents are practitioners of it; the queryable corpus underneath is the infrastructure; MCP is what plugs it into every engineer’s IDE.

Figure 10. The same discipline, at two scales. Context engineering for an AI product and context engineering for a software team share every element that matters — corpus, retrieval, memory, freshness, interface.

The same shift that makes context engineering the differentiating discipline for AI products makes it the differentiating discipline for AI-augmented teams. Context is the new currency.

9. What Comes Next in This Series

This essay is the pillar of a four-part series on context engineering. Three follow-up pieces provide the practitioner’s toolkit:

Context Layers: Giving AI the Information It Needs. The architectural piece. How the six sources of context stack, when to use each, and how to compose them without collision.

Contextual Retrieval: Improving AI Accuracy Without Bigger Models. The performance piece. Chunking, hybrid search, contextual embeddings — and why most accuracy problems are retrieval problems in disguise.

Model Context Protocol (MCP): What You Need to Know. The standards piece. What MCP is, why it exists, and what it unlocks for teams building compound AI systems.

Figure 11. The four pieces together form a working reference for teams building the AI systems of the next two years.

Takeaway

Prompt engineering was a real discipline in a specific moment. It solved the problem the industry had when models were the differentiator. It does not solve the problem the industry has now, which is that models are no longer the differentiator — the context around them is.

The teams that recognize this shift and invest in the four decisions — what to include, how to retrieve, how to organize, how to keep fresh — will spend the next eighteen months compounding an advantage the model no longer provides. Teams that keep tuning prompts will find themselves competing on a surface already commoditized.

Build for context. It is where the moat lives now.

Umaku Built by Omdena