Semantic Memory: Why Ontologies Outlast Models

Every model generation your team has adopted in the last three years is already, in some sense, deprecated — retired, superseded, or quietly deprioritized by its own vendor in favor of a newer checkpoint with a better benchmark score. The one artifact from those three years of work that's still paying dividends, in every deployment we've watched hold up, is not a prompt library or a fine-tuned checkpoint. It's the ontology — the explicit, versioned map of what entities exist in the business and how they relate to each other, built once and maintained deliberately, independent of whichever model happens to be reading it this quarter.

What we mean by ontology, precisely

Not a data dictionary, and not a database schema, though it borrows from both. An ontology, as we build it, is a formal statement of the entities a business actually reasons about — "shipment," "customer account," "SKU," "maintenance window" — and the relationships between them that carry business meaning: a shipment belongs to an order, an order is associated with an account, an account has a risk tier that changes the interpretation of every downstream shipment delay. The distinction that matters: a schema describes how data is stored; an ontology describes how the business thinks, in a form a machine can query. You can migrate the schema underneath an ontology without touching the ontology at all, which is exactly the property that makes it durable.

What breaks when teams skip this step

Teams that skip building an explicit ontology don't fail immediately — they fail at the second or third model migration, when it turns out the entity relationships the first model had "learned" implicitly from prompt examples and few-shot context don't transfer. Every new model needs to be re-taught, in prose, buried in a system prompt, what a "qualified lead" means at this specific company versus the general sense of the term, and that re-teaching happens ad hoc, inconsistently, across every team that touches the model, because there was never a single place that definition lived. We've inherited systems where the definition of "active customer" differed by twelve days of inactivity threshold between three different agents built by three different teams, each one having independently prompted a model with its own interpretation. None of them were wrong. None of them agreed with each other. That's not a model problem — it's the absence of a shared semantic layer that every agent, regardless of vendor, is required to consult before it reasons about a customer.

An ontology is what makes retrieval precise instead of merely plausible

Semantic search over raw documents gets you plausible answers — text that resembles what a correct answer should look like. Retrieval grounded in an explicit ontology gets you answers that are structurally correct, because the retrieval step can traverse real relationships (this shipment's order, that order's account, that account's contract terms) instead of relying on embedding similarity to accidentally surface the right cluster of tokens. The difference shows up hardest in the cases embedding similarity handles worst: negation, multi-hop reasoning across entities that don't share vocabulary, and any relationship that's true but never stated in the same document. An ontology-backed retrieval layer answers "which accounts have shipments delayed past their contracted SLA" by walking a graph. A pure vector-search layer answers it by hoping a document exists somewhere that happens to mention delay and SLA in the same paragraph, for every account that matters.

Building it is elicitation work, not engineering work

The actual cost of building an ontology isn't the graph database or the schema tooling — that part is a week of work. It's sitting with domain experts long enough to surface the distinctions they treat as obvious and never write down: that a "return" and a "cancellation" are different entities with different downstream obligations even though both end with the customer not receiving the product, or that "capacity" means something different to a plant scheduler than it does to a sales forecaster referencing the same word. That elicitation is slow, unglamorous, and the single highest-leverage thing we do at the start of a decision-intelligence engagement, because everything built afterward — every agent, every retrieval pipeline, every model swap — inherits its correctness from whether that map was built right the first time.