GraphRAG versus vector search when the question is "who knows this client?"

A firm buys an AI tool, points it at the shared drive, and asks it the question it has always wanted answered: who here has a relationship with this client's finance director?

It gets an answer. Three names, confidently given. Two of them are wrong.

This is not a bad product or a weak model. It is a predictable consequence of using similarity retrieval for a question whose answer is not a passage of similar text. Worth understanding, because the same failure will recur with every model generation until the retrieval layer changes.

How each one actually works

Vector retrieval turns your question into a vector, turns every chunk of every document into a vector, and returns the chunks nearest to the question in that space. It is retrieval by resemblance, it is very fast, and for a large class of questions it is exactly right. Ask what a report concluded about network resilience and the passage that answers it looks like the question. Similarity works.

Graph retrieval does something categorically different. It walks recorded relationships. From this stakeholder, to the engagements they were involved in, to the colleagues who delivered them. The answer is produced by traversal, not by resemblance, and it can be as many hops deep as the question needs.

Similarity search finds text that looks like the question. Relationship questions do not look like their answers.

Why the finance director question breaks similarity search

Trace what would have to be true for vector retrieval to answer it correctly.

There would need to exist, somewhere in the corpus, a passage that says something close to "these colleagues have a relationship with this finance director". No such passage exists. Nobody writes that sentence.

What exists instead is scattered across documents that individually contain none of the answer: a steering committee attendee list from 2022, a proposal naming a delivery team, an email thread where the director raises a concern and a manager responds, a report cover page. The answer is the intersection of those, and no chunk contains it.

So retrieval returns the chunks that mention the director. The model receives them and does what it is built to do, which is produce a fluent, plausible response from the material in front of it. Names that appear near the director's name get returned as people who know the director. Frequency of co-occurrence becomes a proxy for relationship, and it is a bad one: the analyst who was copied on forty status reports outranks the manager who sat opposite them in eight meetings.

The output is not hallucination in the usual sense. Every name is real, every document is real. The relationship is inferred rather than recorded, and the user cannot tell the difference, because the answer arrives in the same tone whether it is derived or invented.

What the benchmarks say

The published numbers support this and are worth quoting carefully, because they are commonly overstated.

A benchmark by AWS and Lettria on SEC filings found that adding graph structure lifted correctness from 16.7% to 56.2% on complex relationship-tracking queries. That is nearly a fourfold improvement, and it is on exactly the class of question described above: multi-hop, entity-relationship, answers spread across documents.

Vendors report hallucination reductions in the range of 70% to 90% for grounded retrieval generally, and FalkorDB has published a 90% reduction against traditional RAG. Treat vendor figures with appropriate scepticism, ours included. What is consistent across all of them is the direction and the conditions: the gains are large where questions are relational and multi-hop, and close to nil where questions are single-passage lookups.

The property that matters more than accuracy

Set the benchmarks aside for a moment, because there is a difference here that matters more in a firm than a percentage.

When graph retrieval returns a path, the path is the evidence. This partner delivered that engagement, evidenced by this proposal. That stakeholder was involved in it, evidenced by this thread. Each hop names a document, so the whole answer can be inspected step by step.

When vector retrieval plus a model returns an answer, you get prose, possibly with citations. The citations tell you which documents were in the context window. They do not tell you that the relationship asserted in the sentence appears anywhere in those documents, because it may have been inferred from their juxtaposition. Citations of that kind are a weaker guarantee than they appear, and in a firm the difference is decisive: a partner about to walk into a client meeting needs to know whether a claim is recorded or deduced.

A cited answer tells you what the model read. A traversed answer tells you what the firm knows.

The negative case, which is the one that matters

Here is the failure that ends deployments.

Ask a similarity system about an area where the firm has no experience. It will still return the nearest chunks, because nearest is always defined, and the model will still produce an answer. What comes back is the firm's most adjacent-looking work, presented with the same confidence as a real answer.

That is the most dangerous output a system like this can produce, because it is acted on. A bid gets written claiming experience the firm does not have. Someone finds out in the room.

A graph cannot do this. If there is no edge, there is no path, and no path is a result: we have no evidence of this. That is why OrgAtlas draws capabilities with nothing behind them as hollow rather than omitting them. Absence is a finding, and a system that cannot report it is unusable for exactly the decisions where being right matters most.

What this means if you are evaluating tools

Four questions, ordered by how much they discriminate.

1. Ask it a three-hop question about your own firm and check every name. Not a summarisation question. Something like who here has worked with this specific person. Then verify each answer independently.
2. Ask it about something you know the firm has never done. The correct answer is that there is no evidence. Anything else tells you what its behaviour is under uncertainty, which is the behaviour you will be relying on.
3. For each claim, ask what it stands on. Distinguish "these documents were in context" from "this specific relationship is recorded in this specific document". They are usually presented identically.
4. Ask which questions are routed to which retrieval path. A vendor with a serious hybrid architecture will answer this readily. A vendor who describes everything as AI-powered search is doing similarity for everything.

The underlying point is not architectural preference. It is that firms ask relationship questions about their own work, and relationship questions need something that recorded the relationships.

Next: why provenance, not accuracy, is the property that makes a system usable in a firm