Why search finds the subject but misses the point

Say you're looking for a quiet café where you can catch up with a friend. Search brings back good coffee, glowing reviews and live music every night. A fine evening out. Possibly a terrible place to hear how your friend's week went.

Or you want yesterday's version of a shared document without replacing today's edits. The first result explains how to restore it. Your colleagues would prefer that you kept looking.

In both imagined searches, the results are about the right subject. They miss the part that makes the answer useful to you. Reranking is a second pass over search results that can bring that distinction into focus. It takes the candidates already found, compares them again with the question and changes their order.

The words are close. The purpose is not.

A large search has to narrow the field. Keyword matching finds shared words. Embedding search turns questions and passages into numerical representations that can connect related wording, even when the words differ. Many systems use both. That helps a question about “getting an old draft back” find a page called “version history.” [1]

With a common embedding setup, the documents are processed ahead of time and the question is processed when it arrives. Search compares those separately prepared representations. This makes a big collection manageable. But a page can be close to your subject while missing a small, decisive condition. The embeddings guide shows how that first comparison works.

“Quiet enough to talk” is a condition, not decoration. So is “without replacing.” Drop those words and you have changed the afternoon, or the document.

A few words can change which answer belongs first

For our invented document search, three passages make the first cut:

• A: Restore an earlier version. Replace the shared file with its previous contents.
• B: Download an earlier version. Save a separate copy without changing the shared file.
• C: Version-history retention. Find out how long older versions remain available.

A and B both offer yesterday's words. Only B leaves your colleagues' work alone.

One common reranker processes the whole question and one passage together, then gives that pair a relevance score: a number for how well the passage fits the question. This model is called a cross-encoder. Processing the pair together lets relationships between their words influence the score, rather than relying only on representations made separately. [2]

For A, the model sees your wish to avoid replacement beside instructions that replace the file. For B, it sees “separate copy.” A model trained to judge relevance may use that relationship to move B up. The application repeats the comparison for the other candidates and sorts the scores.

Now change the question: “How do I replace the shared file with yesterday's version?” A becomes the useful passage. Nothing about the document changed. What you wanted changed.

The café example works the same way. Live music is a selling point when you're looking for a band and a possible drawback when you're trying to have a conversation. A useful result isn't simply a page with plenty of words in common. It fits the particular job those words are asking it to do.

It cannot choose a page it never saw

The reranker receives a shortlist, not the whole collection. If B never arrived, it can shuffle A and C all afternoon. Neither will acquire the missing copy instructions.

Adding another model isn't automatically an improvement. In a small, tidy collection, the first search may already put B at the top. A second pass has earned its place only if it helps enough to justify the extra work and waiting.

The top result still has to fit the situation

Suppose B comes first with a score of 0.9. That tells you where to look next; it doesn't settle whether your account has the copy option or whether yesterday's version still exists. The passage may describe an edition your organization doesn't use.

You get the old draft as a separate file. Your colleagues keep their edits. Or you find somewhere to hear your friend's story without competing with a drummer. A few words in a question can carry most of what matters. Reranking is one way to give them another chance to count.

More on models, scores and testing

What the extra pass costs and changes

In the pair-by-pair design above, fifty candidates mean fifty question–passage pairs, not necessarily fifty network requests: software can process pairs in batches. That extra computation is why a fast search often supplies a smaller set for reranking. Tiny collections can sometimes be scored directly. [3] Other reranking designs examine a list of passages together. [9]

That gives the people running a search system a much better question than “Do we need a cleverer model?” Did the helpful passage arrive at all? Was it present but poorly placed? Or did an AI assistant receive it and still give the wrong answer? Those are different failures. The RAG guide follows the later journey from retrieved passages to an assistant's answer.

Even a passage that appears to have arrived can be incomplete. A service may shorten a long document before scoring it. If “this creates a separate copy” falls outside the supplied text, the model is missing the very sentence we hoped it would notice. The page title in a log cannot tell us whether that sentence survived. [7]

For someone choosing a reranker, that distinction keeps the comparison honest. Useful test questions have passages known to answer them, and sometimes no adequate answer in the collection. Comparing the order before and after reranking shows what the extra pass changed. Recording missing passages separately stops the ranking model from taking the blame for something it never received. [8]

Score meanings vary. Sentence Transformers documents raw scores and an optional conversion into the zero-to-one range that preserves their order. Cohere documents calibrated relevance scores, intended to estimate relevance. Neither description makes a score proof that an instruction is current, permitted or appropriate to a particular account. Private documents also need approved access and data-handling controls before any model receives them; relevance is not permission. [4] [5] [6]

Sources

1. Elastic: ranking and reranking.
2. Elastic: semantic reranking.
3. Sentence Transformers: retrieve and rerank.
4. Sentence Transformers: cross-encoder usage.
5. Cohere: Rerank.
6. Elastic: document and field access controls.
7. Cohere: rerank API limits.
8. Sentence Transformers: cross-encoder evaluation.
9. Liu et al.: Leveraging Passage Embeddings for Efficient Listwise Reranking with Large Language Models.

The café, workspace, passages and score are invented teaching examples. No model or service was run. Documentation was checked October 30 and November 2, 2026.