Embeddings and vector search: finding the right passage

You remember that a document explains certificate renewal, but its title says “credential rollover.” A literal keyword search can miss the connection. Embeddings help by representing text as numerical vectors that can be compared for similarity. Vector search uses those representations to find candidates whose meaning is close to a query, even when the wording differs.

The important word is candidates. A close match can be outdated, apply to another product version or contain the exception that reverses an apparent recommendation. Good retrieval brings the relevant passage into view with enough source information to judge it. It does not turn a similarity score into proof.

What an embedding represents

An embedding model maps an input into a fixed-length list of numbers. Its training shapes which relationships are reflected in that space. A text embedding model may place similar passages near each other, but its dimensions are not usually a set of human-readable topics. A two-dimensional sketch is a teaching aid, not a picture of every relationship in a high-dimensional model.

Queries and documents must use a compatible embedding space. Two models that both produce 384 numbers do not necessarily assign those numbers the same meaning. Re-embedding a collection after a model change is an explicit migration, with new retrieval checks, rather than a harmless replacement of one API URL.

The Sentence Transformers quickstart shows the basic encode-and-compare workflow. The practical setup below follows that route with a small local collection. It downloads a model, so use a disposable Python environment with network access and sufficient storage. It is an instructional example, not a measured retrieval benchmark.

Search a small collection

python3 -m venv .venv
. .venv/bin/activate
python -m pip install sentence-transformers

from sentence_transformers import SentenceTransformer

records = [
    {"id": "cert-v1", "version": 1, "status": "superseded",
     "text": "Renew the service certificate using the shared gateway job."},
    {"id": "cert-v2", "version": 2, "status": "current",
     "text": "Credential rollover for edge v2 uses its dedicated renewal job."},
    {"id": "disk-v1", "version": 1, "status": "current",
     "text": "Inspect inode availability when a filesystem cannot create files."},
]
model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")
vectors = model.encode([r["text"] for r in records], normalize_embeddings=True)
query = model.encode(["How do I renew edge v2 credentials?"], normalize_embeddings=True)
scores = (query @ vectors.T)[0]
for i in scores.argsort()[::-1]:
    print(records[i]["id"], records[i]["status"], float(scores[i]))
    print(records[i]["text"])

The records are synthetic, forming an illustrative example. The code encodes the query and passages with the same model, normalizes their vectors and takes a dot product. For unit-length vectors, that dot product equals cosine similarity. Read the actual ranked output from your run; the guide does not invent scores or promise a particular ranking.

Inspect whether both certificate passages were retrieved, and whether the current edge-v2 procedure is distinguishable from the superseded shared job. Preserve each passage's identity, effective version and status when presenting it. A semantically convincing older procedure must not quietly become current guidance.

From a loop to an index

The example compares the query with every record. That is easy to inspect for three documents. A larger collection often uses a vector index, a data structure designed to search vectors efficiently. Faiss's getting-started guide demonstrates adding vectors and searching an index. Different index choices trade memory, speed and retrieval accuracy.

Approximate nearest-neighbor search can avoid comparing every candidate, but may miss a result that an exhaustive search would find. Measure that tradeoff using representative queries and known useful passages. A fast query with the wrong evidence is not a retrieval improvement.

Long documents are often split into chunks before embedding. Keep enough surrounding context to understand a condition or exception, and retain the original document link and location. A chunk boundary that separates “do not” from the action it qualifies can create a misleading result even when the vector search works as designed.

Judge relevance and applicability separately

For current operating guidance, apply the reader's permitted scope and relevant product version before accepting a result. Access controls must constrain retrieval, not merely hide a source link after its contents have already been supplied to a model. Dates help, but the newest document is not automatically applicable to an older deployed version.

For a historical question, the superseded certificate procedure may be exactly the right source. Keep that distinction explicit. When effective scope remains unclear, show the disagreement instead of collapsing it into a single recommendation. Once retrieval returns useful, attributable passages, RAG adds the next step: assembling those passages into the context for an answer.