Embeddings and Vectors: The Map of Meaning

Part four of ten. Search used to mean matching words. Embeddings let a system match meaning instead, and almost every AI feature that touches your data stands on that one idea.

1. What is an embedding?

An embedding is a list of numbers that stands for the meaning of a piece of text. An embedding model reads a word, a sentence or a document and turns it into that list. Texts with similar meanings get lists that are close together.

What the interviewer wants. A location, not a summary. An embedding doesn’t store the words. It stores where the meaning sits.

Watch out. Don’t treat embeddings as a way to hide sensitive data. Researchers have shown that text can be partly recovered from them.

2. How can two different sentences end up close together?

The model learned from huge amounts of text which phrases show up in similar situations. “Cancel my order” and “I want to stop this purchase” share few words. They appear in similar contexts, so the model places them near each other.

The trap. Close means related, not identical. “Refund approved” and “refund denied” can sit close together, and they mean opposite things.

Watch out. Negation and numbers are weak spots. Embeddings can blur “with” and “without”, or $50 and $500.

3. How do cosine similarity, Euclidean distance and dot product compare?

Three ways to compare two vectors. Cosine similarity looks only at direction, so it compares meaning regardless of length. Euclidean distance is the straight line between the two points. Dot product mixes direction and length into one score.

What the interviewer wants. The practical point: for normalised vectors, all three give the same ranking. Say why, then say you’d use the measure your embedding model recommends.

Watch out. Cosine distance and cosine similarity run in opposite directions. A similarity near 1 means close, and a distance near 0 means close. Mixing them up flips your results.

4. Why must the question and the documents use the same embedding model?

Because each model builds its own map of meaning. A vector from one model and a vector from another use different axes, even with the same number of dimensions. Comparing them is like measuring the distance between points on two different maps.

Why it matters. It’s an easy mistake when a team upgrades. New questions get embedded with the new model, old documents keep the old vectors, and search quality falls apart quietly.

What the interviewer wants. The model name and version stored next to every vector. That one column saves a painful week later.

5. Do you need a separate vector database?

Not always. If your data already lives in a database that supports vectors, keeping vectors next to the rows is simpler. You keep the joins, security and backups you already trust. A dedicated vector database earns its place at large scale or for search-heavy workloads.

What the interviewer wants. A trade-off, not a brand preference. Name what would push you to a dedicated store, such as query volume or response-time targets.

Watch out. Vector search inside a general database can be slower at huge scale. Measure with your own data before deciding either way.

Five minutes, any AI assistant

Build a tiny map of meaning by hand, so the idea stops being abstract:

Score each word from 0 to 1 on three things: money, data and fruit. Words: invoice, bill, receipt, table, column, banana. Score “Where is my invoice?” the same way. Then rank the words by cosine distance to that question.

Three dimensions instead of a thousand, and you can check every number. Watch which words land near the question, and notice that “table” is ambiguous in a way the scores expose.

Why this is worth a book

Chapter 4 has ten questions. It also covers what a dimension is, normalising a vector, and how databases store vectors. Then what happens to embeddings when the source text changes, and how to choose an embedding model.

That last one is a Senior question, and the answer is mostly about testing on your own data rather than trusting a leaderboard.

100 AI Interview Questions and Answers for Data Professionals is in Kindle (also on Amazon.in), paperback and audiobook.

Part five turns the closed-book exam into an open-book one.

An embedding is not a summary, it is a location.

Published by Pinal Dave on SQLAuthority. More of my work at pinaldave.com.