Dot products and cosine similarity: geometry behind relevance
From angles between vectors to embedding similarity, follow a simple piece of linear algebra into search and AI systems.
Learning goal
Compute dot products and cosine similarity, and explain magnitude versus direction in a search problem.
Prerequisites
- Vectors and matrices
- Basic algebra
- Basic geometry
Dot product
The dot product combines the magnitude and relative direction of two vectors.
Geometric interpretation
θ is the angle between the vectors. Dot product therefore depends on both angle and vector magnitude.
Cosine similarity
Dividing by the norms removes the scale effect and focuses the comparison on relative direction.
Numerical example
The cosine similarity is 0.8 and the angle is about 36.9 degrees.
Connection to search and embeddings
In retrieval, a query and a document can be represented as vectors and compared with a similarity measure. Similarity choice and embedding quality still need validation against real data.
Connection to the Elasticsearch project
The Elasticsearch Search Platform focuses on relevance, text analysis, filtering, and explainability. This lesson is one mathematical foundation for vector retrieval, not a complete relevance model.
Exercises
- Compute cosine similarity for (1,1) and (1,0).
- Construct an example where dot product is large but cosine similarity is lower.
- Explain why normalization is useful when comparing vector direction.
Related projects
Elasticsearch Search Platform
Relevance and search behavior are implemented in an inspectable platform.
Open sourceRelated solutions
Working on search or representation?
Similarity and retrieval can be evaluated against the real data and requirement.