Note
What Are Embeddings?
A beginner-friendly explanation of how embeddings turn text into numbers that computers can compare.
Purpose
Explain what embeddings are in a simple way, without going too deep into vectors, math, or implementation details.
This note builds on the idea from semantic search:
Computers can search by meaning when text is converted into a form they can compare.
Key Idea
An embedding is a way to turn text into numbers that represent meaning.
The numbers are not important by themselves. What matters is that similar ideas create similar number patterns.
Explanation
Computers do not naturally understand meaning the way humans do.
A person can read this question:
How do I stay consistent while learning?
and understand that it is related to:
Building a daily learning routine
Even though the words are not exactly the same, the ideas are close.
A computer needs help making that kind of comparison.
That is where embeddings come in.
An embedding model reads text and converts it into a list of numbers.
Simple flow:
Text
↓
Embedding model
↓
Numbers that represent meaning
For example:
Building a daily learning routine
might become something like:
[0.12, -0.45, 0.88, 0.03, ...]
I do not need to understand every number yet.
The important idea is:
Similar meaning → similar number pattern
Different meaning → different number pattern
Simple Analogy
Think of embeddings like giving every idea a location on a map.
Ideas that are similar are placed near each other.
For example:
Learning habits
Daily routines
Staying consistent
Practice plans
These ideas would be close together.
An unrelated idea like:
Pasta recipe
would be far away.
The embedding does not store the sentence as normal text. It stores a number-based representation that helps the computer understand where the idea belongs on the meaning map.
Why Computers Need Numbers
Computers are good at comparing numbers.
They are not naturally good at understanding human meaning.
So instead of asking a computer to directly understand this:
How do I stay consistent while learning?
we first convert it into numbers.
Then we also convert stored notes into numbers.
After that, the computer can compare the numbers and find which note is closest in meaning.
How Embeddings Help Search
Semantic search uses embeddings in two places:
- Stored notes are converted into embeddings.
- The user's question is converted into an embedding.
Then the system compares the question embedding to the stored note embeddings.
Simple workflow:
Stored notes → embeddings
User question → embedding
Compare embeddings
Return closest match
This is how a system can find a useful note even when the question uses different words.
Example
Stored note:
Building a daily learning routine
User question:
How do I stay consistent while learning?
These are different words, but the meaning is close.
An embedding helps the computer recognize that closeness.
Another stored note:
Grocery list for the week
This is not close in meaning, so it should rank lower.
Connection To Semantic Search
Semantic search is the search experience.
Embeddings are one of the tools that make it possible.
Simple relationship:
Embeddings create the meaning numbers.
Semantic search uses those numbers to find related content.
So when I say semantic search helps computers search by meaning, embeddings are the hidden step that makes the meaning searchable.
Connection To Qurio
For Qurio, embeddings could eventually help connect:
- notes
- experiments
- build logs
- questions
- lessons learned
This would make the project easier to search as the content grows.
Instead of only searching for exact words, Qurio could help find related ideas across the learning journey.
For example, a future Qurio search could connect:
How do I stay consistent with projects?
to notes about:
daily learning routines
building in public
small experiments
documentation habits
Even if those notes use different wording.
Practical Use
The practical takeaway is not that I need to understand the math immediately.
The practical takeaway is that embeddings are a bridge between human language and computer comparison.
They let a computer treat meaning as something it can measure.
Related Content
Takeaways
- An embedding turns text into numbers that represent meaning.
- Similar ideas create similar number patterns.
- Embeddings help computers compare ideas instead of only matching exact words.
- Semantic search uses embeddings to find related content.
- Embeddings are one of the foundations behind AI memory, document search, RAG, and personal knowledge systems.
Source Material
This note was informed by Lena Voita's NLP Course, especially the Word Embeddings section.
The course is useful because it explains NLP concepts in a self-paced, visual, and research-oriented way. It connects well with this note because embeddings are one of the first steps in understanding how language can be represented numerically.
Source:
Topics to revisit from this source:
- Word embeddings
- Distributional semantics
- Word2Vec
- GloVe
- How embeddings are evaluated
- How embedding ideas connect to later NLP topics like language modeling, attention, and transfer learning
Questions to Explore Next
- What is a vector?
- How does similarity comparison work?
- What is cosine similarity?
- What is chunking?
- How are embeddings stored?
- Can I create embeddings locally with Ollama?