You, as a vector.
Recent watchesSkips & replaysLikes & followsContext
↓
Encode interaction history
Learn useful combinations
↓
u = fθ(history, context)
AI, EXPLAINED DIFFERENTLY.
It’s not magic.
It’s math.
And a little
fruit drama.
From a strawberry’s breakup to the algorithm that knows your next obsession. Let’s open the black box.
meaning, in motion
[0.82, −0.14, 0.63]🍓
“It’s complicated.”relationship_status: drama
01 / INSIDE THE BLACK BOXDRAG TO ROTATE ↔
YOUR ROUTE THROUGH THE MACHINE
01 / THE BIG IDEA
Learning patterns.
Not reading minds.
A model learns from examples. Then it uses those patterns on something it hasn’t seen. That second part is the real test.
YOU ARE THE MODEL
2 → 4 → 8 → ?
What comes next? Pick the rule you think fits.
Examples are clues, not guarantees.
16 fits “multiply by two.” But a short sequence can fit many rules. Models also make assumptions from limited data.
TRAININGLearn parameters from examples.
INFERENCEUse those parameters on a new input.
02 / INPUTS
Different media.
Same language: numbers.
A computer doesn’t receive “a funny cat.” It receives numerical data. A model learns useful patterns within it.
UNDER THE HOOD
03 / NEURAL NETWORKS
A little error.
A little better.
Signals travel through weighted connections. Training changes the weights to reduce prediction error. A weight is a number, not a tiny thought.
INPUTHIDDEN LAYERSPREDICTION
LIVE TOY TRAINING
Will they watch?
One neuron learns from a positive example. The bigger network illustrates how many such operations connect.
Prediction σ(w)0.500
Target1.000
Loss0.693
p = σ(w) = 1 / (1 + e−w)
L = −ln(p) · w ← w − η(p − 1)
Step 0 · w = 0.000 · learning rate η = 0.5L = −ln(p) · w ← w − η(p − 1)
Go deeper: what happens inside a neuron?
A neuron computes z = Σwᵢxᵢ + b, then applies an activation function a = f(z). Backpropagation calculates how parameters contributed to loss; an optimizer updates them. Our one-neuron demo uses input x = 1 and no bias. Real training uses many varied examples and held-out evaluation. Memorizing training examples is overfitting, not mastery.
04 / EMBEDDINGS & VECTORS
Meaning has
a new address.
A vector is a list of numbers. An embedding is a learned vector that represents an item. Similarity becomes something we can calculate.
THE MEANING MAP
v = [0.82, 0.14, 0.63]
Drag the scene to rotate.
Related examples share a neighborhood.
DramaFootballBeautyComedy
Illustrative hand-positioned 3D vectors, not a trained embedding model. Real vectors can have hundreds or thousands of dimensions. Projections lose information; real dimensions usually mix learned patterns rather than correspond to named topics.
05 / THE MOROCCAN FEED LAB
“Tafaha.”
Watches part seven.
One strawberry breakup later, you’re emotionally invested. Try the feed and watch your toy interest vector change.
9:41▮▮▮
Following For you
♫ original sound · Darija edition
⌂ ⌕ ⊞ ♡ ◉
YOU’RE TEACHING THE FEED
The banana has trust issues.
YOUR TOY USER VECTOR
Interest is inferred from behavior. Enjoyment and attention aren’t always the same.
Fictional reel illustrations; creator names are audience-supplied examples, not endorsements or reproduced posts. Comments count as an action here; their text isn’t sentiment-analyzed or stored.06 / TWO-TOWER RETRIEVAL
Two towers.
One shared space.
One network represents the viewer. Another represents the video. Training makes useful matches score higher in a shared embedding space.
SHARED SPACE
u · v
cosine similarityThe reel, as a vector.
Visual featuresAudio & languageCaption / topicItem history
↓
Encode content & item features
Learn useful combinations
↓
v = gφ(video features)
Why two towers?
Video vectors can be computed ahead of time. At request time, compute the user vector, then search a vector index for promising candidates. You don’t need an expensive joint model for every video on the platform.
What changes when you scroll?
Recent actions can change the input history and resulting user representation. Retraining the underlying network weights is a separate process, not necessarily something that happens after every tap.
HOW THEY LEARN TO MATCH
Training pairs a user with an interacted-with video and contrasts it with sampled alternatives. One illustrative contrastive objective is:
L = −ln [ exp(u · v⁺ / τ)
/ Σⱼ exp(u · vⱼ / τ) ]
/ Σⱼ exp(u · vⱼ / τ) ]
v⁺ is the positive example; the denominator includes it and sampled alternatives. τ controls score sharpness. Gradient updates adjust both towers. Skipped content is ambiguous, and real sampling and objectives are more nuanced.
Architecture teaching model, not TikTok source code. Meta has documented two-tower retrieval for Instagram Explore. The feed above uses transparent, hand-written updates so you can inspect the math; it does not train two neural networks.
07 / RECOMMENDATIONS
A match isn’t
the whole story.
Retrieval finds candidates. Ranking predicts usefulness or engagement. Final selection can account for freshness, diversity and eligibility.
Content pool→Retrieve candidates→Rank→Select→Your feed
DESIGN YOUR FICTIONAL RANKER
The remaining weight rewards novelty. Move the slider to see why the nearest vector doesn’t always come first.
score = α × similarity
+ (1 − α) × novelty
+ (1 − α) × novelty
Our teaching formula; not a platform’s production formula. Scores are not calibrated watch probabilities.
YouTube
Recommendations consider viewing signals and satisfaction. Home and “Up next” use different contexts.
Read YouTube’s explanation ↗TikTok
Watch time, interactions, content information and user context help personalize recommendations. Their exact architecture isn’t reproduced here.
Read TikTok’s explanation ↗Instagram / Meta
Explore has a documented retrieval and ranking pipeline. Different Meta surfaces use different systems.
Read Meta’s engineering notes ↗08 / GENERATIVE AI
One token.
Then another.
A language model predicts a distribution over next tokens. A decoding strategy chooses one, appends it, and repeats.
ILLUSTRATIVE NEXT-TOKEN DISTRIBUTION
The strawberry left the banana for …
Fixed toy logits, not a live language model. Temperature rescales logits: Pᵢ = exp(zᵢ/T) / Σⱼ exp(zⱼ/T). Lower values favor likely choices; higher values flatten the distribution. This does not make an answer more truthful.
09 / RETRIEVAL-AUGMENTED GENERATION
Less guessing.
More receipts.
RAG retrieves external information and gives it to a model as context. It’s an open-book workflow, not automatic retraining.
01 Tokenize02 Chunk03 Embed04 Score05 Answer + cite
A / PREPARE YOUR KNOWLEDGE
This workbench uses a small word-and-punctuation tokenizer, not a production subword tokenizer. IDs identify tokens; they are not embeddings. Overlap repeats boundary tokens so useful context is less likely to be split away.
How does a real tokenizer work?
Many language models use subword tokenization: common pieces get vocabulary IDs, while unfamiliar words split into smaller pieces. Byte-pair encoding (BPE) learns frequent merges; other tokenizers use different methods. Arabic, Latin-script Darija and emoji may split differently. Tokenization rules depend on the model. Token IDs go through an embedding lookup; they are not meaning scores. A chunk is a group of tokens, not a single token.
Why chunk instead of embedding the entire book?
Smaller chunks can make retrieval more specific and fit within model context limits. Too small loses context; too large mixes topics. Overlap trades extra storage and repeated context for continuity. Real systems often respect sentences, headings, and document structure.
B / ASK, RETRIEVE, INSPECT
GROUNDED EXTRACT · NO LIVE LLM
THE SCORING FORMULA
How close is close?
Encode the query and each chunk with compatible representations. Cosine similarity compares direction rather than vector length.
cos(q, d) = Σᵢ qᵢdᵢ√(Σᵢ qᵢ²) × √(Σᵢ dᵢ²)
Real dense embeddings can capture semantic relationships without exact word overlap. This transparent demo uses term-frequency vectors so every number is inspectable; it is lexical retrieval, not a pretrained semantic model.
WORKED EXAMPLE · TOP RESULT
Cosine ranges from −1 to 1 for general vectors. These nonnegative word-count vectors produce 0 to 1. A score of 0.8 is not “80% chance of being correct.”
And after scoring?
Select top-k relevant chunks, optionally rerank with a more expensive model, and provide them with the question to a generator. Ask for citations and an explicit “not found” when evidence is missing. Retrieval can fail; a generator can still misread evidence. Check the cited source.
WHAT A GENERATOR WOULD RECEIVE
The handbook is invented for this demo, not an actual 1337 / UM6P event announcement.
10 / CONNECT THE DOTS
Now you can
explain the magic.
One last check. What does an embedding actually represent?
DATA → TRAINING → REPRESENTATIONS → PREDICTIONS
Embeddings help retrieve. Rankers help select. Generators produce. RAG connects retrieval with generation.
Explore again ↑Embeddings help retrieve. Rankers help select. Generators produce. RAG connects retrieval with generation.