// notes
Research notes
How the things I research and build actually work, explained with pictures: from my master's thesis, my projects, and the papers I'm reading for the MEDIANA project.
- Embeddings: meaning as a point in spaceHow text becomes a list of numbers where similar meanings land close together, how "close" is measured, and where it falls short.
- How RAG worksHow a chatbot answers from your own documents instead of from memory, step by step, with the settings behind Emma.
- Positional encoding and RoPE: how a transformer knows word orderAttention on its own can't tell "dog bites man" from "man bites dog". How position gets in, from the original sine waves to the rotations used by today's models.
- AI agents: a language model with tools and a loopWhat turns a chatbot into an agent, and the ReAct pattern that lets it decide which tool to use, from my master's thesis.
- Topic models before embeddings: LDA and its time-aware cousinsHow LDA finds topics by shuffling words between them, how dynamic topic models follow topics through time, and why neither can spot a brand new one.
- Stemming vs lemmatisation, and why Croatian needs the secondBefore counting words, their many forms have to become one. Chopping endings off is fast and often wrong; looking up the dictionary form is slower and right, and in Croatian the difference is big.
- From embeddings to topics: UMAP, HDBSCAN and c-TF-IDFHow a pile of articles becomes a handful of named topics without telling the computer what the topics are, the three steps inside BERTopic.
- Spotting emerging topics with BERTrendHow a pipeline of embeddings and clustering notices a new story while it's still small, and tells it apart from noise and from topics everyone already covers.
- Using an LLM as a judgeWhen there's no right answer to check against, can a language model grade the results? What the MT-Bench and Chatbot Arena paper found, and the biases to watch for.