Lumen

Lumen is the system I built for my master’s thesis, Development and implementation of an AI agent based on large language models (Faculty of Informatics and Digital Technologies, University of Rijeka, 2025, mentored by prof. dr. sc. Ana Meštrović). The idea: the popular things an AI assistant does, from searching the web to drawing a picture, but on my own computer, with open-source models and as little reliance on outside services as possible, so your data never leaves your own infrastructure.

The agent

At its heart is a LangChain agent running Solar 10.7B Instruct, quantized and run on the GPU with llama.cpp and CUDA. Every service has to fit on one graphics card with 12 GB, so the model’s size mattered most. Solar beats models of a similar size on many benchmarks while staying small, and its instruct tuning made it follow the tool-calling format reliably.

It works in the ReAct pattern: it writes down a thought, picks an action (a tool) and its input, reads the observation that comes back, and repeats until it can give a final answer. The animation above is that loop. It remembers the conversation (earlier messages come from the database with each question), and while it works, the interface shows what it’s doing, “Searching the web…” or “Generating image…”, streamed from the server as it happens.

Its tools

• Web search through SearXNG, a private metasearch engine running in Docker on the same machine.
• Python, for the maths language models are famously bad at. Asked for 321413², it runs print(321413**2) instead of guessing. (Running code the model writes is a risk, so in production this would need a sandbox.)
• Image generation with Stable Diffusion XL and SDXL Lightning weights: a 1024 × 1024 picture in only two denoising steps.
• Email through Mailjet, from lumen@bojan-radulovic.xyz. It started on SendGrid, until SendGrid ended its free plan in May 2025, a small lesson in why the rest of the system avoids outside services.

Talking to it

You can ask by voice (the browser’s speech recognition) and it answers out loud, with a voice generated locally by Kokoro. Image generation and the voice run as separate services that the main server reaches through RabbitMQ, so either can be scaled up, or go down, without taking the rest with it. The live tools on this site work the same way.

Around it

• Server: FastAPI with an SQLite database for users, chats, messages and password reset tokens, sign-in with JSON Web Tokens, and every route checked so you only ever see your own chats.
• Accounts: there’s no sign-up page, since one GPU can’t answer a crowd quickly. An admin invites people by email, and they choose their own password on first sign-in. Forgotten passwords are reset by email too.
• Interface: React, for both phones and computers. A visualizer that moves with the voice, playback controls (speed, download), a menu of past chats, and a classic chat view with images inline.
• Publishing: an Apache reverse proxy on bojan-radulovic.xyz, the domain bought for this, with dynamic DNS for a home connection and a Let’s Encrypt certificate (which the microphone needs anyway).

It all ran on one PC: an Intel i5-12400F, an RTX 4070 SUPER with 12 GB and 64 GB of RAM. That’s also why there’s no demo here: this site lives on a much smaller machine with no GPU.

The thesis ends with what I’d do next: a bigger model for better answers and more languages than English, letting it answer from your own files (RAG), speech recognition on a local model too, and its own mail server. Every step is in the thesis (in Croatian).