Scalability Series - Part 1

Introduction

I am a huge fan of the history of science. I can talk about it for hours and I devour podcasts and videos on the subject.

Take the three-body problem: how do three celestial objects interact gravitationally, and can we precisely predict their trajectories? Spoiler: no. There is no general analytical solution. Henri Poincare, by tackling it, discovered what we now call deterministic chaos, opening a door to chaos theory.

What does this have to do with AI? Imagine a system composed of three entities -- a server, an AI engine, and a user interface -- that must cooperate to function. Much like gravitationally bound celestial bodies: each has its role, each depends on the other two. But beware of the slightest imbalance.

This "three-body system" (server, AI engine, interface) is both a metaphor and a real project for rethinking AI architectures: distributed, robust, and sustainable.

We do not necessarily need to invoke chaos theory... although, as you will see, making this system resilient is anything but trivial.

Thinking of AI as an Ecosystem

In this article series, I present a distributed AI prototype written in Rust, which goes well beyond a simple API call to a vLLM or Ollama server.

This system is an essential building block of mAIstrow: a generative AI platform designed to give control back to users, with a focus on digital sovereignty and energy efficiency. At the heart of this approach: Rust, Go, and distributed architectures built to last.

Why Not Just Use a Standard AI Server?

Today, many AI projects rely on HTTP calls to a centralized server. It is convenient, but it has its limits:

• Fragility: a simple network outage or a server crash, and everything falls apart.
• Dependency: on external APIs, American clouds, opaque models.
• Complexity: it is hard to properly handle multiple users, simultaneous requests, queuing...

Even advanced workflows (such as with n8n) can break down if they do not rest on a properly distributed architecture.

The answer? Design an intelligent system, resilient and built for scaling. Not just a somewhat solid backend: a true ecosystem.

The Three-Body System: Anatomy of a Rust Architecture

Here are the 3 components of the system, each playing its part:

Server (written in Rust for the prototype, Go in production)

• Manages WebSocket connections
• Persists data (SQLite)
• Distributes tasks with a simple algorithm (round-robin)
• Guarantees delivery even if a client disconnects

AI Engine (Rust client)

• Receives and processes requests (e.g., splitting a sentence word by word)
• Resumes an interrupted task thanks to persistence
• Optimized to run with minimal resources

User Interface (minimalist web)

• Allows sending a request
• Receives responses via streaming
• Uses a local UUID for session persistence

The flow appears simple: the user sends a request, the server dispatches it to an engine, the engine processes it and streams the response back.

Easy, right? On paper, yes. But when one of these components goes down? When you need to prioritize hundreds of requests? When a task is cancelled mid-processing?

That is where you need to think about resilience, scalability, and energy efficiency. And believe me: it starts at protocol design time.

The Foundations of mAIstrow: Rust, Frugality, and Abstraction

This prototype is not an academic exercise: it lays the foundations for mAIstrow, a sovereign, distributed, and performant AI platform. We run inference engines locally, without third-party APIs. Less energy, more control.

Rust provides a real playground here, thanks to:

• Traits for modeling protocols, tasks, and persistence
• Asynchronous programming
• SIMD optimizations
• A rigor that forces you to think through every detail

This allows us to abstract business logic from transport and persistence layers. The algorithms stay clear, the system stays flexible.

A Holistic Engineering Philosophy

This project also reflects a personal journey: from my beginnings on a ZX81 at age 6, to my contributions to Plan 9, through trigonometric libraries for aerospace in SPARC V7 assembly, and a bug fixed in 9vx with Ron Minnich himself.

I learned that innovation is born from simplicity and good abstraction.

With DREMML, I want to pass on this vision: building systems that combine technical robustness, user empathy, and sustainability. Training, supporting, building for the long term.

What Comes Next?

The three-body system shows that we can rethink AI differently: as a distributed, resilient, and sovereign ecosystem. This is just the beginning. More is coming.

Coming Up in Future Episodes

• Part 2: Exploring distributed frameworks like Apache Kafka, Pulsar, Redpanda, and Ray.
• Part 3: Async architecture with Rust -- async traits, ergonomics, pitfalls, and performance.
• Part 4: Resilience, scaling, and frugality.
• Part 5: Chaos Engineering -- breaking everything to rebuild better.
• Part 6: Performance measurement (Valgrind, flamegraphs, etc.).
• Part 7 (maybe): Comparing Rust / Go / Node.js / Python (yes, despite the GIL).

Quick Glossary

• SIMD: Single Instruction, Multiple Data. Parallel instructions very useful for optimizing performance.
• GIL (Python): Global Interpreter Lock. Prevents parallel execution of threads in pure Python.
• Plan 9: Experimental OS from the creators of Unix. Powerful abstractions, native networking.
• Deterministic chaos: A fully deterministic system can exhibit unpredictable long-term behavior due to high sensitivity to initial conditions.