modelctl: The Rust CLI That Treats Local LLMs Like System Services

A thin, transparent manager for detached model servers, runtime adapters, and Apple Silicon telemetry.

6-8 min read • View on GitHub • More from wrale

By Repo Explainer • AI-assisted • Published by Ron Kurti

Published Apr 12, 2026 • Local LLM Inference

A terminal control room where a model server is placed into a dock and then continues running after the terminal closes, marked by a PID tag and a small status light. The scene explains that modelctl manages local LLM servers as detached services rather than as one-shot launches. — The key idea is not just starting a server. It is keeping it alive, trackable, and inspectable after the shell is gone.

Key Takeaways

modelctl’s core idea is to manage local LLM servers like ordinary Unix services, with state, liveness, and shutdown behavior that stays visible.
Its value comes from being thinner than a bundled launcher, because it asks for the real binary and leaves the runtime architecture exposed.
The runtime trait keeps the tool open-ended, so new model backends can fit without rewriting the CLI around a single engine.
Apple Silicon telemetry turns the TUI into an ops tool for hardware tinkerers, not just a start-stop wrapper.

A service manager for local models

Most local model tools try to feel like apps. modelctl feels like infrastructure. It launches an LLM server, detaches it, records the PID, and lets you ask whether the process is still alive. That is a small shift in interface, but a big shift in mental model.

The result is closer to systemctl than to a consumer launcher. You are not surrendering control to a black box. You are managing a process that happens to speak tokens.

Why this feels different from Ollama

Dimension	modelctl	Bundled launcher
Underlying binary	You point it at the actual server binary.	The engine is usually packaged behind the app.
Process model	Detached process plus PID state on disk.	Managed as part of the application shell.
Extensibility	Runtime adapters make new backends feel natural.	Support tends to follow the vendor’s roadmap.
Configuration	TOML and explicit flags.	More opinionated, more hidden defaults.
Audience	Power users and tinkerers.	People who want a smoother all-in-one experience.

A split composition showing a sealed black box on one side and an open rack of visible parts on the other, including a binary path, config file, PID file, and telemetry meter. The image explains the contrast between opaque bundled launchers and a transparent service manager. — The comparison is philosophical as much as technical. modelctl is built for visibility, not convenience theater.

The process manager under the hood

The mechanics are simple on purpose: spawn a process, detach it, write state to disk, then verify liveness when you need it.

The interesting part is not that the code can fork a child process. The interesting part is that it does so with discipline. In start, the child calls setsid() in a pre_exec hook, which creates a new session so the server survives after the CLI exits.

State lives on the filesystem, not in a database. A PID file under ~/.local/state/modelctl/<name>.pid is enough to answer the only question that matters: is this server actually running?

// Simplified idea of the lifecycle
let child = Command::new(binary)
    .args(extra_args)
    .pre_exec(|| {
        nix::unistd::setsid().map_err(|e| std::io::Error::from_raw_os_error(e as i32))?;
        Ok(())
    })
    .spawn()?;

write_pid_file(name, child.id())?;

// Later
if is_alive(pid)? {
    println!("running");
} else {
    println!("stale pid file");
}

A runtime trait that keeps the tool open-ended

The Runtime trait is the quiet architectural choice that makes the whole project less fragile. The CLI does not need to know whether the backend is llama_cpp, something custom, or something that has not been wired in yet. It only needs a way to build the command.

That keeps the control plane small. The tool can stay opinionated about process management while staying neutral about the engine underneath it.

Telemetry for people who tune hardware

Generating illustration...

Telemetry is not bolted onto the side. It is sampled separately and fed into the dashboard as a live signal.

This is where the project becomes more than process supervision. The metrics thread samples Apple Silicon residency data in the background, then feeds the TUI without blocking it. For people tuning local inference, that means the dashboard is not decorative. It is operational.

The design stays read-only by choice. The dashboard observes. The command layer acts. That separation is part of the tool’s discipline.

Where this tool fits in the local LLM stack

User need	Best fit	Why
Point-and-click setup	A bundled launcher	It hides the plumbing and gets you running fast.
Scriptable control	modelctl	It exposes the process model and stays CLI-native.
Custom model binaries	modelctl	It does not assume one blessed runtime.
Ad hoc debugging	Either tool	But the CLI wins when you want repeatable commands and logs.

That is the point of the tool. It is not trying to be the easiest path for everyone. It is trying to be the right layer for people who already think in processes, paths, and services.

The Unix philosophy, translated for local AI

Local AI is starting to look less like a novelty and more like an operations problem. Once models run on your machine, you need the same boring things you need everywhere else: lifecycle control, observability, and a way to recover from stale state.

modelctl answers that with a small Rust binary and a very Unix answer to a new problem. No bundle. No mystery. Just enough machinery to start, track, and observe a model server cleanly.