tmbspaceships: Building a Digital Twin of a Technical Mind
A SurrealDB-backed graph, Claude-powered extraction, and agentic chat turn 583 tweets into a navigable map of one engineer’s physics obsession.
- tmbspaceships is trying to preserve expertise, not just archive text, by turning one engineer’s tweets into a searchable knowledge graph.
- Claude handles extraction, but the schema is where the project becomes opinionated, because it encodes topics, themes, entities, and relationships as first-class structure.
- The system is strongest when it traverses the graph in multiple hops, since that reveals conceptual bridges a flat search layer would miss.
- SurrealDB lets the project stay compact while still supporting graph queries, full-text search, and transactional storage in one stack.
The weirdly compelling thing about tmbspaceships is not that it searches tweets. It tries to preserve a technical worldview. The repository turns 583 terse posts from a retired USAF engineer into something closer to a digital twin of expertise, where concepts, entities, and themes can be queried as a living structure rather than flattened into a feed.
The project is trying to preserve a mind, not just index tweets
That distinction matters. A keyword index can tell you where a term appears, but it cannot tell you why it matters in a specialist’s internal taxonomy. This corpus is full of shorthand, recurring obsessions, and domain-specific references, so the real problem is not retrieval. It is interpretation.
The repository’s core move is to treat the posts as evidence of a worldview. Once the text is structured into a graph, the system can answer questions about relationships, not just similarity. That changes the unit of search from a sentence to a network of ideas.
Why tweet search fails for this kind of material
Flat search breaks down when the corpus is dense and idiosyncratic. A query like "antigravity" or "E-M-K axis" may surface the right tweet, but miss the surrounding cluster of related concepts that gives the phrase its meaning. Vector search helps with fuzzy similarity, but it still tends to rank neighbors. It does not naturally explain structure.
raw tweet -> Claude extraction -> structured JSON -> SurrealDB graph
Entities: people, papers, phenomena
Themes: recurring ideas across tweets
Topics: searchable buckets of discussion
Edges: cites, relates_to, belongs_to, discusses| Approach | What it is good at | What it misses | Why tmbspaceships is different |
|---|---|---|---|
| Keyword search | Exact matches and simple lookup | Meaning that depends on context | It needs relationships, not just terms |
| Vector search | Loose semantic similarity | Internal taxonomy and multi-hop reasoning | It models concepts as connected graph nodes |
| Graph traversal | Paths between ideas | Weak recall without structure | It combines traversal with extracted semantic labels |
Claude does the extraction, but the schema does the thinking
The ingestion pipeline, centered in ingest.py and schema.py, is the project’s real intelligence. Claude turns messy posts into JSON, but the schema decides what counts as a topic, what counts as a theme, and how those pieces are allowed to relate. That means the database is not passive storage. It is an interpretation layer.
The source model is deliberately lean. Raw tweets are enriched into node types like tweet, topic, theme, entity, and image, then tied together with semantic edges such as discusses, cites, belongs_to, and relates_to. The result is a graph that can be queried as if the corpus had always been structured that way.
SurrealDB collapses the stack
This project would feel much larger if it were split across a vector store, a graph database, and a relational backend. SurrealDB lets it collapse those jobs into one system. That matters because the article of record here is not infrastructure sprawl. It is how much expressive power a small stack can hold.
| Layer | Typical stack | tmbspaceships | Why it matters |
|---|---|---|---|
| Storage | Relational DB plus object store | SurrealDB | The data stays in one transactional system |
| Similarity | Vector database | SurrealDB full-text plus graph traversal | The corpus can be searched and walked |
| Reasoning | Separate agent layer | Claude with graph tools | The model can inspect structure, then answer |
| Ops | Multiple services | One lean backend | The project stays auditable and compact |
The real trick is not retrieval, it is traversal
The chat agent in chat.py is the clearest sign that this is not a conventional RAG wrapper. It can search topics, inspect details, fetch tweets for a topic, and chain tool calls when one hop is not enough. That makes it behave less like a summarizer and more like a researcher moving through a corpus.
That difference is subtle but important. Retrieval asks, "Which chunk looks relevant?" Traversal asks, "What path through the knowledge structure gets me to the answer?" The second question is harder, but it is also the right one when the corpus has internal logic.
This is also where the repository becomes legible as a knowledge-engineering project. The system is not trying to sound intelligent. It is trying to expose the steps needed to justify an answer.
| Mode | User question becomes | System behavior | Failure mode |
|---|---|---|---|
| Single-hop retrieval | One search result | Returns the nearest-looking text | Misses context and conceptual links |
| Multi-hop traversal | A route through the graph | Chains tools across topics, entities, and tweets | Can take more steps, but reveals structure |
| tmbspaceships | A navigable inquiry | The agent inspects, branches, and synthesizes | Better for dense expert corpora |
The bridge query is the most elegant part of the product
The /bridge?a=&b= endpoint is the project’s sharpest idea. It asks a graph question instead of a search question: how do two concepts connect, and what sits between them? That makes the corpus explorable in a way a typical chatbot cannot match.
A bridge query is valuable because it surfaces structure that no single tweet owns. It can show how a niche topic links to another through shared themes, entities, or related discussions. In a corpus like this, that is often the most useful answer available.
This is a boutique knowledge graph, and that is a strength
The project is scoped to one person, one voice, and one archive. That makes it modest in scale, but unusually coherent in design. A generic knowledge base has to survive every kind of content. This one only has to explain a single expert’s intellectual footprint.
That constraint is not a compromise. It is why the system can be precise. The schema can reflect the source material’s quirks, the extraction prompt can preserve its taxonomy, and the interface can focus on exploration rather than generality.
In other words, the boutique nature of the graph is the point. It is a prototype for expertise preservation, not a platform pretending to know everything.
What the project suggests about AI infrastructure
The broader lesson is that AI infrastructure gets more interesting when the scope gets sharper. You do not always need a bigger model or a more sprawling stack. Sometimes the better move is a tighter schema, a more deliberate graph, and an agent that knows how to ask follow-up questions.
tmbspaceships is a compact example of that philosophy. It turns a narrow corpus into a navigable system of meaning, and it does so with a small enough codebase that the design stays visible. That is the real appeal. The machinery is modern, but the idea is old-school: organize knowledge well, and it becomes easier to think with it.
