Kafka, Redis, and a Dashboard That Stays Honest: Inside `angelroma/kafka-redis-microservices`

The Dashboard Makes a Promise

This project starts with a user promise, not a broker diagram. You submit an order, and the dashboard behaves as if the system is one continuous machine, even though the backend is really a chain of services passing work around.

That is the useful tension here. The repository, angelroma/kafka-redis-microservices, is a compact reference architecture for the hardest part of event-driven systems: preserving a simple mental model while the data moves through many hands.

One Order, Four Systems

The repo becomes easy to understand when you follow one `orderId`. The API gateway accepts the HTTP request, stores the order, emits an `order-created` Kafka event, and pushes a live Socket.IO update. From there, the worker consumes the event, processes the order, and emits either a success or failure event that the notification service can republish for the browser.

// Gateway flow, simplified
app.post('/api/orders', async (req, res) => {
  const order = await Order.create(req.body);
  await kafka.produce('order-created', { key: order.orderId, value: order });
  io.emit('orderCreated', order);
  res.status(201).json(order);
});

Why the Gateway Does More Than Gate

The gateway is not a thin traffic cop. It is the place where the system turns a synchronous request into a durable event stream while still giving the client immediate feedback. That is a subtle but important design choice.

The code also keeps same-order events together by keying Kafka messages with the order id. That matters because real-time systems get messy fast when status updates arrive out of sequence.

const producer = kafka.producer({
  createPartitioner: Partitioners.LegacyPartitioner
});

await producer.send({
  topic: 'order-created',
  messages: [{ key: order.orderId, value: JSON.stringify(order) }]
});

The Worker Is Where the Real Work Happens

The order service is the part of the system that actually changes meaning. It consumes the event, simulates work, checks state, and then publishes a new outcome instead of hiding failure in logs.

That makes the worker more than a consumer. It is a producer of the system’s next fact. In a distributed architecture, that is the unit of accountability.

async function processOrder(order) {
  try {
    await new Promise((resolve) => setTimeout(resolve, 2000));
    // update state, then publish processed event
    await producer.send({
      topic: 'order-processed',
      messages: [{ key: order.orderId, value: JSON.stringify(order) }]
    });
  } catch (error) {
    await producer.send({
      topic: 'order-failed',
      messages: [{ key: order.orderId, value: JSON.stringify({ ...order, error: error.message }) }]
    });
  }
}

Why Redis Is Not Replacing Kafka. It Is Getting Out of the Way

This is the cleanest architectural split in the repo. Kafka carries durable, replayable events. Redis Pub/Sub handles transient fanout to live clients. They are both messaging systems, but they are not doing the same job.

That division is the teachable move. The repo does not use Kafka because it is fashionable. It uses Kafka where durability and ordering matter, then steps out of the way when the browser only needs a quick nudge.

The Frontend’s Cleverest Trick

The dashboard is not a dumb subscriber. It applies its status filter to the live event stream, which means the view stays honest even while updates keep arriving in the background.

That is the detail that turns the whole system from a demo into a lesson. The user is not just seeing data. The user is seeing a coherent slice of a changing system.

newSocket.on('orderCreated', (newOrder) => {
  if (statusFilter === 'ALL' || statusFilter === newOrder.status) {
    setOrders((prevOrders) => [newOrder, ...prevOrders]);
  }
});

This is a small bit of code with a big product effect. It prevents the dashboard from feeling noisy or self-contradictory when multiple statuses are streaming in at once.

What This Repo Gets Right, and What It Leaves Out

As a reference implementation, it is sharp. The code reads cleanly, the service boundaries are obvious, and the stack covers the full path from button click to live update. For learning how event-driven systems hang together, that is already a lot.

It is also honest about its scope. The Kafka setup is local-dev friendly, the gateway emits Socket.IO directly, and the system is not pretending to solve multi-node scaling for every layer. That restraint is a feature, not a flaw.

The Pattern This Repo Teaches

The larger lesson is simple and useful. Event-driven systems are not hard because they move data. They are hard because they must move data without disturbing the user’s mental model of what is happening.

This repository handles that problem with a crisp split of responsibilities: Kafka for durable order flow, MongoDB for persisted state, Redis for transient fanout, and the browser for a filtered, human-readable view. That is the pattern worth keeping.