Urban-Pulse-Ai: Urban Pulse AI: The Complaint Router That Refuses to Guess

A multimodal civic stack for Bengaluru that cross-checks text, voice, and images before it ever decides where a report should go.

8 min read • View on GitHub • More from Shreyas0047

A city clerk sits at a desk while three intake streams arrive at once: a written complaint, a voice recorder waveform, and a printed photo. A mechanical routing device above the desk sends one case cleanly forward and diverts another into a manual review tray. The image explains that the system is built to route only when evidence is consistent.
Urban Pulse AI treats intake like a safety boundary, not a conversation.
Key Takeaways

Most civic apps optimize for intake. Urban Pulse AI optimizes for correct routing under uncertainty. That is a much harder problem, and a much more useful one if you care about what happens after a citizen hits submit.

The repo is built around a simple but rare idea: when evidence conflicts, the system should abstain. Instead of pretending a pothole, tree fall, and drainage complaint are the same thing, it compares signals, lowers confidence when they disagree, and sends uncertain cases to human review.

Why this complaint system is different

The usual civic intake flow is basically a form plus a classifier. Urban Pulse AI adds a safety boundary. It does not just ask, “What category is this?” It asks, “Do the signals support each other enough to automate a route?”

That changes the product shape. A normal chatbot tries to look helpful. This system tries to be hard to fool. In practice, that means a wrong answer is not treated as a successful guess. It is treated as a failure mode.

The core trick is not multimodal fusion. It is deciding when the signals are too inconsistent to trust.

A close-up of two interlocking dials, one for text and one for image, mounted over a locked routing gate. When the dials align, the latch lifts and a path opens. When they disagree, the latch drops and a folder labeled manual review catches the case. The image explains confidence calibration as a gate, not a score alone.
Agreement opens the lane. Contradiction closes it.

How the AI decides when not to decide

The heart of the stack is the decision engine in ai_service/decision_engine.py. The interesting part is not that it outputs a confidence score. It is that the score is explicitly adjusted by contradiction. If the text says one thing and the vision signal says another, confidence is penalized before any routing decision is made.

def calibrate_confidence(base_confidence, text_image_consistency, contradiction=False):
    confidence = base_confidence
    if contradiction:
        confidence -= 0.2
    if text_image_consistency == "agree":
        confidence += 0.1
    if confidence < threshold:
        return "Needs Manual Review"
    return confidence

That pattern matters because it makes refusal a first-class outcome. The system is not trying to squeeze every report into a category. It is trying to protect the city from bad automation. In a civic workflow, that is a better trade-off than being aggressively certain.

The repo also uses a broader multimodal merge step. Text classification, speech transcription, and image extraction are not just stacked together. They are compared for coherence, then fed into a gate that can route, defer, or abstain.

Why Bengaluru routing matters

Generic complaint classification is not enough. A report becomes operational only when it lands in the right ward and the right department. Urban Pulse AI hardcodes that local reality into the stack with Bengaluru-specific mapping logic.

Generic civic intakeUrban Pulse AI
Captures a complaint and assigns a broad category.Maps location to Bengaluru wards and BBMP departments.
Trusts a single model output.Cross-checks text, voice, and image before routing.
Optimizes for speed of submission.Optimizes for correctness under uncertainty.
Fails by misrouting.Fails safe by sending unclear cases to manual review.
Works anywhere in theory.Works where local administrative structure actually matters.

That local layer is the difference between a dashboard demo and a civic tool. If the system knows which ward owns the issue, and which department should receive it, the AI output becomes actionable instead of ornamental.

The system is built to degrade gracefully

The repo does not assume the heavy model will always load. The Florence-2 runtime is memory-aware, uses serialized inference, and checks available budget before bringing the vision model online. That is the kind of detail that separates a prototype from an operational system.

If the visual stack is under pressure, the service can fall back instead of collapsing the whole flow. That matters in real deployments, because civic intake cannot stop just because one model is expensive or temporarily unavailable.

The benchmark suite is the credibility anchor

The strongest sign of maturity is not the UI. It is the fact that the repo includes a benchmark dataset and evaluation scripts. The project is trying to measure itself against labeled civic issues, which is exactly what you want if you are building decision support rather than a demo.

That makes the system legible as engineering, not just product theater. The evaluation layer says the same thing as the decision gate: don’t guess when you can verify.


Urban Pulse AI suggests a better template for civic AI. Do less automatically, but do it with more evidence. Preserve human judgment where the signals are weak. Route only what is confident enough to route, and make the uncertainty visible when it is not.