secure-genai-gateway: Secure GenAI Gateway: The Firewall for Prompts, Outputs, and AI Spend
A centralized Python gateway that scrubs sensitive input, blocks risky responses, encrypts audit trails, and gives enterprises one place to govern every LLM request.
- Secure GenAI Gateway turns LLM access into a governed perimeter, not a direct API call.
- The pipeline blocks unsafe prompts before provider spend happens and rechecks model output before users see it.
- Regex-based filters and encrypted audit logging favor speed, predictability, and operational control over cleverness.
- Offline emulation makes the gateway useful in local development and CI without depending on live cloud credentials.
Most teams still bolt safety onto LLM features one app at a time. That works until privacy, compliance, and cost land on different owners and nobody can explain who approved what. This repo takes a harder position: if enterprise AI is going to be real infrastructure, it needs a perimeter.
The AI firewall enterprises keep reinventing
The project’s strongest idea is the GenAI DMZ. Put one gateway between business apps and model providers, then force every request through the same policy chokepoint. That shifts governance out of scattered microservices and into a single control plane.
In the repo’s own framing, the gateway is designed to sit between production applications and LLM providers such as AWS Bedrock, OpenAI, and Claude. That matters because the dangerous part of GenAI is not only the model call. It is everything that can leak into it, and everything that can leak back out.
One request, four gates
The orchestration engine in src/gateway.py runs a strict sequence: ingress analysis, upstream dispatch, egress analysis, then hardening. If the prompt trips a security rule, the request is blocked before the provider call. If the response looks unsafe, it is stopped before the user sees it.
That is the real product move. The gateway does not just proxy traffic. It decides whether traffic should exist at all, and it leaves an encrypted trail behind either way.
The filters are simple on purpose
The security filters lean on precompiled regex patterns in config rather than model-based detection. That is a conscious tradeoff. Regex is fast, predictable, and easy to reason about in a gateway whose job is to make hard calls quickly.
| Concern | Ad hoc app integration | Secure GenAI Gateway |
|---|---|---|
| PII scrubbing | Handled differently in each service | Centralized before provider calls |
| Prompt injection checks | Often inconsistent or missing | Applied at ingress in one place |
| Output inspection | Usually skipped | Checked before the response returns |
| Audit logging | App-specific and uneven | Encrypted centrally |
| Provider switching | Requires code changes across apps | Concentrated in the gateway layer |
| Policy ownership | Spread across teams | Owned as a shared control plane |
| Offline development | Usually depends on live APIs | Can emulate locally without credentials |
This is not trying to be clever AI security. It is trying to be boring in the best way: cheap to run, easy to audit, and hard to bypass.
The gateway assumes the model is untrusted
Most products stop at prompt filtering. This repo also treats the model response as an attack surface. That matters because leakage can happen in both directions: sensitive input on the way in, and restricted or revealing output on the way out.
The egress check makes the gateway feel more like a security boundary than an SDK. If the model tries to echo secrets, policies can still catch it before exposure.
Why offline emulation is the underrated feature
One of the most practical details in src/llm_client.py is offline emulation mode. If AWS credentials are missing, the gateway can keep running locally or in CI instead of breaking the workflow or burning provider credits.
That makes the project friendlier to development, testing, and policy iteration. It also reveals the product instinct behind the code: this is meant to be an operational layer, not a demo wrapper around a model endpoint.
The repo also includes encrypted audit logging with Fernet-backed protection, IAM-based runtime credential resolution, and a Docker setup with a non-root user. Those are small signals, but together they point to a project that is thinking like infrastructure, not like a prototype.
What this replaces, and what it does not
Compared with app-level integrations, a centralized gateway reduces duplicated policy logic and makes governance visible. Compared with a traditional API gateway, it is opinionated about AI-specific risks like prompt injection, output leakage, and auditability.
It is not trying to be a full agent platform. It is trying to be the control layer in front of one. That narrow scope is why the repo works as an argument: enterprises do not just need model access. They need a perimeter around it.
| Architecture | Best at | Weakness |
|---|---|---|
| App-level LLM calls | Fast feature shipping | Policy drift across services |
| Traditional API gateway | Routing and throttling | No AI-specific safety checks |
| Secure GenAI Gateway | Ingress and egress governance | May be more than small teams need |
