cloudflare-ai-gateway-replicate-test: The One Header That Makes Replicate Feel Synchronous

A tiny Bash script hides a bigger lesson about AI infrastructure. Put a gateway in front of an async model API, layer auth cleanly, and the whole system starts to feel native.

6 min read • View on GitHub • More from replicate

A courier carries a sealed request envelope through a checkpoint, then hands it off to a workstation on the far side. The scene explains how a gateway can reshape a request in transit without changing what the upstream service ultimately does. — The repo’s central trick is not routing. It is turning a two-step model workflow into a single request that feels normal to the caller.

Key Takeaways

A single `Prefer: wait=30` header is the real innovation, because it turns Replicate’s async prediction flow into a synchronous-feeling request.
The repo separates upstream provider auth from gateway auth, which lets Cloudflare sit in front without owning the model credentials.
This is less a library than a smoke test for a bigger idea: gateways can change the shape of developer experience, not just the route.
The minimal Bash surface area makes the architecture legible, which is exactly why the demo works as an explainer.

Most AI gateway stories start with dashboards and policy controls. This one starts with a header. In cloudflare-ai-gateway-replicate-test, Prefer: wait=30 collapses Replicate’s create-and-poll rhythm into a single blocking request, so the integration reads like ordinary HTTP even though the model call still lives upstream.

The trick: one request, not a polling loop

Replicate is natively asynchronous. You create a prediction, then you usually poll until it finishes. This script asks the server to hold the line for up to 30 seconds and return the result if it resolves in time, which removes the extra loop and makes the call feel synchronous to the caller.

set -a
source .env
set +a

: "${CF_ACCOUNT_ID:?}"
: "${REPLICATE_API_TOKEN:?}"

curl -sS "https://gateway.ai.cloudflare.com/v1/${CF_ACCOUNT_ID}/${GATEWAY_ID}/replicate/predictions" \
  -H "Authorization: Bearer ${REPLICATE_API_TOKEN}" \
  -H "Prefer: wait=30" \
  ${CF_AIG_API_TOKEN:+-H "cf-aig-authorization: Bearer ${CF_AIG_API_TOKEN}"} \
  -d '{...}'

That is the whole move. The repo is not trying to hide complexity with a framework or an SDK. It is showing how a gateway plus one well-chosen header can make an async provider behave like a normal request-response API.

What this repo actually is

The codebase is deliberately tiny: one Bash script, a .env.example, a README, and a safety net .gitignore. That minimalism is the point. This is a smoke test and a reference implementation, not a reusable library.

We're happy to announce that as of today Replicate is officially part of Cloudflare.

Andreas Jansson and Ben Firshman, Replicate Blog Authors · Why Replicate is joining Cloudflare

How the request gets reshaped

A synchronous-feeling call is assembled from an async provider, a gateway perimeter, and a single wait instruction.

The script loads environment variables, validates the Cloudflare account ID and Replicate token, then builds a Cloudflare gateway URL that mirrors Replicate’s prediction endpoint shape. From there, the headers do the interesting work: one authenticates to Replicate, another optionally authenticates to Cloudflare, and Prefer: wait=30 tells the provider how long to hold the connection open.

Two auth layers, one clean path

The separation matters. Authorization: Bearer is for the upstream provider, while cf-aig-authorization belongs to Cloudflare. That means the gateway can add a perimeter, logging, and policy without breaking the contract the model provider already expects.

A close-up of a hand filling out a shipping manifest beside two distinct seals and a stopwatch. The scene explains the difference between upstream provider authorization and gateway authorization, with the wait instruction acting like a timed hold on the package. — Two identities, one request path. The gateway adds a second boundary without forcing the upstream API to change its shape.

Why this pattern matters

Path	Request style	Auth layering	Observability	Caching / rate limiting	Developer experience	Complexity
Direct Replicate	Create, then poll	Single provider token	Provider-side only	Provider-specific	Clear, but async	Lower upfront, more client logic
Cloudflare AI Gateway + Replicate	Single request with wait instruction	Provider token plus gateway token	Gateway and provider views	Gateway can centralize policy	Feels native to the caller	Moderate, but cleaner at the edge
Generic API gateway	Depends on custom config	Usually multiple secrets	Varies by product	Often available, but generic	Can be flexible, often less model-aware	Higher when adapting AI flows

The obvious wins are observability, caching, and rate limiting. The subtler win is ergonomic: the gateway changes the shape of the interaction itself. Instead of forcing every client to learn Replicate’s async rhythm, the edge can present a simpler contract to the developer.

Where it fits in the AI stack

This repo is interesting because it sits between model provider and platform. Direct Replicate gives you a model API. A generic gateway gives you traffic control. Cloudflare AI Gateway, in this specific demo, does both while also smoothing the request pattern so the integration feels native to a shell script.

That is why the tiny hardcoded gateway slug is not a flaw in the article’s logic. It is a reminder that this is a demonstration of shape, not scale. The important thing is not that the script is reusable. The important thing is that the architecture is visible at a glance.

The tiny repo with a bigger architecture lesson

A lot of infrastructure products promise centralization. Fewer of them change the developer’s mental model. This repo shows the better version of that promise: a gateway can sit in front of an async AI provider and make the call feel simpler, cleaner, and more local without pretending the underlying system is synchronous.