Wayfarer: An AI Travel Companion That Actually Picks Up the Phone
By bridging the gap between LLM reasoning and the global telephone network, Wayfarer transforms the travel assistant from a search engine into a functional concierge.
- Wayfarer automates real-world travel logistics by connecting LLM reasoning to the global telephone network via Twilio and ElevenLabs.
- The system bypasses training data cutoffs by injecting real-time local context from Google Places and Firecrawl into the AI's prompt.
- A Node.js WebSocket proxy maintains sub-second latency by facilitating a direct WebRTC connection between the telephony provider and the voice engine.
- The backend ensures reliable agent performance through a prioritized scraper waterfall that extracts contact details from multiple web sources when APIs fail.
The Glass-Box Concierge
Most travel applications treat communication as an exercise left to the reader. They present a static phone number and a call button, immediately shifting the burden of language barriers and timezone math onto the user. Wayfarer takes a dramatically different approach. It uses a low-latency voice engine and automated telephony to actually perform the real-world chore of calling a restaurant or hotel.
The iOS client provides a window into this autonomous action. As the Node.js backend negotiates a reservation in Italian via Twilio and ElevenLabs, the SwiftUI interface renders a live transcript over a WebSocket connection. The user watches the conversation unfold in real time, transforming a black-box AI operation into a transparent, glass-box experience.
Beyond the Knowledge Cutoff
An AI calling a restaurant is only useful if it knows what to ask. Standard LLMs suffer from training data cutoffs, making them oblivious to today's weather, local holidays, or temporary closures. Wayfarer solves this through aggressive context injection.
Before a call begins, the backend extracts the user's GPS coordinates and feeds them into Google Places and Firecrawl. Firecrawl scrapes local news and event pages, compiling an ephemeral snapshot of the destination. This data is injected into the AI's system prompt, ensuring the digital concierge speaks with immediate, local relevance rather than relying on stale training weights.
The Low-Latency Dial Tone
The architectural core of Wayfarer is its WebSocket proxy. To maintain the sub-second latency required for a natural phone conversation, the backend cannot afford to intercept and process every audio frame sequentially. Instead, it establishes a direct WebRTC connection between the telephony provider and the ElevenLabs Conversational AI API.
The Node.js server sits in the middle as a stateful observer. It intercepts the initial handshake to inject the payload containing current city, country, and latitude. From that point on, the proxy simply monitors the stream, forwarding ping and pong frames to keep the connection alive and translating complex AI events into simplified JSON for the mobile client.
When APIs Fail: The Scraper Waterfall
The physical world is messy, and official APIs often fail to capture it. Google Places frequently hides direct phone numbers or restricts access to reservation links. To ensure the AI agent always has a valid target, Wayfarer implements a multi-strategy discovery layer.
The backend utilizes a prioritized fallback system. It begins with high-confidence sources like the Google Places API. If contact information is missing, the system cascades down to specialized scrapers targeting TripAdvisor, Yelp, and regional directories like PagineGialle for Italy. Each source returns a hardcoded confidence score, allowing the orchestration layer to select the most reliable phone number before initiating the outbound call.
| Feature | Traditional Travel App | Wayfarer |
|---|---|---|
| Data Source | Static database or single API | Live scraped data with fallback waterfall |
| Call Action | Opens dialer for the user | Triggers an autonomous AI agent |
| Language Barrier | User must speak local language | Agent speaks 15+ local languages automatically |
| Context Awareness | Relies on manual search | Injects real-time weather and local events |
A New Interface for the Physical World
Wayfarer represents a shift in how we interact with software. The 'Call' button is no longer a terminal endpoint where the app's responsibility ends and the user's begins. Instead, it becomes an implementation detail of a broader orchestration engine.
By seamlessly integrating web scraping, low-latency text-to-speech, and automated telephony, the project demonstrates that an AI assistant can be more than a chatbot. It can be a functional actor in the physical world, navigating legacy communication networks so the user does not have to.