The Challenge
Voice-First Complexity
Salons need hands-free booking—clients call while multitasking. Voice transcription had to be accurate, fast, and handle accents.
Multi-Step Conversations
Booking requires collecting service type, stylist preference, date/time, and contact details—all through natural conversation flow.
VIP Client Segmentation
Manual tracking of high-value clients was unreliable. Needed automated 4-tier VIP system based on booking history and spend.
The Solution
Conversational AI with GPT-4o + Deepgram
Built a voice-enabled conversational agent using OpenAI Assistants API and Deepgram for real-time speech-to-text. The system guides clients through multi-turn booking conversations, extracting structured data with function calling.
- ✓Voice & Text Support: Deepgram transcription with fallback to text chat
- ✓Session Management: Persistent conversation threads survive page refreshes
- ✓Structured Extraction: Function calling converts natural language to booking objects
// Define booking function for AI to call
const tools = [{
type: "function",
function: {
name: "create_booking",
description: "Create a salon booking",
parameters: {
type: "object",
properties: {
service: { type: "string" },
stylist: { type: "string" },
datetime: { type: "string" },
name: { type: "string" },
phone: { type: "string" }
},
required: ["service", "datetime", "name", "phone"]
}
}
}];
// AI decides when to call function
const run = await openai.beta.threads.runs.create({
thread_id: threadId,
assistant_id: assistantId
});Key Features
Intelligent Booking Flow
Conversational Interface
Natural language booking with GPT-4o. Handles voice input via Deepgram and maintains conversation context across sessions.
Service Selection
Dynamic service catalog with pricing. AI suggests services based on conversation context and client history.
Smart Scheduling
Date, Time & Stylist Selection
Real-time availability checking with preferred stylist matching. System handles timezone conversions automatically.
Booking Confirmation
Stripe-powered payment integration with automated confirmation emails and calendar invites sent to clients.
Technical Architecture
4-Tier VIP System
Automated client segmentation based on booking frequency and lifetime value. VIP status unlocks priority booking, exclusive stylists, and personalized offers.
Standard access, basic support
Priority scheduling, 5% discount
Exclusive stylists, 10% discount, complimentary services
Concierge service, 15% discount, private appointments
Real-Time Voice Processing
Deepgram provides sub-second transcription with 95%+ accuracy. WebSocket connection streams audio to backend, transcriptions fed directly to GPT-4o for immediate response generation.
// WebSocket connection to Deepgram
const deepgram = createClient(apiKey);
const connection = deepgram.listen.live({
model: "nova-2",
language: "en-AU",
smart_format: true
});
connection.on("Results", (data) => {
const transcript = data.channel.alternatives[0].transcript;
// Send to OpenAI Assistant
await sendMessageToAssistant(threadId, transcript);
});Technology Stack
Frontend
Backend
Database
AI/ML
Integrations
Deployment
Results & Impact
Key Learnings
- →Voice requires context management: Multi-turn conversations need persistent session state to avoid repetition
- →Function calling is powerful but brittle: Clear parameter definitions and validation prevent booking errors
- →VIP automation drives retention: Automated segmentation increased repeat bookings by recognizing loyalty
- →Real users reveal edge cases: Testing uncovered timezone bugs, accent challenges, and unclear AI prompts that needed refinement
Explore More Projects
See how I've applied AI and full-stack development to other challenges
Next: Retreat Matchmaker