Your Groq key
Paste it once. It's saved to a .env file in this folder and never
leaves your computer except in requests to Groq.
Which provider?
The orchestrator speaks plain OpenAI-compatible HTTP, so the provider is just a URL and a model name. Nothing in the call logic knows the difference — which is what stops one vendor's rate limit from being an outage.
Groq is cheaper per token; Cerebras has a far bigger free daily allowance. Both are sub-second. Switching takes effect on the next call — no restart.
Which model should speak?
The only number that matters on a phone call is how long the caller sits in silence before the agent starts talking. This measures that from your computer, on your network — three runs each, on a realistic mid-call turn. Budget is 800 ms.
On cost: a 2.5-minute intake call costs about $1.75 all-in at
your measured $0.70/min. Every model here prices the whole call at well under
a cent, so the model is a rounding error — and Llama 3.3 70B is the
dearest option on your account ($0.59/$0.79 per million against
$0.075/$0.30 for GPT-OSS 20B), not the cheapest. Pick on latency.
Why this isn't obvious: every model Groq offers is a reasoning model,
and reasoning tokens are silence on a call — the caller waits while it
thinks invisibly. The app already turns reasoning down as far as each model
allows (none on Qwen, low on GPT-OSS). Also note
the model only has to phrase one short question — your orchestrator
already decided which question and checked the rules. That's an easy job,
which is why the small model usually wins here.
Don't have a key yet?
- Go to console.groq.com and sign in.
- Left sidebar → API Keys → Create API Key.
- Copy it right away — Groq shows it once and never again.
- Paste it above and hit Save & test.
The test tries three models and picks a working one
automatically. You may see llama-3.3-70b-versatile come back as
retired — Groq shut it down on 16 August, which is why this checks rather
than assumes.
Talk to it
Type what a caller would say. This is the same code a real phone call runs.
A real call always carries a caller ID, so the phone slot is pre-filled and the script only asks you to confirm it. Tick the box to test a blocked number — then it has to ask for the digits instead. Changing either restarts the call.
Memory
Anything green is remembered and will never be asked about again.
Worth trying
- Give your county, then stay quiet about it. It should never ask twice.
- Say "no kids." That's an answer, not a blank — it shouldn't ask again.
- Say you're in Fresno County. Not on Schank's list — it should decline warmly and offer a referral rather than book you.
- Ask if they handle bankruptcy. Should say no and refer out.
- Refuse to give your phone number. It should move on, not nag.
Point Retell at this app
Retell has no dashboard setting for this. The Custom LLM option isn't in their UI at all — the response engine is a field on the agent record, set through their API. That's what these buttons do.
- Leave this app running. It already is.
- Start ngrok in a terminal:
ngrok http 8080— it prints a line likehttps://a1b2-c3d4.ngrok-free.app. - Paste both below and press the buttons.
1 · Retell API key
Retell dashboard → Settings → API Keys. If you see more than one kind, you want the backend / server-side one — the public key is for browser web calls and can't create agents. Give it Agent: Edit (and Phone: Edit only if you'll bind a real number).
Not sure which you've got? Paste it and press Save — this checks what the key can actually do and tells you.
2 · Your ngrok address
Paste the https://… line. I'll convert it to the websocket
URL Retell needs. The URL carries a private secret segment — the tunnel
accepts nothing else, so the setup pages here are never reachable from
outside this machine.
3 · Create the agent
Makes a new agent pointed at this app. It never edits an existing one, so nothing you already have is touched.
Then open that agent in Retell and use the Test Audio tab for a browser call — no phone number needed, and your sandbox number keeps ringing whatever it rings today. To use a real number instead, bind it below.
4 · Optional — point a real number at it
Saves the current binding first, so Undo puts it back exactly.
Waiting for a call
When a call connects, the conversation appears here as it happens.
Memory, live
Each firm is one file in
config/firms/ — the same file the call routing reads. Saves are
versioned; “Make live” switches which firm THIS line answers as, on the
next call. Editing a firm that is not live changes nothing on the phone.
What the firm offers
Tick what's true. The agent's answers are generated from this — change a box and what it says to callers changes everywhere, with no restart.
Unchecked matters are politely declined and referred out — a DVRO caller is only booked if DVRO is ticked.
When a call books a consultation, a prep brief is emailed here. Blank = off. Needs RESEND_API_KEY in .env.
Each one is only asked when the answer before it applies. Takes effect on the next call.
Spoken exactly as: “The office is open .”
The built greeting already carries the AI disclosure and recording notice.
Static until the live calendar is connected — only times the firm truly holds.
What the agent will say
Live preview — the exact wording a caller hears. To use the firm's own wording for any answer, type it in the box under that answer; leave the box blank to keep the generated line.
Anything the boxes don't cover
One per line, as trigger words | the answer. The trigger words
are what the caller might say. Example:
parking, where do i park | There's free parking in the building garage.
Replay your 46 real calls
Feeds the caller audio from your QA corpus through the new slot machine and
compares what it captures against what the live agent actually shipped.
The live agent sent 14 of 46 alerts as "Unknown Unknown" and
19 with unknown@example.com.
What this can and can't tell you. It measures capture only — whether facts the caller actually said get recorded instead of dropped. It cannot tell you about turn count or booking rate, because these callers were answering the old agent's questions in the old order. Those two numbers need live calls.