Voice AI agent capabilities have been steadily improving. One of the things that has genuinely surprised me is the quality of voice; both in AI voice cloning and in AI generated voice agents.
This week's challenge is to experiment with voice AI from the inside. You've likely heard the news about job cuts in customer service, booking, and task management. The likelihood that you've already encountered an AI voice agent in the real world is almost 100%. But seeing how these platforms work from the inside gives you a completely different perspective.
You may or may not choose to publish anything, but I hope this challenge helps you get into an AI agent platform and better understand the capabilities and limitations of voice.
To do this for myself, I built an agent that calls me every week, asks me a set of questions, and then analyzes the transcripts and creates notes for me like a phone-based journal. There are already apps that do voice-powered journaling, so the goal was less about creating an entirely new workflow and more about getting you to play around with a voice agent platform firsthand. The challenge is structured around building a journal prompting agent, but if that doesn't resonate, I encourage you to explore voice in whatever way feels most useful to you.
Before we start: if you are getting this email forwarded to you and are not yet part of the lab - you can join the tiny AI lab here.
The Challenge:
Build a voice agent, give it a script, and have it call you. The point is to get inside one of these platforms, understand how they work, and hear what an AI voice actually sounds like in real conversations that you can edit.
If you want a starting point for what the agent should do, the weekly reflection format works well: three short questions, five minutes, done. But you can give it any script you want.
Before you start — one thing to know: The platforms require you to set up a phone number before they can call you. It usually costs around $2/month through the platform. It takes about two minutes to set up but it's a step worth knowing about before you begin. If you don’t want to pay to set up a phone number you can just use the test function in the app.
TOOLS TO TRY
All three options below have free tiers and work without any code. Pick whichever feels most approachable the setup is similar across all of them. You can also ask your regular AI of choice which option would be best for you using the prompt below:
Retell AI — easiest to get a call live quickly. Good first stop if you've never done this before.
Bland AI — slightly more setup but more control over the call flow.
ElevenLabs — best voice quality of the three. Worth it if you want to hear how natural AI voice can actually sound.
The prompt to get you started
Copy this and run it in your preferred AI system:
"I want to build a weekly AI voice agent that calls me and asks reflection questions at the end of each week. Help me with three things in order:
1. Platform: Recommend the best no-code platform for outbound voice agents (free or low cost). I'm non-technical.
2. Questions: Write 3 reflection questions in a warm, conversational tone. Focus on [wins / challenges / intentions — pick one or describe your focus].
3. Setup: Give me step-by-step instructions to build this on the platform you recommended, including how to configure the phone number and trigger the first call."
Too Easy?
If the core challenge felt quick or too easy, keep going. Ask AI to store the transcripts for you then help you keep them organized in a format that works best for you.
"My voice agent on [Retell AI / Bland AI / ElevenLabs] saves a transcript after every call. I want to automatically store these in [Notion / Airtable / Google Sheets — pick one] so I can search and read back over time.
Help me with two things in order:
1. Format: Recommend how to structure each entry (e.g. date, summary, full transcript, key themes). Keep it simple and searchable. 2. Automation: Give me step-by-step instructions to connect my voice platform to [storage tool] automatically after each call. Assume I have no automation experience and am non-technical."
Why this matters?
On the opportunity side: Voice AI is moving faster than most people realize. Recent reports by Gartner predict that organizations will replace 20-30% of service agents with Generative AI in 2026. Companies like Salesforce and Amazon have already made support cuts citing AI as the primary reason. The people who understand how these systems work, what they can do, where they break down, how to configure them are going to be in a good position to see around corners. Whether your job involves customer experience, operations, HR, or anything that touches a phone or a workflow, understanding this technology from the inside gives you a real advantage in deciding where to use it and where organizations may be looking to expand use.
On the protection side: Voice cloning has crossed what researchers are calling the "indistinguishable threshold" meaning human listeners can no longer reliably tell a cloned voice from a real one. The practical consequences are already here. In recent news reports a UK-Based Firm lost over 200,000 Euros after an employee received a voice call from the CEO. People have been scammed with voice cloning of family members or friends and voice clones can be created with as little as 3-5 seconds of audio.
Building a voice agent yourself, even a simple one changes your relationship with these tools. You understand how they're constructed, what the tells sound like, and how they mimic emotion, urgency, patience leveraging human speech. That awareness I hope is genuinely useful in both helping you find opportunities and protect yourself against the harms of voice AI.
That’s it for this week.
Hit reply and tell me what you thought of the challenge this week.
I read every response, and your experience helps me shape these challenges. If you get stuck or have questions, reach me at [email protected]