AI Companion

AI Companion is a robot built for conversation. An ESP32-S3 captures your voice through a digital microphone, streams it to a realtime AI model over Wi-Fi, and plays the reply through its own amplifier and speaker. While it talks, two servos move the head so it feels present rather than like a speaker in a box.

It is the second version of my Desk Companion. The first was built in under two weeks and ran on pre-recorded phrases. This one started again without a deadline and with one goal: something you can actually hold a conversation with.

The ESP32-S3 runs everything. Speech comes in over I2S from the microphone, goes to the AI model over Wi-Fi, and the spoken reply plays through the amplifier while the text appears on screen.

The head sits on two servos, one for pitch and one for yaw, and moves while it speaks.

There is only one button. A single press, a double press and holds of three, five and ten seconds each do something different. More buttons would have meant a bigger enclosure, so the logic moved into the firmware instead.

I built the firmware in modules: display and animation, audio capture, networking, the conversation loop and servo motion. Each was generated with AI, brought up and tested on the hardware on its own, then merged. Most of the work came after that, running it on the device and working out whether each fault was in the code or the wiring.

Getting the reply back fast enough

The first working version took the long way round. The microphone recorded what I said, speech to text turned it into words, the words went to the cloud, the answer came back as text, and text to speech turned it into audio. That is four conversions before a single word reached the speaker, and the delay made it feel like filling in a form. Switching to a realtime speech model removed the conversions at both ends, and the response time dropped to something that feels like a conversation.

Hiding the motion

I wanted it to feel like a companion, not a gadget, so the head had to move naturally and the hardware doing it had to disappear. Two servos, one for yaw and one for pitch, had to fit inside the head and neck, stay out of sight and still give the full range of movement. It took several printed prototypes, each tested for weak points and reprinted, before the mechanism worked and stayed hidden.

The reboots

The hardest fault to fix. The ESP32-S3 reset whenever too much happened at once: audio playing, Wi-Fi transmitting and both servos moving drew more current than the supply could hold, and the board browned out. I tried several buck converters. A 2 to 3 A module was not enough, and two replacements did not fully solve it either. The resets went from constant to occasional, and the next revision needs a power path designed around the peak current rather than the average.

A power supply designed around the peak current draw, so the last of the resets go away.