Review: ChatGPT’s New Advanced Voice Mode

Was this newsletter forwarded to you? Sign up to get it in your inbox.

Usually, technology moves in increments.

An iPhone with a marginally improved camera here, a Kia Sorento with a slightly better safety rating there. When you look back after a decade, the technology has clearly advanced—but each step along the way was so small that you didn’t really notice it when it landed.

Occasionally, though, you’ll encounter a new technology that ditches the incremental. Instead, it appears to truss the future to a sturdy rope and pull it hand over hand into the present.

In moments like these, previously state-of-the-art technology fossilizes before your eyes. You can see its desiccated bones crumple together into a dusty pile that you look at with nostalgia and pity.

That’s the experience of using ChatGPT’s new Advanced Voice Mode feature—and then returning to its precambrian precursors, Siri and Alexa.

I got alpha access to it last week, and I reviewed the basics of Advanced Voice Mode—including demos of my major use cases—on YouTube and X. If you’re interested, I suggest you check it out.

I’d like to dive deeper on a few use cases that underscore the leap-forward nature of this technology. The first is self-reflection, and the second is learning. But first, let’s start with what Advanced Voice Mode is and why it’s so different from what came before it.

What is Advanced Voice Mode?

ChatGPT’s Advanced Voice Mode understands speech natively, meaning it doesn’t just read and write text. It reads and writes speech, too. This creates an experience that is distinctly better—more fluid, more fluent, and more authentic—than any other voice interaction I’ve ever had with a computer.

Advanced Voice Mode replaces ChatGPT’s standard Voice Mode, which has been around for about a year. The old Voice Mode used to work like this:

1. You speak to ChatGPT,
2. The interface turns your voice to text using a transcription model,
3. It feeds the text into its underlying language model, GPT-4, to get a response in text,
4. The interface takes the text answer from GPT-4 and feeds it into a separate text-to-speech model, and then
5. ChatGPT speaks the words back to you.

That’s a lot of steps! It caused a significant amount of latency, and it also created a lot of room for misunderstanding. When you translate speech into text, you can lose a lot of nuance. A sarcastic tone might be taken literally, or it might not discern that there are actually two speakers in the room.

As a result, Voice Mode felt a little bit like doing an escape room with your hard-of-hearing grandparent, or trying to order a medium-rare steak in English from a puzzled waiter in a small village outside of Seoul. There was a sense of distance, of being trapped—not by the limits of intelligence on the other end, but by the limits of your expressive ability and theirs. This manifested as a certain pressure in my chest.

With the old ChatGPT Voice Mode, you couldn’t stop talking for fear of being interrupted, and you had to speak loudly and clearly for fear of being misheard. You expected, more often than not, that something might be misunderstood. You were constantly catering to the needs of the model, and so it was not relaxing. (Though, to be fair, it was still better than Alexa or Siri.)

The new Advanced Voice Mode eliminates steps 2 and 4 from the process above. It can natively understand speech, so you’re just speaking directly to the language model. The biggest immediate change is that a conversation with ChatGPT feels much more authentic and responsive. When I began using it, the pressure in my chest was suddenly gone. I felt more relaxed and expansive.

This opens up a new and important use case: ChatGPT as an aid to conversational reflection.

Reflecting with ChatGPT Advanced Voice Mode 

I am usually a pretty chill and laid-back guy—that is, until you wrong me.

And, unfortunately, you will probably wrong me.