🧙🏼 OpenAI's voice API: real-word use cases

Also: startups building an AI brain

Author

Dario Chincha
October 11, 2024

Howdy, wizards.

Today I’ve got some kick-ass demos from real companies using OpenAI’s new voice API + a list of startups working to create an “AI brain”.

DARIO’S PICKS

OpenAI just shared some examples on how developers have been using the new realtime API for creating AI voice apps.

Here’s 3 cool demos of how real-world companies are using the new voice API:

  • Chatbase, a platform that makes customer chatbots to put on any website, is integrating the realtime API for live voice conversations. However, it’s currently “super expensive” according to the company, so they’re working on rate-limits and other things to make it work. The usage cost of the realtime API is a widely mentioned limit to scaling currently.

  • Twilio integrates OpenAI's Realtime API to enable live voice translation in contact centers. It allows multilingual support without the need for human translators between agents and callers.

  • Hallway, a platform for creating animated AI avatars (a bit like Twitch but with AIs), uses the real-time API to give its animated characters a voice. They’re using their own tech, Expression engine, which animates and lip-syncs in real time, too.

  • Bonus: this demo is super unpolished (no interface) but shows how the realtime API can be used together with function calling to integrate third-party tools. It’s a code review that shows how, through real-time voice, you could have a financial analyst with access to company financials, Perplexity search, an advanced calculator, and more.

‎ Why it matters‎ ‎ This shows off some ways how real-time AI voice chat could be integrated into existing businesses to make it more personalised, engaging and scalable. Costs are still a significant hurdle to implementing this tech at scale though, although I expect that to improve fast over the coming months.

TOGETHER WITH MASTERWORKS

HubSpot offers an intuitive, top-rated custom relationship management platform designed specifically for small businesses.

With its user-friendly interface and robust features, you can effortlessly manage leads, track sales performance, and gain a deeper understanding of your customers. Plus, it allows you to store and manage up to 1,000,000 contacts with no limits on users or customer data, making it a perfect fit for businesses at any stage of growth.

Whether you're just starting out or scaling up, HubSpot's comprehensive set of tools will help you stay on top of customer conversations, close deals faster, and grow your business. Best of all, HubSpot’s CRM is completely free to use. It’s time to focus on what matters most: your customers. Let HubSpot handle the rest.

DARIO’S PICKS

a16z published an article by Justine Moore, where she explores the concept of “exporting your brain”—using AI to capture and translate our thoughts and experiences into something tangible. She shares her experience using ChatGPT as a daily journal to improve communication and self-understanding.

They’ve also made a great list of startups innovating in this space.

I’ve broken down this market map with direct links so you can try these tools yourself:

AI clones: Delphi, SecondSoul, SenSay

AI ownership / permissioning: Anon, Vana

‎ Why it matters‎ ‎ If you’ve used ChatGPT, you know how helpful it is when it remembers context. Imagine extending that to all your knowledge, memories, and preferences—interacting with them seamlessly using AI. This could be one of the most useful AI applications (indeed, with big privacy concerns).

PS I know it might seem like I have some kind of collaboration with a16z since I keep sharing their articles—but I don’t. They’re just great at distilling information and spotting under-the-radar companies early on. So I can’t help but share it with you!

FROM OUR PARTNERS

200+ hours of research on AI tools, prompting techniques & hacks packed in a solid 3 hour masterclass.

THAT’S ALL FOLKS!

Was this email forwarded to you? Sign up here.

Want to get in front of 13,000 AI enthusiasts? Work with me.

This newsletter is written & curated by Dario Chincha.

Affiliate disclosure: To cover the cost of my email software and the time I spend writing this newsletter, I sometimes link to products and other newsletters. Please assume these are affiliate links. If you choose to subscribe to a newsletter or buy a product through any of my links then THANK YOU – it will make it possible for me to continue to do this.