🧙🏼‍♂️ Meet Apple Intelligence

Howdy, wizards.

I’m launching Tool Picks. A new section of this newsletter featuring our weekly top-pick of AI tools, rigorously tested and reviewed by our editorial team. We only recommend tools that are so good, they deserve to be part of your toolbox. You can follow our new reviews here (remember to bookmark).

In this edition, we’ll cut through the noise and look at the most important, AI-related updates from Apple’s WWDC 2024 keynote.

1. Apple Intelligence - Apple’s own AI system. Very useful and hands-down brilliant marketing.

2. Siri’s AI comeback (starring ChatGPT). The new Siri feels like ChatGPT but better.

3. AI-powered writing & image tools. Not groundbreaking, but a leap forward in terms of UX.

4. The privacy aspect of it. Apple is a trustworthy bridge for everyday users to AI.

5. The non-AI stuff they released. Voice isolation, spatial photos, iPhone mirroring and more.

6. GPTs: top newcomers on whatplugin.ai

But first, let me introduce you to Tool Picks –

 Tool Picks 

The best AI tools we’ve tried recently. We personally test and review AI tools for the tasks you need them the most. Read about how we test & review.

🥇 SciSpace

Best for: Academic Research

Summary: Search engine for science that offers generous free access and has its own copilot

Runners up: 🥈 Consensus, 🥉 Elicit

***

🥇 Gamma

Best for: Creating presentations with AI

Summary: Gamma can generate stunning presentations in a matter of minutes, and is quite affordable

Runners up: 🥈 beautiful.ai, 🥉 Simplified

***

🥇 Gizmo

Best for: Making flashcards for studying with AI

Summary: Lets you generate and customise flashcards easily, allows you to generate unlimited cards for free

Runners up: 🥈 PDF2Anki, 🥉 Revisely

 Dario’s Picks - Apple WWDC 2024

1. Apple Intelligence - Apple’s own AI system

Apple (finally) launched its own AI system. They’re not calling it AI, though, it’s Apple Intelligence, and it will be deeply integrated across its operating systems and devices. It will enable a heap of new capabilities across Apple’s own apps – AI-powered Siri, writing & image tools and much more.

Apple Intelligence highlights (I get into the nitty-gritty in the sections below):

• It can take actions across your apps

• It has all your personal info and context

• It can use different language models securely, whether on-device or third-party

• It will enable AI-powered Siri, writing & image tools and more

Apple Intelligence is coming in Beta this fall (probably 5-7 months into the future). Will only be compatible with quite new versions of iPhone/iPad/Mac in order to be used, see model list here (scroll to the bottom).

Why it matters 

Apple didn’t really release a ton of groundbreaking features – but they still did something clever. They’ve sewed the best of all the existing use cases for AI deeply into their own ecosystem, where it’ll have way more personal context (your routine, relationships, communications, ++) than standalone tools.

The branding of this experience as Apple Intelligence under the tagline “AI for the rest of us” is hands-down brilliant marketing.

Given Apple’s distribution (2.2 billion active devices worldwide), this is massive news for AI adoption and for the user experience of AI.

***

2. Siri’s AI comeback (starring ChatGPT)

Apple Intelligence will power Siri, enabling it to take actions, and giving it on-screen awareness and personal context about you.

AI-powered Siri highlights:

• Actions within and across apps: Siri will be able much better at taking action within apps (“enhance this photo for me”) and across your apps (“add this photo to an email”).

• Also works across non-native apps, thanks to App intents - a framework that lets developers of third-party apps define a set of actions for Siri, Shortcuts, Spotlight and other native apps.

• On-screen awareness: you’ll be able to ask Siri about anything currently on your screen, and have it take actions based on it.

• Personal context: Siri will be able to pull data from your apps – thanks to a new semantic index of things like your emails, messages, photos, calendar events and files.

• ChatGPT built-in as a feature in Siri. Since Apple’s on-device AI isn’t as powerful as the frontier models, Siri has the ability to tap into the knowledge of GPT-4o when needed. Before sharing any information with ChatGPT, Siri asks for your permission.

  • It decides whether to process your request on-device or in the cloud, based on its complexity; more on this further down, where I talk about privacy.

  • The ChatGPT integration will be available for free, and for ChatGPT subscribers, you’ll be able to sync your account and access paid features within Apple’s apps.

  • Apple also stated that they’re intending to add support for other AI models in the future.

Why it matters

There’s strong consensus of AI moving in the direction of being able to take actions and having personal context; and I think Apple and Siri are very well positioned to lead the way (in terms of distribution, UX and security).

AI-powered Siri looks a lot more like ChatGPT now but better – fully integrated with your devices and apparently has better privacy.

***

3. AI-powered writing & image tools

Apple Intelligence is powering new AI writing tools helps you write better, and a new set of image generation capabilities across apps.

• Writing tools: system-wide text editing tools that will let you proofread, summarise, rewrite, and more – or simply just describe the change you want to make to the text. You’ll also have the option to generate brand new text using the “Compose with ChatGPT” feature.

• Smart Script: Feature that cleans up your handwriting appearance with AI and spellchecks handwritten notes.

• Math Notes: New feature in the iPad Calculator app allowing handwritten equations to be solved using AI.

• Image Playground: A new, on-device image-generator available inside Apple’s apps, such as Messages, Notes, Keynote and Pages. It’ll also get its own dedicated app. It combines the freedom of prompting it to create an image, with options for pre-defined concepts like themes, costumes and accessories. It also has previews before generating your final image.

• AI-upgrade to Photos: semantic search for images and videos, and a clean up tool that removes unwanted background objects in photos.

• Genmojis: Generate AI-powered emojis right from your keyboard. Use as stickers, as reactions in messages, or inline in your messages.

Why it matters

The new writing tools aren’t groundbreaking but they’re taking the best features from the AI writers already available on the market, and seamlessly integrating them everywhere you write. Again, a leap forward in terms of UX.

Image Playground is a clever take on image generation that’s probably better for most people than DALL-E, Midjourney and the like. The pre-defined concepts, simplification of styles, ability to preview, and the fact that it’s available right inside the apps you will likely use the generated image (such as Messages) makes it easier and more enjoyable to use.

***

4. The privacy aspect of it

Apple Intelligence has a ton of personal information about you, yet it’s secure (according to the company, anyways). It has the ability to retrieve and analyse the most relevant data from across your apps, as well as referencing the content you’re currently viewing. It uses a semantic index to selectively feed your data to generative models so that they can best assist you.

When you make a request, it analyses whether or not it can be processed on-device. For on-device processing, the privacy aspect is similar to other processes on your iPhone: you’re in control of your data, where it is stored and who can access it. Your data isn’t stored or sent to Apple.

If it needs greater computational power than Apple’s native AI, it sends relevant data to privacy-enhanced cloud-based models that can handle more complex requests (e.g. ChatGPT). For cloud-based processing, Apple announced Private Cloud Compute: a more private and secure way to use these models, as they run on servers created using Apple Silicon. It requires the servers it interacts with to be publicly logged for inspection.

Why it matters

Apple’s promise of “a new standard of privacy in AI” addresses a key concern for most consumers. It’s especially relevant given the seemingly never-ending sequence of dramas and screw-ups by the leading AI players.

With their reputation still intact and massive distribution, Apple might be the perfect bridge to introduce everyday users to AI, both on-device and more powerful cloud-based systems.

***

5. The non-AI stuff they released

Here’s some cool, new things Apple released that weren’t primarily AI related.

• New passwords app: Keeps track of passwords across devices.

• macOS Sequoia will allow mirroring of your iPhone.

• VisionPro: Spatial photos and ultrawide Mac display on visionOS 2.

• AirPod Pro gets Voice isolation and Siri interactions.

• Real-time translations on Apple watch: a great one for travellers.

• iPhone customization: Freely place app icons, lock certain apps, ++.

• iMessage upgrades: Schedule messages, SMS messaging via satellite, tapback, text formatting (bold, underline, italicise).

 GPTs 

Top newcomers on whatplugin.ai’s top 1,000 list of GPTs in the last week. Read about how rankings work.

 The wizard’s favourite AI newsletters 

what i’m reading right now

TLDR 💨 - essential tech news in 5 mins a day. Read by 1.2 million people

simple.ai - 🕵🏻‍♂️ - deep-dives on AI by Hubspot’s co-founder

Bagel Bots 🥯 - best hands-on tips & tricks

The Neuron 😸 - easy weekday read on AI’s latest developments

Connecting Dots ⠎ - good one on business strategy & innovation

Affiliate disclosure: To cover the cost of my email software and the time I spend writing this newsletter, I sometimes link to products and other newsletters. Please assume these are affiliate links. If you choose to subscribe to a newsletter or buy a product through any of my links then THANK YOU – it will make it possible for me to continue to do this.