WWDC 2024: Apple Intelligence 5 Major Features

This article is originally published here.

WWDC just concluded, and wow, that was AI-packed!

This year’s Apple Worldwide Developers Conference (WWDC) has been one of the most exciting and highly anticipated events ever, especially with all the buzz about AI. Apple has been notoriously tight-lipped about its AI developments while other companies were tripping over themselves to launch their AI products.

Today, the wait is over. We have finally seen the fruits of Apple’s meticulous work on AI. Here are my top five picks of all the AI announcements made by Apple at WWDC 2024:

1. Siri with ChatGPT
2. AI-Powered Writing Tools
3. Genmoji: AI-generated Emojis
4. Image Playground: Apple’s AI Image Generator
5. New Photos App

Siri with ChatGPT

Siri has received some long-needed major upgrades: a more natural way of speaking, smarter Apple Intelligence generative AI models, and a brand new UI/UX.

What’s even cooler? Siri will now have on-screen awareness of what you’re currently looking at and can take in-app actions. For instance, if you’re filling out a form that asks for your driver’s license number, Siri can find a picture of your license, extract the number, and fill it in for you automatically.

Wait… Siri will monitor my on-screen activity. That’s kind of creepy.

Siri will also get a visual makeover. Instead of the usual symbol, it now opens with a glowing light around the edges of your iPhone’s screen. Thanks to upgraded language comprehension capabilities, you’ll be able to speak to Siri “more naturally” than ever before.

If Siri can’t answer a question, you can connect it to OpenAI’s ChatGPT-4o model for free. Apple promises that your info and request details won’t be logged, keeping your privacy intact.

Take a look at how it works below:

Is this the reason why OpenAI made ChatGPT’s core features and most powerful model free for everyone? Apple gives you an assurance that privacy protections are built in for users who access ChatGPT —your IP address is obscured, and OpenAI won’t store requests.

Apple also announced the App Intents API, which provides developers the functionality to deeply integrate their app’s actions and content with system experiences across platforms, including Siri, Spotlight, widgets, controls, and more.

You can learn more about App Intents API here.

New Writing Tools

Apple is integrating new AI-powered writing capabilities across apps and nearly everywhere you write, including Mail, Notes, and third-party apps.

Rewrite

The new Rewrite feature will generate alternate versions of your draft, letting you pick one that better matches your style or tone. Based on the preview, Rewrite will appear as a small blue button whenever text is selected.

Click it, and you’ll see options to make your writing more “friendly,” “professional,” or “concise.” You can even prompt Rewrite to handle specific requests, like “Write this as a poem.”

Proofread and Summarize

RIP Grammarly and Quillbot! Apple’s Proofread feature offers grammar, word choice, and sentence structure suggestions for free across iOS. It provides edits with explanations to help improve your writing.

For longer pieces, the Summarize tool can extract key points for a quick review or create a TL;DR for recipients. It also works in your inbox, summarizing long emails so you can grasp the essential points of each message quickly.

Apple also updated the Smart Reply feature. It isn’t new, but the recent upgrades powered by Apple Intelligence make it even more accurate.

So, these new and free features baked into the OS just killed third-party writing tools out there.

Genmoji: AI-generated Emojis

Taking emojis to an entirely new level, Apple introduced Genmoji.

Users can now create original emojis to express themselves just by typing a description, and their Genmoji appears along with additional options.

Here are some examples:

You can even create a Genmoji of friends and family based on their photos. Just like emoji, Genmoji can be added inline to messages or shared as a sticker or reaction in a Tapback. This adds a fun, personalized twist to your messaging experience.

From my experience, AI-generated icons or emojis are still a hit-and-miss even with the most powerful AI tools and models. I wonder how well Apple will execute this feature.

AI-Powered Photos App

Searching for photos and videos is now more convenient with natural language support. For example, you can search for specific moments by saying, “Frank playing with her dog in the park.” The search in videos is enhanced, allowing you to find specific moments within clips.

With Memories, users can type a description, and Apple Intelligence will write a story with chapters based on themes from the photos, arranging them into a movie with a matching song suggestion from Apple Music.

For someone who has thousands of photos buried in my gallery, this one’s a pretty sweet feature, I can’t wait to try it.

According to Apple, user photos and videos remain private on the device and are not shared with Apple or others.

Image Playground

Image Playground is Apple’s answer to Midjourney, OpenAI’s Dall-E, or Google’s Imagen. It’s a text-to-image tool powered by Apple Intelligence.

It is integrated into apps like Messages and Notes, and is available as a standalone app, it allows for easy experimentation with various concepts and styles. Users can type descriptions, select personal photos, and pick their favorite style to generate images.

Here’s what the standalone app looks like:

There’s no mention of Apple using their own image model, so they might be using OpenAI’s Dall-E 3.

In Notes, the Image Wand feature in the Apple Pencil tool palette turns sketches into delightful images.

Rough sketches can be transformed into cooler images, and users can even select empty spaces to create an image using context from the surrounding area.

If you’re a developer, Apple will be releasing Image Playground API to integrate it into your apps. This is pretty exciting.

How to get access to Apple Intelligence?

I know you and I are both excited to try out these new features. Hold on a moment, because there are some important details you need to know first. These AI-powered features won’t be available on all devices, and the system requirements might be a bit of a surprise.

On the iPhone side, only the iPhone 15 Pro and iPhone 15 Pro Max are getting access to Apple Intelligence. So, if you’re using an older iPhone, even one that supports iOS 18, you won’t be able to get access.

For iPad and Mac users, the requirements are also pretty specific. You’ll need a device with the M1 chipset or later. That means older iPads and Macs won’t be able to take advantage of Apple Intelligence either.

The first beta of Apple Intelligence will be coming this fall.

Final Thoughts

Overall, I was impressed with what Apple announced at WWDC 2024.

The technology showcased is undoubtedly advanced, but some of it feels like catching up rather than groundbreaking innovation. Many of these features have been available in some form from other services: Office 365 has rewrite capabilities, Google Photos allows searches by description, WhatsApp has had generative stickers for a while, and Google Photos introduced a magic eraser feature long ago.

While I appreciate the seamless integration of these capabilities into the OS, none of it feels particularly jaw-dropping. However, I am hopeful to see some cool applications powered by the exposed APIs.

What are your thoughts on these announcements? Which feature are you most excited about? Personally, I'm thrilled about the integration of ChatGPT and the new writing tools.

‍