Apple Intelligence impresses now, and it's still very early
At WWDC, Apple introduced Apple Intelligence as an expansion of its AI capabilities. It's incredibly impressive in action, and if you let it, it will change how you work.
Apple Intelligence is coming this fall
The span and ambition of Apple Intelligence are staggering now, and it's not even in the betas yet. This project that has been years in the making can help you search video content, answer personalized questions, and create any image you can image.
The keynote at the annual developer conference spent nearly half of the run time of the presentation on these features. After that, we also got to see it firsthand.
Writing tools with Apple Intelligence
As one who needs to bash out words for part of my job, the writing tools of Apple Intelligence stood out. They work anywhere there is a text field.
Select some text and just as you'd see the cut/copy/paste buttons, there will be Apple Intelligence writing tools.
You'll find it in a blogging app, an email, messages, or in the browser. Apple Intelligence will proofread your work, rewrite it, or adjust the tone.
Apple Intelligence can help give you confidence in your writing
When proofreading, it will highlight each change. You can step through them individually and learn what changed, or you can accept them all at once.
If you rewrite the text, you can choose a style that gives you a different take on what you had. For example, if you take a nasty email to a neighbor, you can resculpt it with Apple Intelligence it to be more professional, more neutral, and the like.
In our trial, Apple Intelligence took an internship application email and swapped out the intro, exclamation points, and rephrased some questionable sentences to make it sound far more professional than what was originally written.
In our demo, it looked just as good as it did in Apple's keynote. We'll be comparing it against Grammarly and Microsoft Word's grammar and spelling checkers in the future.
Working with images via Apple Intelligence
There are three primary ways that Apple Intelligence works with images. It can make Genmoji characters, it can create unique images based on a description, and it can edit your photos.
Automatically editing your photos is long overdue, with Google and Samsung touting AI abilities to manipulate your shots. Now, Apple is taking its first steps into the arena.
Apple's goal is still clearly to maintain the original intent of the photo. So you're not going to be adding in people, opening someone's eyes, turning a crying baby into a smiling one, or anything like that.
Apple Intelligence clean up tools helps remove distractions
You can choose a photo, and it will scan the photo automatically, looking for any distractions. There's another great-looking animation as it processes the image and highlights these distractions.
Just tap any of them, and they'll be removed, filling in the pixels in their absence. Apple put a lot of effort into retaining textures, finding the edges of hair, and other nuances to make edited images in Photos look as natural as possible.
Most distractions will be bigger objects like people walking in the background, a bird that flew into the frame, and other photo-bombers. Blemishes, can also be potentially removed.
It will depend on various circumstances that we'll be sure to test on our own when we can. But, since you can select distractions manually, something like a red dot on your face can likely be removed.
Generating entirely new images is a different beast. Apple even created a dedicated app -- Image Playground -- to create these images.
Generate images in three distinct styles
A picture can be created in three styles -- sketch, animation, and illustration. Notably, photo realistic isn't amongst the options.
Apple trained their generative models on publicly available art, licensed art, and commissioned art to specifically match what it was going for.
When an image is generated, it performs multiple checks for adult or copyrighted content. This happens when it is submitted, on the remote servers, and before it's sent to the user.
Users can provide feedback, too, if something goes awry. Images generated are tagged as created with Image Playground in the EXIF data, as we've already discussed.
Image Playground lets you generate images
Our main takeaway after the time we had with Image Playground was just how well it worked. Images came back very fast, and when they did, users could quickly move between the different styles on the fly.
If you add more criteria to the image, it will iterate and evolve new ones. And if you remove any, it backpedals to the previous version it had created.
The whole thing felt fun, easy, and approachable. We expect that this will be the most obvious tool that people use and share results of, as Apple Intelligence rewritten text is far more difficult to spot.
Apple Intelligence and Siri
Starting in iOS 18 and macOS Sequoia, Siri leverages Apple Intelligence to improve performance, accuracy, and what the voice assistant can do.
Arguably, this may be the most substantial portion of the Apple Intelligence repertoire. There are millions of queries done per-day by Apple customers now -- with varying success.
Siri will be able to tap into your personal information, all locally on the device. This means that it can be far more aware of you and your needs.
It's able to answer questions about your texts, your emails, photos, videos, schedule, and more. Importantly, this is something no other AI assistant is capable of doing with the same degree of privacy.
It starts with a new slick invocation animation.
Siri now animates from the edges with iOS 18
When invoked via the side button, Apple has added these subtle effects where it looks like you're pressing into the screen. After a brief pause, an animation spreads around the edge of the display that's colorful and lively.
It's a step up from the glowing orb. And, it extends to the volume buttons too.
One big change is how natural it feels to talk to Siri. You can stumble across your words or even change your mind mid-query, and Siri will understand. Apple demonstrated this in the keynote.
Siri was asked to "Set an alarm for 7 AM, sorry, I actually meant 5:45." Sure enough, an alarm was set for 5:45.
Context is also improved, with the assistant retaining awareness across multiple questions.
Siri was asked about the San Francisco Giants, and it understood who "they" were when asked "When do they play next" after the initial query. Siri even knew what to do when we ended by asking "add that to my schedule."
These interactions within apps are using App Intents, something developers can utilize as well. They also give unique looks to each interaction.
Double-tap the bottom to type to Siri
Each interaction was brisk and fluid, even in this unreleased beta, which was particularly impressive.
Some of these interactions, though, will be limited, depending on the apps you use. For example, you may have these complex, context-aware conversations in some apps like Apple Home.
You still can't yet say, "turn on the lights in the living room and open the shades 30 percent." These things will come, though, and developers can do more by utilizing App Intents.
ChatGPT answers the hard questions
While Siri seems to have a great grasp on your personal knowledge, there are times when more info proves useful. That's where ChatGPT's optional integration comes into play.
How it works is you ask a question. Siri chews on it for a minute and suggests that ChatGPT may be better at helping.
It then explicitly asks you every time if you'd like the query passed to ChatGPT. Contrary to some fearmongering already going on, ChatGPT isn't privy to every query you make.
If you approve the use of ChatGPT, and only if you approve it, Siri will then pass along limited info about the query. It includes photos if applicable, before returning your answer.
Siri can leverage ChatGPT to help answer questions
There are few important things to note about this. Users have ultimate control.
ChatGPT must be enabled in settings and will ask for permission each time it is used. A user's IP is also obfuscated each time, so OpenAI can't match you to past searches, which Apple has explicitly prohibited as per the OpenAI's contract terms.
Finally, OpenAI is not allowed to train on these searches either.
However, this all changes if you log in with an OpenAI account. If you do, you're bound by the policies you agreed to when creating that profile.
We'll see in the fullness of time if the responses that ChatGPT change when logged into an OpenAI account. We'll be testing that as the summer progresses.
A promising start, but not a universal one
We left our demo blown away by how impressive Apple Intelligence was. It permeates the entire operating system of macOS, iOS, and iPadOS, and will touch so many of your daily tasks.
Many of these touches we didn't discuss here here. Mainly, the intelligent management of Mail, or sorting your notifications based on what's important.
Apple Intelligence is only going to expand
Most of the AppleInsider staff are deeply interested in the new features. We're expecting the features to be gated behind a wait list, though.
But, regardless of that wait list, we're not going to get it on our HomePods in 2024. And, probably not on the existing devices in the family at all.
If you have a HomePod -- or multiple -- they won't have the benefits of Apple Intelligence or the updated Siri. That extends to other devices, too, like your Apple Watch or any unsupported Macs or iPhones.
In the interim, you'll want to invoke Siri from your iPhone manually so that your HomePod doesn't take over. You'll have to adjust your queries based on the device you're talking to.
These are just growing pains, though. The future is bright, and there is much room for Apple Intelligence to expand.
Apple Intelligence will be launching with the beta of iOS 18 this summer before launching as a beta feature this fall. It works on all Macs with an Apple Silicon processor, and the iPhone 15 Pro and iPhone 15 Pro Max.
Read on AppleInsider
Comments
I see that this article wasn’t proofread…
Looking forward to a time when AI articles are proofread using AI…
It wouldn’t surprise me in the least to find that Apple is working on their own solution with the intention of no longer needing to use ChatGPT or some other third-party.
What I do see Apple doing is trying to pass all this off as their own doing to the unaware masses who don't know.
PCC is the ultimate marriage of vertical integration, stellar hardware and software capes, and a deep commitment to privacy. Nobody else can match it. The biggest threat to PCC doesn't come from competitors but from brain dead regulators (hi there EC).
That’s ridiculous. I mean, totally ridiculous. You really need to know something before posting nonsense. Apple has been doing this well before anyone else. They call it by its proper term; machine learning. Artificial intelligence is just a term dreamed up a time ago to sound more exciting for the masses.
if you look at what Apple us doing here, you’d see that they’ve been working on nit for years. But Apple has learned from the Maps fiasco to not introduce a major initiative before it’s ready. Look at what Microsoft and Google did with theirs. Both companies had to withdraw their LLMs shortly after introducing them because of major problems. Google’s engineers even wrote a letter to management n telling to not bring it out because it wasn’t ready and there would be problems. They’ve now withdrawn AI search until they fix it. Microsoft has withdrawn the latest Windows 11 update because of a security nightmare and there’s no information as to when Recall will come out, if it ever will, which is what some people in the industry believe.
so Apple doesn’t have the talent? Really, that’s garbage.
Really good question I'd also like to understand better.
Oh good god, they announced a new set of features, it's being talked on a rumor site and you post this silly nonsense? Troll harder.
I don't believe we're back to the days of Copland - the macOS that never shipped. At least Apple paid hotel+flights for me to attend a demo.
These days most products do ship with AirPower being one that only made it to a keynote.
But shipping outside US are where things get ugly. Apple is active in 175 countries. iPhone 14 got satellite SOS. 2 years later and 16 countries (9%) are covered. In most of the world customers pay for hardware that does nothing.
Look at this mess of a KB-post on availability of services: https://support.apple.com/en-us/118205
Google Gemini is available in 150 countries supporting around 30 languages. MS Azure AI is doing 143 combinations.
Apple Intelligence is scheduled for "en-us" and that is it. Apple Intelligence is mainly FUD to those of us not running on en-us.
2) Not unless you want it to. Your paranoia regarding cloud services sure is something.
3) Maybe, and I sure would prefer an Apple-owned "world knowledge" model versus the current model which will be sending off those queries to 3rd parties like OpenAI, Microsoft, etc (as we know, other 'modules' are coming as deals are made). Sherlocked? Not sure that applies, as there's already competition in this space and Apple plans to offer other models. If they add their own, it would still be entirely possible they keep offering third party models as well, like search engines in Safari.
They're doing nothing of the sort. If a query doesn't fit on-device AI or Private Cloud Compute, a dialogue comes up that very specifically asks if you'd like to send the query to ChatGPT, which you have to approve every single time.
Software helping us humans to process information comes in many forms. And has for a long time.
Apples AI is a combination of several such processing paradigms, as is most other recently marketed services.
On-device processing and privacy är where Apple shines.