Apple Intelligence impresses now, and it's still very early

Posted:
in iOS

At WWDC, Apple introduced Apple Intelligence as an expansion of its AI capabilities. It's incredibly impressive in action, and if you let it, it will change how you work.

Apple Intelligence being announced at WWDC by Craig Federighi
Apple Intelligence is coming this fall



The span and ambition of Apple Intelligence are staggering now, and it's not even in the betas yet. This project that has been years in the making can help you search video content, answer personalized questions, and create any image you can image.

The keynote at the annual developer conference spent nearly half of the run time of the presentation on these features. After that, we also got to see it firsthand.

Writing tools with Apple Intelligence



As one who needs to bash out words for part of my job, the writing tools of Apple Intelligence stood out. They work anywhere there is a text field.

Select some text and just as you'd see the cut/copy/paste buttons, there will be Apple Intelligence writing tools.

You'll find it in a blogging app, an email, messages, or in the browser. Apple Intelligence will proofread your work, rewrite it, or adjust the tone.

Writing an email with Apple Intelligence's help
Apple Intelligence can help give you confidence in your writing



When proofreading, it will highlight each change. You can step through them individually and learn what changed, or you can accept them all at once.

If you rewrite the text, you can choose a style that gives you a different take on what you had. For example, if you take a nasty email to a neighbor, you can resculpt it with Apple Intelligence it to be more professional, more neutral, and the like.

In our trial, Apple Intelligence took an internship application email and swapped out the intro, exclamation points, and rephrased some questionable sentences to make it sound far more professional than what was originally written.

In our demo, it looked just as good as it did in Apple's keynote. We'll be comparing it against Grammarly and Microsoft Word's grammar and spelling checkers in the future.

Working with images via Apple Intelligence



There are three primary ways that Apple Intelligence works with images. It can make Genmoji characters, it can create unique images based on a description, and it can edit your photos.

Automatically editing your photos is long overdue, with Google and Samsung touting AI abilities to manipulate your shots. Now, Apple is taking its first steps into the arena.

Apple's goal is still clearly to maintain the original intent of the photo. So you're not going to be adding in people, opening someone's eyes, turning a crying baby into a smiling one, or anything like that.

Removing a person in the background with Apple Intelligence
Apple Intelligence clean up tools helps remove distractions



You can choose a photo, and it will scan the photo automatically, looking for any distractions. There's another great-looking animation as it processes the image and highlights these distractions.

Just tap any of them, and they'll be removed, filling in the pixels in their absence. Apple put a lot of effort into retaining textures, finding the edges of hair, and other nuances to make edited images in Photos look as natural as possible.

Most distractions will be bigger objects like people walking in the background, a bird that flew into the frame, and other photo-bombers. Blemishes, can also be potentially removed.

It will depend on various circumstances that we'll be sure to test on our own when we can. But, since you can select distractions manually, something like a red dot on your face can likely be removed.

Generating entirely new images is a different beast. Apple even created a dedicated app -- Image Playground -- to create these images.

Three different styles of image creation
Generate images in three distinct styles



A picture can be created in three styles -- sketch, animation, and illustration. Notably, photo realistic isn't amongst the options.

Apple trained their generative models on publicly available art, licensed art, and commissioned art to specifically match what it was going for.

When an image is generated, it performs multiple checks for adult or copyrighted content. This happens when it is submitted, on the remote servers, and before it's sent to the user.

Users can provide feedback, too, if something goes awry. Images generated are tagged as created with Image Playground in the EXIF data, as we've already discussed.

Image Playground is a new app in iOS 18
Image Playground lets you generate images



Our main takeaway after the time we had with Image Playground was just how well it worked. Images came back very fast, and when they did, users could quickly move between the different styles on the fly.

If you add more criteria to the image, it will iterate and evolve new ones. And if you remove any, it backpedals to the previous version it had created.

The whole thing felt fun, easy, and approachable. We expect that this will be the most obvious tool that people use and share results of, as Apple Intelligence rewritten text is far more difficult to spot.

Apple Intelligence and Siri



Starting in iOS 18 and macOS Sequoia, Siri leverages Apple Intelligence to improve performance, accuracy, and what the voice assistant can do.

Arguably, this may be the most substantial portion of the Apple Intelligence repertoire. There are millions of queries done per-day by Apple customers now -- with varying success.

Siri will be able to tap into your personal information, all locally on the device. This means that it can be far more aware of you and your needs.

It's able to answer questions about your texts, your emails, photos, videos, schedule, and more. Importantly, this is something no other AI assistant is capable of doing with the same degree of privacy.

It starts with a new slick invocation animation.

Siri has a new edge animation in iOS 18
Siri now animates from the edges with iOS 18



When invoked via the side button, Apple has added these subtle effects where it looks like you're pressing into the screen. After a brief pause, an animation spreads around the edge of the display that's colorful and lively.

It's a step up from the glowing orb. And, it extends to the volume buttons too.

One big change is how natural it feels to talk to Siri. You can stumble across your words or even change your mind mid-query, and Siri will understand. Apple demonstrated this in the keynote.

Siri was asked to "Set an alarm for 7 AM, sorry, I actually meant 5:45." Sure enough, an alarm was set for 5:45.

Context is also improved, with the assistant retaining awareness across multiple questions.

Siri was asked about the San Francisco Giants, and it understood who "they" were when asked "When do they play next" after the initial query. Siri even knew what to do when we ended by asking "add that to my schedule."

These interactions within apps are using App Intents, something developers can utilize as well. They also give unique looks to each interaction.

Typing a question to Siri
Double-tap the bottom to type to Siri



Each interaction was brisk and fluid, even in this unreleased beta, which was particularly impressive.

Some of these interactions, though, will be limited, depending on the apps you use. For example, you may have these complex, context-aware conversations in some apps like Apple Home.

You still can't yet say, "turn on the lights in the living room and open the shades 30 percent." These things will come, though, and developers can do more by utilizing App Intents.

ChatGPT answers the hard questions



While Siri seems to have a great grasp on your personal knowledge, there are times when more info proves useful. That's where ChatGPT's optional integration comes into play.

How it works is you ask a question. Siri chews on it for a minute and suggests that ChatGPT may be better at helping.

It then explicitly asks you every time if you'd like the query passed to ChatGPT. Contrary to some fearmongering already going on, ChatGPT isn't privy to every query you make.

If you approve the use of ChatGPT, and only if you approve it, Siri will then pass along limited info about the query. It includes photos if applicable, before returning your answer.

ChatGPT giving an answer inside of Siri
Siri can leverage ChatGPT to help answer questions



There are few important things to note about this. Users have ultimate control.

ChatGPT must be enabled in settings and will ask for permission each time it is used. A user's IP is also obfuscated each time, so OpenAI can't match you to past searches, which Apple has explicitly prohibited as per the OpenAI's contract terms.

Finally, OpenAI is not allowed to train on these searches either.

However, this all changes if you log in with an OpenAI account. If you do, you're bound by the policies you agreed to when creating that profile.

We'll see in the fullness of time if the responses that ChatGPT change when logged into an OpenAI account. We'll be testing that as the summer progresses.

A promising start, but not a universal one



We left our demo blown away by how impressive Apple Intelligence was. It permeates the entire operating system of macOS, iOS, and iPadOS, and will touch so many of your daily tasks.

Many of these touches we didn't discuss here here. Mainly, the intelligent management of Mail, or sorting your notifications based on what's important.

Apple Intelligence uses your personal information
Apple Intelligence is only going to expand



Most of the AppleInsider staff are deeply interested in the new features. We're expecting the features to be gated behind a wait list, though.

But, regardless of that wait list, we're not going to get it on our HomePods in 2024. And, probably not on the existing devices in the family at all.

If you have a HomePod -- or multiple -- they won't have the benefits of Apple Intelligence or the updated Siri. That extends to other devices, too, like your Apple Watch or any unsupported Macs or iPhones.

In the interim, you'll want to invoke Siri from your iPhone manually so that your HomePod doesn't take over. You'll have to adjust your queries based on the device you're talking to.

These are just growing pains, though. The future is bright, and there is much room for Apple Intelligence to expand.

Apple Intelligence will be launching with the beta of iOS 18 this summer before launching as a beta feature this fall. It works on all Macs with an Apple Silicon processor, and the iPhone 15 Pro and iPhone 15 Pro Max.



Read on AppleInsider

Alex1N
«1

Comments

  • Reply 1 of 30
    blastdoorblastdoor Posts: 3,432member
    One thing PCC makes me think — neural interface. Accessing cloud compute resources with your thoughts sounds pretty powerful, but clearly you want that to be highly secure. That would be a good space to lead in. 
    watto_cobra
  • Reply 2 of 30
    aderutteraderutter Posts: 618member
     We'll see in the fullness of time if the responses that ChatGPT change when logged into an OpenAI account. ”

    I see that this article wasn’t proofread… 

    Looking forward to a time when AI articles are proofread using AI…


    Stabitha_ChristierueM68000williamlondonAlex1Nwatto_cobra
  • Reply 3 of 30
    But, regardless of that wait list, we're not going to get it on our HomePods in 2024. And, probably not on the existing devices in the family at all. 


    If you have a HomePod -- or multiple -- they won't have the benefits of Apple Intelligence or the updated Siri. That extends to other devices, too, like your Apple Watch or any unsupported Macs or iPhones.

    Is that confirmed or is it possible HomePods will handoff queries to our iPhones like they already sometimes do. For instance, asking a HomePod where somebody is will use Find My on my iPhone to actually get the info from and then it will tell me. Maybe Apple Intelligence will work similarly with a HomePod?
    williamlondonappleinsideruserAlex1Njony0watto_cobra
  • Reply 4 of 30
    boboliciousbobolicious Posts: 1,154member
    "Users have ultimate control."
    Do all roads still lead to iCloud and Apple servers ?
    Will ChatGP ultimately end up being 'Sherlocked'...?

    williamlondon
  • Reply 5 of 30
    "Users have ultimate control."
    Do all roads still lead to iCloud and Apple servers ?
    Huh? All roads don’t lead to iCloud or Apple servers. You can turn iCloud off. It’s also possible to use some other service. If you don’t want to sync photos to iCloud, use Google or something else. Same goes for your contacts and calendar and passwords, etc. Nothing is stopping you from that.

    Will ChatGP ultimately end up being 'Sherlocked'...?
    It wouldn’t surprise me in the least to find that Apple is working on their own solution with the intention of no longer needing to use ChatGPT or some other third-party. 
    williamlondonssfe11Penzijony0watto_cobra
  • Reply 6 of 30
    indieshackindieshack Posts: 333member
    noun: vapourware; noun: vaporware
    1. software or hardware that has been advertised but is not yet available to buy, either because it is only a concept or because it is still being written or designed.
    williamlondonnubus
  • Reply 7 of 30
    Will ChatGP ultimately end up being 'Sherlocked'...?
    It wouldn’t surprise me in the least to find that Apple is working on their own solution with the intention of no longer needing to use ChatGPT or some other third-party. 
    Apple doesn't have talent to "Sherlock" ChatGPT, if they could they'd already have their own service instead of outsourcing to get it.
    What I do see Apple doing is trying to pass all this off as their own doing to the unaware masses who don't know. 
    williamlondon
  • Reply 8 of 30
    blastdoorblastdoor Posts: 3,432member
    PCC may eventually be viewed as Tim Cook's greatest legacy. It's not as flashy as the iPhone because it's a behind the scenes kind of a thing (much as Cook is more of a behind the scenes kind of guy himself, at least relative to Jobs), but it could end up being just as big of a deal -- maybe a bigger deal. 

    PCC is the ultimate marriage of vertical integration, stellar hardware and software capes, and a deep commitment to privacy. Nobody else can match it. The biggest threat to PCC doesn't come from competitors but from brain dead regulators (hi there EC). 
    watto_cobra
  • Reply 9 of 30
    riverkoriverko Posts: 234member
    Could anyone find the clean up feature in current Mac OS beta in the Photos app? Somehow have no clue if it is there or where i should look.
    watto_cobra
  • Reply 10 of 30
    melgrossmelgross Posts: 33,585member
    "Users have ultimate control."
    Do all roads still lead to iCloud and Apple servers ?
    Will ChatGP ultimately end up being 'Sherlocked'...?

    I hate that term, “sherlocked”. Apple comes out with software for System 9, then OS X can’t use it and Apple is taking time to get it working there. A third party developer comes out with an app called Watson, to mimic Apple’s old Sherlock. Apple finally comes out with a version that works on OS X and immediately some people yell that Apple “Sherlocked” him. I always thought that was nuts.
    ilarynxfastasleepAlex_VPenzijony0watto_cobra
  • Reply 11 of 30
    melgrossmelgross Posts: 33,585member

    Will ChatGP ultimately end up being 'Sherlocked'...?
    It wouldn’t surprise me in the least to find that Apple is working on their own solution with the intention of no longer needing to use ChatGPT or some other third-party. 
    Apple doesn't have talent to "Sherlock" ChatGPT, if they could they'd already have their own service instead of outsourcing to get it.
    What I do see Apple doing is trying to pass all this off as their own doing to the unaware masses who don't know. 
    That’s ridiculous. I mean, totally ridiculous.  You really need to know something before posting nonsense. Apple has been doing this well before anyone else. They call it by its proper term; machine learning. Artificial intelligence is just a term dreamed up a time ago to sound more exciting for the masses.

    if you look at what Apple us doing here, you’d see that they’ve been working on nit for years. But Apple has learned from the Maps fiasco to not introduce a major initiative before it’s ready. Look at what Microsoft and Google did with theirs. Both companies had to withdraw their LLMs shortly after introducing them because of major problems. Google’s engineers even wrote a letter to management n telling to not bring it out because it wasn’t ready and there would be problems. They’ve now withdrawn AI search until they fix it. Microsoft has withdrawn the latest Windows 11 update because of a security nightmare and there’s no information as to when Recall will come out, if it ever will, which is what some people in the industry believe.

    so Apple doesn’t have the talent? Really, that’s garbage.
    AfarstardanoxroundaboutnowAlex1NfastasleepAlex_VPenzisconosciutojony0watto_cobra
  • Reply 12 of 30
    williamlondonwilliamlondon Posts: 1,379member
    aderutter said:
    “ We'll see in the fullness of time if the responses that ChatGPT change when logged into an OpenAI account. ”

    I see that this article wasn’t proofread… 

    Looking forward to a time when AI articles are proofread using AI…


    Captain Pedantic arrives on the scene and saves no one.
    But, regardless of that wait list, we're not going to get it on our HomePods in 2024. And, probably not on the existing devices in the family at all. 


    If you have a HomePod -- or multiple -- they won't have the benefits of Apple Intelligence or the updated Siri. That extends to other devices, too, like your Apple Watch or any unsupported Macs or iPhones.

    Is that confirmed or is it possible HomePods will handoff queries to our iPhones like they already sometimes do. For instance, asking a HomePod where somebody is will use Find My on my iPhone to actually get the info from and then it will tell me. Maybe Apple Intelligence will work similarly with a HomePod?
    Really good question I'd also like to understand better.
    noun: vapourware; noun: vaporware
    1. software or hardware that has been advertised but is not yet available to buy, either because it is only a concept or because it is still being written or designed.
    Oh good god, they announced a new set of features, it's being talked on a rumor site and you post this silly nonsense? Troll harder.
    tmayrezwitsroundaboutnowAlex1Npascal007Alex_Vsconosciutojony0brometheuswatto_cobra
  • Reply 13 of 30
    mac_dogmac_dog Posts: 1,076member
    aderutter said:
    I see that this article wasn’t proofread… 
    That’s your job…apparently. 

    Looking forward to a time when AI articles are proofread using AI…
    😏


    williamlondonaderutter
  • Reply 14 of 30
    nubusnubus Posts: 486member
    noun: vapourware; noun: vaporware
    1. software or hardware that has been advertised but is not yet available to buy, either because it is only a concept or because it is still being written or designed.

    I don't believe we're back to the days of Copland - the macOS that never shipped. At least Apple paid hotel+flights for me to attend a demo.
    These days most products do ship with AirPower being one that only made it to a keynote.

    But shipping outside US are where things get ugly. Apple is active in 175 countries. iPhone 14 got satellite SOS. 2 years later and 16 countries (9%) are covered. In most of the world customers pay for hardware that does nothing.

    Look at this mess of a KB-post on availability of services: https://support.apple.com/en-us/118205

    Google Gemini is available in 150 countries supporting around 30 languages. MS Azure AI is doing 143 combinations.
    Apple Intelligence is scheduled for "en-us" and that is it. Apple Intelligence is mainly FUD to those of us not running on en-us.
    rezwitspascal007IreneW
  • Reply 15 of 30
    AppleZuluAppleZulu Posts: 2,097member
    melgross said:

    Will ChatGP ultimately end up being 'Sherlocked'...?
    It wouldn’t surprise me in the least to find that Apple is working on their own solution with the intention of no longer needing to use ChatGPT or some other third-party. 
    Apple doesn't have talent to "Sherlock" ChatGPT, if they could they'd already have their own service instead of outsourcing to get it.
    What I do see Apple doing is trying to pass all this off as their own doing to the unaware masses who don't know. 
    That’s ridiculous. I mean, totally ridiculous.  You really need to know something before posting nonsense. Apple has been doing this well before anyone else. They call it by its proper term; machine learning. Artificial intelligence is just a term dreamed up a time ago to sound more exciting for the masses.

    if you look at what Apple us doing here, you’d see that they’ve been working on nit for years. But Apple has learned from the Maps fiasco to not introduce a major initiative before it’s ready. Look at what Microsoft and Google did with theirs. Both companies had to withdraw their LLMs shortly after introducing them because of major problems. Google’s engineers even wrote a letter to management n telling to not bring it out because it wasn’t ready and there would be problems. They’ve now withdrawn AI search until they fix it. Microsoft has withdrawn the latest Windows 11 update because of a security nightmare and there’s no information as to when Recall will come out, if it ever will, which is what some people in the industry believe.

    so Apple doesn’t have the talent? Really, that’s garbage.
    I agree with your conclusion that Apple has the capacity to impact this field, and even that in recent times the term Artificial Intelligence has been over-hyped. That said, the term itself has been in use since at least 1956 and probably earlier, so the term is not a new thing. 
    williamlondonroundaboutnowAlex1Nwatto_cobra
  • Reply 16 of 30
    aderutter said:
    “ We'll see in the fullness of time if the responses that ChatGPT change when logged into an OpenAI account. ”

    I see that this article wasn’t proofread… 

    Looking forward to a time when AI articles are proofread using AI…


    AI proofreading definitely looks handy in some circumstances and it’s good that Apple are (presently) aiming to steer the ship wide of AI’s risk towards advancing the huge mis/disinformation issue in society, politics, business and journalism. Dunno how we’re going to get around that looming iceberg...
    watto_cobra
  • Reply 17 of 30
    fastasleepfastasleep Posts: 6,447member
    "Users have ultimate control."
    Do all roads still lead to iCloud and Apple servers ?
    Will ChatGP ultimately end up being 'Sherlocked'...?

    1) Yes. You can turn all this stuff off. Based on your posting history, I imagine you have every single feature turned off on all your devices, and you live in an air gapped Faraday cage

    2) Not unless you want it to. Your paranoia regarding cloud services sure is something.

    3) Maybe, and I sure would prefer an Apple-owned "world knowledge" model versus the current model which will be sending off those queries to 3rd parties like OpenAI, Microsoft, etc (as we know, other 'modules' are coming as deals are made). Sherlocked? Not sure that applies, as there's already competition in this space and Apple plans to offer other models. If they add their own, it would still be entirely possible they keep offering third party models as well, like search engines in Safari. 
    williamlondonwatto_cobra
  • Reply 18 of 30
    fastasleepfastasleep Posts: 6,447member

    Will ChatGP ultimately end up being 'Sherlocked'...?
    It wouldn’t surprise me in the least to find that Apple is working on their own solution with the intention of no longer needing to use ChatGPT or some other third-party. 
    Apple doesn't have talent to "Sherlock" ChatGPT, if they could they'd already have their own service instead of outsourcing to get it.
    What I do see Apple doing is trying to pass all this off as their own doing to the unaware masses who don't know. 
    They're doing nothing of the sort. If a query doesn't fit on-device AI or Private Cloud Compute, a dialogue comes up that very specifically asks if you'd like to send the query to ChatGPT, which you have to approve every single time. 
    Alex_VMacProwilliamlondonbrometheuswatto_cobra
  • Reply 19 of 30
    zimmermannzimmermann Posts: 332member
    nubus said:
    noun: vapourware; noun: vaporware
    1. software or hardware that has been advertised but is not yet available to buy, either because it is only a concept or because it is still being written or designed.

    I don't believe we're back to the days of Copland - the macOS that never shipped. At least Apple paid hotel+flights for me to attend a demo.
    These days most products do ship with AirPower being one that only made it to a keynote.

    But shipping outside US are where things get ugly. Apple is active in 175 countries. iPhone 14 got satellite SOS. 2 years later and 16 countries (9%) are covered. In most of the world customers pay for hardware that does nothing.

    Look at this mess of a KB-post on availability of services: https://support.apple.com/en-us/118205

    Google Gemini is available in 150 countries supporting around 30 languages. MS Azure AI is doing 143 combinations.
    Apple Intelligence is scheduled for "en-us" and that is it. Apple Intelligence is mainly FUD to those of us not running on en-us.



    i must agree on this. Only recently bike-routes have come available  in the Netherlands in Cards. All this wonderful stuff will slowly come available to other countries and other languages. I can’t wait. 

    edited June 20 watto_cobra
  • Reply 20 of 30
    IreneWIreneW Posts: 305member
    melgross said:

    Will ChatGP ultimately end up being 'Sherlocked'...?
    It wouldn’t surprise me in the least to find that Apple is working on their own solution with the intention of no longer needing to use ChatGPT or some other third-party. 
    Apple doesn't have talent to "Sherlock" ChatGPT, if they could they'd already have their own service instead of outsourcing to get it.
    What I do see Apple doing is trying to pass all this off as their own doing to the unaware masses who don't know. 
    That’s ridiculous. I mean, totally ridiculous.  You really need to know something before posting nonsense. Apple has been doing this well before anyone else. They call it by its proper term; machine learning. Artificial intelligence is just a term dreamed up a time ago to sound more exciting for the masses.
    .
    Actually, Machine Learning is defined as a very distinct subset of Artificial Intelligence, and AI is absolutely not a newly dreamed up term. Generative models and LLMs also has their specific meanings.
    Software helping us humans to process information comes in many forms. And has for a long time.

    Apples AI is a combination of several such processing paradigms, as is most other recently marketed services.

    On-device processing and privacy är where Apple shines.
    williamlondonwatto_cobra
Sign In or Register to comment.