Siri for iOS 18 to gain massive AI upgrade via Apple's Ajax LLM

Posted:
in iOS edited May 13

Several of Apple's standard system components are on the verge of receiving significant AI-related enhancements, with Safari, Spotlight Search, and Siri being first in line for the treatment.

An image with a Siri icon, Spotlight search bar, Safari icon, and Messages icon
Apple AI will make Siri, Safari, Spotlight, and Messages better



Over the past several months, there's been various chatter regarding AI-related features bound for iOS 18, and AppleInsider has since learned some specifics on the functionality of Apple's Ajax large language model (LLM), as well as information on features the company has been testing alongside it.

This report focuses only on information that could be verified independently through people familiar with the software. In addition to specific features, they've collectively shared some info on the iPhone maker's internal AI test environments as well.

Before exploring the planned AI upgrades in detail, it's important to consider the motivation behind Apple's move to develop its own generative AI technology and what it aims to achieve by doing so.

Why is Apple incorporating generative AI into its upcoming operating systems?



As the popularity of generative AI software continues to increase, it's only logical for Apple to apply LLMs for the betterment of its existing default applications, which are widely deployed across billions of devices worldwide. Those familiar with the company's early efforts believe the company maintains a clear vision for its AI-enhanced apps and services, with an unmistakable sense of direction.

An image combining the Siri icon with the ChatGPT icon
Apple will enhance its operating systems with GPT-like features



Apple's approach to generative AI will focus on practical benefits for the end-user while simultaneously attempting to preserve user privacy by using on-device LLM.

Rather than offering short-term entertainment value in the form of an AI chatbot, the company aims to improve its existing portfolio of system applications via generative AI. Features like text summarization, document analysis, and AI-enhanced search options would all directly benefit end-users in meaningful ways.

Apple intends to introduce AI-related improvements to several of its built-in system apps beginning next month, including Siri, Spotlight Search, Messages, Mail, and Safari.

AI-powered text summarization for Safari and Siri



As detailed in our exclusive report on the next iteration of Safari, Apple is testing AI-related enhancements for the app, with text summarization being one of the key features in the works.

An image showing a Safari window open to AppleInsider's website. The open menu shows selections including Intelligent Browsing.
Safari's Intelligent Browsing feature will provide page summaries and other info



With the release of Safari 18, Apple is expected to introduce article summarization through a new Intelligent Search feature -- meaning users will have the option to generate a brief summary of any webpage currently on screen.

Apple's built-in AI software can analyze the keywords and phrases within a webpage or document and generate a short summary containing only the most important information.

Siri is also due to receive a similar update, according to people familiar with Apple's plans, as the company intends to integrate the digital assistant more closely with its built-in Messages application.

The company's AI software can analyze message contents in pre-release versions of Apple's next-gen operating systems. The LLM is also reportedly capable of generating responses that relay the content of messages in a simplified way.

Internal test environments offer significant insight into the overall capabilities of Ajax, especially regarding on-device response generation - another key feature Apple is working on.

Ajax will be able to generate basic responses entirely on-device



Apple has been exploring offline response generation software for over a year at this point, and it could very likely make its debut next month at WWDC in one form or another.

A Spotlight search result for 'kitten' showing various images of kittens
Spotlight will get much better with more intelligent results and sorting



Perhaps the most valuable feature Apple has developed is the software's capability to generate sentence-long responses and function unabated without cloud-based processing.

When provided with text input, the on-device AI generates multiple responses within a matter of milliseconds. Within Apple's test environments, these responses are organized based on accuracy, speed, and other relevant factors.

In creating responses, Apple's Ajax LLM will check whether or not the text input contains the name of a saved contact and display the contact's information, if necessary. The software will also communicate with the Calendar app and take into account events when creating a response.

While Ajax can generate rudimentary text-based responses on-device, more advanced replies or text summaries appear to necessitate server-side processing. This aligns with other reports claiming that Apple is in talks with OpenAI and Google about licensing their cloud-based AI technology.

As for generating text from a prompt, Apple reportedly explored ways of doing this entirely on-device, but it remains unclear whether it succeeded in accomplishing offline text generation.

Though more sophisticated AI features will likely require connectivity, Apple's on-device processing can still be used for more basic features, such as text analysis and rudimentary summarization through keywords and sentences.

Text analysis and summarization for Safari and Messages



In its AI-related test applications and associated environments, Apple strongly emphasizes text analysis and summarization - all of which are handled by the company's on-device LLM, Ajax.

A screenshot in an iPhone frame showing the Arc browser summarizing an article
Alternate browsers like Arc offer article summaries, and soon Safari will too



Before testing can begin, Apple's engineers first provide the LLM with the necessary text input. The software can receive text input through a dedicated text box, through digital documents, or it can use information from the Safari and Messages applications.

Upon receiving text input, Apple's on-device AI selects keywords and phrases within a text, designating them as text topics. Sentences containing explanations, descriptions, definitions, or those which denote varieties of objects are also isolated from the remainder of the text.

In analyzing texts, the software takes into account all relevant information available. It can recognize and classify entities such as companies, people, and locations. For instance, if a name appears at the top of a text, the software will likely recognize the name as belonging to the text's author.

The selected key sentences and topics ultimately provide a rudimentary text summary, which Apple's on-device response generation software can use for a more coherent answer.

As the company's text analysis software is capable of functioning entirely on-device, this could give Apple an advantage against the competition in the area of privacy protection.

How will the new AI improvements affect user privacy?



Apple's on-device AI software also appears to have a strong focus on preserving the privacy of its users, which would align with the company's existing privacy-focused features - such as iCloud Private Relay.

Apple's privacy icon of two figures shaking hands below a padlock
Apple holds privacy high as a basic human right and will prioritize it with AI



Even in Apple's test environments, user privacy warnings still exist. According to individuals familiar with the software, the company's test utilities will display a message before the Ajax LLM uses any information from Safari or the Messages app.

By eliminating the need for cloud-based processing -- at least in the most basic of situations -- the chance for privacy-related issues is greatly reduced. If user data is not transmitted, the odds of it becoming compromised are obviously much lower.

Compared to existing implementations of AI in rival products and services, Apple's implementation is arguably innovative in its attempt at privacy preservation and efficient on-device processing.

How does Ajax compare against the competition?



Apple's AI-related improvements look to compete with the ever-increasing number of AI-enabled applications, which are available from a wide variety of third parties.

Siri telling a joke: 'How did Darth Vader know what Luke was getting for Christmas?' 'He felt his presents.'
Siri is due for a big upgrade



Virtually every app, from Slack to Google Chrome, now features some degree of AI integration, in addition to the various AI chatbot apps currently on the market.

Products such as the Rabbit R1 or Humane AI Pin could also be regarded as competition, as they're commonly perceived as virtual personal assistants or smartphone replacements.

By creating an AI-infused version of Siri with on-device processing, Apple hopes to overshadow many existing implementations of generative AI - especially AI-themed physical products.

Both the Rabbit R1 and the Humane AI Pin function through cloud-based processing, with the latter even requiring a separate monthly subscription to operate. These devices are also commonly regarded as unfinished or only partially functional, as noted in various user reviews.

The Rabbit R1 is a small orange-ish handheld device with a rabbit mascot and spin wheel UI
Apple will compete with products like the Rabbit R1 by offering vertically integrated AI software on established hardware



An AI-capable version of Siri would also allow for greater privacy protection with on-device processing, all without the cost of an added monthly subscription. Given Apple's established market presence, quality control practices, and available resources, Siri has renewed potential to become a serious competitive threat.

During Apple's most recent earnings call on May 2, Apple's CEO Tim Cook fielded two questions about the company's monetization of generative AI but did not provide concrete responses. Earlier in the call, Cook affirmed Apple's "unwavering focus on privacy" and said that the company has "advantages that differentiate" it from the competition regarding AI.

Relative to AI chatbot applications and services such as OpenAI's ChatGPT, an updated version of Siri would have its advantages in the form of on-device functionality. It would also leverage its existing integration with other built-in Apple applications, allowing for greater functionality than any ordinary chatbot.

As for Safari's upcoming Intelligent Browsing feature with text summarization, it likely aims to compete against prominent third-party browsers such as Arc Browser, which also contains its own text summarization feature.

Given Apple's recent interest in artificial intelligence, we should see some of these features debut at the annual Worldwide Developers' Conference later this year.



Read on AppleInsider

«1

Comments

  • Reply 1 of 29
    PemaPema Posts: 45member
    Spotlight search. Hope that this upgrade provides some useful search. As Spotlight search currently stands you will be lucky, very lucky, to find anything of any use on the Mac. 
    The Mac is a great machine and so is the iPhone. But search on the Mac and on the iPhone is about as terrible as it gets. 
    Any time I am stuck with having to look for a file or document I reach for an anti migraine tablet. I know I am going to get a massive headache at all the useless junk that is thrown at me which has nothing to do with the search criteria that I entered.  >:)
    williamlondonrmusikantowAlex1N
  • Reply 2 of 29
    neilmneilm Posts: 989member
    The key question is: what will be the device hardware requirements for AI aided services?

    For instance my iPhone XS is the oldest that can run the current iOS 17, so it may well not be supported by iOS 18. While my XS still works well — even the original battery! — and does everything I want it to, I may be prodded to upgrade by the availability of useful AI.
    OnPartyBusinessnubusmattinozwatto_cobraAlex1N
  • Reply 3 of 29
    neilm said:
    The key question is: what will be the device hardware requirements for AI aided services?

    For instance my iPhone XS is the oldest that can run the current iOS 17, so it may well not be supported by iOS 18. While my XS still works well — even the original battery! — and does everything I want it to, I may be prodded to upgrade by the availability of useful AI.
    I think the worst case is it will be A14/M1 and above. That’s when they introduced the 16-core Apple Neural Engine NPU. But maybe the cutoff will be an 8-core NPU, which was introduced with the A12/A12X, so your XS might be okay? One last hurrah before it gets left behind next year?
    edited May 3 watto_cobragregoriusm
  • Reply 4 of 29
    iOS_Guy80iOS_Guy80 Posts: 841member
    neilm said:
    The key question is: what will be the device hardware requirements for AI aided services?

    For instance my iPhone XS is the oldest that can run the current iOS 17, so it may well not be supported by iOS 18. While my XS still works well — even the original battery! — and does everything I want it to, I may be prodded to upgrade by the availability of useful AI.
    That’s why new iPad announcements May 7 and of course iPhone 16 in September.
    watto_cobra
  • Reply 5 of 29
    elijahgelijahg Posts: 2,780member
    Pema said:
    Spotlight search. Hope that this upgrade provides some useful search. As Spotlight search currently stands you will be lucky, very lucky, to find anything of any use on the Mac. 
    The Mac is a great machine and so is the iPhone. But search on the Mac and on the iPhone is about as terrible as it gets. 
    Any time I am stuck with having to look for a file or document I reach for an anti migraine tablet. I know I am going to get a massive headache at all the useless junk that is thrown at me which has nothing to do with the search criteria that I entered.  >:)
    Really? I find Spotlight great. Especially compared to Windows search, which is makes a chocolate teaspoon seem otherworldly.
    Aulani40domiStrangeDaysjas99
  • Reply 6 of 29
    JamesCudeJamesCude Posts: 52member
    That would be kind of nuts for Apple to be such a renowned software company and yet they would still look to OpenAI for something as simple as an LLM??
    watto_cobrawilliamlondonLettuce
  • Reply 7 of 29
    AulaniAulani Posts: 7member
    Pema said:
    Spotlight search. Hope that this upgrade provides some useful search. As Spotlight search currently stands you will be lucky, very lucky, to find anything of any use on the Mac. 
    The Mac is a great machine and so is the iPhone. But search on the Mac and on the iPhone is about as terrible as it gets. 
    Any time I am stuck with having to look for a file or document I reach for an anti migraine tablet. I know I am going to get a massive headache at all the useless junk that is thrown at me which has nothing to do with the search criteria that I entered.  >:)
    Spotlight works great for me, on my Mac and iPhone/iPad. Couldnt live without it. 

    Perhaps you could use some instruction on composing better search prompts?
    lordjohnwhorfinwatto_cobra40domiwilliamlondongregoriusmStrangeDaysjas99
  • Reply 8 of 29
    dutchlorddutchlord Posts: 231member
    Does that mean Siri suddenly understands Dutch? If not, any “massive upgrade” has zero value for me and Siri remains switched off like the past few years.
    40dominubuswilliamlondonCrossPlatformFrogger
  • Reply 9 of 29
    22july201322july2013 Posts: 3,612member
    I love the Safari Reader feature in Safari, which also doubles as an ad-remover. I'm surprised I haven't noticed companies get upset about Apple's ad remover technology in this feature.

    Similarly, I would love to see Apple use AI to give Safari a new "Summarizer" feature which, when turned on, would display the web page in a manner similar to Safari Reader, but would summarize the page. I would expect web-content creators to get upset that someone is abridging their content, but since they haven't been upset about the Safari Reader, maybe they won't get upset about the Safari Summarizer either.

    Maybe there are no legal grounds to object to a user creating summaries of web content for personal purposes. If the summary was being performed by an online service, there might be a legal problem with that. But if it's done on device, it's probably legal. This could be a problem for Google and Microsoft, because their AI services are all done online. Google and Microsoft are charging users for "summaries" of information from third party websites.
    Alex1Nwatto_cobragregoriusmradarthekatjas99
  • Reply 10 of 29
    dutchlord said:
    Does that mean Siri suddenly understands Dutch? If not, any “massive upgrade” has zero value for me and Siri remains switched off like the past few years.
    No offense but this is somewhat niche… 
    watto_cobrawilliamlondondanox
  • Reply 11 of 29
    With cloud AI deeply integrated into Google and Microsoft apps, I can see enterprise firms looking to Macs for on-device AI. I don’t think enterprise firms would like these cloud AI’s scraping their servers of all their secure documents and data and who knows what happens to that data down the road. The security concerns have to be huge.
    watto_cobramattinozradarthekatjas99
  • Reply 12 of 29
    gatorguygatorguy Posts: 24,339member
    I love the Safari Reader feature in Safari, which also doubles as an ad-remover. I'm surprised I haven't noticed companies get upset about Apple's ad remover technology in this feature.

    Similarly, I would love to see Apple use AI to give Safari a new "Summarizer" feature which, when turned on, would display the web page in a manner similar to Safari Reader, but would summarize the page. I would expect web-content creators to get upset that someone is abridging their content, but since they haven't been upset about the Safari Reader, maybe they won't get upset about the Safari Summarizer either.

    Maybe there are no legal grounds to object to a user creating summaries of web content for personal purposes. If the summary was being performed by an online service, there might be a legal problem with that. But if it's done on device, it's probably legal. This could be a problem for Google and Microsoft, because their AI services are all done online. Google and Microsoft are charging users for "summaries" of information from third party websites.
    Google doesn't charge me for AI-enabled article summaries, which like Apple has ads removed. I use Google Simplify, their "Reader" mode, and Google Summary which makes use of Generative AI, multiple times a day. No ads with either feature. Oh, and Summary runs on-device, just as privately as what Apple is supposedly planning. We will all know more if/when Apple officially announces it. 

    At the same time, Google respects a website owner's right to monetize their content and won't "summarize" without the content owner's permission.  There have also been no legal objections to it either, as far as I know, so I fully expect Apple will handle it the same as Google; AI Summarize, sans ads, will only be available with a website owner's permission. 
    https://support.google.com/assistant/answer/14163109?hl=en  ;
    edited May 4 ctt_zhmuthuk_vanalingamgregoriusmroundaboutnow
  • Reply 13 of 29
    ctt_zhctt_zh Posts: 82member
    With cloud AI deeply integrated into Google and Microsoft apps, I can see enterprise firms looking to Macs for on-device AI. I don’t think enterprise firms would like these cloud AI’s scraping their servers of all their secure documents and data and who knows what happens to that data down the road. The security concerns have to be huge.
    How does "cloud AI" scrape private corporate servers?
    williamlondon
  • Reply 14 of 29
    40domi40domi Posts: 132member
    So far there isn't an AI product that is anything other than misleading hype and a toy for kids.
    Is summarising text useful & helpful? Most people already do that in their minds, missing important context & info, making them ill informed, tools to summarise aren't going to improve that.
    For me hope, Siri gets a big improvement and photo editing, which has been very basic for years, to make sure we buy crap apps and the developers still complain about Apple 😳 I would like to see best take in it & an eraser at the very least.
    Other than that I'm not buying into the fake hype!
    williamlondonjas99danoxsconosciuto
  • Reply 15 of 29
    40domi40domi Posts: 132member
    dutchlord said:
    Does that mean Siri suddenly understands Dutch? If not, any “massive upgrade” has zero value for me and Siri remains switched off like the past few years.
    Spot on, it is Siri's biggest downside, in which Universe is it Ok to sell a product that doesn't work in your native language? Hopefully AI can help with that, Siri should work with every language the phone is sold! However I think you'll find Siri works in Dutch language, since IOS 15


    edited May 5 williamlondonStrangeDayssconosciuto
  • Reply 16 of 29
    40domi40domi Posts: 132member

    dutchlord said:
    Does that mean Siri suddenly understands Dutch? If not, any “massive upgrade” has zero value for me and Siri remains switched off like the past few years.
    No offense but this is somewhat niche… 
    No offence taken, however your view is very blinkered and a big deal for people that don't speak the limited languages it so far supports. It's not niche, it's essential that it supports every language it's sold in!

    williamlondonmacguipascal007mac_dogsconosciutoavon b7Scot1
  • Reply 17 of 29
    40domi40domi Posts: 132member

    gatorguy said:
    I love the Safari Reader feature in Safari, which also doubles as an ad-remover. I'm surprised I haven't noticed companies get upset about Apple's ad remover technology in this feature.

    Similarly, I would love to see Apple use AI to give Safari a new "Summarizer" feature which, when turned on, would display the web page in a manner similar to Safari Reader, but would summarize the page. I would expect web-content creators to get upset that someone is abridging their content, but since they haven't been upset about the Safari Reader, maybe they won't get upset about the Safari Summarizer either.

    Maybe there are no legal grounds to object to a user creating summaries of web content for personal purposes. If the summary was being performed by an online service, there might be a legal problem with that. But if it's done on device, it's probably legal. This could be a problem for Google and Microsoft, because their AI services are all done online. Google and Microsoft are charging users for "summaries" of information from third party websites.
    Google doesn't charge me for AI-enabled article summaries, which like Apple has ads removed. I use Google Simplify, their "Reader" mode, and Google Summary which makes use of Generative AI, multiple times a day. No ads with either feature. Oh, and Summary runs on-device, just as privately as what Apple is supposedly planning. We will all know more if/when Apple officially announces it. 

    At the same time, Google respects a website owner's right to monetize their content and won't "summarize" without the content owner's permission.  There have also been no legal objections to it either, as far as I know, so I fully expect Apple will handle it the same as Google; AI Summarize, sans ads, will only be available with a website owner's permission. 
    https://support.google.com/assistant/answer/14163109?hl=en  ;
    Perhaps you read, too many summaries in Google, to be so ill informed?
    williamlondonjas99
  • Reply 18 of 29
    gatorguygatorguy Posts: 24,339member
    40domi said:

    gatorguy said:
    I love the Safari Reader feature in Safari, which also doubles as an ad-remover. I'm surprised I haven't noticed companies get upset about Apple's ad remover technology in this feature.

    Similarly, I would love to see Apple use AI to give Safari a new "Summarizer" feature which, when turned on, would display the web page in a manner similar to Safari Reader, but would summarize the page. I would expect web-content creators to get upset that someone is abridging their content, but since they haven't been upset about the Safari Reader, maybe they won't get upset about the Safari Summarizer either.

    Maybe there are no legal grounds to object to a user creating summaries of web content for personal purposes. If the summary was being performed by an online service, there might be a legal problem with that. But if it's done on device, it's probably legal. This could be a problem for Google and Microsoft, because their AI services are all done online. Google and Microsoft are charging users for "summaries" of information from third party websites.
    Google doesn't charge me for AI-enabled article summaries, which like Apple has ads removed. I use Google Simplify, their "Reader" mode, and Google Summary which makes use of Generative AI, multiple times a day. No ads with either feature. Oh, and Summary runs on-device, just as privately as what Apple is supposedly planning. We will all know more if/when Apple officially announces it. 

    At the same time, Google respects a website owner's right to monetize their content and won't "summarize" without the content owner's permission.  There have also been no legal objections to it either, as far as I know, so I fully expect Apple will handle it the same as Google; AI Summarize, sans ads, will only be available with a website owner's permission. 
    https://support.google.com/assistant/answer/14163109?hl=en  ;
    Perhaps you read, too many summaries in Google, to be so ill informed?
    Oooh, thanks for that detailed rebuttal with evidence to the contrary! /s

    'ya didn't follow the link, huh?
    ctt_zhmuthuk_vanalingam
  • Reply 19 of 29
    nubusnubus Posts: 443member
    dutchlord said:
    Does that mean Siri suddenly understands Dutch? If not, any “massive upgrade” has zero value for me and Siri remains switched off like the past few years.
    No offense but this is somewhat niche… 
    Microsoft Copilot is shipping with wide language support - Dutch included. Apple is trying to run a global company but only support US. This also happened with iPhone 14. The device had 1 USP (SOS calls) and it didn't work outside North America. No wonder the Chinese are moving to products supporting their language and region.
    ctt_zhwilliamlondonpascal007
  • Reply 20 of 29
    gatorguygatorguy Posts: 24,339member
     This could be a problem for Google and Microsoft, because their AI services are all done online

    Nope. These things are on-device using Google AI:

    • Gemini Nano: Google's most efficient AI model for on-device tasks, which powers features like:
      • Summarize in Recorder: Summarizes recorded conversations, interviews, and presentations without a network connection
      • Smart Reply in Gboard: Suggests high-quality responses to save time
    • Magic Eraser: Erases larger objects and people from photos without leaving traces
    • Read aloud: Reads news articles aloud in English or another language
    • Transcription: Transcribes recorded conversations and identifies the two speakers
    • Summarizing audio recordings in the Recorder app
    • Generative AI reply suggestions in chat apps like WhatsApp
    • An enhanced Magic Eraser that generates new pixels to fill in spaces left by removed objects
    • Best Take:  A photo editing feature that combines similar photos into one, allowing users to select their favorite expressions for each person in the photo. 
    • Magic Audio Eraser: Removes unwanted background noise from videos on the Google Pixel 8 and Pixel 8 Pro

    So while some are done online, the resource-intensive Video Boost for instance, most of the AI features introduced with the Pixel 8 are done privately on the user's phone rather than sent to Google to accomplish. That makes Google's current use of on-device AI processing similar to the rumored integration of AI features on this fall's Pro iPhones.

    However, if the reports are accurate, not all of Apple's AI features can be accomplished on-device. The more advanced features will still need cloud assistance.




    edited May 5 ctt_zhgregoriusmmuthuk_vanalingamroundaboutnowLettuce
Sign In or Register to comment.