Siri for iOS 18 to gain massive AI upgrade via Apple's Ajax LLM
Several of Apple's standard system components are on the verge of receiving significant AI-related enhancements, with Safari, Spotlight Search, and Siri being first in line for the treatment.
Apple AI will make Siri, Safari, Spotlight, and Messages better
Over the past several months, there's been various chatter regarding AI-related features bound for iOS 18, and AppleInsider has since learned some specifics on the functionality of Apple's Ajax large language model (LLM), as well as information on features the company has been testing alongside it.
This report focuses only on information that could be verified independently through people familiar with the software. In addition to specific features, they've collectively shared some info on the iPhone maker's internal AI test environments as well.
Before exploring the planned AI upgrades in detail, it's important to consider the motivation behind Apple's move to develop its own generative AI technology and what it aims to achieve by doing so.
Why is Apple incorporating generative AI into its upcoming operating systems?
As the popularity of generative AI software continues to increase, it's only logical for Apple to apply LLMs for the betterment of its existing default applications, which are widely deployed across billions of devices worldwide. Those familiar with the company's early efforts believe the company maintains a clear vision for its AI-enhanced apps and services, with an unmistakable sense of direction.
Apple will enhance its operating systems with GPT-like features
Apple's approach to generative AI will focus on practical benefits for the end-user while simultaneously attempting to preserve user privacy by using on-device LLM.
Rather than offering short-term entertainment value in the form of an AI chatbot, the company aims to improve its existing portfolio of system applications via generative AI. Features like text summarization, document analysis, and AI-enhanced search options would all directly benefit end-users in meaningful ways.
Apple intends to introduce AI-related improvements to several of its built-in system apps beginning next month, including Siri, Spotlight Search, Messages, Mail, and Safari.
AI-powered text summarization for Safari and Siri
As detailed in our exclusive report on the next iteration of Safari, Apple is testing AI-related enhancements for the app, with text summarization being one of the key features in the works.
Safari's Intelligent Browsing feature will provide page summaries and other info
With the release of Safari 18, Apple is expected to introduce article summarization through a new Intelligent Search feature -- meaning users will have the option to generate a brief summary of any webpage currently on screen.
Apple's built-in AI software can analyze the keywords and phrases within a webpage or document and generate a short summary containing only the most important information.
Siri is also due to receive a similar update, according to people familiar with Apple's plans, as the company intends to integrate the digital assistant more closely with its built-in Messages application.
The company's AI software can analyze message contents in pre-release versions of Apple's next-gen operating systems. The LLM is also reportedly capable of generating responses that relay the content of messages in a simplified way.
Internal test environments offer significant insight into the overall capabilities of Ajax, especially regarding on-device response generation - another key feature Apple is working on.
Ajax will be able to generate basic responses entirely on-device
Apple has been exploring offline response generation software for over a year at this point, and it could very likely make its debut next month at WWDC in one form or another.
Spotlight will get much better with more intelligent results and sorting
Perhaps the most valuable feature Apple has developed is the software's capability to generate sentence-long responses and function unabated without cloud-based processing.
When provided with text input, the on-device AI generates multiple responses within a matter of milliseconds. Within Apple's test environments, these responses are organized based on accuracy, speed, and other relevant factors.
In creating responses, Apple's Ajax LLM will check whether or not the text input contains the name of a saved contact and display the contact's information, if necessary. The software will also communicate with the Calendar app and take into account events when creating a response.
While Ajax can generate rudimentary text-based responses on-device, more advanced replies or text summaries appear to necessitate server-side processing. This aligns with other reports claiming that Apple is in talks with OpenAI and Google about licensing their cloud-based AI technology.
As for generating text from a prompt, Apple reportedly explored ways of doing this entirely on-device, but it remains unclear whether it succeeded in accomplishing offline text generation.
Though more sophisticated AI features will likely require connectivity, Apple's on-device processing can still be used for more basic features, such as text analysis and rudimentary summarization through keywords and sentences.
Text analysis and summarization for Safari and Messages
In its AI-related test applications and associated environments, Apple strongly emphasizes text analysis and summarization - all of which are handled by the company's on-device LLM, Ajax.
Alternate browsers like Arc offer article summaries, and soon Safari will too
Before testing can begin, Apple's engineers first provide the LLM with the necessary text input. The software can receive text input through a dedicated text box, through digital documents, or it can use information from the Safari and Messages applications.
Upon receiving text input, Apple's on-device AI selects keywords and phrases within a text, designating them as text topics. Sentences containing explanations, descriptions, definitions, or those which denote varieties of objects are also isolated from the remainder of the text.
In analyzing texts, the software takes into account all relevant information available. It can recognize and classify entities such as companies, people, and locations. For instance, if a name appears at the top of a text, the software will likely recognize the name as belonging to the text's author.
The selected key sentences and topics ultimately provide a rudimentary text summary, which Apple's on-device response generation software can use for a more coherent answer.
As the company's text analysis software is capable of functioning entirely on-device, this could give Apple an advantage against the competition in the area of privacy protection.
How will the new AI improvements affect user privacy?
Apple's on-device AI software also appears to have a strong focus on preserving the privacy of its users, which would align with the company's existing privacy-focused features - such as iCloud Private Relay.
Apple holds privacy high as a basic human right and will prioritize it with AI
Even in Apple's test environments, user privacy warnings still exist. According to individuals familiar with the software, the company's test utilities will display a message before the Ajax LLM uses any information from Safari or the Messages app.
By eliminating the need for cloud-based processing -- at least in the most basic of situations -- the chance for privacy-related issues is greatly reduced. If user data is not transmitted, the odds of it becoming compromised are obviously much lower.
Compared to existing implementations of AI in rival products and services, Apple's implementation is arguably innovative in its attempt at privacy preservation and efficient on-device processing.
How does Ajax compare against the competition?
Apple's AI-related improvements look to compete with the ever-increasing number of AI-enabled applications, which are available from a wide variety of third parties.
Siri is due for a big upgrade
Virtually every app, from Slack to Google Chrome, now features some degree of AI integration, in addition to the various AI chatbot apps currently on the market.
Products such as the Rabbit R1 or Humane AI Pin could also be regarded as competition, as they're commonly perceived as virtual personal assistants or smartphone replacements.
By creating an AI-infused version of Siri with on-device processing, Apple hopes to overshadow many existing implementations of generative AI - especially AI-themed physical products.
Both the Rabbit R1 and the Humane AI Pin function through cloud-based processing, with the latter even requiring a separate monthly subscription to operate. These devices are also commonly regarded as unfinished or only partially functional, as noted in various user reviews.
Apple will compete with products like the Rabbit R1 by offering vertically integrated AI software on established hardware
An AI-capable version of Siri would also allow for greater privacy protection with on-device processing, all without the cost of an added monthly subscription. Given Apple's established market presence, quality control practices, and available resources, Siri has renewed potential to become a serious competitive threat.
During Apple's most recent earnings call on May 2, Apple's CEO Tim Cook fielded two questions about the company's monetization of generative AI but did not provide concrete responses. Earlier in the call, Cook affirmed Apple's "unwavering focus on privacy" and said that the company has "advantages that differentiate" it from the competition regarding AI.
Relative to AI chatbot applications and services such as OpenAI's ChatGPT, an updated version of Siri would have its advantages in the form of on-device functionality. It would also leverage its existing integration with other built-in Apple applications, allowing for greater functionality than any ordinary chatbot.
As for Safari's upcoming Intelligent Browsing feature with text summarization, it likely aims to compete against prominent third-party browsers such as Arc Browser, which also contains its own text summarization feature.
Given Apple's recent interest in artificial intelligence, we should see some of these features debut at the annual Worldwide Developers' Conference later this year.
Read on AppleInsider
Comments
The Mac is a great machine and so is the iPhone. But search on the Mac and on the iPhone is about as terrible as it gets.
Any time I am stuck with having to look for a file or document I reach for an anti migraine tablet. I know I am going to get a massive headache at all the useless junk that is thrown at me which has nothing to do with the search criteria that I entered.
For instance my iPhone XS is the oldest that can run the current iOS 17, so it may well not be supported by iOS 18. While my XS still works well — even the original battery! — and does everything I want it to, I may be prodded to upgrade by the availability of useful AI.
Similarly, I would love to see Apple use AI to give Safari a new "Summarizer" feature which, when turned on, would display the web page in a manner similar to Safari Reader, but would summarize the page. I would expect web-content creators to get upset that someone is abridging their content, but since they haven't been upset about the Safari Reader, maybe they won't get upset about the Safari Summarizer either.
Maybe there are no legal grounds to object to a user creating summaries of web content for personal purposes. If the summary was being performed by an online service, there might be a legal problem with that. But if it's done on device, it's probably legal. This could be a problem for Google and Microsoft, because their AI services are all done online. Google and Microsoft are charging users for "summaries" of information from third party websites.
At the same time, Google respects a website owner's right to monetize their content and won't "summarize" without the content owner's permission. There have also been no legal objections to it either, as far as I know, so I fully expect Apple will handle it the same as Google; AI Summarize, sans ads, will only be available with a website owner's permission.
https://support.google.com/assistant/answer/14163109?hl=en ;
Is summarising text useful & helpful? Most people already do that in their minds, missing important context & info, making them ill informed, tools to summarise aren't going to improve that.
For me hope, Siri gets a big improvement and photo editing, which has been very basic for years, to make sure we buy crap apps and the developers still complain about Apple 😳 I would like to see best take in it & an eraser at the very least.
Other than that I'm not buying into the fake hype!
No offence taken, however your view is very blinkered and a big deal for people that don't speak the limited languages it so far supports. It's not niche, it's essential that it supports every language it's sold in!
Perhaps you read, too many summaries in Google, to be so ill informed?
'ya didn't follow the link, huh?
Nope. These things are on-device using Google AI:
So while some are done online, the resource-intensive Video Boost for instance, most of the AI features introduced with the Pixel 8 are done privately on the user's phone rather than sent to Google to accomplish. That makes Google's current use of on-device AI processing similar to the rumored integration of AI features on this fall's Pro iPhones.
However, if the reports are accurate, not all of Apple's AI features can be accomplished on-device. The more advanced features will still need cloud assistance.