Apple's on-device email categorization is a feature years in the making

Posted:
in iOS edited July 8

Apple recently announced an all-new email categorization feature for iOS 18, but a newly-discovered prototype of the UI dates back to around five years ago, before the release of iOS 13.

Screenshots of Apple's Mail app in iOS 13 and iOS 18 with mail sorting categories
Apple's email categorization feature was originally created with iOS 13 in mind



At the company's annual WWDC on June 10, Apple showcased a significant update to its in-house mail application. By analyzing the text contents of emails through machine learning, the new-and-improved Mail app will be able to organize emails into several pre-defined categories:


  • Primary - displays time-sensitive email messages from people the user likely knows, along with other relevant information

  • Transactions - for receipts and order confirmations

  • Updates - which encompasses newsletters and social media notices

  • Promotions - for marketing and sales-related emails

AppleInsider

first broke the news on this feature ahead of WWDC in our exclusive report on Project BlackPearl. This was the development-era codename for Apple's latest email categorization feature, which introduces on-device email categorization through machine learning.

Following the feature's official announcement, we learned that Apple developed an early prototype of the feature's user interface around five years ago. Speaking to a known collector of Apple prototype devices, AppleInsider was able to obtain imagery and a video of the iOS 13-era UI Apple created for its email sorting feature.

An in-depth look at Apple's prototype UI for email classification



The early prototype of Apple's email categorization feature was discovered on a development device, running an InternalUI version of iOS 13.0. InternalUI builds of iOS are commonly used by Apple's software engineering teams for development, testing, and quality control purposes.

Smartphone displaying an email inbox with messages from LinkedIn, Skrill, Reddit, and StockX, on a wooden table.
Apple created a prototype of its email categorization feature more than five years ahead of launch



Because of this, they often contain early implementations of then-unreleased or postponed features and settings, as is the case with the email categorization functionality. Such versions of iOS are also commonly found on specialized prototype or development-type devices with unique attributes.

The early prototype interface for Apple's email categorization was found on an iPhone XS Max with a unique model number beginning with 993. Model numbers like these can often be found on the company's prototype units and devices intended for OS development.

To activate the new email classification UI, users of Apple's pre-release operating systems had to navigate to Settings > Internal Settings > Mail. The option labeled "Enable Prototype UI" was used to toggle the in-development user interface.

Once activated, the Mail application displays a series of brightly colored icons towards the top of the application window. The prototype user interface bears great resemblance to the finished product Apple previewed in June.

The differences and similarities between the iOS 13 prototype and iOS 18 preview of Apple's email categorization



In the version shown off during Apple's WWDC keynote, the different mail category icons were displayed in the same overall position with arguably minor stylistic differences compared to the prototype UI.

Smartphone screen with Mailboxes showing inbox categories: all mail, primary, social, updates, and promotions, each with zero unread emails. A finger is touching the screen.
The iOS 13 prototype of Apple's email categorization feature is quite similar to its iOS 18 counterpart



While the iOS 18 version features icons that are gray until tapped or selected, the icons on the iOS 13 prototype are always displayed in full color. Interestingly, even the colors of the icons are similar across both OS versions, with only yellow being missing from the final iOS 18 variant of the feature.

Tapping the category icons yields the same result on both the iOS 13 prototype and the iOS 18 version seen in Apple's keynote video. When an icon is tapped, the full name of the category is revealed, and emails belonging to said category are displayed on-screen.

The iOS 13 prototype of the email categorization feature is largely non-functional, apart from the core UI elements themselves. While it is currently unclear why this feature was postponed, it's not the first time Apple has delayed an OS feature.

In 2020, an early InternalUI build of iOS 14 was leaked from a development-fused iPhone 11 in China. The filesystem was initially sent out to a small number of individuals, but it eventually made its way to the press, which uncovered unannounced features such as App Clips and Wallpaper Collections.

While App Clips would become available later in 2020, the Wallpaper Collections feature was nowhere to be seen until two years later. Apple finally released the feature iOS 16, two major operating system releases after its initial implementation.

The early development-era prototype of Apple's on-device email categorization functionality provides useful insights as to how key operating system features can change ahead of release and could offer some ideas as to where Apple originally got the idea for its feature.

Another round of Sherlocking?



The use of machine learning for the categorization of emails is hardly a new or innovative concept. Third-party email services have offered similar functionality for a number of years at this point.

Gmail inbox showing tabs for Primary, Social, Promotions, and Updates with several unread messages from various contacts. Navigation and search bar visible at the top.
Gmail has had email categorization since 2013



Gmail, for instance, has had AI-powered email sorting since 2013. This is more than a decade before Apple finally decided to implement and announce its take on the feature.

Google's email categorization functionality offers a selection of categories similar to those Apple announced recently:


  • Primary

  • Social

  • Promotions

  • Updates

  • Forums



The company's website states that Gmail uses machine learning to classify emails based on different "signals." Such signals included the text contents of an email, the email sender, and information about how other users interacted with similar emails, among other things.

To classify incoming emails, machine learning algorithms have to analyze the contents of an email, the surrounding context, and the language used. This information is then used to determine an email's purpose and put it into a specific category.

At its WWDC keynote, Apple said that its email categorization feature would be able to organize emails in more or less the same way. Apple also announced a new Digest view for Mail, which displays all messages from a particular business, indicating that email senders are also taken into account.

Overall, there are undeniable similarities between Apple's and Google's email sorting features. Both make use of machine learning and organize emails into more or less the same categories.

Screenshot of a Yahoo Mail inbox with an advertisement offering vision correction services starting at $299.
Yahoo Mail has its own email sorting feature known as Smart Views. Image credit - Yahoo



Other third-party email services, such as Yahoo, also offer similar email categorization functionality in the form of Smart Views. Apple could simply be playing catch-up and trying to capitalize upon current trends with its Apple Intelligence AI initiative

In recent years, the prominence of artificial intelligence and machine learning technology has increased significantly. Everyday consumers now have access to powerful AI tools such as OpenAI's ChatGPT and Google's Gemini, all of which can make day-to-day tasks significantly easier.

In 2023, Yahoo introduced generative AI improvements to its Mail service, which include AI-generated email replies that match the users' tone. Improved search functionality was also added, meaning that users could type a question such as "How much did I spend on groceries last week," and the software would display the email with the desired information.

While Yahoo's features rely on the AI of Google Cloud, Apple's AI enhancements make use of the company's in-house cloud-based or on-device LLMs. This allows Apple to offer similar features, such as email categorization and AI-generated replies, while maintaining user privacy, giving it an advantage over rival products.

Apple's version of email categorization is sure to benefit users in many different ways. It could lead to better time management, increased productivity, and a reduction in stress, as users will no longer need to manually sort through a cluttered inbox to find an important message.

Other iOS 18 Mail features, such as smart Replies and AI-generated email summaries also present noteworthy improvements. Through Smart Replies, users will be able to reply to emails much more quickly with AI-assisted responses, while summarization will boil complex emails down to essential data points.

These latest Mail-related features and enhancements were all announced at WWDC and are expected to become available later in 2024. They will roll out with Apple Intelligence in iOS 18, iPadOS 18, and macOS Sequoia.



Read on AppleInsider

Comments

  • Reply 1 of 12
    I use GMail's categorization pretty frequently to delete scads of emails quickly.  It'll be nice to be able to do that directly on my phone instead of going to a web page.  And yes, I know I could use the GMail app on my phone.  I choose not to do so.
    JaiOh81williamlondonAlex1NOferbaconstangwatto_cobra
  • Reply 2 of 12
    MisterKitMisterKit Posts: 508member
    So apparently the concept worked on iPhones as old as the 6s. Hmmmm........
    williamlondonwatto_cobra
  • Reply 3 of 12
    Dougie.SDougie.S Posts: 44member
    MisterKit said:
    So apparently the concept worked on iPhones as old as the 6s. Hmmmm........
    Or maybe didn’t which is why it’s only coming now.
    williamlondonwatto_cobra
  • Reply 4 of 12
    hmlongcohmlongco Posts: 552member
    MisterKit said:
    So apparently the concept worked on iPhones as old as the 6s. Hmmmm........
    Well, apparently it didn't work since it was never released...
    williamlondonwatto_cobra
  • Reply 5 of 12
    I use GMail's categorization pretty frequently to delete scads of emails quickly.  It'll be nice to be able to do that directly on my phone instead of going to a web page.  And yes, I know I could use the GMail app on my phone.  I choose not to do so.
    I do the same, I open up Spam folder and immediately delete forever 
    Oferbeowulfschmidtwatto_cobra
  • Reply 6 of 12
    kkqd1337kkqd1337 Posts: 446member
    Apple have neglected mail for so long I doubt most users would even give it a second chance.

    After weighing up all the options, I personally chose a Microsoft Exchange paid for account for mail and am pretty happy with the Outlook app.
    williamlondon
  • Reply 7 of 12
    XedXed Posts: 2,704member
    kkqd1337 said:
    Apple have neglected mail for so long I doubt most users would even give it a second chance.

    After weighing up all the options, I personally chose a Microsoft Exchange paid for account for mail and am pretty happy with the Outlook app.
    1) Are you talking about mail as a service or Mail as an app?

    2) I have several iCloud-based mail accounts all utilizing the Mail apps for iOS and macOS, as well as MS and Gmail-based mail accounts that also utilize the Mail apps in iOS and macOS. So what exactly do you think is so neglected that I shouldn't be using anything from Apple for my email?
    mike1watto_cobra
  • Reply 8 of 12
    iadlibiadlib Posts: 99member
    There is absolutely no valid reason this feature took this long to develop. I feel like Apple needs to hire more software engineers. They came out swinging with the iPhone and OS X and for years rode that momentum all the while allowing the basic apps to flounder. They should have invested more in the basics than they have. They are only now playing catch up with various features to even be competitive vs slack, Google, etc. this is what happens when you are blinded by profit. 
  • Reply 9 of 12
    9secondkox29secondkox2 Posts: 2,907member
    Now I know why Sjrouji had that smirk when that lady interviewed he and Ternus. 

    He knees Apple has been cranking away on this for years. 

    You don’t create what Apple has come up with in a couple years. This was a big lead-in. 

    Apple was probably barf-laughing the whole time at what “ai” companies were doing while they were building the real stuff. 
    watto_cobra
  • Reply 10 of 12
    chadbagchadbag Posts: 2,013member
    Hopefully you can turn this off.  I have no need of it categorizing my email.  
  • Reply 11 of 12
    netroxnetrox Posts: 1,464member
    You know that it could be just because of potential patent and infringement issues?
    watto_cobra
  • Reply 12 of 12
    fastasleepfastasleep Posts: 6,447member
    I hope this works well, is flexible, and that there is real usable spam filtering and more reliable cross-device filters in Mail. Currently I have to filter everything server-side on my cPanel-based web server, and it sucks. 

    Also, some feature parity with iOS/iPadOS — like being able to sort by sender, which you can only do on the Mac! So annoying. 

    Lastly, I'm wondering why Mac Mail doesn't pre-cache content in downloaded messages. On a slow internet connection, just switching to a new unread message can take a few seconds just to display. This seems completely unnecessary. 

    I want to love Mail despite all my frustrations with it, so hoping this is a bigger overhaul than just the category sorting and summary stuff (another welcome change!).
    watto_cobra
Sign In or Register to comment.