Craig Federighi is on a mission to let you move work across all of your devices, instantly...

Posted:
in macOS

From using an iPhone as a webcam on Mac to Universal Control, for years Apple has been making it easier to work across multiple devices -- and now it has a plan to mean you can instantly switch between them, too.

Turning iPad, Mac, and more into a single unit
Turning iPad, Mac, and more into a single unit



Universal Control was the wow moment of WWDC 2021, as Craig Federighi dragged a file from his iPad, across his MacBook Pro screen, and dropped it into Final Cut Pro on an iMac. Universal Control gave us the ability to use one keyboard and trackpad -- in this case his MacBook Pro -- and seamlessly use other nearby devices.

No wonder Federighi looked excited. But it turns out that this may have been less his turn to present something at WWDC, and more of a project he is particularly invested in.

For a newly-revealed patent application about extending this feature still further, is credited to Federighi and two colleagues. "Continuity of Applications Across Devices" proposes a subtle but significant improvement on what Apple summarizes as its Continuity features.

Universal Control is the most striking of those, but there is also how the music playing on an iPhone can be handed off to a convenient HomePod. Then there's the tremendous AirDrop, the ability to use an iPhone as a camera, or to have your Apple Watch unlock your Mac.

Apple is all-in on this Continuity idea of recognizing that we tend to have multiple devices, and will benefit from integrating them more. The idea in this patent application centers on how we might turn to either the best or the handiest device to do something, only to realise another device would be better.

"[For example], the user may power-up a laptop, launch an e-mail application, find the specific e-mail for which a response is needed, and begin to prepare the response," says Apple. "It would be helpful if the user could transition from use of one device to another, such as transition from reading an e-mail on one device to drafting a reply on another device, in a seamless and cognitively continuous manner."

One Apple ID, one account



You've done this. You've begun replying to an email on your iPhone, only to find that there's more to say than a quick emoji. Right now, you could delete that draft and open up Mail on your Mac to get the benefit of typing on a full keyboard instead of tapping with your finger.

Right now you can even select all of the text you've written, copy it, turn to your Mac, open Mail, find the message, click on Reply, and paste your existing text into your reply. There is also iCloud syncing which for some apps will mean a draft is copied between devices, saving you even the copy-and-paste effort.

But Apple wants it to be that you literally just put down one device and pick up the other, to carry on working precisely where you were.

"[If] the user is using one application on a first device, it would be helpful for the second device to which the user is transitioning-to automatically launch the same application," says Apple, "so that the user may continue without loss of progress."

So you're in Mail, writing a reply, and you realise it would be better to do that on your Mac. Put down your iPhone, open your MacBook Pro's lid, and Mail has launched itself in readiness.

Mail is waiting for you -- and so is the message you were replying to. If you haven't cancelled the one on your iPhone, that's what you see now, the email you've begun writing.

That email reply is just waiting for you to carry on precisely where you were. This seems handy, it seems quite nicely convenient, but Apple argues that actually, there's more to it than convenience.

"Such techniques can reduce the cognitive burden on a user who switches between uses of multiple computing devices, thereby enhancing productivity," says the patent application. "Further, such techniques can reduce processor and battery power otherwise wasted on redundant user inputs."

Ultimately, while Apple would gladly have us buy very many devices, the aim is for us to feel as if we have just one. It comes in different sizes and has different capabilities, but it's the device we need to do what we have to right now.



Read on AppleInsider

Comments

  • Reply 1 of 7
    jhalmosjhalmos Posts: 26member
    Make it work when my MacBook Pro is in clamshell and you have my attention. Lost the simple ability to move my cursor to my iPad when I moved from a Mac mini M1 to a MacBook Pro M2. Argh.
    williamlondonwatto_cobra
  • Reply 2 of 7
    StrangeDaysStrangeDays Posts: 13,102member
    How long before competing device makers sue claiming it’s an unfair advantage…
    jas99williamlondonmacxpresswatto_cobra
  • Reply 3 of 7
    mattinozmattinoz Posts: 2,485member
    Apple issued a similar patent in the early days of the iPad, which, according to the patent description, would allow Mac apps to relay and interface to an iPad-like device. 

    Now, it never really happened like that; indeed, it really was the first edition of watch Apps that were hosted by the phone, but the interface was run by the watch as an app extension. 

    Still, it seemed like a great idea to expand. Why bring the whole app onlineon each device when all you need in the example is only a small part of it? Why couldn't drafting an email be a super-lightweight app extension that jumps between devices with ease?

    Would help the CPU/GPU multi-task the app if if was divided to task interfaces. 




    watto_cobra
  • Reply 4 of 7
    michelb76michelb76 Posts: 698member
    That would require them to make iCloud syncing robust so I guess that's a good thing? Would be nice to have all your open windows, files apps and complete state, syncing instantly. 
    watto_cobra
  • Reply 5 of 7
    jellybellyjellybelly Posts: 138member
    michelb76 said:
    That would require them to make iCloud syncing robust so I guess that's a good thing? Would be nice to have all your open windows, files apps and complete state, syncing instantly. 
    Why wouldn’t the process use local WiFi or BT for very little (unnoticeable) lag, even as syncing to iCloud happens in the background. Dragging from Mac to iPad or vice versa is over WiFi or Bluetooth and seamless with no lag.  I believe continuity works that way.  My Apple Watch unlocks my MBP the instant I’m within a foot of the MBP.  

    If you had an Internet outage for say 5 minutes, Continuity still works with local signal. In addition to WiFi or BT, both devices might have an Apple near-field communication chip and protocol for communication (more privacy). If you quit the Mail App on one device, the remaining edits on the live device will sync to iCloud when reconnected over the restored Internet connection.  Restarting the Mail App you closed will sync to the other live device and will match the iCloud sync. 
    And according to the expected behavior, the closed Mail App might re-open automatically if the intention is to have a synced live document open on each device so for example, you might want to initiate taking a photo from your iOS device to insert in the email and resume typing on the Mac ( Continuity Camera).   

    I like this metaphor that suggests the way it will work—the devices work as though you have a virtual single device with arms and legs ( other devices) to work in concert with your task.  

    Apple might use AI to even detect a sentence that’s stating what your current heart rate is and would automatically request that data from your Apple Watch, including a quick permission confirmation of the private data, and insert that data from your Apple Watch into the sentence on the Mac or other Apple composing device (Mac, iOS or VisionPro for now).
  • Reply 6 of 7
    Gah!  No!  DO NOT WANT!

    When I'm goofing off on the personal laptop and a work call comes in, I don't want everything to jump to the work laptop, it would just be in the way.  And there's a reason I've got 5 computers on my home office desk, and it's not because I want them all to be doing the same thing.

    I do wish copy & paste worked better when I'm dealing with a VM.  Not Apple's fault, I blame VMware - and MicroSloth, because they didn't put command keys on their keyboards and misused control instead.  And of course Linux does the same thing, because that's the hardware it usually runs on.
  • Reply 7 of 7
    michelb76michelb76 Posts: 698member
    michelb76 said:
    That would require them to make iCloud syncing robust so I guess that's a good thing? Would be nice to have all your open windows, files apps and complete state, syncing instantly. 
    Why wouldn’t the process use local WiFi or BT for very little (unnoticeable) lag, even as syncing to iCloud happens in the background. Dragging from Mac to iPad or vice versa is over WiFi or Bluetooth and seamless with no lag.  I believe continuity works that way.  My Apple Watch unlocks my MBP the instant I’m within a foot of the MBP.  

    If you had an Internet outage for say 5 minutes, Continuity still works with local signal. In addition to WiFi or BT, both devices might have an Apple near-field communication chip and protocol for communication (more privacy). If you quit the Mail App on one device, the remaining edits on the live device will sync to iCloud when reconnected over the restored Internet connection.  Restarting the Mail App you closed will sync to the other live device and will match the iCloud sync. 
    And according to the expected behavior, the closed Mail App might re-open automatically if the intention is to have a synced live document open on each device so for example, you might want to initiate taking a photo from your iOS device to insert in the email and resume typing on the Mac ( Continuity Camera).   

    I like this metaphor that suggests the way it will work—the devices work as though you have a virtual single device with arms and legs ( other devices) to work in concert with your task.  

    Apple might use AI to even detect a sentence that’s stating what your current heart rate is and would automatically request that data from your Apple Watch, including a quick permission confirmation of the private data, and insert that data from your Apple Watch into the sentence on the Mac or other Apple composing device (Mac, iOS or VisionPro for now).
    I will just be happy as to not lose data for once, because reliable syncing is apparently an unsolved problem.
Sign In or Register to comment.