Porting operating systems to Apple Silicon leagues harder than migrating software

Posted:
in Mac Software
Working with Apple Silicon directly is a tough task, a developer attempting to port Linux to run natively on the M1 chip advises, with Apple using a highly customized process to boot the Mac that is different from versions used by other 64-bit ARM systems.

Mac mini DTK


In its introduction of Apple Silicon to developers, Apple has provided assistance to developers to port their Intel-compatible apps over to M1. For developers performing more ambitious feats, such as porting Linux over to Apple Silicon, the task is multiple times harder.

In a blog post about the Asahi Linux project, the team discusses its findings in trying to set up an alternative boot kernel on Apple Silicon systems. While most of the feature has been implemented, the lack of support for a command that allows the installation of a non-Apple kernel led to an attempt to document the undocumented system.

The main hurdle faced was that Apple Silicon boots differently from PCs, and works "more akin to embedded platforms" like Android or iOS devices. There are differences and a "few bespoke mechanisms" in use, though Apple apparently made the boot process "feel closer" to an Intel Mac.

These differences result in some behaviors that were unexpected, such as how Apple Silicon Macs handles booting from external storage. It was also found the bootloader cannot show a graphical user interface and that the "Boot Picker" is a "full-screen macOS app, not part of the bootloader."

The developers believe the boot process is "not based on any existing standard" and instead uses "a bespoke Apple mechanism that has slowly evolved from the early days of iOS."

DeviceTree was selected to be used as part of the boot process, in part because it is similar to Apple Device Tree, which Apple Silicon uses. Both Apple Device Tree and the open DeviceTree standard are based on the Open Firmware specification, which was used for booting older PowerPC Macs.

However, the difference in binary format that cannot be easily converted automatically without high-level details about what the data represents is a hurdle for the project to cross. "Trying to unify Apple's and Linux's ideas of how device trees should work would be a nightmare," the project team claims.

To solve the problem, the team worked on "m1n1," a bootloader for Apple Silicon to "take care of as many Apple-isms' as possible" for anyone developing their own Linux or other OS ports. Using the took, which is based on a minimal environment originally made to examine the Nintendo Wii's security CPU, the project has started to document Apple's custom ARM instructions, system registers, and hardware like the Apple Interrupt Controller.

The Asahi Linux project joins another effort by Corellium to port Linux to M1. In January, Corellium mentioned a similar untraditional boot process and the use of a non-standard controller, as well as allegedly managing to boot Linux.

Directly working with Apple Silicon is a big jump in difficulty from the experience of porting Intel-based macOS apps over to the chip.

On March 12, Adobe revealed it had a "smooth experience" in converting Photoshop over. Along with initial Intel app support using Rosetta 2, the development team partnered with Apple to refactor and implement features such as Context Aware Fill to work with M1.

Praise was also offered for Apple's "significant investment in the developer toolchain and experience," elements that teams porting Linux to Apple Silicon simply do not have.

Stay on top of all Apple news right from your HomePod. Say, "Hey, Siri, play AppleInsider," and you'll get latest AppleInsider Podcast. Or ask your HomePod mini for "AppleInsider Daily" instead and you'll hear a fast update direct from our news team. And, if you're interested in Apple-centric home automation, say "Hey, Siri, play HomeKit Insider," and you'll be listening to our newest specialized podcast in moments.
«1

Comments

  • Reply 1 of 29
    netroxnetrox Posts: 992member
    That's why I cannot use M1 Macs for work. I must have Intel compatibility to run VMs at near native speed. Unfortunately, I will have to buy Intel laptop. 
    agilealtitude
  • Reply 2 of 29
    sflocalsflocal Posts: 5,465member
    netrox said:
    That's why I cannot use M1 Macs for work. I must have Intel compatibility to run VMs at near native speed. Unfortunately, I will have to buy Intel laptop. 
    If you run it as a VM, it should run at near native speeds... for ARM-specific OS'es of course.  Windows10 for ARM apparently works, but Microsoft isn't allowing it... yet.

    If you're referring to X86 OS'es, then yes... you'll have to go with an Intel Mac for the time being until someone figures out a way to run x86 OSes on ARM architecture.  It's why I bought a 2020 iMac.  I still have to run multiple flavors of x86 Windows and ASi will not do it for the foreseeable future.  Hopefully, when it comes time to retire my iMac in around 5-7 years, Apple will have some offerings, but for now... don't hold your breath.
    edited March 15 dewmeStrangeDaysAlex1Ntwokatmewwatto_cobra
  • Reply 3 of 29
    netrox said:
    That's why I cannot use M1 Macs for work. I must have Intel compatibility to run VMs at near native speed. Unfortunately, I will have to buy Intel laptop. 
    This has nothing to do with virtual machines.
    Vermelhowatto_cobra
  • Reply 4 of 29
    shaminoshamino Posts: 464member
    netrox said:
    That's why I cannot use M1 Macs for work. I must have Intel compatibility to run VMs at near native speed. Unfortunately, I will have to buy Intel laptop. 
    Your problem will exist whether or not the Linux community figures out how to write a boot-loader for M1 Macs.

    if you need to run x86 code without emulation, then you need an x86-based computer.  Right now, you can still get x86 laptops from Apple - two model MacBook Pros (one 13" and one 16") and one MacBook Air are still Intel-based.  When Apple stops selling these, you'll need to buy your computer from someone else.  But Apple's M1 boot-loader architecture has nothing to do with this.

    But, of course, you don't have to run Windows as a result of this.  That x86 laptop can run Linux natively, which can host your VMs.  If you think the computer will only be used for VM hosting, you can also install something like vSphere ( https://www.vmware.com/products/vsphere-hypervisor.html ) and avoid having anything other than a thin hypervisor between your VMs and the hardware.


    edited March 15 Alex1Napplguywatto_cobra
  • Reply 5 of 29
    lkrupplkrupp Posts: 8,894member
    netrox said:
    That's why I cannot use M1 Macs for work. I must have Intel compatibility to run VMs at near native speed. Unfortunately, I will have to buy Intel laptop. 
    So you’ve made the assumption that this will never happen. Why?
    Alex1Nwatto_cobra
  • Reply 6 of 29
    cloudguycloudguy Posts: 323member
    lkrupp said:
    netrox said:
    That's why I cannot use M1 Macs for work. I must have Intel compatibility to run VMs at near native speed. Unfortunately, I will have to buy Intel laptop. 
    So you’ve made the assumption that this will never happen. Why?
    If you are a pro user, NEVER buy hardware - or software - today in anticipation of functionality to be added tomorrow. And this is even with functionality that a major entity like Apple says is "coming soon." But in this case, Apple hasn't promised these features - Intel compatibility to run x86 VMs at near native speed - soon or even ever. The reason is that your job is, well just that. If he could find equivalent paying work using and promoting Apple hardware I am certain that he would take it. But since that isn't his job then he needs hardware that allows him to do it, whether Apple makes it or not.

    Good grief, it isn't that big a deal. He can still buy HomePod Minis, AirPods, Apple TVs, Apple Watches, iPhones, iPads and an M1 MacBook Air for personal use. But for his job he needs capable hardware. Now were this 2019 or even 2020 you would be able to honestly tell him "just get an Intel-based MacBook Pro, iMac or Mac Mini." But now you really can't. No Mac has anything newer than a 10th gen Intel CPU. Some of them even have 9th gen. But Windows and Linux machines with 11th gen Intel inside have better CPU performance and MUCH better GPU performance. Not only that but 12th gen Intel machines launch later this year, as do next-gen AMD machines. Those are all going to have 30%-50% better performance than any Intel Mac.

    Of course the M1X and M2 Macs are going to blast away anything that x86 will be capable of for some time beyond a top end Xeon or Threadripper CPU meant for servers. But that won't mean anything if those Macs can't run the software that you need for your job. He should wait until it can before he buys it no matter how fast it runs. I am someone who had the same experience. I badly wanted the Mac Mini, but my job required a machine with A. VMWare and B. 32 GB of RAM so that I could run at least 3 VMs simultaneously. So, I had to get an HP desktop - with the same hexacore Intel i5 that is in the Mac Mini - and a pair of RAM sticks. (The combined cost was what I would have paid for the $699 M1 Mac Mini with 8 GB RAM and less than the Intel Mac Mini. Yes, the Apple tax is real.) Later I needed another PC ... but needed to be able to virtualize Windows Server. So I got an 16 GB Intel NUC (again for significantly less than a Mac Mini). I may FINALLY be able to get an M1 Mac Mini later this year if the old Intel Mac Mini that I am currently using as an HTPC kicks the bucket, but that would be a personal device, not a work one. 

    Want and need are two different things and for work the latter has to be the driver. Had I gotten a Mac Mini instead of the HP or Intel NUC, I wouldn't be able to use either for work. And ultimately guys, yes the switch from x86 to ARM is going to cost Apple some pro users. It is no big deal as Apple will replace them with the iPhone and iPad casual users who now use Windows PCs.
    edited March 15 dewmeAlex1Ncanukstormmuthuk_vanalingamrevenant
  • Reply 7 of 29
    tzeshantzeshan Posts: 2,351member
    Apple Silicon should be booted the same as iPhone and iPad. Any people successfully ported Unix to iPhone? 
  • Reply 8 of 29
    dewmedewme Posts: 3,425member
    I'm not holding my breath waiting for x86 VMs running on Apple Silicon. But I also don't think this a serious impediment either. Realistically, the only thing this changes for me is that my next Mac purchase will probably not be an iMac. Instead, I will buy a Mac mini (or whatever display-less Mac Apple has available, the long rumored Mac Semi-Pro) along with a high res (4K minimum) monitor that has at least 2 video input ports. One port is dedicated to the Mac and the other one dedicated to an Intel box, which for me is currently an Intel NUC. I already have the keyboard and mouse for supporting multiple hosts and I'm already sharing files between machines, real and virtual, using NAS and cloud.

    Yeah, I know that having a second dedicated machine for hosting x86 based VMs isn't for everyone. But I actually prefer this solution because having a second machine is kind of nice if one is out of commission, if one is in the middle of a big update, one is running a big CPU intensive operation, or if you need full native (non-VM) execution of a Windows instance. You can still use the Apple Silicon Mac for ARM VMs. I also keep all my VMs on an external SSD so they're not bound to any specific machine.

    Physically, the total desktop footprint of a 4K monitor (I always have 2 monitors), a Mac mini, a NUC, plus keyboard and mouse/trackball is not much different than an iMac, especially if you mount the NUC on the back of the monitor. Maximum versatility in minimum space. Between the Apple Silicon Mac mini and the NUC you can test on just about every combination of targets there are, except for x86 Mac. I suppose that a second Intel Mac mini with Bootcamp instead of a NUC would work too, and fill in that one gap.

    If I was developing professionally for Mac today, and up until the Apple Silicon transition is complete and nobody is supporting x86 Mac apps any longer, I'd be leaning on my boss to supply me with 2 Mac minis, one M1 and one Intel. I don't think this is an unusual or unreasonable request - at all.
    Alex1NcanukstormsocalbrianVermelhoshaminowatto_cobra
  • Reply 9 of 29
    netrox said:
    That's why I cannot use M1 Macs for work. I must have Intel compatibility to run VMs at near native speed. Unfortunately, I will have to buy Intel laptop. 
    why not set up a basic server local to you? i have an M1 MacBook and used a 2015 MacBook Pro with ESXi installed to run a few VMs. the install was a bit wacky and hacky but it is working good now. being an old laptop it has a built-in battery backup as well.

    could use any old Intel machine really. or pick up an old Dell server for a few hundred off eBay if you have some place to put it (fan noise reasons mostly). 
    watto_cobra
  • Reply 10 of 29
    Weird headline indicates operating systems aren’t software, which is nonsensical on its face: operating systems are the quintessential software, upon which separate user space applications depend on to exist, as abstract libraries full of functions that allow users to interface between the physical hardware interfaces and the user space applications.  As such, they must depend on the most concrete of details of a system, down to the most non-portable details of the CPU and all associated hardware: even with the best documentation, that may not be the easiest thing to accomplish if the memory model at the CPU cache coherency level (especially with interrupt handling) is different from another platform, as well as low-level memory synchronization instructions (memory barriers) and the like.

    That being said, all operating systems that run on more than a single type of machine are generally designed with a hardware abstraction layer, and that interacts as needed with low-level boot code, which may also need to be done custom.  Once that’s done, the rest of the OS (once you have compilers designed to properly use the CPU instructions correctly) should largely just be a recompile, combined with ensuring there are device-specific drivers.

    So, if you have more to worry about than adapting compiler toolchains, boot code, a little abstraction code for certain optimized APIs in the OS, and the HAL and device drivers, you’re doing it wrong.

    The rest is mechanical in nature.
    Alex1Nrundhvidwatto_cobra
  • Reply 11 of 29
    bloggerblogbloggerblog Posts: 1,996member
    netrox said:
    That's why I cannot use M1 Macs for work. I must have Intel compatibility to run VMs at near native speed. Unfortunately, I will have to buy Intel laptop. 
    I could be wrong but I thought this article is referring to booting in Linux. VMs shouldn’t be much of a problem
    Alex1N
  • Reply 12 of 29
    crowleycrowley Posts: 7,451member
    lkrupp said:
    netrox said:
    That's why I cannot use M1 Macs for work. I must have Intel compatibility to run VMs at near native speed. Unfortunately, I will have to buy Intel laptop. 
    So you’ve made the assumption that this will never happen. Why?
    There's no assumption there.  It isn't compatible now.  Some people need it to be compatible to work.  Therefore some people cannot work on M1 Macs right now.

    Tomorrow is a different story.
    Alex1Nagilealtitudemaciekskontakt
  • Reply 13 of 29
    cloudguycloudguy Posts: 323member
    netrox said:
    That's why I cannot use M1 Macs for work. I must have Intel compatibility to run VMs at near native speed. Unfortunately, I will have to buy Intel laptop. 
    I could be wrong but I thought this article is referring to booting in Linux. VMs shouldn’t be much of a problem
    If you need x86 Linux instead of ARM Linux it is a problem. Linus Torvalds said as much last year. 
  • Reply 14 of 29
    cloudguycloudguy Posts: 323member
    Weird headline indicates operating systems aren’t software, which is nonsensical on its face: operating systems are the quintessential software, upon which separate user space applications depend on to exist, as abstract libraries full of functions that allow users to interface between the physical hardware interfaces and the user space applications. 

    So, if you have more to worry about than adapting compiler toolchains, boot code, a little abstraction code for certain optimized APIs in the OS, and the HAL and device drivers, you’re doing it wrong.

    The rest is mechanical in nature.
    Point 1: good grief. People who have, you know, actually studied computer architecture and organization please speak up. In the meantime ... it has been common parlance for decades to refer to application software as software and systems software as the OS. And also to refer to both the kernel and the software that runs on top of the kernel but below the application software (conceptually) like the GUI as "the OS" when you really shouldn't, especially when it comes to Linux. So ... I don't get where you are going with this?

    Point 2: "You're doing it wrong ... the rest is mechanical in nature." Ah. That is where you are going with this. The first is FALSE and the second is VERY FALSE. Clearly you haven't done this before. Let the people who have be the ones to talk about this because they are the ones who are actually qualified to do so. Go try to port a major application from x86 to ARM on the same OS (i.e. Linux or Android) and see how "mechanical" it is. Not even that ... merely port a 32 bit application to a 64 bit one. Done it before? It is neither easy or fun. But I guess it should have been "mechanical" and it was "being done wrong" eh? Goodness where do people like this come from?
    dewmeAlex1Nshamino
  • Reply 15 of 29
    cloudguycloudguy Posts: 323member
    tzeshan said:
    Apple Silicon should be booted the same as iPhone and iPad. Any people successfully ported Unix to iPhone? 
    While you are not wrong from the perspective of Apple, you are ignoring that a lot of users in the tech arena have been using macOS, Windows and *nux almost interchangeably for 15 years. While before this would have been impossible, the combination of common hardware platforms and multiplatform applications based on open standards made it (almost) easy. Back in the day ... not so much. PC was once exclusively used to refer to "IBM machine that isn't a mainframe, server or minicomputer." Even after the "PC clones" made it a more generic term, until 2005 Apple and Wintel/WinAMD were so divergent that you couldn't use the same peripherals: the ports were different. 

    Open standards (like USB for ports!), Java and other multiplatform tools, the web, cloud, SaaS etc. created a situation where "everything is a PC" whether it was running Windows, macOS or even Linux. (ChromeOS is the only outlier.) The current generation of tech workers expects to be able to use pretty much any hardware they buy basically the same way. If you are someone who actually "uses a Mac like a Mac" - meaning that you rely primarily on software written for macOS - then you won't understand how big a chance this is for these people. Some people "get it" ... I saw the first raft of articles stating that Macs should no longer be considered PCs last year. But if you are someone that has made building applications, networks and infrastructure based on open multiplatform standards with hardware as interchangeable parts the basis of your entire career - and if you are under 40 you have done exactly that - then this is a major change.

    Enough to take your rack-mounted Mac Minis and your Xeon Mac Pros out of your infrastructure? Well if you were responsible for 500 devices, would you still use 75 of them if they didn't boot the same way? Or would you go for uniformity? Exactly. It wasn't a problem for iPhones and iPads because no one deploys the Apache applications on iPhones and iPads that are used to encrypt outbound web traffic. Instead, they deployed it on rack-mounted Mac Minis running Linux that were used to encrypt web traffic that originated on the iPhones and iPads. These folks would prefer to continue to use the Mac Minis for this because they perform well, are extremely reliable and have small footprints. But if problems like this can't be solved, they won't be able to. They will likely switch to Intel Nucs or small form factor Dells instead, even if a single M1 Mac is capable of handling the load of 2 Dells. 
    Alex1N
  • Reply 16 of 29
    sflocalsflocal Posts: 5,465member
    We are still buying Intel Macs for folks here at the office.  While the apps we use will run under rosetta, the only thing preventing us is that everyone here has at least two monitors and there are too many complaints out there with the M1 Macs not playing well with them, and quite a few with a single external monitor.  
    Alex1N
  • Reply 17 of 29
    cloudguy said:
    lkrupp said:
    netrox said:
    That's why I cannot use M1 Macs for work. I must have Intel compatibility to run VMs at near native speed. Unfortunately, I will have to buy Intel laptop. 
    So you’ve made the assumption that this will never happen. Why?
    If you are a pro user, NEVER buy hardware - or software - today in anticipation of functionality to be added tomorrow. And this is even with functionality that a major entity like Apple says is "coming soon." But in this case, Apple hasn't promised these features - Intel compatibility to run x86 VMs at near native speed - soon or even ever. The reason is that your job is, well just that. If he could find equivalent paying work using and promoting Apple hardware I am certain that he would take it. But since that isn't his job then he needs hardware that allows him to do it, whether Apple makes it or not.

    Good grief, it isn't that big a deal. He can still buy HomePod Minis, AirPods, Apple TVs, Apple Watches, iPhones, iPads and an M1 MacBook Air for personal use. But for his job he needs capable hardware. Now were this 2019 or even 2020 you would be able to honestly tell him "just get an Intel-based MacBook Pro, iMac or Mac Mini." But now you really can't. No Mac has anything newer than a 10th gen Intel CPU. Some of them even have 9th gen. But Windows and Linux machines with 11th gen Intel inside have better CPU performance and MUCH better GPU performance. Not only that but 12th gen Intel machines launch later this year, as do next-gen AMD machines. Those are all going to have 30%-50% better performance than any Intel Mac.

    Of course the M1X and M2 Macs are going to blast away anything that x86 will be capable of for some time beyond a top end Xeon or Threadripper CPU meant for servers. But that won't mean anything if those Macs can't run the software that you need for your job. He should wait until it can before he buys it no matter how fast it runs. I am someone who had the same experience. I badly wanted the Mac Mini, but my job required a machine with A. VMWare and B. 32 GB of RAM so that I could run at least 3 VMs simultaneously. So, I had to get an HP desktop - with the same hexacore Intel i5 that is in the Mac Mini - and a pair of RAM sticks. (The combined cost was what I would have paid for the $699 M1 Mac Mini with 8 GB RAM and less than the Intel Mac Mini. Yes, the Apple tax is real.) Later I needed another PC ... but needed to be able to virtualize Windows Server. So I got an 16 GB Intel NUC (again for significantly less than a Mac Mini). I may FINALLY be able to get an M1 Mac Mini later this year if the old Intel Mac Mini that I am currently using as an HTPC kicks the bucket, but that would be a personal device, not a work one. 

    Want and need are two different things and for work the latter has to be the driver. Had I gotten a Mac Mini instead of the HP or Intel NUC, I wouldn't be able to use either for work. And ultimately guys, yes the switch from x86 to ARM is going to cost Apple some pro users. It is no big deal as Apple will replace them with the iPhone and iPad casual users who now use Windows PCs.
    I would go so far as to say NEVER buy or use Apple products if your future work depends on the current setup.  Apple (in)famously removes functionality, pulling the rug out from under you, as history has shown over and over.
    edited March 15
  • Reply 18 of 29
    tzeshantzeshan Posts: 2,351member
    cloudguy said:
    tzeshan said:
    Apple Silicon should be booted the same as iPhone and iPad. Any people successfully ported Unix to iPhone? 
    While you are not wrong from the perspective of Apple, you are ignoring that a lot of users in the tech arena have been using macOS, Windows and *nux almost interchangeably for 15 years. While before this would have been impossible, the combination of common hardware platforms and multiplatform applications based on open standards made it (almost) easy. Back in the day ... not so much. PC was once exclusively used to refer to "IBM machine that isn't a mainframe, server or minicomputer." Even after the "PC clones" made it a more generic term, until 2005 Apple and Wintel/WinAMD were so divergent that you couldn't use the same peripherals: the ports were different. 

    Open standards (like USB for ports!), Java and other multiplatform tools, the web, cloud, SaaS etc. created a situation where "everything is a PC" whether it was running Windows, macOS or even Linux. (ChromeOS is the only outlier.) The current generation of tech workers expects to be able to use pretty much any hardware they buy basically the same way. If you are someone who actually "uses a Mac like a Mac" - meaning that you rely primarily on software written for macOS - then you won't understand how big a chance this is for these people. Some people "get it" ... I saw the first raft of articles stating that Macs should no longer be considered PCs last year. But if you are someone that has made building applications, networks and infrastructure based on open multiplatform standards with hardware as interchangeable parts the basis of your entire career - and if you are under 40 you have done exactly that - then this is a major change.

    Enough to take your rack-mounted Mac Minis and your Xeon Mac Pros out of your infrastructure? Well if you were responsible for 500 devices, would you still use 75 of them if they didn't boot the same way? Or would you go for uniformity? Exactly. It wasn't a problem for iPhones and iPads because no one deploys the Apache applications on iPhones and iPads that are used to encrypt outbound web traffic. Instead, they deployed it on rack-mounted Mac Minis running Linux that were used to encrypt web traffic that originated on the iPhones and iPads. These folks would prefer to continue to use the Mac Minis for this because they perform well, are extremely reliable and have small footprints. But if problems like this can't be solved, they won't be able to. They will likely switch to Intel Nucs or small form factor Dells instead, even if a single M1 Mac is capable of handling the load of 2 Dells. 
    One workaround is Apple changes the boot mechanism from iOS 14, iPhone 13 to be the same for Apple Silicon. But what about the fate of one year old Apple Silicon Mac?
    watto_cobra
  • Reply 19 of 29
    twokatmewtwokatmew Posts: 19unconfirmed, member
    sflocal said:
    netrox said:
    That's why I cannot use M1 Macs for work. I must have Intel compatibility to run VMs at near native speed. Unfortunately, I will have to buy Intel laptop. 
    If you run it as a VM, it should run at near native speeds... for ARM-specific OS'es of course.  Windows10 for ARM apparently works, but Microsoft isn't allowing it... yet.

    If you're referring to X86 OS'es, then yes... you'll have to go with an Intel Mac for the time being until someone figures out a way to run x86 OSes on ARM architecture.  It's why I bought a 2020 iMac.  I still have to run multiple flavors of x86 Windows and ASi will not do it for the foreseeable future.  Hopefully, when it comes time to retire my iMac in around 5-7 years, Apple will have some offerings, but for now... don't hold your breath.
    I bought a 2020 MacBook Pro for the same reason. 
    edited March 15
  • Reply 20 of 29
    If enough people let Apple know that they won't buy Apple Silicon computers if they can't boot another operating system, I am sure Apple will make that an option. To be fair, though, I am not sure that is all that important. There are plenty of computers out there that can boot multiple operating systems. It doesn't have to be a Mac, does it?
Sign In or Register to comment.