FreeBSD adds support for Snow Leopards' Grand Central Dispatch

[AppleInsider]

Apple's Grand Central Dispatch technology, which debuted in Snow Leopard as a mechanism for optimizing parallelism across multiple cores and processors, has now been ported to FreeBSD.

Apple publicly announced plans to release its GCD technology as open source last month; the FreeBSD team demonstrated its early port of the new feature at EuroBSDCon 2009 in Cambridge, UK just days after Apple's announcement. Out of the box support for GCD is scheduled to appear with the release of FreeBSD 8.1.

The work required to port Apple's GCD event and concurrency framework to other operating systems is more complex than many other higher-level open source packages because GCD requires integration into the kernel (the core component of the operating system which manages processes, memory, and other hardware).

Most Unix-based software is highly portable between Mac OS X, Linux, and BSD, but significant kernel differences between these systems makes porting low-level, kernel-integrated technologies like GCD more work. In particular, Mac OS X uses a unique kernel design based on a hybrid of Mach and BSD. Porting GCD to FreeBSD required adaptations to account for a more conventional kernel environment without a Mach layer, such as using POSIX semaphores instead of Mach semaphores.

FreeBSD's porting efforts should help to make GCD easier to port to other operating systems with conventional Unix or Unix-like kernels, including OpenBSD, NetBSD, Linux, and Solaris.

In addition to kernel support, GCD requires compiler adaptations (a compiler is used to transform human readable source code language into the binary object code of an application). Apple's work on the Clang compiler front end and LLVM (Low Level Virtual Machine, the back end), makes porting this portion of GCD to other operating systems easier. This is particularly the case for FreeBSD, which has been quick to embrace support for Apple's Clang/LLVM open source projects.

The porting of GCD to other operating systems will help broaden the new technology's appeal. MacResearch, a website covering the use of Apple technologies in the scientific research community, observed last month that "with the possibility that Grand Central will become available on other Unix systems, the likelihood that Grand Central will be incorporated into command line tools is greatly increased.”

Significant new outside interest in GCD will also likely result in a wider support base for building parallelism compatible with Apple’s other open technologies, such as OpenCL and the aforementioned Clang/LLVM compiler tools.

FreeBSD and Mac OS X

FreeBSD is a direct heir of UC Berkeley's "standard distribution" of Unix, which was created first to augment and then to replace AT&T's commercial Unix along with its expensive and closed licensing. NeXT incorporated BSD's operating system technology into its desktop workstations in the late 80s, back when using version 4.3 BSD meant licensing AT&T's Unix as well.

After BSD developers removed the vestiges of AT&T's commercial Unix code in 1991, BSD was released under a permissive license on its own; significant portions were incorporated into a wide variety of operating systems from SVR4 Unix to Windows and in many embedded applications such as network routers and firewalls. The widespread use of BSD Unix, and in particular its high quality networking code, helped to foster smooth interoperability among different vendors' products on the Internet.

In 1992, AT&T sued BSD over copyright and trademark infringement issues that were not resolved until 1994. The legal uncertainties raised by those challenges resulted a vacuum filled by Linux, which first appeared in 1991. That resulted in shifting most open source operating system development from BSD to Linux.

Apple's acquisition of NeXT at the end of 1996 resulted in Mac OS X inheriting a BSD foundation; the company had previously offered a Linux distribution (MkLinux) for its PowerPC Mac users who wanted to run an open, Unix-like environment. While open source development is still centered on Linux, Apple's move to adopt NeXT's operating system technology not only made it the world's largest Unix vendor but also shifted a significant amount of the *nix-world's attention back toward BSD.



Apple has since regularly synced its Mac OS X development roadmap to FreeBSD in its efforts to incorporate open source modernizations into what had been the aging codebase of NeXTSTEP, although the company has also borrowed technologies from other BSDs, Linux, and Sun's OpenSolaris.

The company has also contributed technology back or funded the development of projects that various open source operating systems have incorporated, including Bonjour, Clang/LLVM, CUPS (the Common Unix Printing System), and now of course GCD.

[ View this article at AppleInsider.com ]

[Ireland]

Daniel, I don't know why you don't use your real name. Yes, my real name is Ireland. Ireland O'Reilly. Brother to Bill-O. One of these days that WiFi bug will get a post by the AI team, I hope. The thread over on the Apple discussion forums has reached 89 pages now. I think it's worth a story, but that would because I live in the real world.

Wine Grammar!

[bjhomer]

This is great news; Grand Central Dispatch greatly simplifies parallelization.

However, the author seems to have a bit of confusion regarding the acronym; GCD and CGD are not equivalent.

[vinea]

No GCD in the linux kernel unless Linux goes GPL v3 rather than GPL v2.

Or Linus ignores the FSF and goes with the Apache position that GPL V2 Only is compatible with the Apache License V2. Legally there's probably merit to that position but RMS would have a cow or something.

I think the drama would be much higher for option 2 and I don't see Linux going v3. So I'm guessing no GCD for llinux.

Here's a big



for freetards. BSD FTW.

[hmurchison]

Originally Posted by Ireland

Daniel, I don't know why you don't use your real name. Yes, my real name is Ireland. Ireland O'Reilly. Brother to Bill-O. One of these days that WiFi bug will get a post by the AI team, I hope. The thread over on the Apple discussion forums has reached 89 pages now. I think it's worth a story, but that would because I live in the real world.

Duly noted Ireland but lets not try to threadjack. The Wifi issues will have their day (and thread) eventually



As for GCD..the more it's used the better and more powerful it becomes. Good news although not unexpected IMO.

[Glockpop]

Originally Posted by bjhomer

This is great news; Grand Central Dispatch greatly simplifies parallelization.

However, the author seems to have a bit of confusion regarding the acronym; GCD and CGD are not equivalent.

Maybe it was just a typo?

Speaking of which, why are the AI comments all catty and obnoxious?

Ireland, sounds like you got time on your hands. Why don't you write up the wifi issue and submit it to AI? Sounds really fascinating.

[MacTel]

Glad to see open source working once again for Apple. GCD is a perfect technology to go that route. Apple will get benefit from the contributions of the BSD community on GCD in the future.

[crees!]

Originally Posted by bjhomer

However, the author seems to have a bit of confusion regarding the acronym; GCD and CGD are not equivalent.

Nothing to worry about. Just the dyslexia kicking in.

[aplnub]

This should be seen as a positive for open source and Apple. Great news.

[ascii]

This is good but I think it will need to be in Linux too if there's any chance of it becoming a defacto standard for multi-core utilization on Un*xes.

[rangerdavid]

The proof is in the pudding - Let's see benchmarks on the same multi-core hardware, using GCD and non=GCD builds of FreeBSD. Show me the advantage in handling multithreaded software. Anandtech? Anyone?

[7600/132]

Originally Posted by rangerdavid

The proof is in the pudding - Let's see benchmarks on the same multi-core hardware, using GCD and non=GCD builds of FreeBSD. Show me the advantage in handling multithreaded software. Anandtech? Anyone?

The existing port to FreeBSD is unfinished, and is missing many optimizations found in the Mac version. Benchmarking at this point might be a little premature.

But don't just take my word for it, get the story straight from the daemon's mouth.

[wizard69]

Originally Posted by hmurchison

Duly noted Ireland but lets not try to threadjack. The Wifi issues will have their day (and thread) eventually

Funny I don't even know what that issue is. Seems to me if it was a serious issue somebody would start a thread themselves.

As for GCD..the more it's used the better and more powerful it becomes.

The GCD spec and implementation is rather complete, it isn't going to become more powerful just because other platforms use it. The more platforms it runs on however the more developer mindshare it will grab outside of Apple developers.

The goal should be wide spread adoption to simplfy porting of software and to effectively compete against other similar technologies.

Good news although not unexpected IMO.

Yep people shouldn't be surprised as GCD is very good technology. However one must temper the response here as the BSD guys specifical described this as a preliminary release. To really be competitive with what Apple is offering they will have to redo the BSD scheduler and likely other low level parts of the OS. It simply takes time to debug and distill all that tech into a reliable revision. In any event the BSD guys ought to be applauded for going at this with a gusto.



Dave

[wizard69]

I'm not sure it was your intention or not but you make it sound like open source is foriegn to Apple. Frankly I think many in the open source world are glad Apple is around. It is not "again" but just another facet we are seeing here of Apples continued development in this regards.

Originally Posted by MacTel

Glad to see open source working once again for Apple. GCD is a perfect technology to go that route.

Well it isn't bad that is for sure. What I would find most interesting is what drove Apple in this direction. We will likely never know the details but they did bypass other alternative open software solutions for parallel code.

Apple will get benefit from the contributions of the BSD community on GCD in the future.

Yes this could be a huge gain for Apple but it might be a bigger gain for programing in general. Let's face it Apple is wedded to Objective C, but the BSD community could do a lot to establish new language revisions, that would adapt GCD, to things besides Objective C. Everything from ForTran to Ruby could concievably be re rigged to benefit from GCD tech. Of course in some cases that might be a lot of work, so instead we might see new languages spring up built around the concepts Apple puts forth here. There is a big potential for new scripting languages for one.



Dave

[robmel]

Not surprising the FreeBSD would be right up there. When Mac OS X was first arriving Apple recruited from the FreeBSD core team to support the integration. Jordan Hubbard is still there.

What's so exciting about this is that GCD is a fairly obvious idea but totally essential and elegantly implemented. Now that the ceiling on individual processor speed has been reached, the requirement for efficiently parcelling work out between multiple cores is evident. GCD takes the responsibility for thread management away from the app developer and puts it in the OS -- where it should be.

It'll take maybe 3-5 years before this is properly leveraged into mainstream apps. That's when your 16-core MacBook will really sing.

[bobrik]

Originally Posted by vinea

No GCD in the linux kernel unless Linux goes GPL v3 rather than GPL v2.

Or Linus ignores the FSF and goes with the Apache position that GPL V2 Only is compatible with the Apache License V2. Legally there's probably merit to that position but RMS would have a cow or something.

I think the drama would be much higher for option 2 and I don't see Linux going v3. So I'm guessing no GCD for llinux.

Apple could choose whatever license they liked/wanted since it's their code, so it left me thinking whether it could be Apple's intention not to have GCD on Linux. Apple could have chosen BSD license and nothing could prevent GCD being used in Linux kernel directly.

However, since BSDs and Linux don't have similar kernel internals (unlike Apple's Darwin and FreeBSD, which do), maybe this is irrelevant, since there would have to be a completely new implementation of the kernel components on Linux anyway and no code copying, which would mean the license of Apple's GCD does not matter for Linux port of kernel components.

[Ireland]

Originally Posted by wizard69

Funny I don't even know what that issue is. Seems to me if it was a serious issue somebody would start a thread themselves.

Like the 89 page thread on the Apple Discussion Boards?

[digitalclips]

Originally Posted by Ireland

Like the 89 page thread on the Apple Discussion Boards?

From the BBC blog you linked in. Thanks.

" At 8:34pm on 15 Oct 2009, DaveGreen55 wrote:
I had the problems with my iphone, but on advice updated with the latest software update and it has resolved all issues - i.e. lack of wifi, coma mode - I am now very happy iphone owner again!"

[delreyjones]

Originally Posted by rangerdavid

The proof is in the pudding - Let's see benchmarks on the same multi-core hardware, using GCD and non=GCD builds of FreeBSD. Show me the advantage in handling multithreaded software. Anandtech? Anyone?

I don't think the standard kind of benchmark applies here, i.e the benchmark where we measure a computer's speed with and without the feature. If I understand GCD correctly, the main benefit will be measured in developer-time, not CPU-time. For example, a programmer who's already proficient in GCD may be able to create software that exploits multiple cores in a matter of days, whereas it might take months for a programmer without this tool to exploit multi-cores. But both solutions would run equally fast after they're debugged, it's just that GCD makes it much cheaper to debug.

[Ireland]

Originally Posted by digitalclips

From the BBC blog you linked in. Thanks.

" At 8:34pm on 15 Oct 2009, DaveGreen55 wrote:
I had the problems with my iphone, but on advice updated with the latest software update and it has resolved all issues - i.e. lack of wifi, coma mode - I am now very happy iphone owner again!"

You really cherry-picked that one eh? I watched the show and the general consensus seemed to be 3.1.2 fixes issues, but diffidently not all of them. It specifically didn't fix the WiFi issue for most the vast majority of people with it. Besides, I don't need to read comments to know I own two iPhones with 3.1.2 and WiFi issues on both of them.

[Foo2]

Is GCD integration in FreeBSD 8.1 mentioned on the freebsd.org website? I only see the upcoming 8.0 RC1 mentioned.
A link would be helpful.

[MacTel]

Originally Posted by wizard69

I'm not sure it was your intention or not but you make it sound like open source is foriegn to Apple. Frankly I think many in the open source world are glad Apple is around. It is not "again" but just another facet we are seeing here of Apples continued development in this regards.

I not sure how you got that idea since "once again" refers to that they have done it before or are doing open source projects.

Apple picks what technologies they will use whether open source or home grown. Webkit, MySQL, and Apache are some existing projects that they've joined and contributed to with varying degrees of contributions. Bonjour, CUPS (Apple hired the developer), and now GCD are technologies Apple created and have sent to the open source community as new projects. I am just glad to see they have new projects going out.

GCD is a technology that is at the kernel level, so it is critical to the operation of the OS that it works extremely well. My hope is that the open source community (BSD and Linux) grab hold of GCD and incorporate it. While they are doing that they may find and fix bugs and/or improve the code and Apple will benefit from that. Heck it might even benefit Microsoft if they review the code.

My company has many developers around the country and world that contribute code to open source projects. We are the benefactors of opensource since our Oracle instances run on the Oracle Linux distro. I'm a DBA so I'm glad to see the community working for everyone.

[MacGui]

Originally Posted by Glockpop

Speaking of which, why are the AI comments all catty and obnoxious?

because this is AI.

check the all the various threads, or maybe you already have. catty, obnoxious, bitchy... that's pretty much the way it is. the s/n is not as high as other forums.

moderation of moderation makes for lots of squabbles and infighting. but if you can wade through the chafe there is some knowledge to be gained. mostly i just come here for the articles not the comments.

[bigpics]

Originally Posted by robmel

It'll take maybe 3-5 years before this is properly leveraged into mainstream apps. That's when your 16-core MacBook will really sing.

Can't wait!

Originally Posted by MacGui

because this is AI.

check the all the various threads, or maybe you already have. catty, obnoxious, bitchy... that's pretty much the way it is. the s/n is not as high as other forums.

moderation of moderation makes for lots of squabbles and infighting. but if you can wade through the chafe there is some knowledge to be gained. mostly i just come here for the articles not the comments.

At least this thread is filled with more reflection and less aggression. I can barely program "Hello, world" but I love following (as best I can) what's really happening under the hood.

[vinea]

Originally Posted by MacTel

GCD is a technology that is at the kernel level, so it is critical to the operation of the OS that it works extremely well. My hope is that the open source community (BSD and Linux) grab hold of GCD and incorporate it. While they are doing that they may find and fix bugs and/or improve the code and Apple will benefit from that. Heck it might even benefit Microsoft if they review the code.

MS has had blocks for a while. The funny thing is that MS could use GCD while Linux could not.

[blecch]

I just have to say: that is a beautiful and informative diagram!

You might also consider Blue Box in Rhapsody - and Classic in OS X - to be at least spiritual descendants of Apple's (forgotten, but rather interesting) Macintosh Application Environment (MAE), which was virtual Mac OS for AIX/Solaris/HP-UX, and which was in turn a descendant of A/UX's virtual Mac environment:

A/UX startmac -> MAE -> Rhapsody Blue Box -> Mac OS X Classic

[rangerdavid]

Originally Posted by delreyjones

I don't think the standard kind of benchmark applies here, i.e the benchmark where we measure a computer's speed with and without the feature. If I understand GCD correctly, the main benefit will be measured in developer-time, not CPU-time. For example, a programmer who's already proficient in GCD may be able to create software that exploits multiple cores in a matter of days, whereas it might take months for a programmer without this tool to exploit multi-cores. But both solutions would run equally fast after they're debugged, it's just that GCD makes it much cheaper to debug.

Ah, so wait - What we should benchmark is an app without and with GCD complied into its code benched on the two versions of BSD, to see any speed advantages that incorporating the GCD code into the app, and running it on a GCD-enabled OS, may have? That's what I would like to see.

Dave

[hezekiahb]

Originally Posted by Ireland

You really cherry-picked that one eh? I watched the show and the general consensus seemed to be 3.1.2 fixes issues, but diffidently not all of them. It specifically didn't fix the WiFi issue for most the vast majority of people with it. Besides, I don't need to read comments to know I own two iPhones with 3.1.2 and WiFi issues on both of them.

My wifi is flawless, love my iPhone. It isn't perfect, but beats the pants off my friends blackberry.

[lfmorrison]

Originally Posted by bobrik

Apple could choose whatever license they liked/wanted since it's their code, so it left me thinking whether it could be Apple's intention not to have GCD on Linux. Apple could have chosen BSD license and nothing could prevent GCD being used in Linux kernel directly.

However, since BSDs and Linux don't have similar kernel internals (unlike Apple's Darwin and FreeBSD, which do), maybe this is irrelevant, since there would have to be a completely new implementation of the kernel components on Linux anyway and no code copying, which would mean the license of Apple's GCD does not matter for Linux port of kernel components.

When Apple first made the announcement to open-source GCD, I was interested in the impact its license would have on GPL'ed open-source operating systems, so I spent a little time reading up on it.

It seemed to me at the time that Apple had implemented GCD in two distinct components which worked with each other: the user-land library, and the underlying kernel required to support the library.

It is reasonable to predict that the implementation of kernel-level support for Linux would have to be substantially different from OS X's or FreeBSD due to the different kernel architectures. It seems to me that there could be little, if any, code duplication for technological reasons, rendering the license limitation somewhat moot.

But under the system Apple seemed to have deployed, application developers probably wouldn't be encouraged to work directly with the kernel's support layer anyway -- they'd be linking against the userland library, whose job would be to act as an intermediate, platform-agnostic wrapper between the kernel and the application.

As for the userland library, it is a distinct component from the kernel. The Linux kernel is released under a custom version of the GPL which makes it clear that the kernel's license does not extend itself to any userland programs or libraries which may run on top of the kernel. So duplicating code from the userland library to run on Linux probably wouldn't be too much of a problem.

However, just as is the case on every other platform for which GCD is deployed, it would be up to each of the open-source 3rd-party application developers to determine whether or not their own applications' license permits them to legally link their own applications against the userland GCD library.

[ivan.rnn01]

blocks explained in human readable text
Wikipedia
ADC

Yeah, that's great, Apple. No, no, really good job.
Two independent language elements are awesome: '__block' storage modifier and new home-brew unary operation '^'.
Why just not write

code:

---

typedef void (__block *pblock)();

---

[talksense101]

I hope this technology catches up with all of Linux land. It will help us get more bang for the buck for our hardware investments.

The __block keyword is used for data storage that is exclusive to blocks that reference it. It is not exposed to local or global scope. That is the first extension. The second extension of ^ is for declaring a block variable. Both of these extensions are for different reasons.

[ivan.rnn01]

No "__block keyword", no "different reasons", lee-news-sea-noobies.

[lfmorrison]

Originally Posted by ivan.rnn01

No "__block keyword", no "different reasons", lee-news-sea-noobies.

Your response makes no sense whatsoever to me. Care to expand? Perhaps in coherent sentences???

[ivan.rnn01]

Originally Posted by lfmorrison

Your response makes no sense whatsoever to me. Care to expand? Perhaps in coherent sentences???

Not that much. This is not a response at all.

Well... OK. Remarks, which contain only copies of descriptions of Apple's C language extensions and void qualifications like "made for different reasons", are useless bullshit.

People, who are interested in the problem, have already bothered to read articles and to look much more deeply in that. This is no more about the difference between defines and keywords. Neither is it about calling conventions, storage modifiers and namespaces.