The announcement was made by Google Chief Executive Larry Page in a post to the company's official blog. In it, Page recalled when he and company co-founder Sergey Brin first met Rubin back in 2004.
"He believed that aligning standards around an open-source operating system would drive innovation across the mobile industry," Page said. "Most people thought he was nuts. But his insight immediately struck a chord because at the time it was extremely painful developing services for mobile devices."
The CEO praised Rubin for having "exceeded" the "crazy ambitious goals" Google had for Android. According to Page, Rubin simply "decided it's time to hand over the reins and start a new chapter at Google."
Taking his place will be Sundar Pichai, who has previously worked on the Chrome browser and Apps teams at Google.
The pace of innovation has never been greater, and Android is the most used mobile operating system in the world." - Google CEO Larry Page
"The pace of innovation has never been greater, and Android is the most used mobile operating system in the world: we have a global partnership of over 60 manufacturers; more than 750 million devices have been activated globally; and 25 billion apps have now been downloaded from Google Play," Rubin said.
The executive shakeup comes less than six months after Apple had its own major change for its mobile operating system, iOS. Scott Forstall, who previously served as Apple's iOS chief, was forced out at the company last October.
Before Rubin joined Google, he worked at Apple from 1989 to 1992. He then moved on to General Magic, then Danger, which made the Sidekick handset, before starting Android, Inc. He joined Google when the search giant acquired Android in 2005.
Google's Android and Apple's iOS are the two dominant platforms in the modern smartphone space.