Apple details new 'AppleBot' web crawler used by Siri and Spotlight
Apple has confirmed the existence of a long-rumored web crawling service -- first noticed last November --?and provided some details of its operations in a recently-updated support document.
According to Apple, AppleBot is a crawler that primarily serves Siri and Spotlight. Both services offer web search capabilities, though until now it was thought that Apple routed queries through an existing search engine such as Google or Bing.
AppleBot operates similarly to Google's Googlebot, and Apple advises developers that it will follow Googlebot instructions if no AppleBot-specific instructions are found. Website operators are urged to contact Apple's network operations center if they have questions or concerns about the crawler.
It's not clear how long AppleBot has been operational in production. The support document first appeared in February, but was only populated with content last month.
One web developer spotted Apple's crawler last November, after seeing numerous hits from a crawler in the 17.0.0.0/8 block. That block is assigned to Apple, one of just a handful of companies that control an entire A block.
According to Apple, AppleBot is a crawler that primarily serves Siri and Spotlight. Both services offer web search capabilities, though until now it was thought that Apple routed queries through an existing search engine such as Google or Bing.
AppleBot operates similarly to Google's Googlebot, and Apple advises developers that it will follow Googlebot instructions if no AppleBot-specific instructions are found. Website operators are urged to contact Apple's network operations center if they have questions or concerns about the crawler.
It's not clear how long AppleBot has been operational in production. The support document first appeared in February, but was only populated with content last month.
One web developer spotted Apple's crawler last November, after seeing numerous hits from a crawler in the 17.0.0.0/8 block. That block is assigned to Apple, one of just a handful of companies that control an entire A block.
Comments
I truly hope Apple do this and free us from the warped and biased searches Google now offer. (Bait ... )
Don't worry, Google is listening to you ... trying to figure out what ads to send you ... lol
Wow. Apple has really opened up in some area, much better than some other "open" company, yea, Google, I am talking about you.
What are you talking about? So because Apple is using their own Applebot to fulfill Siri queries that means they opened up?
There's absolutely nothing wrong with Google.
Does Google have any legal control over the use of "Googlebot instructions"? I.e., is Apple allowed to have its own search technology use them, or does Google have any control over this? For example, do Googlebot instructions follow a particular syntax that Google may have protected as intellectual property?
Their motto is "do no evil"...so there! They can't be evil!
They are perfect and anyone who questions anything they do is a stupid head.
Their motto is "do no evil"...so there! They can't be evil!
No corporation is perfect, neither is Apple.
Google has obviously made some missteps but no more or less than any other in their position.
'absolutely perfect'
Given the importance of search on the web, relying on others is not a good model. Even if they don't launch it for a while, its presence can give them leverage...
This is going into my 'Hall of Shame' Gallery
Why's that? Google makes an amazing search engine among other products.
I tried using Duck Duck Go but it sucks for anything local. Google just provides me with the best search results for my purposes.
Why's that? Google makes an amazing search engine among other products.
I tried using Duck Duck Go but it sucks for anything local. Google just provides me with the best search results for my purposes.
...and also all of the baggage that goes with an advertising company. Lots of baggage.
There's room for yet another search business model, and maybe Apple has/will find it.
It's not about their search algorithms...it's about tracking every breath you take...its insidious...
You have the choice to use it, and until there is something better, most will. But you should be surprised that many would prefer a viable alternative.
I've been saying the same thing for years too. I seem to recall a few years ago that ideas was shot down by many in AI for reasons such as 'Apple couldn't compete with Google' or 'They didn't have the technology' and even 'Why would Apple want to?'
My answer to all these is, given the trust Apple has out there by its users, if Apple offer search without strings it will be another massive blow to Google's ability to make money off iOS and OS X users just as Maps has been. That's reason enough for me. If Apple's search is the default most will use it IMHO.
...and also all of the baggage that goes with an advertising company. Lots of baggage.
There's room for yet another search business model, and maybe Apple has/will find it.
What baggage may that be? They display non intrusive ads. If they really bother you, you can use an ad blocker and the problem is solved.
Microsoft has pumped billions of $ into search and even they can't find their footing in search, I highly doubt Apple will be able to either other than with a very niche market. Google makes an incredible product and that's why people use it.
It's not about their search algorithms...it's about tracking every breath you take...its insidious...
You have the choice to use it, and until there is something better, most will. But you should be surprised that many would prefer a viable alternative.
There are alternatives and people aren't using them. Google for now just has the best working solution.
I've been saying the same thing for years too. I seem to recall a few years ago that ideas was shot down by many in AI for reasons such as 'Apple couldn't compete with Google' or 'They didn't have the technology' and even 'Why would Apple want to?'
My answer to all these is, given the trust Apple has out there by its users, if Apple offer search without strings it will be another massive blow to Google's ability to make money off iOS and OS X users just as Maps has been. That's reason enough for me. If Apple's search is the default most will use it IMHO.
Most iOS users maybe, but that's really just a small niche compared to the full market out there. Also who's to say that most people wouldn't just switch back to Google? People care about the quality of the search results rather than who is providing it. If Apple does release a search it had better be better than the launch of Apple Maps which was one of the most botched launches ever. Apple isn't a services company but rather a hardware and software company.
LOL nothing personal ... But why? "There's absolutely nothing wrong with Google. " is a pretty bold statement that's why. I collect these from AI. Years later they are fun to trot out. You know, the sort of thing I mean, like this iPhone related quote ... "It’s kind of one more entrant into an already very busy space with lots of choice for consumers ... But in terms of a sort of a sea-change for BlackBerry, I would think that’s overstating it."
Well,Google won't go away. The fact is I'm often searching for very specific things of economic interests. That is a good part of search is to enable buying something. So in that regard Google serves my needs very well. At least for work related searches.
Of course not all searching is related to commerce. This is where Apple could leverage its own search engine to keep Siri AI like. Could you imagine Siri answering searches with spoken ads. Sickening isn't it.