AppleInsider AppleInsider Forums


Go Back   AppleInsider > Future Hardware
Register Members List New Posts Mark Forums Read

Reply
 
Thread Tools Display Modes
Old 01-19-2007, 01:26 PM   #1
AppleInsider
Kasper's Automated Slave
 
Join Date: Nov 1997
Posts: 6,169
SATA-based Xserve RAID prototype escapes from Apple (photos)

It's not often that a prototype of an unreleased Apple hardware product escapes the company's kung fu grip and lands smack-dab in the lap of one of our loyal readers. But hey, that's what appears to have happened. So here's some fresh meat for friday:

What you see below looks like a run-of-the-mill Xserve RAID, though first appearances can sometimes be deceiving. The only noticeable change to the face of the unit is the presence of 6 fibre channel activity lights instead of two. However, a quick peep inside confirms the mounting suspicion. Apple has yet to commission the release of this puppy, which according to some internal stamping is dubbed "Q57."

A label on the unit refers to the hardware configuration as: '512MB/1000GB 4DRIVE/2GB FC'.*

Prototype SATA Xserve RAID

Changes to the internal hardware include:

Interior of new SATA ADM

Backplane of new SATA ADM

The pre-production unit contains SATA drive modules, which appear to be the same as those used with the currently shipping Xserve server component (not the RAID, which still maintains use of Ultra ATA drives). Of the four modules, a number are marked 'Accusys Q57', indicating that they may be marginally different than the SATA drive modules utilized by the current Xserve.*Increased maximum bandwidth and lowered power consumption of future SATA drives should both increase the efficiency of the Xserve RAID and allow for cooler operation.*

RAID Controller Module

Also changed is the RAID controller module, which no longer houses the fibre channel port.* Only the ethernet and UPS interface ports remain.

Six Fibre Channel Ports

The fibre channel ports are now incorporated in the cooling module.* Matching the increase in front activity lights, Q57 sports six fibre channel ports.

Xserve RAID Prototype - Backplane

When using the latest version of RAID Admin, the device firmware is reported as:* 2.0d32 dev/A3.10. Unfortunately, a large portion of the RAID Admin functionality is disabled. Currently shipping Xserve RAID units are operating on firmware version 1.5 and it appears that, while the administration software recognizes the prototype, it cannot be configured.*For this reason, the abilities of the 4 extra fibre channel ports cannot be determined.* *

On the currently shipping Xserve RAID, when one Xserve is connected to both ports it has access to all 14 drives. When two Xserves are connected (one to each port), both have access to 7 of the drives.* It's likely that newer RAID Admin software will allow arbitrary assignment of drives to any of six connected Xserves.* It is also possible that these newer Xserves RAIDs have an integrated 6-port fibre channel switch.* This won't be known until an actual release of the hardware when new firmware and admin software is available.

While it's not clear if and when Apple plans to update its Xserve RAID systems to coincide with those advancements present in Q57, the prototype presents a logical progression to their cutting-edge storage solution, allowing the company to standardize the Xserve RAID drive modules with those currently used in the Xserve.

Of course, Apple may also forgo the release of an SATA RAID completely in favor of an SAS-based solution.

Update: A reader points out the prototype consists of only half a RAID.

"There is only one controller in it, as current Xserves are actually two 7 drive RAIDS in an enclosure (when you want to access all 14 drives, you have to connect both controllers to the host, and then strip across the two volumes in software).

So I would assume that in a full Xserve, not a test / demo/ prototype, that it would have 2 controllers, 2 sets of 6 fibre channel ports, etc. as there is only 1 controller and 1 6 port fibre channel bay shown in the photos you have, but the spaces below them are meant to take those additional controllers / fan assemblies.

Also apple could be duplicating the fibre ports, as those are one of the major weaknesses of the xserves, the fibre ports are a single point of failure."
AppleInsider is offline   Reply With Quote
Old 01-19-2007, 02:55 PM   #2
emig647
Registered User
 
Join Date: Feb 2004
Location: Portland
Posts: 2,055
That is a really cool find. If only getting more unreleased products was this easy. I'm actually pretty impressed with Apple's internal designs. They really take the time to make it a professional product.


We all want things we can't have... So go to http://www.stuffspace.com!
emig647 is offline   Reply With Quote
Old 01-19-2007, 02:57 PM   #3
Matthew Yohe
Registered User
 
Join Date: Apr 2005
Posts: 262
"But hey, that's what appears to have happened."

...
Matthew Yohe is offline   Reply With Quote
Old 01-19-2007, 02:59 PM   #4
matracer
Registered User
 
Join Date: Feb 2005
Posts: 91
Is ZFS better than RAID? I was just wondering whether ZFS would make RAID redundant.
matracer is offline   Reply With Quote
Old 01-19-2007, 02:59 PM   #5
emig647
Registered User
 
Join Date: Feb 2004
Location: Portland
Posts: 2,055
""But hey, that's what appears to have happened.""

LOL @ that


We all want things we can't have... So go to http://www.stuffspace.com!
emig647 is offline   Reply With Quote
Old 01-19-2007, 03:01 PM   #6
emig647
Registered User
 
Join Date: Feb 2004
Location: Portland
Posts: 2,055
Quote:
Originally Posted by matracer View Post
Is ZFS better than RAID? I was just wondering whether ZFS would make RAID redundant.
Though I'm not all that familiar with ZFS, I suppose that depends on what RAID configuration you're talking about?

(does anyone know if the XServe RAID can be reconfigured to any type? ie. 0, 1, 2, 3, 5, etc.)


We all want things we can't have... So go to http://www.stuffspace.com!
emig647 is offline   Reply With Quote
Old 01-19-2007, 03:31 PM   #7
bbatsell
Registered User
 
Join Date: Jan 2007
Posts: 9
I personally would love to see a new XServe RAID handle SAS drives (I haven't seen a SATA drive hit 15k yet, but I could be wrong). The extra fibre ports and the ability to access any array across either backplane through either controller would be very nice, but I know isn't likely.

Quote:
Is ZFS better than RAID? I was just wondering whether ZFS would make RAID redundant.
ZFS can use RAID-Z across multiple drives (called a pool in ZFS terminology). RAID-Z is an adaptation of RAID 5. So, basically, if RAID 5 makes sense for your application, then ZFS should work fine. If you do a lot of writes where computing and writing the parity data that RAID 5 requires would result in a slowdown, then using a ZFS pool wouldn't be the best option. (I personally would use RAID 10 here, but everyone has a different level they like for different applications.) You could still use the ZFS filesystem on top of the hardware-based RAID ? stripe, of course.

Quote:
Though I'm not all that familiar with ZFS, I suppose that depends on what RAID configuration you're talking about?

(does anyone know if the XServe RAID can be reconfigured to any type? ie. 0, 1, 2, 3, 5, etc.)
The current XServe RAID can handle 0, 1, 3, 5, 0+1, and what they call "Enhanced JBOD". I have no idea what is enhanced about it, as I have never tried it. (For those unacquainted, JBOD stands for "Just a Bunch Of Disks" and is termed "Concatenation" in Disk Utility's software RAID. It's just a series of disks treated as one volume with data written sequentially across the drives.)
bbatsell is offline   Reply With Quote
Old 01-19-2007, 03:39 PM   #8
Johnny Mozzarella
Registered User
 
Join Date: Mar 2006
Posts: 956
Integrating the fibre channel switch is a really nice touch.
It's one less seperate component to purchase and to take up space.
For Apple's target market this is going to make the XServe RAID an even better value than it already is.
Johnny Mozzarella is offline   Reply With Quote
Old 01-19-2007, 03:45 PM   #9
monkeyastronaut
Registered User
 
Join Date: Apr 2003
Location: united mexican states
Posts: 1,326
it's FAKE!!!11!!!2 you can't fool me, photoshop scammer!
monkeyastronaut is offline   Reply With Quote
Old 01-19-2007, 03:46 PM   #10
Johnny Mozzarella
Registered User
 
Join Date: Mar 2006
Posts: 956
Quote:
Originally Posted by matracer View Post
Is ZFS better than RAID? I was just wondering whether ZFS would make RAID redundant.
ZFS is file system software.
RAID is hardware or software.

The Zen File System incorporates principles found in RAID to increase reliabilty and performance.
While there is overlap they are really complimentary technologies.
Especially when ZFS is used on RAID hardware.

Here is an analogy...
GPS(Global Positioning System) is a set of algorythems for determining your position.
It is a great system, but not if you don't have any satellites.
Once you have satellites(hardware) to complement the system(software) then it is more useful.

***I know I am potentially opening a huge can of worms by just mentioning GPS***


Last edited by Johnny Mozzarella; 01-19-2007 at 03:53 PM..
Johnny Mozzarella is offline   Reply With Quote
Old 01-19-2007, 03:58 PM   #11
addabox
Registered User
 
Join Date: May 2002
Location: 0aktown
Posts: 9,255
Quote:
Originally Posted by Johnny Mozzarella View Post
ZFS is file system software.
RAID is hardware or software.

The Zen File System incorporates principles found in RAID to increase reliabilty and performance.
While there is overlap they are really complimentary technologies.
Especially when ZFS is used on RAID hardware.

Here is an analogy...
GPS(Global Positioning System) is a set of algorythems for determining your position.
It is a great system, but not if you don't have any satellites.
Once you have satellites(hardware) to complement the system(software) then it is more useful.

***I know I am potentially opening a huge can of worms by just mentioning GPS***
THE IPHONE SUCKS WITHOUT GPS!!!!!!!!!! APPLE IS TEH DOOMED!!!!!!! GAAAAAAAA!!!!!!!!


party's over
addabox is offline   Reply With Quote
Old 01-19-2007, 04:09 PM   #12
King Chung Huang
Registered User
 
Join Date: Nov 2001
Location: Calgary, Alberta, Canada
Posts: 342
Sweet! I'm in the market for a new Xserve RAID. I wonder when the update will be released?
King Chung Huang is offline   Reply With Quote
Old 01-19-2007, 04:27 PM   #13
Johnny Mozzarella
Registered User
 
Join Date: Mar 2006
Posts: 956
Quote:
Originally Posted by King Chung Huang View Post
Sweet! I'm in the market for a new Xserve RAID. I wonder when the update will be released?
If past experience is any indicator, as soon as I buy the current model.
Johnny Mozzarella is offline   Reply With Quote
Old 01-19-2007, 05:46 PM   #14
hmurchison
Global Moderator
 
Join Date: Nov 2001
Location: Seattle, WA
Posts: 10,465
I'm with bbatsell

The Xserve RAID need to migrate to SAS.

I'd like to see two different models hit.

Deliver the same casing SAS XR with 14 drive bays. Then ship a 3U 22 drive bay XR with 2.5" SFF drives. This allows companies to tier their storage right in the RAID box. Use 15k SAS drives for your production data and SATA drives for your nearline storage. Works well with SAN file systems as well because you don't need basic stuff sitting on your expensive drives.

Fibre is losing traction. Even Brocade is blinking and buying IP companies.

Brocade aquires Silverback IP Data Vendor

ZFS is nice as well. The checksum features go beyond what you get with a basic RAID 5 setup which cannot prevent corruption.


Mac mini - 2 , iPod Nano- 1
G4 Cube - 5 , iPod Shuffle -1
Bloggity Blog
hmurchison is online now   Reply With Quote
Old 01-19-2007, 09:15 PM   #15
jbh0001
Registered User
 
Join Date: Jan 2006
Posts: 62
Quote:
Originally Posted by Johnny Mozzarella View Post
The Zen File System incorporates principles found in RAID to increase reliabilty and performance.
Umm...
ZFS stands (or stood) for Zettabyte File System.
Zen is a philosophical approach to life.

...unless, of course, you're wanting to talk about a particular philosophical approach to file systems. In that case, wait until I leave the room before you start that discussion.
jbh0001 is offline   Reply With Quote
Old 01-19-2007, 09:36 PM   #16
JeffDM
Global Moderator
 
Join Date: Jun 2004
Location: .US
Posts: 9,127
It looks nice, but not definitely not something I can afford. Personally, I'd prefer eSATA II with a port multiplier in an external box, though even those are still a bit expensive. That would be somewhere between the previous FC standard and the new one and well ahead of any gigabit link, in speed of course. FC and ethernet have a distance advantage to consider.


Last edited by JeffDM; 01-19-2007 at 09:50 PM..
JeffDM is online now   Reply With Quote
Old 01-19-2007, 10:14 PM   #17
Chucker
Registered User
 
Join Date: Dec 2001
Posts: 5,066
Quote:
Originally Posted by jbh0001 View Post
Umm...
ZFS stands (or stood) for Zettabyte File System.
Zen is a philosophical approach to life.
Strictly speaking, ZFS once stood for Zettabyte File System but now stands for nothing at all.
Chucker is offline   Reply With Quote
Old 01-19-2007, 10:53 PM   #18
sjk
Registered User
 
Join Date: Apr 2002
Location: Hawaii
Posts: 540
Quote:
Originally Posted by Chucker View Post
Strictly speaking, ZFS once stood for Zettabyte File System but now stands for nothing at all.
As in zilch?
sjk is offline   Reply With Quote
Old 01-19-2007, 10:57 PM   #19
sjk
Registered User
 
Join Date: Apr 2002
Location: Hawaii
Posts: 540
Quote:
Originally Posted by jbh0001 View Post
Zen is a philosophical approach to life.

...unless, of course, you're wanting to talk about a particular philosophical approach to file systems. In that case, wait until I leave the room before you start that discussion.
Hmm, that might be a discussion I'd enjoy having with people reasonable well-informed on that subject.
sjk is offline   Reply With Quote
Old 01-20-2007, 12:11 AM   #20
Shookster
Registered User
 
Join Date: Aug 2006
Posts: 107
So if there are three (or six) FC ports per controller, that would mean bandwidth of 6Gbps (or 12Gbps), right, providing you had enough ports for the machines as well?
Shookster is offline   Reply With Quote
Old 01-20-2007, 11:04 AM   #21
jbh0001
Registered User
 
Join Date: Jan 2006
Posts: 62
Quote:
Originally Posted by Chucker View Post
Quote:
Originally Posted by jbh0001 View Post
Umm...
ZFS stands (or stood) for Zettabyte File System.
Zen is a philosophical approach to life.
Strictly speaking, ZFS once stood for Zettabyte File System but now stands for nothing at all.
Hence the parenthetical "or stood" in my original post.
jbh0001 is offline   Reply With Quote
Old 01-20-2007, 01:50 PM   #22
wmf
Registered User
 
Join Date: Feb 2002
Location: Austin, TX
Posts: 1,158
Let's count the ways this sucks:

FC is still too expensive.
ADMs are still too expensive.
It appears to still use the "split-brain" controller design instead of the redundant active-active controller design that everyone else uses.
wmf is offline   Reply With Quote
Old 01-23-2007, 12:38 AM   #23
Haggar
Registered User
 
Join Date: Jul 2006
Posts: 530
Apple has offered the choice of formatting a drive using UFS for a while now, but not all applications are compatible with that file system. Will applications have to be updated to run on ZFS?

Does ZFS support resource forks?

Is ZFS case sensitive, and will Apple allow users to choose whether they want case sensitivity?

Can a ZFS formatted hard drive be used as a Mac OS startup disk?
Haggar is offline   Reply With Quote
Old 01-23-2007, 01:19 AM   #24
Chucker
Registered User
 
Join Date: Dec 2001
Posts: 5,066
Quote:
Originally Posted by Haggar View Post
Apple has offered the choice of formatting a drive using UFS for a while now, but not all applications are compatible with that file system. Will applications have to be updated to run on ZFS?

Does ZFS support resource forks?
It supports multiple streams, so this would be implementable.

Quote:
Is ZFS case sensitive, and will Apple allow users to choose whether they want case sensitivity?
They can always provide an option.

Quote:
Can a ZFS formatted hard drive be used as a Mac OS startup disk?
ZFS is non-bootable at this point, though bootstrap solutions exist.
Chucker is offline   Reply With Quote
Old 01-23-2007, 04:11 AM   #25
sjk
Registered User
 
Join Date: Apr 2002
Location: Hawaii
Posts: 540
Re: ZFS supporting resource forks
Quote:
Originally Posted by Chucker View Post
It supports multiple streams, so this would be implementable.
Aside from boot-ability (for now), do you know of any advantages that HFS+ still has over ZFS? Maybe HFS+ still has better performance for certain situations?
sjk is offline   Reply With Quote
Old 01-24-2007, 01:15 PM   #26
mwswami
Registered User
 
Join Date: Mar 2006
Location: Silicon Valley
Posts: 166
10.5TB XServer RAID

Looks like Apple has quietly upgraded the XServer RAID with 750GB drives though they are still Ultra ATA. The 14x750GB configuration (10.5TB) costs an additional $1400 over the 7TB version.
mwswami is offline   Reply With Quote
Old 01-24-2007, 04:30 PM   #27
dfiler
Registered User
 
Join Date: Aug 2002
Location: Pittsburgh, PA
Posts: 2,666
Quote:
Originally Posted by hmurchison View Post
I'm with bbatsell

The Xserve RAID need to migrate to SAS.

...

ZFS is nice as well. The checksum features go beyond what you get with a basic RAID 5 setup which cannot prevent corruption.
I'd agree about adding a SAS option. It appears to be _the_ future for storage servers.

About ZFS... From my reading, RAID-Z simply looks like one vendor's implementation of RAID-5. Good RAID implementations have all kinds of error checking done right on-board. Some vendors like to appear special by explaining how their implementation is better than "standard" RAID. When in actuallity, no two vendors have the same definition of how each RAID level is accomplished.

We recently purchased a SnapServer, expandable to 44TB. If apple had supported SAS and combining of multiple chassis into a single storage server, we would have gone the apple route.
dfiler is offline   Reply With Quote
Old 01-24-2007, 04:44 PM   #28
Chucker
Registered User
 
Join Date: Dec 2001
Posts: 5,066
Quote:
Originally Posted by dfiler View Post
About ZFS... From my reading, RAID-Z simply looks like one vendor's implementation of RAID-5.
RAID-Z has copy-on-write; RAID-5 does not.
Chucker is offline   Reply With Quote
Old 01-24-2007, 05:23 PM   #29
JeffDM
Global Moderator
 
Join Date: Jun 2004
Location: .US
Posts: 9,127
Quote:
Originally Posted by hmurchison View Post
I'm with bbatsell

The Xserve RAID need to migrate to SAS.

I'd like to see two different models hit.
I don't think it's a need to totally migrate to SAS, but to offer it in addition to SATA. The new Xserve shows that you can support SATA and SAS in the same bay, just slap in a new drive module with the other drive type. Making one that is SAS-only would make it exorbitantly expensive and less flexible than a dual-standard design.

Quote:
Deliver the same casing SAS XR with 14 drive bays. Then ship a 3U 22 drive bay XR with 2.5" SFF drives. This allows companies to tier their storage right in the RAID box. Use 15k SAS drives for your production data and SATA drives for your nearline storage. Works well with SAN file systems as well because you don't need basic stuff sitting on your expensive drives.
The 14 bay design would hold more data than a 22 bay SFF drive design. The SFF SAS drives are half the capacity as the larger sized SAS drives of the same general speed. I don't know why this is, because the platters for the 10k and 15k drives are very small in diameter anyway.
JeffDM is online now   Reply With Quote
Old 01-24-2007, 05:39 PM   #30
dfiler
Registered User
 
Join Date: Aug 2002
Location: Pittsburgh, PA
Posts: 2,666
Quote:
Originally Posted by Chucker View Post
RAID-Z has copy-on-write; RAID-5 does not.
I think that is where the rub is. In my opinion, "RAID-5" doesn't preclude copy-on-write.

Various RAID vendors tweak the text-book definition of a standard RAID level and then claim that it is a brand new type of RAID. When in actuality, all they've done is implement a nice or slightly different version of a basic RAID type.

Each RAID level is, in my opinion, more like a category than an exact definition.

For instance, I'm running a SyncRAID card in my old G4 as a multi-TB media server. They initially tried to call it RAID-XL but eventually gave up and just admitted it was RAID-3.

Vendors like to give their RAID implementations a new name so that they have something to advertise. "RAID-5 with hardware ensured transaction integrity" just isn't as marketable as "RAID-Z".


Last edited by dfiler; 01-24-2007 at 05:45 PM..
dfiler is offline   Reply With Quote
Old 01-24-2007, 11:14 PM   #31
wmf
Registered User
 
Join Date: Feb 2002
Location: Austin, TX
Posts: 1,158
I think they should switch from FC to SAS for the host ports, but continue using SATA drives.
wmf is offline   Reply With Quote
Old 01-27-2007, 12:14 PM   #32
macgyvr64
Registered User
 
Join Date: Dec 2004
Posts: 1
On a related note -- how is JBOD different from RAID 0? I thought RAID 0 was 'concatenation', and now we have JBOD, too?
macgyvr64 is offline   Reply With Quote
Old 01-27-2007, 12:26 PM   #33
JeffDM
Global Moderator
 
Join Date: Jun 2004
Location: .US
Posts: 9,127
Quote:
Originally Posted by wmf View Post
I think they should switch from FC to SAS for the host ports, but continue using SATA drives.
That's an interesting idea.

Quote:
Originally Posted by macgyvr64 View Post
On a related note -- how is JBOD different from RAID 0? I thought RAID 0 was 'concatenation', and now we have JBOD, too?
No, they are different. Striping basically takes a block of data and splits that block of data between the drives equally. This increases sequential read and write speed. Striping requires that the drives (or the partitions on all drives) be the same size to stripe across.

JBOD doesn't increase sequential read and write peeds for saving or retrieving large files. Maybe JBOD helps random reads and writes, I've never thought of that. JBOD allows different sizes of drives to be used together.
JeffDM is online now   Reply With Quote
Old 01-27-2007, 01:39 PM   #34
wmf
Registered User
 
Join Date: Feb 2002
Location: Austin, TX
Posts: 1,158
Quote:
Originally Posted by macgyvr64 View Post
On a related note -- how is JBOD different from RAID 0? I thought RAID 0 was 'concatenation', and now we have JBOD, too?
JBOD is multiple disks with no RAID, so each physical disk appears as a separate volume. For example, if you attach a bunch of disks to your Mac and format each one individually with no RAID, you get a JBOD. With RAID 0, it looks like one big volume.
wmf is offline   Reply With Quote
Old 01-27-2007, 02:27 PM   #35
Chucker
Registered User
 
Join Date: Dec 2001
Posts: 5,066
Quote:
Originally Posted by wmf View Post
JBOD is multiple disks with no RAID, so each physical disk appears as a separate volume. For example, if you attach a bunch of disks to your Mac and format each one individually with no RAID, you get a JBOD. With RAID 0, it looks like one big volume.
No. JBOD combines multiple disks, regardless of size, into one and the same volume.
Chucker is offline   Reply With Quote
Old 01-27-2007, 06:11 PM   #36
hmurchison
Global Moderator
 
Join Date: Nov 2001
Location: Seattle, WA
Posts: 10,465
Quote:
Originally Posted by JeffDM View Post
I don't think it's a need to totally migrate to SAS, but to offer it in addition to SATA. The new Xserve shows that you can support SATA and SAS in the same bay, just slap in a new drive module with the other drive type. Making one that is SAS-only would make it exorbitantly expensive and less flexible than a dual-standard design.



The 14 bay design would hold more data than a 22 bay SFF drive design. The SFF SAS drives are half the capacity as the larger sized SAS drives of the same general speed. I don't know why this is, because the platters for the 10k and 15k drives are very small in diameter anyway.
SAS share the same backplane with SATA connections so you actually have to willfully engineer SATA support out of a SAS backplane. You want a SAS backplane that is full featured with staggered startup and eventually dual ported design for high availability connections but also maintain SATA support for nearline storage. I'm not sure how many SAS only products exist but I think you'll find that most SAS products now are SAS/SATA backplanes.

I think Apple needs to focus on two distinct needs. The 14 Bay 3.5" drive Xserve RAID accomplishes storage need for many people. However the reason why you will see 2.5" drives in 2U and 3u chassis is to cover the market that is less concerned about total storage and most concerned about IOPS per rack. More spindles = more IOPS and often this is more important for some vertical markets.

There is a 300GB 2.5" drive coming

http://www.wwpi.com/index.php?option...1692&Itemid=39

though it's only 4200k. Seagate makes the highest density SFF drive at 146GB and 10k speeds. I believe they just announced 73GB 15k SFF drives. Perpendicular recording will boost these numbers soon.


Mac mini - 2 , iPod Nano- 1
G4 Cube - 5 , iPod Shuffle -1
Bloggity Blog
hmurchison is online now   Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -5. The time now is 11:51 AM.


Powered by vBulletin® Version 3.8.4
Copyright ©2000 - 2009, Jelsoft Enterprises Ltd.