Trying to build 7-45 TB storage on the cheap with only Dell hardware

dell at bobich.net dell at bobich.net
Fri Jun 6 04:14:32 CDT 2008


On Thu, 5 Jun 2008, Ray Van Dolson wrote:

> On Mon, Jun 02, 2008 at 09:14:54AM -0700, Ryan Bair wrote:
>> I love the MD1000 PERC5/6 combination. Its really great performance
>> and scalability at a good price.
>>
>> I keep all of my drives in a big 15 block then chain them together
>> with the dual EMMs. On the OS level I use LVM to tie together the
>> individual MD1000's into a single mega volume group. Keep in mind that
>> you can hook up  to 45TB of raw disk space (with 1TB drives) to a
>> single PERC card. With a 2U server and 3 PERC cards, you can really
>> get a lot of storage on a single machine.
>>
>> If you need to scale across multiple servers, you may want to look at
>> GlusterFS.
>>
>> Some comments inline.
>
> This sounds like something we are interested in setting up.  Can you
> give insight into the following potential setup?
>
> Ideally we'd like to have one front-end box and one MD1000 loaded with
> disks in two separate colo rooms.  We'd like the backend storage to be
> mirrored between the two MD1000's and if one MD1000 fails the other
> can automatically step in and continue to be accessed via *either*
> front-end box (regardless if it's directly attached to the MD1000 or
> not).
>
> I envision each front end box having a direct connection to one MD1000
> and an iSCSI connection to the other... then using LVM or software RAID
> to mirror between the two and formatting the whole thing with GFS.
>
> Better way to do this?

I don't think mirroring using LVM or software RAID will work in this 
scenario. You will need something like DRBD and both connections to be on 
the same level of abstraction.

Have the MD1000s connected to one machine each, and have that machine 
export the content via DRBD for mirroring, and have iSCSI run on top of 
that, with failover using RHCS or heartbeat. Depending on whether you have 
very few big files or lots of small files and how many writes go on, you 
may or may not be better off with running GFS on top of this as an 
active/active pair.

There are variations on the theme that I'm too lazy to go into right 
now as it could go somewhat epic, but this article should give you the 
gist of it as far as a simple configuration (active/passive with 
heartbeat failover, no clustering) goes: 
http://www.pcpro.co.uk/realworld/82284/san-on-the-cheap/page1.html

Gordan



More information about the Linux-PowerEdge mailing list