Scaling my file server with my audience

Jerker Nyberg jerker at Update.UU.SE
Fri Jan 19 03:10:02 CST 2007

Hello Colin,

Check out MogileFS - an application level open source distributed file 

Since you have a lot of downloadable files, MogileFS (or something like 
it) might be useful. Buy several cheap servers with a couple of (SATA) 
disks each and keep your data there. Scale up storage with more servers 
and more disks. The limiting factor for your large downloads is probably 
the number of disks. If the code part of your site is small enough you 
could keep it in cache on a single server with plenty of RAM.

A couple of years ago I scaled up webhosting from a single server to two 
racks with two load balancers (Heartbeat/LVS), around seven frontends (web 
and mail) with storage on two file servers (NFS) and two database servers. 
However, after a while with increasing load things get quite dependant on 
each other, a hot spot on a couple of disks could disturb all the 
frontends which in turn could make the content located other the other 
file servers slow as well... In order to cope with increasing load and to 
get better stability this setup was later divided by my colleagues into 
several clusters, each with a single NFS/SQL-server and a few frontends 
for mail/web. The loadbalancers were then running Keepalived(LVS). Stable 
and scalable for that kind if load (many websites with small files on a 
read/write file system), but probably not the best way for you to go with 
your HTTP download site where you have better control over the 

Jerker Nyberg.
Uppsala Sweden.

On Fri, 19 Jan 2007, Colin Dermott wrote:

> I'm interested in this set up.  I don't have much experience with NFS,
> however.  Do the NFS clients cache to local disk or will the webservers
> hit the NFS server for each file request?  As it stands now, my disk
> array is the bottleneck.  I have also exhausted the upgrade path for
> RAM in my machine at 12GB.
> The median file size in my hot set would be around 50-100MiB,
> possibly more as the most popular files seem to be game (WoW)
> recordings (can be several hundred MiB) and Linux ISO downloads.
> The median of my entire set would be much smaller - say 3MiB.

More information about the Linux-PowerEdge mailing list