Scaling my file server with my audience

[Colin Dermott]

On 1/19/07, Tino Schwarze <linux-poweredge.lists at tisc.de> wrote:
> Hi Colin,

Hi Tino.  Thanks for your reply.

> On Thu, Jan 18, 2007 at 10:28:42AM +1000, Colin Dermott wrote:
> > I have been running a HTTP download site for free software on a single
> > PowerEdge server for the past 13 months and it has now grown to the
> > stage where I need to scale up, but I can't actually scale this
> > machine any further (vertically).
>
> First try to identify your bottlenecks. Do you saturate the network?
> Where gets most of the time spent for the requests? Can your RAID
> deliver the data fast enough?

The bottleneck at this stage is the RAID, however I am not able to
add any more RAM to this machine either.

> There are some lightweight, high-performance web servers out there which
> you might want to try (look for Gatling). It's possible that your
> current setup is not maxed out yet.

That is interesting.  I will definitely check Gatling out.  At the moment
I use Apache 2.2.4 with the worker MPM.

I am always interested in performance tuning, however I am still
curious as to how I would best proceed once this machine is truly
maxed.

Does a cluster of Squid/Apache reverse-proxies sound reasonable?

This is the only way I can think of being able to continually add more
disk, RAM and CPU as the size of my audience and the size of my
"hot set" of files increases.

> Bye,
>
> Tino.