My Webserver Setup and Specifications, Load and Performance (after a Slashdot optimisation session.)

I had a great day today which is why I’m drinking more than one bottle of beer tonight. The UMPCPortal server got hit with a Slashdot article. And then an Engadget article. And then a Gizmodo article.  I woke up at 0800 to a DM from JKKMobile saying ‘Server Down’ so I rolled over, put the Archos Internet Tablet in its dock, picked up the U820 UMPC and got to work (while I sipped a nice cuppa that my wife had brought me.)

The great thing about getting hit with a Slashdot article is that it stresses your server with REAL traffic. Clearly my server wasn’t doing too well. With a Dual-Core 3Ghz CPU it was hitting CPU limits and with 100 concurrent Apache instances, out of memory and into swap space. Swap space is not something you want to be hitting on a web server.

After measuring network load, watching mod-status and top for a while I decided that I needed to reduce the number of Apache sessions to 80 which did a good job at keeping the swap partition unused. I then played around with some WordPress cache software. None of it really worked well.

After some research and a handy tip from Hanno, I checked out some PHP cache software. This is something I’ve been thinking about for at least 12 months as my product database is written with ‘Chippy code.’ That means it’s super inefficient and generates the page dynamically from the database for every view. For pages that don’t change from month to month, that’s a poor architecture!

I settled on a very simple, a surprisingly simple, Xcache install via apt-get on my server. The install was quick and smooth and almost immediately I was seeing a drop in CPU load.

For the rest of the day I watched the traffic flow and everything has been working really well. At least I think it is.

I though it might be a good idea to share the stats I’ve seen today. It could help someone to scale or measure their own server. Feedback on this will also help me too so here goes.

Server: Dedicated, E8300 (dual core, 3.0Ghz) processor, 2GB RAM.

Server Software: Mysql, PHP5, Apache 2.

Applications: WordPress (about 15 instances including WordPress Mu and Buddypress), OPenX, Gallery2 and a hand-written PHP product website. Small amount of FTP and Email.

Normal load: about 20-25K pages per day. Most including some PHP/Mysql.

Current stats (Today, since Slashdot impact)

Server uptime: 9 hours 11 minutes 42 seconds
Total accesses: 1255349 – Total Traffic: 16.1 GB
37.9 requests/sec – 510.1 kB/second – 13.5 kB/request

CPU load has been between 40 and 60% average.

Apache2.conf prefork MPM config:

StartServers         2
MinSpareServers      2
MaxSpareServers      5
MaxClients          120
ServerLimit     120
MaxRequestsPerChild  4000

Each Apache process instance grows up to 30MB. Average about 15-20MB over the 120 processes.

(I’ve tuned the above to fit within the 2GB of Memory that I have. Swap kills! CPU levels are OK)

The plan is:

Upgrade memory to 4GB and increase MaxClients to 200. I’m hoping that the server can handle up to 100 requests/second of PHP from my code, from WordPress, from OPenX (a significant part of my setup) and from various other LAMP components. I will also take a CPU upgrade though. Serving sessions and closing them off quickly is going to save memory. With an upgrade to a quad-core I’m hoping I can reach 150-200 requests per second on the single server.

I’m quite happy with the performance and the cost of the server which is about $160 per month from the very responsive and helpful Future Hosting service I’ve been using for about 2 years now but the question is, which part of my website is taking the most CPU and Memory. Is there 20% that is responsible for 80% of the load? If I knew how to find it, I’d probably be able optimise and double the capacity.

As I said, I’d be happy to have your feedback. How much traffic do you serve and what environment do you have?