Main Stories
Slash Boxes

Slash Open Source Project

Slashcode Log In

Log In

[ Create a new account ]

Building a large Slash cluster

posted by Krow on 11:53 AM October 17th, 2002   Printer-friendly   Email story
We're getting close to going live with a Slash cluster that will need to be able to handle at least 1 million (pinky finger to the mouth :) hits per month with room for substantial growth. In the extended copy (below), I propose a cluster system as a strawman for you to pick holes in and suggest better ways to set it up, make better hardware decisions, or just add your own lessons learned, etc. Pick away!

The entire cluster will be x86 boxes running FreeBSD. We're planning on keeping our own copy of Slashcode CVS up-to-date and running the same version that powers Slashdot.org, but with a fair amount of custom tweaks and plugins.

In addition to raw serving power, this cluster needs to be available. If any single machine dies we want to switch its tasks over to another machine for a short period of time while the dead box is fixed/replaced and brought back online. So here's the idea:

5 server-class machines:

Each box is a 1-processor 1.2 GHz P4 and single IDE system disk. Question: How much memory would you put in these?

  • 1 firewall stateful packet filter, doubling as a load-balancer for the 2 webservers listed below, and with the ability to run as a backup SMTP machine
  • 1 DNS/SMTP machine with the ability to run as a backup firewall
  • 2 webserver boxes each running Apache
  • 1 webserver running Apache dedicated to serving images and also serving some corporate backend stuff.
2 dataserver-class machines:

Both boxes are dual-processor 2.x GHz P4 with multiple SCSI disks. Questions: How much memory would you put in these? How would you arrange the drives?

  • 1 primary dataserver machine dedicated to running mysqld, but with the ability to run a backup slashd, nfsd, and loghost
  • 1 backup dataserver running a slave mysqld to the primary dataserver, slashd, nfsd (to serve slashd-generated SHTML to the 2 webservers, and acting as a syslog loghost
All machines will be interconnected though a 100Mbit switch (9Gbps total capacity switch fabric) What do you think of this cluster architecture? Is it overkill, underkill? What would be your dream architecture to serve at least 1 million "pages" a month from the word go and have the ability to scale upwards?
This discussion has been archived. No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
 Full
 Abbreviated
 Hidden
More | Login
Loading... please wait.
  • Look at Slashdot's config, first of all. They handle a shitload of hits every hour.

    Now, if I were you, I would start with LVS [linuxvirtualserver.org]. Have a primary, and a secondary.

    Behind them a raid5 distribution machine (fileserver, runs slashd, has https admin interface). Then a big sql box, and a smaller one for replication. The rest of the machines, webservers.

    --
    lottadot [lottadot.com]
    • What type of simulations exist? Surely someone has a slashdot site and a set of queries/comment adds that would simulate a "slashdot.org" type site. There are a number of raw simulation software to test web sites -- modify them to submit comments and stories.

    • Yes Slashdot has a good description of their configuration in the tech notes [slashdot.org]. Their doc is a bit old (6/13/00), but I imagine the architecture is prolly the same.

      Anyway they write that they have RAIDed the sql box (Type III in their notation), but don't say that they've RAIDed the dataserver/NFS box (Type II). This seems opposite to what you described? (Course it would be nice to just RAID both machines and be done with it :)