In reply to:
OK, the KB article about the setup is rather vague. I can understand the security issues involved and stuff, but any additional stats that can be given?
The setup varies from machine to machine. We're not really trying to be secretive, though (and the kbase article is somewhat vague since it's hard to keep this sort of information up-to-date). I'll try to answer your questions as best as possible without making it super confusing (some of our weird internal stuff is hard to explain).
1) To determine memory (physical and virtual, i.e., swap) on a machine, you can type "free", or by viewing the file "/proc/meminfo". Most new user (web) machines are dual Xeons (1.8 - 2.3 Ghz, I think) with about a gig of RAM and a gig of swap. We do balance stuff out so that older, less buff machines will have less users or sites on them. We have some internal machines (for databases and whatnot) that are super buff.
We have some pretty old machines out there that are still alive and kicking, and hosting lots of sites with 512 Mb of RAM and dual 500s.
As far as total number of machines, I think a little over 250 last time I counted, not counting network devices. Some of these are internal machines and some are dedicated.
In terms of number of sites, it depends whether you're counting subdomains or not. Our DNS servers are currently (as of today) serving 56,779 zones or so, but of course many of these zones have many subdomains.
In reply to:
Also how does the .something part of the svrpath work? i.e. why is it /home/.something/user instead of just /home/user? Are each of those .somethings a different backup tape or physical drive?
This has been discussed on the forum before... it's kind of a weird internal concept. Basically, as you may know, user data is mounted via NFS on Network Appliance filers. The .foo/ is a separate mount point, which we internally refer to as a "dataglob"; the names are random and generated from wordlists of different types of words.
It's just kind of a way of keeping things in manageable chunks (in terms of moving data around, etc.). Machines are automatically configured to mount the appropriate dataglobs for any users that are on the machine. Overall, I think we currently have 8 NetApp 760 heads, and a whole bunch of shelves (not sure how many Tb of data, but I think about 3.5 unless my math is wrong). So there aren't that many different physical filers, but each one is divided into multiple volumes, and then the "dataglobs" add another layer of abstraction.
Pictures of some of these devices....
Here and here (these links were posted to the forum a while back, so some of you may have seen them already).
