Back in December of last year Sun's Jonathon Swartz offered a free T2000 demo to anyone willing to test the machine and then blog about it in public.
I thought it might be fun, therefore, to take a look at some of the resulting blogs. To do it I used "Sun T2000 tested" as google keywords and read through the first three pages that came up looking for hits that weren't obviously commercial reviews or otherwise not what I was looking for.
The first one I found was a doozy. Boy, this guy hated the product:
The Sun Fire T2000 is just more of the same from Sun Microsystems: it's overhyped and looks great on paper, but costs a fortune to buy, is difficult to configure and use, and doesn't make any sense at all unless you're already on UltraSPARC-based servers. On the other hand, if you've already paid through the nose for Sun hardware in the past, why would you be willing to spend another astronomical sum on replacement machines, when you could much more cheaply switch to an Opteron-based solution?
If I'm reading him right, he couldn't get it working - because he couldn't get an RJ45 connection in place and so couldn't assign the machine its default network configuration:
Unfortunately, the network connection is disabled by default, so you need to initially connect through the serial port. But wait! You can't use the 9-pin serial port because, for some mysterious reason, it is not connected to the control module. So how, then, do you connect to the bloody thing? You have to use an RJ45 serial connection. This is the first time I've ever seen such a thing, so I did not have a cable or adapter for it -- and the machine did not come with any kind of adapter.
I know, that's laughable, but it's also real: four of the first ten results I got on April 16/06 using "T2000" as a search term in blogsearch.google.com contained complaints about this issue. The whole serial start-up configuration routine has always struck me as a bad compromise - Where these things are being shipped in ones and twoses, Sun should pre-configure them according to customer specifications so they come up clean when plugged in and turned on.
The next one Niagara vs ftp.heanet.ie Showdown was at the opposite extreme. Run by some people who clearly know quite a lot about Solaris and Apache and whose business focuses on the provision "of broadband internet Services to ireland's Universities, Institutes of Technology and researchers" this test produced a glowing review:
Bottom line, the T2000 was able to handle over 3 times the number of transactions per-second and about 60% more concurrent downloads than the current ftp.heanet.ie machine can (a dual Itanium with 32Gb of memory) running identical software. Its advantages were even bigger than that again, when compared to a well-specced x86 machine. Not bad!
They report their tuning effort and results in detail. For example:
How many requests the machine can handle in a second is probably the most valuable statistic when talking about webserver performance. It's a direct measure of how many user requests you can handle. Fellow ASF committer, Dan Diephouse, has been producing some interesting stats for requests-per-second for webservices (and they are impressive), however we were more interested in how many plain-old static files the machine could really ship in a hurry. And without further ado, those numbers are;
Sun's own benchmarks have quoted up to 2500 requests per second, which we didn't find particularly impressive. Our current box - merely a dual Itanium - can do 2700 requests per-second without much trouble. I'm happy to confirm though, that the tricks we do to reduce Apache's memory usage on Linux have as much of an effect on Solaris. Our results are averaged over 5 runs of the testing, during which the T2000 managed a very, very impressive 5718 requests per second. Not bad!
Next on the hit list is a blog series by a guy named John Sinteur. His tests focus on Saxon and Xalan xml/xslt transformations. In that process he compares a T2000 to a couple of PPC Macs, some Intel and AMD stuff, and a Sun V440.
The Macs hold up well, the higher end AMDs complete transforms in less time per unit, but the T2000 blows everything away on volume - this is an on going series, but here's an on-the-fly comment that could well pass for his summary to date:
It's clear that if you're running a website with a lot of traffic, the T2000 is an excellent choice. Your visitors won't get the fastest possible response time, but you'll handle far more visitors for the same amount of dollar investment than if you're buying the other machines I've tested so far. If you've only got a few visitors, or each visitor needs the fastest possible response time, don't pick the T2000. The T2000 isn't a formula 1 car, it's an 18-wheeler truck.
The next hit was on Mark Mayo's VMUnix Blues. Headlined "So where's my T2000?" because he doesn't have the machine, this one is more a review of Sun Canada's interest in making good on a Sun corporate initiative. Some samples:
Well, I filled out the original form the night the program was announced. People from Sun marketing contacted me to confirm some stuff. I told them what I wanted to do. They were excited. A week later someone at Sun in Ottawa called me. I said 'yeah, send it my way. we'll beat the snot out of it.' I volunteered that our current interface to Sun is with a local VAR. He said, ok, 'I'll work with them and your local sales office and get a box your way ASAP'. And that's the last I heard of it....
March 22nd. One month after I posted this, nearly 3 months after I first applied, and still no T2000. By comparison, two weeks ago I requested an XServe from Apple for Open Directory testing and received it 5 days later. I had never tested anything from Apple before, and currently have zero Apple kit in the server room. Hello? Sun? Anybody there?
Update 3: April 14th. Nearly another month, and even after a face-to-face meeting with Sun Canada President Andy Canham where I was told 'don't worry, we'll get you your T2000' there's still no signs of it.
Sorry Mark, they're from Toronto and you're not - which means you're nobody; get used to it.
On a positive note, the last one I looked at was a report from Robert Milkowski -apparently in Poland.
His headline? "T2000 beats E6500" - here's the key bit:
We put T2000 (8x core, 1GHz) instead of E6500 with 12x US-II 400MHz into our production. We've got heavily multithreaded applications here. Server is doing quite a lot of small NFS transactions and some basic data processing. We didn't recompile applications for T1 - we used the same binaries as for US-II. Applications do use FPU rarely.T2000 gives as about 5-7x the performance of E6500 in that environment.
Well, "quite" good I would say :)
We did real production benchmarks using different servers. Servers were put into production behind load-balancers, then weights on load-balancers were changed so we got highest number of dynamic PHP requests per second. It must sustain that number of requests for some time and no drops or request queue were allowed. With static requests numbers for Opteron and T2000 were even better but we are mostly interested in dynamic pages.
T2000 is over 4x faster than IBM dual Xeon 2.8GHz!
Except x335 server which was running Linux all the other servers were running Solaris 10. Our web server is developed on Linux platform so it's best tuned on it. After fixing some minor problems web server was recompiled on Solaris 10 update1 (both SPARC and x86). No special tuning was done to application and basic tuning on Solaris 10 (increased backlog, application in FX class). Web server was running in Solaris Zones. On x4100 and T2000 servers two instances of web server were run due to application scalability problems. On smaller servers it wasn't needed as CPU was fully utilized anyway. Minimal I/O's were issued to disks (only logs). Putting application into FX class helped a little bit.
I think here are two messages here: first, Sun's gamble on using blogging to generate buzz is starting to look like it will become successfull; and, second, it's pretty clear that the more expertise you bring to testing a T2000, the better your results are going to be.