The Opteron 6276: a closer look
by Johan De Gelas on February 9, 2012 6:00 AM EST- Posted in
- IT Computing
- CPUs
- Bulldozer
- AMD
- Opteron
- Cloud Computing
- Interlagos
SQL Server 2008 Enterprise R2
We have been using the Flemish/Dutch web 2.0 website Nieuws.be for some time. 99% of the loads on the database are selects and about 5% of them are stored procedures. You can find a more detailed description here.
We have improved our testing methodology and updated the SQL Server version, so the results are no longer comparable with previous results. We used to publish the highest throughput possible, but we have found that it is not entirely fair. At the highest throughput, there was a very small (<1%) percentage of connection errors (client side timeouts), but those timeouts could make the results vary by about 5-10%. A better configured .NET data provider improved this situation. We adapted the .Net data provider to support the same timeout as the MS SQL Server standard timeout (60s), and we now meticulously scan our logs for errors and discard all results that have any error rates.
Since turbo modes are disabled in the "Balanced" Power management policy and we want to evaluate both power and performance, we tested with both the "Balanced" and "High Performance" power management policies.
We have reported this before: when it comes to pure OLAP MS SQL server throughput, the Opteron Magny-Cours is unbeatable. Notice that both the Xeon and the new Opteron 6276 can hardly leverage Turbo (e.g. "High Performance" mode) even though both Intel and AMD talk about the potential to run at higher clockspeeds even when all cores are active. While this integer intensive workload comes nowhere close to consuming what a Linpack run would require, the TDP headroom is no longer there to enable a clockspeed boost.
Looking at the results, the Opteron 6276 disappoints somewhat as it is not capable of outperforming its older brother. However, it performs relatively close to the Xeon and is thus far from a dud.
At maximum CPU load, the response times paint a very similar picture as the throughput numbers:
When we look at the response times, the Opteron 6174's leadership is confirmed and emphasized. The fact that a 16-cluster 2.3GHz Opteron offers 30% more throughput and twice as fast response times as its 8-cluster 3GHz brother is a clear testimony to the excellent scaling capabilities of SQL server.
Since performance/watt is an extremely important metric, we follow up with a power measurement:
There's no doubt about it: the Opteron 6174 is the performance/watt champion in this particular task.
This is the classical way to evaluate server performance, but should you base your purchase on these numbers alone? Frankly, no. The 100% full load evaluation is incomplete, and it's not related to the real world way of using a database. It just shows what your servers can spit out when running at their maximum, a situation most people either try to avoid or never see as so many other bottlenecks (I/O, lock contention) kick in before you see 100% CPU utilization. It is the best method to evaluate HPC machines, but a short-sighted method for almost any server application (web, database, etc.).
In other words, server benchmarks at 100% are just one datapoint, but we should test at lower concurrencies as well. That is why we simulated 40, 80, 100, 125, 200, 300, 400, 600 and 800 users with our vApus stress testing client. Each user starts a query with between 900 and 1100 ms "thinking time" (so on average 1 second). At 600 and 800 users, our servers achieve their maximum throughput, but how do these servers handle "low" and "midrange" workloads? Let us see what happened when we tested with everyday "normal" loads.
Facebook
Twitter
RSS
46 Comments
View All Comments
Jaguar36 - Thursday, February 9, 2012 - link
I too would love to see more HPC related benchmarks. Finite Element Analysis (FEA) or Computational Fluid Dynamic (CFD) programs scale very well with increased core count, and are something that is highly CPU dependent. I've found it very difficult to find good performance information for CPUs under this load.I'd be happy to help out developing some benchmark problems if need be.
dcollins - Thursday, February 9, 2012 - link
These would indeed be interesting benchmarks to see. These workloads are very floating point heavy so I imagine that the new Opterons will perform poorly. 16 modules won't matter when they only have 8 FPUs. Of course, I am speculating here.Going forward, these types of workloads should be moving toward GPUs rather than CPUs, but I understand the burden of legacy software.
silverblue - Friday, February 10, 2012 - link
They have 8 FPUs capable of 16x 128-bit or 8x 256-bit instructions per clock. On that level, it shouldn't be at a disadvantage.bnolsen - Sunday, February 12, 2012 - link
GPUs are pretty poor for general purpose HPC. If someone wants to fork out tons of $$$ to hack their problem onto a gpu (or they get lucky and somehow their problem fits a gpu well) that's fine but not really smart considering how short release cycles are, etc.I have access to a quad socket magny cours built mid last year. In december I put together a sandy-e 3930k portable demo system. Needless to say the 3930k had at least 10% more throughput on heavy processing tasks (enabling all intel sse dropped in another 15%). It also handily beat our dual xeon nehalem development system as well. With mixed IO and cpu heavy loads the advantage dropped but was still there.
I'd love to be able to test these new amds just to see but its been much easier telling customers to stick with intel, especially with this new amd cpu.
MySchizoBuddy - Friday, March 9, 2012 - link
"GPUs are pretty poor for general purpose HPC."tell that to the #2, #4 and #5 most powerful supercomputers in the world. I'm sure no one told them.
hooflung - Thursday, February 9, 2012 - link
I think I'd rather see some benchmarks based around Java EE6 and an appropriate container such as Jboss AS 7. I'd also like to see some Java 7 application benchmarks ( server oriented ).I'd also like to see some custom Java benchmarks using Akka library so we can see some Software transactional memory benchmarks. Possibly a node.js benchmark as well to see if these new technologies can scale.
What I've seen here is that the enterprise circa 2006 has a love hate relationship with AMD. I'd also like to see some benchmarks of the Intel vs AMD vs SPARC T4 in both virtualized and non virtualized J2EE environments. But this article does have some really interesting data.
jibberegg - Thursday, February 9, 2012 - link
Thanks for the great and informative article! Minor typo for you..."Using a PDU for accurate power measurements might same pretty insane"
should be
"Using a PDU for accurate power measurements might seem pretty insane"
phoenix_rizzen - Thursday, February 9, 2012 - link
MySQL has to be the absolute worst possible choice for testing multi-core CPUs (as evidenced in this review). It just doesn't scale beyond 4-8 cores, depending on CPU choice and MySQL version.A much better choice for "alternative SQL database" would be PostgreSQL. That at least scales to 32 cores (possibly more, but I've never seen a benchmark beyond 32). Not to mention it's a much better RDBMS than MySQL.
MySQL really is only a toy. The fact that many large websites run on top of MySQL doesn't change that fact.
PixyMisa - Friday, February 10, 2012 - link
This is a very good point. While it can be done, it's very fiddly to get MySQL to scale to many CPUs, much simpler to just shard the database and run multiple instances of MySQL. (And replication is single-threaded anyway, so if you manage to get one MySQL instance running with very high inserts/updates, you'll find replication can't keep up.)Same goes for MongoDB and, of course, Redis, which is single-threaded.
We have ten large Opteron servers running CentOS 6, five 32-core and five 48-core, and all our applications are sharded and virtualised at a point where the individual nodes still have room to scale. Since our applications are too large to run un-sharded anyway, and the e7 Xeons cost an absolute fortune, the Opteron was the way to go.
The only back-end software we've found that scales smoothly to large numbers of CPUs is written in Erlang - RabbitMQ, CouchDB, and Riak. We love RabbitMQ and use it everywhere; unfortunately, while CouchDB and Riak scale very nicely, they start out pretty darn slow.
We actually ran a couple of 40-core e7 Xeon systems for a few months, and they had some pretty serious performance problems for certain workloads too - where the same workload worked fine on either a dual X5670 or a quad Opteron. Working out why things don't scale is often more work than just fixing them so that they do; sometimes the only practical thing to do is know what platform works for what workload, and use the right hardware for the task at hand.
Having said all that, the MySQL results are still disappointing.
JohanAnandtech - Friday, February 10, 2012 - link
"It just doesn't scale beyond 4-8 cores, depending on CPU choice and MySQL version."You missed something: it does scale beyond 12 Xeon cores, and I estimate that scaling won't be bad until you go beyond 24 cores. I don't see why the current implementation of MySQL should be called a toy.
PostgreSQL: interesting several readers have told me this too. I hope it is true, because last time we test PostgreSQL was worse than the current MySQL.