Original Link: https://www.anandtech.com/show/3768/seamicro-announces-sm10000-server-with-512-atom-cpus-and-low-power-consumption



Two years ago when I first covered Intel’s Atom architecture I proposed that Moore’s Law has paved the way for two things: 1) ridiculously fast microprocessors, and 2) fast enough microprocessors.

The first category is used to push the bleeding edge of software. Everything from scientific computation to 3D gaming. If it’d never been done before, Moore’s Law enabled companies like AMD, Intel and NVIDIA to build the microprocessors we needed to make it happen.

The second category is a more recent development. If you don’t need the compute power, Moore’s Law enabled the creation of smaller, cheaper, more power efficient microprocessors to deliver performance that’s good enough. These types of chips are found in everything from netbooks to smartphones.


Intel's Atom (Silverthorne) Processor

A number of folks are arguing that the same approach can be applied to server workloads. They argue that the majority of time the servers that drive your favorite websites or cloud services are idle, at least from the CPU perspective. For servers that aren’t virtualized, this is largely true. You don’t build servers for average load, you build them to make sure they can withstand maximum load. The unfortunate result is that when these servers are running during periods of low traffic, they aren’t very power efficient.

For a small operation like AnandTech this isn’t much of a problem. But on a larger scale, it adds up. Data centers are easily power constrained. High density servers give us dozens of microprocessor cores in the space of several rack units. That’s great for compute, but terrible for power consumption.

Also keep in mind that even the slowest servers you can buy today are still pretty powerful. In many cases you need physical box redundancy but not the added horsepower of having more hardware. Add in live fail-over support to minimize downtime and you’ve got even more wasted power.

Today if you don’t need the performance a multi-core Xeon can offer you, but you need tons of physical servers, there are very few options. You can stick with a simple single core server but then your power consumption even at idle is still in the range of dozens of watts. You could make the argument that as CPUs get more powerful, there’s room for a category of “fast enough” servers. And thankfully we already have a processor that’s “fast enough”. The Atom.

Take 512 Atom based servers, cram them into a box that consumes 2kW of power, give it boatloads of networking and you’ve got the SM10000 by SeaMicro. This $139K box is designed to replace dozens of quad core Xeon/Opteron boxes that remain idle most of the time. According to SeaMicro if your business model is that you’re giving something away for free on the Internet, then the SM10000 might be for you.

SeaMicro’s CEO comes from the network field, while its CTO is a former Sun and AMD microprocessor architect (he was apparently one of five chief architects on AMD’s Bulldozer core). The company not only makes the 512 Atom server but also a custom ASIC inside that makes the technology work.

The SM philosophy is simple; you don’t take a space shuttle to the grocery store. If you don’t need such a beefy server for your workload, why continue to use one?



Understanding the SeaMicro Architecture

The secret begins at the motherboard level. It all starts with a single core Intel Atom Z530 (1.6GHz, Silverthorne) and US15 chipset (Poulsbo). Astute readers will recognize this as the previous generation Intel Atom platform for MIDs, codenamed Menlow. SeaMicro chose single core Menlow and not the newer Pine Trail platform in order to hit its power targets. Moorestown would probably be a good fit as well but the chips only recently started shipping. Hanging off Poulsbo is 2GB of DDR2 memory.

SeaMicro excludes the I/O hub and instead connects its custom ASIC over Poulsbo’s PCIe x2 interface. The custom ASIC emulates all I/O features, everything from SATA to Gigabit Ethernet is handled by the SeaMicro chip. As far as the Atom CPU is concerned, it has a bunch of I/O devices that hang off of Poulsbo. The virtualized I/O is a key part of making SeaMicro’s technology work.

Those three chips (+ DRAM) make up the basic building block of a SM10000 server. They occupy a PCB area about the size of a credit card: 2.2” x 3”. Since this basic building block is physically autonomous, SeaMicro refers to it as a single server.

SeaMicro then takes eight of these server building blocks and puts them on a card measuring 5” x 11”. Instead of using one SeaMicro ASIC per Atom, the ratio is one ASIC per two Atom processors.


Eight Atom "servers", four SeaMicro ASICs and a 32-lane electrical PCIe interface to the rest of the box

Each one of these cards has a pair of electrical PCIe x16 connectors that plug into the SM10000’s back plane.

A single SM10000 can support up to 64 of these cards, which is how you end up with 512 Atom CPUs in a 10U chassis. Intra-system communication occurs over a multidimensional torus bus interface. The link is built by connecting all of the SM ASICs together, allowing each Atom server to communicate with any other server in the system.

Despite being well connected, the server architecture doesn’t support shared memory (each Atom has exclusive access to its 2GB of DRAM). The torus interface is instead used to share the virtualized I/O amongst all of the servers. If server/CPU 0 wants to access the virtual HDD on server 206, it can. Each hop takes 8 microseconds so it’s fairly low latency for storage and network I/O but not fast enough for memory.

Since each Atom CPU is paired with 2GB of memory, the total machine has a terabyte of DDR2 memory. But like I said earlier, the memory isn’t shared so you have a 2GB maximum limit on each server. This in itself imposes a restriction on the type of applications you’ll run on a SM10000. If you need more than 2GB of memory per server in your rack, the SM10000 isn’t for you.

Poulsbo’s memory controller doesn’t support ECC, which is fine for MIDs but can be a problem for some enterprise customers. SeaMicro claims that most of its customers aren’t bothered by the lack of ECC. There’s no hope for future ECC support unless Intel eventually embraces the Atom platform for servers.

Networking

SeaMicro not only wants to replace some of your server hardware with its boxes, but also some of Cisco’s networking equipment. A single SM10000 is designed to replace your top rack switch.

The idea is you’d take the uplink provided to your backbone and plug it directly into one of the ports on the back of the SM10000. All load balancing, terminal server and switching functionality is handled by the SM10000 itself. It’s all Linux based so you should be able to add a firewall as well.

On the back of the machine you’ll see rows of ethernet ports, up to 64 to be exact. On each one of these cards is a separate CPU that is used to handle all of the network functionality of the server. It helps SeaMicro justify the pricing of the server as you’re replacing not only your server hardware but also some expensive networking gear.

Each server has a physical Gigabit Ethernet interface on it. A fully populated SM10000 can have up to 64 Gigabit Ethernet ports, or it can be configured to have 16 10GbE ports. If you don’t need that much bandwidth you can just use the Ethernet ports you need.

The networking is fully virtualized so each Atom “server” gets its own IP address and thinks it has its own connection to the outside world.

Storage

SeaMicro’s ASIC virtualizes four SATA ports per Atom processor. The SM10000 can support up to 64 physical 2.5” HDDs or SSDs. The customer will configure the machine to determine what four physical disks or slices of disks will map to each Atom CPU.

The SM ASIC emulates RAID-0, but nothing more. SeaMicro states this is because its target market is to replace dozens of simple servers that have limited or no storage. If you’re replacing a couple hundred web servers that only use their storage for OS and little else, the SeaMicro approach makes sense.

OS

Linux is fully supported today but currently there’s no official Windows support. SeaMicro claims the box works just fine running a VM with Windows Server installed however Microsoft doesn’t officially support the configuration. SM is working with Microsoft on fixing that but for now, if you want support, you need to be running Linux.



Final Words

The base configuration of the SeaMicro SM10000 comes with 512 Atom servers, 1TB of DDR2 memory and no storage. SeaMicro charges $139,000 per SM10000 in this configuration.

It is easy to see how this technology might scale. Simple drop the number of cards used in the server and now you have a lower end configuration. SeaMicro needs to be able to make enough revenue to support its business model so I wouldn’t expect any ultra cheap Atom servers out of the company anytime soon. Scaling down appears to be a top priority, although scaling up is also a possibility.

The limitations are of course numerous. There’s no official Windows support, so this for Linux only deployment. Each server is limited in performance not only by the CPU but also by the memory capacity (2GB). There’s no ECC, no real RAID support (other than striping). It’s very clear that the SM10000 can potentially serve one specific niche very well: a setup with tons of servers that have low CPU usage requirements.

The benefits are clear. If you’re running tons of servers that are mostly idle, the SM10000 could save you boatloads on power. By stripping out most of the components on the motherboard and simplifying each server down to something that consumes roughly 4W of power, SeaMicro does have a very valid solution to the power problem in data centers.

The power savings scale up tremendously. On a large scale the cost of running servers really boils down to hardware, then software then power in order of increasing cost. If you’re running tens of thousands of servers, power consumption becomes a real problem. And this is where SeaMicro is most effective:

I suspect that anyone faced with rising data center power costs will be at least interested in what SeaMicro has to offer.

The obvious argument against the SeaMicro approach is virtualization. Combine multiple servers into one beefy Xeon or Opteron machine and then you start to use those processors at their peak efficiency by keeping them loaded most of the time.

It’s difficult to tell based on a press presentation how well the SeaMicro solution fares against a beefy Xeon server running hundreds of VMs. I don’t doubt that it is a lower power solution, the question is whether or not the performance characteristics work with your specific needs.

From where I stand, the SM10000 looks like the type of product that if you could benefit from having it, you’ve been waiting for something like it. In other words, you will have been asking for something like the SM10000 for quite a while already. SeaMicro is simply granting your wish.

Log in

Don't have an account? Sign up now