Today’s processors: the value angle

Almost a year has passed since our first CPU value article. Since then, we’ve taken a look at the value propositions of current graphics processors, but we haven’t taken another shot at CPUs—until today, that is. Fresh from our review of AMD’s triple-core Phenom X3 processors, we’ve gathered all of our benchmark results and tossed current official pricing into the equation to see how much bang Intel’s and AMD’s latest processors deliver for your buck.

In this look at the CPU market, we explore how value scales across two, three, and four cores, whether bargain prices make AMD’s Phenoms attractive from a value perspective, and how Intel’s blazing-fast dual-core CPUs like the Core 2 Duo E8400 fit into the picture. Our results span 20 processors, 18 different benchmarks, and some custom tests that take power efficiency and more complete system pricing into account. Keep reading to see where the best values lie in the CPU market.

Quantifying CPU value
We noted in our GPU value article that it’s much easier to calculate value with microprocessors than with graphics processors. That’s true, but CPU value isn’t a completely straightforward affair either. We have two main variables to work with—performance and price—so working out a “performance per dollar” figure is easy enough. However, that equation doesn’t take into account a wealth of other performance factors, including memory bandwidth, storage bottlenecks, and graphics processing power in games. Power consumption also plays an important role, especially when it comes to the cost of a cooling solution. We’ll focus on CPU performance-per-dollar almost exclusively in this article, but while reading, keep in mind that there are other factors to take into account.

With that out of the way, let’s look at our test subjects. We have 20 processor configurations, 11 of which bear the Intel logo:

The other nine chips come from AMD:

As in our last CPU value piece, we’ve taken bulk prices straight from Intel and AMD. Volume prices aren’t always representative of what consumers pay, but the two are close enough right now, and selecting “official” figures saves us the trouble of working out averages from fickle online retailers. Again, there are other factors to take into account here, such as platform and power costs. We’ll look at how platform pricing fits into all this soon, but let’s focus on just the processors for now.

Oh, also, we’ve included AMD’s Phenom 9500 and 9600 processors in our tests. These two chips are plagued by the infamous TLB erratum, the fix for which (enabled in most BIOSes by default, we might add), hurts performance significantly. We’ve disabled the fix for our tests, so our results should roughly represent the performance of the new, erratum-free 9550 and 9650 CPUs. We nevertheless resisted the temptation to simply rename our test chips, because the 9550 and 9650 are based on a new stepping of silicon and may perform slightly differently.

To paint a reasonably thorough picture of how all these processors handle common tasks, we’ve selected 18 benchmarks spanning everything from games, general productivity applications, and media encoding to folding, 3D rendering, and game development. The vast majority of these tests cover real-world applications, which we think is fitting considering the practical aim of this article.

Organizing all this data in a readable fashion isn’t an easy enterprise, so we’ve relied on a mix of charts and scatter plots once again. For each benchmark, we’ll start with a look at raw performance, then move on to a bar chart that ranks chips based on how much performance they deliver per dollar. When we’re dealing with tests that generate higher scores with faster CPUs, we’ll look at “score point per dollar.” However, when the performance metric shifts to time in seconds, where lower scores represent better performance, we’ll rank chips based on test run rate per dollar. The test run rate will be the reciprocal of the test score, expressed in megahertz.

The third tool we use quantify value is our trusty scatter plot:

Those who’ve checked out our previous two value articles will know the drill. The best possible chip would be at the top left, costing nothing while delivering maximum performance, while the worst would be at the bottom right, offering the lowest performance at the highest cost.

There’s a twist here, though. Since we have lots of chips, and our two-way Core 2 Extreme QX9775 would prolong the X axis substantially due to its high price, we’ve made two compromises in order to keep our scatter plots readable. For one, you won’t find the QX9775 setup listed at all in our scatter plots. This isn’t much of a sacrifice, since the priciest configuration almost always offers the worst value. The second compromise involves cropping the bottom of our Y axis. In the plot above, for example, the Y scale starts at 2000 instead of zero. We prefer to maintain a proper scale for our graphs here at TR, but in this case, we’ve elected to err on the side of readability. The scatter plots are a little tricky to decipher as it is, and without cropping, all our results would often be squeezed into roughly two square inches of graph area.

Let’s now have a brief look at our test systems and, after that, our results.

Test notes
We underclocked a Core 2 Extreme QX9650 in order to simulate our Core 2 Quad Q9450, but performance should be exactly the same. Power consumption should be a different matter, so we left the Q9450 out of those tests. We used the same method to simulate a Core 2 Duo E8400 with an underclocked E8500 and a Phenom X3 8450 with an underclocked 8750.

Our testing methods
As ever, we did our best to deliver clean benchmark numbers. Tests were run at least three times, and the results were averaged.

Our test systems were configured like so:

Please note that testing was conducted in two stages. Non-gaming apps and Supreme Commander were tested with Vista patches KB940105 and KB929777 (nForce/790FX systems only) and ForceWare 163.11 drivers. The other games were tested with the additional Vista patches KB938194 and KB938979 and ForceWare 163.71 drivers.

Thanks to Corsair for providing us with memory for our testing. Their products and support are far and away superior to generic, no-name memory.

Our single-socket test systems were powered by OCZ GameXStream 700W power supply units. The dual-socket systems were powered by PC Power & Cooling Turbo-Cool 1KW-SR power supplies. Thanks to OCZ for providing these units for our use in testing.

Also, the folks at NCIXUS.com hooked us up with a nice deal on the WD Caviar SE16 drives used in our test rigs. NCIX now sells to U.S. customers, so check them out.

The test systems’ Windows desktops were set at 1280×1024 in 32-bit color at an 85Hz screen refresh rate. Vertical refresh sync (vsync) was disabled.

We used the following versions of our test applications:

WorldBench 6 beta 2
• Team Fortress 2
• Lost Planet: Extreme Condition with DirectX 10
• BioShock 1.0 with DirectX 10
• Valve VRAD map build benchmark
• Valve Source Engine particle simulation benchmark
• Cinebench R10 64-bit Edition
• POV-Ray for Windows 3.7 beta 21a 64-bit
• notfred’s Folding benchmark CD 8/8/07 revision
• The Panorama Factory 4.5 x64 Edition
• Windows Media Encoder 9 x64 Edition
• LAME MT 3.97a 64-bit

The tests and methods we employ are usually publicly available and reproducible. If you have questions about our methods, hit our forums to talk with us about them.

WorldBench
WorldBench’s overall score is a pretty decent indication of general-use performance for desktop computers. This benchmark uses scripting to step through a series of tasks in common Windows applications and then produces an overall score for comparison. WorldBench also records individual results for its component application tests, allowing us to compare performance in each. We’ll look at the overall score, and then we’ll show individual application results alongside the results from some of our own application tests.

Already, we see some of the same trends as in our past articles. The cheapest offerings—AMD’s Athlon X2 5600+ and Intel’s Core 2 Duo E7200—score highest on our performance-per-dollar chart, while the outrageously expensive Core 2 Extreme QX9775 two-way setup delivers the least bang for your buck.

The scatter plot helps bring the performance question into perspective. The Core 2 Duo E7200 and Core 2 Duo E8400 fare better than alternatives at similar price points, and the Athlon 64 X2 5600+ doesn’t look so hot. Quad-core chips don’t look especially attractive here, either, which isn’t surprising considering that few WorldBench tests take advantage of more than two cores.

We should probably take a minute to say something about AMD. As you can see in the scatter plot, all of the AMD chips are clumped together below the Intel offerings. The differences in performance between the two aren’t huge—remember that we cropped our Y axes to make the results readable—but AMD’s offerings don’t perform as well in WorldBench’s application suite overall.

Productivity and general use software

MS Office productivity

The AMD CPUs do well in Microsoft Office. Our value chart and scatter plot both show the Athlon X2 5600+ beating out the Core 2 Duo E7200, and the Athlon 64 X2 6000+ and 6400+ rival the Core 2 Quad Q9300 and Core 2 Duo E8500. However, a look at the Y axis and our performance chart suggests that the Office test isn’t particularly CPU-bound, making for relatively small differences in actual performance.

Firefox web browsing

The picture is a little more mixed in Firefox, where the E7200 gets back at the 5600+ and the E8400 steps all over the 6400+. Because Firefox doesn’t seem to have much use for more than a couple of CPU threads, Phenoms and Core 2 Quads don’t really shine.

Image processing and multitasking

Photoshop

Like Firefox, this Adobe Photoshop CS2 test doesn’t benefit significantly from more than a couple of CPU cores. Our scatter plot shows the Core 2 Duo E8400 towering over pricier Core 2 Quads, not to mention AMD’s entire Phenom X4 lineup. Here again, the E7200 and E8400 both look very well positioned.

The Panorama Factory photo stitching
The Panorama Factory handles an increasingly popular image processing task: joining together multiple images to create a wide-aspect panorama. This task can require lots of memory and can be computationally intensive, so The Panorama Factory comes in a 64-bit version that’s multithreaded. We asked it to join four pictures, each eight megapixels, into a glorious panorama of the interior of Damage Labs. We’ve added up the total operation time to give us an overall measure of performance.

At last, an application that showcases the potential of quad-core processors. Intel’s Core 2 Quads and AMD’s Phenoms zoom past comparable dual-core chips in The Panorama Factory, and even AMD’s slowest quad-core Phenom manages to outperform the Core 2 Duo E8400.

Where’s the sweet spot here? When we take a gander at sub-$200 offerings, the Core 2 Duo E7200 and E8400 look to us like the best choices. If you want to spend more than $200, it’s a toss-up between the Core 2 Quad Q6600 and the Q9300—the former being a little slower but less expensive. We’d probably side with the Q9300, despite its lower value score, due to its superior performance and higher power efficiency. More on that in a moment.

Multitasking – Firefox and Windows Media Encoder

Multitasking normally showcases the strengths of multiple CPU cores, so we’re a little surprised to see the Core 2 Duo E8400 and E8500 do so well here. The E8500 may cost the same as the Q9300, but it outperforms the quad-core chip by about 13%. The E8400 looks even better, though, considering that it’s nearly as fast as the E8500, costs $80 less, and ranks third in performance per dollar.

Team Fortress 2
We’ll kick off our gaming tests with some Team Fortress 2, Valve’s class-driven multiplayer shooter based on the Source game engine. In order to produce easily repeatable results, we’ve tested TF2 by recording a demo during gameplay and playing it back using the game’s timedemo function. In this demo, we played as the Heavy Weapons Guy, with a medic in tow, dealing some serious pain to the blue team.

We used a relatively low display resolution with low levels of filtering and AA in order to prevent the graphics card from becoming a primary performance bottleneck, so we could show you the performance differences between the CPUs. We tested at 1024×768 resolution with the game’s detail levels set to their highest settings. HDR lighting and motion blur were enabled. Antialiasing was disabled, and texture filtering was set to trilinear filtering only.

Judging by both our CPU usage graph and our performance results above, Team Fortress 2 only takes advantage of one full CPU core. The Core 2 Duo E7200 and E8400 continue to do well here, occupying prime real estate in our scatter plot and delivering good performance per dollar in our value chart.

Somewhat embarrassingly for AMD, the $133 E7200 beats out the best Athlon and Phenom processors on the market. And really, the E7200 may be all you need to have a good gaming experience in TF2. The E8400 certainly looks like a tantalizing step up, though, and it’s in third place in the value chart.

BioShock
We tested BioShock by manually playing through a Big Daddy fight five times while recording frame rates using the FRAPS utility.

This method has the advantage of simulating real gameplay quite closely, but it comes at the expense of precise repeatability. We believe five sample sessions are sufficient to get reasonably consistent results. In addition to average frame rates, we’ve included the low frame rates, because those tend to reflect the user experience in performance-critical situations. In order to diminish the effect of outliers, we’ve reported the median of the five low frame rates we encountered.

For this test, we largely used BioShock‘s default image quality settings for DirectX 10 graphics cards, but again, we tested at a relatively low resolution of 1024×768 in order to prevent the GPU from becoming the main limiter of performance.

AMD salvages its honor in BioShock thanks to a strong performance from the Phenom X4 9750. However, we should point out that BioShock doesn’t seem all that bound by CPU performance—at least that’s how we’d explain why half the processors we tested produced close to the same frame rates, and why the 9750 somehow outdid the higher-clocked Phenom X4 9850.

We believe Intel’s Core 2 Duo E7200 lies in the sweet spot here, trailed closely by the Phenom X3 8450. Speedier dual-core chips aren’t much faster, and extra cores don’t seem to help a whole lot.

Lost Planet: Extreme Condition
Lost Planet puts the latest hardware to good use via DirectX 10 and multiple threads—as many as eight, in fact. Lost Planet‘s developers have built a benchmarking tool into the game, and it tests two different levels: a snow-covered outdoor area with small numbers of large villains to fight, and another level set inside of a cave with large numbers of small, flying creatures filling the air. The former doesn’t appear to be CPU-bound, so we’ll be looking at the latter.

We tested this game at 1152×864 resolution, largely with its default quality settings. The exceptions: texture filtering was set to trilinear, edge antialiasing was disabled, and “Concurrent operations” was set to match the number of CPU cores available.

Lost Planet‘s ability to harness multiple CPU cores effectively may be rather unique among today’s games, but it shows what may be in store for future titles. Here, it seems you can’t go wrong with any quad-core chip priced around $200, be it a Phenom X4 or Core 2 Quad. The Phenom X4 9500 looks particularly attractive, outdoing the Core 2 Duo E8400 at almost the same price point while nipping at the heels of a Core 2 Quad Q6600 that costs $30 more.

Audio and video encoding

LAME MT audio encoding
LAME MT is a multithreaded version of the LAME MP3 encoder. LAME MT was created as a demonstration of the benefits of multithreading specifically on a Hyper-Threaded CPU like the Pentium 4. Of course, multithreading works even better on multi-core processors. You can download a paper (in Word format) describing the programming effort.

Rather than run multiple parallel threads, LAME MT runs the MP3 encoder’s psycho-acoustic analysis function on a separate thread from the rest of the encoder using simple linear pipelining. That is, the psycho-acoustic analysis happens one frame ahead of everything else, and its results are buffered for later use by the second thread. That means this test won’t really use more than two CPU cores.

Our results focus on the 64-bit version of LAME MT built with Intel’s compiler. We are encoding a massive 10-minute, 6-second 101MB WAV file at a variable bit rate.

LAME MT’s dual-threaded execution allows Intel’s speedy 45nm Core 2 Duos to take the cake yet again. The E7200 and E8400 both offer compelling value.

Windows Media Encoder x64 Edition video encoding
Windows Media Encoder is one of the few popular video encoding tools that uses four threads to take advantage of quad-core systems, and it comes in a 64-bit version. Unfortunately, it doesn’t appear to use more than four threads, even on an eight-core system. For this test, we asked Windows Media Encoder to transcode a 153MB 1080-line widescreen video into a 720-line WMV using its built-in DVD/Hardware profile. Because the default “High definition quality audio” codec threw some errors in Windows Vista, I instead used the “Multichannel audio” codec. Both audio codecs have a variable bitrate peak of 192Kbps.

Windows Media Encoder juggles multiple CPU threads comfortably, allowing quad-core chips to stretch their legs—and the Phenom X4 9500 to trump the Core 2 Duo E8400. Further up the ladder, the Core 2 Quad Q6600 just barely edges out the Phenom X4 9850.

As prices climb, performance returns begin to diminish. Perhaps the best examples of this trend are the Core 2 Extreme QX9650 and 9770, both of which fail to outperform significantly the vastly cheaper Core 2 Quads. The two-way Core 2 Extreme QX9775 setup isn’t in our scatter plot, but our performance chart shows it’s barely any faster than the QX9770.

Incidentally, if you’re wondering why the Phenom X3s fare so poorly, it’s because Windows Media Encoder only makes use of two of their cores. AMD has told us it’s working with Microsoft on a fix.

WinZip file compression

The Core 2 Duo E7200 and Core 2 Quad Q9300 offer nearly identical performance in WinZip, suggesting that this is yet another application that doesn’t make effective use of quad-core processors. As we saw with other apps that don’t take advantage of more than two cores, the E7200 and E8400 look to be the best values south and north of the $150 mark, respectively. AMD’s Athlon and Phenom chips all trail their Intel rivals by substantial margins here.

3ds max rendering

Our 3ds max results somewhat mirror those of the Windows Media Encoder test: AMD’s Phenom X4 9500 outdoes the Core 2 Duo E8400, while the Core 2 Quad Q6600 trumps the fastest Phenoms. Moving up the price scale, Core 2 Extreme offerings don’t exactly justify their lofty price premiums with markedly better performance. Even the eight-core QX9775 setup fares poorly when we look at performance per dollar.

Cinebench rendering
Cinebench is a benchmark based on Maxon’s Cinema 4D rendering engine. It’s multithreaded and comes with a 64-bit executable. This test runs with just a single thread and then with as many threads as CPU cores are available.

Cinebench gives more love to the Phenoms—so much so, in fact, that the Phenom X4 9500 tops our value chart. Our scatter plot confirms its attractive positioning. The Core 2 Quad Q9300 is probably the next step we’d consider, since other sub-$250 quad-core offerings all lie quite close to one another.

POV-Ray rendering
We caved in and moved to the beta version of POV-Ray 3.7 that includes native multithreading. The latest beta 64-bit executable is still quite a bit slower than the 3.6 release, but it should give us a decent look at comparative performance, regardless.

POV-Ray makes good use of three and four cores, and the Phenoms perform well here as a result. In fact, the $235 Phenom X4 9850 outperforms the $266 Core 2 Quad Q9300 and sits at nearly the same performance level as the $999 Core 2 Extreme QX6850. The $215 Phenom X4 9750 ranks higher on our value chart than the X4 9850, but for the record, we really like the X4 9850’s unlocked multiplier.

Folding@Home
We have a slick little Folding@Home benchmark CD created by notfred, one of the members of Team TR, our excellent Folding team. For the unfamiliar, Folding@Home is a distributed computing project created by folks at Stanford University that investigates how proteins work in the human body, in an attempt to better understand diseases like Parkinson’s, Alzheimer’s, and cystic fibrosis. It’s a great way to use your PC’s spare CPU cycles to help advance medical research. I’d encourage you to visit our distributed computing forum and consider joining our team if you haven’t already joined one.

The Folding@Home project uses a number of highly optimized routines to process different types of work units from Stanford’s research projects. The Gromacs core, for instance, uses SSE on Intel processors, 3DNow! on AMD processors, and Altivec on PowerPCs. Overall, Folding@Home should be a great example of real-world scientific computing.

notfred’s Folding Benchmark CD tests the most common work unit types and estimates performance in terms of the points per day that a CPU could earn for a Folding team member. The CD itself is a bootable ISO. The CD boots into Linux, detects the system’s processors and Ethernet adapters, picks up an IP address, and downloads the latest versions of the Folding execution cores from Stanford. It then processes a sample work unit of each type.

On a system with two CPU cores, for instance, the CD spins off a Tinker WU on core 1 and an Amber WU on core 2. When either of those WUs are finished, the benchmark moves on to additional WU types, always keeping both cores occupied with some sort of calculation. Should the benchmark run out of new WUs to test, it simply processes another WU in order to prevent any of the cores from going idle as the others finish. Once all four of the WU types have been tested, the benchmark averages the points per day among them. That points-per-day average is then multiplied by the number of cores on the CPU in order to estimate the total number of points per day that CPU might achieve. We’ll be looking at that last figure for this value comparison. You can check detailed numbers for each unit type in our Phenom X3 review.

This may be a somewhat quirky method of estimating overall performance, but our sense is that it generally ought to work. We’ve discussed some potential reservations about how it works here, for those who are interested. We have included results for each of the individual WU types below, so you can see how the different CPUs perform on each.

These results suggest folding enthusiasts ought to at least consider AMD’s Phenom line. That said, the performance of the Core 2 Quad Q9300 towers over cheaper quad-core offerings, and it’s ranked third in our value chart.

Valve Source engine particle simulation
Next up are a couple of tests we picked up during a visit to Valve Software, the developers of the Half-Life games. They had been working to incorporate support for multi-core processors into their Source game engine, and they cooked up a couple of benchmarks to demonstrate the benefits of multithreading.

The first of those tests runs a particle simulation inside of the Source engine. Most games today use particle systems to create effects like smoke, steam, and fire, but the realism and interactivity of those effects are limited by the available computing horsepower. Valve’s particle system distributes the load across multiple CPU cores.

The results in Valve’s particle simulation benchmark are reminiscent of those we saw in Lost Planet‘s Cave benchmark a few pages back. Here, however, the Core 2 Quad Q6600 comes out ahead of the Phenom pack and sets up camp at the top of our value chart. The Q9300 isn’t far behind, offering slightly higher performance at a slightly higher price.

Valve VRAD map compilation
This next test processes a map from Half-Life 2 using Valve’s VRAD lighting tool. Valve uses VRAD to precompute lighting that goes into games like Half-Life 2. This isn’t a real-time process, and it doesn’t reflect the performance one would experience while playing a game. Instead, it shows how multiple CPU cores can speed up game development.

Here’s another test where the Core 2 Quad Q6600 excels. Intel’s cheapest quad-core CPU finds itself at the top of our value chart again, and our scatter plot shows it comfortably above similarly-priced Phenoms from AMD. The Core 2 Quad Q9300 comes in second place in the value chart, providing a good alternative that costs a little more but also runs a little faster, all while consuming less power overall.

Power efficiency
We’ve talked a little bit about power envelopes throughout the past few pages, so let’s have a closer look at how our 20 CPUs compare when you factor power efficiency into the value equation. We measured power utilization with an Extech 380803 power meter. The meter reads power use at the wall socket, so it incorporates power use from the entire system—the CPU, motherboard, memory, graphics solution, hard drives, and anything else plugged into the power supply unit. (We plugged the computer monitor into a separate outlet, though.) We measured how each of our test systems used power across a set time period, during which time we ran Cinebench’s multithreaded rendering test.

Almost all of the systems had their power management features (such as SpeedStep and Cool’n’Quiet) enabled during these tests via Windows Vista’s “Balanced” power options profile. The exception here was the dual QX9775 “Skulltrail” system, since its BIOS didn’t support SpeedStep.

For this piece, we’ve limited the scope of our results to our render energy test, which tells us how much energy each system used to render our test scene in Cinebench. For a more detailed look at our methodology and power efficiency tests for these chips, you can check out our full results in our review of AMD’s Phenom X3 processors.

For now, let’s look at power efficiency per dollar. The 1/microjoules value in our power efficiency per dollar graph is really 1/(watt-seconds/1000000), or 1,000,000 m^-2 kg^-1 s². That’s a little obscure, but it quantifies power efficiency in a readable fashion based on the source data, which is in joules.

High-end, quad-core chips are the most power-efficient in this test (and they should be, since they spend less time at their fully loaded state rendering the scene). Still, the Core 2 Duo E7200 trumps other contenders in our power-efficiency-per-dollar chart. Looking at our scatter plot, we see that the E7200 outdoes all of our AMD chips and comes close to the level of efficiency of the Core 2 Quad Q6600 and Core 2 Duo E8500.

The next step up is undeniably the Core 2 Quad Q9300, which places second in our value rankings. This chip rivals the power efficiency of the QX9650 at a fraction of the price, and our scatter plot shows it sitting quite close to the magical top left corner of the plot. On the other side, the two-way Core 2 Extreme QX9775 provides similar power efficiency at more than ten times the cost.