In the Lab A behind-the-scenes look at Maximum PC testing
The New Benchmarks Are Here!
The New Benchmarks Are Here!
We proudly present our new benchmarks and
zero-point system for 200492 MAXIMUMPC DECEMBER 2003
I
t’s been a long time coming, but after
much testing and debate, we’re ready
to present a new set of system bench-
marks. This month, we describe the new
tests, along with a new “zero-point sys-
tem” that will be referenced in our bench-
marking charts. Next month, you’ll see
the new tests and zero-point rig used in
actual practice in our system reviews.
For our latest benchmarking refresh,
we continue to use real-world apps like
Photoshop and MusicMatch to gauge sys-
tem performance. Synthetic tests that
run raw math calculations have their
place in some product evaluations, but
they can’t, for example, tell you how a
particular CPU upgrade might affect
frame rates in your favorite game. Plus,
as advocates for PC enthusiasts world-
wide, we want to do everything we can
to encourage the use of real-world tests.
Here’s why: Hardware vendors will
always tweak their drivers to do well in
benchmarks. When they tweak them to
excel in synthetic apps, the driver revi-
sions can actually lead to lower perfor-
mance in real-world apps. But when they
tweak drivers for real-world apps, soft-
ware users actually reap tangible benefits.
Before we nailed down the exact apps
and testing procedures for our new bench-
marking suite, we ran our proposed tests
on various hardware configurations. Our
goal was to make sure our tests produced
consistent results, and that every bench-
mark would run on today’s most common
hardware platforms.
On the following pages, we’ll first
introduce you to our new tests, and then
we’ll describe the hardware profile of our
zero-point rig, which serves as both a
reference point for comparative testing,
and as a host system for individual com-
ponents that are being reviewed in the
magazine.Our New Tests
SYSmark2004
If you’re big on benchmark-
ing, you already know the story
behind BAPCO’s SYSmark testing
suites. Every version of the suite
uses the kernels of real-world
apps to gauge overall system
performance—CPU speed, bus
speed, memory speed, and even
hard drive speed. The bench-
mark basically loads a kernel,
runs the app through an action
script, and times how long the
script takes to complete. After all the
scripts have been timed, the benchmark
spits out three numbers: a measurement
of performance in content creation apps,
a measurement of performance in office
apps, and an overall average of both
scores. AMD has historically complained
that SYSmark unfairly favors Intel CPUs,
but both Intel and AMD were given an
opportunity to contribute to the develop-
ment of SYSmark2004 , so we think the
controversy is now moot.
The latest version of the test-
ing suites runs a total of 17 different
apps. For a complete list of apps, go to
http://www.bapco.com. When SYSmark2004 is
referenced in system reviews, you’ll see
its final, cumulative score (the higher the
score, the better a system’s performance).
We do, however, often report individual
content creation and office app scores in
other types of coverage.
Now for the bad news: At press
time, BAPCO was still working on
SYSmark2004’s code and ironing out bugs.
We expect to switch to SYSmark2004 as
soon as it’s publicly available and fully
tested in our Lab. Our guess is that it’ll
first appear in our February issue, andthat for next month’s issue, you’ll find
SYSmark2002 test scores.Premiere Pro
Adobe’s video-editing warhorse Premiere
gets not only a face-lift, but also an entire-
ly new software core. Most importantly,
the app was rewritten to fully support
Intel’s Hyper-Threading technology and
multiprocessor machines.
Previous versions of Premiere favored
the processing efficiency of AMD CPUs,
but the rewrite seems to favor Intel’s raw
clock speed advantage. AMD fanboys
will certainly cry foul over the situation,
but the bottom line is that Premiere is an
important app, and we’d be remiss if we
didn’t update to the latest version. Plus, as
all the world’s software developers recom-
pile and update their code, they find new
performance to be had in the
P4 architecture.
Before we run our own homegrown
Premiere Pro test, we defrag the system’s
hard drive. We then load the test file—
a 313MB AVI file of birds soaring over a
shoreline—and run it through an action
script. The script combines the file with
an MP3 track, adds numerous transitions,Our Photoshop action script uses the app’s
entire complement of filters—save three that
broke our benchmark.