Willus.com's 2002 Win32 Compiler Benchmarks:
First published January 30, 2002; last updated Dec 29, 2011 [see update
[Be sure to also check out my latest 2011 benchmarks.]
I've written a lot of programs (mostly in C) to do
numerical computations and simulations, and ever since I had to wait for
one to finish, I've been interested in faster computers and faster
compilers. As a graduate student, I grew to appreciate the
availability of free software, particularly the GNU C compiler, a superb
product that is the result of nearly 20 years of software
community work. My first 32-bit PC compiler was DJ Delorie's port of
GNU C to the MS-DOS environment, and it was very nice to be able to
compile code for a flat, 32-bit environment.
Since that time, almost ten years ago, I have migrated primarily to a
Win32 environment, and there are now several available Win32 C/C++
compilers (some of which also compile FORTRAN),
many of them being free. Because of my interest in the
topic, I've tried to outline them on my
Win32 C/C++ Compilers web page,
and now, for the first
time, I've finally done a set of benchmarks to try to give some sense
of comparison between the different compilers.
Updates and Other Links
I am certainly not the first to benchmark compilers. There are numerous
other excellent reviews, including a
Win32 Fortran Compiler Comparison, and
The Great Computer Language Shootout. You can no doubt find many others using
There is also a
G4 spec2K discussion thread with some relevance to compiler
performance on Google Groups.
[Update 6/02:] Thanks to "Justus" for this
to a page on Intel C++ vs. Gnu C.
[Update 8/02:] In their July 2002 issue,
HP World Magazine compared gcc 2.95, gcc 3.04, Intel's Win32 compiler
(Windows 2000 Server), and Intel's Linux compiler (SuSE Linux 8.0) on
a 600 MHz Pentium III HP Vectra PC. Scores (bigger=faster): gcc 2.95 scored
89, gcc 3.04 scored 100, Intel Win32 scored 136, and Intel Linux scored 162.
The benchmark is called the "OBLcpu benchmark 2.0," which, I guess, is
HP World's standard suite of C++ CPU benchmark programs.
[See 10/02 update on summary page for comments on
the -ffast-math flag and MinGW 2.0 performance.]
[Update 12/02:] I missed this article until now, but
in July, Dean Kent at
Real World Technologies
involving Borland, Delphi, Metrowerks, and Microsoft.
[Update 2/03:] Here's a great link
Intel C++ v7.0 to gcc 3.2.1 from coyotegulch.com.
Note that the comparison is in Linux, not Win32.
a long discussion thread about compiler performance here
[Update 9/03:] AMD has published the first SPEC
CFP2000 benchmark that I am aware of where
use the -ffast-math flag with gcc (and they got a very good score). This
means that -ffast-math is a legitimate SPEC compiler option.
[Update 10/03:] See my
MinGW/Gnu C Tips page for some in-line x87 math
routines (sincos, pow, exp, and atan2) which boost the performance
of the MinGW/Gnu C compilers on i386 architectures.
[Update 2/04:] Added results for 3.06 GHz Xeon
with 1 MB cache and Athlon 64 3000+ (laptop).
[Update 3/04:] Added results for Athlon 64 3200+
[Update 6/04:] Scott Robert Ladd at
coyotegulch.com is keeping up his
compiler comparisons between gcc and Intel
(unlike me!). This time he does gcc v3.4 and the yet to be released v3.5
(since renamed to v4.0)!
[Update 7/04:] Since I've read that Intel
is betting a big part of their future on the Pentium-M core (or
something like it), I added results for a 900 MHz Pentium-M
(my wife's Panasonic Toughbook W2 laptop). This required a whole new
category of processor since a Pentium-M is sort of between a P-III and
a P4 (it supports SSE-2 and has a 400 MHz FSB). See
the 1/05 update below, also.
[Update 8/04:] Added results for a 3.6 GHz Xeon (90 nm Nacona core w/EM64T) with 1 MB cache.
[Update 9/04:] Scott Robert Ladd is at it again,
testing GCC 4.0 against Intel 8.1.
[Update 1/05:] I was able to add results from
a 2 GHz Pentium-M (with 2 MB L2 cache) to my benchmarks.
Intel seems well justified in
focusing their efforts on bringing this chip to the desktop world
to replace the Netburst architecture. It
is definitely on par with a 3.6 GHz Pentium-4 and an
Athlon64 3200+. In fact, using the best scores on my five benchmarks
for each processor, averaged and normalized to the 3.6 GHz Xeon (higher
is slower), the results were 1.0 for the Xeon, 1.01 for the Pentium-M,
and 1.15 for the Athlon64 3200+. I would not have guessed that
the Pentium-M would perform so well. I can only guess that Intel
doesn't trumpet the performance more loudly because they don't want
to hurt their Pentium-4 sales.
[Update 5/05:] Scott Robert Ladd published a GCC 4.0 Review.
[Update 6/06:] Principled Technologies did a nice
Intel v9 vs. Gcc 4.1.1 comparison on Linux using SPEC CPU2000. Intel's performance is about 20% better.
[Update 11/06:] If you haven't viewed the
GCC benchmarks page, it's
worth a look. There are loads of plots of SPEC performance on a daily
basis, and there are some tantalizing jumps in the 32-bit performance
on Athlon systems of late (specfp2k). There are also comparisons to
Intel 7.1 and PGI.
[Update 3/07:] I checked through my
comments section on the summary page for the first time in over a year
and had to disable the posting mechanism because of spammers. I'll try to get
the original comments back up soon.
[Update 9/07:] I just found the
Laboratory distributed computing site, which has several excellent
benchmark results, including a 2005 comparison between
Pathscale, Intel, PGI, and GCC compilers for the AMD64 platform.
[Update 1/10:] I just ran some quick
MinGW gcc benchmarks on Windows 7.
[Update 11/11:] I just added some new results to my
MinGW gcc benchmarks page.
[Update 12/11:] I posted my new
2011 compiler benchmarks.
The usual benchmark caveat goes something like this: these are only
benchmarks, and as such may not represent the results you will actually
get if you compare these compilers yourself. There is probably a
different code that could be devised which would score best on each
different compiler. But I did try to take advantage of
honest-to-goodness simulations that I actually use (and mostly that I or
my colleagues actually wrote the code for). Also, I'm a reasonable
programmer (I know what it means to "vectorize" C programs for a Cray), but I
don't pretend to be an expert on writing super-fast, super-efficient
code. Mostly I try to write code as elegantly as possible without doing
anything that will result in an obvious performance penalty. Several
mathematical routines in these simulations were taken from Numerical
Recipes in C, though the Numerical Recipes routines typically don't
account for most of the CPU time used.