Willus.com's 2011 Win32/64 C Compiler Benchmarks:
6. Compiler Issues
As I indicated in the overview, compiling over a million lines of code with
six compilers is no walk in the
park. It is a testament to all of these compilers that I did eventually get
every benchmark to run with each compiler, but there were some issues with every compiler,
though some were more frustrating than others.
Behemoth Install Packages
This is more of a personal issue, and it's probably the Unix/Linux blood in me,
but I like my compiler to be simple, minimal, and command-line friendly, and
Intel fails in all these areas (Microsoft has a pretty large install
package, but isn't bad if you get it via the SDK download).
The install package from Intel is disappointingly bloated.
Intel's download file is 1.6 GiB and the subsequent installation
says it has to install over 40 separate items
(Inspector this, VTune Amplifier that, SQL server this,
.NET framework that, etc., etc., blah, blah, blah)
and consumes approximately 10 GiB of disk
space (Tiny CC, at the opposite end of the spectrum, has a miniscule 1 MiB
install package and uses only 3 MiB of
disk space). I'm sorry, but 1.6 GiB for a compiler package? That's insane.
It's bigger than many operating systems.
Plus I then had to install Microsoft's
compiler to get Intel's to work. I cannot understand why Intel continues to
require Microsoft's compiler. Microsoft's install was much better than
Intel's (you can easily configure what it installs) but still required an
additional 1.1 GiB of disk space. The subsequent
use of batch files by Intel to set up the environment for command-line compiles
was amazingly covoluted and added half a dozen separate new directories to my
exe path, include path, and lib path, many of them needlessly specified twice.
I was able to eventually distill this
down to only the directories I really needed in each path.
Fortunately, Windows 7 lets you sort your installed programs by when
they were installed--a very nice feature for situations like this where one
install package installs a number of separate items.
[Note, Jan 21, 2012: Uninstalling Intel's compiler took 10 full minutes on my PC.]
Large Statically-Initialized Arrays
These caused a lot of problems. Digital Mars and
Microsoft, apparently still dreaming of the good old 1990's, would not compile
statically-intialized character strings larger than 64 KiB(!), and Digital Mars
ran out of memory during the compile phase on a 3 MiB statically-initialized
character array (not a string) even though Win32 processes on my PC have access to at least
2 GiB RAM. The other compilers had
no such limits, but large, statically-initialized arrays did seem to cause issues for Intel and gcc
with inter-procedural optimizations (IPO) turned on. When I commented out
that 3 MiB statically
initialized array (for a large unused font) from one of the source files,
the gcc IPO build times dropped from over 20 minutes to around 3.5 minutes (almost
all of this decrease on the final link).
That same large array caused the Intel IPO
build to fail completely during the library build.
Even worse, after removing a couple of large statically-initialized arrays
(which were not used in the benchmark) from MESHER, one of the Intel-compiled
versions went from
mysteriously crashing to running just fine.
Fortunately, for the purposes of
my benchmarks, I was able to comment out all of the largest
that were causing problems. Ironically, the tiniest compiler (Tiny CC) had
no issues in this regard (though it doesn't offer IPO optimizations).
Windows API Support
Support was lacking in Digital Mars and Tiny CC for some of the
more recent API calls. Tiny CC had to be supplemented with MinGW
headers for some Windows include files--this is pointed out in the Tiny CC
help pages. Again, where necessary, I was able
to comment out a few troublesome spots which were not used by the
benchmark runs anyway.
One benchmark, x264, requires a C99-compliant compiler. Only Intel and gcc had
options for this. The others did not, so there are some missing results under