Willus.com Home   |   Archive   |   About  

   CONTENTS

I. BACKGROUND
    1. Overview
    2. The Compilers
    3. Compiler Options
    4. The Programs
    5. Test Hardware
    6. Compiler Issues
    7. Other notes

II. RESULTS
    1. BW1D
    2. BZIP2
    3. CRAFTY
    4. K2PDFOPT (v1.30)
    5. LAME
    6. MESHER
    7. MODEL3D
    8. RESIZER
    9. TRANSCEND
    10. X264
    11. AVERAGE

III. SUMMARY

IV. COMMENTS

  
  
  
Willus.com's 2011 Win32/64 C Compiler Benchmarks:
I. BACKGROUND

6. Compiler Issues

As I indicated in the overview, compiling over a million lines of code with six compilers is no walk in the park. It is a testament to all of these compilers that I did eventually get every benchmark to run with each compiler, but there were some issues with every compiler, though some were more frustrating than others.

Behemoth Install Packages
This is more of a personal issue, and it's probably the Unix/Linux blood in me, but I like my compiler to be simple, minimal, and command-line friendly, and Intel fails in all these areas (Microsoft has a pretty large install package, but isn't bad if you get it via the SDK download). The install package from Intel is disappointingly bloated. Intel's download file is 1.6 GiB and the subsequent installation says it has to install over 40 separate items (Inspector this, VTune Amplifier that, SQL server this, .NET framework that, etc., etc., blah, blah, blah) and consumes approximately 10 GiB of disk space (Tiny CC, at the opposite end of the spectrum, has a miniscule 1 MiB install package and uses only 3 MiB of disk space). I'm sorry, but 1.6 GiB for a compiler package? That's insane. It's bigger than many operating systems. Plus I then had to install Microsoft's compiler to get Intel's to work. I cannot understand why Intel continues to require Microsoft's compiler. Microsoft's install was much better than Intel's (you can easily configure what it installs) but still required an additional 1.1 GiB of disk space. The subsequent use of batch files by Intel to set up the environment for command-line compiles was amazingly covoluted and added half a dozen separate new directories to my exe path, include path, and lib path, many of them needlessly specified twice. I was able to eventually distill this down to only the directories I really needed in each path. Fortunately, Windows 7 lets you sort your installed programs by when they were installed--a very nice feature for situations like this where one install package installs a number of separate items. [Note, Jan 21, 2012: Uninstalling Intel's compiler took 10 full minutes on my PC.]

Large Statically-Initialized Arrays
These caused a lot of problems. Digital Mars and Microsoft, apparently still dreaming of the good old 1990's, would not compile statically-intialized character strings larger than 64 KiB(!), and Digital Mars ran out of memory during the compile phase on a 3 MiB statically-initialized character array (not a string) even though Win32 processes on my PC have access to at least 2 GiB RAM. The other compilers had no such limits, but large, statically-initialized arrays did seem to cause issues for Intel and gcc with inter-procedural optimizations (IPO) turned on. When I commented out that 3 MiB statically initialized array (for a large unused font) from one of the source files, the gcc IPO build times dropped from over 20 minutes to around 3.5 minutes (almost all of this decrease on the final link). That same large array caused the Intel IPO build to fail completely during the library build. Even worse, after removing a couple of large statically-initialized arrays (which were not used in the benchmark) from MESHER, one of the Intel-compiled versions went from mysteriously crashing to running just fine. Fortunately, for the purposes of my benchmarks, I was able to comment out all of the largest statically-initialized arrays that were causing problems. Ironically, the tiniest compiler (Tiny CC) had no issues in this regard (though it doesn't offer IPO optimizations).

Windows API Support
Support was lacking in Digital Mars and Tiny CC for some of the more recent API calls. Tiny CC had to be supplemented with MinGW headers for some Windows include files--this is pointed out in the Tiny CC help pages. Again, where necessary, I was able to comment out a few troublesome spots which were not used by the benchmark runs anyway.

C99
One benchmark, x264, requires a C99-compliant compiler. Only Intel and gcc had options for this. The others did not, so there are some missing results under x264.


      
  <<  Previous: Test Hardware

Next: Other notes   >>

 
This page last modified
Sunday, 08-Feb-2015 18:58:10 MST