(Mon Jan 2 13:49:54 2012)
Thank you for these great practical benchmarks! I have been wondering for a while if the relative GCC/MSVC/Intel performances have changed in recent years (I work in the UK on crop/climate computational modeling). I appreciate the time it must have taken you to do this.
-by Jim W
|
(Thu Jan 19 02:02:54 2012)
Thanks for your work, I appreciated reading it. I would be interested how LLVM 3.0 / CLang would perform. They claim to have faster compilations and better error messages. But I don't have data about performance compared to GCC. Sadly there seems to be only LLVM 2.9 MinGW version to download.
-by uniq
|
(Thu Jan 19 10:48:31 2012)
I used to find that GCC sped up transcendental functions when using the flag "-mfpmath=sse,387". When used with -ffast-math", this allows GCC to use 80387 functions. I think they were faster than their SSE equivalents, although presumably if the SSE were properly vectorized this advantage would go away.
This might be old news now...
-by Ed
|
(Thu Jan 19 21:13:44 2012)
I tried -mfpmath=sse,387 on a couple of these cases and never saw any speed improvements with it.
-by willus.com
|
(Fri Jan 20 07:26:30 2012)
You look to be confusing MSVS (Visual Studio) with CL (the actual compiler). MSVS is an IDE, and the Express version is a free version of the IDE which also includes a version of the compiler, CL which is limited to 32bit executables. However the compiler, including the generation of 64bit executables, is available seperately for free with the the platform SDK.
And dont forget to use the whole program optimization options.
-by anonymous #1
|
(Sat Jan 21 09:43:18 2012)
Thank you for the comments on Microsoft's compiler. I have updated the text to clarify this and have added results with the whole-program optimization flag (/GL). I also added 64-bit results for MS VC 10.
-by willus.com
|
(Sun Feb 26 08:10:47 2012)
How closely did you check that all compilers and different options produce the same result in all programs?
Especially in these two cases it would be good to know to what extent you verified that the programs actually worked:
http://willus.com/ccomp_benchmark2.shtml?p13+s16
http://willus.com/ccomp_benchmark2.shtml?p16+s16
-by anonymous #2
|
(Tue Feb 28 18:37:26 2012)
Benchmark results were the same across the compilers (to within practical precision for floating point cases). In the specific cases you reference, the gcc and Intel output files, for example, were identical.
-by willus.com
|
(Sun Apr 1 10:32:07 2012)
It would be great to see Open Watcom C/C++ compiler included in the tests. It is an open source compiler with more C99 support than Visual C++.
-by TuringIncomplete
|
(Sun Apr 1 13:03:09 2012)
The latest Digital Mars C++ compiler is 8.52, not 8.42.
-by Walter Bright
|
(Mon Apr 2 03:44:36 2012)
Why not comapre LLVM, Open64 ?
-by anonymous #3
|
(Mon Apr 2 07:51:07 2012)
Digital Mars version corrected. My apologies.
-by willus.com
|
(Mon Apr 2 07:55:48 2012)
I looked for LLVM ports to Windows, but I did not find any that look (a) pre-built and (b) straightforward to use. Open Watcom wasn't chosen mostly because I was trying to keep the field down to a reasonable size.
-by willus.com
|
(Mon Apr 2 08:50:28 2012)
Was there a reason you didn't include Embarcadero (aka Borland BCC) compiler?
-by Eric
|
(Sat Apr 7 08:30:17 2012)
Which Embarcadero version would you have me try? The free v5.5 compiler has apparently not changed since my last review. Is their 30-day free trial version of "C++Builder XE2 Architect" significantly advanced from their free v5.5 version?
-by willus.com
|
(Sun Apr 8 18:28:47 2012)
TuringIncomplete--see latest update and comments on "Compilers" page about Open Watcom.
-by willus.com
|
(Sun Jul 8 07:15:03 2012)
Thank you for this fine benchmark and the time you spent on creating it. It was a fine read. It's great that you included the executable sizes as well as some words about how hard/easy the compilers were to set up.
-by Mexxi
|
(Tue Oct 16 11:25:04 2012)
Hi:
I am working on a SPEC benchmark and working on trying to include a compiler benchmark. Would you be interested in helping us out? Looks for something that can be easily be automated and timed. If interested, you have my email. Thanks.
-by Jimmy Leon
|
(Tue Jan 15 12:43:33 2013)
This is great work. I will reccomended it. If you find the time to update this site, it would be interesting to see how the 2012 Microsoft compiler would compare, with it's new autovectorization features.
-by Jan
|
(Fri Feb 22 06:20:50 2013)
There is an ancient c compiler from
Borland the 5,5 its old but for most
users it will do the job with out all the
bloat from microsoft.
-by Alan
|
(Fri Nov 29 19:16:08 2013)
Thanks for doing all this work and posting it. Science!
-by William Cushing
|
(Wed Mar 5 21:02:24 2014)
Regarding the comment about the 64-bit vs 32-bit advantage of 70% (Crafty), this is expected.
The internals of most modern chess engines use a bitboard structure 64 bits wide and do a huge amount of bit operations on them. As such, 64 bit OS + binary yields huge speed improvements.
Chess engines that do not use bitboards, or mailbox engines, see almost no speedup at all from moving to 64-bit. (They only benefit from the extra registers, which the 64-bit ones do as well.) I'm fairly certain Crafty originated as a mailbox engine and then moved to bitboards. Thus it should be easy to verify this concept by simply looking at older versions.
-by Eric
|
(Wed Mar 5 21:20:16 2014)
Consider switching to Stockfish for future tests. Running its benchmark also produces a signature that tells you if it ran properly. Maybe Crafty also has this, not sure.
It's as strong as the best commercial engines and utilizes a distributed testing framework to evaluate patch effects on its strength. Unlike crafty, it is in c++.
-by Eric
|
(Thu Jan 8 01:17:26 2015)
Just want to say thanks for the excellent work and write up. It was all very well thought and carried out. Gerard
-by Gerard
|