Stefan Pohl Computer Chess

Latest Website-News (2018/10/13): The holiday-break is over - testing resumed. Next testrun: Stockfish 180927. Result not before Friday.

The long thinking-time testruns of LC Zero are suspended, because the neural-net learning of LC Zero was restarted and until now, the new 20xxx nets are much weaker, than the latest 10xxx nets. My testruns will continue, when the new nets are getting as strong as the older 10xxx nets. I will do some fast testruns of LC Zero vs. Stockfish from time to time, to measure the progress of the 20xxx nets beyond pure selfplay-Elo.

My SALC V5 openings and books are ready for download. Check out the "SALC openings"-section on this website for further information. Download SALC V5.02 here

Stay tuned.

Stockfish testing

Playing conditions:

Hardware: i7-6700HQ 2.6GHz Notebook (Skylake CPU), Windows 10 64bit, 8GB RAM

Fritzmark: singlecore: 5.3 / 2521 (all engines running on one core, only), average meganodes/s displayed by LittleBlitzerGUI: Houdini: 2.6 mn/s, Stockfish: 2.2 mn/s, Komodo: 2.0 mn/s

Hash: 512MB per engine

GUI: LittleBlitzerGUI (draw at 130 moves, resign at 400cp (for 4 moves))

Tablebases: None

Openings: HERT testset (by Thomas Zipproth) (download the file at the "Download & Links"-section or here)(I use a version of HERT, where the positions in the file are ordered in a different way - makes no difference for testing-results, dont be confused, when you download my gamebase-file and the game-sequence doesnt match with the sequence of your HERT-set...)

Ponder, Large Memory Pages & learning: Off

Thinking time: 180''+1000ms (= 3'+1'') per game/engine (average game-duration: around  7.5 minutes). One 5000 games-testrun takes about 7 days.The version-numbers of the Stockfish engines are the date of the latest patch, which was included in the Stockfish sourcecode, not the release-date of the engine-file, written backwards (year,month,day))(example: 170526 = May, 26, 2017). Since July, 2018 I use the abrok-compiles of Stockfish again (http://abrok.eu/stockfish), because they are now much faster than before - now only 1.3% slower than BrainFish-compiles. So, there is no reason anymore to not use these "official" development-compiles.

Download BrainFish (and the Cerebellum-Library): here

Each Stockfish-version plays 1000 games versus Komodo 12, Houdini 6, Fire 7.1, Ethereal 11, Fizbo 2. All engines are running with default-settings, except: Move Overhead is set to 300ms, if an engine allows to do so. And for asmBrainFish, the option BestBookMove is set to true (default=false), for best performance, using Cerebellum-Library.

To avoid distortions in the Ordo Elo-calculation, from now, only 2x Stockfish (latest official release + the latest version) and 1x asmFish and 1x Brainfish are stored in the gamebase (all older engine-versions games will be deleted, every time, when a new version was tested). Stockfish, asmFish and BrainFish older Elo-results can still be seen in the Elo-diagrams below. BrainFish plays always with the latest Cerebellum-Library of course, because otherwise BrainFish = Stockfish.

Latest update: 2018/09/22: Stockfish 180910

(Ordo-calculation fixed to Stockfish 9 = 3450 Elo)

See the individual statistics of engine-results here

Download the current gamebase here

Download the archive (all played games with HERT (268000 games)) here

See a ORDO-rating of the complete HERT-archive-base here

     Program                    Elo    +    -   Games   Score   Av.Op.  Draws

   1 asmBrainFish 180510      : 3528    8    8  5000    77.9 %   3292   39.3 %
   2 BrainFish 180728 bmi2    : 3528    8    8  5000    77.8 %   3292   39.0 %
   3 Stockfish 180910 bmi2    : 3488    7    7  5000    73.3 %   3301   44.5 % (new)
   4 asmFish 180503 bmi2      : 3479    8    8  5000    73.0 %   3292   44.1 %
   5 Stockfish 9 180201       : 3450    6    6  8000    74.4 %   3251   41.5 %
   6 Houdini 6 pext           : 3421    4    4 12000    60.0 %   3338   50.5 %
   7 Komodo 12 bmi2           : 3390    5    5 12000    55.9 %   3341   50.3 %
   8 Fire 7.1 popc            : 3275    4    4 12000    40.8 %   3350   47.5 %
   9 Ethereal 11 pext         : 3231    6    6  7000    35.9 %   3343   46.5 %
  10 Fizbo 2 bmi2             : 3188    5    5 12000    30.0 %   3357   35.1 %
  11 Shredder 13 x64          : 3186    5    5 11000    31.1 %   3346   40.9 %
  12 Booot 6.3.1 popc         : 3175    7    7  6000    32.1 %   3318   42.9 %
  13 Andscacs 0.94 popc       : 3139    7    7  6000    28.1 %   3318   37.4 %

The 4 different Fishes in this Elo-list:

- Stockfish

- asmFish = Stockfish manually rewritten in assembler (look here)

- BrainFish = Stockfish playing with Cerebellum-Library by Thomas Zipproth (look here)

- asmBrainFish = asmFish playing with Cerebellum-Library (UCI-option BestBookMove set to true)

The version-numbers (180622 for example) of the engines are the date of the latest patch, which was included in the Stockfish sourcecode, not the release-date of the engine-file. Especially the asmFish-engines are often released much later!!

Below you find a diagram of the progress of Stockfish in my tests since the end of 2016. 

And below that diagram, the older diagrams.

You can save the diagrams (as a JPG-picture (in originial size)) on your PC with mouseclick (right button) and then choose "save image"...

The Elo-ratings of older Stockfish dev-versions in the Ordo-calculation can be a little different to the Elo-"dots" in the diagram, because the results/games of new Stockfish dev-versions - when getting part of the Ordo-calculation - can change the Elo-ratings of the opponent engines and that can change the Elo-ratings of older Stockfish dev-versions (in the Ordo-calculation / ratinglist, but not in the diagram, where all Elo-"dots" are the rating of one Stockfish dev-version at the moment, when the testrun of that Stockfish dev-version was finished).