Stefan Pohl Computer Chess

Latest Website-News (2018/11/16): Testrun of Stockfish 181108 finished - no progress. Next testrun: Komodo 12.2.2 MCTS. It will play vs. 8 opponents (instead of 5), because it is the first MCTS-engine in my testruns. So, the testrun will take around 10 days...

The long thinking-time testruns of LC Zero are suspended, because the neural-net learning of LC Zero was restarted and until now, the new 20xxx nets are much weaker, than the latest 10xxx nets. My testruns will continue, when the new nets are getting as strong as the older 10xxx nets. I will do some fast testruns of LC Zero vs. Stockfish from time to time, to measure the progress of the 20xxx nets beyond pure selfplay-Elo.

My SALC V5 openings and books are ready for download. Check out the "SALC openings"-section on this website for further information. Download SALC V5.02 here

Stay tuned.

Stockfish testing

Playing conditions:

Hardware: i7-6700HQ 2.6GHz Notebook (Skylake CPU), Windows 10 64bit, 8GB RAM

Fritzmark: singlecore: 5.3 / 2521 (all engines running on one core, only), average meganodes/s displayed by LittleBlitzerGUI: Houdini: 2.6 mn/s, Stockfish: 2.2 mn/s, Komodo: 2.0 mn/s

Hash: 512MB per engine

GUI: LittleBlitzerGUI (draw at 130 moves, resign at 400cp (for 4 moves))

Tablebases: None

Openings: HERT testset (by Thomas Zipproth) (download the file at the "Download & Links"-section or here)(I use a version of HERT, where the positions in the file are ordered in a different way - makes no difference for testing-results, dont be confused, when you download my gamebase-file and the game-sequence doesnt match with the sequence of your HERT-set...)

Ponder, Large Memory Pages & learning: Off

Thinking time: 180''+1000ms (= 3'+1'') per game/engine (average game-duration: around  7.5 minutes). One 5000 games-testrun takes about 7 days.The version-numbers of the Stockfish engines are the date of the latest patch, which was included in the Stockfish sourcecode, not the release-date of the engine-file, written backwards (year,month,day))(example: 170526 = May, 26, 2017). Since July, 2018 I use the abrok-compiles of Stockfish again (http://abrok.eu/stockfish), because they are now much faster than before - now only 1.3% slower than BrainFish-compiles. So, there is no reason anymore to not use these "official" development-compiles.

Download BrainFish (and the Cerebellum-Library): here

Each Stockfish-version plays 1000 games versus Komodo 12, Houdini 6, Fire 7.1, Ethereal 11, Fizbo 2. All engines are running with default-settings, except: Move Overhead is set to 300ms, if an engine allows to do so. And for asmBrainFish, the option BestBookMove is set to true (default=false), for best performance, using Cerebellum-Library.

To avoid distortions in the Ordo Elo-calculation, from now, only 2x Stockfish (latest official release + the latest version) and 1x asmFish and 1x Brainfish are stored in the gamebase (all older engine-versions games will be deleted, every time, when a new version was tested). Stockfish, asmFish and BrainFish older Elo-results can still be seen in the Elo-diagrams below. BrainFish plays always with the latest Cerebellum-Library of course, because otherwise BrainFish = Stockfish.

Latest update: 2018/11/16: Stockfish 181108

(Ordo-calculation fixed to Stockfish 9 = 3450 Elo)

See the individual statistics of engine-results here

Download the current gamebase here

Download the archive (all played games with HERT (289000 games)) here

See a ORDO-rating of the complete HERT-archive-base here

     Program                    Elo    +    -   Games   Score   Av.Op.  Draws

   1 asmBrainFish 180510      : 3530    8    8  5000    77.9 %   3294   39.3 %
   2 BrainFish 180728 bmi2    : 3530    8    8  5000    77.8 %   3294   39.0 %
   3 Stockfish 181108 bmi2    : 3497    8    8  5000    74.0 %   3303   44.1 % (new)
   4 asmFish 180503 bmi2      : 3481    8    8  5000    73.0 %   3294   44.1 %
   5 Stockfish 9 180201       : 3450    6    6  9000    75.2 %   3243   40.6 %
   6 Houdini 6 pext           : 3423    4    4 13000    61.5 %   3327   48.8 %
   7 Komodo 12 bmi2           : 3394    4    4 13000    57.8 %   3330   49.5 %
   8 Fire 7.1 popc            : 3279    4    4 13000    42.8 %   3338   47.6 %
   9 Ethereal 11 pext         : 3233    6    6  8000    38.7 %   3323   47.4 %
  10 Fizbo 2 bmi2             : 3188    5    5 13000    31.5 %   3345   35.8 %
  11 Shredder 13 x64          : 3188    5    5 11000    31.1 %   3348   40.9 %
  12 Booot 6.3.1 popc         : 3177    7    7  6000    32.1 %   3320   42.9 %
  13 Xiphos 0.4.6 bmi2        : 3168    7    7  6000    30.0 %   3328   41.6 %
  14 Andscacs 0.94 popc       : 3141    7    7  6000    28.1 %   3320   37.4 %

The 4 different Fishes in this Elo-list:

- Stockfish

- asmFish = Stockfish manually rewritten in assembler (look here)

- BrainFish = Stockfish playing with Cerebellum-Library by Thomas Zipproth (look here)

- asmBrainFish = asmFish playing with Cerebellum-Library (UCI-option BestBookMove set to true)

The version-numbers (180622 for example) of the engines are the date of the latest patch, which was included in the Stockfish sourcecode, not the release-date of the engine-file. Especially the asmFish-engines are often released much later!!

Below you find a diagram of the progress of Stockfish in my tests since the end of 2016. 

And below that diagram, the older diagrams.

You can save the diagrams (as a JPG-picture (in originial size)) on your PC with mouseclick (right button) and then choose "save image"...

The Elo-ratings of older Stockfish dev-versions in the Ordo-calculation can be a little different to the Elo-"dots" in the diagram, because the results/games of new Stockfish dev-versions - when getting part of the Ordo-calculation - can change the Elo-ratings of the opponent engines and that can change the Elo-ratings of older Stockfish dev-versions (in the Ordo-calculation / ratinglist, but not in the diagram, where all Elo-"dots" are the rating of one Stockfish dev-version at the moment, when the testrun of that Stockfish dev-version was finished).