Stefan Pohl Computer Chess

Latest Website-News (2021/03/06): NN-testrun finished: Longtime testrun of Lc0 0.27.0 67741 vs. latest Stockfish-Dev (210226), using Noomen openings from TCEC SuFis: New "SuFi for the poor" -testrun. See the results and download the games in the "NN vs SF"- section (scroll down to the bottom of the site!). And watch some short and spectacular games directly on my website in the "View SF vs Lc0 games"- section - Enjoy!

Next AB-testrun: Stockfish with miniNNUE (7.5 MB) by pleomati.

Next NN-testruns: Ceres 0.89 67741, followed by Lc0 0.27.0 Phoenixstein 13.0 and Lc0 0.27.0 Phoenixstein 14.1.

Some 1000 games quicktests of nnue-nets (SF 210131 default vs. SF 210131 testnet). Latest update: 2021/03/07. Some of the best nets were retested with 3x longer thinkingtime. Look at the results here

Direct download of the strongest net so far (3x longer time) is here

Stay tuned.

Stockfish testing

Playing conditions:

Hardware: Since 20/07/21 AMD Ryzen 3900 12-core (24 threads) notebook with 32GB RAM. Now, 20 games are played simultaneously (!), so from now, each testrun will have 6000 or 7000 games (instead of 5000 before) and will take only 2 days, not 6-7 days as before! From now, all engine-binaries are popcount/avx2, of course, because bmi2-compiles are extremly slow on AMD. To keep the rating-list engine-names consistent, the "bmi2"- or "pext"-extension in the engine-name is still in use for older engines - otherwise ORDO will not calculate all played games by this engine as one engine...

Speed: (singlethread, TurboBoost-mode switched off, chess starting position) Stockfish: 1.3 mn/s, Komodo: 1.1 mn/s

Hash: 256MB per engine

GUI: Cutechess-cli (GUI ends game, when a 5-piece endgame is on the board)

Tablebases: None for engines, 5 Syzygy for cutechess-cli

Openings: HERT_500 testset (by Thomas Zipproth) (download the file at the "Download & Links"-section or here)

Ponder, Large Memory Pages & learning: Off

Thinking time: 180''+1000ms (= 3'+1'') per game/engine (average game-duration: around  7.5 minutes). One 7000 games-testrun takes about 2 days.The version-numbers of the Stockfish engines are the date of the latest patch, which was included in the Stockfish sourcecode, not the release-date of the engine-file, written backwards (year,month,day))(example: 200807 = August, 7, 2020). The used SF compile is the AVX2-compile, which is the fastest on my AMD Ryzen CPU. SF binaries are taken from abrok.eu (except the official SF-release versions, which are taken form the official Stockfish website).

Download BrainFish (and the Cerebellum-Libraries): here

To avoid distortions in the Ordo Elo-calculation, from now, only 3x Stockfish (latest official release + the latest 2 dev-versions) and 1x Brainfish are stored in the gamebase (all older engine-versions games will be deleted, every time, when a new version was tested). Stockfish and BrainFish older Elo-results can still be seen in the Elo-diagrams below. BrainFish plays always with the latest Cerebellum-Libraries of course, because otherwise BrainFish = Stockfish.

Latest update: 2021/03/01: Stockfish 210226 (+5 Elo to Stockfish 13)

(Ordo-calculation fixed to Stockfish 13 = 3723 Elo)

See the individual statistics of engine-results here

Download the current gamebase here

     Program                      Elo    +    -   Games   Score   Av.Op.  Draws

   1 Stockfish 210226 avx2      : 3728    8    8  7000    76.6 %   3500   44.9 %
   2 CFish 12 3xCerebellum      : 3725    8    8  7000    86.1 %   3387   27.3 %
   3 Stockfish 210111 avx2      : 3724    7    7  7000    78.0 %   3479   42.5 %
   4 Stockfish 13 210218        : 3723    6    6 10000    72.8 %   3526   51.0 %
   5 SF Fat Fritz 2 avx2        : 3718    7    7  8000    73.7 %   3513   49.3 %
   6 CFish 12 avx2              : 3702    8    8  7000    84.6 %   3387   29.1 %
   7 Stockfish 12 200902        : 3683    4    4 28000    76.9 %   3446   41.4 %
   8 SF Fat Fritz 2 github      : 3677    7    7  7000    73.3 %   3483   48.5 %
   9 KomodoDragon 1.0 avx2      : 3647    5    5 19000    69.4 %   3479   47.5 %
  10 SF 200910 miniNNue avx2    : 3614    7    7  7000    72.1 %   3435   43.2 %
  11 Stockfish 200731 popc      : 3600    7    7  7000    80.5 %   3344   36.2 %
  12 Stockfish 11 200118        : 3563    5    5 17000    69.5 %   3401   41.6 %
  13 Stockfish 10 181129        : 3524    5    5 15000    78.5 %   3287   37.7 %
  14 KomodoDragon 1.0 MCTS      : 3479    6    6  7000    57.6 %   3425   57.1 %
  15 Stockfish 9 180201         : 3474    8    8  5000    74.9 %   3272   41.7 %
  16 Komodo 14.1 x64            : 3453    6    6  8000    56.3 %   3409   55.6 %
  17 Komodo 14 bmi2             : 3443    4    4 19000    51.3 %   3438   51.6 %
  18 Houdini 6 pext             : 3439    2    2 56000    55.0 %   3404   46.9 %
  19 Nemorino 6.00 avx2         : 3438    4    4 25000    46.1 %   3476   50.4 %
  20 Komodo 13.3 bmi2           : 3437    6    6  8000    62.8 %   3341   49.9 %
  21 Fire 8 popc                : 3430    6    6  8000    39.7 %   3519   50.8 %
  22 Komodo 13.1 bmi2           : 3424    5    5 11000    62.0 %   3333   48.8 %
  23 Slow Chess 2.5 avx2        : 3422    5    5 15000    37.0 %   3535   46.0 %
  24 Komodo 12.3 bmi2           : 3411    7    7  7000    62.7 %   3313   49.4 %
  25 Nemorino 6.05 avx2         : 3406    7    7  7000    42.8 %   3467   51.0 %
  26 Ethereal 12.75 avx2        : 3397    4    4 25000    40.8 %   3478   49.2 %
  27 Ethereal 12.62 avx2        : 3389    6    6  8000    49.1 %   3401   54.6 %
  28 Slow Chess 2.4 popc        : 3373    5    5 12000    43.7 %   3426   52.3 %
  29 Ethereal 12.50 popc        : 3356    6    6  7000    45.8 %   3395   55.2 %
  30 RubiChess 2.0 avx2         : 3355    6    6 12000    30.2 %   3526   45.5 %
  31 Slow Chess 2.3 popc        : 3343    4    4 14000    42.3 %   3406   52.2 %
  32 Komodo 14 MCTS             : 3339    7    7  5000    44.4 %   3384   53.4 %
  33 Ethereal 12.25 pext        : 3337    5    5 12000    35.2 %   3470   46.4 %
  34 Pedone 3 avx2              : 3329    6    6  8000    33.4 %   3466   46.7 %
  35 Slow Chess 2.2 popc        : 3328    6    6 11000    32.9 %   3482   42.7 %
  36 Igel 2.9.0 popavx2         : 3328    5    5 12000    32.5 %   3474   47.5 %
  37 RubiChess 1.9dev nnue      : 3319    6    6  8000    31.8 %   3470   45.6 %
  38 Ethereal 12.00 pext        : 3316    5    5  9000    43.1 %   3369   50.8 %
  39 Ethereal 11.75 pext        : 3308    6    6  9000    39.3 %   3391   53.2 %
  40 Xiphos 0.6 bmi2            : 3302    3    3 31000    36.2 %   3420   48.9 %
  41 Fire 7.1 popc              : 3300    3    3 41000    41.9 %   3371   50.7 %
  42 Xiphos 0.5.6 bmi2          : 3287    7    7  7000    41.2 %   3355   54.6 %
  43 Minic 2.51 nasc_nutr       : 3282    6    6  7000    31.2 %   3435   45.0 %
  44 Ethereal 11.53 pext        : 3280    6    6  7000    42.2 %   3341   53.4 %
  45 Komodo 12.3 MCTS           : 3275    7    7  7000    42.7 %   3333   46.3 %
  46 Ethereal 11.25 pext        : 3270    8    8  6000    38.4 %   3361   51.0 %
  47 rofChade 2.3 bmi2          : 3256    6    6 11000    33.8 %   3387   47.5 %
  48 Booot 6.4 popc             : 3243    7    7  6000    31.1 %   3393   46.5 %
  49 Schooner 2.2 popc          : 3241    7    7  6000    31.3 %   3389   50.3 %
  50 Laser 1.7 bmi2             : 3217    7    7  6000    30.8 %   3370   45.8 %
  51 Fizbo 2 bmi2               : 3212    8    8  5000    36.0 %   3324   39.0 %
  52 Fritz 17                   : 3211    7    7  6000    29.4 %   3376   44.2 %
  53 Shredder 13 x64            : 3209    8    8  6000    31.9 %   3358   42.6 %
  54 RubiChess 1.8 popc         : 3207    6    6  7000    32.0 %   3344   46.1 %
  55 Defenchess 2.2 popc        : 3204    8    8  5000    26.6 %   3393   41.8 %
  56 Booot 6.3.1 popc           : 3199    8    8  5000    34.0 %   3327   44.1 %
  57 Andscacs 0.95 popc         : 3167    9    9  5000    23.1 %   3390   35.4 %

The version-numbers (180622 for example) of the engines are the date of the latest patch, which was included in the Stockfish sourcecode, not the release-date of the engine-file. Especially the asmFish-engines are often released much later!!

Below you find a diagram of the progress of Stockfish in my tests since August 2020. 

And below that diagram, the older diagrams.

You can save the diagrams (as a JPG-picture (in originial size)) on your PC with mouseclick (right button) and then choose "save image"...

The Elo-ratings of older Stockfish dev-versions in the Ordo-calculation can be a little different to the Elo-"dots" in the diagram, because the results/games of new Stockfish dev-versions - when getting part of the Ordo-calculation - can change the Elo-ratings of the opponent engines and that can change the Elo-ratings of older Stockfish dev-versions (in the Ordo-calculation / ratinglist, but not in the diagram, where all Elo-"dots" are the rating of one Stockfish dev-version at the moment, when the testrun of that Stockfish dev-version was finished).