Stefan Pohl Computer Chess

private website for chessengine-tests


LC0 / Neural Nets versus Stockfish testing

 

Playing conditions:

 

Hardware: i7-8750H 2.6GHz (Hexacore) Notebook, RTX 2060 GPU, Windows 10 64bit, 16GB RAM

Speed:  Stockfish (running on 11 hyperthreading-threads, Intel Turbo-Mode off): 9000 kn/s, Lc0 (with old 32930 20x256 net): 16000 n/s in starting position. Since Lc0 0.26.3, Lc0 uses Cuda 11.1, which makes it around +37% faster. To keep the Leela-Ratio on 1.0, I slowed down the GPU with the Afterburner-Tool.

Hash / NN Cache: 4096 GB Hash for Stockfish / 5000000 NN-Cachesize for Lc0

GUICutechess-cli (GUI ends game, when a 5-piece endgame is on the board)

Tablebases: None for engines, 5 Syzygy for cutechess-cli

Openings: Unbalanced Human Openings V3 6moves with eval [+0.90;+0.99]. Learn more in the "Anti Draw Openings"- section.

Ponder, Large Memory Pages & learning: Off

Thinking time: Lc0 2'+1'' and Stockfish 3'+1.5'' (means a perfect Leela-Ratio of 1.0). Average game-duration: 8 minutes, one 750 games-testrun takes around 4 days. 

 

Each Lc0 / Neural Net plays 750 games vs. Stockfish 13 with my new Unbalanced Human Openings V3

 

Learn more about Lc0 (getting started in a GUI, links to net-downloads, FAQs, development-informations and the Leela-Blog) here

 

Latest update: 2021/06/23: Lc0 0.27.0 744204

 

Download all played games (games of the old test-setup, too): here

The results of the old test-setup can be seen here

     Program                            Elo    +    -   Games   Score   Av.Op.  Draws

   1 Stockfish 13 bmi2                : 3723    5    5  8250    67.2 %   3595   50.3 %
   2 Lc0 0.27.0 68695 (30x384)        : 3661   17   17   750    41.2 %   3723   56.8 %
   3 Lc0 0.27.0 69363 (30x384)        : 3648   17   17   750    39.4 %   3723   52.9 %
   4 Lc0 0.27.0 69146 (30x384)        : 3648   17   17   750    39.4 %   3723   54.0 %
   5 Lc0 0.27.0 J94-100 (30x384)      : 3646   16   16   750    39.1 %   3723   51.3 %
   6 Lc0 0.27.0 LS 15 (20x256)        : 3597   18   18   750    32.8 %   3723   50.4 %
   7 Lc0 0.27.0 751236 (15x192)       : 3575   18   18   750    30.1 %   3723   49.2 %
   8 Lc0 0.27.0 J104.1-30 (10x128)    : 3565   19   19   750    28.9 %   3723   49.2 %
   9 Lc0 0.27.0 743947 (10x128)       : 3558   19   19   750    28.1 %   3723   48.4 %
  10 Lc0 0.27.0 744204 (10x128)       : 3555   19   19   750    27.7 %   3723   46.9 %
  11 Lc0 0.27.0 743665 (10x128)       : 3552   19   19   750    27.3 %   3723   48.0 %
  12 Lc0 0.27.0 703810 (10x128)       : 3546   19   19   750    26.7 %   3723   46.0 %


Games        : 8250 (finished)

White Wins   : 3864 (46.8 %)
Black Wins   : 237 (2.9 %)
Draws        : 4149 (50.3 %)

 

Mention, that this is not a ratinglist, but only a performance test of Lc0 with different NNs versus Stockfish. Because Lc0 vs. Stockfish is definitly the most interesting head-to-head competition of NN vs. AB-engines. For a real ratinglist including Lc0 running on a RTX-GPU (with a valid Leela-Ratio of 1.0), please visit Andreas Strangmueller's excellent website. Just click here


 

Stockfish vs Lc0 longtime testing ("SuFi for the poor")

 

Each testrun 300 games with 150 Noomen lowdraw-openings (selected openings from TCEC superfinals) and 5'+3'' thinking-time (Lc0) / 7.5'+4.5'' (Stockfish). This thinking-time gives a perfect Leela-Ratio of 1.0 on the used PC hardware: i7-8750H 2.6GHz (Hexacore, TurboBoost mode off) Notebook, RTX 2060 GPU. Average game-duration: 20 minutes. Stockfish (running on 11 hyperthreading-threads, Intel Turbo-Mode off) 9000 kn/s, Lc0 (with old 32930 20x256 net) on RTX 2060 mobile: 16000 n/s in starting position.

Hash / NN Cache: 4096 GB Hash for Stockfish / 10000000 NN-Cachesize for Lc0

GUICutechess-cli (GUI ends game, when a 5-piece endgame is on the board)

Tablebases: None for engines, 5 Syzygy for cutechess-cli

Openings: 150 Noomen lowdraws openings (J. Noomen selected non-drawish openings out of his TCEC superfinal openings of previous TCEC seasons). Download here

Ponder, Large Memory Pages & learning: Off

Thinking timeLc0 5'+3'' and Stockfish 7.5'+4.5'' (means a perfect Leela-Ratio of 1.0). Average game-duration: 20 minutes.

 

Download all played games here

 

Latest update: 2021/04/28 Stockfish 210415 vs Lc0 0.27.0 68695

See some short and spectacular wins of this match directly here on the website in the "View SF vs Lc0 games"- section!

Stockfish 210415 bmi2 vs Lc0 0.27.0 68695     : 300 (+ 62,=218,- 20), 57.0 % (+49 Elo)
Stockfish 210406 bmi2 vs Lc0 0.27.0 68002     : 300 (+ 60,=219,- 21), 56.5 % (+45 Elo)
Stockfish 210226 bmi2 vs Lc0 0.27.0 67741     : 300 (+ 75,=205,- 20), 59.2 % (+65 Elo)
Stockfish 201225 bmi2 vs Lc0 0.26.3 66680     : 300 (+ 60,=223,- 17), 57.2 % (+50 Elo)
Stockfish 201022 bmi2 vs Lc0 0.26.3 J92-260   : 300 (+ 75,=207,- 18), 59.5 % (+67 Elo)
Stockfish 200928 bmi2 vs Lc0 0.26.3rc2 J92-190: 300 (+ 68,=215,- 17), 58.5 % (+60 Elo)
Stockfish 12 bmi2 vs Lc0 0.26.2 J92-130:        300 (+ 74,=203,- 23), 58.5 % (+60 Elo)
SF 200823 82215d0fd0df vs Lc0 0.26.1 t60-4619:  300 (+ 85,=199,- 16), 61.5 % (+82 Elo)
SF 200810 112bb1c8cdb5 vs Lc0 0.26.1 LS 15:     300 (+ 78,=196,- 26), 58.7 % (+62 Elo)