Stefan Pohl Computer Chess

private website for chessengine-tests


Lc0 or other GPU-Neural Nets versus KomodoDragon 3 testing

 

Playing conditions:

 

Hardware: i7-8750H 2.6GHz (Hexacore) Notebook, RTX 2060 GPU, Windows 10 64bit, 16GB RAM

Cuda version installed: Cuda 11.6

Speed:  Because of my slow hardware and the facts, that (1) the nets of Lc0 getting bigger and bigger and (2) in TCEC the Leela-Ratio is meanwhile around 2.0, I decided not to slow down my RTX 2060 anymore and dont give Lc0 less thinking-time as before (where the Leela-Ratio was 1.0). This leads to a Leela-Ratio of around 2.0, which is an advantage for Lc0 (but my slow hardware is a disadvantage). Additionally I raised the time-increment to 2 seconds (instead of 1 second before), because Lc0 needs more time on a slow hardware and a big net. So the new thinking-time is 2min+2sec.

KomodoDragon plays with 11 Threads (=5.5 cores). Lc0 minibatchsize parameter is set to the best value for each netsize, depending on Lc0's benchmark with backendbench --clippy. For the most (bigger) nets, the best value is 30.

Hash / NN Cache: 2048 GB Hash for KomodoDragom / 1000000 NN-Cachesize for Lc0

GUICutechess-cli (GUI ends game, when a 5-piece endgame is on the board)

Tablebases: None for engines, 5 Syzygy for cutechess-cli

Openings: UHO_2022_6mvs_+109_+119.pgn. Download my UHO 2022 openings here

Ponder, Large Memory Pages & learning: Off

Thinking time: 2min+2sec. One 600 games-testrun takes around nearly 4 days. 

 

Each Lc0 / Neural Net plays 600 games vs. KomodoDragon 3 with my UHO 2022 openings

 

Learn more about Lc0 (getting started in a GUI, links to net-downloads, FAQs, development-informations and the Leela-Blog) here

 

Latest update: 2022/05/13: Lc0 0.29rc1 800815

 

Download all played games (games of the old test-setup, too): here

The results of the old test-setup can be seen here

     Program                          Elo    +    -   Games   Score   Av.Op.  Draws

   1 KomodoDragon 3 avx2            :    0    8    8  1800    56.3%    -44   55.5%
   2 Lc0 0.29rc1 782922 (20x512)    :   -8   16   16   600    48.8%      0   54.3%
   3 Lc0 0.29rc1 800815 (15x512)    :  -48   16   16   600    43.3%      0   56.2%
   4 Lc0 0.29rc1 791810 (15x192)    :  -77   16   16   600    39.2%      0   56.0%


Games        : 1800 (finished)

White Wins   : 788 (43.8 %)
Black Wins   : 13 (0.7 %)
Draws        : 999 (55.5 %)

 

Below the gamebase recalculated with my Gamepairs Rescorer Batch-Tool. Realizing Vondele's (Stockfish maintainer) idea: "Thinking uniquely in game pairs makes sense with the biased openings used these days. While pentanomial makes sense it is a bit complicated so we could simplify and score game pairs only (not games) as W-L-D (a traditional  score of 2-0, or 1.5-0.5 is just a W)."

   # PLAYER                       :  RATING  ERROR  PLAYED   W    D    L   Score
   1 KomodoDragon 3 avx2          :       0
   2 Lc0 0.29rc1 782922 (20x512)  :     -20    30     300    72  139   89  47.2%
   3 Lc0 0.29rc1 800815 (15x512)  :     -95    28     300    42  137  121  36.8%
   4 Lc0 0.29rc1 791810 (15x192)  :    -158    32     300    24  125  151  28.8%

 

You can download my Gamepairs Rescorer Tool right here

 

Mention, that this is not a ratinglist, but only a performance test of Lc0 with different NNs versus KomodoDragon. For a real ratinglist including Lc0 running on a RTX-GPU (with a valid Leela-Ratio of 1.0), please visit Andreas Strangmueller's excellent website. Just click here