Stefan Pohl Computer Chess

private website for chessengine-tests

Lc0 or other GPU-Neural Nets versus KomodoDragon 3 testing

Playing conditions:

Hardware: i7-8750H 2.6GHz (Hexacore) Notebook, RTX 2060 GPU, Windows 10 64bit, 16GB RAM

Cuda version installed: Cuda 11.7

Speed: Because of my slow hardware and the facts, that (1) the nets of Lc0 getting bigger and bigger and (2) in TCEC the Leela-Ratio is meanwhile around 2.0, I decided not to slow down my RTX 2060 anymore and dont give Lc0 less thinking-time as before (where the Leela-Ratio was 1.0). This leads to a Leela-Ratio of around 2.0, which is an advantage for Lc0 (but my slow hardware is a disadvantage). Additionally I raised the time-increment to 2 seconds (instead of 1 second before), because Lc0 needs more time on a slow hardware and a big net. So the new thinking-time is 2min+2sec.

KomodoDragon plays with 11 Threads (=5.5 cores). Lc0 minibatchsize parameter is set to the best value for each netsize, depending on Lc0's benchmark with backendbench --clippy. For the most (bigger) nets, the best value is 30.

Hash / NN Cache: 2048 GB Hash for KomodoDragom / 1000000 NN-Cachesize for Lc0

GUI: Cutechess-cli (GUI ends game, when a 5-piece endgame is on the board)

Tablebases: None for engines, 5 Syzygy for cutechess-cli

Openings: UHO_2022_6mvs_+110_+119.pgn. Download my UHO 2022 openings here

Ponder, Large Memory Pages & learning: Off

Thinking time: 2min+2sec. One testrun takes around nearly 4 days. Average game-duration: 9min 11sec

Each Lc0 / Neural Net plays 600 games vs. KomodoDragon 3 with my UHO 2022 openings

Learn more about Lc0 (getting started in a GUI, links to net-downloads, FAQs, development-informations and the Leela-Blog) here

Latest update: 2023/03/11: Lc0 0.30dev 809983

Download latest Lc0 dev binaries here

Download all played games (games of the old test-setup, too): here

The results of the old test-setup can be seen here

Program Elo + - Games Score Av.Op. Draws

1 Lc0 0.30dev BT2-3250 (15x768) : 22 20 20 600 53.2% 0 56.7% 2 Lc0 0.30dev 784968 (20x512) : 16 19 19 600 52.3% 0 58.3% 3 Lc0 0.30dev 809983 (19x512) : 16 20 20 600 52.3% 0 55.0% 4 Lc0 0.30dev 806012 (15x512) : 16 18 18 600 52.3% 0 63.2% 5 Lc0 0.30dev 784010 (20x512) : 13 20 20 600 51.8% 0 58.3% 6 Lc0 0.30dev BT2 (15x768) : 12 20 20 600 51.8% 0 57.8% 7 Lc0 0.30dev 805123 (15x512) : 9 19 19 600 51.3% 0 61.0% 8 Lc0 0.30dev 784765 (20x512) : 9 18 18 600 51.3% 0 59.7% 9 KomodoDragon 3 avx2 : 0 4 4 13800 52.2% -15 57.3% 10 Lc0 0.29rc1 783467 (20x512) : -5 20 20 600 49.3% 0 57.0% 11 Lc0 0.30dev 803001 (15x512) : -7 19 19 600 49.0% 0 58.3% 12 Lc0 0.29rc1 782922 (20x512) : -8 19 19 600 48.8% 0 54.3% 13 Lc0 0.29.0 807407 (15x512) : -11 19 19 600 48.5% 0 55.0% 14 Lc0 0.30dev 806488 (15x512) : -11 19 19 600 48.4% 0 58.2% 15 Lc0 0.30dev 803399 (15x512) : -13 19 19 600 48.2% 0 60.0% 16 Lc0 0.30dev 802177 (15x512) : -16 18 18 600 47.8% 0 57.8% 17 Lc0 0.30dev 804287 (15x512) : -22 19 19 600 46.9% 0 57.5% 18 Lc0 0.29rc1 801237 (15x512) : -27 19 19 600 46.2% 0 58.3% 19 Lc0 0.29rc1 611246 (30x384) : -29 20 20 600 45.9% 0 57.5% 20 Lc0 0.29rc1 801663 (15x512) : -41 20 20 600 44.2% 0 56.0% 21 Lc0 0.29rc1 800815 (15x512) : -48 19 19 600 43.3% 0 56.2% 22 Lc0 0.29rc1 791921 (15x192) : -64 20 20 600 40.9% 0 52.8% 23 Lc0 0.29rc1 791810 (15x192) : -77 19 19 600 39.2% 0 56.0% 24 Lc0 0.29rc1 606511 (24x320) : -91 20 20 600 37.3% 0 53.3%

Games : 13800 (finished)

White Wins : 5805 (42.1 %) Black Wins : 85 (0.6 %) Draws : 7910 (57.3 %)

Below the gamebase recalculated with my Gamepairs Rescorer Batch-Tool. Realizing Vondele's (Stockfish maintainer) idea: "Thinking uniquely in game pairs makes sense with the biased openings used these days. While pentanomial makes sense it is a bit complicated so we could simplify and score game pairs only (not games) as W-L-D (a traditional score of 2-0, or 1.5-0.5 is just a W)."

# PLAYER : RATING ERROR PLAYED W D L Score 1 Lc0 0.30dev BT2-3250 (15x768): 42 30 300 82 172 46 56.0% 2 Lc0 0.30dev 809983 (19x512) : 32 29 300 83 161 56 54.5% 3 Lc0 0.30dev 806012 (15x512) : 32 30 300 76 175 49 54.5% 4 Lc0 0.30dev 784968 (20x512) : 29 30 300 82 161 57 54.2% 5 Lc0 0.30dev 784010 (20x512) : 26 30 300 80 162 58 53.7% 6 Lc0 0.30dev BT2 (15x768) : 25 30 300 80 161 59 53.5% 7 Lc0 0.30dev 784765 (20x512) : 20 27 300 76 165 59 52.8% 8 Lc0 0.30dev 805123 (15x512) : 19 30 300 74 168 58 52.7% 9 KomodoDragon 3 avx2 : 0 10 Lc0 0.29rc1 783467 (20x512) : -11 30 300 72 147 81 48.5% 11 Lc0 0.30dev 803001 (15x512) : -15 30 300 59 169 72 47.8% 12 Lc0 0.29rc1 782922 (20x512) : -20 30 300 72 139 89 47.2% 13 Lc0 0.29.0 807407 (15x512) : -22 27 300 62 157 81 46.8% 14 Lc0 0.30dev 806488 (15x512) : -22 30 300 61 159 80 46.8% 15 Lc0 0.30dev 803399 (15x512) : -26 30 300 55 168 77 46.3% 16 Lc0 0.30dev 802177 (15x512) : -32 30 300 63 147 90 45.5% 17 Lc0 0.30dev 804287 (15x512) : -42 28 300 50 164 86 44.0% 18 Lc0 0.29rc1 801237 (15x512) : -53 28 300 43 169 88 42.5% 19 Lc0 0.29rc1 611246 (30x384) : -55 28 300 43 167 90 42.2% 20 Lc0 0.29rc1 801663 (15x512) : -82 29 300 44 143 113 38.5% 21 Lc0 0.29rc1 800815 (15x512) : -95 28 300 42 137 121 36.8% 22 Lc0 0.29rc1 791921 (15x192) : -133 32 300 25 141 134 31.8% 23 Lc0 0.29rc1 791810 (15x192) : -158 32 300 24 125 151 28.8% 24 Lc0 0.29rc1 606511 (24x320) : -185 32 300 20 115 165 25.8%

You can download my Gamepairs Rescorer Tool right here

Mention, that this is not a ratinglist, but only a performance test of Lc0 with different NNs versus KomodoDragon. For a real ratinglist including Lc0 running on a RTX-GPU (with a valid Leela-Ratio of 1.0), please visit Andreas Strangmueller's excellent website. Just click here

Datenschutzerklärung