Stefan Pohl Computer ChessHome of famous UHO openings and EAS RatinglistLc0 or other GPU-Neural Nets versus Stockfish 15.1 testing
The evaluation of the UHO 2024 openings started. NN-testing had to be suspended, because the PC is needed for the evaluation. Estimated time needed: around 75-80 days from today (2023/11/14), if all works without crashes or other problems...
Playing conditions:
Hardware: Ryzen 7 6800H 2.6GHz Notebook, RTX 3060 GPU, Windows 11 64bit, 32GB RAM Cuda version installed: Cuda 11.7 Speed: Stockfish 15.1 plays with 14 Threads (=7 cores) and reaches 10 MN/s in the middlegame. Lc0 minibatchsize parameter is set to the best value for each netsize, depending on Lc0's benchmark with backendbench --clippy. Hash: 2 GB Hash for Stockfish 15.1 / 8192 RamLimitMb for Lc0 GUI: Cutechess-cli (GUI ends game, when a 5-piece endgame is on the board) Tablebases: None for engines, 5 Syzygy for cutechess-cli Openings: UHO_2022_6mvs_+120_+129.pgn. Download my UHO 2022 openings here Ponder, Large Memory Pages & learning: Off Thinking time: 2min+2sec for Lc0 and 1min+1sec for Stockfish 15.1: I measured nps on my system and compared these values with the TCEC: My CPU is way too fast, compared with Lc0 running on my RTX 3060 GPU, so it makes sense to set the thinking-time of Stockfish to only 50% of the thinking-time of Lc0. For compensating the fast CPU and the fact, that in TCEC Lc0 benefits from fast hardware and long thinking-time (both is better for Lc0, not for Stockfish) One testrun takes around nearly 5 days. Average game-duration: 6min 45sec
Each Lc0 / Neural Net plays 1000 games vs. Stockfish 15.1 with my UHO 2022 openings
Learn more about Lc0 (getting started in a GUI, links to net-downloads, FAQs, development-informations and the Leela-Blog) here
Latest update: 2023/11/04: Lc0 0.31dev BT3-2860000 (small regression, compared to TCEC 25 SuFi-net (BT3-2860000 is the successor of this net))
Download all played games (games of the old test-setups, too): here Program Elo + - Games Score Av.Op. Draws 1 Stockfish 15.1 avx2 : 0 4 4 16000 58.6% -61 49.4%
White Wins : 8032 (50.2 %)
Below the gamebase recalculated with my Gamepairs Rescorer Batch-Tool. Realizing Vondele's (Stockfish maintainer) idea: "Thinking uniquely in game pairs makes sense with the biased openings used these days. While pentanomial makes sense it is a bit complicated so we could simplify and score game pairs only (not games) as W-L-D (a traditional score of 2-0, or 1.5-0.5 is just a W)." # PLAYER : RATING ERROR PLAYED W D L (%) CFS(%) ------------------------------------------------------------------- You can download my Gamepairs Rescorer Tool right here
Mention, that this is not a ratinglist, but only a performance test of Lc0 with different NNs versus Stockfish. For a real ratinglist including Lc0 running on a RTX-GPU (with a valid Leela-Ratio of 1.0), please visit Andreas Strangmueller's excellent website. Just click here
|