Stefan Pohl Computer Chess

NBSC Advanced Armageddon Openings for Computerchess

- the future of Computerchess (Part 2)

 

No Black Short Castling advanced scoring Armageddon openings

 

 

No longer any draws in computerchess !!! No boring draw-marathons, no draw-death of computerchess – never!

 

Download NBSC Advanced Armageddon openings here

 

Base idea is from Larry Kaufman (Komodo-Team):

The first idea for Armageddon openings for computerchess was, to allow white to castle short, only and black to castle long, only. This so called SALC-Armageddon worked pretty well in my testruns. But, some months ago, I had the idea for a new advanced scoring system for Armageddon computerchess:

 

Classical Armageddon scoring is:

Win for white = 1 point for white

Draw = 1 point for black

Win for black = 1 point for black

 

My Advanced Armageddon scoring is:

Win for white = 1 point for white

Draw = 1 point for black

Win for black = 2 points for black (!!!)

 

Because the chessengines dont know, that they are playing Armageddon-Chess, I believe, it makes sense, to use the advanced scoring: The engines try to win a game, even though they play with black and in the classical Armageddon scoring, there is no difference between a win for black and a draw. So, there is no benefit for the engine, to win a game, when playing black. With my advanced Armageddon-scoring, black gets 2 points for winning a game, because the Armageddon-openings give a measureable advantage for white and it is very difficult for black to win a game. So, trying to win a game even though the engine plays black, makes sense with the advanced scoring, because black gets 2 points for a win. With the classical Armageddon scoring, black does not benefit from winning a game.

It is clear, that the advanced scoring-system lowers the whitescore-level. And in SALC-Armageddon, the whitescore was not very high even though the classical Armageddon scoring is used. So, I decided to give a white a higher advantage and allow white to castle long and short. Black is still allowed to castle long. This „NBSC“ (=No Black Short Castling) Armageddon gives white more advantage and increases the whitescore.

The NBSC line is 1. Na3 Nh6 2. Nb1 Rg8 3. Nc3 Rh8 4. Nb1 Ng8

 

I built 3 NBSC-Armageddon opening-sets (and some small tools for classical and advanced Armageddon scoring):

 

NBSC_Noomen_3mvs: Built from the Noomen 3mvs opening set (deleted double endpositions and opening-lines, which are not good for computerchess IMHO (Dutch, King's gambit etc). 250 lines remaining.

 

NBSC_SuperGM_4mvs: 2612 lines. Both players 2600+ Elo. Out of the Megabase 2018. All endpositions calculated by Komodo 13.1 (7 hyperthreading Threads on a Quadcore, 30''/move). All endpositions in an eval interval of [-0.40/+0.60] (before the NBSC-line was added, of course).

 

NBSC_30k_5mvs: 30578 lines. Both players 2300+ Elo. Out of the Megabase 2018. All endpositions calculated by Komodo 13.1 (7 hyperthreading Threads on a Quadcore, 20''/move). All endpositions in an eval interval of [-0.40/+0.60] (before the NBSC-line was added, of course).

 

I recommend to use the smallest of the 3 opening sets, which is big enough for your engine play or testing.

 

The Book-folder contains openings-books for FritzGUI, ArenaGUI and polylgot-book format (the ShredderGUI has a serious bug in its opening-book using: It does not recognize, that the 2 move sequence of the rook (from h8 to g8 and back to h8) makes the short castling for black illegal. So, an opening-book built from NBSC-lines will not work in the ShredderGUI)

 

Here the results of the testruns (the results of NBSC Armageddon openings are just mind-blowing!):

 

Testing conditions:

2'+1'', Singlethread, i7-8750H Hexacore mobile CPU, 256MB Hash, cutechess-cli (no TB for engines, but 5 Syzygy for cutechess), Contempt=0 for all Stockfish. All openings replayed with reversed colors. Round Robin with 1500 games with SF 11, SF 10, SF 9 and SF 8. Each SF played 250 games vs. Each of the 3 opponents = 1500 games per testrun. ORDO for the ratings (3400 Elo base-value)

 

8 classical (non-Armageddon) opening-sets for comparsion:

Balsa_2724, Chad 8ply, Hert500, Stockfish Framework 8moves v3, SuperGM 4moves, SALC V5 fc 500, Drawkiller tournament and J.Noomens "action" openings (TCEC, Kings Indian and Gambit-lines in one file). Sorted by Elo-spreading (Elo-distance between SF 11 and SF 8 (first to last)).

 

Balsa 2724:

     Program                Elo    +    -   Games   Score   Av.Op.  Draws
   1 Stockfish 11 bmi2    : 3455   14   14   750    60.3 %   3382   72.8 %
   2 Stockfish 10 bmi2    : 3430   13   13   750    55.5 %   3390   76.7 %
   3 Stockfish 9 bmi2     : 3379   14   14   750    46.1 %   3407   74.1 %
   4 Stockfish 8 bmi2     : 3336   14   14   750    38.1 %   3421   67.9 %

Draw-rate                    : 72.9 % (smaller is better)
Elo spreading (first to last): 119 Elo (bigger is better)

 

SuperGM 4moves:

     Program                Elo    +    -   Games   Score   Av.Op.  Draws
   1 Stockfish 11 bmi2    : 3459   14   14   750    60.9 %   3380   69.2 %
   2 Stockfish 10 bmi2    : 3436   13   13   750    56.7 %   3388   70.7 %
   3 Stockfish 9 bmi2     : 3382   14   14   750    46.6 %   3406   68.9 %
   4 Stockfish 8 bmi2     : 3323   14   14   750    35.9 %   3426   61.1 %

Draw-rate                    : 67.5 % (smaller is better)
Elo spreading (first to last): 136 Elo (bigger is better)

 

Chad 8ply:

     Program                Elo    +    -   Games   Score   Av.Op.  Draws
   1 Stockfish 11 bmi2    : 3467   14   14   750    62.3 %   3378   66.0 %
   2 Stockfish 10 bmi2    : 3431   13   13   750    55.7 %   3390   71.5 %
   3 Stockfish 9 bmi2     : 3376   14   14   750    45.6 %   3408   69.3 %
   4 Stockfish 8 bmi2     : 3325   14   14   750    36.3 %   3425   62.5 %

Draw-rate                    : 67.3 % (smaller is better)
Elo spreading (first to last): 142 Elo (bigger is better)

 

 

Stockfish Framework 8moves v3:

     Program                Elo    +    -   Games   Score   Av.Op.  Draws
   1 Stockfish 11 bmi2    : 3466   14   14   750    62.1 %   3378   68.7 %
   2 Stockfish 10 bmi2    : 3431   13   13   750    55.8 %   3390   75.6 %
   3 Stockfish 9 bmi2     : 3383   14   14   750    46.7 %   3406   71.3 %
   4 Stockfish 8 bmi2     : 3320   14   14   750    35.4 %   3427   63.9 %

Draw-rate                    : 69.9 % (smaller is better)
Elo spreading (first to last): 146 Elo (bigger is better)

 

 

Hert 500:

     Program                Elo    +    -   Games   Score   Av.Op.  Draws
   1 Stockfish 11 bmi2    : 3473   14   14   750    63.3 %   3376   67.7 %
   2 Stockfish 10 bmi2    : 3429   14   14   750    55.3 %   3390   69.6 %
   3 Stockfish 9 bmi2     : 3377   14   14   750    45.7 %   3408   69.3 %
   4 Stockfish 8 bmi2     : 3321   14   14   750    35.6 %   3426   62.9 %

Draw-rate                    : 67.4 % (smaller is better)
Elo spreading (first to last): 152 Elo (bigger is better)

 

J.Noomen "action-mix" openings (TCEC, Kings Indian, Gambit-lines):

     Program                Elo    +    -   Games   Score   Av.Op.  Draws
   1 Stockfish 11 bmi2    : 3476   14   14   750    63.8 %   3375   61.2 %
   2 Stockfish 10 bmi2    : 3425   14   14   750    54.7 %   3392   66.9 %
   3 Stockfish 9 bmi2     : 3382   14   14   750    46.6 %   3406   64.9 %
   4 Stockfish 8 bmi2     : 3317   14   14   750    34.9 %   3428   59.2 %

Draw-rate                    : 63.1 % (smaller is better)
Elo spreading (first to last): 159 Elo (bigger is better)

 

my SALC V5 full-closed 500 openings:

     Program                Elo    +    -   Games   Score   Av.Op.  Draws
   1 Stockfish 11 bmi2    : 3482   14   14   750    64.8 %   3373   53.3 %
   2 Stockfish 10 bmi2    : 3437   14   14   750    56.7 %   3388   56.3 %
   3 Stockfish 9 bmi2     : 3381   14   14   750    46.5 %   3406   52.9 %
   4 Stockfish 8 bmi2     : 3300   14   14   750    32.1 %   3433   48.7 %

Draw-rate                    : 52.8 % (smaller is better)
Elo spreading (first to last): 182 Elo (bigger is better)

 

my Drawkiller tournament openings:

     Program                Elo    +    -   Games   Score   Av.Op.  Draws
   1 Stockfish 11 bmi2    : 3527   15   15   750    71.5 %   3358   41.3 %
   2 Stockfish 10 bmi2    : 3455   14   14   750    59.5 %   3382   46.5 %
   3 Stockfish 9 bmi2     : 3360   15   15   750    42.9 %   3413   44.7 %
   4 Stockfish 8 bmi2     : 3259   15   15   750    26.1 %   3447   35.2 %

Draw-rate                    : 41.9 % (smaller is better)
Elo spreading (first to last): 268 Elo (bigger is better)

 

 

 

[XXX]= results with classical Armageddon scoring

 

NBSC Advanced Armageddon Noomen 3-moves:

     Program                Elo    +    -   Games   Score   Av.Op.  Draws
   1 Stockfish 11 bmi2    : 3543   15   15   803    74.0 %   3347    0.0 %
   2 Stockfish 10 bmi2    : 3468   14   14   785    61.8 %   3374    0.0 %
   3 Stockfish 9 bmi2     : 3359   14   14   794    42.8 %   3414    0.0 %
   4 Stockfish 8 bmi2     : 3229   16   16   816    22.1 %   3459    0.0 %

Draw-rate                    : 0 % (smaller is better)
Elo spreading (first to last): 314 Elo (bigger is better) [295 Elo]
White Score                  : 46.5 % [49.5 %]
Number of wins for Black (= 2 points for Black in advanced scoring): 99

 

NBSC Advanced Armageddon SuperGM 4-moves:

     Program                Elo    +    -   Games   Score   Av.Op.  Draws
   1 Stockfish 11 bmi2    : 3561   15   15   802    76.6 %   3344    0.0 %
   2 Stockfish 10 bmi2    : 3434   14   14   793    56.0 %   3387    0.0 %
   3 Stockfish 9 bmi2     : 3362   14   14   784    43.5 %   3413    0.0 %
   4 Stockfish 8 bmi2     : 3243   15   15   809    24.1 %   3454    0.0 %

Draw-rate                    : 0 % (smaller is better)
Elo spreading (first to last): 318 Elo (bigger is better) [299 Elo]
White Score                  : 48.5 % [51.5 %]
Number of wins for Black (= 2 points for Black in advanced scoring): 94

 

NBSC Advanced Armageddon 30k 5-moves:

     Program                Elo    +    -   Games   Score   Av.Op.  Draws
   1 Stockfish 11 bmi2    : 3568   16   16   790    77.5 %   3342    0.0 %
   2 Stockfish 10 bmi2    : 3441   14   14   788    57.6 %   3382    0.0 %
   3 Stockfish 9 bmi2     : 3346   15   15   783    40.9 %   3418    0.0 %
   4 Stockfish 8 bmi2     : 3245   15   15   807    24.5 %   3454    0.0 %

Draw-rate                    : 0 % (smaller is better)
Elo spreading (first to last): 323 Elo (bigger is better) [311 Elo]
White Score                  : 49.1 % [51.9 %]
Number of wins for Black (= 2 points for Black in advanced scoring): 84

 

As you can see, the NBSC-Armageddon results are "out-of-this-world": no draws, of course, and an Elo-spreading, that ist around 2.25x wider, than using "classical" openings: Average Elo-spreading of Balsa, SuperGM, Chad 8ply, SF Framework openings, HERT 500 and J.Noomen openings is = 142 in this test-series. Average Elo-spreading of the NBSC Armageddon openings (3-5 moves) is = 318 (!!!)

 

To make clear, what this really means, mention, that you need an errorbar in your engine-testings, which is 2.25x bigger, when using classical openings, compared to the errorbar, you need, when using NBSC Advanced Armageddon openings, to make a ranking of engines (Engine A better than B, better than C) statistically valid.

Because the ratings of engines are spreaded so much by the NBSC Advanced Armageddon openings. And mention, that you need around 4x more (!) games in order to half the size of the errorbar. That means, you have to play only around 20%-25% amount of games for the same statistical quality of testing-results, when using NBSC Advanced Armageddon openings, compared to classical opening-sets !!! So, all testwork can be done in a 75%-80% shorter time. Or, when you play the same number of games, the statistical quality of the results is way better than ever before!

 

 

IMPORTANT rules for using the NBSC Armageddon Openings:

Not made for playing no Armageddon - so it is absolutely necessary to set all 1/2-1/2 results (draw) of the played games to 0-1 (win for Black), when an engine-tournament or testrun is finished. 

Not made for playing versus other books or opening-sets: In an engine-tournament or testrun ALL engines must use the same Armageddon openings-set (or book).

 

Base idea of Armageddon openings: Larry Kaufman (Komodo-Team)

Developed and constructed by Stefan Pohl (SPCC)

 

(C) NBSC Advanced Armageddon Openings: Stefan Pohl (SPCC) 2020