heimdall

Heimdall is a superhuman chess engine written in Nim. As far as I know, this is the strongest Nim engine that has ever been tested (please let me know should that not be the case), sitting around the top 60 rank globally.

Logo by @kan, thank you!

Building and Installation

Note: Do not run a bare make command! This will not update the neural networks submodule and is meant to be used by OpenBench only.

Just run make native, this is the easiest (Nim 2.2.0 is required, see here). It will build the most optimized executable possible, but AVX2 support is expected on the target platform. Heimdall also requires the clang compiler to be built, as executables generated by gcc are horrendously slow for some reason.

You can also run make zen2 to build a modern version of Heimdall for slightly older CPUs (AVX2 support is still required here). For even slightly older CPUs, try make modern instead. For (very) old CPUs without AVX2 support, run make legacy. In every case, the resulting executable will be located at bin/$(EXE) (bin/heimdall by default).

Or you can grab the latest version from the releases page

Note: Unless you know what you're doing and how nim.cfg works, you probably don't want to build Heimdall using nimble. Just use the Makefile

Advanced: Building with a custom network

Note: If you intend to use a network that has the same architecture as the one Heimdall ships with, you don't need to do this. Just set the EvalFile UCI option to the path of the network file.

If you do intend to embed a different neural network than the one heimdall defaults with, there are a bunch of things to change. You can see that nim.cfg defines the following options:

-d:evalFile="../networks/files/mistilteinn.bin"
-d:hlSize=1536
-d:ftSize=704
-d:inputBuckets=16
-d:outputBuckets=8
-d:evalNormalizeFactor=259
-d:mergedKings
-d:horizontalMirroring

These parameters fully describe Heimdall's network architecture (see here for details) and are what you need to change to allow it to build with a different one. Specifically:

• ftSize controls the size of the feature transformer (aka input layer)

• hlSize controls the size of the first hidden layer

• inputBuckets and outputBuckets are pretty self-explanatory (if you need me to explain, this section is not for you)

• evalNormalizeFactor: The normalization factor as outputted by this program. Also make sure to modify the constants (A_s and B_s) in src/util/wdl.nim

  Feel free to ask for help on how to do this. Not doing this will make Heimdall's normalized eval output completely unreliable, as it will be based on the parameters for a different network

• mergedKings controls whether the network uses merged king planes (requires a bucket layout where no two kings can be in the same bucket)

• horizontalMirroring enables supports for horizontal mirroring

• evalFile is the path, relative to src/, where the network file is located (it will be embedded in the final executable)

The options without an equal sign can be disabled by commenting them out.

You're also going to need to modify the input bucket layout in src/nnue/model.nim (assumes a1=0). Be mindful of the horizontal symmetry if you're using a horizontally mirrored network (look at how it's already done for Heimdall's network, it's not hard).

Then all you need to do is build the engine as usual. You're a smart guy (gal?), I'm sure you can figure it out. Do reach out if you have problems, though.

Note: Heimdall requires perspective networks, where the first subnetwork is the side-to-move perspective and the second is the non side to move

Note 2: Only single-(hidden-)layer networks are supported (for now)

Testing

Just run nimble test: sit back, relax, get yourself a cup of coffee and wait for it to finish (it will take a long time)

Note: The test suite requires Python and expects both heimdall and stockfish to be installed and in the system's PATH. Alternatively, it is possible to specify the location of both Heimdall and Stockfish (run python tests/suite.py -h for more information)

Configuration

Heimdall is a UCI engine, which means that it's not meant to be used as a standalone program (although you can do that, as it defaults to a pretty-printed output unless the environment variable NO_COLOR is set or it detects that it's not attached to a TTY). To use it at its best, you can add it to any number of chess GUIs like Arena, En Croissant or Cutechess. I strive to have Heimdall work flawlessly with any GUI (within reason), so please let me know if you find any issues!

Heimdall supports the following UCI options:

• HClear: Clears all history tables. This is done automatically at every new game, so you shouldn't need to do this normally
• TTClear: Clears the transposition table. Like history clearing, this is done at every new game, so you shouldn't need this
• Ponder: Allows Heimdall to search while its opponent is also searching. A go ponder command will not start a ponder search unless this is set!
• ShowWDL: Display the predicted win, draw and loss probability (see NormalizeScore below for more info). Not all GUIs support this, so only enable it if you know the one you're using does!
• UCI_Chess960: Switches Heimdall to playing Fischer random chess (also known as chess960). Heimdall supports Double Fischer random chess as well!
• EvalFile: Path to the neural network to use for evaluation. Its default value of <default> will cause Heimdall to use the network embedded in the executable. Do not set this to anything other than a valid path that the engine can access, or it will crash (and no, empty strings don't work either!). Keep in mind that the network has to be of the same size and architecture as Heimdall's own (check here for details)
• NormalizeScore: Enables score normalization. This means that displayed scores will be normalized such that +1.0 means a 50% probability of winning when there's around 58 points of material on the board (using the standard 1, 3, 3, 5, 9 weights for pawns, minor pieces, rooks and queens). Thanks to the stockfish folks who developed the WDL model! This option is enabled by default
• EnableWeirdTCs: Allows Heimdall to play with untested/weird/outdaded time controls such as moves to go or sudden death: Heimdall will refuse to search with those unless this is set! See here for more details on why this exists
• MultiPV: The number of best moves to search for. The default value of one is best suited for strength, but you can set this to more if you want the engine to analyze different lines. Note that a time-limited search will share limits across all lines!
• Threads: How many threads to allocate for search. By default Heimdall will only search with one thread
• Hash: The size of the hash table in mebibytes (aka REAL megabytes). The default is 64
• MoveOverhead: How much time (in milliseconds) Heimdall will subtract from its own remaining time to account for communication delays with an external program (usually a GUI or match manager). Particularly useful when playing games over a network (for example through a Lichess bot or on an internet chess server). This is set to 0 by default

Search

Heimdall implements negamax search with alpha-beta pruning in a PVS framework to search the game tree and utilizes several heuristics to help it navigate the gigantic search space of chess

Evaluation

Heimdall currently uses NNUE (Efficiently Updatable Neural Network) to evaluate positions. All of heimdall's networks are trained with bullet using data obtained from selfplay of previous versions, while previous HCE releases used the lichess-big3 dataset for tuning. The current network architecture consists of a horizontally mirrored perspective network using merged king planes, featuring a single hidden layer of 1536 neurons with 16 input buckets and 8 output buckets, and is commonly represented as (704x16hm->1536)x2->1x8

Notes for engine testers

Heimdall is designed (and tested) to play at the standard time controls of time + increment: since I do not have the hardware nor the time to test others (like sudden death or moves to go), support for outdated/nonstandard time controls has been hidden behind the EnableWeirdTCs option. Unless this option is set, Heimdall will refuse to play either if its own increment is missing/zero or if it is told to play with a moves to go time control (this one is especially important because it is not taken into account at all in time management!): this technically means Heimdall is not fully UCI compliant unless EnableWeirdTCs is enabled, but I believe this trade-off is worth it, as it means that if it does indeed perform worse at untested time controls then the tester will have full knowledge as to why that is. If that upsets you or makes you want to not test Heimdall, that's fine! I'm sorry you feel that way, but this is my engine after all :)

More info

Heimdall is sometimes available on Lichess under its old name (Nimfish), feel free to challenge it! I try to keep the engine running on there always up to date with the changes on the master branch. The hardware running it is quite heterogenous however, so expect big rating swings

Strength

Lots of people are kind enough to test Heimdall on their own hardware. Here's a summary of the rating lists I'm aware of (please contact me if you want me to add yours!)

Version	Estimated	CCRL 40/15 (1CPU)	TCEC	CCRL FRC 40/2	CCRL Blitz 2+1 (1CPU)	MCERL	CEGT 40/20	CCRL 40/15 (4CPU)
0.1	2531	2436	-	N/A	-	-	-	-
0.2	2706	2669	-	N/A	-	-	-	-
0.3	2837	-	-	N/A	-	-	-	-
0.4	2888	2864	-	2926	-	-	-	-
1.0	3230	3193	3163*	3372	-	-	-	-
1.1	3370	-	-	-	-	-	-	-
1.1.1	3390**	3362	-	3556	3394	3456	3284	-
1.2	3490	-	-	-	-	3470	-	-
1.2.{1,2}	3500	3376	-	3621	3476	3479	3300	3441
1.3	3548***	-	-	-	-	-	-	-

*: Beta version, not final 1.0 release

**: Estimated at LTC (1CPU, 40+0.4s, 128MB hash) against Stash v36 (-0.2 +- 11.1)

***: Check 1.3's release notes for info about how this was calculated

Note: Unless otherwise specified, estimated strenght is measured for standard chess at a short time control (8 seconds with 0.08 seconds increment) with 1 search thread and a 16MB hash table over 1000 game pairs against the previous version (except for version 0.1 where it was tested in a gauntlet) using the Pohl opening book (up to version 1.0) and the UHO_Lichess_4852_v1 book for later versions, and is therefore not as accurate as the other ratings which are provided by testers running the engine at longer TCs against a pool of different opponents.

Notes

This repository was extracted from a monorepo that you can check out here (look into the Chess/ directory): all history before the first commit here can be found there!

Credits

Many thanks to all the folks on the Engine Programming and Stockfish servers on Discord: your help has been invaluable and Heimdall literally would not exist without the help of all of you. In no particular order, I'd like to thank:

• @analog-hors (okay, she's first for a reason): for her awesome article about magic bitboards as well as providing the initial code for the HCE tuner and the NN inference to get me started on NNUE
• @ciekce: for helping me debug countless issues
• @sroelants: provided debugging help and lots of good ideas to steal
• @tsoj: Saved my ass by solving some major performance bottlenecks and helping me debug my broken threading code
• @viren, @zuppadcipolle, @toanth, @fuuryy, @agethereal: Debugging help
• @DarkNeutrino: for lending cores to my OB instance
• @ceorwmt: for helping with datagen
• @cj5716: Provided lots of ideas to steal
• @jw1912: For creating bullet and helping with debugging twofold LMR (+140 Elo!)

Y'all are awesome! <3

P.S. I probably forgot someone, please let me know should that be the case!