🐝 Play Chess like a Honey Bee

Submitting Your Model

The goal is to create a chess-playing language model with under 1 million parameters, which is roughly the number of neurons in a honey bee's brain.

At this scale, efficiency and clever architecture choices are key! We are not targeting superhuman performance, but rather exploring how well small models can learn the rules of chess. The goal is to play legal moves.

Getting Started

1. Clone this repository:

   git clone ssh://huggingface.co/spaces/LLM-course/Chess1MChallenge
   

2. Check an example solution in the example_solution/ folder for reference

3. Train your model using the provided training script or your own approach, and evaluate it locally:

   python -m src --model ./my_model
   

4. Submit using the official script:

   python submit.py --model_path ./my_model --model_name my-chess-model
   

5. Run evaluation on this page to see your results on the leaderboard

Evaluation Procedure

Your model will be evaluated as follows:

1. Parameter check: Must have < 1M parameters
2. Security check: Model cannot use python-chess to filter legal moves
3. Game play: 500 moves against opponent engine (games restart every 25 moves)
4. Move generation: 3 retries allowed per move, greedy decoding
5. Scoring: Legal move rate (first try and with retries)

The evaluation is fully deterministic (seeded randomness, deterministic opponent).

Requirements

• Model must be under 1M parameters
• Model must use the ChessConfig and ChessForCausalLM classes (or compatible)
• Include the tokenizer with your submission
• Do not use python-chess to filter moves during generation

Tips for Better Performance

• Experiment with different architectures (layers, heads, dimensions)
• Try weight tying to save parameters
• Focus on learning the rules of chess, not just memorizing openings
• Check the example_solution/ folder for ideas