Tony_tech/chess-project
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

scripts

Browse Source
This commit is contained in:
Stefan
2025-12-15 14:54:27 +01:00
parent 7383af7884
commit 35ba74e5c0
5 changed files with 1006 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
scripts/README.md
+365
View File
@@ -0,0 +1,365 @@
# Build Scripts
## Overview
This directory contains setup and build scripts for the chess tutor application:
1. **Tactical Puzzles Setup** - Downloads and configures tactical puzzles from Lichess
2. **Wikipedia Cache Builder** - Downloads Wikipedia articles for chess openings
---
# Tactical Puzzles Setup Script
## Quick Start
### Docker (Recommended for production)
**Use the helper script from your host machine:**
```bash
# Default: 100 puzzles per pattern (800 total)
./scripts/docker-setup-puzzles.sh
# Customize number of puzzles
./scripts/docker-setup-puzzles.sh --max-puzzles 500
# Force re-download
./scripts/docker-setup-puzzles.sh --force
```
The script will:
- Check if your container is running
- Download puzzles inside the container
- Store them in persistent Docker volumes
- Puzzles persist across container restarts
### Local Development
**Run this command once before using the tactical practice feature:**
```bash
cd chess_tutor
python3 scripts/setup_tactical_puzzles.py
# Or specify the number of puzzles:
python3 scripts/setup_tactical_puzzles.py --max-puzzles 100
python3 scripts/setup_tactical_puzzles.py --max-puzzles 500
```
## What It Does
The setup script automatically:
1. ✅ **Checks dependencies** - Verifies `zstd` is installed (installs it if needed)
2. ✅ **Downloads database** - Fetches the Lichess puzzle database (~500MB compressed)
3. ✅ **Decompresses** - Extracts the CSV file (~3.5GB uncompressed)
4. ✅ **Filters puzzles** - Extracts 20 high-quality puzzles for each tactical pattern:
- PIN
- FORK
- SKEWER
- DISCOVERED_CHECK
- DOUBLE_ATTACK
- OVERLOADING
- BACK_RANK_WEAKNESS
- TRAPPED_PIECE
5. ✅ **Converts format** - Transforms Lichess format to our JSON fixture format
6. ✅ **Saves fixtures** - Writes to `fixtures/tactics/*.json`
7. ✅ **Creates marker** - Places `.tactical_puzzles_configured` file to prevent re-running
## Command Line Options
```bash
python3 scripts/setup_tactical_puzzles.py [OPTIONS]
Options:
--max-puzzles N Number of puzzles to extract per pattern (default: 20)
--force Force re-run setup without prompting
-h, --help Show help message
Examples:
# Default: 20 puzzles per pattern (160 total)
python3 scripts/setup_tactical_puzzles.py
# 100 puzzles per pattern (800 total)
python3 scripts/setup_tactical_puzzles.py --max-puzzles 100
# 500 puzzles per pattern (4000 total) - recommended for production
python3 scripts/setup_tactical_puzzles.py --max-puzzles 500
# Force re-run without prompting
python3 scripts/setup_tactical_puzzles.py --max-puzzles 200 --force
```
## Requirements
- **Python 3.7+** (with `python-chess` library - auto-installed if missing)
- **zstd** (auto-installed on macOS/Linux if missing)
- **~4GB disk space** (for downloaded and decompressed database)
- **Internet connection** (for downloading ~500MB file)
## Time Estimate
- **First run**: 5-10 minutes (depending on internet speed)
- **Subsequent runs**: Instant (uses cached database)
## Quality Criteria
The script filters puzzles based on:
- **Popularity**: ≥ 50 (well-liked by users)
- **Rating**: 1200-2000 (appropriate difficulty for learning)
- **Plays**: ≥ 50 (well-tested)
- **Theme**: Must match the tactical pattern
Only the top 20 puzzles (by popularity) are selected for each pattern.
## File Structure
```
chess_tutor/
├── scripts/
│ ├── setup_tactical_puzzles.py # Main setup script
│ └── README.md # This file
├── downloads/ # Created by script
│ ├── lichess_db_puzzle.csv.zst # Downloaded database (cached)
│ └── lichess_db_puzzle.csv # Decompressed database (cached)
├── fixtures/
│ └── tactics/ # Created by script
│ ├── pin.json # 20 PIN puzzles
│ ├── fork.json # 20 FORK puzzles
│ ├── skewer.json # 20 SKEWER puzzles
│ └── ... # Other patterns
└── .tactical_puzzles_configured # Marker file (created by script)
```
## Re-running Setup
If you want to re-run the setup (e.g., to get fresh puzzles):
```bash
# Option 1: Delete the marker file and re-run
rm chess_tutor/.tactical_puzzles_configured
python3 scripts/setup_tactical_puzzles.py
# Option 2: The script will ask if you want to re-run
python3 scripts/setup_tactical_puzzles.py
# Answer 'y' when prompted
```
## Cleaning Up
To save disk space after setup:
```bash
# Delete the downloaded database (keeps the fixtures)
rm -rf chess_tutor/downloads/
# The fixtures in chess_tutor/fixtures/tactics/ will remain
```
## Troubleshooting
### "zstd not found"
The script will attempt to auto-install `zstd`. If it fails:
**macOS:**
```bash
brew install zstd
```
**Ubuntu/Debian:**
```bash
sudo apt-get install zstd
```
**Fedora/RHEL:**
```bash
sudo dnf install zstd
```
### "python-chess not found"
The script will attempt to auto-install `python-chess`. If it fails:
```bash
pip3 install python-chess
```
### "Download failed"
Check your internet connection and try again. The Lichess database is updated daily, so temporary issues may occur.
### "No puzzles found for pattern X"
This is rare but can happen if the database doesn't have enough puzzles matching the criteria. The script will warn you but continue with other patterns.
## Manual Setup (Alternative)
If the automatic script doesn't work, you can manually:
1. Download: https://database.lichess.org/lichess_db_puzzle.csv.zst
2. Decompress with `zstd -d lichess_db_puzzle.csv.zst`
3. Filter puzzles using the Python code in `docs/LICHESS_PUZZLES.md`
4. Convert to JSON format and save to `fixtures/tactics/`
## Source
- **Database**: https://database.lichess.org/
- **License**: Creative Commons CC0 (public domain)
- **Documentation**: See `docs/LICHESS_PUZZLES.md` for detailed information
## Support
If you encounter issues:
1. Check the terminal output for specific error messages
2. Ensure you have Python 3.7+ installed: `python3 --version`
3. Ensure you have internet connectivity
4. Try re-running the script
5. Check `docs/LICHESS_PUZZLES.md` for manual setup instructions
---
# Wikipedia Cache Builder
## Overview
Downloads full Wikipedia articles for all chess opening families and caches them locally for offline use.
## Quick Start
```bash
# Install dependencies (if not already done)
npm install
# Run the Wikipedia cache builder
npm run cache:wikipedia
```
## Benefits
- ✅ **Offline access** - Works completely offline once cached
- ✅ **Instant loading** - No network delay
- ✅ **No rate limiting** - Avoids Wikipedia API rate limits
- ✅ **Full content** - Gets complete articles, not just summaries
- ✅ **Version controlled** - Safe to commit to git
## What It Does
1. **Extracts opening families** from your opening database (`public/openings/*.json`)
2. **Searches Wikipedia** for each family using the OpenSearch API
3. **Downloads full article content** including:
- Introduction
- Main sections (History, Ideas, Variations, etc.)
- Up to 5 relevant sections per opening
4. **Saves locally** to `public/wikipedia/*.json`
5. **Creates an index** at `public/wikipedia/index.json`
## Output Structure
Each opening family gets a JSON file like `public/wikipedia/french-defense.json`:
```json
{
"openingFamily": "French Defense",
"title": "French Defence",
"url": "https://en.wikipedia.org/wiki/French_Defence",
"sections": [
{
"title": "Introduction",
"text": "The French Defence is a chess opening..."
},
{
"title": "History",
"text": "The first known game with the French Defense..."
}
],
"lastModified": "2025-01-15T...",
"license": "CC BY-SA 3.0",
"licenseUrl": "https://creativecommons.org/licenses/by-sa/3.0/",
"fetchedAt": 1705334400000
}
```
## Committing to Git
The generated files are **safe to commit** to your repository:
- Wikipedia content is licensed under CC BY-SA 3.0 (allows redistribution with attribution)
- Each file includes proper license information
- Files are versioned, so you can review changes before committing
## When to Run
- **Initial setup**: Run once to cache all Wikipedia articles
- **After adding new openings**: Re-run to fetch articles for new families
- **Periodically**: Re-run every few months to get updated Wikipedia content
## Rate Limiting
The script includes a 1-second delay between requests to be respectful to Wikipedia's servers.
**Estimated time:**
- 50 opening families = ~1-2 minutes
- 100 opening families = ~2-3 minutes
## Fallback Strategy
The app uses a 3-tier approach for Wikipedia content:
1. **Local cache** (`public/wikipedia/*.json`) - Fastest, always available
2. **localStorage cache** - In-browser cache for API fetches
3. **Live API** - Fallback if local cache missing
This ensures Wikipedia content is always available, even for openings not in your local cache.
## License & Attribution
Wikipedia content is licensed under **CC BY-SA 3.0**:
- ✅ Commercial use allowed
- ✅ Modification allowed
- ✅ Redistribution allowed
- ⚠️ Attribution required (included in JSON files)
- ⚠️ Derivative works must use same license
More info: https://creativecommons.org/licenses/by-sa/3.0/
## Implementation Details
**Script location:** `scripts/fetch-wikipedia-openings.ts`
**Dependencies:**
- Node.js 18+
- tsx (TypeScript executor)
**API endpoints used:**
- Wikipedia OpenSearch API (for finding articles)
- Wikipedia MediaWiki Parse API (for full content)
**Output directory:** `public/wikipedia/`
## Example Usage
```bash
# Run the cache builder
npm run cache:wikipedia
# Output:
# 🌐 Wikipedia Opening Cache Builder
# 📚 Extracting opening families from database...
# ✓ Found 47 unique opening families
#
# 📖 Processing: French Defense
# Searching Wikipedia for: "French Defense"
# ✓ Found: "French Defence"
# Fetching full article...
# ✓ Fetched 5 sections
# ✓ Saved to: french-defense.json
# ...
# ✨ Wikipedia Cache Build Complete!
# ✓ Successful: 45
# ⚠️ Skipped: 2
# ❌ Failed: 0
```
scripts/docker-setup-puzzles.sh
Executable
+96
View File
@@ -0,0 +1,96 @@
#!/bin/bash
#
# Helper script to download Lichess tactical puzzles in Docker container
#
# Usage:
# ./scripts/docker-setup-puzzles.sh [options]
#
# Options:
# --max-puzzles NUM Number of puzzles per pattern (default: 100)
# --force Force re-download even if already configured
# --container NAME Container name (default: chess-tutor)
#
set -e
# Default values
CONTAINER_NAME="chess-tutor"
MAX_PUZZLES=100
FORCE_FLAG=""
# Parse arguments
while [[ $# -gt 0 ]]; do
case $1 in
--max-puzzles)
MAX_PUZZLES="$2"
shift 2
;;
--force)
FORCE_FLAG="--force"
shift
;;
--container)
CONTAINER_NAME="$2"
shift 2
;;
*)
echo "Unknown option: $1"
echo "Usage: $0 [--max-puzzles NUM] [--force] [--container NAME]"
exit 1
;;
esac
done
echo "=========================================="
echo "Chess Tutor - Puzzle Setup (Docker)"
echo "=========================================="
echo "Container: $CONTAINER_NAME"
echo "Puzzles per pattern: $MAX_PUZZLES"
echo "=========================================="
echo ""
# Check if container is running
if ! docker ps --format '{{.Names}}' | grep -q "^${CONTAINER_NAME}$"; then
echo "❌ Error: Container '$CONTAINER_NAME' is not running"
echo ""
echo "Start it with: docker-compose up -d"
exit 1
fi
echo "✅ Container is running"
echo ""
# Check if already configured (unless --force)
if [ -z "$FORCE_FLAG" ]; then
if docker exec "$CONTAINER_NAME" test -f .tactical_puzzles_configured 2>/dev/null; then
echo "⚠️ Puzzles are already configured!"
echo ""
read -p "Do you want to re-download? (y/N): " -n 1 -r
echo
if [[ ! $REPLY =~ ^[Yy]$ ]]; then
echo "Exiting..."
exit 0
fi
FORCE_FLAG="--force"
fi
fi
# Run the setup script inside the container
echo "🎯 Starting puzzle download..."
echo "This will download ~500MB and may take 5-10 minutes"
echo ""
docker exec -it "$CONTAINER_NAME" \
python3 scripts/setup_tactical_puzzles.py --max-puzzles "$MAX_PUZZLES" $FORCE_FLAG
echo ""
echo "=========================================="
echo "✅ Puzzle setup complete!"
echo "=========================================="
echo ""
echo "The puzzles are now available in your Chess Tutor app."
echo "They will persist across container restarts."
echo ""
echo "To check the downloaded puzzles:"
echo " docker exec $CONTAINER_NAME ls -lh fixtures/tactics/"
echo ""
scripts/find-missing-wikipedia.js
+66
View File
@@ -0,0 +1,66 @@
#!/usr/bin/env node
const fs = require('fs');
const path = require('path');
const ecoFiles = ['ecoA', 'ecoB', 'ecoC', 'ecoD', 'ecoE'];
const missingWiki = [];
for (const file of ecoFiles) {
const filePath = path.join(__dirname, '..', 'public', 'openings', `${file}.json`);
const data = JSON.parse(fs.readFileSync(filePath, 'utf8'));
for (const [fen, opening] of Object.entries(data)) {
if (opening.isEcoRoot && !opening.wikipediaSlug) {
missingWiki.push({
eco: opening.eco,
name: opening.name,
moves: opening.moves
});
}
}
}
// Sort by ECO code
missingWiki.sort((a, b) => a.eco.localeCompare(b.eco));
console.log('Major openings (ECO roots) WITHOUT Wikipedia slugs:\n');
console.log('Total:', missingWiki.length, '\n');
// Group by ECO category
const byCategory = {};
for (const opening of missingWiki) {
const category = opening.eco[0];
if (!byCategory[category]) byCategory[category] = [];
byCategory[category].push(opening);
}
for (const [cat, openings] of Object.entries(byCategory).sort()) {
console.log(`\n=== ECO ${cat} (${openings.length} openings) ===`);
openings.slice(0, 15).forEach(o => {
console.log(`${o.eco.padEnd(4)} ${o.name}`);
});
if (openings.length > 15) {
console.log(`... and ${openings.length - 15} more`);
}
}
// Show some notable ones
console.log('\n\n=== Notable Missing Openings ===');
const notable = [
'Sicilian', 'French', 'Caro-Kann', 'Pirc', 'Alekhine',
'Scandinavian', 'Queen', 'English', 'Indian', 'Benoni',
'Nimzo', 'Grunfeld', 'Dutch', 'Reti', 'Bird'
];
const notableMatches = missingWiki.filter(o =>
notable.some(n => o.name.toLowerCase().includes(n.toLowerCase()))
);
if (notableMatches.length > 0) {
notableMatches.forEach(o => {
console.log(`${o.eco.padEnd(4)} ${o.name}`);
});
} else {
console.log('(None found - all major openings have Wikipedia slugs!)');
}
scripts/setup_tactical_puzzles.py
Executable
+392
View File
@@ -0,0 +1,392 @@
#!/usr/bin/env python3
"""
One-time setup script to download and configure high-quality tactical puzzles from Lichess.
This script:
1. Downloads the Lichess puzzle database
2. Extracts puzzles for each tactical pattern
3. Converts them to our JSON fixture format
4. Validates them with our tactical library
5. Creates a marker file to indicate setup is complete
Usage:
python3 scripts/setup_tactical_puzzles.py
"""
import os
import sys
import json
import csv
import subprocess
import urllib.request
from pathlib import Path
from typing import List, Dict, Any
# Add parent directory to path to import chess libraries
sys.path.insert(0, str(Path(__file__).parent.parent))
try:
import chess
import chess.pgn
except ImportError:
print("❌ Error: python-chess library not found.")
print("Installing python-chess...")
subprocess.check_call([sys.executable, "-m", "pip", "install", "python-chess"])
import chess
import chess.pgn
# Configuration
LICHESS_PUZZLE_URL = "https://database.lichess.org/lichess_db_puzzle.csv.zst"
DOWNLOAD_DIR = Path(__file__).parent.parent / "downloads"
FIXTURES_DIR = Path(__file__).parent.parent / "fixtures" / "tactics"
SETUP_MARKER = Path(__file__).parent.parent / ".tactical_puzzles_configured"
# Mapping of our patterns to Lichess themes
PATTERN_THEMES = {
"pin": ["pin"],
"fork": ["fork"],
"skewer": ["skewer"],
"discovered_check": ["discoveredAttack"],
"double_attack": ["doubleCheck", "fork"],
"overloading": ["overloading"],
"back_rank_weakness": ["backRankMate"],
"trapped_piece": ["trappedPiece"],
}
# Quality criteria
MIN_POPULARITY = 50
MIN_RATING = 800 # Easy puzzles start here
MAX_RATING = 2200 # Hard puzzles go up to here
MIN_PLAYS = 50
# Default number of puzzles per pattern (can be overridden via command line)
DEFAULT_PUZZLES_PER_PATTERN = 20
def check_zstd_installed() -> bool:
"""Check if zstd is installed for decompression."""
try:
subprocess.run(["zstd", "--version"], capture_output=True, check=True)
return True
except (subprocess.CalledProcessError, FileNotFoundError):
return False
def install_zstd():
"""Attempt to install zstd."""
print("📦 Installing zstd...")
# Detect OS and install accordingly
if sys.platform == "darwin": # macOS
try:
subprocess.check_call(["brew", "install", "zstd"])
print("✅ zstd installed successfully via Homebrew")
except (subprocess.CalledProcessError, FileNotFoundError):
print("❌ Failed to install zstd. Please install Homebrew first:")
print(" /bin/bash -c \"$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)\"")
sys.exit(1)
elif sys.platform.startswith("linux"):
try:
# Try apt-get (Debian/Ubuntu)
subprocess.check_call(["sudo", "apt-get", "update"])
subprocess.check_call(["sudo", "apt-get", "install", "-y", "zstd"])
print("✅ zstd installed successfully via apt-get")
except (subprocess.CalledProcessError, FileNotFoundError):
print("❌ Failed to install zstd. Please install it manually:")
print(" Debian/Ubuntu: sudo apt-get install zstd")
print(" Fedora/RHEL: sudo dnf install zstd")
sys.exit(1)
else:
print("❌ Unsupported OS. Please install zstd manually:")
print(" Windows: Download from https://github.com/facebook/zstd/releases")
sys.exit(1)
def download_puzzle_database() -> Path:
"""Download the Lichess puzzle database."""
DOWNLOAD_DIR.mkdir(parents=True, exist_ok=True)
compressed_file = DOWNLOAD_DIR / "lichess_db_puzzle.csv.zst"
decompressed_file = DOWNLOAD_DIR / "lichess_db_puzzle.csv"
# Check if already downloaded and decompressed
if decompressed_file.exists():
print(f"✅ Puzzle database already exists at {decompressed_file}")
return decompressed_file
# Download if not exists
if not compressed_file.exists():
print(f"📥 Downloading Lichess puzzle database from {LICHESS_PUZZLE_URL}")
print(" This may take several minutes (file is ~500MB compressed)...")
try:
urllib.request.urlretrieve(LICHESS_PUZZLE_URL, compressed_file)
print(f"✅ Downloaded to {compressed_file}")
except Exception as e:
print(f"❌ Failed to download: {e}")
sys.exit(1)
# Decompress
print(f"📦 Decompressing {compressed_file.name}...")
print(" This may take several minutes (decompressed file is ~3.5GB)...")
try:
subprocess.check_call(["zstd", "-d", str(compressed_file), "-o", str(decompressed_file)])
print(f"✅ Decompressed to {decompressed_file}")
except subprocess.CalledProcessError as e:
print(f"❌ Failed to decompress: {e}")
sys.exit(1)
return decompressed_file
def extract_puzzles_for_pattern(csv_file: Path, pattern: str, themes: List[str], max_puzzles: int) -> List[Dict[str, Any]]:
"""Extract high-quality puzzles for a specific tactical pattern."""
print(f"🔍 Extracting {pattern.upper()} puzzles (max: {max_puzzles})...")
puzzles = []
with open(csv_file, 'r', encoding='utf-8') as f:
reader = csv.DictReader(f)
for row in reader:
puzzle_themes = row['Themes'].split()
rating = int(row['Rating'])
popularity = int(row['Popularity'])
nb_plays = int(row['NbPlays'])
# Check if puzzle matches our criteria
if (any(theme in puzzle_themes for theme in themes) and
popularity >= MIN_POPULARITY and
MIN_RATING <= rating <= MAX_RATING and
nb_plays >= MIN_PLAYS):
puzzles.append(row)
# Stop when we have enough
if len(puzzles) >= max_puzzles:
break
# Sort by popularity (best first)
puzzles.sort(key=lambda x: int(x['Popularity']), reverse=True)
print(f" Found {len(puzzles)} high-quality {pattern.upper()} puzzles")
return puzzles[:max_puzzles]
def lichess_to_fixture(puzzle_row: Dict[str, Any], pattern_type: str) -> Dict[str, Any]:
"""Convert Lichess puzzle to our fixture format.
Lichess puzzle format:
- FEN: Position BEFORE opponent's first move
- Moves: Space-separated UCI moves alternating opponent/player
- First move: Opponent's move (sets up the puzzle)
- Remaining moves: Player move, opponent response, player move, etc.
"""
fen = puzzle_row['FEN']
moves_uci = puzzle_row['Moves'].split()
# Apply first move (opponent's move) to get starting position
board = chess.Board(fen)
opponent_move = chess.Move.from_uci(moves_uci[0])
board.push(opponent_move)
initial_fen = board.fen()
# Determine side to move from initial FEN
side_to_move = "white" if " w " in initial_fen else "black"
# Process all moves in the sequence
# moves_uci[0] = opponent's setup move (already applied)
# moves_uci[1] = player's first move (solution start)
# moves_uci[2] = opponent's response
# moves_uci[3] = player's second move
# etc.
move_sequence = []
for i, move_uci in enumerate(moves_uci[1:], start=1): # Skip first move (already applied)
move = chess.Move.from_uci(move_uci)
move_san = board.san(move)
# Determine who makes this move
# Odd indices (1, 3, 5...) = player moves
# Even indices (2, 4, 6...) = opponent moves
is_player_move = (i % 2 == 1)
move_sequence.append({
"uci": move_uci,
"san": move_san,
"player": is_player_move
})
board.push(move)
# Get final position after all moves
resulting_fen = board.fen()
# First player move (for backward compatibility)
first_player_move_uci = moves_uci[1]
board_temp = chess.Board(initial_fen)
first_player_move = chess.Move.from_uci(first_player_move_uci)
first_player_move_san = board_temp.san(first_player_move)
return {
"id": puzzle_row['PuzzleId'],
"initialFen": initial_fen,
"sideToMove": side_to_move,
"rating": int(puzzle_row['Rating']), # Add rating for difficulty filtering
"bestMove": {
"san": first_player_move_san,
"uci": first_player_move_uci
},
"moves": move_sequence, # Full move sequence
"resultingFen": resulting_fen,
"expectedPattern": {
"type": pattern_type.upper().replace("_", "_"),
# Note: Exact squares will be detected by our tactical library
},
"context": f"Lichess puzzle {puzzle_row['PuzzleId']} (Rating: {puzzle_row['Rating']}, Popularity: {puzzle_row['Popularity']})",
"tags": puzzle_row['Themes'].split()
}
def save_fixtures(pattern: str, fixtures: List[Dict[str, Any]]):
"""Save fixtures to JSON file."""
FIXTURES_DIR.mkdir(parents=True, exist_ok=True)
output_file = FIXTURES_DIR / f"{pattern}.json"
fixture_data = {
"description": f"High-quality {pattern.upper()} tactical puzzles from Lichess database",
"source": "https://database.lichess.org/",
"generatedAt": "auto-generated",
"cases": fixtures
}
with open(output_file, 'w', encoding='utf-8') as f:
json.dump(fixture_data, f, indent=2, ensure_ascii=False)
print(f"✅ Saved {len(fixtures)} puzzles to {output_file}")
def create_setup_marker():
"""Create a marker file to indicate setup is complete."""
with open(SETUP_MARKER, 'w') as f:
f.write("Tactical puzzles configured successfully\n")
print(f"✅ Created setup marker at {SETUP_MARKER}")
print(" (The app will auto-detect Lichess puzzles from fixture metadata)")
def main():
"""Main setup function."""
import argparse
parser = argparse.ArgumentParser(
description='Download and configure high-quality tactical puzzles from Lichess',
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog="""
Examples:
# Download 20 puzzles per pattern (default)
python3 scripts/setup_tactical_puzzles.py
# Download 100 puzzles per pattern
python3 scripts/setup_tactical_puzzles.py --max-puzzles 100
# Download 500 puzzles per pattern for production
python3 scripts/setup_tactical_puzzles.py --max-puzzles 500
"""
)
parser.add_argument(
'--max-puzzles',
type=int,
default=DEFAULT_PUZZLES_PER_PATTERN,
help=f'Maximum number of puzzles to extract per pattern (default: {DEFAULT_PUZZLES_PER_PATTERN})'
)
parser.add_argument(
'--force',
action='store_true',
help='Force re-run setup without prompting'
)
args = parser.parse_args()
print("=" * 70)
print("🎯 Chess Tutor - Tactical Puzzles Setup")
print("=" * 70)
print(f"Configuration: {args.max_puzzles} puzzles per pattern")
print("=" * 70)
print()
# Check if already configured
if SETUP_MARKER.exists() and not args.force:
print("⚠️ Tactical puzzles are already configured!")
response = input("Do you want to re-run the setup? (y/N): ").strip().lower()
if response != 'y':
print("Exiting...")
sys.exit(0)
else:
SETUP_MARKER.unlink()
# Step 1: Check/install zstd
print("Step 1: Checking dependencies...")
if not check_zstd_installed():
print("⚠️ zstd not found (required for decompression)")
install_zstd()
else:
print("✅ zstd is installed")
print()
# Step 2: Download and decompress database
print("Step 2: Downloading Lichess puzzle database...")
csv_file = download_puzzle_database()
print()
# Step 3: Extract puzzles for each pattern
print("Step 3: Extracting puzzles for each tactical pattern...")
total_puzzles = 0
for pattern, themes in PATTERN_THEMES.items():
puzzles = extract_puzzles_for_pattern(csv_file, pattern, themes, args.max_puzzles)
if len(puzzles) == 0:
print(f"⚠️ Warning: No puzzles found for {pattern.upper()}")
continue
# Convert to fixture format
fixtures = []
for puzzle in puzzles:
try:
fixture = lichess_to_fixture(puzzle, pattern)
fixtures.append(fixture)
except Exception as e:
print(f" ⚠️ Skipping puzzle {puzzle['PuzzleId']}: {e}")
# Save to file
if fixtures:
save_fixtures(pattern, fixtures)
total_puzzles += len(fixtures)
print()
# Step 4: Create marker file
print("Step 4: Finalizing setup...")
create_setup_marker()
print()
print("=" * 70)
print(f"✅ Setup complete! {total_puzzles} tactical puzzles are ready to use.")
print("=" * 70)
print()
print("Next steps:")
print("1. Refresh your Chess Tutor app (if running)")
print("2. The warning banner will automatically disappear")
print(" (The app detects Lichess puzzles by checking the fixture metadata)")
print()
print("3. Go to http://localhost:3050/learning")
print("4. Select a coach and practice tactical patterns!")
print()
print("Note: The downloaded database is cached in the 'downloads' directory.")
print(" You can delete it to save space if needed.")
if __name__ == "__main__":
main()
scripts/wikipedia-summary.js
+87
View File
@@ -0,0 +1,87 @@
#!/usr/bin/env node
const fs = require('fs');
const path = require('path');
const ecoFiles = ['ecoA', 'ecoB', 'ecoC', 'ecoD', 'ecoE'];
const withWiki = [];
const withoutWiki = [];
for (const file of ecoFiles) {
const filePath = path.join(__dirname, '..', 'public', 'openings', `${file}.json`);
const data = JSON.parse(fs.readFileSync(filePath, 'utf8'));
for (const [fen, opening] of Object.entries(data)) {
if (opening.isEcoRoot) {
if (opening.wikipediaSlug) {
withWiki.push({
eco: opening.eco,
name: opening.name,
slug: opening.wikipediaSlug
});
} else {
withoutWiki.push({
eco: opening.eco,
name: opening.name
});
}
}
}
}
// Sort
withWiki.sort((a, b) => a.name.localeCompare(b.name));
withoutWiki.sort((a, b) => a.name.localeCompare(b.name));
console.log('━'.repeat(70));
console.log('WIKIPEDIA INTEGRATION SUMMARY');
console.log('━'.repeat(70));
console.log();
console.log(`✅ ECO roots WITH Wikipedia: ${withWiki.length}`);
console.log(`❌ ECO roots WITHOUT Wikipedia: ${withoutWiki.length}`);
console.log();
// Show major openings with Wikipedia
console.log('━'.repeat(70));
console.log('MAJOR OPENINGS WITH WIKIPEDIA (sample):');
console.log('━'.repeat(70));
const majorOpenings = [
'Sicilian', 'French', 'Caro-Kann', 'Pirc', 'Alekhine',
'Scandinavian', 'Italian', 'Spanish', 'Scotch', 'Vienna',
'English', 'Reti', 'Bird', 'Polish', 'Nimzo', 'Queen',
'King', 'Benoni', 'Catalan', 'Grunfeld', 'Dutch', 'Ruy Lopez'
];
const foundMajor = withWiki.filter(o =>
majorOpenings.some(m => o.name.toLowerCase().includes(m.toLowerCase()))
);
foundMajor.slice(0, 40).forEach(o => {
console.log(` ${o.eco.padEnd(4)} ${o.name}`);
});
if (foundMajor.length > 40) {
console.log(` ... and ${foundMajor.length - 40} more major openings`);
}
console.log();
console.log('━'.repeat(70));
console.log('OPENINGS MISSING WIKIPEDIA:');
console.log('━'.repeat(70));
if (withoutWiki.length === 0) {
console.log(' None! All ECO roots have Wikipedia articles.');
} else {
withoutWiki.forEach(o => {
const isVariation = o.name.includes(':');
const marker = isVariation ? ' └─' : ' ';
console.log(`${marker} ${o.eco.padEnd(4)} ${o.name}`);
});
}
console.log();
console.log('━'.repeat(70));
console.log(`Coverage: ${withWiki.length}/${withWiki.length + withoutWiki.length} ECO roots (${Math.round(withWiki.length / (withWiki.length + withoutWiki.length) * 100)}%)`);
console.log('━'.repeat(70));