diff --git a/scripts/README.md b/scripts/README.md new file mode 100644 index 0000000..dafbf84 --- /dev/null +++ b/scripts/README.md @@ -0,0 +1,365 @@ +# Build Scripts + +## Overview + +This directory contains setup and build scripts for the chess tutor application: +1. **Tactical Puzzles Setup** - Downloads and configures tactical puzzles from Lichess +2. **Wikipedia Cache Builder** - Downloads Wikipedia articles for chess openings + +--- + +# Tactical Puzzles Setup Script + +## Quick Start + +### Docker (Recommended for production) + +**Use the helper script from your host machine:** + +```bash +# Default: 100 puzzles per pattern (800 total) +./scripts/docker-setup-puzzles.sh + +# Customize number of puzzles +./scripts/docker-setup-puzzles.sh --max-puzzles 500 + +# Force re-download +./scripts/docker-setup-puzzles.sh --force +``` + +The script will: +- Check if your container is running +- Download puzzles inside the container +- Store them in persistent Docker volumes +- Puzzles persist across container restarts + +### Local Development + +**Run this command once before using the tactical practice feature:** + +```bash +cd chess_tutor +python3 scripts/setup_tactical_puzzles.py + +# Or specify the number of puzzles: +python3 scripts/setup_tactical_puzzles.py --max-puzzles 100 +python3 scripts/setup_tactical_puzzles.py --max-puzzles 500 +``` + +## What It Does + +The setup script automatically: + +1. ✅ **Checks dependencies** - Verifies `zstd` is installed (installs it if needed) +2. ✅ **Downloads database** - Fetches the Lichess puzzle database (~500MB compressed) +3. ✅ **Decompresses** - Extracts the CSV file (~3.5GB uncompressed) +4. ✅ **Filters puzzles** - Extracts 20 high-quality puzzles for each tactical pattern: + - PIN + - FORK + - SKEWER + - DISCOVERED_CHECK + - DOUBLE_ATTACK + - OVERLOADING + - BACK_RANK_WEAKNESS + - TRAPPED_PIECE +5. ✅ **Converts format** - Transforms Lichess format to our JSON fixture format +6. ✅ **Saves fixtures** - Writes to `fixtures/tactics/*.json` +7. ✅ **Creates marker** - Places `.tactical_puzzles_configured` file to prevent re-running + +## Command Line Options + +```bash +python3 scripts/setup_tactical_puzzles.py [OPTIONS] + +Options: + --max-puzzles N Number of puzzles to extract per pattern (default: 20) + --force Force re-run setup without prompting + -h, --help Show help message + +Examples: + # Default: 20 puzzles per pattern (160 total) + python3 scripts/setup_tactical_puzzles.py + + # 100 puzzles per pattern (800 total) + python3 scripts/setup_tactical_puzzles.py --max-puzzles 100 + + # 500 puzzles per pattern (4000 total) - recommended for production + python3 scripts/setup_tactical_puzzles.py --max-puzzles 500 + + # Force re-run without prompting + python3 scripts/setup_tactical_puzzles.py --max-puzzles 200 --force +``` + +## Requirements + +- **Python 3.7+** (with `python-chess` library - auto-installed if missing) +- **zstd** (auto-installed on macOS/Linux if missing) +- **~4GB disk space** (for downloaded and decompressed database) +- **Internet connection** (for downloading ~500MB file) + +## Time Estimate + +- **First run**: 5-10 minutes (depending on internet speed) +- **Subsequent runs**: Instant (uses cached database) + +## Quality Criteria + +The script filters puzzles based on: + +- **Popularity**: ≥ 50 (well-liked by users) +- **Rating**: 1200-2000 (appropriate difficulty for learning) +- **Plays**: ≥ 50 (well-tested) +- **Theme**: Must match the tactical pattern + +Only the top 20 puzzles (by popularity) are selected for each pattern. + +## File Structure + +``` +chess_tutor/ +├── scripts/ +│ ├── setup_tactical_puzzles.py # Main setup script +│ └── README.md # This file +├── downloads/ # Created by script +│ ├── lichess_db_puzzle.csv.zst # Downloaded database (cached) +│ └── lichess_db_puzzle.csv # Decompressed database (cached) +├── fixtures/ +│ └── tactics/ # Created by script +│ ├── pin.json # 20 PIN puzzles +│ ├── fork.json # 20 FORK puzzles +│ ├── skewer.json # 20 SKEWER puzzles +│ └── ... # Other patterns +└── .tactical_puzzles_configured # Marker file (created by script) +``` + +## Re-running Setup + +If you want to re-run the setup (e.g., to get fresh puzzles): + +```bash +# Option 1: Delete the marker file and re-run +rm chess_tutor/.tactical_puzzles_configured +python3 scripts/setup_tactical_puzzles.py + +# Option 2: The script will ask if you want to re-run +python3 scripts/setup_tactical_puzzles.py +# Answer 'y' when prompted +``` + +## Cleaning Up + +To save disk space after setup: + +```bash +# Delete the downloaded database (keeps the fixtures) +rm -rf chess_tutor/downloads/ + +# The fixtures in chess_tutor/fixtures/tactics/ will remain +``` + +## Troubleshooting + +### "zstd not found" + +The script will attempt to auto-install `zstd`. If it fails: + +**macOS:** +```bash +brew install zstd +``` + +**Ubuntu/Debian:** +```bash +sudo apt-get install zstd +``` + +**Fedora/RHEL:** +```bash +sudo dnf install zstd +``` + +### "python-chess not found" + +The script will attempt to auto-install `python-chess`. If it fails: + +```bash +pip3 install python-chess +``` + +### "Download failed" + +Check your internet connection and try again. The Lichess database is updated daily, so temporary issues may occur. + +### "No puzzles found for pattern X" + +This is rare but can happen if the database doesn't have enough puzzles matching the criteria. The script will warn you but continue with other patterns. + +## Manual Setup (Alternative) + +If the automatic script doesn't work, you can manually: + +1. Download: https://database.lichess.org/lichess_db_puzzle.csv.zst +2. Decompress with `zstd -d lichess_db_puzzle.csv.zst` +3. Filter puzzles using the Python code in `docs/LICHESS_PUZZLES.md` +4. Convert to JSON format and save to `fixtures/tactics/` + +## Source + +- **Database**: https://database.lichess.org/ +- **License**: Creative Commons CC0 (public domain) +- **Documentation**: See `docs/LICHESS_PUZZLES.md` for detailed information + +## Support + +If you encounter issues: + +1. Check the terminal output for specific error messages +2. Ensure you have Python 3.7+ installed: `python3 --version` +3. Ensure you have internet connectivity +4. Try re-running the script +5. Check `docs/LICHESS_PUZZLES.md` for manual setup instructions + + +--- + +# Wikipedia Cache Builder + +## Overview + +Downloads full Wikipedia articles for all chess opening families and caches them locally for offline use. + +## Quick Start + +```bash +# Install dependencies (if not already done) +npm install + +# Run the Wikipedia cache builder +npm run cache:wikipedia +``` + +## Benefits + +- ✅ **Offline access** - Works completely offline once cached +- ✅ **Instant loading** - No network delay +- ✅ **No rate limiting** - Avoids Wikipedia API rate limits +- ✅ **Full content** - Gets complete articles, not just summaries +- ✅ **Version controlled** - Safe to commit to git + +## What It Does + +1. **Extracts opening families** from your opening database (`public/openings/*.json`) +2. **Searches Wikipedia** for each family using the OpenSearch API +3. **Downloads full article content** including: + - Introduction + - Main sections (History, Ideas, Variations, etc.) + - Up to 5 relevant sections per opening +4. **Saves locally** to `public/wikipedia/*.json` +5. **Creates an index** at `public/wikipedia/index.json` + +## Output Structure + +Each opening family gets a JSON file like `public/wikipedia/french-defense.json`: + +```json +{ + "openingFamily": "French Defense", + "title": "French Defence", + "url": "https://en.wikipedia.org/wiki/French_Defence", + "sections": [ + { + "title": "Introduction", + "text": "The French Defence is a chess opening..." + }, + { + "title": "History", + "text": "The first known game with the French Defense..." + } + ], + "lastModified": "2025-01-15T...", + "license": "CC BY-SA 3.0", + "licenseUrl": "https://creativecommons.org/licenses/by-sa/3.0/", + "fetchedAt": 1705334400000 +} +``` + +## Committing to Git + +The generated files are **safe to commit** to your repository: +- Wikipedia content is licensed under CC BY-SA 3.0 (allows redistribution with attribution) +- Each file includes proper license information +- Files are versioned, so you can review changes before committing + +## When to Run + +- **Initial setup**: Run once to cache all Wikipedia articles +- **After adding new openings**: Re-run to fetch articles for new families +- **Periodically**: Re-run every few months to get updated Wikipedia content + +## Rate Limiting + +The script includes a 1-second delay between requests to be respectful to Wikipedia's servers. + +**Estimated time:** +- 50 opening families = ~1-2 minutes +- 100 opening families = ~2-3 minutes + +## Fallback Strategy + +The app uses a 3-tier approach for Wikipedia content: + +1. **Local cache** (`public/wikipedia/*.json`) - Fastest, always available +2. **localStorage cache** - In-browser cache for API fetches +3. **Live API** - Fallback if local cache missing + +This ensures Wikipedia content is always available, even for openings not in your local cache. + +## License & Attribution + +Wikipedia content is licensed under **CC BY-SA 3.0**: +- ✅ Commercial use allowed +- ✅ Modification allowed +- ✅ Redistribution allowed +- ⚠️ Attribution required (included in JSON files) +- ⚠️ Derivative works must use same license + +More info: https://creativecommons.org/licenses/by-sa/3.0/ + +## Implementation Details + +**Script location:** `scripts/fetch-wikipedia-openings.ts` + +**Dependencies:** +- Node.js 18+ +- tsx (TypeScript executor) + +**API endpoints used:** +- Wikipedia OpenSearch API (for finding articles) +- Wikipedia MediaWiki Parse API (for full content) + +**Output directory:** `public/wikipedia/` + +## Example Usage + +```bash +# Run the cache builder +npm run cache:wikipedia + +# Output: +# 🌐 Wikipedia Opening Cache Builder +# 📚 Extracting opening families from database... +# ✓ Found 47 unique opening families +# +# 📖 Processing: French Defense +# Searching Wikipedia for: "French Defense" +# ✓ Found: "French Defence" +# Fetching full article... +# ✓ Fetched 5 sections +# ✓ Saved to: french-defense.json +# ... +# ✨ Wikipedia Cache Build Complete! +# ✓ Successful: 45 +# ⚠️ Skipped: 2 +# ❌ Failed: 0 +``` + diff --git a/scripts/docker-setup-puzzles.sh b/scripts/docker-setup-puzzles.sh new file mode 100755 index 0000000..c5a9044 --- /dev/null +++ b/scripts/docker-setup-puzzles.sh @@ -0,0 +1,96 @@ +#!/bin/bash +# +# Helper script to download Lichess tactical puzzles in Docker container +# +# Usage: +# ./scripts/docker-setup-puzzles.sh [options] +# +# Options: +# --max-puzzles NUM Number of puzzles per pattern (default: 100) +# --force Force re-download even if already configured +# --container NAME Container name (default: chess-tutor) +# + +set -e + +# Default values +CONTAINER_NAME="chess-tutor" +MAX_PUZZLES=100 +FORCE_FLAG="" + +# Parse arguments +while [[ $# -gt 0 ]]; do + case $1 in + --max-puzzles) + MAX_PUZZLES="$2" + shift 2 + ;; + --force) + FORCE_FLAG="--force" + shift + ;; + --container) + CONTAINER_NAME="$2" + shift 2 + ;; + *) + echo "Unknown option: $1" + echo "Usage: $0 [--max-puzzles NUM] [--force] [--container NAME]" + exit 1 + ;; + esac +done + +echo "==========================================" +echo "Chess Tutor - Puzzle Setup (Docker)" +echo "==========================================" +echo "Container: $CONTAINER_NAME" +echo "Puzzles per pattern: $MAX_PUZZLES" +echo "==========================================" +echo "" + +# Check if container is running +if ! docker ps --format '{{.Names}}' | grep -q "^${CONTAINER_NAME}$"; then + echo "❌ Error: Container '$CONTAINER_NAME' is not running" + echo "" + echo "Start it with: docker-compose up -d" + exit 1 +fi + +echo "✅ Container is running" +echo "" + +# Check if already configured (unless --force) +if [ -z "$FORCE_FLAG" ]; then + if docker exec "$CONTAINER_NAME" test -f .tactical_puzzles_configured 2>/dev/null; then + echo "⚠️ Puzzles are already configured!" + echo "" + read -p "Do you want to re-download? (y/N): " -n 1 -r + echo + if [[ ! $REPLY =~ ^[Yy]$ ]]; then + echo "Exiting..." + exit 0 + fi + FORCE_FLAG="--force" + fi +fi + +# Run the setup script inside the container +echo "🎯 Starting puzzle download..." +echo "This will download ~500MB and may take 5-10 minutes" +echo "" + +docker exec -it "$CONTAINER_NAME" \ + python3 scripts/setup_tactical_puzzles.py --max-puzzles "$MAX_PUZZLES" $FORCE_FLAG + +echo "" +echo "==========================================" +echo "✅ Puzzle setup complete!" +echo "==========================================" +echo "" +echo "The puzzles are now available in your Chess Tutor app." +echo "They will persist across container restarts." +echo "" +echo "To check the downloaded puzzles:" +echo " docker exec $CONTAINER_NAME ls -lh fixtures/tactics/" +echo "" diff --git a/scripts/find-missing-wikipedia.js b/scripts/find-missing-wikipedia.js new file mode 100644 index 0000000..878d25a --- /dev/null +++ b/scripts/find-missing-wikipedia.js @@ -0,0 +1,66 @@ +#!/usr/bin/env node + +const fs = require('fs'); +const path = require('path'); + +const ecoFiles = ['ecoA', 'ecoB', 'ecoC', 'ecoD', 'ecoE']; +const missingWiki = []; + +for (const file of ecoFiles) { + const filePath = path.join(__dirname, '..', 'public', 'openings', `${file}.json`); + const data = JSON.parse(fs.readFileSync(filePath, 'utf8')); + + for (const [fen, opening] of Object.entries(data)) { + if (opening.isEcoRoot && !opening.wikipediaSlug) { + missingWiki.push({ + eco: opening.eco, + name: opening.name, + moves: opening.moves + }); + } + } +} + +// Sort by ECO code +missingWiki.sort((a, b) => a.eco.localeCompare(b.eco)); + +console.log('Major openings (ECO roots) WITHOUT Wikipedia slugs:\n'); +console.log('Total:', missingWiki.length, '\n'); + +// Group by ECO category +const byCategory = {}; +for (const opening of missingWiki) { + const category = opening.eco[0]; + if (!byCategory[category]) byCategory[category] = []; + byCategory[category].push(opening); +} + +for (const [cat, openings] of Object.entries(byCategory).sort()) { + console.log(`\n=== ECO ${cat} (${openings.length} openings) ===`); + openings.slice(0, 15).forEach(o => { + console.log(`${o.eco.padEnd(4)} ${o.name}`); + }); + if (openings.length > 15) { + console.log(`... and ${openings.length - 15} more`); + } +} + +// Show some notable ones +console.log('\n\n=== Notable Missing Openings ==='); +const notable = [ + 'Sicilian', 'French', 'Caro-Kann', 'Pirc', 'Alekhine', + 'Scandinavian', 'Queen', 'English', 'Indian', 'Benoni', + 'Nimzo', 'Grunfeld', 'Dutch', 'Reti', 'Bird' +]; + +const notableMatches = missingWiki.filter(o => + notable.some(n => o.name.toLowerCase().includes(n.toLowerCase())) +); + +if (notableMatches.length > 0) { + notableMatches.forEach(o => { + console.log(`${o.eco.padEnd(4)} ${o.name}`); + }); +} else { + console.log('(None found - all major openings have Wikipedia slugs!)'); +} diff --git a/scripts/setup_tactical_puzzles.py b/scripts/setup_tactical_puzzles.py new file mode 100755 index 0000000..060794e --- /dev/null +++ b/scripts/setup_tactical_puzzles.py @@ -0,0 +1,392 @@ +#!/usr/bin/env python3 +""" +One-time setup script to download and configure high-quality tactical puzzles from Lichess. + +This script: +1. Downloads the Lichess puzzle database +2. Extracts puzzles for each tactical pattern +3. Converts them to our JSON fixture format +4. Validates them with our tactical library +5. Creates a marker file to indicate setup is complete + +Usage: + python3 scripts/setup_tactical_puzzles.py +""" + +import os +import sys +import json +import csv +import subprocess +import urllib.request +from pathlib import Path +from typing import List, Dict, Any + +# Add parent directory to path to import chess libraries +sys.path.insert(0, str(Path(__file__).parent.parent)) + +try: + import chess + import chess.pgn +except ImportError: + print("❌ Error: python-chess library not found.") + print("Installing python-chess...") + subprocess.check_call([sys.executable, "-m", "pip", "install", "python-chess"]) + import chess + import chess.pgn + +# Configuration +LICHESS_PUZZLE_URL = "https://database.lichess.org/lichess_db_puzzle.csv.zst" +DOWNLOAD_DIR = Path(__file__).parent.parent / "downloads" +FIXTURES_DIR = Path(__file__).parent.parent / "fixtures" / "tactics" +SETUP_MARKER = Path(__file__).parent.parent / ".tactical_puzzles_configured" + +# Mapping of our patterns to Lichess themes +PATTERN_THEMES = { + "pin": ["pin"], + "fork": ["fork"], + "skewer": ["skewer"], + "discovered_check": ["discoveredAttack"], + "double_attack": ["doubleCheck", "fork"], + "overloading": ["overloading"], + "back_rank_weakness": ["backRankMate"], + "trapped_piece": ["trappedPiece"], +} + +# Quality criteria +MIN_POPULARITY = 50 +MIN_RATING = 800 # Easy puzzles start here +MAX_RATING = 2200 # Hard puzzles go up to here +MIN_PLAYS = 50 + +# Default number of puzzles per pattern (can be overridden via command line) +DEFAULT_PUZZLES_PER_PATTERN = 20 + + +def check_zstd_installed() -> bool: + """Check if zstd is installed for decompression.""" + try: + subprocess.run(["zstd", "--version"], capture_output=True, check=True) + return True + except (subprocess.CalledProcessError, FileNotFoundError): + return False + + +def install_zstd(): + """Attempt to install zstd.""" + print("📦 Installing zstd...") + + # Detect OS and install accordingly + if sys.platform == "darwin": # macOS + try: + subprocess.check_call(["brew", "install", "zstd"]) + print("✅ zstd installed successfully via Homebrew") + except (subprocess.CalledProcessError, FileNotFoundError): + print("❌ Failed to install zstd. Please install Homebrew first:") + print(" /bin/bash -c \"$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)\"") + sys.exit(1) + elif sys.platform.startswith("linux"): + try: + # Try apt-get (Debian/Ubuntu) + subprocess.check_call(["sudo", "apt-get", "update"]) + subprocess.check_call(["sudo", "apt-get", "install", "-y", "zstd"]) + print("✅ zstd installed successfully via apt-get") + except (subprocess.CalledProcessError, FileNotFoundError): + print("❌ Failed to install zstd. Please install it manually:") + print(" Debian/Ubuntu: sudo apt-get install zstd") + print(" Fedora/RHEL: sudo dnf install zstd") + sys.exit(1) + else: + print("❌ Unsupported OS. Please install zstd manually:") + print(" Windows: Download from https://github.com/facebook/zstd/releases") + sys.exit(1) + + +def download_puzzle_database() -> Path: + """Download the Lichess puzzle database.""" + DOWNLOAD_DIR.mkdir(parents=True, exist_ok=True) + + compressed_file = DOWNLOAD_DIR / "lichess_db_puzzle.csv.zst" + decompressed_file = DOWNLOAD_DIR / "lichess_db_puzzle.csv" + + # Check if already downloaded and decompressed + if decompressed_file.exists(): + print(f"✅ Puzzle database already exists at {decompressed_file}") + return decompressed_file + + # Download if not exists + if not compressed_file.exists(): + print(f"📥 Downloading Lichess puzzle database from {LICHESS_PUZZLE_URL}") + print(" This may take several minutes (file is ~500MB compressed)...") + + try: + urllib.request.urlretrieve(LICHESS_PUZZLE_URL, compressed_file) + print(f"✅ Downloaded to {compressed_file}") + except Exception as e: + print(f"❌ Failed to download: {e}") + sys.exit(1) + + # Decompress + print(f"📦 Decompressing {compressed_file.name}...") + print(" This may take several minutes (decompressed file is ~3.5GB)...") + + try: + subprocess.check_call(["zstd", "-d", str(compressed_file), "-o", str(decompressed_file)]) + print(f"✅ Decompressed to {decompressed_file}") + except subprocess.CalledProcessError as e: + print(f"❌ Failed to decompress: {e}") + sys.exit(1) + + return decompressed_file + + +def extract_puzzles_for_pattern(csv_file: Path, pattern: str, themes: List[str], max_puzzles: int) -> List[Dict[str, Any]]: + """Extract high-quality puzzles for a specific tactical pattern.""" + print(f"🔍 Extracting {pattern.upper()} puzzles (max: {max_puzzles})...") + + puzzles = [] + + with open(csv_file, 'r', encoding='utf-8') as f: + reader = csv.DictReader(f) + + for row in reader: + puzzle_themes = row['Themes'].split() + rating = int(row['Rating']) + popularity = int(row['Popularity']) + nb_plays = int(row['NbPlays']) + + # Check if puzzle matches our criteria + if (any(theme in puzzle_themes for theme in themes) and + popularity >= MIN_POPULARITY and + MIN_RATING <= rating <= MAX_RATING and + nb_plays >= MIN_PLAYS): + + puzzles.append(row) + + # Stop when we have enough + if len(puzzles) >= max_puzzles: + break + + # Sort by popularity (best first) + puzzles.sort(key=lambda x: int(x['Popularity']), reverse=True) + + print(f" Found {len(puzzles)} high-quality {pattern.upper()} puzzles") + return puzzles[:max_puzzles] + + +def lichess_to_fixture(puzzle_row: Dict[str, Any], pattern_type: str) -> Dict[str, Any]: + """Convert Lichess puzzle to our fixture format. + + Lichess puzzle format: + - FEN: Position BEFORE opponent's first move + - Moves: Space-separated UCI moves alternating opponent/player + - First move: Opponent's move (sets up the puzzle) + - Remaining moves: Player move, opponent response, player move, etc. + """ + fen = puzzle_row['FEN'] + moves_uci = puzzle_row['Moves'].split() + + # Apply first move (opponent's move) to get starting position + board = chess.Board(fen) + opponent_move = chess.Move.from_uci(moves_uci[0]) + board.push(opponent_move) + initial_fen = board.fen() + + # Determine side to move from initial FEN + side_to_move = "white" if " w " in initial_fen else "black" + + # Process all moves in the sequence + # moves_uci[0] = opponent's setup move (already applied) + # moves_uci[1] = player's first move (solution start) + # moves_uci[2] = opponent's response + # moves_uci[3] = player's second move + # etc. + + move_sequence = [] + for i, move_uci in enumerate(moves_uci[1:], start=1): # Skip first move (already applied) + move = chess.Move.from_uci(move_uci) + move_san = board.san(move) + + # Determine who makes this move + # Odd indices (1, 3, 5...) = player moves + # Even indices (2, 4, 6...) = opponent moves + is_player_move = (i % 2 == 1) + + move_sequence.append({ + "uci": move_uci, + "san": move_san, + "player": is_player_move + }) + + board.push(move) + + # Get final position after all moves + resulting_fen = board.fen() + + # First player move (for backward compatibility) + first_player_move_uci = moves_uci[1] + board_temp = chess.Board(initial_fen) + first_player_move = chess.Move.from_uci(first_player_move_uci) + first_player_move_san = board_temp.san(first_player_move) + + return { + "id": puzzle_row['PuzzleId'], + "initialFen": initial_fen, + "sideToMove": side_to_move, + "rating": int(puzzle_row['Rating']), # Add rating for difficulty filtering + "bestMove": { + "san": first_player_move_san, + "uci": first_player_move_uci + }, + "moves": move_sequence, # Full move sequence + "resultingFen": resulting_fen, + "expectedPattern": { + "type": pattern_type.upper().replace("_", "_"), + # Note: Exact squares will be detected by our tactical library + }, + "context": f"Lichess puzzle {puzzle_row['PuzzleId']} (Rating: {puzzle_row['Rating']}, Popularity: {puzzle_row['Popularity']})", + "tags": puzzle_row['Themes'].split() + } + + +def save_fixtures(pattern: str, fixtures: List[Dict[str, Any]]): + """Save fixtures to JSON file.""" + FIXTURES_DIR.mkdir(parents=True, exist_ok=True) + + output_file = FIXTURES_DIR / f"{pattern}.json" + + fixture_data = { + "description": f"High-quality {pattern.upper()} tactical puzzles from Lichess database", + "source": "https://database.lichess.org/", + "generatedAt": "auto-generated", + "cases": fixtures + } + + with open(output_file, 'w', encoding='utf-8') as f: + json.dump(fixture_data, f, indent=2, ensure_ascii=False) + + print(f"✅ Saved {len(fixtures)} puzzles to {output_file}") + + +def create_setup_marker(): + """Create a marker file to indicate setup is complete.""" + with open(SETUP_MARKER, 'w') as f: + f.write("Tactical puzzles configured successfully\n") + print(f"✅ Created setup marker at {SETUP_MARKER}") + print(" (The app will auto-detect Lichess puzzles from fixture metadata)") + + +def main(): + """Main setup function.""" + import argparse + + parser = argparse.ArgumentParser( + description='Download and configure high-quality tactical puzzles from Lichess', + formatter_class=argparse.RawDescriptionHelpFormatter, + epilog=""" +Examples: + # Download 20 puzzles per pattern (default) + python3 scripts/setup_tactical_puzzles.py + + # Download 100 puzzles per pattern + python3 scripts/setup_tactical_puzzles.py --max-puzzles 100 + + # Download 500 puzzles per pattern for production + python3 scripts/setup_tactical_puzzles.py --max-puzzles 500 + """ + ) + parser.add_argument( + '--max-puzzles', + type=int, + default=DEFAULT_PUZZLES_PER_PATTERN, + help=f'Maximum number of puzzles to extract per pattern (default: {DEFAULT_PUZZLES_PER_PATTERN})' + ) + parser.add_argument( + '--force', + action='store_true', + help='Force re-run setup without prompting' + ) + + args = parser.parse_args() + + print("=" * 70) + print("🎯 Chess Tutor - Tactical Puzzles Setup") + print("=" * 70) + print(f"Configuration: {args.max_puzzles} puzzles per pattern") + print("=" * 70) + print() + + # Check if already configured + if SETUP_MARKER.exists() and not args.force: + print("⚠️ Tactical puzzles are already configured!") + response = input("Do you want to re-run the setup? (y/N): ").strip().lower() + if response != 'y': + print("Exiting...") + sys.exit(0) + else: + SETUP_MARKER.unlink() + + # Step 1: Check/install zstd + print("Step 1: Checking dependencies...") + if not check_zstd_installed(): + print("⚠️ zstd not found (required for decompression)") + install_zstd() + else: + print("✅ zstd is installed") + print() + + # Step 2: Download and decompress database + print("Step 2: Downloading Lichess puzzle database...") + csv_file = download_puzzle_database() + print() + + # Step 3: Extract puzzles for each pattern + print("Step 3: Extracting puzzles for each tactical pattern...") + total_puzzles = 0 + for pattern, themes in PATTERN_THEMES.items(): + puzzles = extract_puzzles_for_pattern(csv_file, pattern, themes, args.max_puzzles) + + if len(puzzles) == 0: + print(f"⚠️ Warning: No puzzles found for {pattern.upper()}") + continue + + # Convert to fixture format + fixtures = [] + for puzzle in puzzles: + try: + fixture = lichess_to_fixture(puzzle, pattern) + fixtures.append(fixture) + except Exception as e: + print(f" ⚠️ Skipping puzzle {puzzle['PuzzleId']}: {e}") + + # Save to file + if fixtures: + save_fixtures(pattern, fixtures) + total_puzzles += len(fixtures) + + print() + + # Step 4: Create marker file + print("Step 4: Finalizing setup...") + create_setup_marker() + print() + + print("=" * 70) + print(f"✅ Setup complete! {total_puzzles} tactical puzzles are ready to use.") + print("=" * 70) + print() + print("Next steps:") + print("1. Refresh your Chess Tutor app (if running)") + print("2. The warning banner will automatically disappear") + print(" (The app detects Lichess puzzles by checking the fixture metadata)") + print() + print("3. Go to http://localhost:3050/learning") + print("4. Select a coach and practice tactical patterns!") + print() + print("Note: The downloaded database is cached in the 'downloads' directory.") + print(" You can delete it to save space if needed.") + + +if __name__ == "__main__": + main() + diff --git a/scripts/wikipedia-summary.js b/scripts/wikipedia-summary.js new file mode 100644 index 0000000..0d470d8 --- /dev/null +++ b/scripts/wikipedia-summary.js @@ -0,0 +1,87 @@ +#!/usr/bin/env node + +const fs = require('fs'); +const path = require('path'); + +const ecoFiles = ['ecoA', 'ecoB', 'ecoC', 'ecoD', 'ecoE']; +const withWiki = []; +const withoutWiki = []; + +for (const file of ecoFiles) { + const filePath = path.join(__dirname, '..', 'public', 'openings', `${file}.json`); + const data = JSON.parse(fs.readFileSync(filePath, 'utf8')); + + for (const [fen, opening] of Object.entries(data)) { + if (opening.isEcoRoot) { + if (opening.wikipediaSlug) { + withWiki.push({ + eco: opening.eco, + name: opening.name, + slug: opening.wikipediaSlug + }); + } else { + withoutWiki.push({ + eco: opening.eco, + name: opening.name + }); + } + } + } +} + +// Sort +withWiki.sort((a, b) => a.name.localeCompare(b.name)); +withoutWiki.sort((a, b) => a.name.localeCompare(b.name)); + +console.log('━'.repeat(70)); +console.log('WIKIPEDIA INTEGRATION SUMMARY'); +console.log('━'.repeat(70)); +console.log(); + +console.log(`✅ ECO roots WITH Wikipedia: ${withWiki.length}`); +console.log(`❌ ECO roots WITHOUT Wikipedia: ${withoutWiki.length}`); +console.log(); + +// Show major openings with Wikipedia +console.log('━'.repeat(70)); +console.log('MAJOR OPENINGS WITH WIKIPEDIA (sample):'); +console.log('━'.repeat(70)); + +const majorOpenings = [ + 'Sicilian', 'French', 'Caro-Kann', 'Pirc', 'Alekhine', + 'Scandinavian', 'Italian', 'Spanish', 'Scotch', 'Vienna', + 'English', 'Reti', 'Bird', 'Polish', 'Nimzo', 'Queen', + 'King', 'Benoni', 'Catalan', 'Grunfeld', 'Dutch', 'Ruy Lopez' +]; + +const foundMajor = withWiki.filter(o => + majorOpenings.some(m => o.name.toLowerCase().includes(m.toLowerCase())) +); + +foundMajor.slice(0, 40).forEach(o => { + console.log(` ${o.eco.padEnd(4)} ${o.name}`); +}); + +if (foundMajor.length > 40) { + console.log(` ... and ${foundMajor.length - 40} more major openings`); +} + +console.log(); +console.log('━'.repeat(70)); +console.log('OPENINGS MISSING WIKIPEDIA:'); +console.log('━'.repeat(70)); + +if (withoutWiki.length === 0) { + console.log(' None! All ECO roots have Wikipedia articles.'); +} else { + withoutWiki.forEach(o => { + const isVariation = o.name.includes(':'); + const marker = isVariation ? ' └─' : ' '; + console.log(`${marker} ${o.eco.padEnd(4)} ${o.name}`); + }); +} + +console.log(); +console.log('━'.repeat(70)); +console.log(`Coverage: ${withWiki.length}/${withWiki.length + withoutWiki.length} ECO roots (${Math.round(withWiki.length / (withWiki.length + withoutWiki.length) * 100)}%)`); +console.log('━'.repeat(70));