641d531d88
Sprint 3: Anti-bot retrieval layer + metrics module
...
Anti-bot changes (all 6 adapters):
- Browser-grade User-Agent rotation (Chrome/Firefox on Linux/Windows)
- Shared browser_headers() with Accept, Accept-Language, DNT
- Session-consistent UA fingerprint (picked once, not per-request)
- jitter_sleep() replaces fixed time.sleep() on all adapters
- Exponential backoff on 429/503 already on reddit, now consistent
New shared module:
- adapters/__init__.py: browser_user_agent(), browser_headers(), jitter_sleep()
- adapters/_http.py: HTTPClient class for future browser-mode adapters
Metrics module (from Sprint 2 carry):
- oracle/metrics.py: MetricsRun for log_adapter/log_verdicts/log_scores
- oracle/weekly.py: SYSTEM HEALTH section wired to adapter_health
- oracle/cli.py: metrics subparser with --adapters/--publish/--scores/--alerts
Before: bot signatures like 'ai-oracle/0.1', 'python:athena:v0.1'
After: 'Mozilla/5.0 (X11; Linux x86_64; rv:139.0) Gecko/20100101 Firefox/139.0'
2026-07-22 14:26:42 +00:00
6396b66b45
Add Hacker News adapter + wire into pipeline
...
- adapters/hackernews.py: HN Firebase API adapter with AI keyword filtering
- Word-boundary matching to avoid substring traps (Britain/Guinea)
- Score: log(points) + log(comments), actual HN scores
- Wired into ENABLED_SOURCES + verification in pipeline.py
- Live test: 10 AI/ML stories fetched, all clean
2026-07-08 05:32:28 +00:00
Epictetus