Commit Graph

3 Commits

Author SHA1 Message Date
Epictetus 641d531d88 Sprint 3: Anti-bot retrieval layer + metrics module
Anti-bot changes (all 6 adapters):
- Browser-grade User-Agent rotation (Chrome/Firefox on Linux/Windows)
- Shared browser_headers() with Accept, Accept-Language, DNT
- Session-consistent UA fingerprint (picked once, not per-request)
- jitter_sleep() replaces fixed time.sleep() on all adapters
- Exponential backoff on 429/503 already on reddit, now consistent

New shared module:
- adapters/__init__.py: browser_user_agent(), browser_headers(), jitter_sleep()
- adapters/_http.py: HTTPClient class for future browser-mode adapters

Metrics module (from Sprint 2 carry):
- oracle/metrics.py: MetricsRun for log_adapter/log_verdicts/log_scores
- oracle/weekly.py: SYSTEM HEALTH section wired to adapter_health
- oracle/cli.py: metrics subparser with --adapters/--publish/--scores/--alerts

Before: bot signatures like 'ai-oracle/0.1', 'python:athena:v0.1'
After: 'Mozilla/5.0 (X11; Linux x86_64; rv:139.0) Gecko/20100101 Firefox/139.0'
2026-07-22 14:26:42 +00:00
Epictetus 424da91364 FIX: first_seen re-stamp bug — persist true source publish date, idempotent upsert, backfill DB 2026-07-13 22:16:59 +00:00
Epictetus 4ba270c166 Add Hugging Face adapter — model releases & adoption signal
- adapters/huggingface.py: HF API adapter (models + datasets)
- Dual fetch: sort=likes (popularity) + sort=lastModified (fresh)
- AI relevance filter: pipeline_tag, library_name, tag matching
- Score: adoption (likes/downloads log-scale) + relevance (pipeline/library/tags)
- score_type: actual (real likes/downloads from HF API)
- Cross-source signal: GLM-5.2 top on both HN and HF
- Wired into ENABLED_SOURCES + verification in pipeline.py
- Live verify: DB likes match live API exactly (GLM-5.2: 3607, DeepSeek-R1: 13448)
- 99 total entries across 5 sources, all pipeline green
2026-07-08 05:54:27 +00:00