8017ded3ba
fix(adapters): shared retry helper + run_log failure_class + enable RSS (issues #1 #2 #9 )
...
- adapters/__init__.py: add http_get() unified retry (429/5xx only, max 2
attempts, capped exp backoff) + AdapterHTTPError carrying failure_class;
SourceAdapter.last_failure_class set on failure for pipeline capture.
- arxiv/github/huggingface/hackernews/reddit: route HTTP through http_get.
Preserves GitHub 403 rate-limit retry and Reddit 403/429 fast-bail.
- schema.sql + pipeline.py: add run_log.failure_class column; rollup most-
severe class across sources (5xx>4xx>429>error>zero_fetch>ok).
- pipeline.py: ENABLE RSS in ENABLED_SOURCES (was registered, disabled).
- RSS smoke test surfaced 3 broken feeds (anthropic 404, googleai 404,
metaai 301) — left as-is, captured in feed_failures; URL fix is separate
discovery task, not guessed.
Verified: full dry-run fetches all 6 sources; github live fetch OK;
Reddit 429 fast-bail preserved; no import/syntax errors.
2026-07-10 16:34:05 +00:00
6396b66b45
Add Hacker News adapter + wire into pipeline
...
- adapters/hackernews.py: HN Firebase API adapter with AI keyword filtering
- Word-boundary matching to avoid substring traps (Britain/Guinea)
- Score: log(points) + log(comments), actual HN scores
- Wired into ENABLED_SOURCES + verification in pipeline.py
- Live test: 10 AI/ML stories fetched, all clean
2026-07-08 05:32:28 +00:00
Epictetus