Commit Graph

7 Commits

Author SHA1 Message Date
Epictetus 23cce4d609 fix(adapters): shared retry helper + run_log failure_class + enable RSS (issues #1 #2 #9)
- adapters/__init__.py: add http_get() unified retry (429/5xx only, max 2
  attempts, capped exp backoff) + AdapterHTTPError carrying failure_class;
  SourceAdapter.last_failure_class set on failure for pipeline capture.
- arxiv/github/huggingface/hackernews/reddit: route HTTP through http_get.
  Preserves GitHub 403 rate-limit retry and Reddit 403/429 fast-bail.
- schema.sql + pipeline.py: add run_log.failure_class column; rollup most-
  severe class across sources (5xx>4xx>429>error>zero_fetch>ok).
- pipeline.py: ENABLE RSS in ENABLED_SOURCES (was registered, disabled).
- RSS smoke test surfaced 3 broken feeds (anthropic 404, googleai 404,
  metaai 301) — left as-is, captured in feed_failures; URL fix is separate
  discovery task, not guessed.

Verified: full dry-run fetches all 6 sources; github live fetch OK;
Reddit 429 fast-bail preserved; no import/syntax errors.
2026-07-10 16:38:57 +00:00
Epictetus 7ee1af3d7b Disable RSS adapter from ENABLED_SOURCES (back to 5 approved sources); keep code for future use 2026-07-08 18:03:37 +00:00
Epictetus 2c6701f5a3 Add RSS Feeds adapter (6th source) — commercial AI news from 10 feeds
Bug fix: date parsing was ISO-only (RFC 2822 feeds filtered out).
Added email.utils.parsedate_to_datetime() fallback for RSS dates.

10 feeds: TechCrunch AI, VentureBeat AI, The Verge AI, AI News,
The Decoder, MIT Tech Review AI, OpenAI Blog, Anthropic, Google AI, Meta AI.

3 dead feeds (Anthropic/Google/Meta 404), 7 working.
Score: authority (primary blogs 1.5x, industry 1.3x) × recency decay (48h half-life)
Age cutoff: 14 days. Score type: estimated.

Wired into pipeline.py ENABLED_SOURCES. Added RSS URL verification.
3/3 spot-checks passed (all URLs reachable, HTTP 200).

Commit: 62031be→c24a89f
2026-07-08 15:27:59 +00:00
Epictetus 4ba270c166 Add Hugging Face adapter — model releases & adoption signal
- adapters/huggingface.py: HF API adapter (models + datasets)
- Dual fetch: sort=likes (popularity) + sort=lastModified (fresh)
- AI relevance filter: pipeline_tag, library_name, tag matching
- Score: adoption (likes/downloads log-scale) + relevance (pipeline/library/tags)
- score_type: actual (real likes/downloads from HF API)
- Cross-source signal: GLM-5.2 top on both HN and HF
- Wired into ENABLED_SOURCES + verification in pipeline.py
- Live verify: DB likes match live API exactly (GLM-5.2: 3607, DeepSeek-R1: 13448)
- 99 total entries across 5 sources, all pipeline green
2026-07-08 05:54:27 +00:00
Epictetus 6396b66b45 Add Hacker News adapter + wire into pipeline
- adapters/hackernews.py: HN Firebase API adapter with AI keyword filtering
- Word-boundary matching to avoid substring traps (Britain/Guinea)
- Score: log(points) + log(comments), actual HN scores
- Wired into ENABLED_SOURCES + verification in pipeline.py
- Live test: 10 AI/ML stories fetched, all clean
2026-07-08 05:32:28 +00:00
Epictetus b51be6ed48 Phase 5: cron entry script, soft-cap archive, run_log zero-fetch degradation
- oracle-pipeline.sh: single cron entry point (pipeline -> summarize -> archive)
- archive.py: soft-cap archival to entries_archive (preserve, not delete)
- pipeline.py: record zero-fetch (rate-limited) runs as degraded in run_log.notes
Verified: full script runs exit 0, run_log captures per-source status.
2026-07-08 04:07:54 +00:00
Epictetus 67c002b665 Initial commit: Oracle AI research pipeline (adapters, pipeline, summarize, query)
Source-controlled baseline before Phase 5 cron. Excludes oracle.db,
logs/, and __pycache__ via .gitignore. Pipeline verified running
clean end-to-end (run_log write confirmed before conn.close()).
2026-07-08 04:03:36 +00:00