Skip to content

EvidenceLens is a research tool — not medical advice. Open Payments lookups are computed from public records via fuzzy matching and may contain false positives. Always verify against primary sources.

Release notes

Changelog

Selected highlights below. GitHub Releases is the canonical, complete history.

Scoring
  • XGBoost LTR model replaces static RRF weights — trained on 12 weeks of click-through data with 11 feature groups (BM25, vector, citation, recency, author authority, journal impact, funding source, study type, COI flag, result wave, query intent).
  • A/B experiment system live in Postgres — experiments table drives per-request variation assignment with hash-based deterministic bucketing.
  • Interleaving evaluator compares ranking configs online using team-draft interleaving on live traffic.
Agent
  • Agent SSE proxy supports tool-use: search, lookup-payments, fetch-full-text, and cite now stream back as structured server events.
  • Multi-turn conversation memory stored in Redis with TTL-based eviction (configurable 15–120 min).
  • Ollama backend support for local LLM inference — no API key needed for agentic queries.
  • WebLLM quantized (Q4) 1.5B model loads on-demand in-browser for fully offline question answering.
MCP
  • MCP server v2025-06 compliant over both stdio and HTTP+SSE transports.
  • Tools exposed: evidencelens_search (hybrid query), evidencelens_get_document (full text + metadata), evidencelens_get_citations (forward/backward graph), evidencelens_lookup_payments (Open Payments).
  • MCP gateway integration lets Claude Desktop and Cline query the corpus without API keys.
API
  • Gateway publishes REST + GraphQL + WebSocket endpoints — GraphQL supports subscriptions for real-time search result streaming.
  • BYOK proxy middleware added: requests to /api/byok/* route through the user's own Anthropic / OpenAI / Ollama key with per-request rate limiting.
  • OpenAPI 3.1 spec published at /api/docs with try-it-out support.
  • Turnstile captcha middleware (optional) protects /api/search and /api/byok from scripted abuse.
Frontend
  • COI badge component surfaces Open Payments data inline with search results — color-coded by payment tier ($0, $1–$10K, $10K+).
  • Three-wave result rendering: first wave (5 results) paints after ~300 ms, second wave (10) at ~1 s, final wave (35) at ~3 s.
  • Citation graph explorer renders forward/reference links as an interactive directed graph using D3 force simulation.
  • Dark mode respects system preference and persists manual toggle to localStorage.
Pipeline
  • Processor pipeline handles full-text extraction from PubMed XML, arXiv TeX, ClinicalTrials.gov HTML, and FDA drug label PDFs.
  • Open Payments ingester indexes 17M+ records from the CMS NPPES dump — physician name matching uses a three-pass strategy (exact, normalized, fuzzy via token-Jaccard).
  • Deduplication merge in the processor collapses duplicate PMID–DOI pairs, keeping the richest metadata variant.
  • BGE-M3 embedder runs on CPU with ONNX Runtime by default; GPU acceleration via NVIDIA Container Toolkit is opt-in with --profile gpu.
Ingest
  • All 29 ingesters reach production: PubMed (EFetch), arXiv, bioRxiv, medRxiv, ClinicalTrials.gov, EU CTIS, FDA Drugs@FDA, FDA Adverse Events, NIH RePORTER, Open Payments, WHO ICTRP, Cochrane, Crossref, PubMed Central, Figshare, Zenodo, Dryad, ANDS, OpenAIRE, Europe PMC, Gene Expression Omnibus, dbGaP, BioProject, SRA, UniProt, PDB, ClinicalTrials.gov results, FDA Orange Book, and FDA Purple Book.
  • Ingesters publish to NATS JetStream with dedup keys and idempotent consumer semantics.
  • Ofelia cron sidecar schedules ingesters on configurable intervals — PubMed every 6 h, trials every 12 h, FDA daily, others weekly.
Launch
  • EvidenceLens goes public as a free, open-source biomedical evidence search engine.
  • Single docker compose up brings up the full stack: ingesters, processor, embedder, indexer, scorer, gateway, agent, MCP server, frontend, Postgres, NATS, Meilisearch, Milvus, Neo4j, Redis, and ofelia.
  • Initial corpus: ~42M documents across all sources with 1024-d BGE-M3 embeddings.
  • Search delivers hybrid (BM25 + vector + citation + recency) results with WebSocket streaming, CMS Open Payments COI badges, and a BYOK agent layer.