BYOK synthesis
EvidenceLens runs at $0 server-side LLM cost. The search engine is free and needs no key. To get a conversational synthesis on top of your results, you supply your own provider key. We proxy the request, stream the response back, and never store the key.
Supported providers
The settings panel offers a key field for:
- Anthropic — Claude models, with prompt caching enabled by default.
- OpenAI — GPT-4o / GPT-4o-mini.
- Groq — Llama 3.3 70B and others. Groq issues a free key with no card, so it's the cheapest way to try BYOK.
The proxy also accepts other OpenAI-compatible endpoints (OpenRouter, Together, DeepInfra) and a self-hosted Ollama instance. If you don't want to use a key at all, switch to the WebLLM tier — inference runs entirely in your browser.
How it works
- You paste your key in the settings panel. It is stored only in your browser's
localStorageand never sent to EvidenceLens for storage. - Each synthesis sends the key over TLS to
POST /llm/synthesizeon the gateway, as anAuthorization: Bearerheader alongsidex-providerand an optionalx-model. The gateway forwards to the agent service, which calls your chosen provider and streams Server-Sent Events back. - The agent validates the key on first use with a cheap probe, then caches the result for 10 minutes keyed by
SHA-256(provider + key). Only provider, model, token counts, latency, and error codes are written to telemetry — never the key itself.
Why BYOK
This is a free public service on a small budget. Server-side LLM costs would scale with traffic and force aggressive rate limits or a shutdown. BYOK keeps the search engine permanently free while letting power users opt into whatever model and budget they prefer.
Privacy
Keys live in your browser and the in-memory request lifecycle of the proxy. They are not written to disk, not logged, not shared with third parties. If you'd rather not trust the proxy at all, use WebLLM and the entire interaction stays on your machine.
Rate limits
Synthesis is limited to 30 requests per minute per IP.