Proxy & Async Logging

## The idea Most observability tools ask you to install an SDK and wrap your call sites. Helicone's headline integration asks you to change a URL. It runs a hosted reverse proxy that speaks the provider's own API shape, so an existing client keeps working when you point it at Helicone instead of the provider. For OpenAI-shaped traffic the proxy host is `https://oai.helicone.ai/v1`. Helicone publishes a host per provider (for example `https://anthropic.helicone.ai` for Anthropic's API); the pattern is the same in each case — same paths, same request bodies, same responses. ## The two credentials A proxied call carries two authorizations, and confusing them is the classic failure: - `Authorization: Bearer <PROVIDER_KEY>` — your OpenAI key. Helicone passes it through so the provider can bill and authenticate you. - `Helicone-Auth: Bearer <HELICONE_API_KEY>` — your Helicone key. It tells Helicone which account and project the log belongs to. If you omit `Helicone-Auth`, the request is not attributable to your account. If you put the Helicone key in `Authorization`, OpenAI rejects it as an invalid key. Note that the Helicone key value itself includes the `Bearer ` prefix — it is a normal bearer-style header, not a bare token field. ## What Helicone records Because the proxy sits between you and the provider, it sees the whole exchange and needs no cooperation from your code to log: - the request body, including the rendered prompt and any tool definitions; - the response body, including the completion; - wall-clock latency as measured at the proxy; - the model, prompt/completion token counts, and a cost derived from the model's published price; - the status code, so failures and provider errors show up beside successes. That is where the cost dashboards come from: Helicone multiplies observed token usage by a per-model rate table, so "spend by model", "spend by day" and (once you add custom property headers) "spend by feature or customer" all fall out of the same log rows without you emitting a single metric. One accuracy caveat worth knowing: streamed responses only carry a usage block if you ask the provider to include it. When usage is absent, the cost number for that row is derived rather than reported, so a heavily streamed workload can drift from the provider's own invoice. ## Setting it up in a typical client Every mainstream client library has a base-URL option and a default-headers option, which is all the integration needs. Prefer setting the base URL from configuration rather than hardcoding it — you want the ability to point straight at the provider again without a code change (that becomes important the day the proxy is unhealthy). Setting the Helicone header as a client default means every call made through that client is logged. You can still add or override headers per request when you want to attach call-specific metadata. ## What being in the path buys and costs The price of the one-line setup is that a third party is now inside your critical request path: one more network hop of latency and one more thing that can be down. The benefit is symmetrical — because Helicone handles the request before the provider does, it can *change* what happens to it (serving a stored response, applying a policy, retrying a failure). Logging alone does not require the proxy; behaviour-changing gateway features do. That tradeoff, not the setup itself, is what interviewers actually push on: the setup is a URL and a header, and the interesting question is when you are willing to accept the dependency. ## Verifying it works After the swap, a successful call looks identical to your application — same response object, same latency order of magnitude. The check is on Helicone's side: the request should appear in the dashboard within seconds, with a model, a token count and a cost attached. If calls succeed but nothing appears, the `Helicone-Auth` header is the first thing to inspect; if calls start failing with an auth error from the provider, the two keys have been crossed.