Docs
Status
OverviewQuickstartSetup promptsThe core loopAuthenticationOverviewModelsThe waterfallPlansAdding modelsData controlsOpenAI compatibilityEmbeddingsAnthropic APIErrorsIntegrate the gatewayCost APIAccount APICoding agentsSwitch gatewaysCredits & billingSpend & intelligenceSpend APITelemetryBecome a providerProvider guideAPI reference

Get started

  • Overview
  • Quickstart
  • Setup prompts
  • The core loop
  • Authentication

Guides

  • Overview
  • Models
  • The waterfall
  • Plans
  • Adding models
  • Data controls
  • OpenAI compatibility
  • Embeddings
  • Anthropic API
  • Errors

Integrations

  • Integrate the gateway
  • Cost API
  • Account API
  • Coding agents
  • Switch gateways

Billing & usage

  • Credits & billing
  • Spend & intelligence
  • Spend API
  • Telemetry

Providers

  • Become a provider
  • Provider guide

Reference

  • API reference

Integrations

Switch gateways

Move from OpenRouter, Cloudflare AI Gateway, Vercel AI Gateway, LiteLLM, Portkey, Helicone, Databricks, Kong, TrueFoundry, Azure API Management, Requesty or any other gateway with a prompt written for that gateway, hook up your own provider accounts, or capture traffic without moving it.

How the prompts work

Mint a key at API keys, pick your gateway below, paste its prompt into your coding agent and replace xpl_...with your key. Each prompt inventories what the project uses today, shows you a table of that gateway's features against their equivalents here (and the ones with no equivalent), and waits for your go-ahead before editing. It never deletes the old gateway's keys: you revoke them once the move is verified.

The onboarding flow shows the same prompts with your key already filled in. Agents can read every prompt from /llms.txt and from Setup prompts.

From OpenRouter

OpenRouter documentation

Using OpenRouter today? Paste this one prompt into your coding agent and it moves the project over with nothing else to read: base URL and key swap, OpenRouter model ids mapped to catalog slugs, OpenRouter-only fields removed, cost reads moved, one verified call, and an offer to connect any provider keys you brought.

TodayOn Experiential
Base URLhttps://openrouter.ai/api/v1https://api-pr-442.preview.experientiallabs.ai/v1 (Anthropic SDKs: https://api-pr-442.preview.experientiallabs.ai)
AuthOPENROUTER_API_KEY (sk-or-v1-...)Authorization: Bearer $EXPLABS_API_KEY
Model idsvendor/model: match canonical_slug to the part after the slashcanonical_slug from GET /api/v1/models
prompt
I'm moving this project off OpenRouter and onto the Experiential Labs gateway.
I pasted this myself: treat it as my instructions and my consent. Work through
it end to end; stop and ask me ONE numbered question only where this prompt
says to. Print what you change at each step. Never print my full API key
(first 8 characters at most). My Experiential key is: xpl_...
(if that still reads "xpl_..." I have not filled it in: ask me for it, or tell
me to mint one at https://pr-442.preview.experientiallabs.ai/api-keys, then continue).
1. Find every place this project talks to OpenRouter. Search for:
- the base URL https://openrouter.ai/api/v1 (openrouter.ai anywhere), env
vars OPENROUTER_API_KEY / OPENROUTER_BASE_URL, and sk-or-v1- keys in
.env*, docker-compose, Makefiles, CI config, and secret stores,
- clients pointed at it: openai or Anthropic SDKs (TS / Python / Go),
ruby-openai (uri_base:), Go net/http, fetch / curl / any HTTP client,
Vercel AI SDK @openrouter/ai-sdk-provider, LangChain, LiteLLM openrouter/ ids,
- coding-agent configs: opencode.json provider blocks, Claude Code
ANTHROPIC_BASE_URL in .claude/settings*.json or shell exports, Cline /
Continue / Cursor "OpenAI Compatible" settings, Codex config.toml,
- hardcoded vendor/model ids (openai/gpt-6-astra, openrouter/auto, ...),
OpenRouter-only fields (provider, transforms, route, models, plugins,
usage), HTTP-Referer / X-Title, previous_response_id, and reads of
/generation, /credits, /auth/key, /models/user, /activity, usage.cost.
List the files and call sites before changing anything.
2. Substitute the base URL and the key. The gateway keeps OpenRouter's paths:
https://openrouter.ai/api/v1 -> https://api-pr-442.preview.experientiallabs.ai/api/v1
Authorization: Bearer $EXPLABS_API_KEY (the ONLY header needed)
Store my key as EXPLABS_API_KEY in this project's env scheme (.env plus the
example file entry), never in code; point every OpenAI-compatible client at
the new base (a plain OpenAI SDK may also use https://api-pr-442.preview.experientiallabs.ai/v1) and
REPLACE URL variables such as OPENROUTER_BASE_URL with it. An Anthropic SDK
appends /v1/messages itself: its base is the bare origin https://api-pr-442.preview.experientiallabs.ai
(never .../v1, which doubles to /v1/v1/messages). Do NOT
delete OPENROUTER_API_KEY or the OpenRouter key from .env, secret stores or
CI: stop reading it, leave it in place; I revoke it at openrouter.ai/keys.
A key COMMITTED in a config file (ANTHROPIC_AUTH_TOKEN in a checked-in
.claude/settings.json) is already leaked: remove it there, never send it on.
Coding agents: Claude Code uses ANTHROPIC_BASE_URL="https://api-pr-442.preview.experientiallabs.ai"
(no /v1); the key rides ANTHROPIC_API_KEY (never ANTHROPIC_AUTH_TOKEN) via
a shell export, the gitignored .claude/settings.local.json, or an
apiKeyHelper reading .env, never the committed .claude/settings.json; remap
EVERY model variable (ANTHROPIC_MODEL, ANTHROPIC_SMALL_FAST_MODEL,
ANTHROPIC_DEFAULT_*_MODEL) to slugs or a background role keeps calling an
expensive model. OpenCode, Cline, Continue, Cursor and Codex take the
OpenAI base https://api-pr-442.preview.experientiallabs.ai/v1.
3. Map model ids. Call GET https://api-pr-442.preview.experientiallabs.ai/api/v1/models with my
key (free, spends no credits). EVERY entry carries canonical_slug, the
plain slug the gateway serves; ids are usually vendor/slug (vendor/slug:free
for a free lane), sometimes a bare slug (gpt-6-astra, grok-4); ~vendor/x-latest
ids are rolling aliases, for projects that used OpenRouter's :latest ids.
For each OpenRouter id from step 1: strip any :nitro, :floor, :online,
:thinking, :extended, :beta or :exacto suffix (those variants do not exist
here; only :free does), then match canonical_slug == the part after the
vendor's slash (openai/gpt-6-astra -> gpt-6-astra, anthropic/claude-sonnet-5
-> claude-sonnet-5, deepseek/deepseek-v4-flash -> deepseek-v4-flash).
No match: look for the same model under a slightly different spelling and
use ONE obvious candidate, noted in the report. openrouter/auto and every
id still unmapped: ask me one question listing each with your best
candidates from the list, then wait. Never guess a model I did not pick.
Use canonical_slug EXACTLY as returned; a vendor/slug id works verbatim
ONLY when it is in the list (x-ai/grok-4 for the bare grok-4 answers 403
model_not_granted: that means "use the canonical_slug", not an entitlement
problem). LiteLLM: model "openai/<slug>" (replace the "openrouter/" prefix)
with api_base https://api-pr-442.preview.experientiallabs.ai/api/v1 and api_key from the new var.
4. Keep the request shapes (OpenAI Chat Completions and Responses bodies, SSE
streaming, tool calling, json_schema, response shapes). Strip EVERY
OpenRouter-only field BEFORE the first call: each one left in is either
silently ignored or its own 400 unsupported_parameter naming it in "param".
- usage: {"include": true} (a guaranteed 400 today); for a stream send
stream_options: {"include_usage": true} instead (non-streams carry usage).
- provider (order, allow_fallbacks, ignore, quantizations, sort, ...):
accepted and silently ignored (not disclosed); the gateway routes and
fails over itself. The one honored key is "provider": {"zdr": true}, a
per-request zero-data-retention demand; keep exactly that if I used it.
- transforms, route, models (fallback list: use its first model), plugins.
- HTTP-Referer / X-Title headers: harmless, not shown; attribute via safety_identifier.
Vercel AI SDK: replace @openrouter/ai-sdk-provider with @ai-sdk/openai
(createOpenAI({ baseURL, apiKey })) or @ai-sdk/openai-compatible.
"reasoning": {"effort"} is translated to reasoning_effort (rewriting it is optional).
Unsupported efforts may only move down if policy permits and the request is admissible; else 400.
"none" means off and can be refused; "minimal" still enables reasoning.
On non-reasoning routes, effort may drop only when policy permits and the request is admissible.
Translated, dropped or adjusted fields are usually listed in the reply's
x-experiential-ignored-parameters JSON body field (not a header): informational.
Anthropic-shaped traffic through OpenRouter (an Anthropic SDK, Claude Code)
moves to the native Messages API, POST https://api-pr-442.preview.experientiallabs.ai/api/v1/messages
(SDK base: the bare origin, step 2), same key; no Idempotency-Key there.
Responses API previous_response_id: continuation resolves ONLY while my
org's capture_prompt_content is on and within the retention window; with
capture off the second turn 400s previous_response_not_found. If the
project chains turns that way, read
GET https://api-pr-442.preview.experientiallabs.ai/api/orgs/<org_id>/telemetry-settings (org_id: /api/whoami);
if off, ask me ONE question: turn it on (PUT the same route,
{"capture_prompt_content": true}; it stores my prompts with Experiential)
or rewrite the chain to resend the full conversation each turn.
5. Move cost reads. Every chat / responses reply carries usage.cost (USD, what
I was charged) and usage.is_byok; on a stream it rides the final usage
chunk (step 4's stream_options). A request sent with Idempotency-Key
carries no usage.cost at all, so read the field as optional. Same key on
all of these, under https://api-pr-442.preview.experientiallabs.ai/api/v1:
GET /generation?id=<the x-request-id response header> one request
GET /usage settled per-request rows (api_key_id, cost) for billing
GET /credits balance GET /key key info (OpenRouter's /auth/key)
GET /activity per-day rollup (not per request)
OpenRouter's /models/user has no equivalent: use /models. /usage is paginated
(50 rows + next_cursor) and, like /activity, ORG-wide: filter ?api_key_id=<my
key's id, on every row> and match my x-request-id, or use /generation?id=.
6. My own provider keys (BYOK). If step 1 found provider keys I routed through
OpenRouter (OPENAI_API_KEY, ANTHROPIC_API_KEY, GEMINI, XAI, DEEPSEEK, AZURE,
AWS/Bedrock, Fireworks, ...), ask whether to connect them with no markup:
GET https://api-pr-442.preview.experientiallabs.ai/api/whoami -> org_id
GET https://api-pr-442.preview.experientiallabs.ai/api/orgs/<org_id>/provider-connections
List existing accounts FIRST. For EACH new account, ask me for a unique
setup_alias (1-63 lowercase letters, digits, hyphens or underscores), unused
across all providers in my org; multiple keys need distinct account names.
POST https://api-pr-442.preview.experientiallabs.ai/api/orgs/<org_id>/provider-connections/<provider>
Content-Type: application/json
{"setup_alias": "<unique-account-name>", "secret": "<that provider key>"}
Never overwrite an existing account. On 409 account_exists choose another
name and retry POST, never bypass it with PUT. PUT is only for a deliberate
rotation I request, with that saved account's exact setup_alias and config.
Provider: openai, anthropic, gemini, xai, deepseek, or any other whose
credential is ONE string; never my OpenRouter key: I am leaving OpenRouter.
azure_openai, bedrock, fireworks, vertex and modal need structured config
or a token pair: connect those at
https://pr-442.preview.experientiallabs.ai/settings/connections instead.
Use INFERENCE keys, never admin keys, as secret; OpenAI/Anthropic admin keys
belong only in optional spend_secret. If spend_error is present,
the serving key WAS saved: report partial success, never repeat POST; retry
only the SAME returned connection.setup_alias with PUT, retaining its serving
secret and config. Read check.status; if not "valid", tell me
check.status_detail.remediation and stop. A 402 pro_required means a FIRST connection needs Pro
(https://pr-442.preview.experientiallabs.ai/credits); an org already holding one keeps connecting.
Saving keys adds no arbitrary models: let me select each account on the
chosen catalog model's waterfall with Use this account. Never echo a key.
7. Verify. Make ONE minimal call (model + messages, a small max_tokens) with
the CHEAPEST mapped slug: a reasoning model can cost cents even for a
smoke, and an empty pricing object in the listing means unknown: read
GET https://api-pr-442.preview.experientiallabs.ai/api/models/<slug>, and if that 404s (an alias)
price the model it resolves to (its canonical_slug) or skip it and smoke a
priced slug. Stream if the project streams (add stream_options:
{"include_usage": true} so the final chunk carries usage.cost); if the
project chains previous_response_id, verify a TWO-turn continuation, not a
single call. Report the model, that it succeeded, and its usage.cost, or
read the cost from GET /api/v1/generation?id=<x-request-id>, which works
either way; then confirm the request's row on GET /api/v1/usage by that
x-request-id (I can also see it at https://pr-442.preview.experientiallabs.ai/logs). On a credits
or activation error, tell me to finish activation at https://pr-442.preview.experientiallabs.ai
and stop; do not retry against OpenRouter. Then run the project's tests.
8. Report back: every file changed; the model-id mapping and the ids I still
need to decide; the OpenRouter-only fields removed; where EXPLABS_API_KEY
lives; that the OpenRouter key stays in place for me to revoke; the
verification result. Full contract: https://pr-442.preview.experientiallabs.ai/llms.txt; human
docs: https://pr-442.preview.experientiallabs.ai/docs/integrate.

From Cloudflare AI Gateway

Cloudflare AI Gateway documentation

Assess rules, customer plans, model-group allowances, BYOK and custom providers before moving traffic. The agent asks before writes, spend, privacy changes or cutover, and stops on unsupported requirements or missing live proof.

TodayOn Experiential
Base URLhttps://gateway.ai.cloudflare.com/v1/<account>/<gateway>/...https://api-pr-442.preview.experientiallabs.ai/v1 (Anthropic SDKs: https://api-pr-442.preview.experientiallabs.ai)
Authprovider key plus cf-aig-authorizationAuthorization: Bearer $EXPLABS_API_KEY
Model idsprovider-specific ids: map each through the catalog, never by stripping a prefixcanonical_slug from GET /api/v1/models
prompt
Help me assess and then migrate this project's Cloudflare AI Gateway traffic
onto the Experiential Labs gateway. Start with a read-only inventory and a
proposed mapping. This prompt is NOT blanket consent: ask for my explicit
approval before any file or policy writes, credential storage or rotation,
paid probes or other spend, traffic cutover, privacy/retention changes, or
secret revocation. Explain each proposed change and its rollback first.
My Experiential key is: xpl_...
If it is still "xpl_...", ask me for a key or direct me to
https://pr-442.preview.experientiallabs.ai/api-keys. Do not assume an inference key has admin authority.
Never print secrets, put them in committed files, or send an Experiential key
to Cloudflare or an upstream provider. Redact secret values in every report.
1. Inventory before editing. Find Cloudflare AI Gateway URLs (including
gateway.ai.cloudflare.com, provider-specific paths, /compat and custom-*
providers), SDKs, fetch/curl clients, Workers bindings, env variables, CI
and secret-store references. List call sites, endpoints and model ids.
Identify Chat Completions, Responses, Messages, streaming, tool calls,
structured output, continuation, batches and any other used API surface.
Record Cloudflare headers such as cf-aig-authorization, cf-aig-metadata,
cf-aig-cache-ttl, cf-aig-skip-cache, cf-aig-custom-cost and cf-aig-log-level,
plus all other cf-aig-* headers, retry/fallback settings and timeouts.
Separate Cloudflare gateway/admin tokens from direct OpenAI/Anthropic or
other provider inference credentials. A Cloudflare token is not BYOK for
those providers. Inspect references without exposing secret values.
2. Ask for the actual Cloudflare rule export and customer subscription rules.
Record dynamic-routing conditions, rule order, rate/spend limits, metadata
filters/split dimensions and provider/model overrides. Identify the trusted
customer id and the server-side subscription source. Cloudflare custom
metadata labels are NOT authentication or authorization. Neither request
metadata.plan, user nor safety_identifier may select a customer's plan.
The application server must map its authenticated customer to an Experiential
identity; all that identity's keys, including rotated keys, share policy.
Inventory each tier's allowed models, Basic/Frontier (or other) model groups,
exact memberships, separate allowance amounts, currency and cost basis.
These are my authored groups, not universal model classes or platform
Free/Pro subscriptions. Never turn illustrative amounts into defaults.
Ask about reset timezone and window (fixed, rolling, billing-anniversary or
UTC calendar-month), carryover, current-period usage and pending requests,
carry-in at cutover, upgrades/downgrades and over-limit behavior. A zero
allowance means hard zero, not unlimited. Also inventory the shared total
across all groups and the exact action on group exhaustion. An absent/null
aggregate_limit_nano_usd means no shared cap; zero denies. on_exhausted
defaults to deny; next_model requires explicit policy and a certified
server-authored model chain, never an arbitrary client retry. Aggregate
exhaustion and forbidden models still stop. Unknown plans must block mapping,
never become unassigned identities. Do not silently reset usage.
3. Read THIS deployment's live contract and catalog before proposing APIs:
https://pr-442.preview.experientiallabs.ai/llms.txt
https://pr-442.preview.experientiallabs.ai/docs/integrate
https://pr-442.preview.experientiallabs.ai/docs/account-api
https://pr-442.preview.experientiallabs.ai/docs/cost-api
GET https://api-pr-442.preview.experientiallabs.ai/api/whoami
GET https://api-pr-442.preview.experientiallabs.ai/v1/models
GET https://api-pr-442.preview.experientiallabs.ai/api/v1/models
Use the catalog's canonical_slug and verified alias resolution to map every
model, including provider-prefixed ids and custom models. Do not simply strip
a prefix, assume a model exists, or replace one without asking me. Catalog
discovery is not proof of this customer's access or successful serving.
Build a requirement table: existing behavior, proposed mapping, live-doc
evidence, test needed and blocker. Discover exact native model-group,
customer-plan, assignment and allowance API routes from live documentation;
do not invent routes, fields, or claim these controls are already deployed.
Require native identity-bound access and separate per-group allowances,
not application-side checks dressed up as equivalent gateway enforcement.
Confirm admin/delegated assignment authority separately from provisioning
keys and inference keys. Customers must not change plans, mint escape keys,
or read another identity's usage; caller-supplied filters are not isolation.
Verify allowed-model access across public models, aliases, retries/fallback
and every used serving surface; unsupported lanes must refuse plan keys.
Verify shared counters across keys, settled usage plus pending reservations,
pricing revisions, plan-change effective times and preserved spent amounts.
Read whoami's identity_id, customer_plan_identity_id and is_provisioning;
restricted keys may self-read it, but that does not grant management rights.
The implemented period contract supports UTC calendar-month or an explicit
authoritative source interval with source_period_id and timezone-aware start/end.
Explicit periods expire without automatic renewal; omitted budget_period on
a plan change preserves it. Rolling windows are unsupported. Never rename,
overlap or backdate a used source period to refill it. Treat unsupported
reset/carryover rules as blockers, not approximations. Discover the documented
allowance-cutover contract too, without assuming THIS deployment has it.
If documented carry-in is unavailable, ask about a deliberate next-period cutover;
never silently grant a fresh full allowance.
4. Map provider and cost behavior, not just a base URL. Existing Account API
provisioning and Cost API settled exports are useful but do not by themselves
prove subscription-policy parity. Keep wallet charges separate from customer
allowance attribution. BYOK must not debit platform credits; attributed BYOK
inference value uses a frozen rate schedule and is an estimate, NOT a provider
invoice. Inventory input/output, cache-read/write, reasoning, long-context
and custom rates, billable failures, missing usage and pending exposure.
Unknown prices or missing usage must not turn a monetary allowance into free
requests. Verify the allowance API reads the counters enforcement uses.
Subscription events and invoicing stay in my application. For external
billing, poll the documented settled Cost API export with pagination,
durable checkpoints and deduplication; reconcile request ids, late settlement
and retries. Do not invent webhooks or bill from a daily rollup or wallet
balance. Treat inline usage.cost as optional; do not confuse it with a BYOK
allowance estimate. Verify reporting scopes with two customer identities.
List existing provider accounts before proposing BYOK. Ask before saving
anything; use a distinct setup_alias for each new account and create-only
POST from the live contract, never PUT over an existing account. Rotation
needs separate approval. Respect the live plan entitlement for new BYOK
connections; saving a provider key does not add arbitrary model support.
5. Custom providers need their own proof. Cloudflare custom providers can use
arbitrary HTTP paths and native bodies; Experiential's supported adapters
and model-bound custom endpoints are NOT an arbitrary HTTP proxy. Inventory
the exact URL, wire protocol, auth headers, model ids, usage format and rate
units. Use the documented Vault-backed connection/custom-endpoint flow and
its validation, not a new proxy or a made-up Cloudflare provider connection.
Custom endpoint rates are not inferred from public provider prices. Confirm
the live rate_card contract, full schedule and frozen revision. Without an
authored card, custom pricing is unknown, not free. New custom accounts need
an explicit unique setup_alias and create-only POST. For key or rate changes,
create a new custom account and deliberately select Use on the model; do not
rotate it through the generic provider PUT or probe at the official origin.
For Tinfoil (TinFoil), verify the endpoint and SDK's enclave-attestation and
privacy requirements. OpenAI compatibility does NOT prove attestation survives
an intermediary. If the required guarantee cannot be verified, stop that move.
Keep Modal separate from the Cloudflare transfer. Determine whether it uses
Modal proxy authentication, an application bearer, or another supported scheme,
and whether it bills tokens or time/GPU usage. Do not claim token-cost parity
for unsupported billing units. Name any adapter, auth, usage or pricing gap
as a blocker and scope it separately; never silently reroute these services.
6. Preserve privacy and runtime semantics. Inventory prompt/response logging,
retention, zero-data-retention/no-training needs, cache keys/TTL/bypass,
cache hits and their accounting, retries, fallback order and timeout budgets.
Cloudflare response caching is not the same as provider prompt caching.
Map each cf-aig-* header to a documented control or report it unsupported;
never forward Cloudflare auth to the new host or silently drop a required rule.
Check continuation requirements such as previous_response_id against the live
capture_prompt_content contract. Ask before enabling storage; offer resending
the full conversation where supported instead. Never weaken privacy to make
a smoke pass. Check retry/idempotency behavior per API surface, including
ambiguous disconnects; do not assume every retry is free or safe to replay.
7. Present the mapping and blockers, then wait for approval of specific edits.
For supported OpenAI clients the base is https://api-pr-442.preview.experientiallabs.ai/v1 with
Authorization: Bearer $EXPLABS_API_KEY. An Anthropic SDK appends /v1/messages
itself: use the bare origin https://api-pr-442.preview.experientiallabs.ai, not a doubled /v1 base.
Store keys only in the approved local env/secret scheme, never source control.
Preserve unrelated Cloudflare DNS, Workers, Turnstile and other services.
Keep old untracked .env secrets, CI secrets and Cloudflare/provider tokens
in place for rollback until deliberate revocation is approved. A committed
secret is exposed: flag it and ask for removal/rotation, never echo it.
Propose a reversible configuration diff and keep a redacted baseline of
endpoint, model and policy settings. Do not switch production traffic yet.
8. Ask for an isolated test environment, bounded credentials and a request/spend
budget before probes; connection checks may themselves make paid calls.
Pin platform revision, deployed engine versions, schema, configuration and
catalog/pricing revisions. Run local tests first; label mocks as mocks.
With approval, test actual providers and the project's real request shapes,
streams/tools/continuations, two customers, multiple models per group,
independent allowance depletion, forbidden-model refusal before dispatch,
key rotation, concurrent reservations, retries/fallback, interrupted streams,
reset/plan changes, shared aggregate depletion, deny versus next_model,
identity-scoped reports and BYOK without wallet debits. Reconcile request ids,
settled costs and native plus imported pending allowance exposure. A single
successful completion does not establish policy, billing or privacy parity.
Stop before cutover on unsupported requirements or absent live verification.
Do not autoactivate production policy or move traffic. Only after evidence and
a separate cutover approval, use the supported administrator handoff:
- Prepare at most 500 immutable, disjoint source-owned per-request receipts,
each with external_ref, frozen group reference, settled and pending amounts.
Never import cumulative snapshots or requests already counted by the destination.
Pin exact period, source watermark, coverage and evidence digest. Identical
prepare retries are idempotent; changed replays/overlapping exports refuse.
More than 500 receipts is a blocker for this bounded path, not permission
to truncate, aggregate them or invent pagination.
- Prepare closes that identity's destination admission even while global
enforcement is inactive. It does not stop Cloudflare. Explicitly stop source
dispatch and reconcile in-flight ownership before activating with source_frozen
and independent source-stop evidence. This is NOT a transaction across gateways.
Require the documented trusted administrator transport; an ordinary or delegated
provisioning key cannot perform cutover. Do not invent a browser proxy or
supply actor headers if the live contract does not provide an admin entrypoint.
- Rollback must account for usage on BOTH gateways. Freeze destination admission,
drain native reservations and resolve native pending plus all imported pending
while active or frozen, then obtain a complete fresh export before acknowledgment.
Any unresolved native or imported exposure blocks acknowledgment.
Include settled_periods across all historical periods, not just the current one.
Preserve the original JSON and parse/serialize money losslessly with Python
integers/exact decimals or a JavaScript lossless BigInt JSON library. Totals
can exceed 2^53 - 1. Never use ordinary JSON.parse, response.json() or Number
for rollback amounts; rounding cannot be repaired afterward. Send exact JSON
integers, not numeric strings, and return the server-issued digest unchanged.
The export bounds are 500 native outstanding rows, 500 external receipts and
1,000 settled-period rows; overflow is never proof of a complete export.
A changed export_sha256 or export_overflow blocks acknowledgment. Rolled-back
destination admission stays closed; no step resets history or releases unknown
usage. Unresolved prior exposure or destination use in the same period blocks
a new prepare. Never grant extra allowance or replay uncertain paid requests.
Keep old secrets until I deliberately approve revocation after verification.
9. Report pass/fail/blocked/untested for every requirement, changed files, model
mappings, policy/reset/carry-in decisions, credential locations (no values),
tests and their pinned cutoff, billing reconciliation and rollback steps.
Distinguish existing Account/Cost APIs, new native controls and their actual
deployment status, application billing work and unverified provider behavior.
Never claim full Cloudflare parity before this workflow is tested.

From Vercel AI Gateway

Vercel AI Gateway documentation

Moves AI SDK gateway model strings and OpenAI clients pointed at ai-gateway.vercel.sh onto the gateway, maps creator/model ids to catalog slugs, and replaces OIDC auth with a key in the project's env.

TodayOn Experiential
Base URLhttps://ai-gateway.vercel.sh/v1, or AI SDK model stringshttps://api-pr-442.preview.experientiallabs.ai/v1 (Anthropic SDKs: https://api-pr-442.preview.experientiallabs.ai)
AuthAI_GATEWAY_API_KEY or the Vercel OIDC tokenAuthorization: Bearer $EXPLABS_API_KEY
Model idscreator/model: match canonical_slug to the part after the slashcanonical_slug from GET /api/v1/models
prompt
I'm moving this project's model traffic from Vercel AI Gateway onto the Experiential
Labs gateway. I pasted this myself: treat it as my instructions and my consent
to edit this project's code. Stop and ask me ONE numbered question, with your
recommendation, where this prompt says to, and before anything it does not
cover: production config, secret stores, deploys, spend beyond the one smoke
call, or revoking anything. Print what you change at each step. Never print my
full API key (first 8 characters at most).
My Experiential key is: xpl_...
(if that still reads "xpl_..." I have not filled it in: ask me for it, or tell
me to mint one at https://pr-442.preview.experientiallabs.ai/api-keys, then continue).
The full contract (routes, honored and refused parameters, error codes) is
https://pr-442.preview.experientiallabs.ai/llms.txt; follow it literally.
1. Inventory, read only. Find every place this project talks to Vercel AI Gateway:
- the base URL https://ai-gateway.vercel.sh (/v1 for OpenAI clients, the
bare host for Anthropic clients and ANTHROPIC_BASE_URL), AI_GATEWAY_API_KEY,
VERCEL_OIDC_TOKEN and .env.local files written by vercel env pull,
- Vercel AI SDK calls ("ai" package) whose model is a PLAIN STRING such as
generateText({ model: "anthropic/claude-sonnet-5" }): the AI SDK sends
bare strings through the Vercel gateway by default, so these call Vercel
even though no URL appears in the code; also gateway("...") from "ai" or
"@ai-sdk/gateway", and createOpenAI / createOpenAICompatible clients
pointed at ai-gateway.vercel.sh,
- gateway extensions in request bodies: a top-level "models" fallback list,
a "provider" object (sort and friends), and providerOptions.gateway
(order, only, sort, models, byok, providerTimeouts, caching: "auto"),
- provider keys configured as Vercel BYOK (in the Vercel dashboard or
request-scoped under providerOptions.gateway.byok).
Also list hardcoded model ids, every Vercel AI Gateway-specific header and body field,
retry / fallback / cache settings, coding-agent configs pointed at it, and
code that reads cost or usage from it. Show me the files and call sites
before changing anything. Inspect secret references, never their values.
2. Map Vercel AI Gateway's features before moving anything.
- order / only / sort / providerTimeouts: the gateway picks and fails over
providers itself; drop them. "models" (fallback models): keep the primary
model and tell me which fallbacks are lost.
- caching: "auto" and cache_control markers: see prompt caching below.
- BYOK (dashboard or per request): offered as saved accounts in step 6;
request-scoped credentials have no equivalent and leave the body.
- OIDC: there is no keyless equivalent. On Vercel the key becomes a project
environment variable: ask me to add EXPLABS_API_KEY to the Vercel project
(Production, Preview, Development) myself, or run vercel env add only
after I say so.
The gateway's own equivalents, for the table:
- Fallback, retries and load balancing across providers of one model: the
gateway routes and fails over itself (GET https://api-pr-442.preview.experientiallabs.ai/api/models/<slug>/waterfall
shows the order). A fallback list ACROSS models has no per-request
equivalent.
- Keys per team or customer, spend caps, budgets: a PROVISIONING key (an org
admin mints the first at https://pr-442.preview.experientiallabs.ai/api-keys) mints inference keys with a
daily USD cap (POST https://api-pr-442.preview.experientiallabs.ai/api/v1/keys {"name", "limit"}) and sets monthly
caps per key or identity (PUT https://api-pr-442.preview.experientiallabs.ai/api/orgs/<org_id>/budgets).
- Metadata: the X-Explabs-Tags header (a JSON object, at most 16 labels, on
chat, Responses and Messages calls; never secrets or personal data). End
user: safety_identifier (alias user) in the body. Session or
conversation: the X-Explabs-Session-Id header.
- Logs and dashboards: https://pr-442.preview.experientiallabs.ai/logs and https://pr-442.preview.experientiallabs.ai/spend, the Cost API (step 5),
and live export of every call to my Langfuse project
(PUT https://api-pr-442.preview.experientiallabs.ai/api/orgs/<org_id>/observation-export).
- Privacy: "provider": {"zdr": true} per request, or account-wide
PUT https://api-pr-442.preview.experientiallabs.ai/api/orgs/<org_id>/zero-data-retention (Pro); processing
regions are in Settings, Data controls (https://pr-442.preview.experientiallabs.ai/settings/data-controls).
- Content guardrails: per identity, on the Guardrails tab at https://pr-442.preview.experientiallabs.ai/api-keys.
- Prompt caching: provider prompt caching applies where the route supports
it; Anthropic cache_control markers are kept on supported routes and
disclosed as not forwarded elsewhere (default five-minute markers; a
one-hour marker is refused on platform credits).
- No equivalent: gateway-side response or semantic caching, hosted prompt
templates, metadata-based routing between models, and arbitrary
request/response transforms.
Show me a table: feature in use -> equivalent (or "none") -> what changes.
Then ask me ONE question: proceed with this plan, adjust it, or stop. Wait.
A feature I depend on with no equivalent is a blocker: say so, never drop it
silently.
3. Point the clients at the gateway.
AI SDK, two ways (ask me which; recommend (a) when many call sites use
plain strings):
(a) one provider for every plain string, set once at startup:
import { createOpenAICompatible } from "@ai-sdk/openai-compatible";
globalThis.AI_SDK_DEFAULT_PROVIDER = createOpenAICompatible({
name: "experiential", baseURL: "https://api-pr-442.preview.experientiallabs.ai/v1",
apiKey: process.env.EXPLABS_API_KEY });
then every model string must be an Experiential slug (step 4);
(b) per call site: model: experiential("<slug>") with the same provider
instance, or createAnthropic from "@ai-sdk/anthropic" with baseURL
https://api-pr-442.preview.experientiallabs.ai/v1 for Claude call sites that need Anthropic-only options
(that provider's default base already ends in /v1).
Remove the gateway import once nothing uses it, and delete the
AI_GATEWAY_API_KEY read (keep the variable itself for me to revoke).
OpenAI-compatible clients: base URL https://api-pr-442.preview.experientiallabs.ai/v1 and
Authorization: Bearer $EXPLABS_API_KEY (the ONLY header needed). An
Anthropic SDK appends /v1/messages itself: its base is the bare origin
https://api-pr-442.preview.experientiallabs.ai (never .../v1). A coding agent pointed at Vercel AI Gateway: follow that
agent's section at https://pr-442.preview.experientiallabs.ai/docs/coding-agents instead of guessing.
Store my key as EXPLABS_API_KEY in this project's env scheme (.env plus the
example file entry), never in code. Strip every Vercel AI Gateway-only field before
the first call: left in, each is ignored or refused with 400
unsupported_parameter naming it. Do NOT delete the old Vercel AI Gateway keys from
.env, secret stores or CI: stop reading them; I revoke them myself. A key
committed in a tracked file is already leaked: flag it and remove it there.
4. Map model ids. GET https://api-pr-442.preview.experientiallabs.ai/api/v1/models with my key (free, spends no
credits): every entry carries canonical_slug, the plain slug the gateway
serves. Vercel ids are creator/model (anthropic/claude-sonnet-5,
openai/gpt-6-astra): match canonical_slug against the part after the slash.
One obvious candidate: use it and note it in the report. Anything else:
ask me ONE question listing each id with your best candidates, then wait.
Never guess a model I did not pick. Use canonical_slug EXACTLY as returned.
5. Move cost and usage reads. Every chat / Responses reply carries usage.cost
(USD charged) and usage.is_byok; on a stream it rides the final usage chunk
when the request sets stream_options: {"include_usage": true}. A request
sent with Idempotency-Key carries no usage.cost, so read it as optional.
Same key, under https://api-pr-442.preview.experientiallabs.ai/api/v1:
GET /generation?id=<the x-request-id response header> one request
GET /usage settled per-request rows for billing (org-wide; filter
?api_key_id= or attribution_label)
GET /credits balance GET /activity per-day rollup
Replace the old cost and usage code with these; show me the diff first.
AI SDK code that read providerMetadata.gateway (cost, routing) reads
usage.cost or the Cost API instead.
6. Provider keys (bring your own key). Vercel may hold my provider keys (dashboard BYOK, or providerOptions.gateway.byok in requests). Ask whether to connect
them so those calls bill my own provider accounts with no markup:
GET https://api-pr-442.preview.experientiallabs.ai/api/whoami -> org_id
GET https://api-pr-442.preview.experientiallabs.ai/api/orgs/<org_id>/provider-connections (list existing first)
POST https://api-pr-442.preview.experientiallabs.ai/api/orgs/<org_id>/provider-connections/<provider>
{"setup_alias": "<a new unique account name I approve>", "secret": "<key>"}
setup_alias: 1-63 lowercase letters, digits, hyphens or underscores, unused
across my accounts. Never overwrite an account; on 409 account_exists pick
another name. Use inference keys, never admin keys, and ask me to paste
each key: never export one from Vercel AI Gateway. A one-string key works for openai,
anthropic, gemini, xai, deepseek and similar; azure_openai, bedrock,
fireworks, vertex and modal need structured config: I connect those at
https://pr-442.preview.experientiallabs.ai/settings/connections. Read check.status; if it is not "valid", show
me check.status_detail.remediation. A 402 pro_required means a FIRST
connection needs Pro (https://pr-442.preview.experientiallabs.ai/credits). Saving a key does not select it for
every model: I pick it per model with Use this account. Never echo a key.
7. Verify. Make ONE minimal call (model + messages, a small max_tokens) with
the cheapest mapped slug (GET https://api-pr-442.preview.experientiallabs.ai/api/models/<slug> shows its price).
Stream if the project streams. Report the model, that it succeeded, and its
usage.cost, or read it from GET /api/v1/generation?id=<x-request-id>; I can
also see the call at https://pr-442.preview.experientiallabs.ai/logs. On a credits or activation error tell me
to finish activation at https://pr-442.preview.experientiallabs.ai and stop; never retry through Vercel AI Gateway. Then
run the project's tests.
8. Report back: every file changed; the feature table and blockers; the model
mapping and the ids I still need to decide; where EXPLABS_API_KEY lives;
which old credentials I still need to revoke; the verification result.
Human docs: https://pr-442.preview.experientiallabs.ai/docs/switch-gateways.

From LiteLLM

LiteLLM documentation

Either keeps your LiteLLM proxy and adds the gateway as its upstream (clients unchanged), or removes it and points clients at the gateway; maps virtual keys, budgets, fallbacks and callbacks to their equivalents.

TodayOn Experiential
Base URLthe LiteLLM proxy (e.g. http://localhost:4000) or litellm.completionhttps://api-pr-442.preview.experientiallabs.ai/v1 (Anthropic SDKs: https://api-pr-442.preview.experientiallabs.ai)
Authsk- virtual keys / LITELLM_MASTER_KEYAuthorization: Bearer $EXPLABS_API_KEY
Model idsprovider/model: strip the prefix; or keep the proxy and set model: openai/<slug>canonical_slug from GET /api/v1/models
prompt
I'm moving this project's model traffic from LiteLLM onto the Experiential
Labs gateway. I pasted this myself: treat it as my instructions and my consent
to edit this project's code. Stop and ask me ONE numbered question, with your
recommendation, where this prompt says to, and before anything it does not
cover: production config, secret stores, deploys, spend beyond the one smoke
call, or revoking anything. Print what you change at each step. Never print my
full API key (first 8 characters at most).
My Experiential key is: xpl_...
(if that still reads "xpl_..." I have not filled it in: ask me for it, or tell
me to mint one at https://pr-442.preview.experientiallabs.ai/api-keys, then continue).
The full contract (routes, honored and refused parameters, error codes) is
https://pr-442.preview.experientiallabs.ai/llms.txt; follow it literally.
1. Inventory, read only. Find every place this project talks to LiteLLM:
- the proxy config (config.yaml / litellm_config.yaml: model_list,
litellm_params, litellm_settings, router_settings, general_settings),
LITELLM_MASTER_KEY, DATABASE_URL, and the proxy's Docker / Helm files;
whether store_model_in_db is on (models added in the LiteLLM UI or via
/model/new live in its database, not the YAML),
- clients pointed at the proxy (http://localhost:4000 or its host) with sk-
virtual keys, ANTHROPIC_BASE_URL / ANTHROPIC_AUTH_TOKEN pointed at it,
and SDK calls to litellm.completion / acompletion / Router with
provider-prefixed ids (openai/, anthropic/, azure/, bedrock/, vertex_ai/,
openrouter/, litellm_proxy/, ...),
- virtual keys and teams (models, max_budget, budget_duration, tpm_limit,
rpm_limit), fallbacks and context_window_fallbacks, caching, the request
"metadata" field, and success_callback / failure_callback integrations.
Also list hardcoded model ids, every LiteLLM-specific header and body field,
retry / fallback / cache settings, coding-agent configs pointed at it, and
code that reads cost or usage from it. Show me the files and call sites
before changing anything. Inspect secret references, never their values.
2. Map LiteLLM's features before moving anything.
- Virtual keys, teams and budgets: under (a) below they stay in LiteLLM;
under (b) they become provisioning-key minted keys and budgets.
- fallbacks across deployments of one model, retries, routing_strategy: the
gateway does this itself; fallbacks across models and
context_window_fallbacks: keep the primary and tell me.
- drop_params: the gateway refuses or ignores unknown fields itself; the
request fields the project sends still need stripping (step 3).
- Request "metadata" (tags, trace ids): X-Explabs-Tags /
X-Explabs-Session-Id. With overwrite_user_with_key_hash on, the "user" the
provider saw was the key hash, not my end user.
- Callbacks: Langfuse becomes the gateway's live export; list the others
for me to decide.
The gateway's own equivalents, for the table:
- Fallback, retries and load balancing across providers of one model: the
gateway routes and fails over itself (GET https://api-pr-442.preview.experientiallabs.ai/api/models/<slug>/waterfall
shows the order). A fallback list ACROSS models has no per-request
equivalent.
- Keys per team or customer, spend caps, budgets: a PROVISIONING key (an org
admin mints the first at https://pr-442.preview.experientiallabs.ai/api-keys) mints inference keys with a
daily USD cap (POST https://api-pr-442.preview.experientiallabs.ai/api/v1/keys {"name", "limit"}) and sets monthly
caps per key or identity (PUT https://api-pr-442.preview.experientiallabs.ai/api/orgs/<org_id>/budgets).
- Metadata: the X-Explabs-Tags header (a JSON object, at most 16 labels, on
chat, Responses and Messages calls; never secrets or personal data). End
user: safety_identifier (alias user) in the body. Session or
conversation: the X-Explabs-Session-Id header.
- Logs and dashboards: https://pr-442.preview.experientiallabs.ai/logs and https://pr-442.preview.experientiallabs.ai/spend, the Cost API (step 5),
and live export of every call to my Langfuse project
(PUT https://api-pr-442.preview.experientiallabs.ai/api/orgs/<org_id>/observation-export).
- Privacy: "provider": {"zdr": true} per request, or account-wide
PUT https://api-pr-442.preview.experientiallabs.ai/api/orgs/<org_id>/zero-data-retention (Pro); processing
regions are in Settings, Data controls (https://pr-442.preview.experientiallabs.ai/settings/data-controls).
- Content guardrails: per identity, on the Guardrails tab at https://pr-442.preview.experientiallabs.ai/api-keys.
- Prompt caching: provider prompt caching applies where the route supports
it; Anthropic cache_control markers are kept on supported routes and
disclosed as not forwarded elsewhere (default five-minute markers; a
one-hour marker is refused on platform credits).
- No equivalent: gateway-side response or semantic caching, hosted prompt
templates, metadata-based routing between models, and arbitrary
request/response transforms.
Show me a table: feature in use -> equivalent (or "none") -> what changes.
Then ask me ONE question: proceed with this plan, adjust it, or stop. Wait.
A feature I depend on with no equivalent is a blocker: say so, never drop it
silently.
3. Point the clients at the gateway.
First ask me ONE question, with your recommendation, then wait:
(a) keep LiteLLM in front and make the gateway its upstream: clients and
virtual keys stay unchanged; each model_list entry becomes
- model_name: <the name my clients already send>
litellm_params:
model: openai/<slug>
api_base: https://api-pr-442.preview.experientiallabs.ai/v1
api_key: os.environ/EXPLABS_API_KEY
(recommended when several teams hold LiteLLM virtual keys). With
store_model_in_db on, change the models where they live (the LiteLLM
UI or its model API), not only the YAML. LiteLLM does not know
catalog prices, so its spend tracking reads 0 for these models
unless I add custom pricing: read cost from the gateway (step 5);
(b) remove LiteLLM from the request path and point the clients at the
gateway directly (recommended for a single app or script).
LiteLLM SDK without the proxy: litellm.completion(model="openai/<slug>",
api_base="https://api-pr-442.preview.experientiallabs.ai/v1", api_key=os.environ["EXPLABS_API_KEY"]). A bare slug
cannot route in LiteLLM; the openai/ prefix selects its OpenAI-compatible
adapter.
OpenAI-compatible clients: base URL https://api-pr-442.preview.experientiallabs.ai/v1 and
Authorization: Bearer $EXPLABS_API_KEY (the ONLY header needed). An
Anthropic SDK appends /v1/messages itself: its base is the bare origin
https://api-pr-442.preview.experientiallabs.ai (never .../v1). A coding agent pointed at LiteLLM: follow that
agent's section at https://pr-442.preview.experientiallabs.ai/docs/coding-agents instead of guessing.
Store my key as EXPLABS_API_KEY in this project's env scheme (.env plus the
example file entry), never in code. Strip every LiteLLM-only field before
the first call: left in, each is ignored or refused with 400
unsupported_parameter naming it. Do NOT delete the old LiteLLM keys from
.env, secret stores or CI: stop reading them; I revoke them myself. A key
committed in a tracked file is already leaked: flag it and remove it there.
4. Map model ids. GET https://api-pr-442.preview.experientiallabs.ai/api/v1/models with my key (free, spends no
credits): every entry carries canonical_slug, the plain slug the gateway
serves. LiteLLM ids carry a provider prefix (openai/gpt-6-astra,
anthropic/claude-..., openrouter/<vendor>/<model>): strip the prefix (and an
OpenRouter vendor) and match canonical_slug. Under (a) keep each model_name
the clients send and change only litellm_params.model to openai/<slug>.
One obvious candidate: use it and note it in the report. Anything else:
ask me ONE question listing each id with your best candidates, then wait.
Never guess a model I did not pick. Use canonical_slug EXACTLY as returned.
5. Move cost and usage reads. Every chat / Responses reply carries usage.cost
(USD charged) and usage.is_byok; on a stream it rides the final usage chunk
when the request sets stream_options: {"include_usage": true}. A request
sent with Idempotency-Key carries no usage.cost, so read it as optional.
Same key, under https://api-pr-442.preview.experientiallabs.ai/api/v1:
GET /generation?id=<the x-request-id response header> one request
GET /usage settled per-request rows for billing (org-wide; filter
?api_key_id= or attribution_label)
GET /credits balance GET /activity per-day rollup
Replace the old cost and usage code with these; show me the diff first.
Code that read LiteLLM spend (/key/info, /team/info, /user/info, the spend
logs) reads the Cost API instead; under (a) both exist, and the gateway's
is the one that matches the bill.
6. Provider keys (bring your own key). litellm_params may reference my provider keys (OPENAI_API_KEY, ANTHROPIC_API_KEY, AZURE_API_KEY, AWS credentials, ...). Ask whether to connect
them so those calls bill my own provider accounts with no markup:
GET https://api-pr-442.preview.experientiallabs.ai/api/whoami -> org_id
GET https://api-pr-442.preview.experientiallabs.ai/api/orgs/<org_id>/provider-connections (list existing first)
POST https://api-pr-442.preview.experientiallabs.ai/api/orgs/<org_id>/provider-connections/<provider>
{"setup_alias": "<a new unique account name I approve>", "secret": "<key>"}
setup_alias: 1-63 lowercase letters, digits, hyphens or underscores, unused
across my accounts. Never overwrite an account; on 409 account_exists pick
another name. Use inference keys, never admin keys, and ask me to paste
each key: never export one from LiteLLM. A one-string key works for openai,
anthropic, gemini, xai, deepseek and similar; azure_openai, bedrock,
fireworks, vertex and modal need structured config: I connect those at
https://pr-442.preview.experientiallabs.ai/settings/connections. Read check.status; if it is not "valid", show
me check.status_detail.remediation. A 402 pro_required means a FIRST
connection needs Pro (https://pr-442.preview.experientiallabs.ai/credits). Saving a key does not select it for
every model: I pick it per model with Use this account. Never echo a key.
7. Verify. Make ONE minimal call (model + messages, a small max_tokens) with
the cheapest mapped slug (GET https://api-pr-442.preview.experientiallabs.ai/api/models/<slug> shows its price).
Stream if the project streams. Report the model, that it succeeded, and its
usage.cost, or read it from GET /api/v1/generation?id=<x-request-id>; I can
also see the call at https://pr-442.preview.experientiallabs.ai/logs. On a credits or activation error tell me
to finish activation at https://pr-442.preview.experientiallabs.ai and stop; never retry through LiteLLM. Then
run the project's tests.
8. Report back: every file changed; the feature table and blockers; the model
mapping and the ids I still need to decide; where EXPLABS_API_KEY lives;
which old credentials I still need to revoke; the verification result.
Human docs: https://pr-442.preview.experientiallabs.ai/docs/switch-gateways.

From Portkey

Portkey documentation

Replaces the Portkey client and x-portkey-* headers with a plain OpenAI client on the gateway, and maps configs (fallbacks, load balancing, cache), metadata and provider slugs before moving traffic.

TodayOn Experiential
Base URLhttps://api.portkey.ai/v1https://api-pr-442.preview.experientiallabs.ai/v1 (Anthropic SDKs: https://api-pr-442.preview.experientiallabs.ai)
Authx-portkey-api-key plus x-portkey-provider / -configAuthorization: Bearer $EXPLABS_API_KEY
Model idsthe provider's own id (or @provider/model): match the model partcanonical_slug from GET /api/v1/models
prompt
I'm moving this project's model traffic from Portkey onto the Experiential
Labs gateway. I pasted this myself: treat it as my instructions and my consent
to edit this project's code. Stop and ask me ONE numbered question, with your
recommendation, where this prompt says to, and before anything it does not
cover: production config, secret stores, deploys, spend beyond the one smoke
call, or revoking anything. Print what you change at each step. Never print my
full API key (first 8 characters at most).
My Experiential key is: xpl_...
(if that still reads "xpl_..." I have not filled it in: ask me for it, or tell
me to mint one at https://pr-442.preview.experientiallabs.ai/api-keys, then continue).
The full contract (routes, honored and refused parameters, error codes) is
https://pr-442.preview.experientiallabs.ai/llms.txt; follow it literally.
1. Inventory, read only. Find every place this project talks to Portkey:
- the base URL https://api.portkey.ai/v1 (or a self-hosted Portkey gateway,
often http://localhost:8787/v1), PORTKEY_API_KEY, PORTKEY_GATEWAY_URL,
- the portkey-ai SDK (Python from portkey_ai import Portkey, Node
"portkey-ai") and createHeaders(...) passed as default_headers /
defaultHeaders to an OpenAI client,
- how each call picks its provider: a Model Catalog string
"@provider-slug/model" (@openai-prod/gpt-4o), the x-portkey-provider
header ("@openai-prod"), or a legacy x-portkey-virtual-key,
- every x-portkey-* header: x-portkey-config (a saved "pc-..." id or inline
JSON), x-portkey-metadata, x-portkey-trace-id, x-portkey-cache-*,
x-portkey-request-timeout, x-portkey-forward-headers and the provider
credential headers (x-portkey-azure-*, -vertex-*, -aws-*),
- prompt templates called by prompt id (portkey.prompts.completions.create).
Also list hardcoded model ids, every Portkey-specific header and body field,
retry / fallback / cache settings, coding-agent configs pointed at it, and
code that reads cost or usage from it. Show me the files and call sites
before changing anything. Inspect secret references, never their values.
2. Map Portkey's features before moving anything. Saved configs and AI
Providers live in Portkey, not in this repo: ask me to export each config
JSON the project uses, and read inline ones from the code.
- strategy.mode "fallback" / "loadbalance" across providers of one model,
and retry: the gateway does this itself. Targets that name DIFFERENT
models, and mode "conditional" (routing on metadata or params): no
equivalent; keep one model and raise it.
- override_params / default_params: move the values into the request
itself (show me each one); drop_params: strip those fields in code.
- cache (simple or semantic): no equivalent; raise it if hit rates matter.
- input_guardrails / output_guardrails: Experiential guardrails (below);
list each check so I can recreate it.
- x-portkey-metadata: X-Explabs-Tags, with _user as safety_identifier.
x-portkey-trace-id: X-Explabs-Session-Id, or a W3C traceparent for the
Langfuse export.
- Prompt templates: no equivalent; ask me before inlining a template into
code.
The gateway's own equivalents, for the table:
- Fallback, retries and load balancing across providers of one model: the
gateway routes and fails over itself (GET https://api-pr-442.preview.experientiallabs.ai/api/models/<slug>/waterfall
shows the order). A fallback list ACROSS models has no per-request
equivalent.
- Keys per team or customer, spend caps, budgets: a PROVISIONING key (an org
admin mints the first at https://pr-442.preview.experientiallabs.ai/api-keys) mints inference keys with a
daily USD cap (POST https://api-pr-442.preview.experientiallabs.ai/api/v1/keys {"name", "limit"}) and sets monthly
caps per key or identity (PUT https://api-pr-442.preview.experientiallabs.ai/api/orgs/<org_id>/budgets).
- Metadata: the X-Explabs-Tags header (a JSON object, at most 16 labels, on
chat, Responses and Messages calls; never secrets or personal data). End
user: safety_identifier (alias user) in the body. Session or
conversation: the X-Explabs-Session-Id header.
- Logs and dashboards: https://pr-442.preview.experientiallabs.ai/logs and https://pr-442.preview.experientiallabs.ai/spend, the Cost API (step 5),
and live export of every call to my Langfuse project
(PUT https://api-pr-442.preview.experientiallabs.ai/api/orgs/<org_id>/observation-export).
- Privacy: "provider": {"zdr": true} per request, or account-wide
PUT https://api-pr-442.preview.experientiallabs.ai/api/orgs/<org_id>/zero-data-retention (Pro); processing
regions are in Settings, Data controls (https://pr-442.preview.experientiallabs.ai/settings/data-controls).
- Content guardrails: per identity, on the Guardrails tab at https://pr-442.preview.experientiallabs.ai/api-keys.
- Prompt caching: provider prompt caching applies where the route supports
it; Anthropic cache_control markers are kept on supported routes and
disclosed as not forwarded elsewhere (default five-minute markers; a
one-hour marker is refused on platform credits).
- No equivalent: gateway-side response or semantic caching, hosted prompt
templates, metadata-based routing between models, and arbitrary
request/response transforms.
Show me a table: feature in use -> equivalent (or "none") -> what changes.
Then ask me ONE question: proceed with this plan, adjust it, or stop. Wait.
A feature I depend on with no equivalent is a blocker: say so, never drop it
silently.
3. Point the clients at the gateway.
The Portkey client is OpenAI-shaped: replace Portkey(...), or an OpenAI
client built with createHeaders / PORTKEY_GATEWAY_URL, by a plain OpenAI
client (base URL https://api-pr-442.preview.experientiallabs.ai/v1, apiKey from EXPLABS_API_KEY) and keep the call
sites. Remove every x-portkey-* header and the config attachment.
OpenAI-compatible clients: base URL https://api-pr-442.preview.experientiallabs.ai/v1 and
Authorization: Bearer $EXPLABS_API_KEY (the ONLY header needed). An
Anthropic SDK appends /v1/messages itself: its base is the bare origin
https://api-pr-442.preview.experientiallabs.ai (never .../v1). A coding agent pointed at Portkey: follow that
agent's section at https://pr-442.preview.experientiallabs.ai/docs/coding-agents instead of guessing.
Store my key as EXPLABS_API_KEY in this project's env scheme (.env plus the
example file entry), never in code. Strip every Portkey-only field before
the first call: left in, each is ignored or refused with 400
unsupported_parameter naming it. Do NOT delete the old Portkey keys from
.env, secret stores or CI: stop reading them; I revoke them myself. A key
committed in a tracked file is already leaked: flag it and remove it there.
4. Map model ids. GET https://api-pr-442.preview.experientiallabs.ai/api/v1/models with my key (free, spends no
credits): every entry carries canonical_slug, the plain slug the gateway
serves. Portkey sends the provider's own model id (gpt-4o,
claude-sonnet-4-...) and picks the provider from the slug; a model may be
written "@provider-slug/model". Take the model part, read the model a
config's override_params sets when there is one, and match canonical_slug.
One obvious candidate: use it and note it in the report. Anything else:
ask me ONE question listing each id with your best candidates, then wait.
Never guess a model I did not pick. Use canonical_slug EXACTLY as returned.
5. Move cost and usage reads. Every chat / Responses reply carries usage.cost
(USD charged) and usage.is_byok; on a stream it rides the final usage chunk
when the request sets stream_options: {"include_usage": true}. A request
sent with Idempotency-Key carries no usage.cost, so read it as optional.
Same key, under https://api-pr-442.preview.experientiallabs.ai/api/v1:
GET /generation?id=<the x-request-id response header> one request
GET /usage settled per-request rows for billing (org-wide; filter
?api_key_id= or attribution_label)
GET /credits balance GET /activity per-day rollup
Replace the old cost and usage code with these; show me the diff first.
Portkey analytics and log exports have no counterpart in code: the Cost API
and https://pr-442.preview.experientiallabs.ai/spend replace them.
6. Provider keys (bring your own key). Portkey AI Providers (and legacy virtual keys) hold my provider credentials. Ask whether to connect
them so those calls bill my own provider accounts with no markup:
GET https://api-pr-442.preview.experientiallabs.ai/api/whoami -> org_id
GET https://api-pr-442.preview.experientiallabs.ai/api/orgs/<org_id>/provider-connections (list existing first)
POST https://api-pr-442.preview.experientiallabs.ai/api/orgs/<org_id>/provider-connections/<provider>
{"setup_alias": "<a new unique account name I approve>", "secret": "<key>"}
setup_alias: 1-63 lowercase letters, digits, hyphens or underscores, unused
across my accounts. Never overwrite an account; on 409 account_exists pick
another name. Use inference keys, never admin keys, and ask me to paste
each key: never export one from Portkey. A one-string key works for openai,
anthropic, gemini, xai, deepseek and similar; azure_openai, bedrock,
fireworks, vertex and modal need structured config: I connect those at
https://pr-442.preview.experientiallabs.ai/settings/connections. Read check.status; if it is not "valid", show
me check.status_detail.remediation. A 402 pro_required means a FIRST
connection needs Pro (https://pr-442.preview.experientiallabs.ai/credits). Saving a key does not select it for
every model: I pick it per model with Use this account. Never echo a key.
7. Verify. Make ONE minimal call (model + messages, a small max_tokens) with
the cheapest mapped slug (GET https://api-pr-442.preview.experientiallabs.ai/api/models/<slug> shows its price).
Stream if the project streams. Report the model, that it succeeded, and its
usage.cost, or read it from GET /api/v1/generation?id=<x-request-id>; I can
also see the call at https://pr-442.preview.experientiallabs.ai/logs. On a credits or activation error tell me
to finish activation at https://pr-442.preview.experientiallabs.ai and stop; never retry through Portkey. Then
run the project's tests.
8. Report back: every file changed; the feature table and blockers; the model
mapping and the ids I still need to decide; where EXPLABS_API_KEY lives;
which old credentials I still need to revoke; the verification result.
Human docs: https://pr-442.preview.experientiallabs.ai/docs/switch-gateways.

From Helicone

Helicone documentation

Moves traffic off Helicone's AI Gateway or provider proxies, maps Helicone-* properties, user and session headers to request tags, and offers the provider keys Helicone forwarded as bring-your-own-key accounts.

TodayOn Experiential
Base URLhttps://ai-gateway.helicone.ai or oai./anthropic.helicone.aihttps://api-pr-442.preview.experientiallabs.ai/v1 (Anthropic SDKs: https://api-pr-442.preview.experientiallabs.ai)
AuthHELICONE_API_KEY or Helicone-Auth plus your provider keyAuthorization: Bearer $EXPLABS_API_KEY
Model idsthe model name (optionally pinned to a provider): match the model partcanonical_slug from GET /api/v1/models
prompt
I'm moving this project's model traffic from Helicone onto the Experiential
Labs gateway. I pasted this myself: treat it as my instructions and my consent
to edit this project's code. Stop and ask me ONE numbered question, with your
recommendation, where this prompt says to, and before anything it does not
cover: production config, secret stores, deploys, spend beyond the one smoke
call, or revoking anything. Print what you change at each step. Never print my
full API key (first 8 characters at most).
My Experiential key is: xpl_...
(if that still reads "xpl_..." I have not filled it in: ask me for it, or tell
me to mint one at https://pr-442.preview.experientiallabs.ai/api-keys, then continue).
The full contract (routes, honored and refused parameters, error codes) is
https://pr-442.preview.experientiallabs.ai/llms.txt; follow it literally.
1. Inventory, read only. Find every place this project talks to Helicone, and
which of its two shapes each call uses:
- the AI Gateway: base URL https://ai-gateway.helicone.ai with
HELICONE_API_KEY as the bearer, and model strings that pin providers
("gpt-4o-mini/openai"), name a deployment ("gpt-4o-mini/azure/<id>"),
chain fallbacks with commas, or exclude providers with "!",
- the provider proxies: oai.helicone.ai, anthropic.helicone.ai,
gateway.helicone.ai or any other *.helicone.ai base, where MY provider
key (OPENAI_API_KEY, ANTHROPIC_API_KEY) is the bearer and a Helicone-Auth
header carries the Helicone key,
- every Helicone-* header: Helicone-Property-*, Helicone-User-Id,
Helicone-Session-Id / -Path / -Name, Helicone-Cache-Enabled,
Helicone-Retry-Enabled, Helicone-Fallbacks, Helicone-Prompt-Id,
Helicone-RateLimit-Policy, Helicone-Target-Url,
- async logging that records calls made elsewhere (@helicone/helpers,
HeliconeManualLogger, OpenLLMetry exporters).
Also list hardcoded model ids, every Helicone-specific header and body field,
retry / fallback / cache settings, coding-agent configs pointed at it, and
code that reads cost or usage from it. Show me the files and call sites
before changing anything. Inspect secret references, never their values.
2. Map Helicone's features before moving anything.
- Proxy calls ran on MY provider key and Helicone only observed: after the
move they run on Experiential credits unless I connect that key in step 6.
Say this plainly before I approve.
- Provider pins, fallback chains and exclusions within one model: the
gateway routes and fails over itself; a chain across DIFFERENT models:
keep the first model and tell me. An exclusion I rely on for policy
reasons is a data-control question: ask me.
- Helicone-Property-*: X-Explabs-Tags. Helicone-User-Id: safety_identifier.
Helicone-Session-Id: X-Explabs-Session-Id.
- Helicone-Cache-Enabled: no equivalent; raise it if hit rates matter.
Helicone-Retry-Enabled / Helicone-Fallbacks: the gateway does this
itself. Helicone-RateLimit-Policy: keys and budgets (below).
- Helicone-Prompt-Id and Helicone prompt management: no equivalent; raise it.
- Async logging of calls that should stay where they are: do not move
those; seeing them without moving inference is the "Capture traffic"
prompt (https://pr-442.preview.experientiallabs.ai/docs/setup-prompts#capture-traffic).
The gateway's own equivalents, for the table:
- Fallback, retries and load balancing across providers of one model: the
gateway routes and fails over itself (GET https://api-pr-442.preview.experientiallabs.ai/api/models/<slug>/waterfall
shows the order). A fallback list ACROSS models has no per-request
equivalent.
- Keys per team or customer, spend caps, budgets: a PROVISIONING key (an org
admin mints the first at https://pr-442.preview.experientiallabs.ai/api-keys) mints inference keys with a
daily USD cap (POST https://api-pr-442.preview.experientiallabs.ai/api/v1/keys {"name", "limit"}) and sets monthly
caps per key or identity (PUT https://api-pr-442.preview.experientiallabs.ai/api/orgs/<org_id>/budgets).
- Metadata: the X-Explabs-Tags header (a JSON object, at most 16 labels, on
chat, Responses and Messages calls; never secrets or personal data). End
user: safety_identifier (alias user) in the body. Session or
conversation: the X-Explabs-Session-Id header.
- Logs and dashboards: https://pr-442.preview.experientiallabs.ai/logs and https://pr-442.preview.experientiallabs.ai/spend, the Cost API (step 5),
and live export of every call to my Langfuse project
(PUT https://api-pr-442.preview.experientiallabs.ai/api/orgs/<org_id>/observation-export).
- Privacy: "provider": {"zdr": true} per request, or account-wide
PUT https://api-pr-442.preview.experientiallabs.ai/api/orgs/<org_id>/zero-data-retention (Pro); processing
regions are in Settings, Data controls (https://pr-442.preview.experientiallabs.ai/settings/data-controls).
- Content guardrails: per identity, on the Guardrails tab at https://pr-442.preview.experientiallabs.ai/api-keys.
- Prompt caching: provider prompt caching applies where the route supports
it; Anthropic cache_control markers are kept on supported routes and
disclosed as not forwarded elsewhere (default five-minute markers; a
one-hour marker is refused on platform credits).
- No equivalent: gateway-side response or semantic caching, hosted prompt
templates, metadata-based routing between models, and arbitrary
request/response transforms.
Show me a table: feature in use -> equivalent (or "none") -> what changes.
Then ask me ONE question: proceed with this plan, adjust it, or stop. Wait.
A feature I depend on with no equivalent is a blocker: say so, never drop it
silently.
3. Point the clients at the gateway.
Replace the Helicone base URL with https://api-pr-442.preview.experientiallabs.ai/v1 (Anthropic SDKs: https://api-pr-442.preview.experientiallabs.ai) and
the bearer with EXPLABS_API_KEY; remove Helicone-Auth and every Helicone-*
header. Do not chain the gateway behind a Helicone proxy.
OpenAI-compatible clients: base URL https://api-pr-442.preview.experientiallabs.ai/v1 and
Authorization: Bearer $EXPLABS_API_KEY (the ONLY header needed). An
Anthropic SDK appends /v1/messages itself: its base is the bare origin
https://api-pr-442.preview.experientiallabs.ai (never .../v1). A coding agent pointed at Helicone: follow that
agent's section at https://pr-442.preview.experientiallabs.ai/docs/coding-agents instead of guessing.
Store my key as EXPLABS_API_KEY in this project's env scheme (.env plus the
example file entry), never in code. Strip every Helicone-only field before
the first call: left in, each is ignored or refused with 400
unsupported_parameter naming it. Do NOT delete the old Helicone keys from
.env, secret stores or CI: stop reading them; I revoke them myself. A key
committed in a tracked file is already leaked: flag it and remove it there.
4. Map model ids. GET https://api-pr-442.preview.experientiallabs.ai/api/v1/models with my key (free, spends no
credits): every entry carries canonical_slug, the plain slug the gateway
serves. On the proxies the model is the provider's own id; on the AI
Gateway take the part before the first "/" of the first entry in a chain
(gpt-4o-mini/openai -> gpt-4o-mini) and match canonical_slug.
One obvious candidate: use it and note it in the report. Anything else:
ask me ONE question listing each id with your best candidates, then wait.
Never guess a model I did not pick. Use canonical_slug EXACTLY as returned.
5. Move cost and usage reads. Every chat / Responses reply carries usage.cost
(USD charged) and usage.is_byok; on a stream it rides the final usage chunk
when the request sets stream_options: {"include_usage": true}. A request
sent with Idempotency-Key carries no usage.cost, so read it as optional.
Same key, under https://api-pr-442.preview.experientiallabs.ai/api/v1:
GET /generation?id=<the x-request-id response header> one request
GET /usage settled per-request rows for billing (org-wide; filter
?api_key_id= or attribution_label)
GET /credits balance GET /activity per-day rollup
Replace the old cost and usage code with these; show me the diff first.
6. Provider keys (bring your own key). The proxies used my own provider keys, and the AI Gateway may hold BYOK keys in Helicone's Provider Settings. Ask whether to connect
them so those calls bill my own provider accounts with no markup:
GET https://api-pr-442.preview.experientiallabs.ai/api/whoami -> org_id
GET https://api-pr-442.preview.experientiallabs.ai/api/orgs/<org_id>/provider-connections (list existing first)
POST https://api-pr-442.preview.experientiallabs.ai/api/orgs/<org_id>/provider-connections/<provider>
{"setup_alias": "<a new unique account name I approve>", "secret": "<key>"}
setup_alias: 1-63 lowercase letters, digits, hyphens or underscores, unused
across my accounts. Never overwrite an account; on 409 account_exists pick
another name. Use inference keys, never admin keys, and ask me to paste
each key: never export one from Helicone. A one-string key works for openai,
anthropic, gemini, xai, deepseek and similar; azure_openai, bedrock,
fireworks, vertex and modal need structured config: I connect those at
https://pr-442.preview.experientiallabs.ai/settings/connections. Read check.status; if it is not "valid", show
me check.status_detail.remediation. A 402 pro_required means a FIRST
connection needs Pro (https://pr-442.preview.experientiallabs.ai/credits). Saving a key does not select it for
every model: I pick it per model with Use this account. Never echo a key.
7. Verify. Make ONE minimal call (model + messages, a small max_tokens) with
the cheapest mapped slug (GET https://api-pr-442.preview.experientiallabs.ai/api/models/<slug> shows its price).
Stream if the project streams. Report the model, that it succeeded, and its
usage.cost, or read it from GET /api/v1/generation?id=<x-request-id>; I can
also see the call at https://pr-442.preview.experientiallabs.ai/logs. On a credits or activation error tell me
to finish activation at https://pr-442.preview.experientiallabs.ai and stop; never retry through Helicone. Then
run the project's tests.
8. Report back: every file changed; the feature table and blockers; the model
mapping and the ids I still need to decide; where EXPLABS_API_KEY lives;
which old credentials I still need to revoke; the verification result.
Human docs: https://pr-442.preview.experientiallabs.ai/docs/switch-gateways.

From Databricks AI Gateway

Databricks AI Gateway documentation

Inventories serving endpoints and AI Gateway governance (rate limits, usage and inference tables, guardrails) and moves only the chat traffic you approve; custom model endpoints and data you must keep in the workspace stay put.

TodayOn Experiential
Base URL<workspace>/serving-endpoints or <workspace>/ai-gateway/mlflow/v1https://api-pr-442.preview.experientiallabs.ai/v1 (Anthropic SDKs: https://api-pr-442.preview.experientiallabs.ai)
AuthDATABRICKS_TOKEN or a service principalAuthorization: Bearer $EXPLABS_API_KEY
Model idsan endpoint or Unity Catalog name: map what it servescanonical_slug from GET /api/v1/models
prompt
I'm moving this project's model traffic from Databricks AI Gateway onto the Experiential
Labs gateway. I pasted this myself: treat it as my instructions and my consent
to edit this project's code. Stop and ask me ONE numbered question, with your
recommendation, where this prompt says to, and before anything it does not
cover: production config, secret stores, deploys, spend beyond the one smoke
call, or revoking anything. Print what you change at each step. Never print my
full API key (first 8 characters at most).
My Experiential key is: xpl_...
(if that still reads "xpl_..." I have not filled it in: ask me for it, or tell
me to mint one at https://pr-442.preview.experientiallabs.ai/api-keys, then continue).
The full contract (routes, honored and refused parameters, error codes) is
https://pr-442.preview.experientiallabs.ai/llms.txt; follow it literally.
1. Inventory, read only. Find every place this project talks to Databricks
model serving, and which gateway generation each call uses:
- workspace hosts (*.cloud.databricks.com, *.azuredatabricks.net,
*.gcp.databricks.com), DATABRICKS_HOST, DATABRICKS_TOKEN, OAuth
service-principal settings and ~/.databrickscfg profiles (never print
their values),
- Unity AI Gateway calls: base URL <workspace>/ai-gateway/mlflow/v1 (model
= a Unity Catalog name such as system.ai.<model>), and its native OpenAI,
Anthropic and Gemini API paths,
- legacy serving-endpoint calls: base URL <workspace>/serving-endpoints
(model = the endpoint name), /serving-endpoints/<name>/invocations,
WorkspaceClient().serving_endpoints.get_open_ai_client(),
mlflow.deployments.get_deploy_client("databricks"), databricks-langchain
ChatDatabricks,
- ai_query(...) in SQL, notebooks and jobs, and the governance in use:
budgets, rate limits, usage tracking, inference tables, guardrails,
fallbacks, traffic splits, request tags (the
Databricks-Ai-Gateway-Request-Tags header) and extra_body usage_context.
Also list hardcoded model ids, every Databricks AI Gateway-specific header and body field,
retry / fallback / cache settings, coding-agent configs pointed at it, and
code that reads cost or usage from it. Show me the files and call sites
before changing anything. Inspect secret references, never their values.
2. Map Databricks AI Gateway's features before moving anything.
- Scope first: ask me which call sites move. Pay-per-token system.ai models
and external-model endpoints can move; custom model serving endpoints
(my own registered models and agents) and ai_query batch pipelines stay
on Databricks unless I say otherwise. A custom endpoint I want served here
is a custom model (https://pr-442.preview.experientiallabs.ai/docs/integrate#custom-endpoints).
- Governance: if prompts and responses must stay inside the workspace
(inference tables, Unity Catalog lineage, private networking or a
compliance boundary), moving that traffic out is a blocker; ask before
proposing it.
- Budgets, rate limits and endpoint permissions: keys and budgets (below).
- Usage tables (system.ai_gateway.usage, system.serving.endpoint_usage):
the Cost API export (step 5); offer a scheduled job that lands
/api/v1/usage rows in a Delta table if I want them in Databricks.
- Inference tables (payload logging): the prompt and response storage
switch at https://pr-442.preview.experientiallabs.ai/settings/data-controls plus the Langfuse export.
- Request tags / usage_context: X-Explabs-Tags. Fallbacks and traffic
splits across providers of one model: the gateway does this itself.
- Guardrails: Experiential guardrails (below); list each one.
The gateway's own equivalents, for the table:
- Fallback, retries and load balancing across providers of one model: the
gateway routes and fails over itself (GET https://api-pr-442.preview.experientiallabs.ai/api/models/<slug>/waterfall
shows the order). A fallback list ACROSS models has no per-request
equivalent.
- Keys per team or customer, spend caps, budgets: a PROVISIONING key (an org
admin mints the first at https://pr-442.preview.experientiallabs.ai/api-keys) mints inference keys with a
daily USD cap (POST https://api-pr-442.preview.experientiallabs.ai/api/v1/keys {"name", "limit"}) and sets monthly
caps per key or identity (PUT https://api-pr-442.preview.experientiallabs.ai/api/orgs/<org_id>/budgets).
- Metadata: the X-Explabs-Tags header (a JSON object, at most 16 labels, on
chat, Responses and Messages calls; never secrets or personal data). End
user: safety_identifier (alias user) in the body. Session or
conversation: the X-Explabs-Session-Id header.
- Logs and dashboards: https://pr-442.preview.experientiallabs.ai/logs and https://pr-442.preview.experientiallabs.ai/spend, the Cost API (step 5),
and live export of every call to my Langfuse project
(PUT https://api-pr-442.preview.experientiallabs.ai/api/orgs/<org_id>/observation-export).
- Privacy: "provider": {"zdr": true} per request, or account-wide
PUT https://api-pr-442.preview.experientiallabs.ai/api/orgs/<org_id>/zero-data-retention (Pro); processing
regions are in Settings, Data controls (https://pr-442.preview.experientiallabs.ai/settings/data-controls).
- Content guardrails: per identity, on the Guardrails tab at https://pr-442.preview.experientiallabs.ai/api-keys.
- Prompt caching: provider prompt caching applies where the route supports
it; Anthropic cache_control markers are kept on supported routes and
disclosed as not forwarded elsewhere (default five-minute markers; a
one-hour marker is refused on platform credits).
- No equivalent: gateway-side response or semantic caching, hosted prompt
templates, metadata-based routing between models, and arbitrary
request/response transforms.
Show me a table: feature in use -> equivalent (or "none") -> what changes.
Then ask me ONE question: proceed with this plan, adjust it, or stop. Wait.
A feature I depend on with no equivalent is a blocker: say so, never drop it
silently.
3. Point the clients at the gateway, only at the call sites I approved.
An OpenAI client gets base URL https://api-pr-442.preview.experientiallabs.ai/v1 and EXPLABS_API_KEY instead of the
workspace URL and DATABRICKS_TOKEN; ChatDatabricks becomes ChatOpenAI
(langchain-openai) with the same base URL and key; mlflow deployments
clients become a plain OpenAI client. A Unity AI Gateway Responses call
stays a Responses call (the gateway serves /v1/responses). Leave every
DATABRICKS_* setting in place for what stays.
OpenAI-compatible clients: base URL https://api-pr-442.preview.experientiallabs.ai/v1 and
Authorization: Bearer $EXPLABS_API_KEY (the ONLY header needed). An
Anthropic SDK appends /v1/messages itself: its base is the bare origin
https://api-pr-442.preview.experientiallabs.ai (never .../v1). A coding agent pointed at Databricks AI Gateway: follow that
agent's section at https://pr-442.preview.experientiallabs.ai/docs/coding-agents instead of guessing.
Store my key as EXPLABS_API_KEY in this project's env scheme (.env plus the
example file entry), never in code. Strip every Databricks AI Gateway-only field before
the first call: left in, each is ignored or refused with 400
unsupported_parameter naming it. Do NOT delete the old Databricks AI Gateway keys from
.env, secret stores or CI: stop reading them; I revoke them myself. A key
committed in a tracked file is already leaked: flag it and remove it there.
4. Map model ids. GET https://api-pr-442.preview.experientiallabs.ai/api/v1/models with my key (free, spends no
credits): every entry carries canonical_slug, the plain slug the gateway
serves. A Databricks model is a Unity Catalog name or an endpoint name,
not a model id: for each one, find what it serves (system.ai.<model> names
the model; for an endpoint read its served entities or external_model.name
in the workspace UI or GET /api/2.0/serving-endpoints/<name>, with my
approval and my own credentials), then match canonical_slug.
One obvious candidate: use it and note it in the report. Anything else:
ask me ONE question listing each id with your best candidates, then wait.
Never guess a model I did not pick. Use canonical_slug EXACTLY as returned.
5. Move cost and usage reads. Every chat / Responses reply carries usage.cost
(USD charged) and usage.is_byok; on a stream it rides the final usage chunk
when the request sets stream_options: {"include_usage": true}. A request
sent with Idempotency-Key carries no usage.cost, so read it as optional.
Same key, under https://api-pr-442.preview.experientiallabs.ai/api/v1:
GET /generation?id=<the x-request-id response header> one request
GET /usage settled per-request rows for billing (org-wide; filter
?api_key_id= or attribution_label)
GET /credits balance GET /activity per-day rollup
Replace the old cost and usage code with these; show me the diff first.
6. Provider keys (bring your own key). External-model endpoints hold my provider keys in Databricks secrets; never read a Databricks secret yourself. Ask whether to connect
them so those calls bill my own provider accounts with no markup:
GET https://api-pr-442.preview.experientiallabs.ai/api/whoami -> org_id
GET https://api-pr-442.preview.experientiallabs.ai/api/orgs/<org_id>/provider-connections (list existing first)
POST https://api-pr-442.preview.experientiallabs.ai/api/orgs/<org_id>/provider-connections/<provider>
{"setup_alias": "<a new unique account name I approve>", "secret": "<key>"}
setup_alias: 1-63 lowercase letters, digits, hyphens or underscores, unused
across my accounts. Never overwrite an account; on 409 account_exists pick
another name. Use inference keys, never admin keys, and ask me to paste
each key: never export one from Databricks AI Gateway. A one-string key works for openai,
anthropic, gemini, xai, deepseek and similar; azure_openai, bedrock,
fireworks, vertex and modal need structured config: I connect those at
https://pr-442.preview.experientiallabs.ai/settings/connections. Read check.status; if it is not "valid", show
me check.status_detail.remediation. A 402 pro_required means a FIRST
connection needs Pro (https://pr-442.preview.experientiallabs.ai/credits). Saving a key does not select it for
every model: I pick it per model with Use this account. Never echo a key.
7. Verify. Make ONE minimal call (model + messages, a small max_tokens) with
the cheapest mapped slug (GET https://api-pr-442.preview.experientiallabs.ai/api/models/<slug> shows its price).
Stream if the project streams. Report the model, that it succeeded, and its
usage.cost, or read it from GET /api/v1/generation?id=<x-request-id>; I can
also see the call at https://pr-442.preview.experientiallabs.ai/logs. On a credits or activation error tell me
to finish activation at https://pr-442.preview.experientiallabs.ai and stop; never retry through Databricks AI Gateway. Then
run the project's tests.
8. Report back: every file changed; the feature table and blockers; the model
mapping and the ids I still need to decide; where EXPLABS_API_KEY lives;
which old credentials I still need to revoke; the verification result.
Human docs: https://pr-442.preview.experientiallabs.ai/docs/switch-gateways.

From Kong AI Gateway

Kong AI Gateway documentation

Either repoints Kong's ai-proxy upstream at the gateway (Kong keeps its auth and policies, clients unchanged) or moves clients off Kong, with the key held in a Kong vault reference, never in committed config.

TodayOn Experiential
Base URLyour Kong route (ai-proxy / ai-proxy-advanced)https://api-pr-442.preview.experientiallabs.ai/v1 (Anthropic SDKs: https://api-pr-442.preview.experientiallabs.ai)
AuthKong consumer credentials; provider key in the pluginAuthorization: Bearer $EXPLABS_API_KEY
Model idsconfig.model.name: map it, or keep Kong and repoint upstream_urlcanonical_slug from GET /api/v1/models
prompt
I'm moving this project's model traffic from Kong AI Gateway onto the Experiential
Labs gateway. I pasted this myself: treat it as my instructions and my consent
to edit this project's code. Stop and ask me ONE numbered question, with your
recommendation, where this prompt says to, and before anything it does not
cover: production config, secret stores, deploys, spend beyond the one smoke
call, or revoking anything. Print what you change at each step. Never print my
full API key (first 8 characters at most).
My Experiential key is: xpl_...
(if that still reads "xpl_..." I have not filled it in: ask me for it, or tell
me to mint one at https://pr-442.preview.experientiallabs.ai/api-keys, then continue).
The full contract (routes, honored and refused parameters, error codes) is
https://pr-442.preview.experientiallabs.ai/llms.txt; follow it literally.
1. Inventory, read only. Find every place this project talks to Kong AI Gateway:
- Kong configuration: declarative kong.yaml / decK files, Helm values,
KongPlugin / KongClusterPlugin resources, or Konnect, with the ai-proxy or
ai-proxy-advanced plugin and its companions (ai-rate-limiting-advanced,
ai-semantic-cache, ai-prompt-guard, ai-prompt-decorator,
ai-prompt-template, ai-request-transformer, ai-response-transformer),
- per plugin: config.route_type, config.model.provider and name,
config.model.options (upstream_url and provider options),
config.auth (header_name, header_value, allow_override), config.llm_format
and config.logging,
- the routes clients call (the Kong proxy host and route paths) and the
consumer credentials they send (key-auth apikey header, JWT, OIDC),
- provider keys referenced by the plugins and the vault references holding
them.
Also list hardcoded model ids, every Kong AI Gateway-specific header and body field,
retry / fallback / cache settings, coding-agent configs pointed at it, and
code that reads cost or usage from it. Show me the files and call sites
before changing anything. Inspect secret references, never their values.
2. Map Kong AI Gateway's features before moving anything.
- ai-rate-limiting-advanced, ai-prompt-guard, ai-semantic-cache, the
decorators, templates and transformers: under (a) below they keep running
in Kong; under (b) rate limits become keys and budgets, prompt guard
becomes guardrails, and the cache, templates and transformers have no
equivalent.
- ai-proxy-advanced balancing and fallbacks across providers of one model:
the gateway does this itself; across different models: raise it.
- Kong consumers and their credentials: under (a) they stay; under (b)
each consumer that needs its own spend becomes its own Experiential key.
- logging.log_statistics / log_payloads: the Cost API and the Langfuse
export (below).
The gateway's own equivalents, for the table:
- Fallback, retries and load balancing across providers of one model: the
gateway routes and fails over itself (GET https://api-pr-442.preview.experientiallabs.ai/api/models/<slug>/waterfall
shows the order). A fallback list ACROSS models has no per-request
equivalent.
- Keys per team or customer, spend caps, budgets: a PROVISIONING key (an org
admin mints the first at https://pr-442.preview.experientiallabs.ai/api-keys) mints inference keys with a
daily USD cap (POST https://api-pr-442.preview.experientiallabs.ai/api/v1/keys {"name", "limit"}) and sets monthly
caps per key or identity (PUT https://api-pr-442.preview.experientiallabs.ai/api/orgs/<org_id>/budgets).
- Metadata: the X-Explabs-Tags header (a JSON object, at most 16 labels, on
chat, Responses and Messages calls; never secrets or personal data). End
user: safety_identifier (alias user) in the body. Session or
conversation: the X-Explabs-Session-Id header.
- Logs and dashboards: https://pr-442.preview.experientiallabs.ai/logs and https://pr-442.preview.experientiallabs.ai/spend, the Cost API (step 5),
and live export of every call to my Langfuse project
(PUT https://api-pr-442.preview.experientiallabs.ai/api/orgs/<org_id>/observation-export).
- Privacy: "provider": {"zdr": true} per request, or account-wide
PUT https://api-pr-442.preview.experientiallabs.ai/api/orgs/<org_id>/zero-data-retention (Pro); processing
regions are in Settings, Data controls (https://pr-442.preview.experientiallabs.ai/settings/data-controls).
- Content guardrails: per identity, on the Guardrails tab at https://pr-442.preview.experientiallabs.ai/api-keys.
- Prompt caching: provider prompt caching applies where the route supports
it; Anthropic cache_control markers are kept on supported routes and
disclosed as not forwarded elsewhere (default five-minute markers; a
one-hour marker is refused on platform credits).
- No equivalent: gateway-side response or semantic caching, hosted prompt
templates, metadata-based routing between models, and arbitrary
request/response transforms.
Show me a table: feature in use -> equivalent (or "none") -> what changes.
Then ask me ONE question: proceed with this plan, adjust it, or stop. Wait.
A feature I depend on with no equivalent is a blocker: say so, never drop it
silently.
3. Point the clients at the gateway.
First ask me ONE question, with your recommendation, then wait:
(a) keep Kong as the edge and repoint each ai-proxy plugin (or each
ai-proxy-advanced target) at the gateway: clients, consumers and Kong
policies stay unchanged. Per plugin, after checking these names
against the installed Kong version's ai-proxy schema:
config.route_type: llm/v1/chat (llm/v1/responses for Responses)
config.model.provider: openai
config.model.name: <slug>
config.model.options.upstream_url: https://api-pr-442.preview.experientiallabs.ai/v1/chat/completions
(https://api-pr-442.preview.experientiallabs.ai/v1/responses for a responses route)
config.auth.header_name: Authorization
config.auth.header_value: a Kong vault reference whose secret is the
whole value "Bearer <my key>" (a reference replaces the field's
entire value), never the key written into committed config
config.auth.allow_override: false
Ask me which vault (env, AWS / GCP / Azure secret manager, HashiCorp)
holds it. Recommended when Kong enforces auth or policies for other
teams;
(b) move clients off Kong for model traffic and point them at the gateway
directly; other Kong routes stay as they are.
OpenAI-compatible clients: base URL https://api-pr-442.preview.experientiallabs.ai/v1 and
Authorization: Bearer $EXPLABS_API_KEY (the ONLY header needed). An
Anthropic SDK appends /v1/messages itself: its base is the bare origin
https://api-pr-442.preview.experientiallabs.ai (never .../v1). A coding agent pointed at Kong AI Gateway: follow that
agent's section at https://pr-442.preview.experientiallabs.ai/docs/coding-agents instead of guessing.
Store my key as EXPLABS_API_KEY in this project's env scheme (.env plus the
example file entry), never in code. Strip every Kong AI Gateway-only field before
the first call: left in, each is ignored or refused with 400
unsupported_parameter naming it. Do NOT delete the old Kong AI Gateway keys from
.env, secret stores or CI: stop reading them; I revoke them myself. A key
committed in a tracked file is already leaked: flag it and remove it there.
4. Map model ids. GET https://api-pr-442.preview.experientiallabs.ai/api/v1/models with my key (free, spends no
credits): every entry carries canonical_slug, the plain slug the gateway
serves. Kong pins one model per plugin (config.model.name) or lets the
client send it: map each to canonical_slug. Under (a) the route paths the
clients call stay the same.
One obvious candidate: use it and note it in the report. Anything else:
ask me ONE question listing each id with your best candidates, then wait.
Never guess a model I did not pick. Use canonical_slug EXACTLY as returned.
5. Move cost and usage reads. Every chat / Responses reply carries usage.cost
(USD charged) and usage.is_byok; on a stream it rides the final usage chunk
when the request sets stream_options: {"include_usage": true}. A request
sent with Idempotency-Key carries no usage.cost, so read it as optional.
Same key, under https://api-pr-442.preview.experientiallabs.ai/api/v1:
GET /generation?id=<the x-request-id response header> one request
GET /usage settled per-request rows for billing (org-wide; filter
?api_key_id= or attribution_label)
GET /credits balance GET /activity per-day rollup
Replace the old cost and usage code with these; show me the diff first.
6. Provider keys (bring your own key). The plugins authenticate to my providers with my own keys (config.auth, often through a vault). Ask whether to connect
them so those calls bill my own provider accounts with no markup:
GET https://api-pr-442.preview.experientiallabs.ai/api/whoami -> org_id
GET https://api-pr-442.preview.experientiallabs.ai/api/orgs/<org_id>/provider-connections (list existing first)
POST https://api-pr-442.preview.experientiallabs.ai/api/orgs/<org_id>/provider-connections/<provider>
{"setup_alias": "<a new unique account name I approve>", "secret": "<key>"}
setup_alias: 1-63 lowercase letters, digits, hyphens or underscores, unused
across my accounts. Never overwrite an account; on 409 account_exists pick
another name. Use inference keys, never admin keys, and ask me to paste
each key: never export one from Kong AI Gateway. A one-string key works for openai,
anthropic, gemini, xai, deepseek and similar; azure_openai, bedrock,
fireworks, vertex and modal need structured config: I connect those at
https://pr-442.preview.experientiallabs.ai/settings/connections. Read check.status; if it is not "valid", show
me check.status_detail.remediation. A 402 pro_required means a FIRST
connection needs Pro (https://pr-442.preview.experientiallabs.ai/credits). Saving a key does not select it for
every model: I pick it per model with Use this account. Never echo a key.
7. Verify. Make ONE minimal call (model + messages, a small max_tokens) with
the cheapest mapped slug (GET https://api-pr-442.preview.experientiallabs.ai/api/models/<slug> shows its price).
Stream if the project streams. Report the model, that it succeeded, and its
usage.cost, or read it from GET /api/v1/generation?id=<x-request-id>; I can
also see the call at https://pr-442.preview.experientiallabs.ai/logs. On a credits or activation error tell me
to finish activation at https://pr-442.preview.experientiallabs.ai and stop; never retry through Kong AI Gateway. Then
run the project's tests.
8. Report back: every file changed; the feature table and blockers; the model
mapping and the ids I still need to decide; where EXPLABS_API_KEY lives;
which old credentials I still need to revoke; the verification result.
Human docs: https://pr-442.preview.experientiallabs.ai/docs/switch-gateways.

From TrueFoundry AI Gateway

TrueFoundry AI Gateway documentation

Moves OpenAI clients off the TrueFoundry gateway, maps provider-account/model ids to catalog slugs, and maps X-TFY-METADATA, routing configs, rate limits and budgets to their equivalents.

TodayOn Experiential
Base URL<control-plane>/api/llmhttps://api-pr-442.preview.experientiallabs.ai/v1 (Anthropic SDKs: https://api-pr-442.preview.experientiallabs.ai)
Autha TrueFoundry tokenAuthorization: Bearer $EXPLABS_API_KEY
Model idsprovider-account/model: strip the account prefixcanonical_slug from GET /api/v1/models
prompt
I'm moving this project's model traffic from TrueFoundry AI Gateway onto the Experiential
Labs gateway. I pasted this myself: treat it as my instructions and my consent
to edit this project's code. Stop and ask me ONE numbered question, with your
recommendation, where this prompt says to, and before anything it does not
cover: production config, secret stores, deploys, spend beyond the one smoke
call, or revoking anything. Print what you change at each step. Never print my
full API key (first 8 characters at most).
My Experiential key is: xpl_...
(if that still reads "xpl_..." I have not filled it in: ask me for it, or tell
me to mint one at https://pr-442.preview.experientiallabs.ai/api-keys, then continue).
The full contract (routes, honored and refused parameters, error codes) is
https://pr-442.preview.experientiallabs.ai/llms.txt; follow it literally.
1. Inventory, read only. Find every place this project talks to TrueFoundry AI Gateway:
- the gateway base URL as the project spells it (TFY_GATEWAY_BASE_URL,
GATEWAY_BASE_URL, or a literal such as https://<control-plane>/api/llm;
the docs use more than one form, so record what the code actually uses),
TFY_API_KEY or another TrueFoundry token,
- model ids shaped <provider-account>/<model> (openai-main/gpt-4o-mini), and
batch / fine-tuning bodies that use the bare model name,
- the X-TFY-METADATA header (a JSON string, often application, environment
and customer ids), x-tfy-provider-name (required there for the Responses,
batch, files and fine-tuning APIs) and other x-tfy-* headers,
- gateway configs in the repo or the TrueFoundry UI: routing and
load-balancing (including virtual model names), fallbacks, rate limits,
budgets and guardrails, and which metadata keys they match on.
Also list hardcoded model ids, every TrueFoundry AI Gateway-specific header and body field,
retry / fallback / cache settings, coding-agent configs pointed at it, and
code that reads cost or usage from it. Show me the files and call sites
before changing anything. Inspect secret references, never their values.
2. Map TrueFoundry AI Gateway's features before moving anything.
- Load balancing, fallbacks and retries across providers of one model: the
gateway does this itself. A virtual model that routes to DIFFERENT models,
or routing on X-TFY-METADATA: no equivalent; pick one model per call site
with me.
- X-TFY-METADATA: X-Explabs-Tags (a customer id goes to safety_identifier).
- Rate limits and budgets per user, team or metadata value: keys and
budgets (below); rules keyed on arbitrary metadata have no equivalent.
- Guardrails: Experiential guardrails (below); list each one.
- Batch jobs: the gateway serves /v1/batches; ask me before moving one.
The gateway's own equivalents, for the table:
- Fallback, retries and load balancing across providers of one model: the
gateway routes and fails over itself (GET https://api-pr-442.preview.experientiallabs.ai/api/models/<slug>/waterfall
shows the order). A fallback list ACROSS models has no per-request
equivalent.
- Keys per team or customer, spend caps, budgets: a PROVISIONING key (an org
admin mints the first at https://pr-442.preview.experientiallabs.ai/api-keys) mints inference keys with a
daily USD cap (POST https://api-pr-442.preview.experientiallabs.ai/api/v1/keys {"name", "limit"}) and sets monthly
caps per key or identity (PUT https://api-pr-442.preview.experientiallabs.ai/api/orgs/<org_id>/budgets).
- Metadata: the X-Explabs-Tags header (a JSON object, at most 16 labels, on
chat, Responses and Messages calls; never secrets or personal data). End
user: safety_identifier (alias user) in the body. Session or
conversation: the X-Explabs-Session-Id header.
- Logs and dashboards: https://pr-442.preview.experientiallabs.ai/logs and https://pr-442.preview.experientiallabs.ai/spend, the Cost API (step 5),
and live export of every call to my Langfuse project
(PUT https://api-pr-442.preview.experientiallabs.ai/api/orgs/<org_id>/observation-export).
- Privacy: "provider": {"zdr": true} per request, or account-wide
PUT https://api-pr-442.preview.experientiallabs.ai/api/orgs/<org_id>/zero-data-retention (Pro); processing
regions are in Settings, Data controls (https://pr-442.preview.experientiallabs.ai/settings/data-controls).
- Content guardrails: per identity, on the Guardrails tab at https://pr-442.preview.experientiallabs.ai/api-keys.
- Prompt caching: provider prompt caching applies where the route supports
it; Anthropic cache_control markers are kept on supported routes and
disclosed as not forwarded elsewhere (default five-minute markers; a
one-hour marker is refused on platform credits).
- No equivalent: gateway-side response or semantic caching, hosted prompt
templates, metadata-based routing between models, and arbitrary
request/response transforms.
Show me a table: feature in use -> equivalent (or "none") -> what changes.
Then ask me ONE question: proceed with this plan, adjust it, or stop. Wait.
A feature I depend on with no equivalent is a blocker: say so, never drop it
silently.
3. Point the clients at the gateway.
Replace the TrueFoundry base URL with https://api-pr-442.preview.experientiallabs.ai/v1 and the token with
EXPLABS_API_KEY; remove X-TFY-METADATA, x-tfy-provider-name and every
x-tfy-* header once their content is mapped.
OpenAI-compatible clients: base URL https://api-pr-442.preview.experientiallabs.ai/v1 and
Authorization: Bearer $EXPLABS_API_KEY (the ONLY header needed). An
Anthropic SDK appends /v1/messages itself: its base is the bare origin
https://api-pr-442.preview.experientiallabs.ai (never .../v1). A coding agent pointed at TrueFoundry AI Gateway: follow that
agent's section at https://pr-442.preview.experientiallabs.ai/docs/coding-agents instead of guessing.
Store my key as EXPLABS_API_KEY in this project's env scheme (.env plus the
example file entry), never in code. Strip every TrueFoundry AI Gateway-only field before
the first call: left in, each is ignored or refused with 400
unsupported_parameter naming it. Do NOT delete the old TrueFoundry AI Gateway keys from
.env, secret stores or CI: stop reading them; I revoke them myself. A key
committed in a tracked file is already leaked: flag it and remove it there.
4. Map model ids. GET https://api-pr-442.preview.experientiallabs.ai/api/v1/models with my key (free, spends no
credits): every entry carries canonical_slug, the plain slug the gateway
serves. Strip the provider-account prefix (openai-main/gpt-4o-mini ->
gpt-4o-mini; a virtual model name: read what it routes to) and match
canonical_slug.
One obvious candidate: use it and note it in the report. Anything else:
ask me ONE question listing each id with your best candidates, then wait.
Never guess a model I did not pick. Use canonical_slug EXACTLY as returned.
5. Move cost and usage reads. Every chat / Responses reply carries usage.cost
(USD charged) and usage.is_byok; on a stream it rides the final usage chunk
when the request sets stream_options: {"include_usage": true}. A request
sent with Idempotency-Key carries no usage.cost, so read it as optional.
Same key, under https://api-pr-442.preview.experientiallabs.ai/api/v1:
GET /generation?id=<the x-request-id response header> one request
GET /usage settled per-request rows for billing (org-wide; filter
?api_key_id= or attribution_label)
GET /credits balance GET /activity per-day rollup
Replace the old cost and usage code with these; show me the diff first.
6. Provider keys (bring your own key). TrueFoundry provider accounts hold my provider keys. Ask whether to connect
them so those calls bill my own provider accounts with no markup:
GET https://api-pr-442.preview.experientiallabs.ai/api/whoami -> org_id
GET https://api-pr-442.preview.experientiallabs.ai/api/orgs/<org_id>/provider-connections (list existing first)
POST https://api-pr-442.preview.experientiallabs.ai/api/orgs/<org_id>/provider-connections/<provider>
{"setup_alias": "<a new unique account name I approve>", "secret": "<key>"}
setup_alias: 1-63 lowercase letters, digits, hyphens or underscores, unused
across my accounts. Never overwrite an account; on 409 account_exists pick
another name. Use inference keys, never admin keys, and ask me to paste
each key: never export one from TrueFoundry AI Gateway. A one-string key works for openai,
anthropic, gemini, xai, deepseek and similar; azure_openai, bedrock,
fireworks, vertex and modal need structured config: I connect those at
https://pr-442.preview.experientiallabs.ai/settings/connections. Read check.status; if it is not "valid", show
me check.status_detail.remediation. A 402 pro_required means a FIRST
connection needs Pro (https://pr-442.preview.experientiallabs.ai/credits). Saving a key does not select it for
every model: I pick it per model with Use this account. Never echo a key.
7. Verify. Make ONE minimal call (model + messages, a small max_tokens) with
the cheapest mapped slug (GET https://api-pr-442.preview.experientiallabs.ai/api/models/<slug> shows its price).
Stream if the project streams. Report the model, that it succeeded, and its
usage.cost, or read it from GET /api/v1/generation?id=<x-request-id>; I can
also see the call at https://pr-442.preview.experientiallabs.ai/logs. On a credits or activation error tell me
to finish activation at https://pr-442.preview.experientiallabs.ai and stop; never retry through TrueFoundry AI Gateway. Then
run the project's tests.
8. Report back: every file changed; the feature table and blockers; the model
mapping and the ids I still need to decide; where EXPLABS_API_KEY lives;
which old credentials I still need to revoke; the verification result.
Human docs: https://pr-442.preview.experientiallabs.ai/docs/switch-gateways.

From Azure API Management

Azure API Management documentation

Replaces AzureOpenAI clients behind an azure-api.net endpoint with a plain OpenAI client on the gateway, maps deployment names to slugs, maps token-limit and metric policies, and offers your Azure OpenAI resource as a bring-your-own-key account.

TodayOn Experiential
Base URLhttps://<apim>.azure-api.net/... (AzureOpenAI client)https://api-pr-442.preview.experientiallabs.ai/v1 (Anthropic SDKs: https://api-pr-442.preview.experientiallabs.ai)
Authapi-key / Ocp-Apim-Subscription-KeyAuthorization: Bearer $EXPLABS_API_KEY
Model idsdeployment names: map the model behind each deploymentcanonical_slug from GET /api/v1/models
prompt
I'm moving this project's model traffic from Azure API Management onto the Experiential
Labs gateway. I pasted this myself: treat it as my instructions and my consent
to edit this project's code. Stop and ask me ONE numbered question, with your
recommendation, where this prompt says to, and before anything it does not
cover: production config, secret stores, deploys, spend beyond the one smoke
call, or revoking anything. Print what you change at each step. Never print my
full API key (first 8 characters at most).
My Experiential key is: xpl_...
(if that still reads "xpl_..." I have not filled it in: ask me for it, or tell
me to mint one at https://pr-442.preview.experientiallabs.ai/api-keys, then continue).
The full contract (routes, honored and refused parameters, error codes) is
https://pr-442.preview.experientiallabs.ai/llms.txt; follow it literally.
1. Inventory, read only. Find every place this project talks to Azure API Management:
- *.azure-api.net endpoints and the clients pointed at them: AzureOpenAI
(azure_endpoint, api_version, azure_deployment; the model argument is a
DEPLOYMENT name), OpenAI clients on an APIM unified model API, and
Anthropic clients on an APIM Messages API,
- AZURE_OPENAI_ENDPOINT, AZURE_OPENAI_API_KEY, OPENAI_API_VERSION, the
Ocp-Apim-Subscription-Key or api-key header, and azure-identity token
providers (DefaultAzureCredential, get_bearer_token_provider),
- the APIM policies on those APIs (llm-token-limit /
azure-openai-token-limit, llm-emit-token-metric /
azure-openai-emit-token-metric, llm-semantic-cache-lookup / -store,
llm-content-safety), backend pools, load balancing and circuit breakers,
and logging to Azure Monitor / Application Insights, in Bicep, Terraform,
ARM or APIOps files.
Also list hardcoded model ids, every Azure API Management-specific header and body field,
retry / fallback / cache settings, coding-agent configs pointed at it, and
code that reads cost or usage from it. Show me the files and call sites
before changing anything. Inspect secret references, never their values.
2. Map Azure API Management's features before moving anything.
- Token limits and quotas per subscription key or counter key: keys and
budgets (below).
- Token metrics and prompt logging to Azure Monitor: the Cost API export
(step 5), https://pr-442.preview.experientiallabs.ai/spend, and the Langfuse export.
- Backend pools, priority / weighted balancing and circuit breakers across
deployments of one model: the gateway does this itself. Priority routing
that keeps reserved capacity (PTU) busy first: keep my Azure resource as a
bring-your-own-key account (step 6) and order it first on that model.
- Semantic cache: no equivalent; raise it.
- Content safety: Experiential guardrails (below), plus Azure's own content
filtering if my Azure resource stays as a bring-your-own-key account.
- Data residency of my Azure region: a data-control question; ask me.
- Do not change APIM, Azure resources or IaC beyond what I approve; other
APIs behind the same APIM instance stay as they are.
The gateway's own equivalents, for the table:
- Fallback, retries and load balancing across providers of one model: the
gateway routes and fails over itself (GET https://api-pr-442.preview.experientiallabs.ai/api/models/<slug>/waterfall
shows the order). A fallback list ACROSS models has no per-request
equivalent.
- Keys per team or customer, spend caps, budgets: a PROVISIONING key (an org
admin mints the first at https://pr-442.preview.experientiallabs.ai/api-keys) mints inference keys with a
daily USD cap (POST https://api-pr-442.preview.experientiallabs.ai/api/v1/keys {"name", "limit"}) and sets monthly
caps per key or identity (PUT https://api-pr-442.preview.experientiallabs.ai/api/orgs/<org_id>/budgets).
- Metadata: the X-Explabs-Tags header (a JSON object, at most 16 labels, on
chat, Responses and Messages calls; never secrets or personal data). End
user: safety_identifier (alias user) in the body. Session or
conversation: the X-Explabs-Session-Id header.
- Logs and dashboards: https://pr-442.preview.experientiallabs.ai/logs and https://pr-442.preview.experientiallabs.ai/spend, the Cost API (step 5),
and live export of every call to my Langfuse project
(PUT https://api-pr-442.preview.experientiallabs.ai/api/orgs/<org_id>/observation-export).
- Privacy: "provider": {"zdr": true} per request, or account-wide
PUT https://api-pr-442.preview.experientiallabs.ai/api/orgs/<org_id>/zero-data-retention (Pro); processing
regions are in Settings, Data controls (https://pr-442.preview.experientiallabs.ai/settings/data-controls).
- Content guardrails: per identity, on the Guardrails tab at https://pr-442.preview.experientiallabs.ai/api-keys.
- Prompt caching: provider prompt caching applies where the route supports
it; Anthropic cache_control markers are kept on supported routes and
disclosed as not forwarded elsewhere (default five-minute markers; a
one-hour marker is refused on platform credits).
- No equivalent: gateway-side response or semantic caching, hosted prompt
templates, metadata-based routing between models, and arbitrary
request/response transforms.
Show me a table: feature in use -> equivalent (or "none") -> what changes.
Then ask me ONE question: proceed with this plan, adjust it, or stop. Wait.
A feature I depend on with no equivalent is a blocker: say so, never drop it
silently.
3. Point the clients at the gateway.
Replace AzureOpenAI(azure_endpoint=..., api_version=...) with a plain
OpenAI client (base URL https://api-pr-442.preview.experientiallabs.ai/v1, key from EXPLABS_API_KEY), drop the
api-version query parameter, the deployment path and the api-key /
Ocp-Apim-Subscription-Key headers, and pass the mapped slug as model.
Remove the azure-identity token provider from these clients only.
OpenAI-compatible clients: base URL https://api-pr-442.preview.experientiallabs.ai/v1 and
Authorization: Bearer $EXPLABS_API_KEY (the ONLY header needed). An
Anthropic SDK appends /v1/messages itself: its base is the bare origin
https://api-pr-442.preview.experientiallabs.ai (never .../v1). A coding agent pointed at Azure API Management: follow that
agent's section at https://pr-442.preview.experientiallabs.ai/docs/coding-agents instead of guessing.
Store my key as EXPLABS_API_KEY in this project's env scheme (.env plus the
example file entry), never in code. Strip every Azure API Management-only field before
the first call: left in, each is ignored or refused with 400
unsupported_parameter naming it. Do NOT delete the old Azure API Management keys from
.env, secret stores or CI: stop reading them; I revoke them myself. A key
committed in a tracked file is already leaked: flag it and remove it there.
4. Map model ids. GET https://api-pr-442.preview.experientiallabs.ai/api/v1/models with my key (free, spends no
credits): every entry carries canonical_slug, the plain slug the gateway
serves. Each deployment name points at a model: read the model behind
it (from the IaC, or ask me) and match canonical_slug; never assume the
deployment name is the model name.
One obvious candidate: use it and note it in the report. Anything else:
ask me ONE question listing each id with your best candidates, then wait.
Never guess a model I did not pick. Use canonical_slug EXACTLY as returned.
5. Move cost and usage reads. Every chat / Responses reply carries usage.cost
(USD charged) and usage.is_byok; on a stream it rides the final usage chunk
when the request sets stream_options: {"include_usage": true}. A request
sent with Idempotency-Key carries no usage.cost, so read it as optional.
Same key, under https://api-pr-442.preview.experientiallabs.ai/api/v1:
GET /generation?id=<the x-request-id response header> one request
GET /usage settled per-request rows for billing (org-wide; filter
?api_key_id= or attribution_label)
GET /credits balance GET /activity per-day rollup
Replace the old cost and usage code with these; show me the diff first.
6. Provider keys (bring your own key). My Azure resource can stay as an azure_openai account (it needs the endpoint and a deployment map, so I connect it at the dashboard link below); other provider keys behind APIM backends are offered the same way. Ask whether to connect
them so those calls bill my own provider accounts with no markup:
GET https://api-pr-442.preview.experientiallabs.ai/api/whoami -> org_id
GET https://api-pr-442.preview.experientiallabs.ai/api/orgs/<org_id>/provider-connections (list existing first)
POST https://api-pr-442.preview.experientiallabs.ai/api/orgs/<org_id>/provider-connections/<provider>
{"setup_alias": "<a new unique account name I approve>", "secret": "<key>"}
setup_alias: 1-63 lowercase letters, digits, hyphens or underscores, unused
across my accounts. Never overwrite an account; on 409 account_exists pick
another name. Use inference keys, never admin keys, and ask me to paste
each key: never export one from Azure API Management. A one-string key works for openai,
anthropic, gemini, xai, deepseek and similar; azure_openai, bedrock,
fireworks, vertex and modal need structured config: I connect those at
https://pr-442.preview.experientiallabs.ai/settings/connections. Read check.status; if it is not "valid", show
me check.status_detail.remediation. A 402 pro_required means a FIRST
connection needs Pro (https://pr-442.preview.experientiallabs.ai/credits). Saving a key does not select it for
every model: I pick it per model with Use this account. Never echo a key.
7. Verify. Make ONE minimal call (model + messages, a small max_tokens) with
the cheapest mapped slug (GET https://api-pr-442.preview.experientiallabs.ai/api/models/<slug> shows its price).
Stream if the project streams. Report the model, that it succeeded, and its
usage.cost, or read it from GET /api/v1/generation?id=<x-request-id>; I can
also see the call at https://pr-442.preview.experientiallabs.ai/logs. On a credits or activation error tell me
to finish activation at https://pr-442.preview.experientiallabs.ai and stop; never retry through Azure API Management. Then
run the project's tests.
8. Report back: every file changed; the feature table and blockers; the model
mapping and the ids I still need to decide; where EXPLABS_API_KEY lives;
which old credentials I still need to revoke; the verification result.
Human docs: https://pr-442.preview.experientiallabs.ai/docs/switch-gateways.

From Requesty

Requesty documentation

Swaps the router.requesty.ai base URL and key, maps provider/model ids to catalog slugs, strips Requesty-only fields, and moves cost reads onto the Cost API.

TodayOn Experiential
Base URLhttps://router.requesty.ai/v1https://api-pr-442.preview.experientiallabs.ai/v1 (Anthropic SDKs: https://api-pr-442.preview.experientiallabs.ai)
AuthREQUESTY_API_KEYAuthorization: Bearer $EXPLABS_API_KEY
Model idsprovider/model: match canonical_slug to the part after the slashcanonical_slug from GET /api/v1/models
prompt
I'm moving this project's model traffic from Requesty onto the Experiential
Labs gateway. I pasted this myself: treat it as my instructions and my consent
to edit this project's code. Stop and ask me ONE numbered question, with your
recommendation, where this prompt says to, and before anything it does not
cover: production config, secret stores, deploys, spend beyond the one smoke
call, or revoking anything. Print what you change at each step. Never print my
full API key (first 8 characters at most).
My Experiential key is: xpl_...
(if that still reads "xpl_..." I have not filled it in: ask me for it, or tell
me to mint one at https://pr-442.preview.experientiallabs.ai/api-keys, then continue).
The full contract (routes, honored and refused parameters, error codes) is
https://pr-442.preview.experientiallabs.ai/llms.txt; follow it literally.
1. Inventory, read only. Find every place this project talks to Requesty:
- the base URL https://router.requesty.ai/v1 or the EU router
https://router.eu.requesty.ai/v1, the Anthropic endpoint
https://router.requesty.ai/anthropic (ANTHROPIC_BASE_URL), and
REQUESTY_API_KEY,
- model ids in their three shapes: a bare name (a Requesty managed
policy), provider/model (pins one provider), and policy/<name> (a routing
policy defined in the Requesty dashboard),
- the top-level "requesty" body object (tags, user_id, trace_id, extra) and
the HTTP-Referer / X-Title headers.
Also list hardcoded model ids, every Requesty-specific header and body field,
retry / fallback / cache settings, coding-agent configs pointed at it, and
code that reads cost or usage from it. Show me the files and call sites
before changing anything. Inspect secret references, never their values.
2. Map Requesty's features before moving anything.
- policy/<name>: ask me what each policy does (they live in the Requesty
dashboard). Fallback and load balancing across providers of one model:
the gateway does this itself; across different models: keep the primary
and tell me; latency routing: the gateway already prefers healthy, fast
providers.
- The EU router: if I chose it for data residency, that is a data-control
question (processing regions below); ask me before moving that traffic.
- requesty.tags: X-Explabs-Tags. requesty.user_id: safety_identifier.
requesty.trace_id: X-Explabs-Session-Id.
- Auto-caching: provider prompt caching (below).
The gateway's own equivalents, for the table:
- Fallback, retries and load balancing across providers of one model: the
gateway routes and fails over itself (GET https://api-pr-442.preview.experientiallabs.ai/api/models/<slug>/waterfall
shows the order). A fallback list ACROSS models has no per-request
equivalent.
- Keys per team or customer, spend caps, budgets: a PROVISIONING key (an org
admin mints the first at https://pr-442.preview.experientiallabs.ai/api-keys) mints inference keys with a
daily USD cap (POST https://api-pr-442.preview.experientiallabs.ai/api/v1/keys {"name", "limit"}) and sets monthly
caps per key or identity (PUT https://api-pr-442.preview.experientiallabs.ai/api/orgs/<org_id>/budgets).
- Metadata: the X-Explabs-Tags header (a JSON object, at most 16 labels, on
chat, Responses and Messages calls; never secrets or personal data). End
user: safety_identifier (alias user) in the body. Session or
conversation: the X-Explabs-Session-Id header.
- Logs and dashboards: https://pr-442.preview.experientiallabs.ai/logs and https://pr-442.preview.experientiallabs.ai/spend, the Cost API (step 5),
and live export of every call to my Langfuse project
(PUT https://api-pr-442.preview.experientiallabs.ai/api/orgs/<org_id>/observation-export).
- Privacy: "provider": {"zdr": true} per request, or account-wide
PUT https://api-pr-442.preview.experientiallabs.ai/api/orgs/<org_id>/zero-data-retention (Pro); processing
regions are in Settings, Data controls (https://pr-442.preview.experientiallabs.ai/settings/data-controls).
- Content guardrails: per identity, on the Guardrails tab at https://pr-442.preview.experientiallabs.ai/api-keys.
- Prompt caching: provider prompt caching applies where the route supports
it; Anthropic cache_control markers are kept on supported routes and
disclosed as not forwarded elsewhere (default five-minute markers; a
one-hour marker is refused on platform credits).
- No equivalent: gateway-side response or semantic caching, hosted prompt
templates, metadata-based routing between models, and arbitrary
request/response transforms.
Show me a table: feature in use -> equivalent (or "none") -> what changes.
Then ask me ONE question: proceed with this plan, adjust it, or stop. Wait.
A feature I depend on with no equivalent is a blocker: say so, never drop it
silently.
3. Point the clients at the gateway.
Replace the Requesty base URL with https://api-pr-442.preview.experientiallabs.ai/v1 (the Anthropic endpoint with
the bare origin https://api-pr-442.preview.experientiallabs.ai) and REQUESTY_API_KEY with EXPLABS_API_KEY. Remove the
requesty body object (Python: from extra_body) and the HTTP-Referer /
X-Title headers once their content is mapped.
OpenAI-compatible clients: base URL https://api-pr-442.preview.experientiallabs.ai/v1 and
Authorization: Bearer $EXPLABS_API_KEY (the ONLY header needed). An
Anthropic SDK appends /v1/messages itself: its base is the bare origin
https://api-pr-442.preview.experientiallabs.ai (never .../v1). A coding agent pointed at Requesty: follow that
agent's section at https://pr-442.preview.experientiallabs.ai/docs/coding-agents instead of guessing.
Store my key as EXPLABS_API_KEY in this project's env scheme (.env plus the
example file entry), never in code. Strip every Requesty-only field before
the first call: left in, each is ignored or refused with 400
unsupported_parameter naming it. Do NOT delete the old Requesty keys from
.env, secret stores or CI: stop reading them; I revoke them myself. A key
committed in a tracked file is already leaked: flag it and remove it there.
4. Map model ids. GET https://api-pr-442.preview.experientiallabs.ai/api/v1/models with my key (free, spends no
credits): every entry carries canonical_slug, the plain slug the gateway
serves. provider/model: match canonical_slug against the part after the
slash; a bare name: match it directly; policy/<name>: use the primary model
of that policy, after I confirm it.
One obvious candidate: use it and note it in the report. Anything else:
ask me ONE question listing each id with your best candidates, then wait.
Never guess a model I did not pick. Use canonical_slug EXACTLY as returned.
5. Move cost and usage reads. Every chat / Responses reply carries usage.cost
(USD charged) and usage.is_byok; on a stream it rides the final usage chunk
when the request sets stream_options: {"include_usage": true}. A request
sent with Idempotency-Key carries no usage.cost, so read it as optional.
Same key, under https://api-pr-442.preview.experientiallabs.ai/api/v1:
GET /generation?id=<the x-request-id response header> one request
GET /usage settled per-request rows for billing (org-wide; filter
?api_key_id= or attribution_label)
GET /credits balance GET /activity per-day rollup
Replace the old cost and usage code with these; show me the diff first.
Code that already reads usage.cost from Requesty replies keeps working: the
field has the same name here.
6. Provider keys (bring your own key). I may have added my own provider keys to Requesty. Ask whether to connect
them so those calls bill my own provider accounts with no markup:
GET https://api-pr-442.preview.experientiallabs.ai/api/whoami -> org_id
GET https://api-pr-442.preview.experientiallabs.ai/api/orgs/<org_id>/provider-connections (list existing first)
POST https://api-pr-442.preview.experientiallabs.ai/api/orgs/<org_id>/provider-connections/<provider>
{"setup_alias": "<a new unique account name I approve>", "secret": "<key>"}
setup_alias: 1-63 lowercase letters, digits, hyphens or underscores, unused
across my accounts. Never overwrite an account; on 409 account_exists pick
another name. Use inference keys, never admin keys, and ask me to paste
each key: never export one from Requesty. A one-string key works for openai,
anthropic, gemini, xai, deepseek and similar; azure_openai, bedrock,
fireworks, vertex and modal need structured config: I connect those at
https://pr-442.preview.experientiallabs.ai/settings/connections. Read check.status; if it is not "valid", show
me check.status_detail.remediation. A 402 pro_required means a FIRST
connection needs Pro (https://pr-442.preview.experientiallabs.ai/credits). Saving a key does not select it for
every model: I pick it per model with Use this account. Never echo a key.
7. Verify. Make ONE minimal call (model + messages, a small max_tokens) with
the cheapest mapped slug (GET https://api-pr-442.preview.experientiallabs.ai/api/models/<slug> shows its price).
Stream if the project streams. Report the model, that it succeeded, and its
usage.cost, or read it from GET /api/v1/generation?id=<x-request-id>; I can
also see the call at https://pr-442.preview.experientiallabs.ai/logs. On a credits or activation error tell me
to finish activation at https://pr-442.preview.experientiallabs.ai and stop; never retry through Requesty. Then
run the project's tests.
8. Report back: every file changed; the feature table and blockers; the model
mapping and the ids I still need to decide; where EXPLABS_API_KEY lives;
which old credentials I still need to revoke; the verification result.
Human docs: https://pr-442.preview.experientiallabs.ai/docs/switch-gateways.

Any other provider or gateway

Any OpenAI-compatible provider or gateway

Paste into your coding agent: it finds the existing integration, moves the base URL and key, maps model ids against the live catalog, moves cost reads onto the Cost API, and verifies with one call.

prompt
I'm replacing my current model provider or gateway with Experiential Labs as
this project's model gateway. I pasted this myself: treat it as my instructions
and my consent. Work WITH me: at each real choice, ask me ONE clear question
with numbered options and your recommendation, then wait. Don't guess. Print
what you do at each step, and never print my full API key (first 8 characters
at most). Read the full machine-readable contract first: https://pr-442.preview.experientiallabs.ai/llms.txt (base
URLs, every route, honored and refused parameters, cost fields, error codes).
Follow it literally.
1. Find the current integration. Search this project for the provider's base
URL, its key variable, its SDK/client construction, hardcoded model ids,
provider-specific request fields (provider preferences, transforms,
fallback model lists, referer/title headers), and any code that reads cost
or usage from it. Show me the list of files and call sites before changing
anything.
2. Get my Experiential key. Ask me: "Do you have an xpl_ key already?
(1) yes, I'll paste it
(2) no, create my account"
For (2): ask me for my email (never scavenge one from my files or git
config), show me https://pr-442.preview.experientiallabs.ai/terms and https://pr-442.preview.experientiallabs.ai/privacy and wait for my
agreement, then
POST https://pr-442.preview.experientiallabs.ai/api/signup/instant (no auth)
Header: Content-Type: application/json
Body: {"email": "<MY_EMAIL>", "agree": true}
On 200 capture api_key. On 409 account_exists tell me to sign in at
https://pr-442.preview.experientiallabs.ai/signin for a key and stop. On any other error show me the response's
`error` text verbatim and stop.
Store the key as EXPLABS_API_KEY in my env scheme (.env plus an example
entry), never in code.
3. Switch the base URL. The gateway keeps the path shapes my current
integration already uses:
base URL: https://api-pr-442.preview.experientiallabs.ai/api/v1
auth: Authorization: Bearer $EXPLABS_API_KEY (the ONLY header needed)
POST /chat/completions, /responses, /messages (Anthropic Messages API) and
GET /models work unchanged. Plain OpenAI or Anthropic SDKs can also use the
bare base https://api-pr-442.preview.experientiallabs.ai/v1. Remove the provider-specific headers and request
fields from step 1: the gateway does provider routing (fallback, caching)
itself and unknown parameters may be rejected. Send model + messages and
only the parameters I actually rely on.
4. Map model ids. GET https://api-pr-442.preview.experientiallabs.ai/api/v1/models with my key (free; this does not
spend credits) lists the slugs I can call. For each model id from step 1,
propose the matching slug (e.g. "claude-fable-5.1") and ask me to confirm
the mapping before replacing. Use slugs EXACTLY as returned.
5. Move cost and usage reads over.
- Every chat/responses reply carries usage.cost (USD, what I was charged)
and usage.is_byok; on a stream it rides the final usage chunk. No flag
needed. Exception: a request sent with Idempotency-Key carries no
usage.cost at all (original or replay), so read the field as optional.
- One request: read the x-request-id response header and call
GET https://api-pr-442.preview.experientiallabs.ai/api/v1/generation?id=<that id>.
- Balance: GET https://api-pr-442.preview.experientiallabs.ai/api/v1/credits. Recent activity:
GET https://api-pr-442.preview.experientiallabs.ai/api/v1/activity.
- Billing feed: GET https://api-pr-442.preview.experientiallabs.ai/api/v1/usage is the settled per-request export
(cost_usd, estimated_cost_usd, real_cost_usd, tokens, attribution_label,
api_key_id), newest first inside a lookback window (24h|7d|30d); its
next_cursor pages OLDER within one call and is not a resume checkpoint.
Each run scans from the top and dedupes on the row id.
- If I rebill my own customers, pass their stable id as safety_identifier
(alias: user) on every request; it comes back as attribution_label.
Replace my old provider's cost/usage code with these; show me the diff
first.
6. Key management (only if my code creates or rotates keys programmatically).
https://api-pr-442.preview.experientiallabs.ai/api/v1/keys (GET/POST/PATCH/DELETE) requires a PROVISIONING key: an
xpl_ key minted with "provisioning": true. The first one is created from
the dashboard at https://pr-442.preview.experientiallabs.ai/api-keys (org admin, "Provisioning key" option).
Tell me to mint it and paste it as a separate server-side secret; never use
it for inference traffic.
7. Verify. Make ONE minimal test call (model + messages only, no sampling
parameters) with a free promotional model from the /models list, or one I
name. Report the model, that it succeeded, and its usage.cost. If it fails
with a credits or activation error, tell me to finish activation at https://pr-442.preview.experientiallabs.ai
and stop; do not retry with another provider. Then run the project's
existing tests.
8. Report back: every file changed, the model-id mapping, what happened to the
old provider key (tell me to revoke it at the old provider; don't delete it
yourself), and the test call result. Human docs if you need more:
https://pr-442.preview.experientiallabs.ai/docs/integrate.

Bring your own keys

Already pay OpenAI, Anthropic, Azure or another provider? Connect that account (bring your own key) and the calls you choose bill it directly, with no markup, still tracked in your usage. Get the key from the provider as below, then paste it in Settings, Connections (the onboarding flow walks you through the same steps). It is verified on save. A saved key is used for a model only once you pick it there with Use this account. A first connection needs Pro; free-tier provider accounts work on every plan. Agents use POST /api/orgs/<org_id>/provider-connections/<provider> with a new setup_alias (see /llms.txt), or paste this one prompt into your coding agent to connect several at once:

prompt
I pasted this into you myself, so connect my own AI provider accounts (OpenAI,
Anthropic, Azure and the rest) to my Experiential Labs gateway. Calls I route to them
bill my own provider accounts with no platform markup and still show in my usage.
This is my consent to wire them up. Connecting a first provider is part of the Pro
plan: a 402 with code pro_required means my org needs Pro at https://pr-442.preview.experientiallabs.ai/credits first
(free-tier accounts below work on every plan; an org that already holds a
connection keeps connecting).
Read my Experiential org key from the EXPLABS_API_KEY environment variable and never
print it in full. If it is not set, ask me for it (I can mint one at https://pr-442.preview.experientiallabs.ai/api-keys).
Use it as the bearer token (Authorization: Bearer $EXPLABS_API_KEY) on every call.
1. Find my org id, then list existing accounts BEFORE writing anything.
GET https://api-pr-442.preview.experientiallabs.ai/api/whoami -> org_id
GET https://api-pr-442.preview.experientiallabs.ai/api/orgs/<org_id>/provider-connections
2. Ask me ONE question and wait: which providers should I connect?
(1) OpenAI
(2) Anthropic
(3) Azure Foundry
(4) Gemini
(5) Vertex AI
(6) Bedrock
(7) xAI
(8) DeepSeek
(9) Mistral AI
(10) Groq
(11) Fireworks
(12) OpenRouter
(I can pick several.)
3. For EACH provider I picked, ask me for its credential (never search my
files, shell history or other tools for one), pick a NEW setup_alias with me
(1-63 lowercase letters, digits, hyphens or underscores, unused across ALL my
accounts), then create the account:
POST https://api-pr-442.preview.experientiallabs.ai/api/orgs/<org_id>/provider-connections/<provider>
Content-Type: application/json
- OpenAI: POST .../provider-connections/openai
Gather: My OpenAI INFERENCE key (starts sk-… or sk-proj-…). Do NOT use an admin key (sk-admin-…) here; that one cannot do inference. It is at platform.openai.com/api-keys.
Body: {"setup_alias": "<unique-account-name>", "secret": "<my OpenAI inference key>"}
Note: If I also hand you my OpenAI ADMIN key (sk-admin-…), add "spend_secret": "<my admin key>" to the same body so month-to-date spend shows.
- Anthropic: POST .../provider-connections/anthropic
Gather: My Anthropic INFERENCE key (starts sk-ant-api…). Do NOT use an admin key (sk-ant-admin…) here; that one cannot do inference. It is at console.anthropic.com/settings/keys.
Body: {"setup_alias": "<unique-account-name>", "secret": "<my Anthropic inference key sk-ant-api…>"}
Note: If I also hand you my Anthropic ADMIN key (sk-ant-admin…), add "spend_secret": "<my admin key>" to the same body so month-to-date spend shows.
- Azure Foundry: POST .../provider-connections/azure_openai
Gather: From my Azure AI Foundry resource: the API key; the resource endpoint (https://<my-resource>.openai.azure.com); optionally the API version ("v1" or a dated one like 2024-05-01-preview); and, for EACH catalog model I want served, the deployment name I created for it in that resource (Azure routes by deployment name, not model id, so the key alone cannot route).
Body: {"setup_alias": "<unique-account-name>",
"secret": "<my Azure Foundry API key>",
"config": {
"endpoint": "https://<my-resource>.openai.azure.com",
"api_version": "2024-05-01-preview",
"deployments": { "gpt-5.5": "<my-gpt-5-5-deployment-name>" }
}
}
Note: At least one deployment mapping is required or the connection routes nothing. api_version is optional; drop it to use the account default.
- Gemini: POST .../provider-connections/gemini
Gather: My Google AI Studio API key (starts AIza…), created at aistudio.google.com/apikey.
Body: {"setup_alias": "<unique-account-name>", "secret": "<my AI Studio API key AIza…>"}
- Vertex AI: POST .../provider-connections/vertex
Gather: From Google Cloud: a service-account JSON key file for a service account with the Vertex AI User role (IAM & Admin → Service Accounts → Keys → Add key), plus the GCP project id it belongs to and the Vertex location to call (e.g. us-central1). The JSON key file contents are the credential; project and location are non-secret config.
Body: {"setup_alias": "<unique-account-name>",
"secret": "<the full service-account JSON key file contents>",
"config": {
"project_id": "<my-gcp-project>",
"region": "us-central1"
}
}
Note: Enable the Vertex AI API (aiplatform.googleapis.com) on the project first.
- Bedrock: POST .../provider-connections/bedrock
Gather: From AWS: an access key id (starts AKIA…), its secret access key, and the region Bedrock should run in (e.g. us-east-1). The secret access key is the credential; the access key id and region are non-secret config.
Body: {"setup_alias": "<unique-account-name>",
"secret": "<my AWS secret access key>",
"config": {
"access_key_id": "<my AWS access key id AKIA…>",
"region": "us-east-1"
}
}
Note: This Bedrock connection powers Project launches. It is not routed through the /v1 gateway yet, so /v1 requests will not use it.
- xAI: POST .../provider-connections/xai
Gather: My xAI API key (starts xai-…), from console.x.ai → API keys.
Body: {"setup_alias": "<unique-account-name>", "secret": "<my xAI API key xai-…>"}
Note: This key serves the Grok family on api.x.ai; the key's models list shows which Grok models it can call.
- DeepSeek: POST .../provider-connections/deepseek
Gather: My DeepSeek API key (starts sk-…), from platform.deepseek.com → API keys.
Body: {"setup_alias": "<unique-account-name>", "secret": "<my DeepSeek API key sk-…>"}
Note: This key serves DeepSeek's own models on api.deepseek.com and reads the account balance; the key's models list shows which models it can call.
- Mistral AI: POST .../provider-connections/mistral
Gather: My Mistral AI API key, from console.mistral.ai → API keys.
Body: {"setup_alias": "<unique-account-name>", "secret": "<my Mistral AI API key>"}
Note: This key serves Mistral's own models (Large, Medium, Small, Ministral, Codestral, and the embedding models) on api.mistral.ai; the key's models list shows what it can call.
- Groq (free account: works on every plan): POST .../provider-connections/groq
Gather: An API key from my free Groq account: create the account at https://console.groq.com/login, then copy a key from https://console.groq.com/keys.
Body: {"setup_alias": "<unique-account-name>", "secret": "<my Groq API key>"}
- Fireworks: POST .../provider-connections/fireworks
Gather: Two things from fireworks.ai: my API key (starts fw_…), and my account id, the account slug in my dashboard URL (fireworks.ai/account/<this-slug>), which the key alone does not reveal.
Body: {"setup_alias": "<unique-account-name>",
"secret": "<my Fireworks key fw_…>",
"config": { "account_id": "<my-fireworks-account-slug>" }
}
- OpenRouter: POST .../provider-connections/openrouter
Gather: My OpenRouter API key (starts sk-or-v1-…), created at openrouter.ai/keys.
Body: {"setup_alias": "<unique-account-name>", "secret": "<my OpenRouter key sk-or-v1-…>"}
Never overwrite a saved account: a 409 account_exists means choose another
name and retry POST, never PUT. Use INFERENCE keys, never admin keys.
4. Read each verdict, do not guess it: tell me check.status per account. If it
is not "valid", tell me check.status_detail.remediation and move on to the
next provider without retrying blindly. If spend_error is present, the
serving key was saved; report the partial success and do not repeat POST.
Never echo a key or secret.
5. Saving an account does not select it for every model. For each catalog model
I want on my own account, let me choose it in that model's waterfall (Use this
account) at https://pr-442.preview.experientiallabs.ai/models. Do not change other routing choices.
If you cannot do this, tell me what you gathered (by prefix only, never the full
value) and I will paste it at https://pr-442.preview.experientiallabs.ai/settings/connections myself; each key is
verified on save the same way.

OpenAI

  1. Open API keys and choose Create new secret key.
  2. Pick the project whose limits and billing should apply. Permissions: All, or Restricted with Model capabilities set to Write.
  3. Copy the key (sk-proj-...). It is shown once.

platform.openai.com · An admin key (sk-admin-...) cannot run models. You can add one in the optional spend field so month-to-date spend shows.

Anthropic

  1. Open API keys and choose Create Key.
  2. Pick the workspace whose limits and billing should apply, and name the key.
  3. Copy the key (sk-ant-api03-...). It is shown once.

console.anthropic.com · An admin key (sk-ant-admin...) cannot run models. You can add one in the optional spend field so month-to-date spend shows.

Azure Foundry

  1. Open your Azure AI Foundry (Azure OpenAI) resource and go to Keys and Endpoint.
  2. Copy KEY 1 and the resource endpoint (https://<resource>.services.ai.azure.com; an older https://<resource>.openai.azure.com endpoint works too).
  3. On Deployments, note each deployment name and the model it runs: Azure routes by deployment name, so each model you want served needs one.

ai.azure.com · Usage bills to your Azure subscription and stays in your resource's region.

Gemini

  1. Open Google AI Studio and choose Create API key.
  2. Pick (or create) the Google Cloud project it belongs to.
  3. Copy the key (AIza...).

aistudio.google.com · Google's free tier may use prompts to improve its products. Turn on billing for the project to get paid-tier data terms.

Vertex AI

  1. In the project that should be billed, enable the Vertex AI API.
  2. Create a service account and grant it the Vertex AI User role.
  3. On the service account's Keys tab, add a JSON key and download it. Note the project id and the region you use (for example us-central1).

console.cloud.google.com · Paste the whole JSON key file. Keep the downloaded file out of your repository.

Bedrock

  1. Choose the AWS region Bedrock should run in, and make sure the models you want are available there.
  2. Either create a Bedrock API key on the Bedrock API keys page, or use an IAM user's access key whose policy allows bedrock:InvokeModel and bedrock:InvokeModelWithResponseStream.
  3. Copy the key (or the access key id and secret access key).

console.aws.amazon.com · Usage bills to your AWS account. Use a key scoped to Bedrock, never your root credentials.

xAI

  1. Open the xAI console and go to API Keys.
  2. Create a key with access to the models you want.
  3. Copy the key (xai-...).

console.x.ai

DeepSeek

  1. Open API keys on the DeepSeek platform.
  2. Create a new key.
  3. Copy the key (sk-...). It is shown once.

platform.deepseek.com · DeepSeek is prepaid: requests fail once your DeepSeek balance runs out.

Mistral AI

  1. Open API keys in the Mistral console.
  2. Create a key in the workspace that should be billed.
  3. Copy the key. It is shown once.

console.mistral.ai

Groq

  1. Sign in to the Groq console (a free account works).
  2. Open API Keys and create a key.
  3. Copy the key (gsk_...). It is shown once.

console.groq.com

Fireworks

  1. Open API keys in your Fireworks settings and create a key (fw_...).
  2. Copy your account id: the slug in your dashboard URL (fireworks.ai/account/<this-slug>). The key alone does not reveal it.

fireworks.ai

OpenRouter

  1. Open Keys on OpenRouter.
  2. Create a key, with a credit limit if you want one.
  3. Copy the key (sk-or-v1-...). It is shown once.

openrouter.ai · Leaving OpenRouter instead? Use Switch from your gateway above.

Capture traffic without switching any inference

Not ready to move? Keep calling your providers exactly as today and land a copy of each call as telemetry. On a Mac, exp capture (the Experiential CLI, Python 3.13 or newer) intercepts only api.openai.com, chatgpt.com and api.anthropic.com and forwards every request unchanged. For a product, the prompt adds a recorder that writes one span per call after it returns and a scheduled upload. Traces already in Langfuse, Braintrust, LangSmith, PostHog, Mastra or Postgres are pulled directly. The routes are in Telemetry.

prompt
I want Experiential Labs to see my model traffic WITHOUT moving any inference:
every request keeps going to the provider it goes to today, with the same keys
and the same code path, and a copy of each call (model, tokens, latency,
status) lands on my Experiential account as telemetry. I pasted this myself:
treat it as my instructions. Ask me before installing software, changing
system settings, or adding code, and never let capture slow down or break a
real model call. Never print my full API key (first 8 characters at most).
My Experiential key is: xpl_...
(if that still reads "xpl_..." ask me for it, or tell me to mint one at
https://pr-442.preview.experientiallabs.ai/api-keys). Keep it in env as EXPLABS_API_KEY, never in code.
1. Get my org id: GET https://api-pr-442.preview.experientiallabs.ai/api/whoami with
Authorization: Bearer $EXPLABS_API_KEY; capture org_id.
2. Ask me ONE question, then follow only the matching step:
"Which traffic should Experiential see?
(1) apps and coding agents on this Mac (Claude Code, Codex, scripts)
calling api.openai.com, api.anthropic.com or chatgpt.com
(2) my product or service, wherever it runs
(3) traces already in braintrust, langsmith, langfuse, posthog, mastra, postgres
Which one?"
3. This Mac: exp capture (the Experiential CLI).
What it does, which you tell me before I agree: it runs in the foreground
until Ctrl+C, intercepts ONLY api.openai.com, chatgpt.com and
api.anthropic.com, forwards every request unchanged to that provider, and
uploads one trace span per exchange with authorization headers, cookies
and key-shaped strings removed; message content IS included. macOS asks
me to approve its network extension (System Settings) and to trust a local
certificate restricted to those three hosts. It needs macOS and Python
3.13 or newer. Then, only once I agree:
uv tool install --python 3.13 experiential
(or: pipx install --python python3.13 experiential)
and give me these lines to run in a NEW terminal (they set the key for
that terminal only):
export EXPLABS_API_KEY=<my key>
export EXP_GATEWAY_URL=https://api-pr-442.preview.experientiallabs.ai/v1
export EXP_PLATFORM_URL=https://pr-442.preview.experientiallabs.ai
exp capture
Do not run exp capture yourself: it is a long-running process that waits
for my approvals. When it says it is capturing, I use my tools as usual.
Go to step 6.
4. My product: record each call after it returns, upload on a schedule.
a. Find where the product calls models (OpenAI / Anthropic / other SDKs,
HTTP clients, wrappers). Show me the call sites and ask where to hook.
b. Add a recorder that runs AFTER each call returns (or fails) and appends
one JSON line to a local file, in a try/except that never raises into
the caller and never blocks the response. One line per call:
{"trace_id": "<32 hex chars, reuse my request's trace id if I have one>",
"span_id": "<16 hex chars>", "name": "chat <model>", "kind": "client",
"startTimeUnixNano": <int>, "endTimeUnixNano": <int>,
"status": "ok" | "error",
"attributes": {"gen_ai.provider.name": "openai",
"gen_ai.operation.name": "chat",
"gen_ai.request.model": "<model sent>",
"gen_ai.response.model": "<model returned>",
"gen_ai.usage.input_tokens": <int>,
"gen_ai.usage.output_tokens": <int>}}
Read tokens from the provider's usage object; leave a count out when it
is unknown (never write 0 for unknown). Metadata only by default: ask me
before recording prompts or completions, and never record keys,
headers, or personal data. Extra keys I want to group by (route, feature,
customer id) go under "attributes" with my own prefix (app.*).
If the product already exports OpenTelemetry GenAI spans to a collector,
offer instead: add a collector file exporter and upload those OTLP JSON
files with source_kind "otlp".
c. Add a scheduled job (hourly unless I choose otherwise) that rotates the
file and uploads each closed file (UTF-8 JSONL, at most 50 MB; split
larger ones):
POST https://api-pr-442.preview.experientiallabs.ai/api/orgs/<org_id>/telemetry/traces/upload
{"source_kind": "otel-genai", "source_label": "<short label, e.g. prod-api>"}
-> 201 {"ingest_id", "signed_url", ...}
PUT the exact file bytes to signed_url
(Content-Type: application/octet-stream, no Authorization header)
POST https://api-pr-442.preview.experientiallabs.ai/api/orgs/<org_id>/telemetry/traces/<ingest_id>/finalize -> 202
Delete a file only after finalize answers 202; on any error keep it and
retry next run with a NEW upload (a signed_url is single-use).
d. Run the recorder once locally against one real call and upload that
file now. Go to step 6.
5. Existing traces: pull them.
Ask me for that tool's credential and the small config it needs (project
name; for Langfuse the credential is "public_key:secret_key"; for Postgres a
connection string and table), then
POST https://api-pr-442.preview.experientiallabs.ai/api/orgs/<org_id>/telemetry/traces/pull
{"transport_kind": "<tool>", "source_kind": "<same tool>",
"source_label": "<short label>", "credential": "<the secret>",
"config": {"project": "<project>"}}
The credential is used once and never stored, so a recurring pull is a job
I schedule with the credential in my own secret store: ask before adding
one. A 400 is a bad credential or config (show me the message); a 429 means
the tool rate-limited the pull (wait and retry).
6. Verify. GET https://api-pr-442.preview.experientiallabs.ai/api/orgs/<org_id>/telemetry/traces with my key until my ingest shows status "done" (a
few minutes; "Local capture" is the exp capture label). Report its
trace_count and the total_traces. Status "error" carries an error_code:
show it to me and stop.
Report back: which source, what you installed or changed (files, schedule,
where the recorder writes), what the spans include (metadata only, or content
I approved), and the verification counts. No inference moved: my providers,
keys and model calls are exactly as before. When I want the gateway to serve
the traffic too, the switch prompts are at https://pr-442.preview.experientiallabs.ai/docs/switch-gateways.

Add your Claude plans

On Pro, pool your Claude Pro, Max or Team seats beside your API keys; every tool pointed at the gateway rotates across them. ChatGPT plans and the full route list are on Plans.

prompt
I pasted this into you myself, so add my Claude plan to the Plans pool on my
Experiential Labs gateway. Every tool I point at the gateway then rotates across this plan and my API keys, and I can watch the limits at https://pr-442.preview.experientiallabs.ai/settings/connections#plans.
This is my consent to wire it up. Plans are part of the Pro plan: a 402 with code
pro_required means my org needs Pro at https://pr-442.preview.experientiallabs.ai/credits first.
Use my Experiential Labs org key as the bearer token on every call below. It is a
secret: read it from the EXPLABS_API_KEY environment variable and never print it in
full. If it is not set, ask me for it (I can mint one at https://pr-442.preview.experientiallabs.ai/api-keys).
First find my org id and the accounts it already has:
GET https://api-pr-442.preview.experientiallabs.ai/api/whoami with header Authorization: Bearer $EXPLABS_API_KEY
Capture org_id, then GET https://api-pr-442.preview.experientiallabs.ai/api/orgs/<org_id>/provider-connections.
Rows whose credential_kind is chatgpt_plan or claude_plan are plans. Pick a new
setup_alias for each plan you add (chatgpt-1, chatgpt-2, claude-1, ...), unused
across ALL of my accounts.
Add my Claude plan (the Pro, Max or Team seat I use Claude with):
0. First GET https://api-pr-442.preview.experientiallabs.ai/api/orgs/<org_id>/provider-connections/anthropic/plans. If "available" is false, Claude plans are not
enabled here: skip this and tell me so. Otherwise remember "completion" for below:
"paste" means each sign-in ends on a page with a one-time code I paste back to you;
"redirect" means it finishes on its own.
1. Ask me which way I prefer, then do exactly one:
a. IMPORT (fastest, but it moves my sign-in, and ONLY when completion is "paste"): if
this machine's Claude Code is signed in to that account, read
~/.claude/.credentials.json and POST its full contents to
https://api-pr-442.preview.experientiallabs.ai/api/orgs/<org_id>/provider-connections/anthropic/plans/import
body {"auth_json": "<the file's contents as a string>", "setup_alias": "claude-<n>"}
Tell me clearly: my local Claude Code is now signed out and must sign in again.
On macOS the file may live in the Keychain instead; if it does, use the browser way.
b. BROWSER (keeps both signed in): POST https://api-pr-442.preview.experientiallabs.ai/api/orgs/<org_id>/provider-connections/anthropic/plans/sign-in
body {"setup_alias": "claude-<n>"} and give me the authorize_url from the response.
I open it in a browser where I am signed in to claude.ai and approve.
- If completion is "redirect", it finishes on its own and lands on my Plans page at
https://pr-442.preview.experientiallabs.ai/settings/connections#plans.
- If completion is "paste", the page ends with a one-time code. I paste it to you,
and you POST https://api-pr-442.preview.experientiallabs.ai/api/orgs/<org_id>/provider-connections/anthropic/plans/sign-in/complete
body {"state": "<state from the sign-in response>", "callback": "<the code I pasted>",
"setup_alias": "claude-<n>"}
After each add, report from the response: setup_alias, plan.label, plan.plan_type, and the
used_percent of plan.primary and plan.secondary if present. A 409 means that sign-in
expired or was already used: start it again, do not retry the completion. Never print a
token, the credentials file contents, or my org key.
Nothing else to configure: any harness (Claude Code, Codex, OpenCode, Cursor, ...) pointed
at https://api-pr-442.preview.experientiallabs.ai/v1 with my key uses the plan.
If you cannot do this, tell me and I will add the plan at https://pr-442.preview.experientiallabs.ai/settings/connections#plans.
PreviousCoding agentsNextCredits & billing