Thundera Models

Overview

The gateway speaks OpenAI Chat Completions, Embeddings and Audio plus the Anthropic Messages protocol, and routes each call to Anthropic (Claude API, direct), Amazon Bedrock, OpenAI, DeepSeek, Google, Deepgram or ElevenLabs, and exposes TypeSafe's Jev decision model on /v1/systemone. Each app has its own API key with a budget, limits and an allowlist. Every response states what it cost.

Quickstart

Export your key, then send a chat call and read cost from the response. Examples use the fast alias.

curl

export MODELS_API_KEY=...
curl -s https://models.thundera.eu/v1/chat/completions \
  -H "Authorization: Bearer $MODELS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"fast","messages":[{"role":"user","content":"Hello"}]}' \
  | jq '{answer: .choices[0].message.content, cost: .cost.total_usd, left: .cost.budget.remaining_usd}'

Node.js

Use the official openai package with baseURL.

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: process.env.MODELS_BASE_URL ?? "https://models.thundera.eu/v1",
  apiKey: process.env.MODELS_API_KEY,
});

const res = await client.chat.completions.create({
  model: "fast",
  messages: [{ role: "user", content: "Hello" }],
});
console.log(res.choices[0].message.content);
console.log("cost USD", res.cost.total_usd, "remaining", res.cost.budget?.remaining_usd);

Python

import os
from openai import OpenAI

client = OpenAI(
    base_url=os.environ.get("MODELS_BASE_URL", "https://models.thundera.eu/v1"),
    api_key=os.environ["MODELS_API_KEY"],
)
res = client.chat.completions.create(
    model="fast",
    messages=[{"role": "user", "content": "Hello"}],
)
print(res.choices[0].message.content)
cost = res.model_extra["cost"]          # extra fields land in model_extra
print("cost USD", cost["total_usd"])

Go

Plain net/http; sashabaranov/go-openai works too via config.BaseURL, but its typed response drops cost.

package main

import (
	"bytes"
	"encoding/json"
	"fmt"
	"net/http"
	"os"
)

func main() {
	base := os.Getenv("MODELS_BASE_URL")
	if base == "" {
		base = "https://models.thundera.eu/v1"
	}
	body, _ := json.Marshal(map[string]any{
		"model":    "fast",
		"messages": []map[string]string{{"role": "user", "content": "Hello"}},
	})
	req, _ := http.NewRequest("POST", base+"/chat/completions", bytes.NewReader(body))
	req.Header.Set("Authorization", "Bearer "+os.Getenv("MODELS_API_KEY"))
	req.Header.Set("Content-Type", "application/json")

	resp, err := http.DefaultClient.Do(req)
	if err != nil {
		panic(err)
	}
	defer resp.Body.Close()

	var out struct {
		Choices []struct {
			Message struct{ Content string } `json:"message"`
		} `json:"choices"`
		Cost struct {
			TotalUSD float64 `json:"total_usd"`
		} `json:"cost"`
	}
	json.NewDecoder(resp.Body).Decode(&out)
	fmt.Println(out.Choices[0].Message.Content, out.Cost.TotalUSD)
}

Rust

With reqwest (features json) and serde_json.

use serde_json::{json, Value};

#[tokio::main]
async fn main() -> Result<(), reqwest::Error> {
    let base = std::env::var("MODELS_BASE_URL")
        .unwrap_or_else(|_| "https://models.thundera.eu/v1".into());
    let key = std::env::var("MODELS_API_KEY").expect("MODELS_API_KEY");

    let res: Value = reqwest::Client::new()
        .post(format!("{base}/chat/completions"))
        .bearer_auth(key)
        .json(&json!({
            "model": "fast",
            "messages": [{ "role": "user", "content": "Hello" }]
        }))
        .send().await?
        .json().await?;

    println!("{}", res["choices"][0]["message"]["content"]);
    println!("cost USD {}", res["cost"]["total_usd"]);
    Ok(())
}

Authentication

Send Authorization: Bearer <api key>. Keys are RS256 JWTs issued per app in /system and verifiable offline with /.well-known/jwks.json. Status is database-authoritative: regenerating or revoking a key takes effect on the very next call.

In-cluster, x-api-key: <key> is also accepted. CloudFront strips that header, so external callers must use Authorization.

Models & aliases

GET /v1/models is public and returns {object:"list", data:[…]} with two kinds of entries. Model ids are <provider>/<provider model id>; a bare provider id such as gpt-4.1-mini also resolves. Only enabled and priced models are listed: an unpriced model is never served.

Aliases

Aliases are admin-mapped chains. Use them instead of vendor ids. Failover happens only before the first byte is sent, only on retryable upstream failures (throttling, 5xx, timeouts, model unavailable or not entitled) and never on a 400.

All models

Prices are USD per 1M tokens unless noted. Capabilities: streaming, tools, vision, reasoning.

Providers

Prompt caching (Claude). Add cache_control: {"type": "ephemeral"} at the top level (automatic: the cache point moves with the conversation) or on a system/text/image/tool part; {"type":"ephemeral","ttl":"1h"} for one hour. Cache writes cost 1.25× input (1-hour: 2×), reads 0.05–0.1× — all reflected in cost. Other providers ignore the marker.

Streaming

Set "stream": true for Server-Sent Events. The final chunk has choices: [] and carries cost, then data: [DONE]. Keepalives are sent every 15 s.

const stream = await client.chat.completions.create({
  model: "smart",
  stream: true,
  messages: [{ role: "user", content: "Write a haiku" }],
});
for await (const chunk of stream) {
  if (chunk.choices.length) process.stdout.write(chunk.choices[0].delta.content ?? "");
  else console.log("\ncost USD", chunk.cost.total_usd);   // final chunk
}

On /v1/messages the Anthropic event protocol is used and cost arrives in the message_delta event. If a stream aborts and the provider reported no usage, cost.estimated is true.

Tools

Standard OpenAI tools and tool_choice, on every model with the tools capability. Use smart for tool-heavy features.

{
  "model": "smart",
  "messages": [{"role": "user", "content": "Weather in Oslo?"}],
  "tools": [{
    "type": "function",
    "function": {
      "name": "get_weather",
      "description": "Current weather for a city",
      "parameters": {"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]}
    }
  }],
  "tool_choice": "auto"
}

response_format (JSON mode and schema) is also supported.

Reasoning

One vocabulary for every provider. Send any one of these; the gateway translates it to the chosen model's own control:

Mapping: OpenAI reasoning models get reasoning_effort; DeepSeek gets thinking (it thinks by default — send thinking: {"type":"disabled"} for fast agent turns); Claude (direct and on Bedrock) gets the control its generation accepts: Claude 4.6 and later get adaptive thinking with output_config.effort (and a readable summary), Claude 4.5 and earlier get extended thinking with a token budget (low 1024 · medium 4096 · high 16384, below max_tokens); Claude 5 thinks by default and thinking: {"type":"disabled"} turns it off where the model allows it; temperature/top_p are omitted wherever Claude rejects them (always on 4.7+); gpt-oss gets reasoning_effort; Nova 2 gets reasoningConfig. A model that cannot honour the control never receives it: the call still succeeds and the response carries x-models-dropped-params. Claude 4.5 cannot think while tool_choice forces a tool, and Opus 5.5 / Sonnet 5.5 / Fable 5.1 never accept a forced tool, so it becomes auto (reported).

Tool loops: send the reasoning back

The answer carries message.reasoning_content (streams: delta.reasoning_content) and, for Claude, message.reasoning_signature (streams: delta.reasoning_signature). Claude answers also carry message.reasoning_blocks — every signed thinking block of the turn, in order (streams: one delta.reasoning_blocks chunk before the finish chunk). Claude 5 hides its thinking text by default, so a block may hold only a signature. In a multi-turn tool loop, put the assistant message back exactly as received, including these fields. The gateway restores each provider's native form (DeepSeek: reasoning_content; Claude: a signed thinking block) and strips them for providers that do not take them.

{
  "model": "reasoning",
  "reasoning_effort": "medium",
  "tools": [...],
  "messages": [
    {"role": "user", "content": "Weather in Oslo, in Fahrenheit?"},
    {"role": "assistant", "content": null,
     "reasoning_content": "I need the weather tool first…", "reasoning_signature": "EqQBCkgIAR…",
     "tool_calls": [{"id": "t1", "type": "function", "function": {"name": "get_weather", "arguments": "{\"city\":\"Oslo\"}"}}]},
    {"role": "tool", "tool_call_id": "t1", "content": "{\"temp_c\": 4}"}
  ]
}

If a client drops the reasoning and DeepSeek refuses the follow-up turn, the gateway retries once with thinking disabled and says so in x-models-reasoning: disabled_for_tool_turn. On /v1/messages the same works with Anthropic thinking blocks (signature included) and signature_delta events.

Vision

Send image_url content parts. Use data URLs for portability; plain https URLs work only with OpenAI and Google models. The vision alias picks an image-capable model.

{
  "model": "vision",
  "messages": [{"role": "user", "content": [
    {"type": "text", "text": "What is in this picture?"},
    {"type": "image_url", "image_url": {"url": "data:image/png;base64,iVBORw0KGgo..."}}
  ]}]
}

Decisions (JEV)

Jev (TypeSafe's System One model) does not write text. You send a state and named, typed questions; it returns calibrated answers your code can branch on: a yes/no probability (noul), one option out of a closed set (choice), a rating on an ordered rubric (score) or a bounding box in an image. choice and score answers carry probabilities and a confidence from 0 to 1. The call is POST /v1/systemone with the same request and response shape as TypeSafe's API, so TypeSafe's SDKs work against the gateway; the gateway adds the key's budget and limits, request logging and a cost object. Model alias: decision (newest Jev, then the previous release). It is valid only on /v1/systemone; decision or a jev/… id on /v1/chat/completions or /v1/messages returns 400 use_systemone.

JEV or an LLM?

Rule of thumb: let JEV decide whether and where; let an LLM write what. JEV reads text (a string, JSON or an array); audio and video must be turned into text first. English is where accuracy is best; test other languages and lean on confidence.

One call, several questions

Questions are named by you; answers come back under the same names. Mix types freely. The state is read once and every question is evaluated against it, so put many questions in one request instead of one request each.

curl -s https://models.thundera.eu/v1/systemone \
  -H "Authorization: Bearer $MODELS_API_KEY" -H "Content-Type: application/json" \
  -H "X-Thundera-User: user-123" \
  -d '{
    "model": "decision",
    "state": "I was charged twice for my subscription this month. Please refund the extra charge.",
    "questions": {
      "billing": { "type": "noul",   "instructions": "Is this message about billing?" },
      "tone":    { "type": "choice", "instructions": "What is the tone of this message?",
                   "criteria": { "angry": "An upset or hostile message",
                                 "calm": "A neutral or polite message",
                                 "excited": "An enthusiastic or eager message" } },
      "urgency": { "type": "score",  "instructions": "How urgent is this message?",
                   "criteria": ["Can wait", "Needs attention this week", "Needs attention today"] }
    }
  }'

{
  "model": "jev-1.14.0",
  "answers": {
    "billing": { "type": "noul", "noul": 0.98 },
    "tone":    { "type": "choice", "choice": "calm", "confidence": 0.7,
                 "probabilities": { "angry": 0.15, "calm": 0.8, "excited": 0.05 } },
    "urgency": { "type": "score", "score": 1.7, "confidence": 0.55,
                 "legend": { "0": "Can wait", "1": "Needs attention this week", "2": "Needs attention today" },
                 "probabilities": { "0": 0.1, "1": 0.1, "2": 0.8 } }
  },
  "usage": { "input_tokens": 407, "output_tokens": 73 },
  "cost": { "request_id": "…", "model": "jev/jev-1.14.0", "currency": "USD", "total_usd": 0.00001709, "input_usd": 0.00001709 }
}

• noul: probability of yes, 0..1. Near 0.5 means unsure. No confidence field: treat distance from 0.5 as certainty.
• choice: criteria is an object (≤ 255 options, name → description). Returns choice, probabilities, confidence.
• score: criteria is an ordered array (1–10 level descriptions, position = score from 0). Returns a fractional score, legend, probabilities, confidence.
• box: locates something in an image (images).
• state, instructions and criteria descriptions may be a string, an object or an array; refer to a field in a question with backticks (`ticket.subject`).

Confidence-gated routing

The answer says what; confidence says whether to act. Gate each action at a threshold that matches its risk, and send the uncertain middle to a human or an LLM.

const r = await fetch("https://models.thundera.eu/v1/systemone", {
  method: "POST",
  headers: { Authorization: `Bearer ${process.env.MODELS_API_KEY}`, "Content-Type": "application/json" },
  body: JSON.stringify({
    model: "decision", state: ticketText,
    questions: { route: { type: "choice", instructions: "Where should this ticket go?",
      criteria: { billing: "Charges, refunds, invoices", bug: "Something is broken", other: "Anything else" } } },
  }),
}).then((x) => x.json());

const a = r.answers.route;
if (a.confidence >= 0.8) queue(a.choice);          // act automatically
else if (a.confidence >= 0.5) queue(a.choice, { review: true }); // act, flag for review
else escalateToHuman();                                         // or ask an LLM ("smart")

Thresholds are yours to tune: they scale with the cost of being wrong. A moderation block deserves a higher bar than a label. Pin jev/jev-1.13.0 or jev/jev-1.14.0 instead of decision if you tuned thresholds against one release; the response model tells you which one answered.

Fan-out

Ask everything you might need, including speculative questions, then decide in code which answers matter. Batching 13 questions in one call is about 12× cheaper and 10× faster than 13 calls (TypeSafe's benchmark). There is no question-count limit; the request must fit the context: 64k tokens per request and 32k per question (262k with long context). Typical uses: a guardrail for every user message (injection? PII? self-harm? off-topic?), a composite score built from several atomic score questions with weights in your code, ranking retrieved passages with one score per passage.

SDKs

TypeSafe's SDKs accept a base URL; use the gateway key as the API key and decision as the model (Python model=, Node defaultModel; an SDK left on its default jev-latest also works, the gateway resolves it to jev/jev-latest). The base URL is the host without /v1 (the SDK adds /v1/systemone). In-cluster: http://models.thundera.svc.cluster.local.

# pip install typesafe-sdk
import os
from typesafe_sdk import TypeSafeClient, Noul, Choice

client = TypeSafeClient(
    api_key=os.environ["MODELS_API_KEY"],
    base_url="https://models.thundera.eu",
    model="decision",
)
res = client.system_one(
    "I was charged twice. Please refund me.",
    {
        "billing": Noul(instructions="Is this about billing?"),
        "tone": Choice(instructions="Tone?", criteria={"angry": "Upset", "calm": "Neutral or polite"}),
    },
)
print(res.nouls["billing"].noul, res.model)

// npm install @typesafe-ai/sdk
import { choice, noul, TypeSafeClient } from "@typesafe-ai/sdk";

const client = new TypeSafeClient({
  apiKey: process.env.MODELS_API_KEY,
  baseURL: "https://models.thundera.eu",
  defaultModel: "decision",
});
const res = await client.systemOne({
  state: "I was charged twice. Please refund me.",
  questions: {
    billing: noul("Is this about billing?"),
    tone: choice("Tone?", { angry: "Upset", calm: "Neutral or polite" }),
  },
});
console.log(res.answers.billing.noul, res.answers.tone.choice);

Per-user budgets work as everywhere: send X-Thundera-User (Python extra_headers, Node headers option or defaultHeaders). The cost is in the response body (cost) and in x-models-cost-usd.

Images, Box and long context (opt-in)

TypeSafe ships these as experimental, behind TypeSafe-Danger-Zone-Images: true (any request with an assets map or a box question) and TypeSafe-Danger-Zone-Long-Context: true (262k context). The gateway forwards each header only if the key was granted the feature (decision.images, decision.long_context; set in /system → Keys); otherwise 403 feature_not_allowed. Decision models themselves are opt-in per key: a key needs the decision feature (ask an admin; app keys get it when the app adopts JEV). Images are PNG or JPEG in assets as {"type":"image","format":"png","data_base64":"…"} (multipart upload is not supported by the gateway), are billed as ordinary input tokens, and their bytes are never stored in the request log. Every place a placeholder appears counts the image again: put an image in the state once and refer to it by path.

Cost & limits

$0.042 per 1M input tokens, output tokens free (TypeSafe pricing): cost.total_usd = usage.input_tokens × 0.042 / 1,000,000. Every call carries about 270 tokens of fixed overhead, so a small request (a few questions on a short text) bills 300–500 input tokens, about $0.00002; 10 questions on a 2,000-character text is about 900 tokens, $0.00004. The budget pre-check reserves a pessimistic estimate (≤ the context limit, ≤ $0.0027 without long context) and the real charge uses the returned usage. The key's allowlist, RPM/TPM and X-Thundera-User budgets apply. Each call is a request_log row with modality decision; request and response payloads are stored (encrypted) only for keys with payload logging on. Metrics: models_decision_questions_total{app,type} and models_decision_confidence{app,type} (a histogram of confidence values, no content).

Errors follow the gateway shape plus TypeSafe's own: 422 validation_error (with detail[] as in TypeSafe, also at the top level), 400 max_tokens_exceeded (over the context length) or invalid_request_error, 413 body over 32 MiB, 415 unsupported Content-Encoding (gzip and zstd bodies are accepted up to the size limit), 429 with Retry-After, and 503 upstream_overloaded for TypeSafe's 529. Transient failures are retried once and fail over from jev-1.14.0 to jev-1.13.0. GET /v1/systemone/models (and models in GET /v1/models) lists the decision models in TypeSafe's shape.

Embeddings

POST /v1/embeddings with {model, input: string | string[], dimensions?, encoding_format?}, up to 2048 inputs. The embed alias is a single target (1024-d Titan v2) and never fails over, so vectors stay comparable.

curl -s https://models.thundera.eu/v1/embeddings \
  -H "Authorization: Bearer $MODELS_API_KEY" -H "Content-Type: application/json" \
  -d '{"model":"embed","input":["first text","second text"]}'

Audio

Transcriptions

POST /v1/audio/transcriptions, multipart: file (≤ 25 MB), model (transcribe), optional language, prompt, and response_format of json, text or verbose_json. Billed per audio minute.

curl -s https://models.thundera.eu/v1/audio/transcriptions \
  -H "Authorization: Bearer $MODELS_API_KEY" \
  -F file=@meeting.mp3 -F model=transcribe -F response_format=verbose_json

Speech (Premium TTS)

POST /v1/audio/speech with {model, input, voice, response_format?, speed?, voice_settings?} streams audio back (audio/mpeg for ElevenLabs by default). Billed per input character. Use the alias tts (ElevenLabs Multilingual v2, then Flash v2.5) or a model id such as elevenlabs/eleven_multilingual_v2. voice is an ElevenLabs voice id (e.g. TyBr4pQkGki6qkndJSGq) — it never fails over to another vendor; OpenAI models (openai/tts-1) take names such as alloy. input accepts up to 5000 characters (4096 for non-ElevenLabs models). Optional voice_settings: {stability, similarity_boost, style, use_speaker_boost, speed}. response_format: mp3 (default), opus, pcm (ElevenLabs). Errors: voice_not_found (404, also for a cloned voice owned by another key), voice_provider_mismatch (400).

curl -s https://models.thundera.eu/v1/audio/speech \
  -H "Authorization: Bearer $MODELS_API_KEY" -H "Content-Type: application/json" \
  -d '{"model":"tts","input":"Hello from Thundera","voice":"TyBr4pQkGki6qkndJSGq"}' -o hello.mp3

Voice cloning

ElevenLabs Instant Voice Cloning, owned per API key. POST /v1/audio/voices (multipart): name (required, ≤ 100 chars), optional description, one or more files (audio, ≤ 40 MB in total, otherwise 413) → 201 {"id","provider":"elevenlabs","name"}. GET /v1/audio/voices lists your key's voices. DELETE /v1/audio/voices/{voice_id} removes one of your voices (404 voice_not_found if it is not yours or does not exist). Use the returned id as voice in speech calls; another key gets 404 for it. At most 25 voices per key (409 voice_limit_reached). Cloning itself is free of per-character cost; every call is logged without audio.

curl -s https://models.thundera.eu/v1/audio/voices \
  -H "Authorization: Bearer $MODELS_API_KEY" \
  -F name="My voice" -F files=@sample.mp3
curl -s -X DELETE https://models.thundera.eu/v1/audio/voices/$VOICE_ID -H "Authorization: Bearer $MODELS_API_KEY"

Anthropic Messages endpoint

POST /v1/messages accepts the Anthropic wire format and can sit in front of any provider. max_tokens is required; system, tools, tool_choice, images, thinking, output_config.effort, cache_control (block-level and top-level automatic caching) and stream (Anthropic event protocol) are supported. Routed to a anthropic/… model, the call goes to the Claude API natively; signed thinking blocks round-trip unchanged. Point the Anthropic SDK's base URL at https://models.thundera.eu.

curl -s https://models.thundera.eu/v1/messages \
  -H "Authorization: Bearer $MODELS_API_KEY" -H "Content-Type: application/json" \
  -d '{"model":"smart","max_tokens":256,"system":"Be brief.",
       "messages":[{"role":"user","content":"Hello"}]}'

End users & per-user budgets

Attribute a call to your user with X-Thundera-User: <opaque user id> (in-cluster) or the body field user, which also works through CloudFront. For /v1/messages use metadata.user_id.

Each user has a monthly budget (UTC calendar month), the tighter of:

• the key policy end_user_limit_usd (set in /system), and
• the per-call X-Thundera-User-Budget-Usd: <usd> header (or body user_budget_usd): pass the user's plan limits.ai_monthly_usd from GET https://auth.thundera.eu/api/v1/auth/billing/effective-plan/<app>.

When exceeded the call fails with 402 end_user_budget_exceeded.

const res = await client.chat.completions.create(
  { model: "smart", messages },
  { headers: { "X-Thundera-User": user.id, "X-Thundera-User-Budget-Usd": String(plan.limits.ai_monthly_usd) } },
);

Budgets & limits

Guardrails are configured per key in /system.

• USD budget with period daily, monthly or total. It is a hard cap: each call reserves its worst-case cost (input estimate + max_tokens × output rate of the most expensive candidate) under a row lock. A call that might exceed the remaining budget is refused with 402 budget_exceeded; lower max_tokens.
• Alert thresholds (default 50 / 80 / 100 %) notify Slack and Prometheus.
• RPM / TPM limits return 429 rate_limit_exceeded with Retry-After.
• max_tokens_cap: larger requests are clamped; see header x-models-max-tokens-clamped.
• Model allowlist: model ids and/or aliases.
• Payload logging (log_payloads): metadata is always logged; request and response bodies are stored AES-256-GCM encrypted (≤ 64 KB each, 30-day retention) only when on. Off for eam, photomemory and whisper.
• Global monthly ceiling (Settings) blocks every non-exempt key with 402 global_ceiling_reached.

Cost object

Every response carries cost: top-level in chat bodies, in the final chunk when streaming, and in message_delta for the Messages stream.

{
  "request_id": "…", "model": "bedrock/us.amazon.nova-lite-v1:0", "provider": "bedrock", "alias": "fast",
  "currency": "USD", "total_usd": 0.00000204, "input_usd": 3.6e-7, "output_usd": 1.68e-6,
  "pricing_tier": "standard",
  "usage": { "input_tokens": 6, "output_tokens": 7, "cache_read_tokens": 0, "cache_write_tokens": 0,
            "audio_seconds": 0, "characters": 0 },
  "budget": { "period": "monthly", "period_start": "…", "limit_usd": 20,
              "spent_usd": 0.0000204, "remaining_usd": 19.99998 }
}

estimated: true means the provider did not report usage (for example an aborted stream) and tokens were estimated.

Usage API

GET /v1/usage returns the calling key's budget, limits and recent calls.

curl -s https://models.thundera.eu/v1/usage -H "Authorization: Bearer $MODELS_API_KEY"

Errors

Errors use the OpenAI shape, plus type: "error" at the top level for Anthropic clients.

{"type":"error","error":{"message":"…","type":"…","code":"budget_exceeded","request_id":"…"}}

CloudFront limits vs in-cluster

External callers pass through CloudFront. It forwards only Authorization, Accept, Origin and Host, and closes an origin response that is silent for 60 s. Streaming sends keepalives every 15 s; long non-streaming calls get a committed 200 with JSON-legal leading whitespace after 20 s.

In-cluster callers have none of these limits and every header (such as X-Thundera-User) passes. Prefer the in-cluster URL from Thundera apps.

App onboarding

1. Ask an admin to create a key for your app in /system (budget, limits, allowlist, payload logging).
2. The key is stored in SSM /thundera/<app>/models-api-key and synced to the Kubernetes Secret <app>-models, key MODELS_API_KEY.
3. Set the base URL and read the secret in your Deployment:

env:
  - name: MODELS_BASE_URL
    value: http://models.thundera.svc.cluster.local/v1
  - name: MODELS_API_KEY
    valueFrom:
      secretKeyRef: { name: <app>-models, key: MODELS_API_KEY }

4. Allow egress to the gateway in your NetworkPolicy:

egress:
  - to:
      - podSelector: { matchLabels: { app: models } }
    ports:
      - { protocol: TCP, port: 8080 }

Use aliases (fast, smart, …) rather than vendor ids, and pass X-Thundera-User for per-user budgets.