Model Routing Map — Air-Gap (mod-auh1-dev-ask)¶
Captured: 2026-06-23.
There are two distinct call patterns. User → Service is when the user picks a model in the frontend and that model name travels through the stack to Foundry. Service → Service is when a service makes its own LLM/embedding/rerank call using a model name hardcoded in its TOML config.
How it flows¶
User picks model in UI
→ core-service stores model id in llm_config
→ assistant-service receives request
→ looks up model id in models.yaml (baked into image)
→ sends api_model_name to Foundry LiteLLM
→ Foundry routes to vLLM backend
Service needs LLM internally (embedding / rerank / code-gen / etc.)
→ reads model name directly from TOML config (no models.yaml lookup)
→ calls Foundry LiteLLM with that name
→ Foundry routes to vLLM backend
Foundry served models (ground truth)¶
Confirmed via GET /model/info on foundry-litellm.foundry.svc.cluster.local:4000.
| Foundry model name | Actual backend | Used by |
|---|---|---|
qwen/qwen3-5-122b-a10b |
hosted_vllm/Qwen/Qwen3-5-122B-A10B |
assistant-service main LLM · data viz |
qwen3-5-122b-a10b |
hosted_vllm/Qwen/Qwen3-5-122B-A10B |
Alias (no prefix) |
qwen/qwen3-5-122b-a10b-kms |
hosted_vllm/Qwen/Qwen3-5-122B-A10B |
KMS VLM (dedicated instance) |
qwen3-5-122b-a10b-kms |
hosted_vllm/Qwen/Qwen3-5-122B-A10B |
Alias (no prefix) |
Qwen/Qwen3-4B-Instruct-2507 |
hosted_vllm/Qwen/Qwen3-4B-Instruct-2507 |
assistant-service small LLM (replaced qwen3-5-4b) |
qwen3-4b-instruct-2507 |
hosted_vllm/Qwen/Qwen3-4B-Instruct-2507 |
Alias (no prefix) |
qwen/qwen3-5-4b, qwen3-5-4b |
→ Qwen/Qwen3-4B-Instruct-2507 |
Compat redirect aliases — old model removed |
qwen/qwen3-embedding-0-6b |
hosted_vllm/Qwen/Qwen3-Embedding-0.6B |
assistant-service tool embeddings · KMS ingestion embeddings |
qwen3-embedding-0-6b |
hosted_vllm/Qwen/Qwen3-Embedding-0.6B |
Alias (no prefix) |
nvidia/llama-nemotron-rerank-1b-v2 |
hosted_vllm/nvidia/llama-nemotron-rerank-1b-v2 |
assistant-service reranker · KMS retriever reranker |
llama-nemotron-rerank-1b-v2 |
hosted_vllm/nvidia/llama-nemotron-rerank-1b-v2 |
Alias (no prefix) |
Service → Service (internal TOML-configured calls)¶
These model names are read directly from TOML at startup — no models.yaml lookup. All are confirmed working.
assistant-service¶
| Config key | Model name sent to Foundry | Purpose | Status |
|---|---|---|---|
foundry.llm_model |
qwen/qwen3-5-122b-a10b |
Main LLM for all internal reasoning | ✅ Served |
foundry.small_llm_model |
Qwen/Qwen3-4B-Instruct-2507 |
Lightweight tasks, summaries | ✅ Served |
foundry.data_visualization_llm |
qwen/qwen3-5-122b-a10b |
Data viz generation | ✅ Served |
foundry.tool_embedding_model |
qwen/qwen3-embedding-0-6b |
Tool / RAG embeddings | ✅ Served |
foundry.reranker.model |
nvidia/llama-nemotron-rerank-1b-v2 |
Result reranking | ✅ Served |
foundry.code_generator_model |
qwen/qwen3-5-122b-a10b |
Code generation | ✅ Served |
foundry.docx_prepare_context_model |
qwen/qwen3-5-122b-a10b |
DOCX context prep | ✅ Served |
intelligence.profiler_model |
qwen/qwen3-5-122b-a10b |
Agent profiling | ✅ Served |
intelligence.code_generator_model |
qwen/qwen3-5-122b-a10b |
Intelligence code-gen | ✅ Served |
intelligence.query_validator_model |
qwen/qwen3-5-122b-a10b |
Query validation | ✅ Served |
intelligence.column_mapper_model |
qwen/qwen3-5-122b-a10b |
Column mapping | ✅ Served |
knowledge-management-service (KMS)¶
| Config key | Model name sent to Foundry | Purpose | Status |
|---|---|---|---|
organization.vlm_model_name |
qwen/qwen3-5-122b-a10b-kms |
Main LLM / VLM for ingestion & queries | ✅ Served |
organization.embedding_model_id |
qwen/qwen3-embedding-0-6b |
Document embeddings (1024-dim, Weaviate) | ✅ Served |
retriever.reranker.model |
nvidia/llama-nemotron-rerank-1b-v2 |
Retrieval reranking | ✅ Served |
User → Service (UI model picker → models.yaml → Foundry)¶
The user selects a model ID in the frontend. Core-service stores it in llm_config.model. Assistant-service looks up the ID in models.yaml (baked into the image) to get api_model_name, then sends that to Foundry.
models.yaml is baked into the image
api_model_name cannot be overridden via Helm values. Fixes require either an image rebuild OR adding the broken alias as a passthrough model in Foundry's LiteLLM config.
Models with a Foundry-served backend (fixable without rebuild)¶
| UI model id | api_model_name (models.yaml) |
Foundry equivalent | Fix needed |
|---|---|---|---|
qwen3-5-122b-a10b |
foundry-stg/core42-oicm-auh1-shd/qwen-3-5-122b-a10b |
qwen/qwen3-5-122b-a10b |
Add foundry-stg/... alias to Foundry |
Fix command (one-shot, add both aliases to Foundry):
kubectl exec -i -n ask deploy/assistant-service-intelligence -- env ADMIN="$ADMIN" python3 - <<'EOF'
import os, httpx
aliases = [
{"model_name": "aicloud/gpt-oss-120b",
"litellm_params": {"model": "qwen/qwen3-5-122b-a10b"}},
{"model_name": "foundry-stg/core42-oicm-auh1-shd/qwen-3-5-122b-a10b",
"litellm_params": {"model": "qwen/qwen3-5-122b-a10b"}},
]
for a in aliases:
r = httpx.post(
"http://foundry-litellm.foundry.svc.cluster.local:4000/model/new",
headers={"Authorization": "Bearer " + os.environ["ADMIN"]},
json=a, timeout=15
)
print(a["model_name"], "→", r.status_code)
EOF
Alias persistence
LiteLLM API-added aliases survive until the foundry-litellm pod restarts. For permanent fix, add the model entries to the Foundry LiteLLM ConfigMap.
Models that require cloud connectivity (hide from UI in air-gap)¶
These point to Azure, Anthropic, xAI, z-ai, Minimax, or OpenRouter — all unavailable in the air-gapped environment. Users who select them will get a 400/403 error.
Action: Disable or remove these model IDs from the core-service model list in values-mod-auh1-dev-ask.yaml.
| UI model id | api_model_name |
Provider | Reason broken |
|---|---|---|---|
gpt-5 |
azure/gpt-5 |
Azure OpenAI | Azure endpoint not available |
gpt-5-2 |
azure/gpt-5.2 |
Azure OpenAI | Azure endpoint not available |
gpt-5-3-chat |
azure/gpt-5.3-chat |
Azure OpenAI | Azure endpoint not available |
gpt-5-4 |
azure/gpt-5.4 |
Azure OpenAI | Azure endpoint not available |
gpt-5-chat |
azure/gpt-5-chat |
Azure OpenAI | Azure endpoint not available |
gpt-5-mini |
azure/gpt-5-mini |
Azure OpenAI | Azure endpoint not available |
gpt-5-nano |
azure/gpt-5-nano |
Azure OpenAI | Azure endpoint not available |
gpt-4o |
(not in models.yaml) | Azure OpenAI | No routing |
anthropic/claude-sonnet-4-6 |
anthropic/claude-sonnet-4-6 |
Anthropic | No Anthropic endpoint |
anthropic/claude-opus-4-6 |
anthropic/claude-opus-4-6 |
Anthropic | No Anthropic endpoint |
anthropic/claude-sonnet-4-5 |
anthropic/claude-sonnet-4-5 |
Anthropic | No Anthropic endpoint |
anthropic/claude-sonnet-4-5-arabic |
anthropic/claude-sonnet-4-5 |
Anthropic | No Anthropic endpoint |
anthropic/claude-haiku-4-5 |
anthropic/claude-haiku-4-5 |
Anthropic | No Anthropic endpoint |
anthropic/claude-sonnet-4 |
anthropic/claude-sonnet-4 |
Anthropic | No Anthropic endpoint |
anthropic/claude-sonnet-4-arabic |
anthropic/claude-sonnet-4 |
Anthropic | No Anthropic endpoint |
anthropic/claude-sonnet-4-long-context |
anthropic/claude-sonnet-4-long-context |
Anthropic | No Anthropic endpoint |
anthropic/claude-sonnet-4-5-internal-thinking-experimental |
anthropic/claude-sonnet-4-5 |
Anthropic | No Anthropic endpoint |
anthropic/claude-sonnet-4-internal-thinking-experimental |
anthropic/claude-sonnet-4 |
Anthropic | No Anthropic endpoint |
grok-4-1-fast |
x-ai/grok-4.1-fast |
xAI | No xAI endpoint |
grok-4-1-fast-arabic |
x-ai/grok-4.1-fast |
xAI | No xAI endpoint |
grok-4 |
x-ai/grok-4 |
xAI | No xAI endpoint |
glm-4-7 |
z-ai/glm-4.7 |
Zhipu AI | No z-ai endpoint |
glm-5 |
z-ai/glm-5 |
Zhipu AI | No z-ai endpoint |
minimax-m2-1 |
minimax/minimax-m2.1 |
Minimax | No Minimax endpoint |
minimax-m2-5 |
minimax/minimax-m2.5 |
Minimax | No Minimax endpoint |
google-gemini-2-5-flash |
openrouter-global-ask-shd/google-gemini-2-5-flash |
OpenRouter/Google | No OpenRouter endpoint |
Models with no models.yaml mapping (routing unknown)¶
| UI model id | Notes |
|---|---|
qwen2.5-32b-instruct |
No models.yaml entry — model name passed raw to Foundry; not served |
qwen3-next-80b-a3b-thinking-fp8 |
No models.yaml entry — not served |
meta/llama3-3-70b-instruct |
No models.yaml entry — not served |
tiiuae/falcon3-10b-instruct |
No models.yaml entry — not served |
tiiuae/falcon3-7b-instruct |
No models.yaml entry — not served |
ai71-agrillm/falcon3-10b-finetuned-0 |
No models.yaml entry — fine-tuned model, not in Foundry |
ai71-agrillm/llama31-8b-finetuned-0 |
No models.yaml entry — fine-tuned model, not in Foundry |
ai71-agrillm/qwen25-7b-finetuned-0 |
No models.yaml entry — fine-tuned model, not in Foundry |
ai71-agrillm/qwen25-32b-finetuned-0 |
No models.yaml entry — fine-tuned model, not in Foundry |
ai71-agrillm/falcon3-10b-finetuned-1 |
No models.yaml entry — fine-tuned model, not in Foundry |
ai71-agrillm/llama33-70b-finetuned-0 |
No models.yaml entry — fine-tuned model, not in Foundry |
BAAI/bge-m3 |
Embedding model; no models.yaml entry — not served |
AI71ai/agrillm-embedding-bge-m3-v1 |
Embedding model; no models.yaml entry — not served |
Summary — what to do next¶
| Action | What | How |
|---|---|---|
| 1 | Add Foundry aliases for the two fixable models | Run the model/new command above |
| 2 | Make aliases permanent | Edit Foundry LiteLLM ConfigMap in foundry namespace |
| 3 | Disable cloud-only + unmapped models in UI | Remove their entries from core-service values in mod-ask-devops |
| 4 | Proper fix for models.yaml |
Update aicloud/ → openai/ and foundry-stg/... → qwen/ in source; rebuild image |