ASK Assistant Service — Overview
Assistant Service is the AI execution layer of the ASK platform. It receives a user's message, orchestrates tool use (code execution, data visualisation, document retrieval, web search), calls the LLM via Foundry, and streams the response back. It runs as four separate deployments so each workload type can scale independently.
What it does
| Responsibility |
Detail |
| Chat orchestration |
Receives the user message from core-service, loads the agent's system prompt and tool configuration, calls the LLM, and streams the response |
| Tool execution |
Dispatches tool calls to specialist sub-services: code execution (Daytona), data visualisation, document retrieval (KMS), web search |
| Intelligence pipeline |
Runs the assistant-service-intelligence worker for complex multi-step reasoning, code generation, query validation, and column mapping |
| Tooling workers |
Runs the assistant-service-tooling worker for retrieval-augmented generation, reranking, and embeddings |
| Generic worker |
Handles background tasks dispatched via Hatchet (title generation, conversation archiving) |
| Safety checks |
Input/output filtering before the LLM response is returned to the user |
Workload architecture
| Deployment |
Replicas |
Role |
assistant-service |
2 |
API gateway — receives requests, authenticates, dispatches to intelligence |
assistant-service-intelligence |
2 |
Main LLM orchestration — calls Foundry for chat completions |
assistant-service-tooling |
2 |
Embeddings, reranking, RAG retrieval via KMS |
assistant-service-generic-worker |
1 |
Hatchet-driven background jobs (title generation, etc.) |
Integrations
| System |
Role |
| Foundry LiteLLM |
All LLM and embedding calls. Endpoint: http://foundry-litellm.foundry.svc.cluster.local:4000/v1 |
| Core Service |
Authentication (machine token), model config, user session validation |
| Knowledge Management Service (KMS) |
Document retrieval for RAG — called by the tooling worker |
| Hatchet |
Receives async job triggers (title generation, bulk operations) |
| Daytona |
Sandboxed code execution environment — called by the tooling worker for the code_execution tool |
| Redis |
Caches embeddings and conversation state. assistant-service-redis-master.ask.svc.cluster.local:6379 |
| Vault / ESO |
All secrets synced into assistant-service-secret from Vault KV v2 |
| Harbor |
Source of the assistant-service image (tag: staging, mirrored to harbor.cl1.sq4.aegis.internal/ask/) |
Model routing
Assistant Service has two model name systems:
- User → Service: The user picks a model in the UI. Core Service stores the model ID in
llm_config. Assistant Service looks up the ID in models.yaml (baked into the image) to get api_model_name, then sends that to Foundry.
- Service → Service: Internal LLM calls (embeddings, reranking, code generation) use model names hardcoded in the TOML config — no
models.yaml lookup.
See Model References (two-layer) for the full mapping.
Pages in this section