ASK Weaviate — Overview

Weaviate is the vector database behind the Knowledge Management Service. It stores 1024-dimensional document embeddings and serves approximate-nearest-neighbour similarity search for RAG retrieval. It is deployed as a vendored subchart of the KMS Helm chart — not a standalone ArgoCD Application — so its lifecycle follows KMS.

---

What it does

Responsibility	Detail
Embedding storage	Stores document chunk embeddings produced by qwen/qwen3-embedding-0-6b (1024-dim) — one Weaviate class per organisation
Similarity search	Serves ANN queries from the KMS retriever to find the most relevant chunks for a user query
Re-rank input	Returns the top-N candidates that the Nemotron reranker (nvidia/llama-nemotron-rerank-1b-v2) then scores and filters
API-key auth	Enforces per-key access control — the KMS client always sends Authorization: Bearer <api_key>

---

Integrations

System	Role
Knowledge Management Service	Sole write and read client — all embedding storage and retrieval goes through KMS
Foundry LiteLLM	Indirectly — KMS calls Foundry to generate embeddings, then writes them to Weaviate
Ceph RGW (ceph-block)	Weaviate persistence volume — 50Gi RWO PVC on the ceph-block StorageClass
Vault / ESO	AUTHENTICATION_APIKEY_ALLOWED_KEYS sourced from knowledge-management-service-secret (same value as VECTOR_DB__API_KEY)

---

Key configuration facts

Concern	Setting
Image	harbor.cl1.sq4.aegis.internal/ask/semitechnologies/weaviate:1.34.0
HTTP service	weaviate.ask.svc.cluster.local:80
gRPC service	weaviate-grpc.ask.svc.cluster.local:50051
Storage	ceph-block, 50Gi
Auth	AUTHENTICATION_APIKEY_ENABLED: "true" — anonymous access disabled
Scheduling	nodeSelector: node.kubernetes.io/role: ask71 + toleration

Service name is hardcoded

The Weaviate chart hardcodes its resource name to weaviate (no fullnameOverride). The KMS TOML must use weaviate.ask.svc.cluster.local and weaviate-grpc.ask.svc.cluster.local — do not change either side independently.

---

Air-gap fixes applied

Three issues required fixes before Weaviate would run in this environment:

1. Init container disabled — configure-sysctl requires privileged: true, which is forbidden by the baseline Pod Security Admission policy on the ask namespace. Disabled the init container; vm.max_map_count is set on the ask71 nodes instead. See KMS Architecture & Ops.
2. memberlist CGNAT fix — The pod CIDR is 100.x (CGNAT), which memberlist rejects as non-private. The vendored chart template sets CLUSTER_ADVERTISE_ADDR from status.podIP via a downward-API field. See KMS Architecture & Ops §2.
3. API-key auth required — KMS always sends an API key; anonymous-only Weaviate returns 401. Key sourced from the same Vault secret as VECTOR_DB__API_KEY. See KMS Config & Model References.

---

Verify

# Pod is running
kubectl get pods -n ask | grep -i weaviate

# API-key auth works (run from a KMS pod)
WV=$(kubectl get secret -n ask knowledge-management-service-secret \
  -o jsonpath='{.data.VECTOR_DB__API_KEY}' | base64 -d)
kubectl exec -n ask deploy/knowledge-management-service -i -- env WV="$WV" python3 - <<'EOF'
import os, httpx
b = "http://weaviate.ask.svc.cluster.local"
print("ready:", httpx.get(b+"/v1/.well-known/ready", timeout=8).status_code)           # 200
print("auth :", httpx.get(b+"/v1/meta", headers={"Authorization": "Bearer "+os.environ["WV"]}, timeout=8).status_code)  # 200
EOF

---

Related pages

Page	Purpose
KMS Deployment	Image mirroring, chart vendoring, full deploy runbook
KMS Config & Model References	Apikey auth wiring, embedding dimensions, model references
KMS Architecture & Ops	PSA fix, memberlist fix, retriever reranker, day-2 ops