🔒 Air-Gapped AI
Air-Gapped SOC AI Architecture
Deploy local LLMs in air-gapped or restricted environments — Ollama vs vLLM vs LM Studio, hardware specs, multi-analyst serving, and RBI/SEBI compliance architecture patterns.
🇮🇳 Regulatory context: RBI Master Direction on IT (2023) and SEBI CSCRF require that sensitive financial data not leave approved infrastructure. Cloud AI APIs (GPT-4, Claude, Gemini) are not permitted for processing payment data, trade data, or customer PII. Local LLM deployment is the only compliant path for AI-assisted SOC operations in regulated Indian entities.
Architecture Pattern 1 — Single Analyst Workstation
Simplest deployment. Each analyst runs Ollama locally. No server required. Suitable for small teams or proof-of-concept.
┌─────────────────────────────────────────────────────┐
│ ANALYST WORKSTATION │
│ │
│ ┌─────────────┐ ┌──────────────┐ │
│ │ Web Browser│───▶│ Open WebUI │ │
│ │ (Analyst) │ │ localhost:3000│ │
│ └─────────────┘ └──────┬───────┘ │
│ │ │
│ ┌───────▼───────┐ │
│ │ Ollama │ │
│ │ localhost:11434│ │
│ └───────┬───────┘ │
│ │ │
│ ┌───────▼───────┐ │
│ │ Llama 3.2 8B │ │
│ │ (4.7 GB RAM) │ │
│ └───────────────┘ │
│ │
│ NO INTERNET REQUIRED. ZERO DATA EGRESS. │
└─────────────────────────────────────────────────────┘
Architecture Pattern 2 — Shared SOC AI Server
Recommended for teams of 5-20 analysts. One GPU server serves all workstations. Analysts access via browser — no local install needed.
┌──────────────────────────────────────────────────────────────────┐
│ SOC NETWORK (Air-Gapped / Restricted) │
│ │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ Analyst 1 │ │ Analyst 2 │ │ Analyst N │ │
│ │ Workstation │ │ Workstation │ │ Workstation │ │
│ └──────┬───────┘ └──────┬───────┘ └──────┬───────┘ │
│ │ │ │ │
│ └──────────────┬────┘────────────────────┘ │
│ │ HTTP (SOC LAN only) │
│ ▼ │
│ ┌─────────────────────┐ │
│ │ AI SERVER │ │
│ │ Open WebUI :3000 │ │
│ │ Nginx (reverse proxy)│ │
│ │ Ollama API :11434 │ │
│ │ Llama 3.1 70B │ ← NVIDIA A100 80GB │
│ └─────────────────────┘ │
│ │
│ FIREWALL: Block all traffic to/from internet │
│ Only SOC VLAN can reach AI server on port 3000 │
└──────────────────────────────────────────────────────────────────┘
Architecture Pattern 3 — SIEM-Integrated AI Pipeline
Production-grade. SIEM alerts automatically flow through AI enrichment before reaching analyst queue. Reduces manual triage by 40-60%.
SIEM (Splunk / Sentinel)
│
│ Alert webhook
▼
┌──────────────────┐
│ Alert Enrichment │ ← Python/Node.js service
│ Service │ on SOC network
└─────────┬────────┘
│ POST /api/chat
▼
┌──────────────────┐
│ Ollama Server │ ← Local, air-gapped
│ Llama 3.2 8B │
└─────────┬────────┘
│ AI triage result
▼
┌──────────────────┐
│ SIEM Alert Queue│ ← Enriched with:
│ (Analyst View) │ • True/False positive score
└──────────────────┘ • MITRE ATT&CK mapping
• Recommended action
• CERT-In flag (if required)
Ollama vs vLLM vs LM Studio — Which to Use
| Feature | Ollama | vLLM | LM Studio |
|---|---|---|---|
| Ease of setup | ⭐⭐⭐⭐⭐ — one command | ⭐⭐⭐ — Python + CUDA setup | ⭐⭐⭐⭐⭐ — GUI installer |
| Windows support | Yes | Limited (WSL2) | Yes — native |
| Linux support | Yes | Yes — best performance | No |
| macOS support | Yes (Apple Silicon optimised) | Limited | Yes |
| GPU utilisation | Good | Excellent — PagedAttention | Good |
| Concurrent users | Up to ~10 (limited concurrency) | Hundreds — production-grade | 1 user at a time |
| API compatibility | OpenAI-compatible REST API | OpenAI-compatible REST API | OpenAI-compatible REST API |
| Model library | Large (Ollama library + GGUF) | HuggingFace models (AWQ/GPTQ) | GGUF format models |
| Web UI included | No (Open WebUI separate) | No | Yes — built-in |
| Best for | Small-medium SOC teams, easy deployment | High-concurrency production (20+ analysts) | Individual analyst desktop |
| Recommended for Indian SOC | ✅ Yes — most teams | For large BFSI with 20+ analysts | Per-analyst laptop deployment |
vLLM setup for high-concurrency SOC
# Install vLLM (requires NVIDIA GPU)
pip install vllm
# Serve Llama 3.2 with OpenAI-compatible API
python -m vllm.entrypoints.openai.api_server \
--model meta-llama/Llama-3.2-8B-Instruct \
--host 0.0.0.0 \
--port 8000 \
--max-model-len 8192 \
--gpu-memory-utilization 0.90 \
--tensor-parallel-size 2 # For multi-GPU
# Test
curl http://localhost:8000/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{"model":"meta-llama/Llama-3.2-8B-Instruct","messages":[{"role":"user","content":"Analyse this log..."}]}'
Hardware Sizing Guide
⚠️ VRAM is the critical constraint — the model must fit in GPU VRAM for GPU acceleration. If it doesn't fit, it falls back to CPU which is 10-50x slower.
| Setup | Model | GPU | VRAM | System RAM | Approx Cost (India) | Throughput |
|---|---|---|---|---|---|---|
| Analyst laptop (minimum) | Phi-3.5 Mini (2.2 GB) | None (CPU) | N/A | 8 GB | Existing hardware | ~8 tok/sec |
| Analyst workstation | Llama 3.2 8B | RTX 4060 (8 GB) | 8 GB | 16 GB | ₹35,000 GPU | ~25 tok/sec |
| Power analyst / researcher | Llama 3.2 8B / 70B | RTX 4090 (24 GB) | 24 GB | 32 GB | ₹1.6L GPU | ~60 tok/sec (8B) |
| Small team server (5-10 users) | Llama 3.2 8B | RTX 3090 (24 GB) | 24 GB | 64 GB | ₹1.2L GPU + server | ~50 tok/sec shared |
| Medium team server (10-20 users) | Llama 3.1 70B | 2x RTX 3090 (48 GB total) | 48 GB | 128 GB | ₹2.5L GPUs + server | ~20 tok/sec shared |
| Enterprise BFSI (20+ users) | Llama 3.1 70B | NVIDIA A100 80 GB | 80 GB | 256 GB | ₹30-40L (A100) | ~100 tok/sec shared |
| Large bank / RBI-regulated | Llama 3.1 70B | 2x A100 80 GB | 160 GB | 512 GB | ₹60-80L | ~200 tok/sec shared |
OpenAI-Compatible API Wrapper
Ollama and vLLM both expose an OpenAI-compatible API. This means any tool built for OpenAI can be redirected to your local model by changing just the base URL and model name — no code changes needed.
## Drop-in replacement: point OpenAI SDK to local Ollama
from openai import OpenAI
# Instead of:
# client = OpenAI(api_key="sk-...")
# Use local Ollama — no API key needed, no data leaves your network
client = OpenAI(
base_url="http://localhost:11434/v1",
api_key="ollama" # Required by SDK but not validated
)
def soc_query(user_message: str, system_prompt: str = None) -> str:
messages = []
if system_prompt:
messages.append({"role": "system", "content": system_prompt})
messages.append({"role": "user", "content": user_message})
response = client.chat.completions.create(
model="llama3.2", # Change to any pulled Ollama model
messages=messages,
temperature=0.1, # Low temperature for factual security analysis
max_tokens=1000
)
return response.choices[0].message.content
# Usage — works exactly like OpenAI but locally
result = soc_query(
"Analyse this alert: Outbound DNS query to mining-pool-xyz.com from 10.0.1.50",
system_prompt="You are a SOC analyst. Classify alerts and provide CERT-In reporting guidance."
)
print(result)
Nginx reverse proxy for SOC-wide access
# /etc/nginx/sites-available/soc-ai
server {
listen 80;
server_name soc-ai.internal; # Internal DNS name
# Only allow SOC VLAN
allow 10.0.1.0/24; # SOC analyst subnet
allow 10.0.2.0/24; # SOC server subnet
deny all;
# Open WebUI (analyst chat interface)
location / {
proxy_pass http://localhost:3000;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection 'upgrade';
}
# Ollama API (for tool integrations)
location /api/ {
proxy_pass http://localhost:11434;
proxy_read_timeout 300s; # Allow long inference
}
}
Compliance Checklist for Local LLM Deployment
1.
No internet egress from AI server
Mandatory for BFSI/Gov
Configure firewall rules to block all outbound traffic from the AI server. The model runs fully offline after initial download.
2.
Model downloaded before air-gapping
Mandatory
Download model weights (Llama 3.2, Mistral, etc.) before cutting internet access. Weights are static files — no internet needed at inference time.
3.
Access restricted to SOC VLAN only
Mandatory
Nginx/firewall restricts AI server access to SOC analyst subnet. No other network segments can reach it.
4.
No customer/policyholder data in prompts
Mandatory per RBI/SEBI
Define clear data classification: only anonymised or de-identified data may be sent to AI. Never include Aadhaar, PAN, account numbers, or full payment data in prompts.
5.
Prompt logging and audit trail
Mandatory
Log all queries sent to local AI with analyst username, timestamp, and response. Retain logs as per CERT-In 180-day requirement.
6.
Model version documented
Mandatory for regulated entities
Document which model version (e.g., Llama-3.2-8B-Instruct, SHA256 hash of weights) is deployed. Include in IT asset register.
7.
Approval in IT risk framework
Mandatory per RBI IT Framework
Local LLM deployment covered in IT Risk Assessment. Board/IT Committee awareness documented.
8.
Incident response plan for AI failure
Recommended
What happens if AI server goes down? Analysts must be able to triage alerts manually. AI is a productivity tool, not a control.
9.
Periodic model evaluation
Recommended
Quarterly review of model accuracy on your specific log types and alert categories. Update to newer model version if accuracy degrades.
10.
Staff training on AI limitations
Mandatory — document in training records
Analysts trained: AI can hallucinate, AI is not authoritative, AI output must be verified for critical decisions. Cannot replace analyst judgment.