🦙 Local AI
Ollama Local AI Setup Guide
Run AI models entirely on-premise for cybersecurity tasks. No data leaves your network — critical for Indian BFSI, government, and defence organisations with data sovereignty requirements.
🇮🇳 Why local AI matters for Indian organisations: RBI, SEBI, and IRDAI-regulated entities cannot send sensitive data (payment logs, audit trails, policyholder data) to cloud AI APIs. Ollama solves this — run Llama 3.2, Mistral, or Phi-3 entirely inside your own network with zero internet egress.
Step-by-Step Installation
Linux / Ubuntu (recommended for SOC servers)
# Install Ollama
curl -fsSL https://ollama.com/install.sh | sh
# Verify installation
ollama --version
# Pull your first security-relevant model (Llama 3.2 8B — good balance)
ollama pull llama3.2
# Run interactively
ollama run llama3.2
# Run as background service (auto-start)
sudo systemctl enable ollama
sudo systemctl start ollama
# Check service status
sudo systemctl status ollama
Windows (SOC analyst workstation)
# Download Ollama installer from ollama.com/download
# Run OllamaSetup.exe — installs and starts service automatically
# Open PowerShell and pull a model
ollama pull llama3.2
# Run from PowerShell
ollama run llama3.2
# Ollama runs on localhost:11434 by default
# Test with: curl http://localhost:11434/api/generate -d '{"model":"llama3.2","prompt":"Hello"}'
macOS (security researcher workstation)
# Install via Homebrew
brew install ollama
# Or download from ollama.com/download (macOS app)
# Pull and run
ollama pull llama3.2
ollama run llama3.2
# Apple Silicon (M1/M2/M3) — Ollama uses Metal GPU acceleration automatically
# Intel Mac — runs on CPU, slower but functional for Phi-3-mini
Docker (isolated SOC environment)
# Run Ollama in Docker (CPU only)
docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama
# With GPU support (NVIDIA)
docker run -d --gpus=all -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama
# Pull a model inside the container
docker exec -it ollama ollama pull llama3.2
# Test API
curl http://localhost:11434/api/chat -d '{
"model": "llama3.2",
"messages": [{"role": "user", "content": "Analyse this log: Failed password for root from 192.168.1.50"}]
}'
Model Selection for Security Tasks
| Model | Size | Best For | Pull Command | Min RAM |
|---|---|---|---|---|
| Llama 3.2 (8B) | 4.7 GB | Log analysis, alert triage, report writing, rule explanation. Best all-rounder for SOC tasks. | ollama pull llama3.2 |
8 GB |
| Llama 3.1 (70B) | 40 GB | Complex threat analysis, policy generation, detailed incident reports. Near-GPT-4 quality. | ollama pull llama3.1:70b |
48 GB |
| Phi-3.5 Mini | 2.2 GB | Fast triage on analyst laptops, quick log classification. Very fast even on CPU. | ollama pull phi3.5 |
4 GB |
| Mistral 7B | 4.1 GB | Structured output (JSON detection rules), technical writing, policy drafts. | ollama pull mistral |
8 GB |
| CodeLlama 13B | 7.4 GB | Writing detection rules (KQL/SPL/Python), YARA rules, parsing scripts. | ollama pull codellama:13b |
16 GB |
| Gemma 2 (9B) | 5.4 GB | Fast, Google-trained. Good for summarisation and classification tasks. | ollama pull gemma2 |
8 GB |
| Qwen2.5 (7B) | 4.4 GB | Strong multilingual support — useful for Hindi/regional language content in logs. | ollama pull qwen2.5 |
8 GB |
| DeepSeek-R1 (8B) | 4.7 GB | Strong reasoning — good for complex incident investigation chains and root cause analysis. | ollama pull deepseek-r1 |
8 GB |
Recommended setup for Indian SOC teams
→
Small team (1-5 analysts, laptop deployment): Phi-3.5 Mini or Llama 3.2 8B on each analyst laptop. No GPU required — runs on CPU with 8 GB RAM. Slower (15-30 sec/response) but functional.
→
Medium SOC (shared server, 10-20 analysts): Llama 3.2 8B or Mistral 7B on a dedicated server with NVIDIA RTX 3090 or A100. 2-3 second response time. Serve via Ollama API to all workstations.
→
BFSI / regulated entity (air-gapped): Llama 3.1 70B on a server with 2x A100 80GB. Full air-gap — no internet. Highest quality output. Use Open WebUI for a ChatGPT-like interface on the local network.
→
Red team / pen test lab: CodeLlama 13B for writing detection rules and scripts. Llama 3.2 8B for report writing. Runs on gaming laptop with RTX 4070.
Hardware Requirements
| Deployment | CPU | RAM | GPU (optional) | Response Time |
|---|---|---|---|---|
| Analyst laptop (Phi-3.5 Mini) | Any modern Intel/AMD | 8 GB | None required | 15–45 sec (CPU) |
| Analyst laptop (Llama 3.2 8B) | 8+ core Intel/AMD | 16 GB | RTX 3060+ (optional) | 5–15 sec GPU, 60+ sec CPU |
| Team server (Llama 3.2 8B) | 16+ core Xeon/EPYC | 32 GB | NVIDIA RTX 3090 (24GB VRAM) | 2–5 sec |
| Enterprise server (Llama 3.1 70B) | 32+ core Xeon/EPYC | 128 GB | 2x A100 80GB or 4x RTX 3090 | 3–8 sec |
| Air-gapped SOC (production) | Dual Xeon or EPYC | 256 GB | 2x H100 or 4x A100 | 1–3 sec |
Indian server procurement options
→
Dell PowerEdge R750: Available via Dell India. Dual Xeon, up to 512 GB RAM. Configure with NVIDIA A100 for production Llama 70B deployment.
→
HP ProLiant DL380: HP India. Common in Indian banking DC. Supports NVIDIA Tesla T4 (16 GB VRAM) — good for Llama 3.2 8B shared deployment.
→
NVIDIA DGX A100: Available via NVIDIA India partners. Purpose-built for LLM inference. Overkill for most SOC teams but relevant for large BFSI.
→
On-premise gaming GPU workstation: RTX 4090 (24 GB VRAM) workstation — ₹3–4 lakh. Runs Llama 3.2 8B at 20 tokens/sec. Practical for small teams.
→
MeitY-empanelled cloud with GPU: For regulated entities that cannot buy hardware — NIC Cloud and ESDS offer GPU instances in India. Data stays in India.
Ready-to-Use Security Prompts for Ollama
These prompts are tested and optimised for Llama 3.2 and Mistral 7B running locally via Ollama. Copy and use directly in ollama run llama3.2 or via API.
Log Analysis
You are a SOC analyst at an Indian bank. Analyse the following Windows Security Event Log entry and provide: (1) What happened in plain English, (2) Whether this is suspicious or normal, (3) MITRE ATT&CK technique if applicable, (4) Recommended next investigation step, (5) Whether this requires CERT-In reporting.\n\nLog entry:\n[PASTE LOG HERE]
Alert Triage
You are a tier-1 SOC analyst. Given this SIEM alert, determine: (1) True positive or false positive and why, (2) Severity (Critical/High/Medium/Low) with justification, (3) Immediate containment action if true positive, (4) Which Indian regulation requires reporting if any (CERT-In, RBI, SEBI).\n\nAlert:\n[PASTE ALERT HERE]
SIEM Rule Generation
You are a detection engineer. Write a KQL query for Microsoft Sentinel that detects the following attack technique. Include: field names from standard Microsoft table schemas, a comment explaining what it detects, MITRE ATT&CK ID, and 2-3 lines of tuning guidance.\n\nDetect: [DESCRIBE ATTACK IN PLAIN ENGLISH]
Threat Report Summary
Summarise the following threat intelligence report for an Indian CISO audience. Extract: (1) Threat actor name, (2) Indian sectors targeted, (3) Key TTPs as MITRE IDs, (4) IOCs (IPs, domains, hashes), (5) Recommended detections, (6) Executive summary in 3 sentences.\n\nReport:\n[PASTE REPORT TEXT HERE]
Incident Executive Brief
Convert this technical incident summary into a non-technical executive brief for the board. Include: what happened, what data or systems were affected, what actions have been taken, current status, and whether regulatory notification (CERT-In/RBI/SEBI) has been or will be made.\n\nTechnical summary:\n[PASTE INCIDENT DETAILS HERE]
Policy Draft
You are a GRC consultant familiar with Indian cybersecurity regulations. Draft a [POLICY TYPE] policy for a [ORG TYPE, e.g. "private sector bank regulated by RBI"]. The policy must align with CERT-In Directions 2022, RBI IT Framework, and be ready for board approval. Use formal language and include: purpose, scope, policy statements, roles and responsibilities, and review frequency.
Using Ollama API in Security Tools
Basic API call (Python)
import requests
import json
OLLAMA_URL = "http://localhost:11434/api/chat"
MODEL = "llama3.2"
def analyse_log(log_entry: str) -> str:
"""Send a log entry to local Ollama for security analysis."""
payload = {
"model": MODEL,
"messages": [
{
"role": "system",
"content": "You are a SOC analyst at an Indian regulated financial institution. Analyse security logs and map findings to CERT-In reporting requirements where applicable."
},
{
"role": "user",
"content": f"Analyse this log entry:\n{log_entry}"
}
],
"stream": False # Set True for streaming response
}
response = requests.post(OLLAMA_URL, json=payload)
result = response.json()
return result["message"]["content"]
# Usage
log = "Failed password for root from 203.0.113.45 port 52341 ssh2"
print(analyse_log(log))
Open WebUI — ChatGPT interface for your local models
## Install Open WebUI — browser-based ChatGPT-like interface for Ollama
## Serves all analysts on the LAN without installing anything on their machines
# Docker (easiest)
docker run -d -p 3000:8080 \
--add-host=host.docker.internal:host-gateway \
-v open-webui:/app/backend/data \
--name open-webui \
--restart always \
ghcr.io/open-webui/open-webui:main
# Access at http://YOUR_SERVER_IP:3000
# All analysts browse to this URL — no installation needed on analyst workstations
# All model calls stay on your server — nothing leaves your network
# To restrict access to SOC network only:
# iptables -A INPUT -p tcp --dport 3000 -s 10.0.0.0/24 -j ACCEPT
# iptables -A INPUT -p tcp --dport 3000 -j DROP
SIEM integration — auto-analyse alerts
## Splunk alert action: send alert to local Ollama for triage
## Add this as a custom alert action in Splunk
import splunk.Intersplunk as si
import requests
def enrich_with_ai(alert_data):
response = requests.post(
"http://ollama-server:11434/api/chat",
json={
"model": "llama3.2",
"messages": [{
"role": "user",
"content": f"Triage this SIEM alert. Is it a true positive? Severity? Action needed?\n\n{alert_data}"
}],
"stream": False
},
timeout=30
)
return response.json()["message"]["content"]
results, dummyresults, settings = si.getOrganizedResults()
for result in results:
result["ai_triage"] = enrich_with_ai(str(result))
si.outputResults(results)