Air-Gapped AI for Cybersecurity

Ollama is a single binary that downloads, manages, and serves LLMs via an OpenAI-compatible REST API. It is the fastest path from zero to a working local AI endpoint. Learn to deploy it in workstation, server, and air-gapped configurations.

• Install Ollama on Ubuntu 22.04: curl -fsSL https://ollama.com/install.sh | sh — understand what the installer does to your system
• Pull your first model: ollama pull mistral:7b — observe the download process and understand model storage location (~/.ollama/models/)
• Run your first security prompt: ollama run mistral "Analyse this Windows Event ID 4688 entry and tell me if it is suspicious: [paste a real event]"
• Understand Ollama's REST API: curl localhost:11434/api/generate — integrate with Python using the requests library
• Configure Ollama to bind to a network interface for team access: OLLAMA_HOST=0.0.0.0:11434 ollama serve
• Air-gap transfer procedure: ollama pull on internet machine → copy ~/.ollama/models/ to USB → deploy on air-gapped server → verify model integrity with sha256sum
• Install Open-WebUI for a browser-based chat interface: gives analysts a ChatGPT-like UI connected to your local Ollama instance
• Performance benchmarking: measure tokens/second for your hardware on different model sizes
• Configure Ollama as a systemd service for automatic startup after reboot
• Security hardening: restrict Ollama API access to internal network only using UFW rules

llama.cpp is the foundational inference engine — pure C++, runs on any hardware including CPU-only servers. Essential for OT environments, air-gapped systems without GPU approval, and maximum control over inference behaviour.

• Build llama.cpp from source on Ubuntu: git clone https://github.com/ggerganov/llama.cpp && cd llama.cpp && make -j$(nproc)
• Download a GGUF model from HuggingFace (on internet machine): TheBloke/Mistral-7B-Instruct-v0.2-GGUF — understand Q4_K_M vs Q5_K_M vs Q8_0 quality tiers
• Run inference in server mode: ./server -m models/mistral-7b-Q4_K_M.gguf --host 0.0.0.0 --port 8080 -c 4096 -t 8
• Understand key flags: -c (context size), -t (CPU threads — set to physical core count), --no-mmap (important for air-gapped systems), -ngl (GPU layers if GPU available)
• Test the OpenAI-compatible API endpoint: curl http://localhost:8080/v1/chat/completions -d '{"messages":[{"role":"user","content":"Analyse this alert..."}]}'
• Tune performance: experiment with thread counts, batch sizes, and context window sizes for your specific CPU
• Deploy as a background service with supervisor: keeps llama.cpp running on server restart
• GPU acceleration with llama.cpp: add -ngl 35 to offload layers to GPU — measure speedup
• Verify air-gap integrity: build llama.cpp binary on internet machine, transfer, verify SHA-256 on air-gapped server before use

LocalAI provides a complete OpenAI API drop-in replacement. Any tool built for the OpenAI API — LangChain, AutoGen, existing security automation scripts — works with LocalAI by changing one environment variable.

• Install LocalAI using Docker: docker run -p 8080:8080 -v /models:/models localai/localai:latest
• Configure LocalAI with your downloaded GGUF models: YAML model configuration files
• Test drop-in replacement: change OPENAI_BASE_URL to http://localhost:8080/v1 in any existing script — verify it works without any other code changes
• Understand LocalAI's additional capabilities beyond OpenAI compatibility: image generation, audio transcription, embeddings
• Deploy LocalAI in air-gapped mode: pull Docker image on internet machine, export with docker save, import on air-gapped server with docker load
• Configure model routing: serve multiple models simultaneously and route requests to appropriate model by model name
• Benchmark LocalAI vs Ollama for your specific use cases: choose the right tool for your environment

A local AI server that is accessible without authentication on your internal network is still a security risk. This module covers hardening the inference layer itself.

• Network access control: restrict inference API to specific internal subnets using UFW/iptables — analysts on VLAN can access, SCADA network cannot
• Nginx reverse proxy with API key authentication: add a simple token check in front of Ollama/llama.cpp for multi-user environments
• TLS termination: serve the API over HTTPS using internal CA certificates — prevent cleartext prompt data on the internal network
• Prompt logging: configure all incoming prompts to be logged to a centralised log (for SOC environments that need audit trails of AI usage)
• Model integrity verification: SHA-256 hash all model files at deployment, re-verify weekly — detect if model files are tampered with
• Access control for model management: prevent analysts from pulling new models without admin approval (critical for air-gapped environments)
• Resource limits: configure systemd resource limits to prevent a single runaway inference job from consuming all server resources
• Incident response plan for AI server compromise: what to do if your inference server is accessed by an attacker

Build a pipeline that takes raw SIEM alerts, sends them to your local LLM for triage, and returns an AI verdict directly in the alert. Analysts see pre-triaged alerts with reasoning — before they even open them.

• Design the alert enrichment architecture: SIEM → webhook/export → Python enrichment script → local Ollama API → enriched result back to SIEM
• Build alert exporter for Elastic SIEM: Python script that polls the Elastic alerts index every 5 minutes for new alerts
• Build alert exporter for Microsoft Sentinel: use the Sentinel REST API or Logic Apps to export new alerts to a processing queue
• Write the core enrichment prompt: given alert JSON, extract relevant fields, format into a structured prompt for the LLM
• Handle context injection: include relevant MITRE ATT&CK context, your organisation's asset inventory, and recent related alerts in the prompt
• Parse LLM response: extract structured verdict (true positive / false positive / needs investigation) and reasoning from model output
• Write enriched verdict back to SIEM: add a custom field "ai_verdict" and "ai_reasoning" to the alert in Elastic or Sentinel
• Build the Python service as a systemd daemon: runs continuously, processes new alerts in real time
• Error handling: graceful fallback if LLM is unavailable — alert passes through unmodified rather than being lost
• Performance measurement: measure analyst alert review time before and after AI pre-triage

Velociraptor queries return raw forensic data. Local AI transforms that raw data into actionable hunt hypotheses and investigation summaries — without any endpoint data leaving your network.

• Understand the Velociraptor REST API: authenticate and query hunt results programmatically using the pyvelociraptor client
• Build a VQL result analyser: execute a hunt, feed results to local LLM, receive a plain-English summary of findings
• Create a daily hunt hypothesis generator: feed your environment context and recent threat intel to the LLM, receive 5 novel hunt hypotheses
• Build a persistence finding classifier: Velociraptor PersistenceSniper results → LLM classification → suspicious vs legitimate per finding
• Create an IOC contextualiser: raw IOC (IP/domain/hash) from Velociraptor → LLM adds MITRE context, attack stage, and recommended containment
• Build a process tree analyser: Velociraptor process tree output → LLM narrative explaining the attack chain in plain English for executive reporting
• Automate: schedule daily hunt → LLM analysis → email summary to security team with no manual intervention

Combine your local AI with MISP to automatically enrich threat intelligence events, generate attribution hypotheses, and produce analyst-ready threat summaries from raw IOC dumps.

• Connect to MISP REST API: authenticate, list events, pull IOCs programmatically using pymisp
• Build an event enricher: new MISP event → local LLM generates a plain-English threat summary and attribution hypothesis
• Build an IOC contextualiser: given a new IOC, LLM checks it against known attack patterns and suggests relevant detection rules
• Auto-generate MISP event descriptions: raw IOC list → LLM writes a structured event description in MISP's standard format
• Build a threat report to IOC extractor: paste a PDF threat report text → LLM extracts all IOCs in MISP JSON format → auto-import to MISP
• Create campaign clustering: given multiple MISP events, LLM identifies potential campaign connections based on TTP overlap

Operational Technology environments have unique constraints — real-time requirements, legacy hardware, the Purdue Model network architecture, and the fundamental rule that security cannot disrupt the physical process. This module covers AI deployment within these constraints.

• Understand the Purdue Model and where AI inference fits: DMZ placement, one-way data flows, passive capture vs active response
• Deploy Phi-3 Mini (2.3GB, runs on standard workstation) in the IT/OT DMZ: small enough to run without dedicated GPU hardware
• Build a Zeek → AI alert triage pipeline for OT network traffic: classify each connection as legitimate industrial protocol vs anomalous
• Create ICS-CERT advisory processor: paste advisory text → LLM extracts affected products, CVEs, and compensating controls for environments that cannot patch
• Build a process anomaly explainer: historian data point + P&ID context → LLM explains what the anomaly means in process safety terms
• Generate OT-specific IR playbooks: given affected system type and process state → LLM produces a playbook that accounts for safety, continuity, and regulatory notifications (CERC, PNGRB)
• Test with Zeek ICSNPP plugins: parse Modbus, DNP3, EtherNet/IP protocol traffic and feed anomalies to the local AI for analysis

The quality of your local AI output is determined 70% by prompt quality and 30% by model quality. This module teaches security-specific prompt engineering techniques that dramatically improve accuracy on triage, analysis, and reporting tasks.

• Understand the three prompt components for security tasks: role (you are a SOC analyst), context (here is the environment), task (analyse this alert)
• Implement chain-of-thought prompting for complex analysis: "Think step by step: first identify the process tree, then check the network connection, then assess the registry modification"
• Build structured output prompts: instruct the model to always return JSON with specific fields — makes automation reliable
• Implement few-shot prompting: provide 3 examples of alert → correct verdict before asking the model to triage a new alert — improves accuracy by 20-40%
• Handle context window limits: chunking strategies for large log files that exceed 4K or 8K context windows
• Build a prompt library: standardised prompts for each security task your team performs — version-controlled in Git
• Test and compare prompts systematically: use a consistent set of 20 test cases to benchmark prompt variants
• Implement prompt injection prevention: sanitise alert data before inserting into prompts to prevent adversarial content in logs from manipulating LLM output

A generic model knows general security concepts. A fine-tuned model knows your organisation's specific alert patterns, your naming conventions, your infrastructure topology, and your response procedures. Fine-tuning takes 4-24 hours and dramatically improves task-specific performance.

• Understand LoRA (Low-Rank Adaptation): fine-tuning technique that trains only a small adapter — requires far less GPU memory than full fine-tuning
• Build a training dataset from your existing data: SIEM alert → analyst verdict pairs (minimum 500, ideal 2000+)
• Format training data as JSONL instruction-following format: {instruction: ..., input: alert_json, output: analyst_verdict}
• Set up Axolotl: the most practical LoRA fine-tuning framework — runs on a single RTX 4090 for 7B models
• Configure a fine-tuning run: model selection, LoRA rank, learning rate, batch size, epochs — understand what each parameter does
• Run a training job: monitor loss curves, identify overfitting, stop at the right checkpoint
• Evaluate your fine-tuned model: compare against base model on a held-out test set of 100 real alerts
• Merge LoRA weights into base model: create a standalone GGUF file deployable in Ollama or llama.cpp
• India-specific fine-tuning datasets: CERT-In advisories (all public, downloadable), RBI cyber incident reports, SEBI CSCRF guidance
• Governance: maintain a fine-tuning log documenting which datasets were used, when, and by whom — essential for AI governance compliance

Individual AI queries are useful. Pipelines that chain multiple AI calls with tool use, automated actions, and human-in-the-loop checkpoints are transformative. Build production-grade security automation that runs 24/7.

• Design a complete alert triage pipeline: ingest → deduplicate → enrich (threat intel) → LLM triage → auto-close false positives → escalate true positives
• Implement LangChain with local Ollama backend: build a chain that uses tools (WHOIS lookup, threat intel check, log query) before reaching a verdict
• Build a daily threat intelligence digest pipeline: fetch new feeds → extract IOCs → LLM summarises relevance to your environment → email to team
• Create an automated incident report drafting pipeline: analyst fills in key facts → pipeline generates all four required documents (technical, executive, CERT-In, customer)
• Implement human-in-the-loop: any AI verdict with confidence below 70% is routed to human review rather than auto-actioned
• Error handling and retry logic: production pipelines must handle LLM failures, API timeouts, and malformed outputs gracefully
• Pipeline monitoring: log every pipeline execution, measure accuracy over time, alert when model performance degrades

When your SOC team of 50 analysts all need simultaneous access to the AI assistant, Ollama's sequential request handling becomes a bottleneck. vLLM uses PagedAttention to serve multiple concurrent requests from a single GPU with dramatically higher throughput.

• Understand vLLM's PagedAttention: why it enables much higher concurrent request throughput vs naive KV cache management
• Install vLLM: pip install vllm — requires CUDA GPU, minimum 24GB VRAM for 7B models
• Launch vLLM with OpenAI-compatible API: python -m vllm.entrypoints.openai.api_server --model /models/llama-3.1-8b-instruct
• Configure tensor parallelism for multi-GPU: --tensor-parallel-size 2 to spread a 70B model across two A100s
• Set up nginx load balancer in front of vLLM for SSL termination and request routing
• Performance testing: measure requests/minute, average latency, P95 latency under load with 10, 25, 50 concurrent users
• Configure vLLM request queuing: prevent resource exhaustion when all 50 analysts submit simultaneously
• Air-gap deployment: vLLM model loading from local filesystem, no HuggingFace downloads at startup

Production AI systems require governance — tracking which models are deployed, who approved them, what data they were trained on, and how their performance changes over time. This is also where India's emerging AI governance framework applies.

• Build a model registry: document each model deployed — name, version, training data sources, approval date, approved use cases, performance benchmarks
• Implement model performance monitoring: track accuracy of AI verdicts over time, alert when false positive rate exceeds threshold
• Establish a model approval process: security review of new models before deployment, testing on a representative dataset
• Understand India AI governance requirements: MeitY's Digital India Act provisions for AI, SEBI guidance on AI use in financial services
• Build an AI audit log: every prompt sent to the model and every response received, stored on-premises for compliance evidence
• Model update procedure: how to safely update to a new model version without disrupting active analyst workflows
• Incident response for AI system compromise: detection, containment, and recovery if your inference server is breached
• DPDP Act compliance: verify that your AI processing of security logs containing personal data complies with lawful basis requirements

The best AI deployment fails if analysts do not use it effectively. This module covers training your SOC team to work with AI tools productively — including understanding AI limitations, avoiding over-reliance, and building effective workflows.

• Develop an analyst onboarding programme: 4-hour training session covering what the AI can and cannot do, how to interpret AI verdicts, and when to override
• Build prompt templates for each analyst role: L1 triage prompts, L2 investigation prompts, L3 hunting prompts — standardise effective usage
• Establish AI usage guidelines: when AI verdicts can be acted upon directly, when human review is mandatory regardless of AI confidence
• Create a feedback loop: analysts flag incorrect AI verdicts, flagged cases become new training data for the next fine-tuning run
• Measure adoption and effectiveness: analyst satisfaction scores, alerts processed per hour, MTTR before and after AI deployment
• Address AI over-reliance: training programme component specifically on the risks of trusting AI output without verification
• Document lessons learned: what worked, what did not, what you would do differently — share with the security community
• Final capstone: present your complete deployment to stakeholders — technical documentation, ROI analysis, governance documentation, and future roadmap