This is a writeup of a three-month engagement helping a dental practice evaluate AI tools for patient intake processing. The practice wanted to use GPT-4 to extract structured data from scanned intake forms, classify dental X-rays for preliminary triage, and generate visit summaries from provider notes. Simple use cases - applications a competent developer could build in a week. The engagement took three months because every cloud AI option ran into the same wall: HIPAA compliance is not a checkbox. It is a constraint on where patient data can exist, who can access it, and what happens when something goes wrong. After evaluating every major cloud AI provider's BAA terms, the practice built the entire system on-premise with Ollama on a local server. Here is why, and exactly how.
Why Cloud AI and HIPAA Do Not Mix (Despite What Sales Teams Say)
The specifics of what HIPAA requires and where cloud AI falls short are where most writeups get vague.
HIPAA's Security Rule requires covered entities and their business associates to implement administrative, physical, and technical safeguards for electronic Protected Health Information (ePHI). When you send patient data to a cloud AI API, you are creating ePHI in a system you do not control. This triggers several requirements that cloud AI providers handle poorly or not at all:
The Business Associate Agreement Problem
OpenAI, Google, Microsoft, and Amazon all offer BAAs for their AI services. Read closely, here is what they actually cover and what they do not:
| Provider | BAA Available | Covers AI/ML APIs | Covers Image Data | Data Retention Control | Audit Log Access |
|---|---|---|---|---|---|
| OpenAI (API) | Yes (Enterprise) | Yes (text only) | Ambiguous | 30-day default | Limited |
| Google Cloud | Yes | Vertex AI only | Yes (Vertex) | Configurable | Yes (Cloud Audit) |
| Microsoft Azure | Yes | Azure OpenAI | Yes | Configurable | Yes |
| Amazon (Bedrock) | Yes | Yes | Yes | Configurable | Yes (CloudTrail) |
| Anthropic (API) | Yes (Enterprise) | Text only | Ambiguous | 30-day default | Limited |
The word "ambiguous" is the warning sign. Asked whether their BAA explicitly covers medical images (dental X-rays, radiology scans) sent via the Vision API, OpenAI's enterprise sales team answered "the BAA covers data processed through our API endpoints." That is not a yes. It is a lawyer-crafted non-answer that leaves the covered entity holding the liability if HHS comes knocking.
The Data Transit Problem
When a dental X-ray goes to GPT-4V for analysis, that image:
- Leaves your network over TLS
- Hits a cloud load balancer
- Gets routed to a processing server (location unknown to you)
- Gets decoded, processed by the model, and a response generated
- The image sits in server memory (and possibly disk cache) for some period
- May be logged for abuse monitoring (most providers retain request metadata)
- Response traverses back to you
At every step, ePHI exists on infrastructure the covered entity does not own, cannot audit, and cannot guarantee meets its specific HIPAA implementation requirements. The HIPAA Security Rule (45 CFR 164.312) requires access controls, audit controls, integrity controls, and transmission security for ePHI. With cloud AI, all of these are delegated to the provider, on the assumption that their implementation matches the covered entity's compliance requirements.
The Breach Notification Problem
This is what keeps compliance officers up at night: if a cloud AI provider has a breach that exposes patient data sent to it, the covered entity is responsible for breach notification under HIPAA - notifying affected patients within 60 days. But the provider's BAA typically gives them 60 days to notify the covered entity. In the worst case, the breach surfaces on day 59, leaving one day to identify affected patients, draft notification letters, and report to HHS. In practice, cloud providers have been faster than this. But the contractual obligation is what matters for risk assessment.
The On-Premise Alternative: Architecture
The system the practice built instead runs entirely on local hardware inside its existing server closet, on an isolated network segment with zero internet connectivity.
┌─────────────────────────────────────────────────────┐
│ PRACTICE NETWORK │
│ │
│ ┌──────────┐ ┌──────────┐ ┌───────────────┐ │
│ │ Scanners │ │ X-Ray │ │ Practice Mgmt │ │
│ │ (intake │ │ Machines │ │ Software │ │
│ │ forms) │ │ (DICOM) │ │ (Dentrix) │ │
│ └────┬─────┘ └────┬─────┘ └───────┬───────┘ │
│ │ │ │ │
│ └───────┬───────┘──────────────────┘ │
│ │ │
│ ┌──────▼──────┐ │
│ │ AI SERVER │ ← Air-gapped subnet │
│ │ (Ollama) │ 10.0.50.0/24 │
│ │ │ NO internet gateway │
│ │ - LLaVA │ │
│ │ - Llama 3 │ │
│ │ - Mistral │ │
│ └──────┬──────┘ │
│ │ │
│ ┌──────▼──────┐ │
│ │ PostgreSQL │ ← Encrypted at rest │
│ │ (results) │ Full audit logging │
│ └─────────────┘ │
│ │
│ ════════════ NO CONNECTION TO INTERNET ════════════ │
└─────────────────────────────────────────────────────┘
Hardware Requirements and Setup
Healthcare AI workloads are not that demanding for inference. There's no model training involved - just pre-trained models running on individual documents and images. The spec:
| Component | Spec | Cost | Why |
|---|---|---|---|
| GPU | NVIDIA RTX 4060 Ti (16GB) | $399 | 16GB VRAM for larger medical models |
| CPU | Intel Core i5-13400 | $199 | Reliable, ECC-compatible platform |
| RAM | 64GB DDR5 ECC | $249 | ECC for data integrity (HIPAA best practice) |
| Storage | 2TB NVMe SSD (OPAL 2.0) | $159 | Hardware encryption support |
| UPS | CyberPower 1500VA | $179 | Clean shutdown on power loss |
| Case + PSU + Mobo | Mid-tower build | $329 | Quiet, good airflow |
| Total | $1,514 |
A few notes on these choices. ECC RAM is not strictly required by HIPAA, but it prevents bit-flip errors that could corrupt patient data during processing. A single corrupted byte in an extracted patient name or medication list is worse than a crash. The OPAL 2.0 SSD provides hardware-level full-disk encryption that meets HIPAA's encryption requirements without the performance penalty of software encryption. The UPS ensures clean shutdowns - an unclean shutdown during write operations could corrupt the PostgreSQL database containing patient data.
Air-Gapped Network Configuration
The AI server sits on an isolated VLAN with no route to the internet. This is the single most important security control and what makes the architecture different in kind from cloud AI.
# /etc/netplan/01-aiserver.yaml
network:
version: 2
ethernets:
enp0s31f6:
addresses:
- 10.0.50.10/24
# NO gateway4 - no route to internet
# NO nameservers - no DNS resolution
routes:
- to: 10.0.50.0/24
via: 10.0.50.1
# Only route is to the practice subnet
# Firewall rules - belt and suspenders
sudo ufw default deny incoming
sudo ufw default deny outgoing # Critical: deny ALL outbound
sudo ufw allow from 10.0.50.0/24 to any port 11434 # Ollama API
sudo ufw allow from 10.0.50.0/24 to any port 5432 # PostgreSQL
sudo ufw allow from 10.0.50.5 to any port 22 # SSH from admin workstation only
sudo ufw enable
# Verify no internet connectivity
ping -c 3 8.8.8.8
# ping: connect: Network is unreachable ← This is what you want to see
The outbound deny rule is critical. Even if malware reached the AI server, it cannot exfiltrate data because there is physically no route to the internet. The only network paths available are to other devices on the 10.0.50.0/24 subnet. This is defense in depth that no cloud solution can match.
Ollama Configuration for Air-Gapped Operation
Ollama normally checks for model updates and phones home for telemetry. An air-gapped environment requires disabling this and pre-loading all models before disconnecting from the internet.
Initial Model Loading (Before Air-Gap)
# Temporarily connect to internet for initial setup ONLY
# Download models to local storage
ollama pull llama3.1:8b-instruct-q4_K_M # Document analysis
ollama pull llava:13b-v1.6-q4_K_M # X-ray and image analysis
ollama pull mistral:7b-instruct-q4_K_M # Fast classification tasks
ollama pull medllama2:7b # Medical-specific model (if available)
ollama pull nomic-embed-text # Embeddings for similarity search
# Verify all models are cached locally
ollama list
ls -la ~/.ollama/models/
# Now disconnect from internet permanently
# Remove the gateway, apply netplan, enable firewall
Ollama Service Configuration
# /etc/systemd/system/ollama.service.d/override.conf
[Service]
Environment="OLLAMA_HOST=10.0.50.10:11434"
Environment="OLLAMA_KEEP_ALIVE=24h"
Environment="OLLAMA_NUM_PARALLEL=2"
Environment="OLLAMA_MAX_LOADED_MODELS=2"
# Disable any update checks or telemetry
Environment="OLLAMA_NOPRUNE=1"
Environment="OLLAMA_DEBUG=0"
# Apply and restart
sudo systemctl daemon-reload
sudo systemctl restart ollama
sudo systemctl enable ollama
# Verify it is listening only on the internal interface
ss -tlnp | grep 11434
# LISTEN 0 4096 10.0.50.10:11434 *:* users:(("ollama",pid=1234,fd=3))
Use Case 1: Patient Intake Form Processing
Dental practices still deal with mountains of paper. New patient intake forms get scanned to PDF, but extracting structured data from them - patient name, DOB, insurance info, medical history, medications, allergies - is done manually by front desk staff. It takes 8-12 minutes per form. With 15-20 new patients per week, that is 3-4 hours of staff time on data entry alone.
"""
Extract structured patient data from scanned intake forms.
Uses LLaVA for OCR + structured extraction in one pass.
"""
import requests
import base64
import json
OLLAMA_URL = "http://10.0.50.10:11434/api/generate"
INTAKE_PROMPT = """This is a scanned dental patient intake form. Extract the following
information and return as JSON. If a field is not visible or illegible, use null.
Do not guess or infer missing information.
{
"patient_name": {"first": "", "last": ""},
"date_of_birth": "YYYY-MM-DD",
"phone": "",
"email": "",
"address": {"street": "", "city": "", "state": "", "zip": ""},
"insurance": {"provider": "", "member_id": "", "group_number": ""},
"emergency_contact": {"name": "", "phone": "", "relationship": ""},
"medical_conditions": ["list of checked conditions"],
"current_medications": ["list of medications"],
"allergies": ["list of allergies"],
"last_dental_visit": "YYYY-MM-DD or approximate",
"chief_complaint": "reason for visit"
}
Return ONLY valid JSON. Mark illegible fields as null, never guess."""
def process_intake_form(image_path: str) -> dict:
with open(image_path, "rb") as f:
img_b64 = base64.b64encode(f.read()).decode()
resp = requests.post(OLLAMA_URL, json={
"model": "llava:13b-v1.6-q4_K_M",
"prompt": INTAKE_PROMPT,
"images": [img_b64],
"stream": False,
"options": {"temperature": 0.0, "num_predict": 1024}
}, timeout=180)
raw = resp.json()["response"]
raw = raw.strip().removeprefix("```json").removeprefix("```").removesuffix("```").strip()
return json.loads(raw)
In testing with 50 scanned intake forms, LLaVA extracted patient demographics with 91% field-level accuracy. The main failure modes were handwriting recognition (some patients have truly illegible handwriting) and checkbox detection (small checkboxes on medical history sections). A mandatory human review step handles this - the extracted data populates the practice management form, and front desk staff verify and correct rather than type from scratch. This reduced per-form processing time from 8-12 minutes to 2-3 minutes.
Use Case 2: Dental X-Ray Preliminary Classification
One point needs to be unambiguous: AI classification of dental X-rays is not a diagnostic tool. It is not FDA-approved for diagnosis. It does not replace a dentist's clinical judgment. What it does is pre-sort X-rays for the dentist's review queue and flag images that need urgent attention.
The practice takes 200-300 X-rays per week. The dentist reviews each one, but some are routine (periodic bitewings for patients with no history of issues) and some need immediate attention (potential fractures, advanced decay, periapical pathology). AI pre-classification helps the dentist prioritize the review queue.
XRAY_PROMPT = """Analyze this dental X-ray image. Classify the findings into
one of these priority levels and identify any visible conditions.
This is for TRIAGE PURPOSES ONLY, not diagnosis. A dentist will review all images.
Return JSON:
{
"image_type": "bitewing|periapical|panoramic|cephalometric",
"priority": "routine|attention|urgent",
"observations": ["list of visible findings"],
"areas_of_concern": ["specific teeth or regions to examine closely"],
"quality": "diagnostic|acceptable|retake_recommended",
"notes": "any additional observations"
}
Priority guidelines:
- routine: normal anatomy, no obvious pathology
- attention: possible decay, calculus, minor bone loss
- urgent: possible fracture, severe bone loss, periapical radiolucency, suspected pathology
Return ONLY valid JSON."""
In testing with a board-certified dentist reviewing AI classifications against their own assessments on 200 X-rays:
| Metric | Result |
|---|---|
| Priority classification accuracy | 83.5% |
| False negatives (missed urgent cases) | 2.0% (4 out of 200) |
| False positives (flagged routine as urgent) | 8.5% (17 out of 200) |
| Image type detection | 97.0% |
| Quality assessment | 89.5% |
The 2% false negative rate is why this is a triage tool, not a diagnostic tool. Those 4 missed cases were subtle periapical radiolucencies that even experienced dentists sometimes debate. The 8.5% false positive rate is acceptable - the dentist looks at some routine X-rays sooner than necessary, but no urgent cases get deprioritized. The system is tuned to be conservative: if in doubt, classify as "attention" rather than "routine."
Use Case 3: Visit Summary Generation
After each patient visit, providers need to document what happened. Many still dictate notes or type them manually. In this system the hygienist or dentist speaks a brief summary, it gets transcribed (locally using Whisper), and then Llama 3 generates a structured clinical note.
CLINICAL_NOTE_PROMPT = """Convert this dictated visit summary into a structured
dental clinical note. Use standard dental terminology. Include all mentioned
findings, procedures, and recommendations. Do not add information that was not
mentioned in the dictation.
Dictation: {transcription}
Return the note in this structure:
- Chief Complaint
- Clinical Findings
- Procedures Performed (with ADA codes if identifiable)
- Recommendations
- Follow-Up Plan
Use professional clinical language appropriate for a dental record."""
This saves providers 5-8 minutes per patient on documentation. For a practice seeing 30 patients per day, that is 2.5-4 hours of provider time reclaimed daily. At a dentist's hourly rate, the ROI on the $1,514 server is measured in days, not months.
Audit Logging: HIPAA's Most Overlooked Requirement
HIPAA requires audit controls that record and examine activity in systems containing ePHI. In this AI system, every inference request gets logged with who requested it, what data was processed, and what the system returned:
import logging
import hashlib
from datetime import datetime
# Dedicated HIPAA audit logger - writes to append-only log file
audit_logger = logging.getLogger("hipaa_audit")
audit_handler = logging.FileHandler("/var/log/ai-audit/hipaa_audit.log")
audit_handler.setFormatter(logging.Formatter(
'%(asctime)s|%(message)s'
))
audit_logger.addHandler(audit_handler)
audit_logger.setLevel(logging.INFO)
def log_ai_access(user_id: str, patient_id: str, action: str,
data_type: str, model: str, success: bool):
"""Log every AI inference involving patient data."""
audit_logger.info("|".join([
user_id,
hashlib.sha256(patient_id.encode()).hexdigest()[:16], # Pseudonymized
action, # "intake_extraction", "xray_classification", "note_generation"
data_type, # "image", "text", "document"
model,
str(success),
datetime.utcnow().isoformat()
]))
# Usage in the classification pipeline:
log_ai_access(
user_id="hygienist_jsmith",
patient_id="PAT-2024-0847",
action="xray_classification",
data_type="image",
model="llava:13b-v1.6-q4_K_M",
success=True
)
# Audit log rotation and retention (HIPAA requires 6-year minimum)
# /etc/logrotate.d/hipaa-audit
/var/log/ai-audit/hipaa_audit.log {
daily
rotate 2190 # 6 years of daily logs
compress
delaycompress
notifempty
create 0600 aiservice aiservice
postrotate
# Calculate SHA-256 hash of rotated log for integrity verification
sha256sum /var/log/ai-audit/hipaa_audit.log.1 >> /var/log/ai-audit/integrity_hashes.log
endscript
}
Security Hardening Checklist
Beyond the air-gapped network, every security measure on the AI server:
# 1. Full disk encryption (LUKS on top of OPAL)
sudo cryptsetup luksFormat /dev/nvme0n1p3
sudo cryptsetup luksOpen /dev/nvme0n1p3 ai_data
# 2. Disable USB storage (prevent data exfiltration via thumb drive)
echo "blacklist usb-storage" | sudo tee /etc/modprobe.d/disable-usb-storage.conf
sudo update-initramfs -u
# 3. Mandatory access controls with AppArmor
sudo aa-enforce /etc/apparmor.d/usr.bin.ollama
# 4. SSH hardening - key-based auth only, from admin workstation only
# /etc/ssh/sshd_config
# PasswordAuthentication no
# PermitRootLogin no
# AllowUsers aiservice@10.0.50.5
# MaxAuthTries 3
# 5. Automatic security updates (applied from local mirror)
sudo apt install unattended-upgrades
# Configure to pull from local apt mirror on practice network
# 6. File integrity monitoring
sudo apt install aide
sudo aideinit
# Weekly integrity check via cron
0 3 * * 0 root /usr/bin/aide --check >> /var/log/aide/weekly_check.log 2>&1
# 7. PostgreSQL encryption at rest
# postgresql.conf
# ssl = on
# ssl_cert_file = '/etc/ssl/certs/ai_server.crt'
# ssl_key_file = '/etc/ssl/private/ai_server.key'
Model Selection for Healthcare
Not every open-source model is appropriate for healthcare applications. The evaluation:
| Model | Use Case | VRAM Required | Healthcare Suitability |
|---|---|---|---|
| LLaVA 1.6 13B (Q4) | X-ray classification, form OCR | 8-10 GB | Good for triage, not diagnosis |
| Llama 3.1 8B Instruct (Q4) | Clinical note generation | 5 GB | Good for structured text generation |
| Mistral 7B Instruct (Q4) | Fast classification, entity extraction | 4.5 GB | Good for quick categorization tasks |
| BioMistral 7B | Biomedical text understanding | 4.5 GB | Trained on PubMed, better medical terminology |
| Whisper Large v3 | Dictation transcription | 3 GB | Excellent for medical dictation |
One important note: the build deliberately avoided models fine-tuned on patient data from other healthcare systems. The provenance of training data matters for HIPAA compliance. If a model was trained on ePHI without proper authorization, using it in a clinical setting creates legal exposure. The selection sticks to models trained on publicly available data - PubMed articles, medical textbooks, publicly released imaging datasets. Not patient records.
What This Actually Costs vs. HIPAA-Compliant Cloud
The real comparison is not on-premise vs. regular cloud AI. It is on-premise vs. HIPAA-compliant cloud AI, a very different price tier:
| Cost Category | On-Premise (Our Setup) | HIPAA Cloud (Azure/AWS) |
|---|---|---|
| Initial hardware/setup | $1,514 | $0 |
| Monthly compute | $45 (electricity) | $800-2,400 (HIPAA-tier VMs) |
| Monthly AI inference | $0 | $500-3,000 (usage dependent) |
| BAA and compliance overhead | $0 | $200-500/mo (compliance tooling) |
| Security monitoring | $0 (self-hosted) | $150-400/mo (cloud security services) |
| Annual compliance audit prep | ~8 hours internal time | ~20 hours + cloud documentation |
| Monthly total | ~$87 | $1,650-6,300 |
HIPAA-compliant cloud infrastructure is expensive because it requires dedicated instances (not shared), encryption at rest and in transit (which rules out cheaper tiers), comprehensive audit logging (which costs extra on every cloud provider), and the compliance overhead of managing BAAs, conducting vendor risk assessments, and documenting data flows through third-party infrastructure.
On-premise eliminates almost all of this. The data never leaves the building. There is no third-party vendor to assess. There is no BAA to negotiate. The audit trail is on a physically controlled server. The compliance story is simple: "Patient data is processed on locally owned hardware on an air-gapped network. It never touches the internet."
Limitations and Honest Caveats
The approach has real limitations:
- Model quality is lower than frontier models. GPT-4V is better at X-ray analysis than LLaVA 13B. That is an objective fact. For triage and preliminary classification, LLaVA is sufficient. For anything approaching diagnostic assistance, better models are required - and those better models only exist in the cloud today.
- No FDA approval. None of these models, cloud or local, are FDA-approved for diagnostic use. Using AI-generated classifications in clinical decision-making without appropriate clinical oversight is both a legal and ethical problem. This system is explicitly labeled as a triage tool that assists provider workflow, not a diagnostic tool.
- Hardware maintenance is the practice's responsibility. When the GPU fails at 3 PM on a Friday, there is no cloud provider to call. A spare GPU on the shelf or a plan B is required - the practice keeps a spare RTX 4060 Ti in storage ($399 insurance policy).
- Model updates require temporary internet access. New model versions require downloading weights from the internet. This happens quarterly: temporarily connect the server to a separate VLAN with internet access, pull updates, verify checksums, disconnect. It is a 30-minute process with documentation for audit purposes.
- Staff training matters. The AI system is only as good as the staff's understanding of its limitations. A full day of training for front desk staff and clinical providers covered what the AI does, what it does not do, when to trust it, and when to override it.
Getting Started: The Minimum Viable HIPAA AI Setup
For a healthcare team exploring on-premise AI, the simplest starting point:
# 1. Build or buy a server with a 16GB+ GPU ($1,500-2,500)
# 2. Install Ubuntu 22.04 LTS Server
# 3. Install Ollama
curl -fsSL https://ollama.com/install.sh | sh
# 4. Pull models while connected to internet
ollama pull llama3.1:8b-instruct-q4_K_M
ollama pull llava:13b-v1.6-q4_K_M
# 5. Test with non-PHI data to validate your use case
# Use sample images, synthetic patient data, test documents
# 6. Once validated, configure air-gapped network
# Remove internet gateway, configure firewall, test isolation
# 7. Implement audit logging BEFORE processing any real patient data
# 8. Document everything for your HIPAA Security Risk Assessment
# 9. Get your compliance officer's sign-off
# 10. Start with a pilot — one use case, one provider, limited scope
Start small. Prove value with one use case. Then expand. The $1,514 server handles all three use cases described here for a practice seeing 30 patients per day. The ROI comes from time savings - minutes per patient, hours per week, days per year - that let providers focus on patient care instead of paperwork.
The underlying question is whether patient data is processed on hardware the practice owns and controls, or on hardware owned by a third party under terms that change annually. The technology to run AI locally is mature, affordable, and accessible. The remaining obstacle is the institutional default that "cloud is always better." For HIPAA-regulated data, that default does not hold.




Leave a Comment