AI Engineering Risk & Guardrails Memo (Single Page)
Title: AI Engineering Risk & Guardrails Memo
Date: July 1, 2026
Purpose: Define practical controls for shipping safe, reliable AI features.
Top Risks
1. Hallucinations: Confident but incorrect output
2. Prompt Injection: Malicious instructions hidden in user or retrieved
content
3. Data Leakage: Sensitive data exposed in prompts, logs, or outputs
4. Automation Errors: Agents taking unintended actions
5. Model Drift: Performance degrades after data or model changes
Guardrail Strategy (Defense in Depth)
Input Controls
Validate and sanitize user input
Detect suspicious instructions and adversarial patterns
Apply domain constraints before model call
Context Controls
Restrict retrieval sources to trusted corpora
Tag data sensitivity and enforce policy-based filtering
Exclude unverified content for high-stakes responses
Generation Controls
Enforce structured outputs where possible (JSON/schema)
Require citations for factual claims in knowledge tasks
Add refusal rules for unsafe or unauthorized requests
Action Controls (Tools/Agents)
Principle of least privilege for tools
Human confirmation for destructive or financial actions
Idempotency keys and audit trails for side-effecting operations
Monitoring Controls
Track hallucination rate, policy violations, and incident count
Sample outputs for manual review
Alert on latency spikes, cost anomalies, and error surges
Release Policy
Launch in phases (internal → limited beta → general)
Maintain rollback path for model/prompt/retriever versions
Re-evaluate after every significant model or data update
Safe AI systems are built through layered controls, continuous testing, and
operational discipline—not one-time prompt edits.