Guard Tate — LLM Output Safety Layer

Challenge

As more internal teams adopted LLMs, prompt injection, internal-PII leakage, and unsafe-content generation became real risks. Each app reimplementing controls was inefficient and produced inconsistent policy. We also needed a single layer that abstracted away provider differences (OpenAI / Anthropic / internal NIM).

Solution

An OpenAI-compatible proxy runs DeBERTa-v3 injection classification + Presidio PII detection on the input side, and the same classifier plus OPA policy evaluation on the output side. Policies are written in Rego and applied dynamically per department, model, and tenant. Every request is traced with OpenTelemetry.

Results

• Prompt-injection detection F1 of 0.93 on the HarmBench subset
• PII false-positive rate 2.1% / false-negative rate 1.8%
• Proxy overhead p95 +38 ms
• Drop-in adoption across 4 internal apps — only base_URL had to change