Zero trust for an AI pipeline, in plain terms

The idea in one line

Zero trust means no component is trusted just because of where it sits. Every request is checked. For AI features, the component that's easiest to forget in that rule is the model.

Three ways it goes wrong

• Text as an attack. A model reads whatever you give it: emails, web pages, uploaded files. Someone can hide instructions in that text. This is called prompt injection, and no prompt is a reliable shield against it.
• Tools that do too much. If the model can send email, issue refunds or run queries, then fooling the model means getting those things done.
• Data in the wrong places. Prompts, responses and logs often contain customer information. They get stored, sent to vendors and read by support staff.

What containment looks like

• Least privilege for tools. Give each agent only the actions it needs, scoped to the user it's acting for, not your whole system.
• Approval for irreversible actions. Payments, deletions, outbound messages. A person confirms, and sees what is about to happen.
• Separate reading from acting. An agent that reads untrusted content shouldn't also hold the keys to sensitive actions.
• Log what the model saw and did. Redact what you must, but keep enough to reconstruct an incident.
• Short-lived credentials. Nothing the model touches should hold a long-lived secret.

What to do this month

1. List every tool each AI feature can call. Cross out anything it doesn't need.
2. Mark the irreversible ones. Add a human confirmation step.
3. Trace where sensitive data goes. Prompts, logs, vendor retention, analytics.
4. Test with hostile input. Hide an instruction in a document and see what your feature does with it.
5. Write down who can turn it off. And how quickly.