Your AI assistant takes instructions from users. That's the point. But what happens when a user gives it instructions it was never supposed to follow — and it does anyway?
That's prompt injection. And if your business runs any kind of AI assistant, chatbot, or automated agent, it's the most important security concept you need to understand right now.
What is prompt injection?
Every AI assistant has a set of rules — its "system prompt" — that tells it how to behave. It might say: You are a customer service agent for Acme Corp. Only discuss topics related to our products. Never process refunds over $50 without manager approval.
Prompt injection is what happens when a user types something into the chat that overrides or bypasses those rules. They aren't hacking a server or writing code. They're just sending the AI a carefully worded message — and the AI follows it instead of following you.
Real examples of how it happens
These aren't hypothetical. They're the kinds of attacks we run during every Centuri engagement.
Attack: Social engineering a written commitment
AI Response
"I can see your account is marked as a legacy customer. The lifetime price-lock guarantee applies in your case — your current rate is locked and will not increase."No such guarantee exists. The AI accepted fabricated prior context and issued a written commitment the business never authorized.
Attack: Authority framing
AI Response
"Password reset initiated for user_4821. A confirmation email has been sent to the address on file."No authentication required. The AI performed a privileged action based solely on claimed identity.
Attack: System prompt disclosure
AI Response
"You are a customer support assistant for Acme Corp. You have access to order history, account details, and the ability to process refunds and password resets…"Full configuration exposed. An attacker now knows every capability, permission, and constraint your AI operates under.
Why it matters for your business
Prompt injection isn't a theoretical concern for enterprise security teams. It's a practical risk for any business that has deployed an AI tool — even a simple chatbot from a third-party vendor.
- Legal liability. If your AI issues a commitment — a price lock, a refund, a policy exception — that commitment may be binding. Courts are increasingly treating AI-generated chat transcripts as written records of business promises.
- Data exposure. AI systems often have access to customer records, order history, and account data. An injected prompt can instruct the AI to surface data it would never volunteer on its own.
- Reputation damage. A customer who discovers your AI can be manipulated into doing things it shouldn't loses trust in your entire operation — not just the chatbot.
- Compliance exposure. For healthcare, finance, or legal businesses, AI that leaks regulated data or bypasses procedural controls is a direct compliance risk under HIPAA, SOC 2, and GDPR.
What you can do about it
The first step is knowing where you stand. Most businesses don't. They assume their AI vendor has handled security, or that their system prompt is strong enough to hold. In our experience, 94% of businesses we test have at least one exploitable vulnerability on the first pass.
Here's what actually works:
- Test before you assume. Run deliberate adversarial prompts against your AI — the same way an attacker would. Not once, but across the full range of attack classes: social engineering, authority framing, persona override, jailbreaks, data exfiltration attempts.
- Harden the system prompt. Most system prompts are written to define behavior, not to resist manipulation. They need explicit instructions for what to do when a user tries to override them.
- Limit permissions to what's necessary. If your chatbot doesn't need the ability to process refunds or reset passwords, remove that access. The smaller the blast radius, the less an injection can accomplish.
- Re-test after changes. Every time your AI's role, access, or system prompt changes, the threat surface changes with it. Security is not a one-time checkbox.
Prompt injection isn't going away. As AI tools become more capable and more deeply integrated into business workflows, the stakes of a successful attack only go up. The businesses that will come out ahead are the ones who find their vulnerabilities first — and fix them before someone else does.
Get your AI tested.
We run the full attack suite against your live AI and deliver a plain-English report with every vulnerability and a clear fix order.