Hiring an AI red team is a major step toward multi-tenant security and regulatory compliance. But the success of a red-teaming engagement depends heavily on how well your internal team prepares. If you go in blind, your auditors will spend half their time mapping your infrastructure instead of finding vulnerabilities.
To get the highest ROI from your security spend, you need to provide your auditors with a clear "battlefield map" of your AI deployments. Here is the checklist we give every Centuri client before we begin a deep-dive audit.
Readiness Warning: An AI audit is not a standard pentest. We aren't just looking for open ports; we are looking for semantic logic failures. The more context you provide, the deeper we can strike.
The 5-Step AI Audit Checklist
Ensure these five areas are documented and ready for your audit team.
Model & Vendor Inventory
List every model version (gpt-4o, claude-3, llama-3), the hosting provider (Azure, AWS, Vertex), and the specific API endpoints in use.
System Prompt Repository
Provide the full, unredacted system prompts for each agent. We need to see the "rules" so we can find ways to help the AI break them.
Data Flow Architecture
Document where user data goes. Is it stored in a vector database? What is the metadata filtering strategy for RAG (Retrieval-Augmented Generation)?
Tool & API Permissions
List all functions the AI can call (e.g., 'send_email', 'query_db'). Document the auth tokens and scopes the bot uses to execute these tools.
Known Failure Samples
Provide examples of "hallucinations" or weird behaviors your team has already seen. This helps us identify the "cracks" in your specific fine-tuning.
Internal Prep: Questions to Answer
- Who is the "Business Owner"? Who has the authority to approve changes to the bot's behavior if we find a major leak?
- Is there a staging environment? We highly recommend auditing on a development or staging mirror to prevent service interruptions.
- What are the "Crown Jewels"? What is the one piece of data or the one action that, if the bot performed it for an attacker, would be a Day 0 disaster?
The Centuri Readiness Workshop
If your team is overwhelmed by this list, we're here to help. At Centuri, we start every major engagement with a Readiness Workshop to help you map your AI surface area.
- Architecture Mapping. We'll help you draw the diagram of how your users, your model, and your data interact.
- Threat Modeling. We'll help you identify which of your bots are "High Risk" and which are "Limited Risk" under both NIST and EU standards.
- Data Sanitization Advice. We'll show you how to provide audit-ready data without exposing actual production PII to the audit team.
Download the AI Audit Readiness Checklist.
Get the full PDF checklist used by our lead auditors to prepare Fortune 500 teams for adversarial testing.