--- title: "(Beta) AI Security Testing" slug: "ai-security-testing" description: "Learn how to use AI Security Testing in Traceable to discover, test, and secure AI and LLM endpoints using OWASP Top 10 for LLM risks. Configure AI scans with XAST or DAST, run attack simulations, review prompt-response evidence, and prioritize remediation with actionable AI-generated insights." tags: ["AI Security", "AI Security Testing", "API Security Scans", "API Security Testing", "OWASP Top 10"] updated: 2026-04-17T08:16:49Z published: 2026-04-17T08:16:49Z --- > ## Documentation Index > Fetch the complete documentation index at: https://traceabledocs.document360.io/llms.txt > Use this file to discover all available pages before exploring further. # (Beta) AI Security Testing **AI Security Testing** is Traceable enables you to automatically discover, test, and secure AI-powered APIs and large language model (LLM) endpoints. The built-in **AI Scan Policy** available under **Testing** → **Settings** → **Policies** identifies and validates AI endpoints against known risks. Built on the*OWASP Top 10*for AI framework, the policy uses Traceable’s test plugins to simulate real-world AI and LLM attack scenarios. These tests help you understand how your AI endpoints handle prompts, data, and responses, ensuring that your AI applications stay secure and compliant. ## What will you learn in this topic? By the end of this topic, you will understand: - The concept of testing AI and LLM endpoints. - How Traceable detects and classifies AI issues using the test plugins. - The evidence and insights Traceable provides for each issue for faster remediation. --- ## Understand AI scans Traceable provides an **AI Scan Policy** to identify issues in your application's AI endpoints. This policy employs Traceable-defined plugins that mimic real-world attack scenarios, covering various attack types based on the OWASP Top 10 for AI, including *Prompt Injection*, *Sensitive Data Disclosure*, and *AI SQL Injection*. These plugins are organized under the **AI** category in the **Testing** → **Settings** → **Policies** → **Test Plugins** tab. When you select the AI scan policy during scan setup, Traceable leverages these plugins to detect issues. Each issue discovered is assigned a severity level to help you prioritize remediation. Traceable displays the detected issues under **Testing** → **Issues** → **Filter** → **Category**: *AI*, where you can review your issue findings and the corresponding evidence. For steps to create a scan, see the section below. --- ## Create an AI scan You can create an AI scan by navigating to **Testing** → **Scans**, and clicking **Create Scan** in the page’s top right corner. To create an AI scan, complete the following steps: ### Step 1 — Specify scan details In this step, you should specify the following details: ![Provide Scan Details](https://cdn.document360.io/24f14f07-13d1-4684-8fae-6d8f811768ee/Images/Documentation/traceable_ai_security_scan_general_details.png) Provide Scan Details 1. **Scan Name** — A unique name for the scan, for example, *AIEndpointScan*. 2. **Environment** — The environment that contains the AI endpoints you wish to scan, for example, *fintech-app*. 3. **Frequency** — The number of times you wish to execute the scan: - **One Time** — Traceable executes the scan once post-creation. To run the scan as required, see [Start an ad hoc scan](/v1/docs/creating-scan#starting-an-adhoc-scan). - **Daily** — Traceable executes the scan once daily at the time you select. - **Weekly** — Traceable executes the scan every week on the day(s) and time you select. - **Monthly** — Traceable executes the scan every month on the day(s) and time you select. 4. **Incremental Scans** (for *Daily*, *Weekly*, and *Monthly* frequencies only) — Traceable only scans the AI endpoints that were not scanned in the previous run. For more information, see [Incremental Scans](https://docs.traceable.ai/docs/creating-scan#incremental-scans). Once you have specified the above details, click **Next**. ### Step 2 — Specify source and attacks In this step, you should select the **Traffic Type** you wish to test the endpoints against: - **XAST Live** — Traceable executes the scan on live incoming traffic. - **XAST Replay** — Traceable executes the scan on stored APIs. This is available in environments with Replay enabled. For enabling Replay, see [Environment Config](/v1/docs/environment-config). - **DAST** — Traceable executes the scan based on the specifications you upload. Traceable supports the following documentation types: - OpenAPI Spec - WSDL Spec - Postman Collection - GraphQL Schema Based on the traffic you select, Traceable displays the following configurations: **XAST Live/Replay** ![Select Source and Attacks](https://cdn.document360.io/24f14f07-13d1-4684-8fae-6d8f811768ee/Images/Documentation/traceable_ai_security_scan_source_attacks.png) Select Source and Attacks - **Select API Endpoints** — Select the API endpoints you wish to test as part of the scan. You can select from the following: > [!NOTE] > Recommendation > > Traceable recommends that you select the *Is AI Endpoint* equal to (*=*) *True* as the condition to test AI endpoints in your selected environment. You can also click **View Endpoints** to view the endpoints that Traceable has selected based on the condition you select above. - **All Endpoints** — Traceable tests all endpoints associated with the environment you selected in [Step 1](/v1/docs/ai-security-testing-1#step-1-—-specify-scan-details) above. - **Specific Endpoints** — Traceable tests the endpoints you select from the available list. - **Endpoint Selection Condition** — Traceable tests the endpoints based on the conditions you select. - (Optional) **Target URL** — Specify the domain on which you wish to execute the scan, for example, *mydomain.com*. **DAST** ![Select Source and Attacks](https://cdn.document360.io/24f14f07-13d1-4684-8fae-6d8f811768ee/Images/Documentation/traceable_ast_scan_creation_source_attacks(1).png) DAST Scan Creation - **Document Type** — Select the type of specification you wish to use for testing: - **OpenAPI Spec**/**WSDL Spec**/**GraphQL Schema** — - (For *GraphQL Schema* only) **Introspection Enabled** — Select this option to enable Traceable to dynamically extract the schema from the Target URL you specify below. This enables you to run security tests on the discovered schema without uploading a specification. - **API Specification** — Select one or more specs from the list of pre-uploaded ones or upload new ones according to your requirements. > [!NOTE] > Note (for *WSDL Spec* only) > > - You must merge any XSD definitions into the corresponding WSDL file before uploading. Traceable does not support standalone XSD file parsing. > - You must ensure that all references are defined within the same WSDL file. Traceable does not support external or cross-file references. > - Traceable does not support the *complexContent* element in WSDL files. - (For *OpenAPI Spec* only) **Configure API dependencies & sample values** — Define the application’s dependency graph. When an API requires one or more prerequisite APIs to run first, the dependency ensures that those calls are executed in order during the DAST scan. For example, `GET /orders/{order_id}` depends on `POST /service/order` because an order must be created before it can be retrieved. For more information, see [Understanding API Dependencies](https://docs.traceable.ai/v1/docs/api-dependencies). - **Postman Collection** — - **Collection** — Select one or more collections from the list of pre-uploaded ones or upload new collections according to your requirements. - (Optional) **Environment Doc** — Select a Postman environment document from the list of pre-uploaded ones or upload a new document according to your requirements. Traceable uses this document to resolve variable references present in the collection(s) that you selected above. - **Target URL** — Specify the domain on which you wish to execute the scan, for example, *mydomain.com*. Once you have selected the above configurations, you must either select an existing policy or create a new one. Based on the policy, Traceable executes attacks against the selected endpoints to detect issues. Traceable provides you with the following options related to a policy: ![Policy Selection](https://cdn.document360.io/24f14f07-13d1-4684-8fae-6d8f811768ee/Images/Documentation/traceable_ai_security_scan_source_attacks_policy.png) Policy Selection - **Select an existing policy** — Select a policy from the drop-down list. Traceable shows the attacks configured as part of the policy. In the attack list, you can also perform the following actions according to your requirements: - **Create a new policy using the same attack selection** — Click the **Copy** icon to create a policy that inherits the attack selections from the selected policy. In the pop-up window, specify the policy details and customize the attack types according to your requirements. This helps when you wish to tweak attack selections by adding or removing specific attacks without altering the original policy. For more information, see [Policies](https://docs.traceable.ai/v1/docs/ast-policies). - **Edit the selected policy** — Click the **Edit** icon to customize the attack types or policy names according to your requirements. > [!NOTE] > Note > > This option is not available for Traceable defined policies. - **Create a new policy******— Click **Create New** to create a new policy according to your requirements. For more information, see [Policies](https://docs.traceable.ai/v1/docs/ast-policies). Once you create the policy, Traceable automatically selects it to attack the endpoints. > [!NOTE] > Recommendation > > For testing AI endpoints, Traceable recommends selecting the *AIScanPolicy*. Once you have configured the above settings, click **Next**. ### Step 3 — Configure advanced settings In this step, you can configure the advanced settings, such as authentication and scan evaluation criteria: ![Advanced Settings](https://cdn.document360.io/24f14f07-13d1-4684-8fae-6d8f811768ee/Images/Documentation/traceable_ast_scan_creation_advanced_settings.png) Advanced Settings - **Authentication** — Enable this toggle to select one or more predefined authentication hooks Traceable should use during testing. This ensures that Traceable can access secured endpoints during the scan. You can also create a new authentication hook according to your requirements. For more information, see [Authentication](/v1/docs/ast-authentication). - **Select Specific Runners or Labels** — Enable this toggle to select the runners Traceable should use for scan evaluation. You can select specific runners or a runner label from which Traceable should select a runner for scanning. If you do not enable this toggle, Traceable automatically selects a runner for you. For more information, see [Runners](https://docs.traceable.ai/docs/runners). - **Filter Traffic** — Allow granular filtering of traffic to include endpoints that match specific conditions: - **Location** — Specify where Traceable should look for the attribute, for example, the *Request Header*. - **Attribute Key** — Define how Traceable should match the key and specify the name of the key to match. - **Attribute Value** — Define the value corresponding to the attribute key, for example, *Matches Regex*. - **URL Regex** — Include or exclude endpoints based on regular expressions: - **Include URL Regex** — Specify a regular expression to include specific API endpoints in the scan, for example, `.+` includes all URLs. - **Exclude URL Regex******— Specify a regular expression to exclude specific API endpoints in the scan, for example, `.*(logout|health).*` includes all health checks and logout paths. - **Scan Evaluation Criteria** — Define the conditions based on which Traceable should evaluate the scan: You can click **+ Condition** to add one or more conditions according to your requirements. - **Matches All**/**Any** — Define whether Traceable should execute an *AND* or an *OR* operation between the conditions. - **API Endpoints** (default)/**Services******— The scope of assets (all or new) within which the criteria should apply. - **Vulnerability** — The scope of vulnerabilities (any or new) corresponding to the above-selected assets. Based on your selection, Traceable looks for vulnerabilities in the above-selected assets. - **Severity** — The severity associated with the vulnerability, based on which Traceable should evaluate the criteria. - **Operator** — The operator for comparing the above-selected criteria and threshold. - **Threshold** — The number of vulnerabilities Traceable should look for, as part of the scan. - **Vulnerability Age** — The number of days (1-60) for which the vulnerability should be open, for the criteria to be successfully evaluated. - **Advanced Configuration** — Allow granular filtering of traffic to include endpoints matching specific conditions: - **Idle Timeout** — Define how long Traceable should wait if no activity occurs. - **Scan Timeout** — Maximum duration for Traceable to complete the scan. - **Delay Between Requests** — Milliseconds that Traceable should wait between sending individual requests. - **Test Execution Threads** — The number of concurrent threads that Traceable should use for executing the scan. - **Integration** — Integrate Snyk with the scan according to your requirements: For information on setting up the integration, see [Snyk Integration](/v1/docs/snyk-integration). - **Snyk Organization** — The Snyk organization you wish to include the scan results in. - **Snyk Project** — The Snyk project associated with the organization. Once you have configured the above settings, click **Create**. --- ## Actionable AI insights Each AI issue detected in a scan includes an AI-generated insight summary that explains why the issue occurred, how to fix it, and how to prevent it from recurring. These summaries distill complex findings into clear, developer-friendly recommendations for faster, more accurate remediation. ![AI Issue Insights](https://cdn.document360.io/24f14f07-13d1-4684-8fae-6d8f811768ee/Images/Documentation/traceable_ast_issue_insight.png) AI Issue Insights --- ## Issue findings and evidence For each AI issue Traceable discovers, it displays evidence, including real prompt-response transcripts between the test agent and the LLM, showing exactly how the issue was triggered. The evidence is further mapped to OWASP LLM Top 10 categories and includes the trace visibility for better validation. ![AI Issue Evidence](https://cdn.document360.io/24f14f07-13d1-4684-8fae-6d8f811768ee/Images/Documentation/traceable_ast_ai_issues_evidence.png) AI Issue Evidence This information helps you and your security teams quickly understand what went wrong, replicate the condition, if needed, and work towards its remediation. ## Related - [Creating a Scan](/creating-scan.md) - [Scan Creation Recommendations](/scan-creation-recommendations.md)