Conformance Analysis

  • Updated on Jun 3, 2025
  • Published on Sep 16, 2022
  • 8 minute(s) read
Prev Next

Traceable's Conformance Analysis helps you identify deviations in API specifications defined by developers compared to the specifications Traceable creates based on the traffic. Traceable provides two options for performing conformance analysis: One Time and Scheduled.

Conformance analysis classifies API endpoints into four categories:

  • Shadow Endpoints — These are the endpoints Traceable identifies based on the traffic analysis, but these endpoints were not part of the specification you uploaded. For example, if the uploaded API specification has two API Endpoints GET /cart and GET /checkout, however, Traceable also identifies GET /admin as an API endpoint, then GET /admin is a shadow endpoint.

    Note

    In Traceable, GET /cart and POST /cart are two different API endpoints.

  • Orphan Endpoints — These are the endpoints that are part of the API specification uploaded by the user; however, Traceable has either not observed or is still learning these API endpoints. For example, if the API specification has two API endpoints GET /cart and GET /checkout, however, Traceable does not observe GET /checkout during analysis, then GET /checkout is identified as an orphan endpoint.

  • Matched Endpoints with Issues — These are the endpoints in the API specification you uploaded, and Traceable has also observed them. However, Traceable has observed some discrepancies, such as type mismatch, missing parameters, and authentication issues. An example of type mismatch is that you define a parameter as an integer, whereas Traceable observes it as a string.

  • Matched Endpoints without issues — These are the endpoints in the API specification you uploaded. Traceable has also observed these endpoints but did not observe any discrepancies in parameters, such as type mismatch, missing parameters, and shadow parameters.

Setting Up Conformance Analysis

The one-time conformance analysis is designed to execute a comprehensive evaluation once, according to the parameters you specify while starting the analysis. However, you can manually re-run the conformance analysis according to your requirements. To start a conformance analysis, navigate to API Catalog → Conformance Analysis and click Start Analysis. A pop-up window is shown containing the following options:

One-Time Conformance Analysis

  • API Documentation Type — The type of documentation that you wish to upload or choose from, for example, OpenAPI Spec.

  • Select API Documentation — The Conformance Analysis is done against the specification you choose from this drop-down list. You can either select one or all specifications available in the drop-down list or upload the documentation below. The list of API documentation is populated either from the click to upload section or from API Catalog → Settings → API Documentation.

  • Click to upload — The API specifications uploaded here are added to the Select API Documentation drop-down list. You can upload a maximum of 10 API specifications, with a total size not exceeding 20 MB.

    Note

    You can select a specification from the drop-down and upload a new API specification simultaneously. For example, if there are five API specifications available in the drop-down list, you can select all five or fewer, and also add a new specification using the Click to upload section.

  • Job name — Specify the name of the analysis job.

  • Environment — Select an environment from the drop-down list against which you want to run the conformance analysis.

  • Select API Endpoints — Select the API endpoints against which you want to run the analysis. You can decide to run the analysis against:

    • All endpoints

    • Specific endpoints

    • Endpoints selected based on specific conditions

    The listed endpoints are specific to the environment you select above.

  • Drift Scope — The API elements you wish to run the analysis on, such as Path, Query, Request Cookie, and Request Header. You can select from the available options in the drop-down, according to your requirements.

    Note

    • This option is available for OpenAPI Spec documentation types only.

    • By default, Traceable selects the Path, Query, Request Cookie, and Request Header parameters for this field.

Click Submit to start the analysis.

Note

The analysis job result is stored for 10 days, after which they are automatically deleted.

The scheduled analysis is designed to execute a comprehensive evaluation at specific intervals, ensuring regular and consistent analysis. Traceable provides two options for performing scheduled conformance analysis: Manual and System Generated.

Manual Analysis

The configuration of a manual analysis is similar to that of a one-time analysis, however, you can set the analysis to run at specific intervals according to your requirements. To do so, navigate to API Catalog → Conformance Analysis and click on Start Analysis. A pop-up window is shown containing the following options:

Manual Conformance Analysis

  • API Documentation Type — The type of documentation that you wish to upload or choose from, for example, OpenAPI Spec.

  • Select API Documentation — The conformance analysis is done against the specification you choose from this drop-down list. You can either select one or all specifications available in the drop-down list or upload the documentation below. The list of API documentation is populated either from the click to upload section or from API Catalog → Settings → API Documentation.

  • Click to upload — The API specifications uploaded here are added to the Select API Documentation drop-down list. You can upload a maximum of 10 API specifications, with a total size not exceeding 20 MB.

    Note

    You can select a specification from the drop-down and upload a new API specification simultaneously. For example, if there are five API specifications available in the drop-down list, you can select all five or fewer and also add a new specification using the Click to upload section.

  • Job name — Specify a name for the analysis job.

  • Environment — Select an environment from the drop-down list against which you want to run the conformance analysis.

  • Endpoints — Select the API endpoints against which you want to run the analysis. You can decide to run the analysis against:

    • All endpoints

    • Specific endpoints

    • Endpoints selected based on specific conditions

    The listed endpoints are specific to the environment you select above.

  • Schedule — Select the schedule according to which Traceable should run the analysis, for example, Daily 8:30 AM.

  • Drift Scope — The API elements you wish to run the analysis on, such as Path, Query, Request Cookie, and Request Header. You can select from the available options in the drop-down, according to your requirements.

    Note

    • This option is available for OpenAPI Spec documentation types only.

    • By default, Traceable selects the Path, Query, Request Cookie, and Request Header parameters for this field.

Click Submit to start the analysis.

System-generated analysis

The system-generated analysis is an automated analysis that runs independently every 12 hours from the last automated job run. The automated job runs for API documentation that is enabled for naming. For example, if 12 API documentations are available, out of which 8 are enabled for naming, then the automated conformance analysis job runs for 8 API documentations.

Note

Traceable executes the system-generated analysis on the Path, Query, Request Cookie, and Request Header parameters.

To enable API documentation for naming, navigate to API Catalog → Settings → API Documentation, edit an existing API specification, and select the Enabled for Naming toggle, as shown below.

Enable Naming for System-Generated Analysis

You can also use the API Documentation tab to upload a new specification or delete existing documentation. Any API specification uploaded from the API Documentation tab is visible in the Select API Documentation drop-down list, as displayed in the One time and Manual analysis sections. In one-time and manual analysis, you can choose the environment in which you want to run the conformance analysis. In contrast, the system-generated conformance analysis job runs for all the environments.

The results of the system-generated conformance analysis are available in the API Catalog → Conformance Analysis → Scheduled → System Generated tab. These results are available for 3 days. To view results for the analysis of specific environments, select the Environment from the drop-down list in the top right corner.

Analysis report

Once you start the conformance analysis, Traceable runs the job according to your configured settings. Once it is complete, you can click on the job to view the analysis results. The analysis report summary is shown at the top, while the API details are shown on the left side of the page, as shown in the screenshot below.

Conformance Analysis Results

The details of the conformance report have the following sections:

API Endpoints

This section lists the API endpoints in the uploaded Spec section. The Traceable section has the number of API endpoints Traceable has observed. For example, in the above screenshot, the number of API endpoints in the Spec is 2, while the number of API endpoints Traceable observed is 24.

Scope

This section summarizes the parameters used in the conformance analysis. For example, in the above screenshot, the job was run against all API endpoints in the Crapi-aws environment with the crapi-openapi-spec as the API documentation.

Details of API

This section lists the number of:

  • Matched Endpoints with Issues

  • Shadow Endpoints

  • Orphan Endpoints

  • Matched Endpoints without Issues

When you click on either of the above, Traceable displays the analyzed APIs accordingly. For example, the above screenshot displays all the shadow API endpoints. You can click on any row in the right pane to view the details of that API. Similarly, you can view details about other types of APIs by clicking on the respective sections.

Create JIRA

From the analysis result page, you can create JIRA tickets for Shadow and Matched Endpoints with Issues by clicking Create corresponding to a row. By default, the ticket lists the basic details about the issue.

Create JIRA

Note

The Link to job option (as in the above screenshot) is valid only for 10 days since the result generation time.

Viewing Detected Issues

Traceable highlights security gaps (issues) identified through conformance analysis directly on the Issues page. The Issues page offers a detailed view of each issue. This enables you to view the issues directly in a centralized dashboard and take the necessary actions for remediation.

To view the issues raised via conformance analysis, navigate to Catalog → API Risk → Issues, and in the page’s top right corner, click the Filter icon and select either of the following:

  • Source: Conformance Analysis

  • Category: API Governance

This detailed issues view ensures that you can identify, track, and resolve the conformance-related security gaps within your application ecosystem. For more information, see Issues Overview and Issue Management.