--- title: "Istio" slug: "istio" description: "Get step-by-step instructions on integrating Traceable with Istio to monitor and secure API traffic within your service mesh. This guide covers deployment, configuration, and upgrade processes for Traceable components, including the Platform Agent and Istio filter. Learn how to validate successful setup, troubleshoot issues, and maximize visibility into API interactions managed by Istio for enhanced security and performance." updated: 2025-07-18T10:45:10Z published: 2025-07-18T10:45:10Z --- > ## Documentation Index > Fetch the complete documentation index at: https://traceabledocs.document360.io/llms.txt > Use this file to discover all available pages before exploring further. # Istio Istio is an open-source service mesh that integrates with existing distributed applications. It also serves as a platform with APIs that enable integration with logging and telemetry systems. Traceable provides an agent that integrates with Istio to monitor your system and detect anomalies and attacks. Traceable can also block attacks on your system based on the configurations that you make on the platform. For more information on blocking, see [Policies](/docs/protection-policies). The following is a high-level deployment diagram for Traceable's Istio solution. Note that Traceable attaches to only the Istio ingress gateway. ![](https://cdn.document360.io/24f14f07-13d1-4684-8fae-6d8f811768ee/Images/Documentation/traceable_istio_deployment_diagram.png) The deployment steps assume that a standard Istio deployment is already available. #### Traceable module extension (TME) A Traceable Module Extension (TME) is a type of sidecar container that a Traceable Platform agent can inject into an application pod. It can be considered a miniature Traceable Platform agent. ![](https://cdn.document360.io/24f14f07-13d1-4684-8fae-6d8f811768ee/Images/Documentation/traceable_tme_injection(1).png) TME has `ext_cap` as a component. This synchronously receives requests from applications, such as a gateway. The traceable module extension also houses libtraceable, which has the request-blocking logic. TME is used when native instrumentation or a native plugin is unavailable. In such cases, TME captures data inline and sends it to the Traceable Platform agent asynchronously. TME also blocks the requests inline. --- ## Before you begin Before you begin deploying Traceable's tracing agent, note that the deployment assumes a stock deployment of Istio is already available. ### Support matrix | Component | Version | | --- | --- | | Istio | - 1.13.0 and above for Traceable Istio agent 2.x - 1.10.0 and above for Traceable Istio agent 1.x | | Traceable Platform agent | - 1.52.0 and above for Traceable Istio agent 2.x - 1.36.2 and above for Traceable Istio agent 1.x | Verify your Istio deployment. It is a good practice to have your Istio's deployment diagram with you for reference. If you are installing for a demo environment, the control and ingress are in the *Istio-system* namespace; however, the namespace may vary. Verify where the control plane and ingress are installed by entering the following command: ```bash kubectl get svc -n istio-system ``` Following is a sample output of the above command: ```bash $ kubectl get svc -n istio-system NAME                      TYPE           CLUSTER-IP       EXTERNAL-IP      PORT(S)                                                                      AGE istio-ingressgateway      LoadBalancer   10.100.200.50    a1b2c3d4e5.us-west.elb.amazonaws.com   80:31380/TCP,443:31390/TCP                             10d istiod                    ClusterIP      10.100.180.20               15010/TCP,15012/TCP,443/TCP,15014/TCP                                        10d ``` **Explanation:** - **istio-ingressgateway**: This is the external gateway for traffic entering the cluster. It is usually of `LoadBalancer` type in cloud environments. - **istiod**: This is the Istio control plane, typically exposed internally via `ClusterIP`. - The `NAMESPACE` is `istio-system`, which is the default in many setups, but could vary in customized deployments. ### Other notable points - Modify the Istio networking configuration to use Envoy Filter for authorization - Make sure that Traceable's platform agent is already installed. For more information on installing, see [Platform agent](/docs/tpa). - Use the sidecar injector to deploy the Traceable module extension (`tme`). For more information on the Traceable module extension, see [Annotations and Labels](/docs/annotations-and-labels). The `tme` receives data from the Istio module and reports it to traceable-agent. - Point the Envoy filter to the `tme`. - Ensure that port 5442 is available for Traceable module extension (`tme`). --- ## Tracing agent at Istio ingress gateway Configuring the Traceable agent for the Istio ingress gateway consists of the following four steps: 1. Adding a label to the namespace 2. Adding an annotation to the spec in your deployment 3. Enabling Envoy filter 4. Restart Ingress controller #### **Step 1 - Add Label** Add `traceableai-inject-tme=enabled` label to the *istio-system* namespace. ```actionscript kubectl label ns istio-system traceableai-inject-tme=enabled ``` #### **Step 2 - Add Annotation** Add` "tme.traceable.ai/inject:" "true"` annotation to the Istio ingress gateway deployment. Enter the following command: ```actionscript kubectl patch deployment.apps/istio-ingressgateway -p '{"spec": {"template": {"metadata": {"annotations": {"tme.traceable.ai/inject": "true"}}}}}' -n istio-system ``` **Verification of TME Injection**: - After injecting the TME, check the `istio-ingressgateway` pod status. If the TME has been successfully injected, the status should change from `1/1` to `2/2`. - You can further confirm this by running `kubectl describe pod <istio-ingressgateway-POD_ID>` and verifying that the second container listed is indeed the TME. To get the `POD_ID` for a specific pod like `istio-ingressgateway`, you can use the following `kubectl` command: ```plaintext kubectl get pods -n ``` For example, ```plaintext kubectl get pods -n istio-system ``` Look for a pod with a name similar to `istio-ingressgateway-<unique_ID>`. The `<unique_ID>` at the end is the `POD_ID` you are looking for. **Example output**: ```plaintext NAME                                       READY   STATUS    RESTARTS   AGE istio-ingressgateway-7f6bb877-6qhzz        2/2     Running   0          53s ``` In this case, the `POD_ID` is `istio-ingressgateway-7f6bb877-6qhzz`. You can now use this `POD_ID` to run commands like `kubectl describe pod istio-ingressgateway-7f6bb877-6qhzz -n istio-system`. #### **Step 3 - Enable Envoy Filter** Add `traceableai-istio=enabled` label to the Istio ingress gateway deployment. If everything has default names, the following command will add the above label to the Istio ingress in *istio-system* namespace: ```actionscript kubectl patch deployment.apps/istio-ingressgateway -p '{"spec": {"template": {"metadata": {"labels": {"traceableai-istio": "enabled"}}}}}' -n istio-system ``` #### Step 4 - **Restart the Ingress controller** Restart the Ingress controller for the `tme` to attach. Enter the following command: ```actionscript kubectl rollout restart deployment istio-ingressgateway -n istio-system ``` ### **Verify the deployment** Enter the following command to verify the deployment. ```actionscript kubectl get pods -n istio-system ``` Example output ```actionscript NAME                                  READY   STATUS    RESTARTS   AGE istio-egressgateway-96cf6b468-87bkf   1/1     Running   0          13m istio-ingressgateway-7f6bb877-6qhzz   2/2     Running   0          53s istiod-58c5fdd87b-k9j29               1/1     Running   0          13m ``` --- ## **Install Traceable Istio helm chart** Enter the following command to install the Traceable Istio helm chart: ```actionscript helm install traceableai-istio traceableai/traceableai-istio --namespace istio-system ``` The above command creates an Envoy Filter object in the Ingress controller. ### Helm values The following tab lists the Helm values for Traceable’s Istio agents 1.x and 2.x. Use the values as per your deployment. Helm values Istio agent 2.xHelm values Istio agent 1.x The following table details the various Helm values. | Value | Default value | Description | | --- | --- | --- | | `bodyCaptureSize` | 131072 | Defines the maximum payload size in bytes to capture. | | `allowFailureMode` | `true` | When set to `true`, Istio accepts client requests even if the communication with the Traceable Module Extension (tme) fails or the ext_cap service has returned an HTTP 5xx error. | | `moduleExtensionGRPCPort` | 5441 | Traceable Module Extension (`tme`) sidecar gRPC port. | | `moduleExtensionHTTPPort` | 5442 | Traceable Module Extension (`tme`) sidecar HTTP port. | | `moduleExtensionIPAddress` | 127.0.0.1 | Traceable Module Extension (`tme`) sidecar IP address. By default, it is 127.0.0.1 since it is in the same pod as the Istio gateway. | | `workloadSelectors` | `- "traceableai-istio: enabled"` | Labels to add to pods in the mesh to enable Traceable Istio module. | | `allowedContentTypes` | `{'json', 'x-www-form-urlencoded', 'grpc'}` | Capture response bodies whose content-type header contains one of the configured substrings. | | `serviceName` | `istio` | The service name for spans that are captured by the Traceable Istio module. | | `connectTimeout` | 50ms | Timeout to connect to Traceable Module Extension (`tme`). | | `requestHandleTimeout` | 0.2s | Timeout to the `ext.authz` request filter. | | `responseHandleTimeout` | 50 | Timeout to the envoy response handler in milliseconds. | | `webhook.enable` | `false` | Webhook must only be enabled if traceable-agent is running outside the Istio cluster. This Webhook will be used to call into the external traceable-agent to inject the `tme` sidecar. | | `webhook.url` | - | External traceable-agent URL. It must be HTTPS. | | `webhook.caBundle` | - | A base64 PEM encoded CA bundle that will be used to validate the external traceable-agent’s (which is Webhook's server) certificate. | | `webhook.failurePolicy` | ignore | Defines how to handle webhook failures. | | `webhook.useNamespaceSelectors.tme` | `true` | Use namespace selector match labels to inject the `tme` sidecar. | | `webhook.useNamespaceSelectors.java` | `true` | Use namespace selector match labels to inject the Java sidecar. | #### ExtProc filter configurations | Helm value | Default | Description | | --- | --- | --- | | `extProc.allowFailureMode` | `true` | Allows Istio to accept client requests even if communication with TME fails or ext_cap returns HTTP 5xx. | | `extProc.allowModeOverride` | `false` | Enables override of the filter processing mode via TME response. | | `extProc.messageTimeout` | `0.2s` | Timeout for individual messages on the stream in synchronous mode. | | `extProc.observabilityMode` | `false` | Enables "Send and Go" mode, sending each HTTP request/response part without pausing. | | `extProc.requestBodyProcessingMode` | `BUFFERED` | Controls request body processing mode (Allowed values: `STREAMED`, `BUFFERED`, `BUFFERED_PARTIAL`, `NONE`). | | `extProc.requestHeaderProcessingMode` | `SEND` | Controls request header handling (Allowed values: `SEND`, `SKIP`). | | `extProc.requestTrailerProcessingMode` | `SKIP` | Controls request trailer handling (Allowed values: `SEND`, `SKIP`). | | `extProc.responseBodyProcessingMode` | `BUFFERED` | Controls response body processing mode (Allowed values: `STREAMED`, `BUFFERED`, `BUFFERED_PARTIAL`, `NONE`). | | `extProc.responseHeaderProcessingMode` | `SEND` | Controls response header handling (Allowed values: `SEND`, `SKIP`). | | `extProc.responseTrailerProcessingMode` | `SKIP` | Controls response trailer handling (Allowed values: `SEND`, `SKIP`). | | `ext.Proc.grpcType` | `envoy_grpc` | Specifies the `gRPC` service type for `ext_proc` filter. Must be one of: `envoy_grpc` (uses dynamic cluster_name) or `google_grpc` (uses target_uri). | #### Module extension configurations | Helm value | Default | Description | | --- | --- | --- | | `moduleExtension.connectTimeout` | `50ms` | Timeout for connecting to the TME. | | `moduleExtension.grpcPort` | `5441` | gRPC port for the TME sidecar. | | `moduleExtension.ipAddress` | `127.0.0.1` | IP address of the TME sidecar (defaults to the same pod as the Istio gateway). | #### Webhook configurations | Helm value | Default | Description | | --- | --- | --- | | `webhook.caBundle` | `(None)` | Base64 PEM encoded CA bundle for validating webhook server certificates. | | `webhook.enable` | `false` | Enables webhook if the Traceable agent runs outside the Istio cluster. | | `webhook.failurePolicy` | `Ignore` | Defines the webhook failure handling policy. | | `webhook.url` | `(None)` | External Traceable agent URL (must be HTTPS). | | `webhook.useNamespaceSelectors.tme` | `true` | Uses namespace selector match labels to inject the TME sidecar. | | `webhook.namespaceSelectors` | `traceableai-inject-tme: enabled` | Namespace selector labels for TME injection. Only namespaces with these labels will have the webhook applied. | #### General configurations | Helm value | Default | Description | | --- | --- | --- | | `requestTimeout` | `0.2s` | Timeout for the `ext_proc` filter. | | `workloadSelectors` | `“traceableai-istio:enabled“` | Labels to add to pods in the mesh to enable the Traceable Istio module. | | `serviceName` | `istiov2` | The service name for the Traceable agent. | | `environment` | The default configured TPA environment | The name of the environment in which the Traceable agent is running. | | `agentToken` | - | Agent token for authentication | | `filterName` | `traceableai-istio` | Name of the EnvoyFilter resource that will be created. | | `filterNamespace` | - | Namespace where the EnvoyFilter will be created. If not specified, defaults to the release namespace. | > [!NOTE] > Body Size Limitation > > The maximum body size that will be captured in spans is controlled by the `maxBodySize` variable in the Traceable Platform Agent (TPA) Helm chart or the `max_body_size` variable in the TPA Terraform configuration. This setting is defined in the TPA configuration, not in the Istio agent configuration. > > If you need to capture larger payloads, you must adjust this setting in your TPA configuration. The default value is typically 131072 bytes (128KB). --- ### Restart Ingress gateway Restart the ingress gateway so the changes take effect. Enter the following command: ```actionscript kubectl rollout restart deployment istio-ingressgateway -n istio-system ``` --- ### Verify the deployment Enter the following command to verify the deployment: ```actionscript kubectl get envoyfilters.networking.istio.io -n istio-system                         ``` Example output: ```actionscript traceableai-istio           104s ``` --- ## Upgrade Follow these steps to upgrade the Traceable components within your Istio deployment. This process includes updating the Traceable Platform Agent (TPA), upgrading the Istio filter, and verifying the deployment to ensure a successful upgrade. Complete the following steps: 1. **Update the Helm Repository**. ```plaintext helm repo update ``` 2. **Upgrade the Traceable Platform Agent (TPA)** to the latest version to maintain compatibility with the latest Istio filter and features. 3. **Upgrade the Traceable Istio Filter** 1. Search for the Latest Filter Version ```plaintext helm search repo traceableai/traceableai-istio ``` 2. Upgrade the Istio Filter. This command upgrades the Traceable Istio filter in the `istio-system` namespace to the latest version. ```plaintext helm upgrade traceableai-istio traceableai/traceableai-istio --namespace istio-system ``` 3. Restart the Istio Ingress Gateway. Restarting the ingress gateway ensures that the updated filter is applied to incoming traffic. ```plaintext kubectl rollout restart deployment istio-ingressgateway -n istio-system ``` 4. **Verify the deployed chart versions**. 1. Check the Traceable Chart Version in the Traceable Namespace. ```plaintext helm ls -n traceableai-system ``` 2. Check the Traceable Istio Chart Version in the Istio System Namespace. ```plaintext helm ls -n istio-system | grep traceableai-istio ``` 5. **Verify the TME Image Version in the Istio Ingress Gateway**. Describe any `istio-ingressgateway` pod to confirm the TME (Traceable Module Extension) image version: ```plaintext kubectl describe pod -n istio-system ``` 6. **Verify Versions in the Traceable UI** --- - **Check Platform Agent Version**: Confirm that the Platform Agent version reflects the latest release. - **Check Tracing Agent Version**: Ensure the Tracing Agent version matches the latest deployment. - **Check TME Version in Span Attributes**: Inspect the `traceableai.module.version` attribute in incoming spans to verify that they reflect the latest module version post-upgrade. ![](https://cdn.document360.io/24f14f07-13d1-4684-8fae-6d8f811768ee/Images/Documentation/traceable_istio_upgrade_verify.png) ## (Optional) Custom error message You can optionally configure a custom error code and message for the blocking function by editing the Traceable Platform agent configuration, as shown below. Configure this inside the `injector` section. ```yaml ext_cap:  blocking_config:    response_status_code: 403    response_message: "Access Forbidden" injector:  blocking_config:    response_status_code: 403    response_message: "Access Forbidden" ``` If you are using Helm in `values.yaml`, then you would need to set: ```yaml blockingStatusCode: blockingMessage: injector:  blockingConfig:    blockingStatusCode:    blockingMessage: ``` Make sure that the status code is set between 400 and 499. If you add values outside the 4xx range, then 403 (the default status code) is displayed in the Traceable Platform.