Pod mirroring
  • 11 Mar 2022
  • 3 Minutes to read

Pod mirroring


Traceable AI provides the deployment option of traffic mirroring at a Pod level to monitor and protect APIs. This option is enabled by deploying network taps within individual Pods across the Kubernetes clusters. Pod based traffic mirroring is implemented completely out-of-band. Traceable gets access to the API bound traffic by auto-injecting a sidecar as a network-tap inside the Pods that you wish to monitor. By virtue of this deployment, all containers inside the specific pod share the same network stack. This allows us to mirror the traffic from the application container to the sidecar container. The following diagram shows a high-level Pod mirroring architecture. 



Before you begin

  • Make sure that you have identified the applications and pods for which you wish to mirror the traffic. 
  • Make sure that you have installed Traceable platform agent in Kubernetes environment. For more information, see Helm installation.

Mirror configuration

The traffic mirroring configuration injects two containers for each pod. The Traceable mirror container captures the traffic while the Traceable module extension (TME) creates SPANS and sends it to the Traceable Platform agent (TPA). 

Complete the following:

Add label

Add traceableai-inject-mirror=enabled label to the namespace. 

kubectl label ns <ns_to_instrument> traceableai-inject-mirror=enabled 

Following is a sample manifest:


apiVersion: v1 # Update
kind: Namespace
metadata:
  name: somename # update
  labels:
    traceableai-inject-mirror: enabled
...

Option 1 - Auto-instrument all Pods in a namespace

To auto-instrument all the Pods in a namespace, add mirror.traceable.ai/defaultInject=true annotation to the namespace.

kubectl annotate ns <ns_to_instrument> mirror.traceable.ai/defaultInject="true"

Setting this instrumentation to true enables automated instrumentation of the mirror across all the the pods in the namespace.

Option 2 - Auto-instrument a subset of deployment in a namespace

When you do not set the mirror.traceable.ai/defaultInject annotation, the default behavior is similar to when the value is set to false. This means that mirroring is not automatically instrumented across all the pods by default. In this case you would need to enable the following annotations at a pod level. You can configure the manifest files:

apiVersion: v1 # Update
kind: Namespace
metadata:
  name: bullish # update
  labels:
    traceableai-inject-mirror: enabled
…
apiVersion: apps/v1
kind: Deployment
metadata:
  name: demo-app
  namespace: somename
spec:
  replicas: 1
  selector:
    matchLabels:
      app: demo-app
  template:
    metadata:
      annotations:
        mirror.traceable.ai/inject: true

Capture pod egress traffic

Traceable's traffic mirroring for Pods has an option to capture Pod's egress traffic instead of the ingress traffic. By default, the mirroring agent captures the ingress traffic. To capture the egress traffic for all pods in a namespace, annotate the namespace with mirror.traceable.ai/defaultMode: egress. To capture the egress traffic at the pod level, annotate pod with mirror.traceable.ai/mode: egress


Annotations and labels summary

The following table summarizes the annotations and labels for Pod mirroring.

Namespace

Label

LabelDescription
traceableai-inject-mirrorSet this value to enabled for the target namespace

Annotations

AnnotationDescription
mirror.traceable.ai/defaultInjectThis annotation defines the default injection behavior on pods in a namespace, that is, whether you want to inject or do not want to inject.
  • Value set to true - When you set the value to true, then by default injection is enabled. If you do not want injection at the pod level, set the pod annotation mirror.traceable.ai/inject: false.
  • Value set to false - When you set the value to false, then by default injection is disabled. In this case, if you want injection at the pod level, set the pod annotation mirror.traceable.ai/inject: true
When this annotation is not set, the default behavior is similar to when the value is set to false.

Pod

The following table describes annotations and labels for the pod.
Labels
There are no labels for the pod for mirroring.

Annotations

AnnotationDescription
mirror.traceable.ai/injectSet the value to true to enable injection of the mirror module along with the Traceable Module Extension (tme) and false to disable injection.
The default behavior, when this annotation is not specified, depends on the behavior of the namespace. See, mirror.traceable.ai/defaultInject annotation in the namespace section above.

Restart the apps

Restart the deployment for mirror to instrument. Enter the following command:

kubectl rollout restart deployment -n <ns_to_instrument>

Verify the deployment

Enter the following command to verify the deployment:

kubectl get pods -n <namespace>

On successful set up, you would see that number of pods have increased.




Was this article helpful?

What's Next