eBPF and custom selectors

Traceable allows you to install the Traceable Platform agent in the eBPF daemonSet mode in a Kubernetes cluster. You can select the pods you want to instrument using custom selectors instead of Traceable's Labels and Annotations. For more information on Kubernetes Labels and Selectors, see Labels and Selectors. The document provides information about types of selectors, for example, field, label, and annotation selectors. The document also gives examples of sample values.yaml file to select pods for ingress capture and/or egress capture.

Install eBPF and Traceable Platform agent

You need to configure a few properties in the file to install the Traceable Platform agent in a Kubernetes cluster as an eBPF-based daemonSet. Following is a sample file:

token: <<ACCESS_TOKEN>>
environment: <<ENVIRONMENT_NAME>>
runAsDaemonSet: false
daemonSetMirroringEnabled: true
ebpfCaptureEnabled: true
ebpfRunAsPrivileged: true
daemonSetMirroring:
  matchSelectors:
    - field_selectors:
      - "metadata.namespace=ingress-nginx"
  matchSelectorsEgress:
    - field_selectors:
      - "metadata.namespace=ingress-nginx"

Enter the following commands using the above values.yaml file that has daemonSet configuration:

1. Add traceableai repo

   ActionScriptActionScript

   helm repo add traceableai https://helm.traceable.ai

2. Update the repo

   ActionScriptActionScript

   helm repo update

3. Install Traceable Platform agent

   ActionScriptActionScript

   helm install traceable-agent traceableai/traceable-agent -n traceableai --create-namespace --values values.yaml

If you are using a custom build or a downloaded Helm chart, use the following Helm install command instead of the one above:

helm install traceable-agent traceable-agent-<latest_version_number>.tgz  -n traceableai --create-namespace --values values.yaml

You can replace the Traceable Platform agent with the latest available version number. 

The daemonSet mirroring mode of installation can be configured by providing the daemonSetMirroring section. The following sections in the values.yaml define the daemonSet configuration:

daemonSetMirroring: #<--configurations for the daemonSet
  matchSelectors: #<--kubernetes selectors to select pods where daemonSet mirroring should be enabled
  ……
  matchSelectorsEgress: #<--kubernetes selectors to select pods where the mirroring is enabled for egress traffic
  ……
  matchSelectorsIngressAndEgress: #<--kubernetes selectors to select pods where the mirroring is enabled for all traffic (both ingress and egress)
  ......

Selector configuration

Enable Monitoring for Pods

To select the pods that you wish Traceable to monitor, configure the matchSelectors section to match the pods. All the pods selected by this selector are enabled for monitoring by Traceable. By default, all the ingress traffic to the pod is monitored.

Egress Mode

If you want to monitor egress traffic for a selected pod, you can specify it in matchSelectorsEgress section.

Ingress and Egress Mode

If you would like to monitor ingress and egress traffic for a selected pod, specify it in the matchSelectorsIngressAndEgress section.

Refer to the table below to understand what traffic is monitored for a given pod.

Selector types

The following types of selectors can be defined in the criteria:

• Field Selectors

• Label Selectors

• Annotation Selectors

Field Selectors

The Field selectors section specifies a list of Field selectors that can help select pods based on the Pod Fields. The Field selectors follow the Kubernetes Selector syntax.

The following fields are supported for evaluation for Traceable Instrumentation:

• metadata.name

• metadata.namespace

• spec.nodeName

• spec.restartPolicy

• spec.schedulerName

• spec.serviceAccountName

The following custom fields are also supported:

• spec.container.name

For example, 

- field_selectors:
    - "metadata.namespace=cloudapp,spec.restartPolicy=Always"
    - "metadata.namespace=hackgoapp"

This will be evaluated as follows:

metadata.namespace=cloudapp AND spec.restartPolicy=Always AND metadata.namespace=hackgoapp

The Traceable agent selects a given pod for instrumentation if it matches all the selectors under the given selector sequence.

Supported operators

The supported operators for Field Selectors are =, ==, and !=. The first two operators behave in the same manner.

Label Selectors

The label selectors section specifies a list of label selectors that can help select pods based on the pod labels. The Label selectors follow the Kubernetes Selector syntax.

For example,

- label_selectors:
    - "app=nginxapp,service_app=true"
    - "foo=baz"
    - "injector=hackgoapp"

The above rule will be evaluated as:

app=nginxapp AND service_app=true AND foo=baz AND injector=hackgoapp

Traceable Agent will select a given pod for instrumentation if it matches all the selectors under the given selector sequence.

Supported operators

The supported operators for Label Selectors are =, ==, !=, in, notin, and exists. The first three are equality-based requirements that allow filtering based on keys and values. The last three operators are set-based requirements that allow filtering keys based on a set of values. For example,

app = myapp
env == production
tier != frontend
environment in (production, qa)
tier notin (frontend, backend)
partition
!partition

Annotation Selectors

The Annotation selectors section specifies a list of Annotation selectors that can help select pods based on the pod annotations. The annotation selectors follow the Kubernetes Selector syntax.

For example,

- annotation_selectors:
    - "app.kubernetes.io/name=myapp1,app.kubernetes.io/role=ingress"
    - "app.kubernetes.io/created-by=controller-manager"

This rule will be evaluated as:

app.kubernetes.io/name=myapp1 AND app.kubernetes.io/role=ingress AND app.kubernetes.io/created-by=controller-manager

Traceable Agent will select a given pod for instrumentation if it matches all the selectors under the given selector sequence.

Note

The supported operators are same as explained in the Label Selectors section.

Evaluation Behavior

The selectors can be placed or grouped in such a way that the conditions are ANDed or ORed to achieve the desired pod selection criteria. All the conditions put together under one YAML sequence are ANDed together. While conditions in separate YAML sequences are ORed together.

For example, 

matchSelectors:
    - label_selectors:
        - "lab3=val3,lab5=val5"
        - "lab4=val4"
      field_selectors:
        - "metadata.namespace=ingress-nginx"
    - label_selectors:
        - "lab6=val6"

This is evaluated as:

(lab3=val3 AND lab5=val5 AND lab4=val4 AND metadata.namespace=ingress-nginx) OR (lab6=val6)

Note that in the example above, since the label_selectors and field_selectors are in a single sequence, they are ANDed together. While the conditions in two separate sequences are ORed together. Similarly,

matchSelectors:
    - label_selectors:
        - "lab3=val3,lab5=val5"
        - "lab4=val4"
    - field_selectors:
        - "metadata.namespace=ingress-nginx"
    - label_selectors:
        - "lab6=val6"

Will be evaluated as:

(lab3=val3 AND lab5=val5 AND lab4=val4) OR metadata.namespace=ingress-nginx OR lab6=val6

Sample values.yaml

Ingress Example

token: <<REFRESH_TOKEN>>
environment: <<ENVIRONMENT_NAME>>
runAsDaemonSet: false
daemonSetMirroringEnabled: true
ebpfCaptureEnabled: true
ebpfRunAsPrivileged: true
daemonSetMirroring:
  matchSelectors:
    - label_selectors:
        - "app=nginxapp,service_app=true"
        - "foo=baz"
        - "injector=hackgoapp"
    - field_selectors:
        - "metadata.namespace=cloudapp,spec.restartPolicy=Always"
    - annotation_selectors:
        - "app.kubernetes.io/name=myapp"

Egress Example

token: <<REFRESH_TOKEN>>
environment: <<ENVIRONMENT_NAME>>
runAsDaemonSet: false
daemonSetMirroringEnabled: true
ebpfCaptureEnabled: true
ebpfRunAsPrivileged: true
daemonSetMirroring:
  matchSelectors:
    - label_selectors:
      - "app=nginxapp,service_app=true"
      - "foo=baz"
      - "injector=hackgoapp"
    - field_selectors:
      - "metadata.namespace=cloudapp,spec.restartPolicy=Always"
    - annotation_selectors:
      - "app.kubernetes.io/name=myapp"
  matchSelectorsEgress:
    - field_selectors:
      - "metadata.namespace=cloudapp,spec.restartPolicy=Always"