Platform agent configuration overrides

The Traceable Agent (TPA) comes with a default config.yaml file that governs its operations. Modifying this configuration file directly is often not feasible during updates or in specific deployment environments such as Fargate, ECS, and EC2. This document provides a comprehensive guide on various methods to override the default configuration settings of TPA.

Configuration overrides allow for more efficient and manageable adjustments to the TPA configuration without altering the main config.yaml file. This is crucial because:

• Persistence across upgrades: Changes to config.yaml are not retained after TPA upgrades, which poses some challenges for users.

• Deployment constraints: Direct editing of the config.yaml in certain deployment environments (like Fargate, ECS) is complex and impractical.

Methods of Configuring Overrides

Config Override YAML File

This method involves creating a YAML file with the required override settings and passing it to the TPA process. Below is an example of a basic config.yaml override.

Example Override File

# tpa-config-override.yaml
global:
  remote:
    endpoint: "api.traceable.ai:443"
  logging:
    level: debug
collector:
  config:
    processors:
      traceable_traces_buffer:
        no_of_workers: 8

Implementation

To apply this override:

1. Save the override YAML file, for example, /path/to/tpa-config-override.yaml.

2. Pass it using either:

   • Environment variable:

     export TA_CONFIG_OVERRIDE_FILE=/path/to/tpa-config-override.yaml

   • Command-line argument:

     ./traceable-agent --config /path/to/tpa-config-override.yaml

Install Script

Depending on the mode, use the following command-line arguments:

• ebpf:

  --tpa-override-config-file tpa-config-override.yaml

• tpa-only:

  --override-config-file tpa-config-override.yaml

• mirror:

  --tpa-override-config-file tpa-config-override.yaml

Base64 Encoding Override

This method is useful in environments like Fargate, where file system access is limited. The override YAML string is base64 encoded and passed as an environment variable.

Base64 Encoding Process

1. Encode the YAML:

   cat tpa-config-override.yaml | base64

2. Set the environment variable:

   export TA_CONFIG_OVERRIDE_BASE64="your_encoded_string"

Environment Variables on Specific Config Fields

In this method, specific fields in the configuration can be overridden using environment variables.

Naming Convention

• Environment variables are prefixed with TA_OVERRIDE_CONFIG__.

• The YAML path is capitalized, and a double underscore separates each key (__).

Example

agent:
  endpoint: https://myendpoint.com
  token: my-token

Override using environment variables:

export TA_OVERRIDE_CONFIG__AGENT__ENDPOINT=https://new-endpoint.com
export TA_OVERRIDE_CONFIG__AGENT__TOKEN=new-token

Limitations

This method cannot override complex YAML structures like arrays.

Config Override Load Order

When multiple override methods are used, they are executed in the following order of precedence:

1. Main config.yaml

2. Config override file (TA_CONFIG_OVERRIDE_FILE or --config-override)

3. Base64 encoded override (TA_CONFIG_OVERRIDE_BASE64)

4. Config fields using TA_OVERRIDE_CONFIG__ prefixed environment variables

5. Specific pre-existing environment variables (for example, TA_REMOTE_ENDPOINT, TA_REFRESH_TOKEN).

TPA Deployments Using Config Overrides

Details on how different environments (such as Fargate, ECS, EC2) can implement these overrides:

Docker Deployment Example

For Docker, you can place the config override YAML in a file and then mount it to the container. Docker Compose will follow a similar pattern.

docker run --volume /Users/myusername/override-config.yaml:/conf/agent/override-config.yaml \
  --env TA_CONFIG_OVERRIDE_FILE=/conf/agent/override-config.yaml \
  --env TA_REFRESH_TOKEN=<refresh-token> \
  --env TA_ENVIRONMENT=<your-environment> \
  -p 4317:4317 --name traceable-agent -it traceableai/traceable-agent:1.48.0

You can also use the environment variable-based override methods listed above and pass in the environment variables to the container.

Mirroring Terraform Deployment Example

For both ECS and EC2 deployments, set the config_override_yaml variable to the YAML string in a .auto.tfvars file, for example, test.auto.tfvars, or any Terraform variables file used to override Terraform variables.

config_override_yaml = <<EOF
global:
  remote:
    endpoint: "api-dev.traceable.ai:443"
  logging:
    level: debug
collector:
  config:
    processors:
      traceable_traces_buffer:
        no_of_workers: 8
EOF

GCP Mirroring Deployment Example

Using the GCP mirroring setup.sh script, you can pass a YAML file to the create subcommand with some config variables. In this YAML file, the config variable is config_override.

config_override:
  global:
    remote:
      endpoint: "api-dev.traceable.ai:443"
    logging:
      level: debug
  collector:
    config:
      processors:
        traceable_traces_buffer:
          no_of_workers: 8