- 13 Apr 2023
- 5 Minutes to read
- PDF
GCP
- Updated on 13 Apr 2023
- 5 Minutes to read
- PDF
Traffic mirroring can be used to capture a copy of the original data from the source network interface without disrupting your existing infrastructure and without adding any latency to your requests. For example, this mirrored data can be used for telemetry, attack detection, and so on. Traceable provides a GCP setup script that automates data collection by using traffic mirroring. Traceable uses open-source Suricata to capture mirrored traffic for further processing. The topic describes the steps to deploy Traceable’s platform agent and enable traffic mirroring using Google Cloud Platform (GCP) CLI.
For a specific VPC network and region, Traceable can mirror traffic from some or all the instances located in that region. Following is a high-level deployment diagram for Traceable agent in a GCP project.
Before you begin
- Make sure that you have a GCP account. This account should have access to the GCP project containing instances that you want to mirror.
- Roles: Your GCP account should have the following set of roles or higher in the GCP project where you wish to install Traceable agent:
roles/compute.admin
roles/container.clusterViewer
roles/iam.serviceAccountUser
- Keep Traceable's access token handy. The token is used during installation to authenticate the agent. To copy the token, login to your Traceable Platform. In the top-right corner under your profile, navigate to Administration > Account > Agent Token and the click on the copy icon.
Download
There are no separate download steps. Traceable uses a setup script that automatically downloads the agent.
Installation
Traceable Platform agent can be installed as follows:
- Login to cloud console
- Download the setup script
- Run the setup script
Step 1 - Login to cloud console
Navigate to Google cloud console and login with your account. Click on the terminal button in the top-right corner to activate Cloud Shell.
Step 2 - Download setup script
Complete the following steps:
- Enter the following command to download the setup script:ActionScriptActionScript
curl -o setup.sh https://downloads.traceable.ai/install/traffic-mirroring/gcp/latest/setup.sh
- Enter the following command to make the script executable:ActionScriptActionScript
chmod +x setup.sh
Step 3 - Run the setup script
The setup script requires a config.yaml file to setup Traceable's traffic mirroring in your project. Complete the following steps:
- Enter the following command to print sample config file:ActionScriptActionScript
Follow the sample to create./setup.sh sample-create
config.yaml
file. For example, if you want to mirror a regional GKE clustermy-regional-gke-cluster
in regionus-east4
in the projectmy-gcp-project
, you may use:ActionScriptYAML
If you wish to mirror subnettoken: *** agentProject: my-gcp-project region: us-east4 mirror: clusters: - name: my-regional-gke-cluster
my-subnet
in the above example instead, then use:ActionScriptYAMLtoken: *** agentProject: my-gcp-project region: us-east4 mirror: subnets: - my-subnet
Note:Traceable Platform agent must be in the same region as the mirrored instances. - Enter the following command to run the script:ActionScriptActionScript
./setup.sh create config.yaml
Verification
You can verify a successful installation of Traceable Platform agent for Google Cloud Platform by completing the following steps:
- Log into the Traceable platform.
- Select traffic-mirroring from the Environmentdrop-down list.
This would display the mirrored data on the Traceable platform.
You can also see the list of installed traffic mirroring agents by navigating to Administration > Configuration > Data Collection section and searching for traffic-mirroring agents.
Troubleshooting
Installation - login issue
gcloud config list
ACCOUNT
that you want to configure: gcloud auth list
Enter the following command to set the account:gcloud config set account <ACCOUNT>
Installation - Setup script failed to run
Follow the instructions given in the output when setup script fails.
Verification - No traces displayed in Traceable platform
If you can view agents in the Traceable platform, however, you do not see any traces:
- Make sure traffic is coming through your mirrored instances.
- In cloud console from top-left, navigate to ≡ > Compute Engine > Instance Groups. Look for the Traceable instance-group. Click on its name to display the page as shown below:
Choose any instance from the Instance Group Members list and click on SSH button as shown. This will open a new terminal to debug further.
Check traceable and suricata status
Check status of traceable
and suricata
agents by running the following commands:
sudo systemctl status traceable
sudo systemctl status suricata
If these appear to be working fine, navigate to /var/traceable/log
to view log files. Log files are rotated and are timestamped. View the latest log file. All stats are printed at a 5-minute interval:
{"level":"info","time":"2021-09-03T13:19:13.302Z","message":"Stats",
"service":"ext_cap","total":"412","post":"0","get":"387","put":"25",
"delete":"0","head":"0","other":"0","20x":"412","30x":"0","40x":"0",
"50x":"0","RCother":"0","header_summary_blobs_read":"0",
"body_summary_blobs_read":"0","total_blobs_read":"496","queued_work":
"0","active_routines":"0","queue_capacity":"5000","http1x":"412",
"http2":"0","grpc":"0","errors":"0"}
If you need further assistance, reach out to Traceable support.
Upgrade
In Google cloud console navigate to ≡ > Compute Engine > Instance Groups. Look for the Traceable instance-group. Click on its name to display the page as shown below:
Click on Restart/Replace VMs button as shown. This will display a new page as shown below:Click on Restart button. This restarts all collector instances while upgrading the Traceable agent installed on them.
Pause
For a disabled policy, click on the Enable button as shown above to resume mirroring.
Uninstall
To uninstall Traceable resources from your Google cloud project, complete Installation steps 1 and 2.
Scenario 1
When you successfully run the setup script in step 3 of Installation, you will see an output similar to this:
Creating Traceable resource config file traceable-b8613ac0.yaml
If you have this yaml file, enter the following command to uninstall Traceable platform agent:
./setup.sh delete traceable-b8613ac0.yaml
The command prompts for a confirmation before deleting Traceable resources. Enter yes
to confirm.
Scenario 2
If you do not have the above yaml file, you will need to create it manually. Complete the following steps:
- Enter the following command to print the sample delete config file:ActionScriptActionScript
The above command will print a sample delete configuration having three values, that is, project, region, and prefix../setup.sh sample-delete
- In Google cloud console, navigate to ≡ > VPC network > Packet mirroring or search for Packet mirroring through the search bar. Look for the Traceable mirroring policy. As shown in the image below, extract the project, region, and prefix from there. Create a
config.yaml
using these values. For example, in the case below, you may use:ActionScriptActionScriptproject: traffic-mirroring-sharedvpc region: us-east4 prefix: traceable-b8613ac0
In the above screenshot, the policy name is
traceable-b8613ac0-pktm
, so the prefix would betraceable-b8613ac0
. - Enter the following command to uninstall Traceable platform agent:ActionScriptActionScript
The command prompts for a confirmation before deleting Traceable resources. Enter./setup.sh delete config.yaml
yes
to confirm.