Deploying the Open-telemetry on EKS cluster as an ADOT-addon and track the traces using Amazon x-ray

   AWS Distro for Open Telemetry (ADOT) is a secure, AWS-supported distribution of the Open Telemetry project. Users can instrument their ap...

  


AWS Distro for Open Telemetry (ADOT) is a secure, AWS-supported distribution of the Open Telemetry project. Users can instrument their applications just once and, using ADOT, send correlated metrics and traces to multiple monitoring solutions. Amazon EKS now allows users to enable ADOT as an add-on after the cluster is up and running. The ADOT add-on includes the latest security patches and bug fixes and is validated by AWS to work with Amazon EKS. In this blog post, we will take a look at how to install the ADOT add-on in an EKS cluster and then use it to collect metrics and traces from application workloads deployed to the cluster.


Open Telemetry is a set of APIs, SDKs, and tools that are designed for the creation and management of telemetry data such as traces, metrics, and logs. AWS Distro for Open Telemetry Collector (ADOT Collector) is an AWS-supported version of the upstream Open Telemetry Collector that is fully compatible with AWS computing platforms, including EKS. It enables users to send telemetry data to AWS-managed services such as Amazon CloudWatch, Amazon Managed Service for Prometheus, and AWS X-Ray.


The ADOT Collector has the concept of a pipeline that comprises three key types of components, namely 1. Receiver.

2. Processor.

3. Exporter.
    A receiver is how data gets into the collector. It accepts data in a specific format, translates it into the internal format, and passes it to processors and exporters defined in the pipeline. It can be pull- or push-based.
    A processor is an optional component that is used to perform tasks such as batching, filtering, and transformations on data between being received and being exported.
    An exporter is used to determine which destination to send the metrics, logs, or traces to. The collector architecture allows multiple instances of such pipelines to be set up via a Kubernetes YAML manifest.    


The traces pipeline comprises an instance of AWS X-Ray Receiver and AWS X-Ray Exporter and sends traces to AWS X-Ray. The metrics pipeline comprises a Prometheus Receiver, a processor, and an AWS Prometheus Remote Write Exporter that sends metrics data to a workspace in Amazon Managed Service for Prometheus.


The following is a list of prerequisites needed before we can install the ADOT add-on.


1. An EKS cluster supporting Kubernetes version 1.19 or higher.

2. Certificate Manager, if not already installed in the cluster. Make sure that you are using version < 1.6.0 (1.6.0 is not supported).

3. Kubernetes RBAC permissions specifically for EKS add-ons to install the ADOT add-on in your cluster. If you are new to creating the cluster feel free to check out our blog Creation of AWS EKS cluster from Scratch. The second prerequisite is to install the Cert Manager. check out the official documentation
kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.16.1/cert-manager.yaml

Install the ADOT add-on with the following command or enable the Add-on using GUI.

 Command: aws eks create-addon --addon-name adot --addon-version v0.45.0-eksbuild.1 --cluster-name $CLUSTER_NAME 

Command: aws eks create-addon --addon-name adot --addon-version v0.102.1-eksbuild.1 --cluster-name Techiev-first-cluster

To add the Add-on using GUI click the Add-ons options in the EKS dashboard and click Get more Add-ons, choose AWS Distro for OpenTelemetry(ADOT) click next, and create the add-on.

The version string must match the value of the addon Version field in the previously shown output. The output from a successful execution of this command looks as follows:


Wait until the add-on is in ACTIVE status before proceeding to the next step. The status of the add-on can be checked using the following command.


command: aws eks describe-addon --addon-name adot --cluster-name $CLUSTER_NAME


command: aws eks describe-addon --addon-name adot --cluster-name Techiev-first-cluster


Before creating the Add-on in open-telemetry, we need to create the IAM role with the below listed required policy to be in the attached state.


1. AmazonPrometheusRemoteWriteAccess -- for Amazon-managed Prometheus 

2. CloudWatchAgentServerPolicy -- for CloudWatch insights

3. AWSXRayDaemonWriteAccess and AWSXrayWriteOnlyAccess -- for x-ray


We need to create the IAM role with the below custom trust policy for the Service account.

{

    "Version": "2012-10-17",

    "Statement": [

        {

            "Effect": "Allow",

            "Principal": {

                "Federated": "oidc_provider_arn"

            },

            "Action": "sts:AssumeRoleWithWebIdentity",

            "Condition": {

                "StringEquals": {

                    "oidc_provider_arn:sub": "system:serviceaccount:opentelemetry-operator-system:adot-collector"

                }

            }

        }

    ]

}


Here opentelemetry-operator-system is a namespace and adot-collector is a service account name.




Once the add-on is installed, we can able to see the namespace opentelemetry-operator-system.
After installing the add-on Opentelemetry-operator installed automatically.


Once the Role is created, we can use the Role ARN to update the servicesaccount creation yaml file.

After updating all the required fields, we can create the cluster role, role binding, servicesaccount, and open-telemetry collector for X-ray.

kubectl apply -f "manifestfile.yaml" example:

kubectl apply -f adot-clusterrole.yaml

kubectl apply -f adot-clusterrolebinding.yaml

kubectl apply -f adot-serviceaccount.yaml

kubectl apply -f opentelemetrycollector.yaml

kubectl apply -f collector-config-xray.yaml

In the adot-serviceaccount.yaml we need to pass the Newly created Role Arn.

In the collector-config-xray.yaml we need to pass the exporter aws x-ray region

opentelemetrycollector.yaml manifest we need to add our clustername, acccountid, region and aws managed Prometheus workspace write endpoint.

If you have not created the AMP please check out the blog for Creating the AMP and Grafana and Deploy the CloudWatch agent and Fluent inside the K8s cluster using Terraform and helm chart.
The code of all the above manifests is in my repo https://github.com/maneshmohan124/Techiev-observability


If you are deploying the x-ray, we need to add the Otel_exporter endpoints, and if you are using any attributes you need to add those attributes as well. For more details check opentelemetry.sdk.environment_variables


            - name: OTEL_EXPORTER_OTLP_ENDPOINT

              value: http://my-collector-xray-collector:4317

            - name: OTEL_RESOURCE_ATTRIBUTES

              value: service.namespace=GettingStarted,service.name=GettingStartedService kubectl apply -f sample-app.yaml kubectl apply -f traffic-generator.yaml



Once the application and Kubernetes resources have been deployed, we can check the status by running the command below.


kubectl get all -n opentelemetry-operator-system Now we can able to see the traces inside the Aws cloudwatch x-ray. Reference Blog:

Metrics and traces collection using Amazon EKS add-ons for AWS Distro for OpenTelemetry

----------------------------------------!!!! Happy Learning with Techiev !!!!!!!!----------------------------------

-------------------------Subscribe our Youtube Channel by clicking the below link---------------------- ----------------------------!!https://www.youtube.com/@techieview729!!---------------------
Name

AWS,19,Devops,29,linux,11,
ltr
item
Techie View: Deploying the Open-telemetry on EKS cluster as an ADOT-addon and track the traces using Amazon x-ray
Deploying the Open-telemetry on EKS cluster as an ADOT-addon and track the traces using Amazon x-ray
https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjLXY16xBmpohaMRNjaNvb0WxTLdzAxbSVbiPgLPCi8fQsLeiYu6BN7ItMw77BT7tJeQAPDOiDpeF-PqbaL8mpm-Cx6HulwD9Bx-cXQscm_T0uC1arHyVZlXujswclt56xCYcZVRFUotuz5LsJINq_bomea4sIX0M6TATlDodgYUJgoDKC6vrriIzzb1xIK/w640-h370/open-telemetry.png
https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjLXY16xBmpohaMRNjaNvb0WxTLdzAxbSVbiPgLPCi8fQsLeiYu6BN7ItMw77BT7tJeQAPDOiDpeF-PqbaL8mpm-Cx6HulwD9Bx-cXQscm_T0uC1arHyVZlXujswclt56xCYcZVRFUotuz5LsJINq_bomea4sIX0M6TATlDodgYUJgoDKC6vrriIzzb1xIK/s72-w640-c-h370/open-telemetry.png
Techie View
https://www.techiev.com/2024/10/deploying-open-telemetry-on-eks-cluster.html
https://www.techiev.com/
https://www.techiev.com/
https://www.techiev.com/2024/10/deploying-open-telemetry-on-eks-cluster.html
true
7013663511659419322
UTF-8
Loaded All Posts Not found any posts VIEW ALL View Full Article Reply Cancel reply Delete By Home PAGES POSTS View All RECOMMENDED FOR YOU LABEL ARCHIVE SEARCH ALL POSTS Not found any post match with your request Back Home Sunday Monday Tuesday Wednesday Thursday Friday Saturday Sun Mon Tue Wed Thu Fri Sat January February March April May June July August September October November December Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec just now 1 minute ago $$1$$ minutes ago 1 hour ago $$1$$ hours ago Yesterday $$1$$ days ago $$1$$ weeks ago more than 5 weeks ago Followers Follow THIS PREMIUM CONTENT IS LOCKED STEP 1: Share to a social network STEP 2: Click the link on your social network Copy All Code Select All Code All codes were copied to your clipboard Can not copy the codes / texts, please press [CTRL]+[C] (or CMD+C with Mac) to copy