Logs

Collecting Logs in CloudWatch Logs for EKS

If you’ve previously deployed the application to EKS, the Java application runs in an EKS cluster with a managed node group. There are different setups for collecting logs from control plane and from applications. Below are the configurations to setup logging in EKS for CloudWatch Logs.

Control Plane Logging

There are five components in Amazon EKS control plane that are available for collecting logs, API Server, Audit, Authenticator, Controller manager and Scheduler. You can configure it using the command below for your cluster:

1
eksctl utils update-cluster-logging --enable-types all --cluster unicorn-store --approve --region=$AWS_REGION

Afterwards the control plane logs can be found in CloudWatch Logs:

Application Logging

To send logs from the containers to Amazon CloudWatch Logs, we will use Fluent Bit  – a lightweight log processor.

Fluent Bit can be deployed with Amazon CloudWatch Observability Add-on, if you have already installed the add-on in the previous chapter, your Fluent Bit is already up and running in the EKS cluster.

To verify this, you can run the command below:

1
kubectl get daemonset fluent-bit -n amazon-cloudwatch

If it is installed properly, the output will be similar as the following:

1
2
NAME         DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR   AGE
fluent-bit   2         2         2       2            2           <none>          6h1m

Also run the test below and check the result in CloudWatch Logs

1. Run the following command

export SVC_URL=http://$(kubectl get svc unicorn-store-spring -n unicorn-store-spring -o json | jq --raw-output '.status.loadBalancer.ingress[0].hostname')
curl --location $SVC_URL; echo
curl --location --request POST $SVC_URL'/unicorns' --header 'Content-Type: application/json' --data-raw '{
    "name": "'"Something-$(date +%s)"'",
    "age": "20",
    "type": "Animal",
    "size": "Very big"
}' | jq

2. Go to Amazon CloudWatch  console “Logs group” /aws/containerinsights/unicorn-store/application and find the stream starting with unicorn-store-spring

Otherwise, you should either run through the chapter Amazon CloudWatch Observability Add-on, or work with the following steps to install the Fluent Bit component for collecting application logs.

Amazon CloudWatch Observability Add-on is the recommended way to install the Fluent Bit in EKS.

(Optional) Installing the Fluent Bit component for collecting application logs

Section finished

In this section you have learned how to retrieve application logs in Amazon EKS service.