[Container] Kubernetes infrastructure monitoring with Datadog [Monitoring]

This is Ohara from the technical sales department.

This article is about infrastructure monitoring of Kubernetes environments using Datadog's monitoring tools.
We will describe the features and points from the perspective of ``Why is an integrated monitoring tool necessary for a dynamic and variable infrastructure environment like Kubernetes?''
(Information as of March 2022)

Challenges of infrastructure monitoring for Kubernetes environments

Orchestration systems such as Kubernetes are being introduced to improve the scalability and fault tolerance of infrastructure, but traditional static hosts such as virtual machines and physical machines that are used for a long time In dynamic and complex embedded environments like Kubernetes, integrated monitoring tools like Datadog can provide real-time visibility across hosts, containers, applications, and Kubernetes. Monitoring and surveillance are required.

● Increase in number of components to be monitored
- In traditional host-centric infrastructure, the two main layers to be monitored are the application and the host running the application. In an orchestrated environment, a new layer of abstraction is added, and in order to comprehensively track the infrastructure, it is necessary to monitor containers and his Kubernetes itself.

● Distributed applications are always on the move
- Kubernetes constantly moves Pods between hosts, scaling up and down to meet demand. In order to properly understand applications and content, it is necessary to monitor all Pods and the applications running within them. However, because Kubernetes automatically schedules workloads, it is difficult to continually check where these pods are actually running.

● Tags and labels are essential for continuous visibility
- Because a typical Kubernetes cluster has many dynamic/mutable elements, tags and labels are the only reliable way to identify Pods and the applications within them. It's a method. Without labels and tags, it's nearly impossible to aggregate or interpret the ever-changing performance data of your Kubernetes infrastructure.

Monitor Kubernetes platform environments of any size

Kubernetes clusters run on a variety of platforms. With Datadog's 400+ pre-built integrations for all major cloud providers, all your containerized applications come online, no matter what platform they're using behind the scenes. You can monitor its health and performance.

And whether your organization chooses a fully managed platform or hosts with Rancher, OpenShift, or Anthos, you can rely on Datadog to monitor everything from cluster status and low-level resource metrics to distributed traces and logs. brings together all your Kubernetes infrastructure and application data into a single, unified platform.

Datadog automatically enriches your data with Kubernetes, Docker, and cloud provider tags, making it easy to investigate events as they occur, so whether you're running tens or thousands of nodes, Datadog provides a minimal By providing detailed visibility into your Kubernetes cluster during setup, you can safely build, deploy, and scale container environments.

All your Kubernetes data in one place

At Datadog, we understand what's happening at every layer of your Kubernetes environment. You can easily deploy the Datadog Agent to all nodes in your cluster using DaemonSet or Datadog Operator. With Datadog's Kubernetes integration, you can:

◆ Maintain control plane health

Monitor the health and performance of all components of the control plane, including tracking A healthy control plane allows you to properly schedule and orchestrate workloads and keep your cluster running smoothly.

Detect and resolve critical control plane issues before they impact your customers, such as setting automatic alerts

◆ Troubleshooting Kubernetes issues

● Full-stack visibility into your Kubernetes environment
- seamlessly move between metrics, logs, and distributed traces for your Kubernetes workloads and applications to quickly troubleshoot performance issues. Visualize your data in real-time with customizable, easy-to-use dashboards.

● Analyze Kubernetes audit logs
- Troubleshoot API authentication issues that may affect access to your cluster by users or services.

● Drill down using tags
- Datadog automatically collects tags from Kubernetes, Docker, and cloud providers, so you can easily sort, filter, and aggregate your data. Quickly narrow down issues by region, container image, Pod name, or other categories to reduce average resolution time.

◆ Automatically detects service status anywhere

● Dynamically monitor orchestrated services
- Datadog detects changes in your cluster and automatically monitors various cluster components (such as the Kubernetes API server) and common infrastructure technologies (such as Apache Automatically start data collection from Tomcat, Redis, etc.). You can also define custom configuration templates for Agent checks and specify which containers each check should monitor.

◆ Automatic scaling of workloads using any metric

● Deliver a high-quality customer experience at scale.
Use Datadog with Kubernetes' Horizontal Pod Autoscaler to maintain application availability even in the face of unexpected traffic. Scale your workloads based on any metric you monitor with Datadog, from integration-specific metrics (like MySQL query throughput) to custom business metrics. (e.g. daily page views, etc.)

summary

In a dynamic and variable infrastructure environment like Kubernetes, it may be difficult to operate using conventional monitoring tools, so it is recommended to introduce an integrated monitoring tool such as Datadog to improve operational performance. is recommended.

Also, although this article was written from the concept and perspective of Kubernetes container environments, some of the content also applies to the operation of instance auto scaling functions and environments such as Amazon EC2 using cloud environments such as AWS. Therefore, I would be happy if you could refer to this article.