Container Advisor (cAdvisor) provides an overview of CPU, memory, disk, and network utilization for each of the Docker containers. It works by querying the Docker Engine API to get these statistics for each container. This lets you avoid layering the containers with a utility to perform these functions.
Docker Log Driver¶
Docker containers can be configured with a log driver that allows log output to be exported from the Docker container. Using log drivers to export logging information makes running another process on the Docker container for this unnecessary.
A Docker plugin is a way to extend Docker (for example, by adding a log driver).
The Elastic Common Schema (ECS) defines a naming convention for fields used in Elasticsearch. For example, use @timestamp for timestamp.
Elasticsearch is a technology that allows JSON documents to be stored, indexed, and searched in near real time. It can be configured as a cluster with built-in resiliency.
The ELK stack is an established way to enable capturing, indexing, and displaying log data.
The Graylog extended logging format (GELF) improves on syslog logging by providing structured messages that are not size-limited. GELF is one of the built-in logging drivers supported by Docker. The message format is compressed JSON.
Grafana provides a web UI that allows the construction of dashboards showing metric data. This data can be queried against a Prometheus metric store.
Unlike Elasticsearch, Graylog is not a general-purpose indexing, analytics, and search tool. It is designed specifically for log data. This provides a simpler, more focused option with better defaults for logging.
Logstash is a service that allows a series of pipelines to be configured that read, filter, and manipulate data before writing it out. It has support for a multitude of input, filter, and output types. The GELF input reader and Elasticsearch output writer are of particular interest.
Loki is a log aggregation system designed to store and query logs from all your applications and infrastructure. It displays log information inside Grafana, allowing a single UI to be used for both metric data and logs.
Loki Log Driver¶
The Loki log driver is a Loki client that allows log information to be shipped from a Docker log file, similar to other log drivers. The message format is gRPC protobuf.
OpenTelemetry is an organization that works to standardize observability (an umbrella term that includes logging, metrics, and tracing).
Prometheus can be configured to scrape metric data from many endpoints. This metric data can then be queried by metric visualization tools such as Grafana.