Daily-It

개발, AI, 인프라, 자동화와 일상 IT 제품 후기를 직접 써보며 정리하는 기술 블로그입니다.

Grafana Loki Guide: Docker Compose Logging, Alloy, Labels, and Troubleshooting

Summary

Grafana Loki is a log aggregation system for finding log streams with labels and querying them with LogQL, instead of indexing every word in every log line like a traditional full-text search stack. If you are setting up Loki with Docker Compose in 2026, the two details that matter most are easy to miss: Promtail is already in LTS/EOL territory with Grafana Alloy as the forward path, and poor label design can make queries slow or expensive very quickly.

This localized guide keeps the Korean source article’s scope: a small Docker Compose test stack, the Loki/Grafana/agent architecture, Alloy migration direction, label and storage decisions, and the failure cases that usually block a first setup.

In this article

Background

When people first try Loki, they often expect an Elasticsearch-like system that indexes the whole log body. Loki’s model is different. It uses labels such as job, service_name, namespace, or container to find log streams, then uses LogQL to filter inside those streams.

That design can reduce storage and indexing cost, but it also means label choices become an operations decision. Putting values that change on every request, such as request_id, user_id, or trace_id, into labels creates high cardinality and can hurt performance.

The other important change is Promtail. Grafana’s documentation states that Promtail entered long-term support and has an EOL date of March 2, 2026. Existing Promtail examples can still help you understand old setups, but for a new design you should check Grafana Alloy first.

Loki architecture in practice

A basic Loki logging stack usually has three parts.

Component Role
Loki Stores log data and handles LogQL queries.
Grafana Alloy or another log collection agent Reads files, container logs, or Kubernetes logs and sends them to Loki.
Grafana Connects to Loki as a data source so you can search logs, build dashboards, and configure alerts.

For a small local test, a single Loki container is enough. For production, you need to decide storage, replication, retention, query performance, multi-tenancy, alerting, and recovery separately.

Docker Compose test setup

Grafana’s Loki documentation presents Docker and Docker Compose as useful for evaluation, testing, and development. For production, it points readers toward deployment approaches such as Helm or Tanka instead of treating a local Compose file as an operations design.

A minimal test stack can start like this.

services:
  loki:
    image: grafana/loki:3.7.0
    ports:
      - "3100:3100"
    command: -config.file=/etc/loki/local-config.yaml

  grafana:
    image: grafana/grafana:latest
    ports:
      - "3000:3000"
    environment:
      - GF_SECURITY_ADMIN_USER=admin
      - GF_SECURITY_ADMIN_PASSWORD=****
    depends_on:
      - loki

Start the stack.

docker compose up -d

Check whether Loki is ready from the host.

curl http://localhost:3100/ready

If Loki is healthy, you should get a ready-style response. Open Grafana in the browser:

http://localhost:3000

This is a learning setup. Do not treat a single container with local filesystem storage as a long-term log platform.

Why Alloy matters now

Many older Loki tutorials use Promtail. That is the reason a new user can still find plenty of Promtail snippets, even though they are no longer the safest starting point for a new stack.

Promtail EOL: 2026-03-02
Forward path for new collection work: Grafana Alloy

If you already run Promtail, this does not mean the system suddenly stops today. It means a new build or a revised architecture should include an Alloy migration plan. Grafana also documents migration tooling and Alloy-based collection paths, so test the converted configuration before changing production ingestion.

New project: review Alloy first
Existing Promtail deployment: plan and test migration
Short local test: old examples can help, but mark Promtail as EOL/LTS

Label design and storage caveats

Labels are the entry point for Loki queries. Use them for low-cardinality values that describe where a log came from.

Good label candidates look like this.

service_name="api-server"
environment="prod"
namespace="backend"
container="nginx"

Avoid labels like these when the value changes constantly.

request_id="a1b2c3..."
user_id="123456"
ip="203.0.113.10"
trace_id="..."

For high-cardinality fields that you still need to search often, consider structured metadata or filtering inside the log body instead of making them labels.

Storage is just as important. Filesystem storage is fine for local testing, but production should review object storage such as S3, GCS, or Azure Blob together with retention rules. Without retention, logs keep growing until disk usage or object-storage cost becomes the problem.

Failure cases and fixes

Case 1. Grafana cannot connect to the Loki data source

A common mistake is entering http://localhost:3100 in Grafana while Grafana itself is running inside Docker Compose. Inside the Grafana container, localhost means the Grafana container, not the Loki container.

From one Compose service to another, use the service name:

http://loki:3100

From the host machine, this check is still useful:

curl http://localhost:3100/ready

If the host check works but the Grafana data source fails, the first thing to check is whether the data source URL should be http://loki:3100.

Case 2. Loki is healthy, but no logs appear

Loki can be up while the collector is not sending anything. Start with the server health check:

curl http://localhost:3100/ready

Then inspect the collector and Compose logs:

docker compose logs

Check these points before reinstalling the whole stack:

Is the Loki push URL correct?
Is the Docker network/service name correct?
Is the log file path visible inside the collector container?
Did the labels send the logs into a different stream than expected?

Case 3. Too many label values make queries slow

This is one of the most common Loki operations mistakes. Labels such as pod, container, and namespace may be appropriate, but request-level values can create too many streams.

Start by inspecting the label set and querying a narrow stream:

{service_name="api-server"}

For production, keep the rule simple:

Use labels only for low-cardinality values
Do not use user IDs, request IDs, or session IDs as labels
Keep frequently changing values in structured metadata or log content

Case 4. A Promtail tutorial works, but the design is already outdated

Search results still contain many Promtail examples. For a new setup, check Alloy documentation before copying them into a fresh environment.

Back up the existing Promtail configuration
Read the Alloy migration guidance
Validate the converted configuration in a test environment
Check for log loss before production rollout

Case 5. Storage keeps growing because retention was never decided

This is easy to miss during a local test and painful in production. Access logs and debug logs can accumulate faster than expected.

Retention period
Deletion policy
Storage type
Compression/chunk settings
Log level by environment

Case 6. The Docker Compose test file is used as the production design

Docker Compose is useful for evaluation and development. Production needs a separate design for recovery, storage, scaling, security, authentication, and monitoring.

Kubernetes deployment with Helm
Object storage integration
Retention configuration
Grafana authentication and permissions
Monitoring for Loki itself
Backup and disaster recovery plan

Best practices

  • Start small with Docker Compose for learning and validation.
  • For new collection pipelines, review Grafana Alloy before Promtail snippets.
  • Begin with a small label set and add labels only when they improve queries.
  • Keep high-cardinality values out of labels.
  • Design production storage and retention before logs start growing.
  • Check log loss, storage cost, and query performance before polishing dashboards.

Common mistakes

Treating Loki like Elasticsearch

Loki does not index every word in every log line. Label design directly affects query behavior and cost.

Assuming Promtail examples are the current recommended path

Promtail examples are common, but Promtail is in LTS/EOL status. New builds should review Alloy first.

Using localhost from inside a container

Inside Docker Compose, localhost means the current container. Grafana often needs http://loki:3100 to reach Loki.

Running without retention

Logs continue to grow. Without a retention and deletion policy, disk usage or object-storage cost becomes the incident.

Conclusion

Grafana Loki can be a cost-efficient logging stack, but the hard parts are not the first Docker command. The real decisions are label design, the collection agent, storage, retention, and query patterns.

For a new setup in 2026, use current Loki 3.7.x-era documentation, treat Promtail examples as legacy/LTS material, and plan around Grafana Alloy. Docker Compose is a good place to learn, but production should be designed separately with Helm or another operations workflow, object storage, retention, monitoring, and recovery.

References

Original Korean version: This article is based on the Korean version and lightly adapted for English readers. Read the original Korean post.
Please show some love to Korean, too.