self-host/services/grafana/README.md

## Local development

```sh
docker-compose-up -d
```

Check which containers are connected to the Docker network:

```sh
docker network inspect web
```

List networks:

```sh
docker network ls
```

Using new Docker Compose syntax:

```sh
docker compose up -d
```

```sh
docker compose --env-file .env.prd up -d
```

### Prometheus

The tool that serves as the data source for the Grafana representation layer.
Adds timestamps to enable series data. Includes query language (PromQL) that
Grafana uses to retrieve and transform data. Also provides alerting.

> In a typical implementation, Prometheus collects the raw metrics data from various systems, and Grafana connects to Prometheus as a data source to create dashboards with visualizations of those metrics.

### Node exporter

Prometheus has different plug-ins called "exporters". These expose metrics from
different processes and services and present them in a format that Prometheus
can scrape. Node Exporter is one of these - it exposes hardware and OS metrics
(such as CPU use and load, memory, disk capacity etc).

### Loki

Loki is for storing and displaying logs. This is part of Grafana. Promtail (also
part of Grafana) is for collecting the logs.

## Explaining reverse proxy

Yes, that's correct. Let me break down how the overall system works at the top level:

Nginx (Your Reverse Proxy)

Listens on public ports 80 (HTTP) and 443 (HTTPS)
Has separate config files in conf.d/ for each service
Each config file (like grafana.conf) specifies:

The domain name (grafana.systemsobscure.net)
HTTPS certificate configuration
Redirection from HTTP to HTTPS
Which internal service and port to route requests to (e.g., grafana:3000)

Docker Networks

A shared external network (typically called "web") connects Nginx to all your services
Each service also has its own internal network for service-specific communication

Services (Grafana, etc.)

Run on their own internal ports (Grafana on 3000, etc.)
Connect to the shared "web" network so Nginx can reach them
Are NOT directly exposed to the internet

The flow works like this:

User requests grafana.systemsobscure.net
Request hits your server on port 80/443
Nginx receives it, finds the matching server_name in conf.d
Nginx forwards the request to the internal Grafana container on port 3000
Response flows back through the same path

This setup keeps your services secure by only exposing them through the Nginx proxy, which handles all SSL termination and access control.