Metrics That Survive Audit Week

Auditors ask boring questions on purpose. They want to see that metrics tie to controls, not vanity charts. Start with SLIs that map to customer-visible behavior—latency of checkout, not CPU on server seven unless you can prove the chain.

Second, fight cardinality early. High-cardinality labels feel clever until they torch storage budgets. Teach teams to ask for a business reason before accepting a new label, and keep exemplars out of production unless you truly need them.

Third, design dashboards for tired humans. Large type, minimal color reliance, and annotations that explain what changed last week. If a panel needs a legend longer than three lines, split it.

Limitation: this article stays on pull-based Prometheus patterns. Push gateways and tracing backends deserve their own notes.