Custom Log Based Metrics Counter Distribution Design

What Custom Log-Based Metrics Are

Log-based metrics are user-defined or system-defined metrics in Cloud Logging that convert log stream data into time-series metrics ingested by Cloud Monitoring. User-defined log-based metrics support two core metric types: counter metrics and distribution metrics.

How it meets the requirements

• Latency Percentiles: Creating a distribution metric allows Cloud Logging to extract numeric values from jsonPayload.latency_ms and aggregate them into configurable histogram buckets. Cloud Monitoring uses these distributions to calculate percentiles (such as p50, p95, and p99) and evaluate latency threshold alert policies.
• 5xx Server Error Tracking by Service: Creating a counter metric with a filter matching jsonPayload.status >= 500 increments a count each time a server error log is ingested. Adding a custom label that extracts jsonPayload.service_name partitions the count into distinct time series per service, allowing Cloud Monitoring to chart error rates and alert on spikes per microservice.

Key advantages and capabilities

• Tail Latency Analysis: Distribution metrics maintain count, mean, sum of squared deviations, and histogram buckets for calculating statistical percentiles.
• Dimensional Partitioning: Custom labels extract log attributes into dimensions, enabling grouping and filtering in dashboards and alerting policies.
• Continuous Monitoring: Both metric types stream directly into Cloud Monitoring to evaluate trends and alert conditions automatically.
• Cost Control: Metrics summarize high-volume log data into compact time-series data points, minimizing overhead.

Why this is optimal

This approach directly maps the data requirements to the native capabilities of Google Cloud Observability: continuous numeric distributions for statistical latency analysis and labeled counters for tracking discrete categorized event occurrences.