Configure Log Router Sink Bigquery Writer Identity

What Log Sink Writer Identities Are

When you create a log sink in Cloud Logging that routes log entries to another destination (such as a BigQuery dataset, Cloud Storage bucket, or Pub/Sub topic), Cloud Logging creates or assigns a dedicated service account represented in the writerIdentity field of the sink configuration. This identity acts on behalf of the Log Router to deliver logs to external or cross-project resources.

How It Resolves the Issue

• Sink Authentication: Log routing sinks do not automatically inherit access to write to downstream targets, especially across separate Google Cloud projects.
• Required Destination Role: For a BigQuery destination, the writerIdentity principal requires permissions to insert records into tables and create underlying log tables, which is provided by the BigQuery Data Editor (roles/bigquery.dataEditor) role.
• Verification Command: Running gcloud logging sinks describe SINK_NAME reveals the assigned service account (e.g., serviceAccount:service-123456789012@gcp-sa-logging.iam.gserviceaccount.com), which can then be granted the necessary role via IAM policy bindings on the target dataset or project.

Key Capabilities and Best Practices

• Decoupled Security Model: Grants least-privilege permissions strictly to the sink's unique writer identity without exposing broader pipeline permissions.
• Cross-Project Ingestion: Enables secure, centralized log aggregation from multiple workloads (such as Dataflow and Dataproc) into a central data warehouse.
• Automated Schema Evolution: Allows BigQuery to automatically structure and partition incoming log tables based on log timestamps and payloads.

Why This Is Optimal

Granting the specific destination role (roles/bigquery.dataEditor) directly to the sink's writerIdentity satisfies the exact IAM requirement needed for the Log Router to push data into BigQuery without granting unnecessary project-level owner permissions.