Triggering Authenticated Cloud Run from Dataflow Events

What Eventarc with Cloud Run is

Eventarc is a fully managed Google Cloud eventing service that captures events from Google Cloud sources, custom applications, or third-party platforms and routes them asynchronously to specified targets. When integrated with Cloud Dataflow, Eventarc captures state changes (such as JOB_STATE_DONE, JOB_STATE_FAILED, or JOB_STATE_RUNNING) as direct event notifications without requiring custom polling scripts or complex manual notification channels.

How it meets the requirements:

• Event-driven trigger (✓): Eventarc listens directly to Dataflow events, delivering job state change notifications in real time without polling delay.
• Containerized application target (✓): Cloud Run services are native target destinations for Eventarc, allowing microservices in containers to execute custom logic immediately upon receiving the event payload.
• Secure authenticated invocation (✓): To allow Eventarc to invoke authenticated Cloud Run services securely, the trigger must be configured with a user-managed service account assigned the Cloud Run Invoker (roles/run.invoker) role via the --service-account parameter.
• Post-processing and logging (✓): The Cloud Run service receives structured JSON or Protobuf payloads detailing the Dataflow job attributes (e.g., job ID, execution state, timestamp) to drive automated workflows like bucket cleanup or alert logging.

Key advantages of this configuration:

• Serverless and fully managed: Eliminates infrastructure management and scaling overhead for handling event delivery.
• Standardized CloudEvents format: Delivers structured metadata about job lifecycle changes using standard event specifications.
• Fine-grained filtering: Allows filtering events by exact matches or path patterns based on specific Dataflow job IDs or regions.
• Granular access control: Leverages standard Google Cloud Identity and Access Management (IAM) roles to ensure the principle of least privilege.

Why this is optimal:

Using Eventarc to directly bind Dataflow state transitions to an authenticated Cloud Run service represents Google Cloud's recommended architectural pattern for event-driven orchestration. It decouples the data processing engine from downstream reporting and maintenance systems, providing a resilient, secure, and low-latency solution.