An enterprise data platform receives real-time transaction events via Pub/Sub and writes them directly into BigQuery using a BigQuery subscription with use_topic_schema enabled. To implement strict data contract governance across upstream producers and analytical downstream tables, the engineering team must satisfy the following architectural requirements: Validate message structure against an agreed contract definition before messages are accepted from producers. Enforce target data type compatibility when writing formatted fields (such as string-encoded timestamps and numeric amounts) into BigQuery. Quarantine any message that fails schema validation or cannot be written to BigQuery without delaying valid messages with delivery backoff. Preserve detailed failure diagnostics and error metadata for forensic debugging. Which architectural configuration should the team implement?

Attach an Apache Avro or Protocol Buffer schema to the Pub/Sub topic, and configure a dead-letter topic on the BigQuery subscription to capture failed messages with the CloudPubSubDeadLetterSourceDeliveryErrorMessage attribute.: ### What the Pub/Sub and BigQuery Schema Contract Architecture Is This solution establishes an end-to-end data contract by binding an **Apache Avro** or **Protocol Buffer** schema directly to a **Pub/Sub topic**, enforcing structural validation at the ingestion boundary while leveraging native **BigQuery subscription** schema compatibility and **dead-letter topics** for deterministic failure handling. ### How it meets the requirements - **Pre-ingestion structural validation**: Pub/Sub topic schemas validate incoming payloads against Avro or Protocol Buffer definitions. Messages violating field definitions, missing required attributes, or presenting unexpected types are rejected at the publishing boundary. - **Target data type compatibility enforcement**: When `use_topic_schema` is set on a BigQuery subscription, Pub/Sub checks compatibility with the BigQuery table schema. String fields mapped to `TIMESTAMP`, `DATETIME`, `DATE`, `TIME`, `NUMERIC`, or `BIGNUMERIC` columns must adhere strictly to BigQuery formatting rules; otherwise, BigQuery refuses the row. - **Non-blocking quarantine via dead-letter topic**: Attaching a **dead-letter topic** ensures that unwriteable or schema-incompatible messages are routed away immediately without triggering progressive **push delivery backoff**, preventing ingestion pipeline stalls. - **Preservation of forensic diagnostics**: When Pub/Sub routes failed messages to the dead-letter topic, it automatically attaches the `CloudPubSubDeadLetterSourceDeliveryErrorMessage` attribute, providing the exact reason the message failed to load into BigQuery. ### Key advantages and capabilities - **Zero-code ingestion**: Achieves high-throughput streaming directly into BigQuery using the managed **BigQuery Storage Write API** underneath. - **Eliminates delivery bottlenecks**: Dead-letter routing removes failing payloads from the main subscription stream, ensuring healthy throughput for compliant events. - **Detailed root-cause analysis**: Enriches failed payloads with standard metadata attributes detailing schema mismatches. - **Automated serialization checks**: Guarantees strong typing with compact binary serialization using Avro or Protobuf. ### Why this is optimal This architecture enforces contract boundaries natively across serverless Google Cloud services without provisioning intermediate compute clusters, ensuring strict schema governance and automated dead-letter forensics.

Pubsub Bigquery Data Contract Schema Enforcement | professional-cloud-data-engineer

Pubsub Bigquery Data Contract Schema Enforcement

Worked example. The correct answer is already marked and every option is explained below, so there is nothing to select here. To answer questions yourself, start the free trial.

An enterprise data platform receives real-time transaction events via Pub/Sub and writes them directly into BigQuery using a BigQuery subscription with use_topic_schema enabled. To implement strict data contract governance across upstream producers and analytical downstream tables, the engineering team must satisfy the following architectural requirements:

Validate message structure against an agreed contract definition before messages are accepted from producers.
Enforce target data type compatibility when writing formatted fields (such as string-encoded timestamps and numeric amounts) into BigQuery.
Quarantine any message that fails schema validation or cannot be written to BigQuery without delaying valid messages with delivery backoff.
Preserve detailed failure diagnostics and error metadata for forensic debugging.

Which architectural configuration should the team implement?

Attach a JSON Schema definition to the Pub/Sub subscription and configure BigQuery column-level encryption using AEAD functions to automatically reject malformed data into Cloud KMS audit logs.

Configure the BigQuery subscription with use_table_schema enabled and rely on BigQuery time travel and table snapshots to restore and analyze corrupted data elements after ingestion.

Attach an Apache Avro or Protocol Buffer schema to the Pub/Sub topic, and configure a dead-letter topic on the BigQuery subscription to capture failed messages with the CloudPubSubDeadLetterSourceDeliveryErrorMessage attribute.

Define a Data Catalog tag template on the BigQuery table to enforce data types, enable schema auto-detection on the subscription, and capture failed rows via BigQuery session query labels.

Professional Cloud Data Engineer

Pubsub Bigquery Data Contract Schema Enforcement

Related questions