professional-cloud-data-engineer
BigQuery security is built on Google Cloud's Identity and Access Management (IAM) system, which controls who can do what on data resources. IAM uses roles that group permissions together: basic roles (Owner, Editor, Viewer), predefined roles for specific tasks, and custom roles you create for unique needs. When you assign roles at the dataset level, users automatically inherit those permissions for all tables within that dataset. This hierarchical approach makes it easier to manage access at scale while following the principle of least privilege.
Data classification in BigQuery uses policy tags to categorize sensitive information and control access at the column level. You create a taxonomy of policy tags that represent different sensitivity levels, such as confidential, private, or sensitive, and then attach these tags to specific columns in your tables. Sensitive Data Protection can automatically scan your data to discover and classify sensitive content like credit card numbers or personal information. This classification system works with IAM to ensure that users can only access columns they have been granted permission to view.
Column-level security in BigQuery combines policy tags with data masking to protect sensitive information while allowing appropriate access. When you apply a data masking rule to a column, users without fine-grained access see masked data (like "****" instead of actual values), while users with the Fine-Grained Reader role see the unmasked data. You can also use column-level encryption with AEAD encryption functions to encrypt individual column values, providing dual control since users need both table permissions and encryption key access to read the data.
Row-level security lets you filter which rows users can access based on conditions you define. You can implement this using authorized views that contain WHERE clauses to restrict data, or using BigQuery's native row-level security policies with dynamic row filters. For example, you might create a filter that only shows employees data from their own department. This approach is especially useful in multi-tenant scenarios where different customers or departments need to see only their specific data.
Authorized views are a powerful security feature that lets you grant access to a view without exposing the underlying tables. You create a query that selects only the data you want to share, then authorize that view to access the source tables. This way, users query the view but never directly access the base tables. Authorized views can also be used to implement complex security logic, such as combining data from multiple tables or applying dynamic filtering based on the user's identity.
Additional security measures include encrypting data at rest and in transit by default, with options for customer-managed encryption keys through Cloud KMS. VPC Service Controls add another protection layer by creating security perimeters that prevent data from being accessed outside your organization. BigQuery also supports table-level access control, letting you grant permissions to specific tables without giving dataset-wide access, providing finer granularity for your security requirements.
Designing a secure project and dataset architecture is critical for implementing multi-tenant workloads and enforcing strict data governance. To restrict unauthorized data access and maintain isolation, organizations should use a dataset-per-tenant configuration rather than creating thousands of separate projects, which can hit resource limits. While authorized views let you share specific subsets of tables without granting direct access, they are limited to 2,500 resources per dataset and cannot cross regions. Conversely, creating fully written subset tables allows for customized partitioning and clustering for each tenant, though it requires more complex schema changes to manage over time.
Once the dataset structure is established, securing sensitive data columns within these tables requires a combination of dynamic masking, classification, and encryption. Using Sensitive Data Protection allows teams to scan and classify confidential data like PII, which can then be tagged and tokenized before storage. Implementing column-level access control using policy tags restricts viewing rights to specific roles, ensuring that only users with fine-grained reader access can see cleartext values. Furthermore, organizations can apply client-side encryption using the Tink library to encrypt sensitive data columns at a more granular level, allowing secure decryption inside BigQuery using AEAD encryption functions.
Proper table architecture must also define robust data retention and expiration policies to ensure compliance with industry regulations. Organizations must carefully select their storage and protection methods depending on their operational goals:
By evaluating these tools, data engineers can establish a structured framework for managing the complete data lifecycle without risking data loss.
To meet international compliance frameworks like HIPAA or CDMC, data architectures must rigorously govern where data resides and how it is processed. Administrators can use resource location policies to restrict the creation of resources to specific geographic regions, helping organizations comply with strict data residency requirements. Data sovereignty is further managed by storing encryption keys outside the cloud via Cloud External Key Manager (EKM) and using customer-managed encryption keys (CMEK) inside Cloud HSM for FIPS-compliant cryptographic protection. By integrating these storage, access, and location controls, data engineers can evidence security compliance while preventing unauthorized insider or vendor access.
Data governance on Google Cloud is fundamentally established by designing a structured resource hierarchy that aligns with your organization's security and compliance needs. This hierarchy, which flows from the Organization node down to Folders, Projects, and individual Resources, provides the logical framework for applying consistent policies. By segregating data and workloads into different projects based on criteria like environment, sensitivity, or business unit, you create natural boundaries for access control and data isolation.
A well-designed hierarchy is the first step in enforcing governance. You create folders under the organization to represent major divisions, such as Production, Non-Production, and Common shared services. Within these folders, you create dedicated projects for specific purposes, like a Data Governance project for centralized services (e.g., Cloud KMS, Data Catalog) and separate Data Ingestion and Confidential Data projects. This segregation allows you to apply granular IAM policies and organization policies at the most appropriate level, ensuring that security controls like least privilege are enforced where they are most effective.
Once the hierarchy is established, you use IAM to define who can do what on which resource. By assigning permissions to Google Groups at the folder or project level, you streamline access management for different teams (e.g., data engineers, security analysts). To prevent data exfiltration and enforce a defense-in-depth strategy, you implement VPC Service Controls. This creates a security perimeter around projects containing sensitive data, controlling the movement of data in and out of Google-managed services like BigQuery and Cloud Storage, even if IAM permissions are misconfigured.
Governance also involves controlling where data resides. You use organization policy constraints (like gcp.resourceLocations) to restrict where new resources can be deployed, ensuring compliance with data residency requirements. Within your projects, you leverage services like Sensitive Data Protection and Data Catalog to discover, classify, and tag confidential data (e.g., PII). You can then attach policy tags in BigQuery to enforce column-level security, dynamically masking data or restricting access based on a user's granted roles, providing fine-grained control over sensitive information.
Finally, governance is operationalized through automation and monitoring. Organization policies are defined in code (e.g., Terraform) and applied consistently across the hierarchy. Services like Security Command Center provide continuous monitoring for policy violations, misconfigurations, and threats. By integrating Cloud Logging and audit logs, you maintain a comprehensive trail of administrative actions and data access, which is essential for demonstrating compliance and conducting forensic analysis. This layered approach, built upon a deliberate project and resource hierarchy, forms a robust and scalable data governance foundation.
Prepare and test your skills
Prepare and test your skills
Dataset-level security uses IAM roles assigned at the dataset level, granting users access to all tables within that dataset. Column-level security uses policy tags to categorize sensitive information and control access at the column level, combining policy tags with data masking so users without fine-grained access see masked data while those with the Fine-Grained Reader role see unmasked data.
Use authorized views when you need to share specific data subsets without exposing underlying tables, as they allow you to create a query that selects only the data you want to share and then authorize that view to access the source tables. Use subset tables when you need customized partitioning and clustering for each tenant, though this approach requires more complex schema changes over time.
BigQuery Snapshots provide point-in-time copies of tables that protect against human errors and logical data corruption beyond the standard seven-day time travel period. Cross-Region Replication provides continuous copying of database changes to a remote secondary location for geo-redundancy and to support geographically close consumers. Cloud Storage Bucket Lock permanently locks retention policies on storage buckets to satisfy strict compliance obligations.
The Google Cloud resource hierarchy flows from the Organization node down through Folders, Projects, and individual Resources, providing the logical framework for applying consistent policies. This structure enables you to apply granular IAM policies and organization policies at the most appropriate level, ensuring security controls like least privilege are enforced where they are most effective.
An enterprise organization is designing a centralized data repository in BigQuery to store sensitive financial transactions across multiple regions. The data governance and security framework requires the following:
Which combination of administrative and security configurations meets these governance requirements?
StudyCerts
StudyCerts